skx / monkey Goto Github PK

Given input like this:

string.interpolate("Steve ${surname}", { "surname": "Kemp" } );
-> "Steve Kemp"

The following code works as you'd expect:

   let a = 3;
   let b = 3.312;

   puts( a.type() );
   puts( b.type() );

It outputs integer, then float. However this code fails:

  puts( 3.type() );
  puts( 3.14.type() );

The reason for the failure is that the lexer thinks we're seeing an invalid number, rather than stopping when it become obvious that is not the case.

   puts( 3.type() );
   -> no prefix parse function for ILLEGAL found

Related to #61, we should port the range operator from the evalfilter 1..10 returns an array of the values 1, 2 .. 9, 10.

These are useful and will save users having to write them themselves.

This should work:

  function foo( name = "Nobody") {
     puts( "Hello " + name  + "\n" );
   }

   foo();

Right now you cannot run:

• if ( a && b ) ..
• if ( a || b ) ...

I want to allow this to succeed:

function test( name ) {
  switch( name ) {
    case /^steve$/i {
       printf("I'm a regexp\n");
    }
    case "Steven" {
	printf("I know you!\n" );
    }
    default {
	printf("I don't know who you are\n" );
    }
  }
}

test( "Steve" );    // Regexp match
test( "Steven" );   // Literal match
test( "Bob" );      // Unhandled case

NOTE: I explicitly use blocks here, because fall-throughs are evil :)

Current we have :

assert( bool, str );

Would be nice to evaluate the expression:

assert( str, str );

We could do that if we had eval

Right now if you call pragma(), with no arguments, it returns the pragmas in-use.

If you give it an argument pragma("strict"); it returns true.

It should always return the enabled pragmas.

This causes an ugly crash:

    function foo( name ) {
      puts( "Hello " + name  + "\n" );
    }
    foo();

(i.e. A parameter is expected but not found.)

The crash is here:

    panic: runtime error: index out of range

    goroutine 1 [running]:
    github.com/skx/monkey/evaluator.extendFunctionEnv(0xc42007a600, 0x0, 0x0, 0x0, 0x44013f)
    	/home/skx/go/src/github.com/skx/monkey/evaluator/evaluator.go:797 +0x19f
    github.com/skx/monkey/evaluator.applyFunction(0x548260, 0xc42007a600, 0x0, 0x0, 0x0, 0x0, 0x0)

We should just call len on the two parameter arrays and abort if one is too short. (Though that might need more care when implementing #16 )

This program works:

   let x = 10;
   let x = x * 10;

However this does not:

    let x = 10;
    x = x * 10;

We should allow bare (let-free assignments).

Right now if we want to do a string-find / string-match we must use the index-operator to do it manually.

There are two ways to go here:

• We could implement various string matching functions:
  • string.has_prefix( "Steve", "test" ) -> "false"
  • string.has_suffix "Steve", "eve" ) -> "true"
  • string.contains( "Hello, world!", "world" ) -> true
• We could implement a regular expression matcher.
  • let result = steve =~ /e[vV]e/;

The regular expression method is probably cleaner, but we'd need to think about what it would return:

• NIL on failure to match
• A hash with captures, if any were present:

I'd expect:

 let ip = "10.1.2.234";
 let result = ip =~ /^([0-9]+)\.([0-9]+)\.([0-9]+)\.([0-9]+)$/;
 puts(result[0]) -> "10";
 puts(result[1]) -> "1";
 puts(result[2]) -> "2";
 puts(result[3]) -> "234";

But the use of the result there is a bit nasty. The only alternative would be to create a regexp-type so we could say:

     let regexp = "^([0-9]+)\.([0-9]+)\.([0-9]+)\.([0-9]+)$";
     let result = match(regexp, "string input" );
     puts( result[0] );

That might make the most sense, but feedback welcome.

Consider this code:

  let a = 4;
  let b = 4;

  if ( a == b ) {
    puts( "Integers are identical.\n");
  } else {
    puts( "Integers are different - this is a bug!!\n");
  }

  

  let a = "Steve";
  let b = "Steve";

  if ( a == b ) {
    puts( "Strings are identical.\n");
  } else {
    puts( "Strings are different - this is a bug!!\n");
  }

The output shows:

    deagol ~/go/src/github.com/skx/monkey $ ./monkey e.mon 
    Integers are identical.
    Strings are different - this is a bug!!

Should be simple.

We now have support for calling things like:

 "Steve".count("e");

We should allow the user to define similar methods in pure Monkey, rather than in go.

The initial case should allow:

 function TYPE.name( self );

Where TYPE is string, array, hash, etc. It might be I have to cheat and use ":" instead of "." to simplify life, but I could live with that. The important thing is that it is possible at all.

Currently built-in functions can return errors - primarily when they're invoked on objects of the wrong type.

For example calling len against a hash is an error:

 let a = { "Name": "Steve" };
 puts( len(a), "\n");

This will output:

 Error calling `len` : ERROR: argument to `len` not supported, got=HASH

If we had a strict-mode we could terminate the process at this point, similarly on the use of an undefined variable:

 let steve = "Foo";
 puts( steven );

I think we'll add use( strict ); or pragma( "strict" ); to enable this.

This produces the wrong output:

  puts("狐犬"[0]);

(We expect "狐".)

This code works:

function array.sort() {
   let pass = 0;

   for( ! self.sorted?() ) {
      let i = 1;
      let l = len(self);
      for( i < l ) {
         if ( self[i] < self[i-1] ) {
           self = self.swap( i-1, i);
         }
         i++;
      }
      pass +=1;
   }
   return self;
 }

However remove the use of pass and it panics. Seems to be the return-value of the for looks is causing isseus - suspect it doesn't return a value and it should.

We need to cover:

• functions in the standard-library
• strings
• comments
  • single and multi-line
• keywords

Specifically the first and rest functions could be implemented in pure-monkey, rather than C.

 function first( array ) {
   puts( "first() implemented in monkey\n");
   return( array[0] );
 }
 function rest( array ) {
   let result = [];

   if ( len(array) > 1 ) {
     let i = 1;
     for( i < len(a) ) {
        result = push(result, array[i]);
        i++;
     }
   }
   puts( "rest() is implemented in monkey!!!!\n");
   return result;
 }


 //
 // Test-code
 //
 let a = [ "Steve", "Kemp", "Kirsi", "Kemp" ];
 puts( "First element is ", first(a), "\n");

 a = rest(a);
 let i = 0;
 for( i < len(a) ) {
   puts( "Element ", i, " is " , a[i], "\n");
   i++;
 }

The output of this code is as-expected:

  frodo ~/go/src/github.com/skx/monkey $ ./monkey t.mon 
  first() implemented in monkey
  First element is Steve
  rest() is implemented in monkey!!!!
  Element 0 is Kemp
  Element 1 is Kirsi
  Element 2 is Kemp

Sadly it seems the rest of our functions are required to be implemented in C, but we could write and embed a standard-library of monkey-functions..

At the moment all environmental variables are imported as variables with a $-prefix.

For example this shows your current username, assuming a Unix system:

  puts( "Hello ", $USER , "\n");

or:

  puts( "Hello " + $USER + "\n");

However this importing is read-only. So when you execute processes you cannot change the values.

It might make more sense to clone environmental variables to a hash:

  puts( "The PATH is:" ,ENV["PATH"], " You are :" , ENV["USER"], "\n" );

Using a hash would still allow iteration, via keys, but we could add special handling such that writing ot the ENV-hash would allow updating the contents of an environmental variable.

I'm not 100% sure if this would be a useful change, but it feels like it might be useful?

This should output 2:

  puts( len("狐犬") );

We shouldn't need to use the match function to access captures, we can just populate $1, $2, .. etc, as perl does.

We should be able to port some internals from go to monkey, with a little rejigging.

I suspect I went down the wrong-path when I started adding the abliity to run things like this:

 puts( strings.toupper( "steve" ) );

The idea of a standard-library is appealing, but really when we want to turn a string into a modified version we should probably allow that to happen via the object itself. So I'd prefer users to write:

  puts( "steve".toupper() );

Of course not everything is an object, but a similar approach could be applied to at least:

• arrays
• hashes
• strings

Integers/Floats/Booleans will probably be left alone, and we'd keep the "standard library" for files/etc.

This will be a reasonably large change:

• Allow parsing method-calls via a "."
  • e.g. "foo.bar()" is a method call to function "bar" with implicit arg "foo".
• Remove much of the standard library.
  • e.g. strings.toupper, strings.tolower(), etc.

Fun times :)

We recently added mutation-operators in #14, allowing:

 let x = 4;
 x += 44;

However type-mixing isn't permitted, meaning this code fails:

  let y = 3;
  y += 3.14;

This fails because we only implemented +=, -=, *= and /= for integer types. We should allow floats, and float/int pairs to work. It probably also makes sense to allow string concatenation via "+=":

let s = "Steve";
s += " Kemp";
s += "\n";

Sample usage is the obvious:

  function max(a, b) {
      return( a > b ? a : b );
  }

We can replicate read as write to allow writing to an arbitrary open handle.

Once #13 is implemented the following will work:

     let x = 4;
     x = x + 3;

However this will not:

        let x = 4;
        x += 3;

We need to handle +=, -=, *= and /=. No doubt there are other cases we could consider but they will be sufficient to make me happy.

e.g. This should work:

function max(a,b) {
   return( a > b ? a : b );
}

function max2(a,b) {
    if ( a > b ) {
        return a;
    } else {
        return b;
    }
}

puts( "max(1,2) -> ", max(1, 2), "\n" );
puts( "max2(-1,-2) -> ", max2(-1, -2), "\n" );

If we added an assert method we could write some tests in the standard-library, and run them every time we launched.

(We used to test some methods in the evaluator-test-cases, but because they're loaded in package main, via static.go we had to remove them..)

This should work:

 if ( "Steve" ~= /steve/i ) {
       ..

We should also see:

  print( type( /foo/ ) ) ;   -> "regexp"

(So it is a distinct type, not a string.)

#47 covers a reworking of file/directory handling.

In addition to the work described there we should add:

• file.stat()
  • As mentioned.
• unlink
  • Delete a file/directory.
• mkdir
  • Make a directory.
  • mkdir( "/tmp/foo/bar", { "mode": "0755" } )

It might be nice to allow fork/exec/popen, etc, but they're a bit of work. (popen can probably be handled like fopen.)

For the moment I'd suggest we just implement backtick-execution:

  let out = `uptime`

  puts( out['STDOUT'] );
  puts( out['STDERR'] );
  puts( out['ERROR'] );

Right now you can't write this:

 let str = "\"";

(i.e. Trying to write a quote in a string will cause a syntax error.)

This will become important if we want to allow quotes in regular-expressions, as per #8.

At the moment you open a file via file.open() then use file.read or file.lines.

I think it would be better to have a file object. You'd receive that via calling open. From there you could call methods on the object:

let p = open( "/etc/passwd", "r" ); 
let contents = p.readlines();
p.close();

We should also implement stat to return a hash of meta-data about the given path.

For directory handling we could do something similar for example:

// implemented in golang.
let d = opendir( "/etc" );
d.readdir()
d.closedir();

// all entries, including "." and ".." - implement in monkey, via readdir:
let entries = d.entries(); 

// entries matching the pattern -> implement in monkey, via readdir.
let matched = d.glob( "pas*" );

This would break old code/examples, but I think the consistency would be worth it.

Right now you can't run :

• if ( "a" >= "A" ) ...
• if ( "a" <= "A" ) ...

This fails, silently:

 let a = { "Steve": "Kemp" };
 puts( len( a ) );

Related to #19 we could move PI, & E, into the monkey-based init-file(s) if we had the ability to define constants, which couldn't be modified.

Something like:

const PI=3.1.4..

Instead of:

let PI=3.14..

This should be a simple matter of updating our environment.

The following code should be valid:

   let a = 0xff;
   let b = 0b10101010;

We'll need to update the lexer to recognize these values as numbers, and the parse to convert them to decimal as processed.

My evalfilter project uses foreach to iterate over:

• arrays
• characters in strings.

We can port that easily, and once done we'll also support iterating over hash key/value pairs.

When we allowed object-methods to be defined they were originally written 100% in golang.

That meant that the golang-implemented methods call could work, outputting the names of all known methods. However now there are methods implemented in monkey.

We should use reflection/introspection to get all methods that work on a given-type, or we should remove the facility entirely. Having wrong results is the worst possible compromise!

Convert the github actions from the old HCL-based approach, to the new YAML-based configuration.

Should be trivial.

At least

• string.split
• string.substring
• string.replace

This code fails:

 let a = PI;
 let a = a.to_i();
 > Error: Attempting to modify 'a' denied; it was defined as a constant.

However this works:

let a = PI;
puts( a.to_i() );

The reason for this is the assignment-operation copies the const-value of the source variable (PI in this case).

Calling this code led me on a chase:

 [ true, false, true ].uniq();

array.uniq does everything else, up to the point where it sorts the keys of the (temporary) hash it used.

The comparison makes no sense:

if ( true < false ) { ... }

So either :

• Disallow this.
• Fake an order (false comes first?)
• Don't sort the array when the types are mixed/bogus.

Suspect there's no real right/wrong answer here. Just a choice to be made.

We should allow something like:

 let entries = file.glob( "/etc/*.conf" );

The following program contains a crash, which seems to be as a result of #61:

   a = [ "steve", "kemp" ];
   let out = a.find( "steve" );

The following code fails:

  const a =3;

  function foo() {
      let a = "Hello";
      puts( "A:" , a, "\n" );
  }
  puts( "A:", a, "\n");
  foo();

The output is:

  A:3
  Attempting to modify constant denied - a