alcides / aeon Goto Github PK

Implement the PSB2 benchmark library.

We need a property-based testing library to test our aeon library.
Additionally, this library can be used to write specifications for synthesis.

• Consider whether we want decorators for property based testing.
• Create Generators for the most common types.
• Write tests for our standard library (examples: https://github.com/cabrero/pypbt/blob/main/examples/simple_props.py)

We want a webpage where we can show off Aeon and Aeon examples.

We wil need:

• Interesting hello world example with liquid types
• Interesting synthesis example
• A tour of Aeon, showing the syntax
• Explaining the syntax of refinements and its limitations
• Explaining how to do FFI in Python
• Explain the CLI flags
• Document the standard library, once it's stable.

Assume you have the following code:

def Int.toString : (n:Int) -> String = native "str";
def Int.plus : (n:Int) -> (m:Int) -> Int = \x -> \y -> x+y;

You should be able to write:

s : String = 1.toString
v : String = (1.plus 2).toString

Implementing this is not necessarily trivial in the current version. You need:

• Add MethodCall to the Sugar AST.
• Pre-annotate Sugar AST with (possibly non-liquid) types.
• Replace MethodCalls with the correct Core syntax, during elaboration.

Depends on #26.

When there is a type checking or evaluation error, it would be desirable that the terminal shows the surrounding source code, and notes indicating what went wrong (see the rust compiler for an example).

To implement this:

• Sugar AST/Parser needs to separate from Core.
• Sugar nodes need to contain the source location (begin and end).
• Core AST nodes should have a source (that should be pointing at either Sugar nodes, or their original location).
• In case of an error, that should be caught at the sugar processor, and the error should be printed in the context of the sugar program.

The idea of this feature is two-fold:

Firstly, we want to support multiple decorators (e.g., multiple minimize targets that just append to a list of objectives).

@dec1(...)
@dec2(...)
def f : Int := 3;

Secondly, we want to save information together with the AST ("eg: what is the list of fitness functions, what functions to ignore, etc).

How to implement:

• Decorators need to accept a metadata dictionary, and return the new metadata dictionary.
• When decorators are being applied for each function, the new metadata is passed to the next decorator.
• Between function definitions, the metadata dictionary is also passed on to the next one.
• Metadata needs to be passed on to the synthesis procedure.

Instead of @allow_recursion(), we should support just @allow_recursion.

def sqrt : (i: {x:Int | x > 0} ) -> Float = native "__import__('math').sqrt";

could be written:

def sqrt : (i: Int | i > 0) -> Float = native "__import__('math').sqrt";

To implement this, it requires:

• Change the parser
• In Desugaring, convert this parameter syntax to the core syntax.

We want Lean's namespaces for the following reason:

• We want to group functions that we can import for synthesis (or that we want to ignore as a whole).
• We want to be able to define List.map, and then if the name of the type (constructor) is the same of a namespace, you can do: #[1,2,3].map (\x -> x + 1)

The way it should be implemented:

• Add support for using the "." in the identifiers of function definitions.
• Add support for using "." in variables.
• Verify that TypingContext and EvaluationContexts have the right names.

PR #31
while running the example "examples/PSB2/annotations/multi_objective/fizzbuzz.ae" I noticed it took a while to do the parsing, it took an average of over 25 second .

this is an example of one run

(.venv) ➜  aeon git:(metadata_in_decorators) $ python aeon examples/PSB2/annotations/multi_objective/fizzbuzz.ae --log INFO ERROR TYPECHECKER SYNTH_TYPE CONSTRAINT

Time to parse: 25.318535089492798

sometimes also happens with other examples, like "examples/PSB2/annotations/multi_objective/camel_case.ae".
camel_case took an average of over 28 second to execute the parsing progress

but most of the examples take less than a second to do the parsing

The goal of this feature is to create a flag in the compiler that parses the Sugar file, and generated a single, self-contained HTML with the documentation for that file.

A minimal set of desirable features:

• Do not expand imports
• Document each type with the comment immediately above.
• Document each function with the comment immediately above it, its types (with hyper links to the type definitions, eventually across files)
• Annotate the decorators used.

The problematic use case is when we have a module M, which is imported from both files A and B. A wants to ignore everything in M, but B wants to ignore only a few. The current ignore decorator does not support this use case.

Inspired by Aesop, a solution is to wrap the target function in a syn_ignore decorator:

# M.ae
def c1 : Int = 1;
def c2 : Int = 2;

# A.ae
@syn_ignore(c1, c2)
def k : Int := ( ?h : Int )

# B.ae
@syn_ignore(c1)
def g : Int := ( ?g : Int )

This feature might require Metadata in decorators (#30)
Namespaces can be used as ignoring keywords (#26).

Just like go has gofmt, and other languages have the equivalent, we want an option in the aeon compiler that automatically formats sugar files.

To implement:

• The sugar parser/AST should be more of a concrete syntax tree (consider blank lines and comments)
• A pretty printer should take the sugar AST and return a properly formatted file (ideally parameterized by whether we want terminal colors for highlighting errors on the terminal, and by the line length.

A pretty good resource for this is Phillip Wadler's Prettier Pretty Printer and its several implementations on Github.

https://lean-ja.github.io/lean99/build/Problem1.html

We want to create a VSCode plugin for Aeon that contains the following features:

• Syntax Highlighting (ideally Semantic)
• Show localized error messages
• Show the type of a term on hover
• Embed a code action to replace a hole with the actual code

There is a skeleton for this project here.

Add support for type constructors, to support List Int and List Float.

Introduce parametric refinements (Chapter 8), so we can say that the result of a map has the same refinement as the input.

We support multiples Fitness Decorators in a single function for example:

@minimize( ... )
@minimize( ... )
def f (i: Int) : Int { (?hole:Int) }

but we only support multiples fitness decorators if they are of the same kind.
@minimize with @minimize or @multi_minimize, and vice versa. And the same appends for the @maximize and @multi_maximize decorators.

we would like to support multiples fitness decorators with different kinds, for examples:

@minimize( ... )
@maximize( ... )
def f (i: Int) : Int { (?hole:Int) }

this can be done by adding support to the unary negation operator ( - ) that produces the negative of its operand, and using that operator to negate the value of the @maximize and @multi_maximize decorators during the process of decorators resolution.

running the follwoing examples:

python aeon examples/PSB2/annotations/bouncing_balls_annotations.ae --log INFO WARNINGS TYPECHECKER SYNTH_TYPE SYNTHESIZER ERROR -f 

python aeon examples/PSB2/annotations/dice_game_annotations.ae --log INFO WARNINGS TYPECHECKER SYNTH_TYPE SYNTHESIZER ERROR -f

python aeon examples/PSB2/annotations/gcd_annotations.ae --log INFO WARNINGS TYPECHECKER SYNTH_TYPE SYNTHESIZER ERROR -f

python aeon examples/PSB2/annotations/snow_day_annotations.ae --log INFO WARNINGS TYPECHECKER SYNTH_TYPE SYNTHESIZER ERROR -f

python aeon examples/sugar/synthesis_proposal.ae --log INFO WARNINGS TYPECHECKER SYNTH_TYPE SYNTHESIZER ERROR -f

and analysing the log files, the most common errors are:

SYNTH_TYPE | aeon.typechecking.typeinfer:synth:262 - ('Unhandled:', <aeon.core.terms.If object at 0x124faf2d0>)
SYNTH_TYPE | aeon.typechecking.typeinfer:synth:263 - ('Unhandled:', <class 'aeon.core.terms.If'>)

This is a new error:

 SYNTHESIZER | aeon.synthesis_grammar.synthesizer:evaluator:96 - Failed in the fitness function: operands could not be broadcast together with shapes (200,) (200,3) 

and

SYNTHESIZER | aeon.synthesis_grammar.synthesizer:evaluator:96 - Failed in the fitness function: sort mismatch

I believe that the last error will be solved by adding the MetaHandlers to our grammar, because the refinements are what is raising this error

the log files for the examples above:
bouncing_balls_annotations.ae_2023-11-30 09:15:36.182157.log
dice_game_annotations.ae_2023-11-30 09:18:50.250942.log
gcd_annotations.ae_2023-11-30 09:18:54.597519.log
square_digit_annotations.ae_2023-11-30 09:19:38.045659.log
synthesis_proposal.ae_2023-11-30 09:19:43.910071.log

The idea is to use Program Synthesis tools that address SysGus problems (e.g., CVC5 or z3) to solve a subset of programs (no polymorphism, only native operators).

Steps to implement this approach:

• Translate Aeon Core to SysGus syntax (or fail if impossible)
• Call CVC5/z3 bindings to obtain the program
• Translate the resulting program back to Aeon.

We would like to be able to run Aeon programs in the browser. This can be achieved by embedding a Python interpreter in web assembly (PyScript).

Ideally, we could have a Monaco-based editor to run our little aeon scripts online.

Right now the syntax for arguments is:

def my_function (x:Int, y:Int) : Int { x + y }

To be consistent with lean4, and to avoid confusion with function calls (we call my_function 3 4 and not my_function(3,4)), it would be better to support the following syntax:

def my_function (x:Int) (y:Int) : Int { x + y }

This syntax has another big advantage:

We can now replace refined types {x:Int | x > 0} with (x:Int | x > 0)

def my_function (x:Int | x > 0) (y:Int | y > x) : (z:Int | z < 1000) { x + y }

How to implement this feature:

• Change the parser to accept multiple sets or arguments in parenthesis (trivial)
• Automatically translate (x:Int | p) : Int to (x:{x:Int | p }) -> Int = \x ->
• Change the type grammar rule to accept (x:Int | p) instead of {x : Int | p}
• Add support to also accept (x:Int) instead of {x : Int | p}

We would like to be able to support operator precedence like this:
x: Int = 12 - 1 * 5 ;

currently to support this we have to do it like this, due to some grammar limitations:
x: Int = (12 - 1) * 5 ;

Inside the branch recursion_error, there is a test called recursion_test.py testing the following code :

def gcd ( n:Int, z:Int) : Int {
       if z == 0 then n else (gcd(z)(n % z))
 }
        
def main (x:Top) : Int {
       gcd 15 5
}

that passes the pytest tool, but I have the same exact code in the file gcd.ae in the examples folder, and it raises an error:

...
 File "/Users/eduardomadeira/Documents/lasige/aeon/aeon/backend/evaluator.py", line 44, in <lambda>
    return lambda k: eval(t.body, ctx.with_var(t.var_name, k))
  File "/Users/eduardomadeira/Documents/lasige/aeon/aeon/backend/evaluator.py", line 53, in eval
    return eval(t.body, ctx.with_var(t.var_name, eval(t.var_value, ctx)))
  File "/Users/eduardomadeira/Documents/lasige/aeon/aeon/backend/evaluator.py", line 69, in eval
    return bool(c) and eval(t.cond, ctx) or eval(t.otherwise, ctx)
  File "/Users/eduardomadeira/Documents/lasige/aeon/aeon/backend/evaluator.py", line 53, in eval
    return eval(t.body, ctx.with_var(t.var_name, eval(t.var_value, ctx)))
  File "/Users/eduardomadeira/Documents/lasige/aeon/aeon/backend/evaluator.py", line 48, in eval
    e = f(arg)
  File "/Users/eduardomadeira/Documents/lasige/aeon/aeon/backend/evaluator.py", line 44, in <lambda>
    return lambda k: eval(t.body, ctx.with_var(t.var_name, k))
  File "/Users/eduardomadeira/Documents/lasige/aeon/aeon/backend/evaluator.py", line 53, in eval
    return eval(t.body, ctx.with_var(t.var_name, eval(t.var_value, ctx)))
  File "/Users/eduardomadeira/Documents/lasige/aeon/aeon/backend/evaluator.py", line 69, in eval
    return bool(c) and eval(t.cond, ctx) or eval(t.otherwise, ctx)
  File "/Users/eduardomadeira/Documents/lasige/aeon/aeon/backend/evaluator.py", line 53, in eval
    return eval(t.body, ctx.with_var(t.var_name, eval(t.var_value, ctx)))
  File "/Users/eduardomadeira/Documents/lasige/aeon/aeon/backend/evaluator.py", line 48, in eval
    e = f(arg)
  File "/Users/eduardomadeira/Documents/lasige/aeon/aeon/backend/evaluator.py", line 44, in <lambda>
    return lambda k: eval(t.body, ctx.with_var(t.var_name, k))
  File "/Users/eduardomadeira/Documents/lasige/aeon/aeon/backend/evaluator.py", line 53, in eval
    return eval(t.body, ctx.with_var(t.var_name, eval(t.var_value, ctx)))
  File "/Users/eduardomadeira/Documents/lasige/aeon/aeon/backend/evaluator.py", line 46, in eval
    f = eval(t.fun, ctx)
  File "/Users/eduardomadeira/Documents/lasige/aeon/aeon/backend/evaluator.py", line 46, in eval
    f = eval(t.fun, ctx)
  File "/Users/eduardomadeira/Documents/lasige/aeon/aeon/backend/evaluator.py", line 42, in eval
    return ctx.get(t.name)
  File "/Users/eduardomadeira/Documents/lasige/aeon/aeon/backend/evaluator.py", line 34, in get
    return self.variables[name]
RecursionError: maximum recursion depth exceeded in comparison

PR #52

running:
python aeon/__main__.py examples/sugar/test.ae --log INFO ERROR TYPECHECKER SYNTH_TYPE CONSTRAINT

It takes a lot of time parsing but if I comment the import line, it runs smoothly

Liquid Type checking currently obtains the VCs from the leafs upward to the root.

The idea of this issue is to push the binders inward to the leafs of the VCs, so we can get more localized error messages.

When testing the following program :

type Unit;
def math : Unit = native_import "math";
def test (b:{x:Float | 1 <= x <= 100}) -> (e:{y:Float | 1 <= y <= 100}) ->  Int { native "math.pow(b , e)"} 

it raises an Assertation error :
aeon/verification/horn.py:78: AssertionError

because the "type Unit;" is consider a TypeVar instead of a BaseType

Haskell supports f $ g a as a shorthand for f (g a).

The way this can be implemented is adding to the syntax the $ operator with the right precedence.

In Haskell, $ has the following type:

($) :: (a -> b) -> a -> b

So this depends on Polymorphism.

Google Research published a list of smallish programs in Python at the introductory level.

https://github.com/google-research/google-research/tree/master/mbpp

We should implement them.