atahanyorganci / alloy Goto Github PK

Since CST is a strict superset of AST, From<CST> trait can be trivially implemented for AST.

Parse null keyword and convert to Value::Null.

Concrete Syntax Tree (CST) and Abstract Syntax Tree (AST) are closely related structures and having different parser that produce either one but not the other is wasted effort.

Since CST includes info such as delimiters and whitespace characters used a CST can be cast down to AST.

Extra info present in CST can be used to format source files (i.e. cargo fmt) or programatically produce documents from source files.

syn defines items as top level syntax elements, using syn crate parse struct and enums definitions into ItemStruct and ItemEnum.

Alloy has following binary operators in decreasing precedence order

• Power, left associative
• Multiply, Divide, Module, left associative
• Add, Subtract, left associative
• Comparison (<, <=, >, >=), left associative
• Equals, Not Equal, left associative
• Logical And, Logical Or, Logical Xor, left associative

CSTs can be generic structs and some generic arguments can be redundant after some fields are dropped. See example below

#[derive(AST)]
pub struct SumCST<'a> {
    lhs: Spanned<i64>,
    #[space]
    lw: Spanned<&'a str>,
    #[space]
    rw: Spanned<&'a str>,
    rhs: Spanned<i64>,
}

Lifetime argument 'a becomes redundant in generated AST.

String literals are sequence of regular characters and escape sequences delimited by double quotes.

Reference document should be fleshed out and describe current implementation status.

Implementation of #24 using derive macros. Following steps are required for generating AST.

• #28
• #29
• #30
• #31
• #33
• #32
• #34
• #36

While running REPL don't display dissassembly of the bytecode if compilation has failed. Semantics of intermediate bytecode is defined at a instruction level so it's a undefined state.

just check does not traverse history as it does not use git checkout "$curr" to travel back in time.

Identifiers in Alloy start with alphabetic or '_' character and is followed by any number of alphanumeric or '_' characters which in total is not a keyword.

Rational numbers such as 0.5 are parsed as floating point numbers. Floating point values can either omit whole part of the rational number, for instance one half can be written as .5 or fractional part of rational number can be omitted 1. which is floating point one.

Main feature of CST in Alloy that is different from current AST is the span information. This is achieved by using Spanned<T> where T is the underlying AST so Spanned<T> should be transformed into T.

Alloy identifiers cannot start with numeric characters, for instance 1j is not valid Alloy identifier.

Allow parenthesized expressions '(' *expr* ')' in grammar.

Currently Parse<'a> trait requires the implementer to provide a method for constructing Self from Rule. This has to be changed since nom will replace pest as parsing library.

Parse unary expressions that are logical not, unary minus (i.e. -a) and unary plus (i.e. +a).

Fields with#[space] attribute should not exist in the derived struct.

Add fuzz suite for testing parser. Current integration tests use procedural macros to convert Rust syntax into Alloy AST. This limits the surface area covered by testing.

Implement a proper ParserError struct that can be returned from parse_* functions implemented using nom.

Parse true and false tags and convert them into Value::True and Value::False respectively.

Self referential fields, complex fields, have to transformed.

Title

Expression parsing has to be reimplemented in nom.

• #10
• #11
• #13
• #12

Parse implementation of Value unwraps errors rather than forwarding them along to ParserError struct.

Integer literals in Alloy can be regular decimal numbers, integers with bases other than 10 can be written by prefixing the literal with appropriate prefix. Other supported radix are

• Binary prefixed with 0b
• Octal prefixed with 0o
• Hexadecimal with 0x

Crate nom is a parser combinator library focusing on safety and speed. Writing a custom parser will make syntax error reporting more rigorous. Since, nom can be used for parsing binary data same crate can be used for parsing bytecode and compiled files.

Parser rules for AST nodes have to be re-implemented and modify Parse trait.

• #7
• #8
• #9

Current AST implementation in Alloy includes enums such as Expression that is discriminated union of BinaryExpression, UnaryExpression etc.

// src/ast/expression/mod.rs:21
pub enum Expression {
    Value(Value),
    Binary(BinaryExpression),
    Unary(UnaryExpression),
    Identifier(IdentifierExpression),
}

This AST enum has to be derived from CST one candidate CST that can be used to generate Expression AST is provided below.

#[derive(AST)]
pub enum ExpressionCST {
    Value(Spanned<Value>),
    Binary(BinaryExpressionCST),
    Unary(UnaryExpressionCST),
    Identifier(IdentifierExpressionCST),
}

Originally posted by @atahanyorganci in #24 (comment)

Instructions should be stored in byte array [u8] for compact storage.

AST identifiers are generated by stripping trailing 'CST' suffix if it exists otherwise adding 'AST' to original identifier.

• #14
• #15
• #16
• #18
• #17