lycus / flect Goto Github PK

Namely:

• A formula for Homebrew.
• A MacPorts Portfile.
• A Fink package.
• An ebuild for Gentoo.
• A PKGBUILD for Arch Linux.
• A spec file for RPM.

Feel free to reply with more package managers/OSs that I should add to the list if you're interested in putting effort into writing a script for that target.

It'd be nice to have a way to annotate code with metadata.

The syntax might look something like:

[serializable, version = 3]
pub struct Foo {
    # ...
}

We should explore ways to leverage the Erlang VM's easy concurrency mechanisms to facilitate parallel or even distributed compilation.

It would be cool if we could do, say:

machine-one $ flect s
Starting Flect in server mode...

machine-two $ flect s
...

And then on the developer machine:

dev-machine $ flect c --dist true source1.fl source2.fl ...

And it would just automagically connect to the Flect nodes running in server mode on the network and do all the work there.

We'll need some kind of unwinding mechanism for handling conditions like running out of memory and assertion failures sooner or later.

The easiest thing to do would be to have code in the run-time library that simply uses libunwind. However, libunwind is not very friendly to non-C users; its API makes heavy use of macros (on some architectures, entire functions are implemented with macros), and even if we go through all the architecture-specific code and write the declarations exactly as they appear for each architecture (and port macros over to Flect code), there is still the problem that depending on how libunwind is built, the symbol names are different. Worse yet, libunwind doesn't work on all the systems/architectures that we target.

So, I don't know. Maybe there is some other viable option...

We can probably get away with inferring the type of a literal most of the time, so it makes sense to allow omitting it in some cases.

Using Elixir 0.10.1 and running make test fails in the tests/pp-fail suite.

I get the following error in the error_directive_unicode test:

    flect c --stage pp error_directive_unicode.fl ... ** exit {{:function_clause, [{String.Unicode, :next_grapheme, [["\\u0000000A"]], [file: '/private/tmp/elixir-j2M7/elixir-0.10.1/lib/elixir/priv/unicode.ex', line: 171]}, {String, :slice, 3, [file: '/private/tmp/elixir-j2M7/elixir-0.10.1/lib/elixir/lib/string.ex', line: 790]}, {Flect.String, :"-expand_escapes/2-fun-0-", 2, [file: '/Users/novabyte/Desktop/flect-lang/lib/string.ex', line: 30]}, {Enumerable.List, :reduce, 3, [file: '/private/tmp/elixir-j2M7/elixir-0.10.1/lib/elixir/lib/enum.ex', line: 1522]}, {Flect.String, :expand_escapes, 2, [file: '/Users/novabyte/Desktop/flect-lang/lib/string.ex', line: 29]}, {Flect.Compiler.Syntax.Preprocessor, :evaluate_directive, 2, [file: '/Users/novabyte/Desktop/flect-lang/lib/compiler/syntax/preprocessor.ex', line: 274]}, {Enum, :"-map_reduce/3-fun-0-", 3, [file: '/private/tmp/elixir-j2M7/elixir-0.10.1/lib/elixir/lib/enum.ex', line: 663]}, {Enumerable.List, :reduce, 3, [file: '/private/tmp/elixir-j2M7/elixir-0.10.1/lib/elixir/lib/enum.ex', line: 1522]}]}, {:gen_server, :call, [#PID<0.132.0>, {:work, Flect.Config[tool: :c, options: [stage: "pp"], arguments: ["error_directive_unicode.fl"]]}, :infinity]}}
    gen_server.erl:188: :gen_server.call/3
    /Users/novabyte/Desktop/flect-lang/lib/worker.ex:30: Flect.Worker.work/3
    /Users/novabyte/Desktop/flect-lang/test.exs:94: :elixir_compiler_0."-__FILE__/2-fun-9-"/3
    /private/tmp/elixir-j2M7/elixir-0.10.1/lib/elixir/lib/enum.ex:594: Enum."-map/2-lc$^0/1-0-"/2
    /private/tmp/elixir-j2M7/elixir-0.10.1/lib/elixir/lib/enum.ex:594: Enum."-map/2-lc$^0/1-0-"/2
    /private/tmp/elixir-j2M7/elixir-0.10.1/lib/elixir/lib/enum.ex:594: Enum."-map/2-lc$^0/1-0-"/2
    /Users/novabyte/Desktop/flect-lang/test.exs:39: (file)

We have fairly reasonable caret diagnostics in master now, but there is still some work to be done:

• Allow passing multiple locations to the logger (currently allows only one).
• Look source code up in the in-memory list of files instead of reading from disk.
• Allow disabling caret diagnostics (perhaps have a FLECT_DIAGS environment variable).

Not super important, but nice to have if practical:

• Mark the entire token relevant to a particular message.
• Colorize the token relevant to a particular message (e.g. the token would be red for errors).

Once Elixir 0.9.0 is out, we need to stabilize on that so that people don't have to grab the Git master version of Elixir to build Flect.

This makes it more friendly to Ctrl-C and such.

I stumbled across your project from the Pegged library (via an issue you raised that mentioned your CI server).

I'm an avid follower of Mozilla's Rust programming language and a beginner developer with D. I really like the goals of this project, the approach you're taking to Flect's language design is very interesting.

I attempted to build the current master for Flect from source, I was able to successfully build it but running the unit tests (via make test) produces the following error:

$ make test

  Testing tests/lex-pass (lex source files)...

    flect c --stage lex --dump tokens char.fl ... ** (UndefinedFunctionError) undefined function: IO.ANSI.escape/1
    IO.ANSI.escape("%{green, bright}ok (0)")
    /Users/chris/Desktop/flect/test.exs:38: :elixir_compiler_0."-__FILE__/2-fun-5-"/4
    /private/tmp/elixir-Tbxh/elixir-lang-elixir-437adc1/lib/elixir/lib/enum.ex:536: Enum."-map/2-lc$^0/1-0-"/2
    /Users/chris/Desktop/flect/test.exs:18: (file)
    src/elixir_compiler.erl:63: :elixir_compiler."-eval_forms/4-fun-0-"/7
    src/elixir_compiler.erl:62: :elixir_compiler.eval_forms/4
    src/elixir_compiler.erl:30: :elixir_compiler.string/2
    /private/tmp/elixir-Tbxh/elixir-lang-elixir-437adc1/lib/elixir/lib/code.ex:229: Code.require_file/2

real 0.41
user 0.33
sys 0.06
make: *** [test-lex-pass] Error 1

My config.mak looks like this:

# Generated by config.exs on {{2013,2,1},{9,9,47}}.

export FLECT_ARCH       ?= x86
export FLECT_OS         ?= darwin
export FLECT_ABI        ?= x86-sysv32
export FLECT_CROSS      ?= false

export FLECT_CC         ?= clang
export FLECT_CC_TYPE    ?= gcc
export FLECT_CC_ARGS    ?= 
export FLECT_LD         ?= ld
export FLECT_LD_TYPE    ?= ld
export FLECT_LD_ARGS    ?= 

export FLECT_PREFIX     ?= /usr/local
export FLECT_BIN_DIR    ?= /usr/local/bin
export FLECT_LIB_DIR    ?= /usr/local/lib
export FLECT_ST_LIB_DIR ?= /usr/local/lib/static
export FLECT_SH_LIB_DIR ?= /usr/local/lib/shared

I'm using the Erlang release from Homebrew which is currently at:

$ brew info erlang
erlang: stable R15B03-1 (bottled), HEAD
http://www.erlang.org
Depends on: automake, libtool
/usr/local/Cellar/erlang/R15B03-1 (6940 files, 272M) *
https://github.com/mxcl/homebrew/commits/master/Library/Formula/erlang.rb

Let me know if I can provide you with any more information.

We need a test runner that can execute arbitrary commands, keep count of passes/failures, and compare output.

At the moment I can't find any parser generators for Elixir. The language is probably not mature enough yet for developers to invest time to write PEG generators or recursive decent parser generators.

From the look of the source code for the Flect lexer and parser I believe that it's currently being handwritten?

It may be worth taking advantage of the direct support for Erlang with Elixir and use a parser generator for Erlang that you can use to rapidly prototype language syntax changes.

One of my work colleagues is developing a PEG parser generator for Erlang called Neotoma.

I don't particularly like D's version statement monstrosity. Maybe something could be built on top of macros...

There should be a FLECT_ENDIAN variable that is set to either big or little.

Something like:

'\u0000000A'

This would create a character literal with the line feed character (same as '\n'). The long notation is because in practice Unicode code points can have large enough numbers to need 32-bit storage. The number after the \u is the hexadecimal code point number (i.e. not the encoded UTF-8 value).

While being able to mark functions and types pub/priv is great, it doesn't cover the use case where something should be visible inside the bundle (library or executable) it belongs to. This is sort of like internal visibility in C#.

I don't particularly like C#'s internal visibility modifier, though. I think it actively encourages bad encapsulation and design because it can be sprinkled virtually anywhere. So, I think that a way to mark an entire module as internal would make a lot more sense; that way, the language will encourage proper separation of concerns and discourage cluttered module API design.

I'm thinking something like this:

pub mod foo::bar {
    # This module is exported from the bundle.
}

priv mod foo::bar {
    # This module can only be imported inside the bundle.
}

The lexer currently reports completely wrong locations for tokens such as identifiers and number literals. This is because it only threads the current location through functions. Should be trivially fixable by adding a cur_loc parameter to most functions that lex complex tokens.

The problem: We want to be able to retrieve information about various declarations at compile time.

The planned solution: A new meta expression.

The syntax would look something like this:

meta-expr ::= "meta" ( meta-type-expr | meta-fn-expr | meta-trait-expr | meta-glob-expr | meta-macro-expr )
meta-type-expr ::= "type" type
meta-fn-expr ::= "fn" qualified-name
meta-trait-expr ::= "trait" qualified-name
meta-glob-expr ::= ( "glob" | "tls" ) qualified-name
meta-macro-expr ::= "macro" qualified-name

Each of these would return a managed pointer to a particular structure in core::rtti; specifically, @core::rtti::TypeInfo, @core::rtti::FunctionInfo, @core::rtti::TraitInfo, @core::rtti::GlobalVarInfo, @core::rtti::ThreadVarInfo, and @core::rtti::MacroInfo, respectively.

So, we could say:

pub struct Foo {
    pub bar : i32;
    pub baz : f64;
}

let ty : @core::rtti::TypeInfo = meta type Foo;

We could now use ty to discover fields in Foo (i.e. bar and baz).

Ideas, thoughts, suggestions, criticisms, etc, go!

Useful to figure out whether running the runnable parts of the test suite makes sense on the host at all.

In the D programming language it's possible to create functions that can be invoked at compile-time by the compiler to generate new code. This functionality is known as Compile-Time Function Execution (CTFE). It's also possible to achieve similar results using C++ Templates but is often much harder to debug and maintain.

Two D libraries that take great advantage of CTFE are:

• std.regex, this library can generate the FSA used to handle the regex pattern matching as a string mixin that's passed to the compiler (by the compiler) before compilation.
• Pegged, this library can generate a PEG parser at compile-time that's also passed to the compiler before the compilation phase begins.

To take best advantage of CTFE it's important to add support for string mixins so that the compiler can parse and compile the code generated by the compile-time function.

CTFE can be extremely useful when generating code to handle a data serialization format (i.e. Google Protocol Buffers), generate information at compile-time based on version information or read an XML specification to generate a lexer & parser (for example, with the FIX protocol)... and many other use cases.

We already spoke about this on IRC and I know you already know about CTFE in D but I wanted to provide some rationale for it to anyone else interested in Flect.

Let me know what you think.

:)

The lexer currently takes around 10 milliseconds to tokenize a file like this:

mod foo::bar::baz {
    # this is a comment
}

This is weird considering the lexer is written in a fairly performance-oriented way.

Some things that could be problematic:

• Multi-code point tokens are constructed by creating new Erlang binaries for every code point.
• The identifier character lookup is currently a simple binary search in a list. Same for digits, and so on.