The yoakke from lucyelle

The Yoakke Compiler Infrastructure

Language- and Compiler development tools in .NET.

What is Yoakke?

Yoakke is a collection of libraries aimed at compiler- and language developers to help them in their development. Most common tasks - like lexing and parsing - are completely automated, but extensibility is kept in mind at every step. All components let you roll your custom solution, but are capable enough to fit most cases by default.

Status warning

Yoakke is still early in development. You can play around with the libraries, but there may be bugs or missing features. Please open an issue if you find a bug, or miss an important feature. All contributions are welcome!

A note for Visual Studio users

If you are experiencing build issues when using Visual Studio around Source Generators, please try enabling the option described here.

What components are there?

Lexing: Lexers can be defined completely declaratively, as regular expression annotations - inspired by Logos - over token types. Do not worry, the regular expressions are compiled into Deterministic Finite Automatas, so the resulting code is a lot more efficient, than your average RegEx engine.
Parsing: Parsers are defined using a BNF-like notation as annotations over methods that produce the syntax node for the appropriate rule - inspired by rply. The generated parser is a recursive descent one, but supports automatic transformation of direct left-recursion and precedence tables.
Syntax trees: ASTs usually are a pain to implement because of all the crud that goes on with them in C#. With an annotation you can generate equality and hash implementations for AST nodes, or define visitors for them.
Symbols: While lexical symbols are not too complex to implement, most compilers will need the same abstractions for it. Hence, this is also provided in a small library.
Error reporting: Nice error messages on console are a pain. You have to align arrows, properly color text segments, number lines and so on. Fortunately, there is a component that provides you a nice fluent API - inspired by the beautiful codespan library - to generate a message, render it however you want to. You can even create a custom syntax-highlighter for source lines!
Language Server: The Language Server Protocol defines a generic way for language tools to communicate with text editors. A library already implements the protocol, you just need to provide your own handler for events!

Library roadmap

This is a list of the libraries that are planned for implementation, along with their status:

Library	Status	Nuget	Notes
Lexer	🆗
Parser	🆗
Error message reporting	🆗
AST generator, visitor	🆗
Lexical symbols	🆗
Language server protocol	🚧
C syntax (pre-processor, lexer, parser)	🚧
Type-system	📝
Virtual Machine	📝
X86 Assembly	🚧
Debugger	📝
Dependency system (incremental compilation)	📝
Debug adapter protocol	📝

Done: ✅
Consider mostly done: 🆗
Work in progress: 🚧
Planned: 📝

Documentation

Documentation is very much work-in-progress, but you can always find the latest version in the Documentation folder. Feel free to open up Pull-Requests, if you see mistakes!

Developer documentation

Make sure to read our contributing document! Some developer documents about describing the projects is - hopefully - coming soon aswell.

Examples

Work in progress...

Sample projects

Work in progress...

Using Yoakke Badge

If you use Yoakke and would be willing to show it, here is a badge you can copy-paste into your readme:

<a href="https://github.com/LanguageDev/Yoakke" alt="Using Yoakke"><img src="https://raw.githubusercontent.com/LanguageDev/Yoakke/master/.github/resources/UsingYoakke.svg" title="Using Yoakke" alt="Using Yoakke"/></a>