joncatanio / cannoli

Cannoli Programming Language

Cannoli Programming Language

Cannoli is a compiler for a subset of Python 3.6.5 and is designed to evaluate the language features of Python that negatively impact performance. Cannoli is written in Rust and also compiles Python to Rust. The use of Rust as the intermediate representation was chosen for performance purposes and to avoid writing a garbage collector. Cannoli was developed as work for a Master's Thesis at Cal Poly - San Luis Obispo.

Python Support

Cannoli supports a subset of Python 3.6.5, its current state omits many features that could not be completed during the duration of the thesis. The main omissions are exceptions and inheritance. Standard library support is also incomplete but covers numerous proofs-of-concepts that could be applied to other types and modules (see Cannolib).

Optimizations

Cannoli supports two major optimizations that come as a result of applying restrictions to the language. Restrictions are placed on the Python features that provide the ability to delete or inject scope elements and the ability to mutate the structure of objects and classes at run time. The corresponding feature branches are scope-opts and class-opts. The optimizations are built on top of each other, therefore the class-opts branch is a superset of the scope-opts branch. In general, the class-opts branch yields a performance increase of over 50% from the master branch.

Thesis Paper

More information on the results and implementation details of Cannoli can be found in the thesis paper.

👉 Leave the Features: Take the Cannoli - Jonathan Catanio

How to Run

• Install Rust by following the instructions on their official installation guide. Cannoli is both compiled with Rust 1.24.0 and outputs Rust 1.24.0 code. Changing versions with the Rust toolchain can be done with the rustup utility.
• Build the project by running cargo build or cargo build --release in the project's root directory. This will create a target directory with debug or release subdirectories containing the executable.
• Compile a Python file by executing the command ./target/release/cannoli [src.py]

Executing the Compiled Python

Compiling with Cannoli outputs a main.rs file that can be used in a standalone Rust crate. Ideally the Cannoli compiler would utilize rustc to output a binary but this wasn't done. The steps on how to run this compiled file are as follows:

• Create a new crate with the command cargo new --bin sandbox. Where sandbox is the name of the crate.
• Move the main.rs file into the new crate's src directory.
• Update the Cargo.toml file to include cannolib as a dependency, this is Cannoli's standard library. This can be done by linking to a git repository or to a relative crate.

[dependencies]
cannolib = { git = 'https://github.com/joncatanio/cannolib', branch = 'master' }

Would include the master branch of the Cannolib git repo in the crate.

• To add debugging and all optimizations to release mode, also include these lines in the Cargo.toml:

[profile.release]
debug = true
codegen-units = 1

• Finally, build the project from the crate's root directory with cargo build --release and run the executable with the command ./target/release/sandbox.

Cannolib

Cannolib provides library support for Cannoli, including a number of types and modules that offload work that would have otherwise been done by the Cannoli compiler. Cannolib provides the implementation of the overall type system as well as built-in functions similar to those defined in the Python library.

cannoli's Issues

Add Meta Data to AST nodes

Fields like lineno would be really beneficial to have in the AST so the compiler can report better error messages. Doing this in rust is a little more complicated since we can't pattern match structs and there isn't really a way to extend a class like one would in Java. We could apply traits to structs/enums but this only allows us to create similar functions not fields. The best way to achieve this is use a combination of enums and structs. An encapsulating struct can provide common fields with a variant field that represents the AST enum that is currently used.

enum AstNode {
   stmt: Statement,
   expr: Expression,
   ...
}

struct Node {
   lineno: usize,
   ...,
   variant: AstNode
} 

This is just a simple example that allows us to avoid adding common fields to each enum arm.

crate/s

cargo package canolli canollib
hopefully get a few devs interestd in updating pull requests etc..

for docker or small games ie mobile having a slim'ed down python might be good.

# Copyright 2017-2020 Gentoo Authors
# Distributed under the terms of the GNU General Public License v2

# Auto-Generated by cargo-ebuild 0.3.0

EAPI=7

CRATES="
aho-corasick-0.5.3
ansi_term-0.11.0
atty-0.2.14
bitflags-1.2.1
cannoli-0.2.0
clap-2.33.1
hermit-abi-0.1.14
kernel32-sys-0.2.2
lazy_static-1.4.0
libc-0.2.71
memchr-0.1.11
regex-0.1.80
regex-syntax-0.3.9
strsim-0.8.0
textwrap-0.11.0
thread-id-2.0.0
thread_local-0.2.7
unicode-segmentation-0.1.3
unicode-width-0.1.7
unicode_names-0.1.7
utf8-ranges-0.1.3
vec_map-0.8.2
winapi-0.2.8
winapi-0.3.9
winapi-build-0.1.1
winapi-i686-pc-windows-gnu-0.4.0
winapi-x86_64-pc-windows-gnu-0.4.0
"

inherit cargo GIT-R3.ECLASS

DESCRIPTION="cannoli"
# Double check the homepage as the cargo_metadata crate
# does not provide this value so instead repository is used
HOMEPAGE="https://github.com/joncatanio/cannoli"
SRC_URI="$(cargo_crate_uris ${CRATES})"
EGIT_REPO_URI="https://github.com/joncatanio/cannoli.git"
# git-r3_checkout
RESTRICT="mirror"
# License set may be more restrictive as OR is not respected
# use cargo-license for a more accurate license picture
LICENSE="Apache-2.0 MIT Unlicense"
SLOT="0"
KEYWORDS="~amd64"
IUSE=""

DEPEND=""
RDEPEND=""

Update the enum context throughout the parser.

A context should be passed down through to each call. CPython represents this context with keywords like del which may change the context from Load or Store to Del. It shouldn't be conceptually difficult but it is quite a large update that needs to be done when the parser is in a more complete state.

Move lexer/parser into own crate

It would be handy to have the lexer/parser separate from the compiler if possible. That would allow projects to make an interpreted version without having to rewrite the lexer/parser.

Write Cannoli Parser

Write the parser for a subset of the Python 3.6.3 grammar. Focuse on the file_input definition and work from there. Ignore decorators and the eval & exec functionality.

Resources

• Python 3.6.3 Grammar

Add lots of error checking to parser.

In the parser's current state there are not very helpful error messages. The parser operates completely fine on functioning code but doesn't make many validation checks like those found in the CPython parser. These need to be updated as much as possible.

Add Rust Doc Comments

Add concise and helpful rust doc comments prefixed with /// or //! to all modules and crates.

Resources:

• Rust By Example comment overview
• Rust By Example intro to comments
• Rust 2nd Edition Book