GithubHelp home page GithubHelp logo

zetzit / zz Goto Github PK

View Code? Open in Web Editor NEW
1.6K 40.0 52.0 2.08 MB

πŸΊπŸ™ ZetZ a zymbolic verifier and tranzpiler to bare metal C

License: MIT License

Rust 76.65% Shell 0.13% C++ 18.40% C 4.80% HTML 0.02%

zz's Introduction

logo

ZetZ is for systems without dynamic memory, where C is and will remain the defacto standard system interface.
Target bare metal MCUs, embedded linux, WASM, and embed it in other languages.

You can also use it to build cross platform libraries, with a clean portable C-standard api.
Zetz plays nice with others and does not require rewriting everything in zetz to be useful in large projects.

A major innovative feature is that all code is formally verified by symbolic execution in a virtual machine, at compile time.

quick quick start

  1. Install https://github.com/Z3Prover/z3 usually through a distro package
  2. Get the latest binary from http://bin.zetz.it

Discord Community

https://discord.gg/EsMxjWtcf5

editor support

how it looks

ZZ has some go and rust aesthetics, but remains a C dialect at the core.

using <stdio.h>::{printf}

struct Random {
    u32 num;
}

fn rng(Random *self) u32 {
    return self->num;
}

export fn main() int {
    let r = Random{
        num: 42,
    };
    
    printf("your lucky number: %u\n", r.rng());
    return 0;
}

the basic ideas

plain C ABI

ZZ emits plain C and it will always do that. It is one of the main reasons it exists. It will always emit C into a C compiler which will then emit the binary.

most modern languages have their own ABI, deviating from C, so you have to use glue tools to play nice with others. with ZZ being emitted as C, all you do is include the header.

There is no stack unwinding (C++, rust), and no coroutines (go), so all code emits to plain ansi C with no requirements towards compiler features.

ZZ works nicely with vendor provided closed source compiler for obscure systems. Like arduino, esp32, propriatary firmware compilers, and integrates nicely into existing industry standard microkernels like zephyr, freertos, etc.

safety and correctness with symbolic execution

Unlike other modern languages, ZZ has raw unchecked pointers as default. Nothing prevents you from doing crazy things, as long as you have mathematical proof that what you're doing is defined behaviour according to the C standard.

Checking is done by executing your code in SSA form at compile time within a virtual machine in an SMT prover. None of the checks are emitted into runtime code.

The standard library is fully stack based and heap allocation is strongly discouraged. ZZ has convenience tools to deal with the lack of flexibility that comes with that, such as checked tail pointers.

namespaces, autogenerated headers and declaration ordering

No modern language has headers or semantically relevant declaration order and neither does ZZ. but since it interacts with raw C nicely, it also emits headers, and orders declarations so a c compiler likes them.

ZZ puts module namespaces into the C symbol using underscores instead of mangling. so my::lib::hello becomes my_lib_hello, which is C convention.

language reference

mutability: const, mut

by default, everything is const. this is the opposite of C. the mut keyword is used to make a global variable, or function argument mutable.

polymorphism

ZZ follows the C model of polymorphism: any struct can be cast to the same type as its first member. In ZZ the cast is implicit because it is always safe.

struct Vehicle {
    int wheels;
}

struct Car {
    Vehicle base;
}

fn allowed_entry(Vehicle *self) bool {
    return self->wheels <= 2;
}


fn main() {
    Car c{
        base: Vehicle {
            wheels: 4,
        }
    };
    assert(!c.allowed_entry());
}




where and model

ZZ requires that all memory access is mathematically proven to be defined. Defined as in the opposite of "undefined behaviour" in the C specification. In other words, undefined behaviour is not allowed in ZZ.

You will quite often be told that by the compiler that something is not provable, like indexing into an array.

this is not ok:

fn bla(int * a) {
    a[2];
}

you must tell the compiler that accessing the array at position 2 is defined. quick fix for this one:

fn bla(int * a)
    where len(a) == 3
{
    a[2];
}

this will compile. its not a very useful function tho, because trying to use it in any context where the array is not len 3 will not be allowed. here's a better example:

fn bla(int * a, int l)
    where len(a) >= l
{
    if l >= 3 {
        a[2];
    }
}

thanks to the underlying SMT solver, the ZZ symbolic executor will know that a[2] is only executed in the case where len(a) >= l >= 3, so it is defined.

The where keyword requires behaviour in the callsite, and the model keyword declares how the function itself will behave.

fn bla(int a) int
    model return == 2 * a
{
    return a * a;
}

In this simple example, we can declare that a function returns 2 times its input. But it actually does not, so this won't compile.

theory

we can use annotations to define states for types, which neatly lets you define which calls are legal on which type at a given time in the program without ANY runtime code.

theory is_open(int*) bool;

fn open(int mut* a)
    model is_open(a)
{
    static_attest(is_open(a));
    *a = 1;
}

fn read(int require<open> mut* a)
    where is_open(a)
    model is_open(a)
{
    return *a;
}

fn close(int mut* a)
{
    *a = 0;
}

the above example defines a type state transition that is legal: open -> read -> close any other combination will lead to a compile error, such as read before open.

storage: const, static, atomic and thread_local

const and static work exactly like in rust, but with C syntax.

export const uint32_t foo = 3;
static mutable float blarg = 2.0/0.3;
thread_local mutable bool bob = true;
atomic mutable int marvin = 0;

const is inlined in each module and therefore points to different memory in each module. static has a global storage location, but is private to the current module.

in effect, there is no way to declare a shared global writable variable. ZZ has no borrowchecker, and the restriction has nothing to do with preventing multithread races. Instead the declarations are selected so that the resulting exported binary interface can be mapped to any other language.

if you need to export a global writeable memory location (which is still a bad idea, because threads), you can define a function that returns a pointer to the local static.

thread_local and atomic are mapped directly to the C11 keywords. ZZ can use nicer keywords because there are no user defined names at the top level.

visibility: pub, export

by default all declarations are private to a module

"export" can be used to make sure the declaration ends in the final result. that is in the binary and the export header.

"pub" marks a declaration as local to the project. it is usable in other zz modules, but not exported into the resulting binary

struct initialization

To prepare for type elision, all expressions have to have a known type.

struct A {
    int a;
    int b;
}

fn main() {
    A a = A{
        a : 2,
    };
}

conditional compilation / preprocessor

Like in rust, the prepro is not a string processor, but rather executed on the AST after parsing. This makes it behave very different than C, even if the syntax is the same as C.

The right hand side of #if is evaluated immediately and can only access preprocessor scope.

struct A {
    int a;
#if def("TEST")
    uint proc;
#elif def("MAYBE")
    int proc;
#else
    void* proc;
#endif
}

Every branch of an #if / #else must contain a completed statement, and can only appear where a statement would be valid, so this is not possible:

pub fn foo(
#if os("unix")
)
#endif

note that even code that is disabled by conditions must still be valid syntax. It can however not be type checked,

a note on west-const vs east-const

ZZ enforces east-const. C is not a formally correct language, so in order to make ZZ formally correct, we have to make some syntax illegal. In this case we sacrifice west-const, which is incosistent and difficult to comprehend anyway.

west-const with left aligned star reads as if the pointer is part of the type, and mutability is a property of the pointer (which it is).

    int mut* foo;
    foo = 0; // compile error
    *foo = 0 // valid

unless you want to apply mutability to the local storage named foo

    void * mut foo;
    foo = 0; // valid
    *foo = 0 // compile error

Coincidentally this is roughly equivalent to Rust, so Rust devs should feel right at home. Even if not, you will quickly learn how pointer tags works by following the compiler errors.

closures

function pointers are difficult to do nicely but also make them emit to all languages well, so they don't really exist in ZZ. instead you declare closures, which are automatically casted from and to functions

closure rand_t() int;

fn secure_random() int {
    return 42;
}

fn main() {
    rand_t rand = secure_random;
}

metaprogramming or templates: tail variants

technically zz does not have metaprogramming. template functions blow up code size and are difficult to export as plain C types. instead zz makes code reusable by allowing allocations of structs to be larger than their member sizes.

We call this the "tail". And it can be used to make functions on fixed size arrays reusable for other sizes.

here's the String type:

export struct String+ {
    usize   len;
    char    mem[];
}

A + sign behind the name indicates this type has a tail. The tail here is mem, which is specified as array with no size.

a length function could be implemented with this signature:

fn len(String+t mut * self) {
    return t;
}

again, the + indicates a tail. in this case, the tail size is bound to a local variable named t, which can be used in the function body.

when allocating a new stack variable of type String, you also allocate a tail on the same stack

    String+100 s = {0};
    string::append(&s, "hello");
    string::append(&s, "world");
    printf("%.*s", s.len, s.mem);

again, + means tail, but here we specify an integer value of exact numbers of char we would like to add. the tail is measured in number of elements of whatever is the last unsized element in the struct, not in bytes.

String can dynamically expand within the tail memory. in this case, we append some stuff to the string, without ever allocating any heap. simply returning from the current function will clear up any memory used, without the need for destructor ordering or signal safety.

symbols

Symbols are a big global enum that lets you create unique values from anywhere in your code.

    using symbols;

    symbol Car;
    symbol Bike;

    fn drive_this(usize sym)
        where symbol(sym)
    {
        if sym == Car {
            printf("bzzzz\n");
        } else {
            printf("what do i do with a %s?\n", symbols::nameof(sym));
        }
    }

note that you cannot make assumptions about the integer value of a symbol, as it depends on compilation and loading order

new constructors

ZZ autogenerates bindings to more languages than C, and some languages are not fully compatible with C abi. Specifically they don't allow returning structs from functions, a standard way to create "constructor" functions in C.

intstead you should be using a function that takes a mut pointer as its first argument, and call it on a zero initialized stack variable. zz has syntactic sugar for this with the 'new' keyword.

    struct A {
        int a;
        int b;
    }

    fn empty(A mut new * self, int a)
    {
        self->a = a;
    }

    fn main() {
        new a = empty(3);
        assert(a.a == 3)
        assert(a.b == 0)
    }

new creates a new local variable with the correct size and passes it as self argument to the constructor

to create a local with a tail, use new like this:

    new+100 foo = string::empty();

procedural macros

macros in zz are fully compiled and executed at compile time for each call. this allows constructing arbitrary complex macros using regular zz code.

unlike C prepro macros, macros must emit complete expressions. for example you cannot emit an open brace without a closing brace.

a macro is compiled to a standalone executable, automatically including all dependencies. the call arguments and derive context is passed as json to stdin, and the macro is expected to print zz code to stdout.

/! creates literal string with arg0 repeated arg1 times
export macro repeat()  {

    new+1000 a = ast::from_macro();
    err::assert2(a.args[0].t == ast::Expression::LiteralString, "expected arg0: string");
    err::assert2(a.args[1].t == ast::Expression::Literal,       "expected arg1: number");
    let num = (int)atoi(a.args[1].v.string);

    printf("\"");
    for int mut i = 0; i < num; i++ {
        printf("%s", a.args[0].v.string);
    }
    printf("\"");
}

export fn main() int {
    printf("hello %s\n", repeat("world ", 32));
    return 0;
}

inline included C source

ZZ supports importing C source with the using keyword. Imported C source can be inlined at the call site by using the inline keyword. In certain cases the imported C source may depend on symbols defined in a ZZ module. ZZ symbols can be given to an imported C file with the needs keyword.

In the example below the native.h header file depends on the example_Container type to be defined. The lib.zz file imports the header file in line and exposes the Container type to the header file by specifying it with the needs keyword. The Container type is made available as example_Container because the project name is example and the type is Container.

// native.h
#include <stdio.h>
void print(example_Container *self) {
  printf("%s\n", (char *) self->value);
}

void init(example_Container *self, void const *value) {
  self->value = value;
}
// lib.zz
inline using (needs Container) "native.h" as native

export struct Container {
  void *value;
}

export fn create_container(Container new mut *self, void *value) {
  native::init(self, value);
}

pub fn print(Container *self) {
  native::print(self);
}
using example

fn main() int {
  new container = example::create_container("hello");
  container.print();
  return 0;
}

packed structs and unions

ZZ supports packed structs and unions with the packed modifier. This modifier, when used with struct or union, omits alignment padding. This is typically used for architecture independant serialization at the cost of causing alingment faults

Below is an example of a packed and unpacked struct and their static sizes printed to stdout.

using <stdio.h>::{ printf }

struct Packed packed {
  u8 a;
  u8 b;
  int b;
}

struct Unpacked {
  u8 a;
  u8 b;
  int b;
}

fn main() int {
  printf("sizeof(Packed) == lu\n", sizeof(Packed)); // 6
  printf("sizeof(Unpacked) == lu\n", sizeof(Unpacked)); // 8
  return 0;
}

environment variables

ZZ_MODULE_PATHS

When ZZ imports other ZZ modules it will look in a projects modules/ directory by default. The search path can be extended by defining the ZZ_MODULE_PATHS environment variable much like PATH environment variable where multiple paths can be defined separated by a colon (:) on POSIX systems and a semi-coloon (;) on Windows.

ZZ_MODULE_PATHS="$PWD/path/to/modules:/usr/share/zz/modules" zz build

zz's People

Contributors

acatton avatar aep avatar andresmargalef avatar bgeron avatar damon-kwok avatar destynova avatar dm9pzcaq avatar dvtpei avatar eindiran avatar fadedbee avatar jam1garner avatar jbpratt avatar jwerle avatar karroffel avatar peirick avatar rurban avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

zz's Issues

implement struct as bitvec

currently there's a bunch of edge cases in the prover because it makes assumptions about higher types. It really should tread everything as a memory blob and struct members as slices into the blob

`zz build` does not like dashes in project dir name

At commit b141f29.

$ mkdir left-right && cd left-right && zz init && zz build
project 'left-right' created
done emitting                                                                                                                                                                   /home/bnoordhuis/src/left-right/left-right/src/main.zz:3:50: error: expected ';' after top level declarator
int __attribute__ ((visibility ("default"))) left-right_main_main ();
                                                 ^
                                                 ;
1 error generated.
[ERROR] cc: [clang-10] args: [-fPIC -I . -I -fvisibility=hidden -g -fstack-protector-strong -fsanitize=address -Werror=implicit-function-declaration -Werror=incompatible-pointer-types -Werror=return-type -Wpedantic -Wall -Wno-unused-function -Wno-parentheses-equality -Wno-flexible-array-extensions -Wno-gnu-variable-sized-type-not-at-end -Werror=pointer-sign -Werror=int-to-pointer-cast -c target/test/zz/left-right_main.c -o ./target/test/zz/_left-right_main_bcf6c5e5607f650d607377da89bf2f53.o]
failed [Exe] left-right

Yices does not work out of the box

I ran the suggested commands to run hello on the Readme.md and it gave me an error message about not finding z3

Maybe remove the suggestion of installing yices if it doens't work?

procedural macros

Hello @aep,

What are your thoughts on not inserting a semicolon at the end of a macro evaluation?

For example:

macro string() {
  "hello"
}

printf("%s\n", string);

results in a compile error because a semicolon is inserted at the end of "hello"

Inconsistent treatment of keywords in zz.pest

I think that for clarity keywords should always have key_* rules and should be in the keyword rule. In particular (this list may not be correct or complete):

  • is, inline, where, model, for, while, switch, if, else, unsafe have key_* rules but don't use them;
  • struct, union, static, atomic, and thread_local have key_* rules but don't appear in the keyword rule;
  • true, false, as, export, extern, fn, fntype, symbol, theory, needs, using, test, macro don't have key_* rules at all.

The first group should be fixed just to improve readability, as it has no effect on the grammar accepted.

As for the second and third groups it may be that you don't want some of these keywords to be excluded from the identifiers, but I think context-sensitive keywords just make things more confusing. I recommend changing them all too, even though this will exclude some code that was formerly legal.

theory of pointers is inconsistent

I tried to build an example program with the snippet from README.md which uses theory for typestate enumeration / sequencing. However, I ran into syntax errors immediately, indicating that the syntax of theory has changed a bit recently.

So I looked at a working example in the typestate_wrong_order test, which compiles fine. I tried to simplify it to be more like the README example, using a straight int instead of a struct Socket { int fd }.
However, I've gotten stuck at either expected int* got int or static_attest leads to conflicting constraints.

Here's what I have right now:

using <stdio.h>::{printf}

export fn main() -> int {
  int mut i = 0;

  open(&i);
  read(&i);
  close(&i);

  printf("Greetings, tree! i = %d\n", i);
  return 0;
}

pub theory is_open(int *a) -> bool;

fn open(int mut *a)
    model is_open(a)
{
    static_attest(is_open(a) == true);
    static_assert(is_open(a) == true);
    *a = 1;
}

fn read(int mut *a) -> int
    where is_open(a)
    model is_open(a)
{
    return *a;
}

fn close(int mut *a)
    where is_open(a)
    model !is_open(a)
{
    *a = 0;
    static_attest(!is_open(a));
}

I was also unable to compile the new+ example of creating a string with a tail object.

To be honest, I only ever learned enough C to be extremely dangerous, and am quite unfamiliar with theorem provers (outside of a little excursion into Idris which was interesting but impractical).
It might be a good time to refresh the docs to make it easier for people to get started playing with zz? I'd love to help, but I'm still struggling with the basics πŸ˜“

Unicode support

Please implement unicode string support.

In C++, std::wstring is a wrapper for wchar_t* similar to std::string which is a wrapper for char*. wchar_t is defined in C as well [1]. A similar API in C is Glib::ustring.

The major difference to std::string is that a character is defined by 4 bytes rather than 1.

Add TODOs Badge to README

Hi there! I wanted to propose adding the following badge to the README to indicate how many TODO comments are in this codebase:

TODOs

The badge links to tickgit.com which is a free service that indexes and displays TODO comments in public github repos. It can help surface latent work and be a way for contributors to find areas of code to improve, that might not be otherwise documented.

The markdown is:

[![TODOs](https://badgen.net/https/api.tickgit.com/badgen/github.com/zetzit/zz)](https://www.tickgit.com/browse?repo=github.com/zetzit/zz)

Thanks for considering, feel free to close this issue if it's not appropriate or you prefer not to!

(full disclosure, I am the creator/maintainer of tickgit)

external c functions incorrectly accepted as safe

I played a bit, and this compiles:

using <stdio.h>::{printf, scanf};

export fn main() -> int {
    char foo[2];

    scanf("%s", foo);
    printf("hello unsound %s\n", foo);
    return 0;
}

this should be unsound for len(stdin) > 2 or 1 (depends on NUL-byte or not)
ASAN finds this at runtime, but it feels unsound.

zo meny tipos in raedme!11

It's always amusing to see people who want to teach us the Right Way to Write Correct Code but don't care enough to run their epiphany thru a spell-checker (text edit box in any contemporary browser is surely accessible to everyone?), or aren't attentive enough to tap that Shift key at the beginning of a sentence.

It's similar to many Rust projects, which have completely down-to-bottom change control policies and commit messages which don't really explain well the changes made. There must be a belief that having some mega-crutches like borrow checker or thery (sic! from README) keyword alleviates programmers from any further attentivity or thoroughness - you can do the rest in any sh/tty way and still get a stellar result.

Keep up the great work!

Mutation isn't checked properly

This is almost certainly a known issue, but just in case it isn't, and even if it is to document it. Mutation isn't handled properly, for example

fn a_lie(usize mut i) -> usize
model return == i {
    i += 1;
    return i;
}

export fn main() -> int {
    u8 mut arr[2] = {0, 1};
    printf("%d\n", arr[a_lie(1)]);
}

Expected output: Compiler error

Actual output: Stack buffer overflow

Theory behind ZZ

The README is extremely vague on this. How does it actually work?

Specifically, I have the following questions:

  • How are symbolic execution and SMT related?
  • How do you ensure that execution is finite?
  • How ZZ is related to Hoare-Floyd logic?
  • How ZZ is related to refined types?
  • Does ZZ suffer from the quantifier problem?

Is there a way to emit C source code only?

First of all, thanks for this, really great project!

I noticed that right now when we do zz build, it would generate the C code, then invoke a C compiler. The problem here, is that we are doing cross-compilation to RISC-V, hence we need a gcc that supports RISC-V. While I could of course set CC environment variable, I noticed zz hardcoded certain compiler flags that might not be supported well in different environments.

So the question now is: is there a mode that we can just use zz to emit the C code, the at a later stage manually compile the C source code to binary?

Convert TravisCI to GitHub Actions

I'd like to set up CI with GitHub Actions to replace TravisCI and eventually get automated builds triggered by tag releases. We can get binaries for Linux, macOS, and Windows automatically for every tag by leveraging the upload-artifact action.

Q: how to deref C struct?

At commit ec50a85. Test case:

using <stdio.h>::{printf};
using <dirent.h>::{DIR, (struct dirent) as dirent, opendir, readdir, closedir};

export fn main() -> int {
    DIR mut *d = opendir(".");
    for (;;) {
        dirent *e = readdir(d);
        if (e == 0) {
            break;
        }
        static_attest(safe(e));
        static_attest(safe(*e));
        printf("%d\n", e->ino);
    }
    closedir(d);
    return 0;
}

Fails to build with the following error message:

$ zz
comp [ t::main ]  0 / 1 [---------------------------------------------------------------------------------------------------------------------------------------------] 0.00 %  [ERROR] deref(S20_e) is not accessible as struct. it is ::ext::<dirent.h>::struct dirent
  --> /home/bnoordhuis/src/t/src/main.zz:13:25
   |
13 |         printf("%d\n", e->ino);␊
   |                         ^^
   |
   = cannot use as struct here

  --> /home/bnoordhuis/src/t/src/main.zz:13:16
   |
13 |         printf("%d\n", e->ino);␊
   |                ^-------------^
   |
   = last callsite

comp [ ]  1 / 1 [===================================================================================================================================================] 100.00 %  thread 'main' panicked at 'ICE: dependency ::t::main module doesnt exist', src/pipeline.rs:215:25
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

I'm wondering how to make that work. Grepping through zz's source tree, it might simply not be supported yet?

Q: Learning ZZ and Z3 - How to handle errors for file opening, etc.

Learning how to use this and what it all means. How does ZZ handle file opens that fail?

Obviously the example using where is_open isn't supposed to catch that case, since that is not runtime code, so does that mean that read is still supposed to do:

if file == NULL { return 0; }

or similar?

ifdef with c defines

the current preprocessor is bad and needs to be replaced. the design goals are:

  • must be expanded at C compile time, so that the exported C code is platform independent
  • all define variants must be complete values, so that the prover can run all of them independently.
  • variants can not change function signatures
  • default/else branch is required
  • must be able to conditionally include c headers

so far this is the syntax i came up with:

using <stdio.h>::{printf};
using err;
using log;
using string;

#if defined(__WIN32__)
using <windows.h>


const char * os = 
#if defined(__linux__)
"linux"
#else
"who knows"
;

export fn main() -> int

#if defined(__APPLE__)
{
    log::info("hello unknown \n");
    return 0;
}
#elif defined(__linux__)
{
    log::info("hello linux\n");
    return 0;
}
#else
{
    log::info("hello linux\n");
    return 0;
}

struct Bob
#if defined (WITH_SOMETHING)
{
  int something;
}
#else
{
 
}

@jwerle thoughts?

Error when setting CC=gcc

Apologies if this is just user error; I've just found zz and wanted to try running it with GCC.

I can force Clang to be used (which I guess is the default output anyway) by specifying the environment variable CC=clang. However, when I set CC to gcc, this happens:

~/code/zz/zz/examples/switch $ CC=gcc cargo run build --release
...
done emitting                                                                                                                                                            
gcc: error: : No such file or directory
[ERROR] gcc  -fPIC -I . -I -fvisibility=hidden -O03 -flto -fomit-frame-pointer -fno-exceptions -fno-asynchronous-unwind-tables -fno-unwind-tables -Werror=implicit-function-declaration -Werror=incompatible-pointer-types -Werror=return-type -Wpedantic -Wall -Wno-unused-function -Wno-parentheses-equality -Wno-flexible-array-extensions -Wno-gnu-variable-sized-type-not-at-end -Werror=pointer-sign -Werror=int-to-pointer-cast -c target/release/zz/switch_main.c -o ./target/release/zz/_switch_main_b62f26286d75fddaf1d47701e6be86f8.o
failed [Exe] switch   

However, if I copy/paste the GCC command from the output, it builds the .o file with no problems.

I added in some debugging but can't really see what the problem is. It dies at this part of src/make.rs:

            if step.is_dirty() {
                debug!("{} {:?}", self.cc, step.args);
                let status = Command::new(&self.cc)
                    .env("AFL_USE_ASAN", "1")
                    .args(&step.args)
                    .status()
                    .expect("failed to execute cc");
                if !status.success() {
                    error!("cc: [{}] args: [{}]", self.cc, step.args.join(" "));
                    ABORT.store(true, Ordering::Relaxed);
                }
            }

macros in Build Scripts

Currently, there is no way to depend on a library or source code that needs to be built outside of the ZZ build process. Listing source files as cobjects is not enough when the library source may require the traditional ritual of autoconf, configure, make, make install (with prefix).

An example of this are the libuv bindings over here

When a consumer of this module uses it, it will break because there is no way to build the libuv submodule before the ZZ module is built.

@aep What are your thoughts on supporting build scripts? We could have a [scripts] section of the zz.toml file similar to that of npm-scripts or something similar to Rust's Build Scripts.

Module Symbol Exports

Looking around at the docs emitter, it doesn't seem too difficult to emit all the parsed symbols in the loaded modules to something like JSON. This could make it easier to expose ZZ symbols in a module to other systems for tooling.

What are your thoughts on this @aep?

`static_attest` does nothing

This code snippet compiles when it is not supposed to. The static_attest(A[i] <= max_value); statement is used to tell the symbolic executor to verify that the max_value is indeed the biggest number in the array (it isn't) but ZZ doesn't reject the program as being invalid.

export fn main() -> int {
        usize n = 5;
        u64 A[n] = {3, 4, 5, 2, 1};
        u64 mut max_value = A[0];
/*
        for (usize mut i = 1; i < n; i++) {
                if (A[i] > max_value) {
                        max_value = A[i];
                }
        }
*/
        for (usize mut i = 0; i < n; i++) {
                static_attest(A[i] <= max_value);
        }
        return 0;
}

Generics

Vectors of structs seem like such a common paradigm. To elaborate on maybe an idea. Imagine we have some struct Foo.

struct Foo {
   int a;
}

static Vec<Foo> all_foo = [ null; 100 ]

fn main() -> int {
  
  for (int mut i = 0; i < num::len() ; i++) {
     printf("%d",all_foo[i].a); 
  }
  return 0;
}

I'd be curious how this is done today :)

Add suppport for manual specification of compile flags etc.

I haven't found documentation about this (I haven't looked at the code, yet).
It would be nice to specify the additional CFLAGS and LDFLAGS either via zz.toml (this is kinda undocumented) and via environment variables (this is currently missing).

Use case: I have a "semi-broken" clang install, which requires additional linker flags/libraries to successfully link. Otherwise, it errors out with something like the following:

/usr/lib/gcc/x86_64-pc-linux-gnu/9.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: /usr/lib/llvm/9/bin/../../../../lib/clang/9.0.1/lib/linux/libclang_rt.asan-x86_64.a(sanitizer_unwind_linux_libcdep.cc.o): in function `__sanitizer::Unwind_GetIP(_Unwind_Context*)':
(.text+0x1): undefined reference to `_Unwind_GetIP'
/usr/lib/gcc/x86_64-pc-linux-gnu/9.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: /usr/lib/llvm/9/bin/../../../../lib/clang/9.0.1/lib/linux/libclang_rt.asan-x86_64.a(sanitizer_unwind_linux_libcdep.cc.o): in function `__sanitizer::Unwind_Trace(_Unwind_Context*, void*)':
(.text+0x28): undefined reference to `_Unwind_GetIP'
/usr/lib/gcc/x86_64-pc-linux-gnu/9.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: /usr/lib/llvm/9/bin/../../../../lib/clang/9.0.1/lib/linux/libclang_rt.asan-x86_64.a(sanitizer_unwind_linux_libcdep.cc.o): in function `__sanitizer::BufferedStackTrace::UnwindSlow(unsigned long, unsigned int)':
(.text+0xdf): undefined reference to `_Unwind_Backtrace'
/usr/lib/gcc/x86_64-pc-linux-gnu/9.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: /usr/lib/llvm/9/bin/../../../../lib/clang/9.0.1/lib/linux/libclang_rt.asan-x86_64.a(sanitizer_unwind_linux_libcdep.cc.o): in function `__sanitizer::BufferedStackTrace::UnwindSlow(unsigned long, void*, unsigned int)':
(.text+0x2cd): undefined reference to `_Unwind_Backtrace'
clang-9: error: linker command failed with exit code 1 (use -v to see invocation)

(missing library would be probably -lunwind in this case).

Support namespace in project name

I can see much benefit from allowing one to do:

[project]
name = "namespace::library"

allowing:

using namespace::library::{ some_export }

ZZ currently complains that it can't find the module 'namespace::library' because of the :: contained in the name.

@aep what are your thoughts on support something like this? I hope this makes sense

Reference to module root

It would be pretty convenient to support an identifier or keyword that references the root of the module much like certain usage of Rust's crate keyword.

@aep what are your thoughts?

borrow() implementation

I am not sure I have fully understood how this works yet. I can see that: if a module exports a borrow function, it will automatically be called during compilation triggering any theories attached to it. Is that correct? Does that make sense?

Back to C Syntax?

ZZ over time has derived from C. Some things are for good reasons, but some are mostly just because i don't personally like the C syntax. this might create an unnecessary barrier from C to ZZ.

an examples of necessary changes are function type declarations. the C syntax is ambiguous and near impossible to parse.

An example of a change that i did simply because i just like rust syntax more is struct initialization.

I'd like to collect feedback on ZZ's syntax and the biggest changes versus C which may be a porting barrier.

https://www.reddit.com/r/zzlang/comments/f0ai54/back\_to\_c\_syntax

Does not build on windows

As for the step "Install Z3" - I downloaded it but you might expect it to be in the path or rely on
other means how to find it. Elaboration might be required.

Not sure about the reason, why build failed. Did as told ;)
Its windows 10 64 bit with VS2019 and a fairly recent rust.

cargo 1.40.0-nightly (3a9abe3f0 2019-10-15)
rustc 1.40.0-nightly (518deda77 2019-10-18)

E:\gitprojects\zz\examples\hello>cargo run run
    Updating crates.io index
  Downloaded fasthash v0.4.0
  Downloaded toml v0.5.1
  Downloaded cfg-if v0.1.9
  Downloaded serde_json v1.0.40
  Downloaded tempdir v0.3.7
  Downloaded which v3.1.0
  Downloaded rayon v1.1.0
  Downloaded pest_derive v2.1.0
  Downloaded bitflags v1.1.0
  Downloaded termcolor v1.0.5
  Downloaded pest v2.1.1
  Downloaded rsmt2-zz v0.11.1
  Downloaded serde v1.0.92
  Downloaded regex v1.1.7
  Downloaded xoroshiro128 v0.3.0
  Downloaded crossbeam-epoch v0.7.2
  Downloaded seahash v3.0.6
  Downloaded crossbeam-utils v0.6.6
  Downloaded libc v0.2.66
  Downloaded ucd-trie v0.1.1
  Downloaded pest_generator v2.1.0
  Downloaded fasthash-sys v0.3.2
  Downloaded failure v0.1.6
  Downloaded num-traits v0.2.8
  Downloaded ryu v1.0.0
  Downloaded utf8-ranges v1.0.3
  Downloaded rayon-core v1.5.0
  Downloaded pbr v1.0.1
  Downloaded pest_meta v2.1.1
  Downloaded regex-syntax v0.6.7
  Downloaded arrayvec v0.4.11
  Downloaded scopeguard v1.0.0
  Downloaded serde_derive v1.0.92
  Downloaded proc-macro2 v0.4.30
  Downloaded syn v0.15.36
  Downloaded error-chain v0.12.1
  Downloaded maplit v1.0.1
  Downloaded num_cpus v1.10.1
  Downloaded crossbeam-queue v0.1.2
  Downloaded autocfg v0.1.4
  Downloaded backtrace v0.3.40
  Downloaded memoffset v0.5.1
  Downloaded rustc-demangle v0.1.16
  Downloaded backtrace-sys v0.1.32
  Downloaded cc v1.0.48
   Compiling winapi v0.3.7
   Compiling cfg-if v0.1.9
   Compiling libc v0.2.66
   Compiling proc-macro2 v0.4.30
   Compiling lazy_static v1.3.0
   Compiling semver-parser v0.7.0
   Compiling unicode-xid v0.1.0
   Compiling cc v1.0.48
   Compiling syn v0.15.36
   Compiling arrayvec v0.4.11
   Compiling nodrop v0.1.13
   Compiling rustc-demangle v0.1.16
   Compiling memchr v2.2.0
   Compiling serde v1.0.92
   Compiling gcc v0.3.55
   Compiling autocfg v0.1.4
   Compiling winapi-build v0.1.1
   Compiling ucd-trie v0.1.1
   Compiling version_check v0.1.5
   Compiling scopeguard v1.0.0
   Compiling rayon-core v1.5.0
   Compiling bitflags v1.1.0
   Compiling ucd-util v0.1.3
   Compiling regex v1.1.7
   Compiling maplit v1.0.1
   Compiling ryu v1.0.0
   Compiling quick-error v1.2.2
   Compiling winapi v0.2.8
   Compiling unicode-width v0.1.5
   Compiling utf8-ranges v1.0.3
   Compiling strsim v0.8.0
   Compiling either v1.5.2
   Compiling seahash v3.0.6
   Compiling vec_map v0.8.1
   Compiling itoa v0.4.4
   Compiling log v0.4.6
   Compiling crossbeam-utils v0.6.6
   Compiling thread_local v0.3.6
   Compiling semver v0.9.0
   Compiling pest v2.1.1
   Compiling backtrace-sys v0.1.32
   Compiling kernel32-sys v0.2.2
   Compiling num-traits v0.2.8
   Compiling regex-syntax v0.6.7
   Compiling humantime v1.2.0
   Compiling error-chain v0.12.1
   Compiling textwrap v0.11.0
   Compiling fasthash-sys v0.3.2
   Compiling crossbeam-queue v0.1.2
   Compiling rustc_version v0.2.3
   Compiling pest_meta v2.1.1
   Compiling aho-corasick v0.7.3
error: failed to run custom build command for `fasthash-sys v0.3.2`

Caused by:
  process didn't exit successfully: `E:\gitprojects\zz\target\debug\build\fasthash-sys-3b8a307234c8fc88\build-script-build` (exit code: 101)
--- stdout
TARGET = Some("x86_64-pc-windows-msvc")
OPT_LEVEL = Some("0")
TARGET = Some("x86_64-pc-windows-msvc")
HOST = Some("x86_64-pc-windows-msvc")
TARGET = Some("x86_64-pc-windows-msvc")
TARGET = Some("x86_64-pc-windows-msvc")
HOST = Some("x86_64-pc-windows-msvc")
CC_x86_64-pc-windows-msvc = None
CC_x86_64_pc_windows_msvc = None
HOST_CC = None
CC = None
TARGET = Some("x86_64-pc-windows-msvc")
HOST = Some("x86_64-pc-windows-msvc")
CFLAGS_x86_64-pc-windows-msvc = None
CFLAGS_x86_64_pc_windows_msvc = None
HOST_CFLAGS = None
CFLAGS = None
DEBUG = Some("true")
running: "C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Community\\VC\\Tools\\MSVC\\14.24.28314\\bin\\HostX64\\x64\\cl.exe" "/nologo" "/MD" "/Z7" "-Wno-implicit-fallthrough" "-Wno-unknown-attributes" "-msse4.2" "-maes" "-mavx" "-mavx2" "/DT1HA0_RUNTIME_SELECT=1" "/DT1HA0_AESNI_AVAILABLE=1" "/W4" "/FoE:\\gitprojects\\zz\\target\\debug\\build\\fasthash-sys-f21c3bb3e1825f9e\\out\\src\\fasthash.o" "/c" "src/fasthash.cpp"
cargo:warning=cl : Command line error D8021 : invalid numeric argument '/Wno-implicit-fallthrough'
exit code: 2

--- stderr
thread 'main' panicked at '

Internal error occurred: Command "C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Community\\VC\\Tools\\MSVC\\14.24.28314\\bin\\HostX64\\x64\\cl.exe" "/nologo" "/MD" "/Z7" "-Wno-implicit-fallthrough" "-Wno-unknown-attributes" "-msse4.2" "-maes" "-mavx" "-mavx2" "/DT1HA0_RUNTIME_SELECT=1" "/DT1HA0_AESNI_AVAILABLE=1" "/W4" "/FoE:\\gitprojects\\zz\\target\\debug\\build\\fasthash-sys-f21c3bb3e1825f9e\\out\\src\\fasthash.o" "/c" "src/fasthash.cpp" with args "cl.exe" did not execute successfully (status code exit code: 2).

', C:\Users\Jochen.000\.cargo\registry\src\github.com-1ecc6299db9ec823\gcc-0.3.55\src\lib.rs:1672:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.

warning: build failed, waiting for other jobs to finish...
error: build failed

book / structured documentation

In an effort to help others get started with ZZ, we will start writing user guides in GitBook. The entries will be simple and approachable. It will target classic things like Installation & Setup, Prerequisites, Getting Started, Concepts, Hello World, Modules, Dependencies, User Space Repositories, Z3/Yices, Building, and more.

Make hello example simpler

I looked into this project and it was cognitive overload of what was happening. I think a simpler example would give someone a clearer picture whats going on.

Support for web assembly?

Hey, I'm a big fan of Rust, one of the reasons why I keep going back to it is they have amazing support for web assembly compilation. So does C with llvm! I think there's many devs out there who want a better language for web assembly, but C is pretty brutal. I've found myself looking for a good C trancompiled language for awhile, but none of them seem interested in making web assembly a first candidate target. Anyhow, I find ZZ interesting as a Rust dev who loves web assembly, and I would love to see a language like this compete as an alternative in that space.

Some thoughts I have for ZZ that would make it really appeal to web assembly devs (who tend to be javascript fanatics )

  • a package manager that makes installing dependencies easy
  • a more narrower standard library that basically only does memory allocation (since that's all wasm has access to) and wasm size is important ( think no_std + alloc )
  • easy command support so that a dev could compile .zz with a simple one line command that uses maximum optimization for wasm for reducing file size
  • one binary distributions for popular environments (windows, osx, linux )
  • an easy way to do #define export __attribute__((visibility("default")))
  • support for way better macros that can do things like construct json structures

I sometimes program web assembly in non-Rust languages, and I've created a fairly mature library that acts as a bridge between web assembly and the browser dom that is fairly technology agnostic.

This tech could accelerate zz in the web assembly space greatly. Here's an example of a hello world:

#include "../../js_ffi.h"

export int main() {
	int log = jsffiregister("window.alert");
	jsfficall1(JS_UNDEFINED,log,TYPE_STRING,(JSValue)(int)&"Hello World!");
	return 0;
}
clang -O3 --target=wasm32 -nostdlib -I./ -Wl,--no-entry -Wl,-allow-undefined -Wl,--export-dynamic -o main.wasm src/main.c

implement async on win32

#96 introduces the basic api design for async, but this needs to be validated if the api works well on win32.

C Import convenience

importing C code is not a good enough experience.

i believe the best course of action is to implement a rough c parser, equivalent in correctness to doxygen, as we only care about consuming the C api (header).

Better array syntax

Wouldn't it be much easier to declare arrays like C++ refs, just with constant or variable length for automatic bounds checking, without manual where clause.

Like &a[2] or &a[len] as signature,and then you also won't need to deref it.

Allow search path to be extended with environment variable

Hi @aep,

What are your thoughts on allowing one the ability to add a path to the search paths HashSet with an environment variable? This could allow users to specify the location of their modules if they are not located in standard directories when running zz:

$ ZZ_PATH=./path/to/modules zz build

I think something like:

if let Ok(zz_path) = std::env::var("ZZ_MODULES_PATH") {
	let module_paths = zz_path.split(":");
	for path in module_paths {
		searchpaths.insert(std::path::Path::new(&path).to_path_buf());
	}
} 

*p and p[0] are treated differently

using <stdio.h>::{printf, scanf};

fn foo(int *p) -> int {
  // return p[0]; // not ok
  return *p; // ok
}

export fn main() -> int {
    int mut y = 123;
    foo(&y);
    return 0;
}

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.