dyne / zenroom Goto Github PK

Embedded no-code VM executing human-like language to manipulate data and process cryptographic operations.

License: GNU Affero General Public License v3.0

Makefile 0.67% Shell 12.84% C 66.79% Lua 11.98% C++ 0.51% HTML 4.23% CSS 0.03% Roff 0.07% JavaScript 0.02% Python 1.18% Dockerfile 0.01% CMake 1.10% Go 0.05% Java 0.03% TypeScript 0.26% Meson 0.12% GDB 0.01% Rust 0.06% PHP 0.04% Objective-C 0.03%

musl-libc lua vm language blockchain portable portable-virtual-machine dyne osx arm

zenroom's Introduction

Zenroom crypto VM

Zenroom is a secure language interpreter of the domain-specific Zencode, making it easy to execute fast cryptographic operations on any data structure.

The Zenroom VM is very small, has no external dependency, is fully deterministic and is ready to run end-to-end encryption on any platform: desktop, embedded mobile, cloud micro-services, and web browsers. Zenroom works inside applications written in Javascript, Python3, Rust or Golang.

Zencode has a no-code approach. It is a domain-specific language (DSL) similar to human language. One can process large data structures through complex cryptographic and logical transformations.

Zencode helps developers to empower people who know what to do with data: one can write and review business logic and data-sensitive operations without learning to code.

Timeline

2017 - Proof of Concept
2018 - Prototype and Alpha release series
2019 - Stable release series v1 (now EOL)
2022 - Stable release series v2 until LTS v2.22.1
2023 - Stable release series v3 until LTS v3.23.4
2024 - Current stable v4 (latest HEAD)

Read the full Changelog for more infos

This software has zero dependencies. If you chose to use it in your project be assured that we intend to support the same code to still run in 20 years from now on any target platform.

Links

Continue to the developer website

Read the Zencode whitepaper

Visit the product website for a friendly introduction to the love we put in craftsmanship.

For many quick running examples visit the ApiRoom online IDE powered by Zenroom.

Zenroom is licensed as AGPLv3; we are open to grant exceptions on a commercial basis.

Applications

Many applications already include Zenroom and use the Zencode language.

Getting Started

To quickly try out Zenroom using the Zencode language with some examples navigate to ApiRoom and start typing into the browser.

The Zenroom VM runs locally in your browser (needs WASM) and ApiRoom provides various examples to show operational crypto flows.

ApiRoom is also an IDE (Integrated Development Environment) and by signing in with a username and password you can save your contracts and download them as a Dockerfile micro-service ready to deploy.

Tutorials

Build

Dependencies: makefile, cmake, zsh, gcc, libreadline-dev

Optional: musl-libc, emscripten for wasm builds

Use this command sequence:

git clone https://github.com/dyne/zenroom
cd zenroom
make linux

to create the CLI executable in src/zenroom

make linux-lib

to create the shared library in src/libzenroom-x86_64.so

make

to list more available targets

Meson + Ninja

Practical build scripts for GNU/Linux are provided using Meson + Ninja

make meson

Will produce a zenroom executable and a libzenroom shared lib in zenroom/build.

License

Designed and written by Denis "Jaromil" Roio with the help of Puria Nafisi Azizi and Andrea D'Intino.

Includes code contributions by Alberto Lerda, Matteo Cristino, Danilo Spinella, Luca Di Domenico and Rebecca Selvaggini.

Reviews and suggestions contributed by: Richard Stallman, Daniele Lacamera, Enrico Zimuel, Sofía Celi, Sebastian Blichfeld, Adam Burns, Thomas Fuerstner and Jürgen Eckel.

Zenroom complies with the REUSE license specification and redistributes:

Lua 5.4 - Copyright © 1994–2023 Lua.org, PUC-Rio.
Apache Milagro Crypto Library (AMCL)
Various Lua libraries released under Apache/MIT license

Special thanks to our colleagues in the DECODE project whose research has inspired the birth of this project: Francesca Bria, George Danezis, Ola Bini, Mark de Villiers, Ivan Jelincic, Alberto Sonnino, Jim Barritt, Christian Espinoza, Samuel Mulube and Nina Boelsums.

This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details.

You should have received a copy of the GNU Affero General Public License along with this program. If not, see https://www.gnu.org/licenses/.

zenroom's People

Contributors

Stargazers

Watchers

zenroom's Issues

Statement " Given my name is in a 'string' named 'myName' " crashes in zenroom npm is parameter is passed via data

@puria

In zenroom NPM, the script works only if you pass it from "-k":

Scenario 'ecdh': Create the keypair from a name passed from data/keys
Given my name is in a 'string' named 'myName'
When I create the keypair
Then print my data

{
	"myName": "User123456"
}

In zenroom CLI it works when passed both as -a or -k

broken javascript

Likely related to last PRs the javascript-* targets are broken and consequently the online demo

Need to fix urgently in particular the javascript-demo target (wasm based)

To reproduce use make check-js on javascript-asmjs or javascript-wasm

Questions about C API usage

First of all this is a very interesting project that nicely encapsulates the complexities of cryptography using a nice DSL. I'm experimenting with the C API as described here, but have a few questions about its usage:

What is the difference between the parameter keys and data? When will the Zencode fail if keys data is passed into data instead of keys (or vice versa)?
Is there a way to suppress all printing to console by the library? Currently the following is printed when making calls to the API:

[*] Memory manager selected: lightweight
 .  ECDH curve is GOLDILOCKS
 .  Memory in use: 277 KB

Is there documentation somewhere about all the configuration options that can be passed?
Does the return of the C API only return 0 for success and 1 for failure, or does it return additional error codes?

Change from DECODE-0.3 to 0.4 is disabling way to pass data to zenroom

Hello,
Using zenroom version DECODE-0.3 I can pass data to it to encrypt and decrypt data, i.e:
echo '{"myval":1}' | ./zenroom -k device_keys.json encode.lua - > my_encrypt_data

but in the version DECODE-0.4 I can't use this feature as now STDIN is being treating as lua code then I can't use the DATA global variable in my lua scripts as before.

https://github.com/DECODEproject/zenroom/blob/e7d5d8c7648d3f4cef289410accde7e1aa52a4cc/src/zenroom.c#L274

> echo '{"val":1}' | ./zenroom-static encode.lua -
[*] Zenroom - crypto language restricted execution environment DECODE-0.3
 .  Copyright (C) 2017-2018 Dyne.org foundation
 .  reading KEYS from file
 .  loaded file: device_keys.json (143 bytes)
 .  reading DATA from stdin
 .  loaded file: test2.lua (370 bytes)
 .  using default configuration
 .  done loading all extensions
Mydata Gp3HC8aXXaXwhxm92BMswrejGniqMNcdhamuWEEuRkhV5GnXoLkjW5Yh8dD1dRDQ1k9AHe68Mhxmb1vpBCdXTchEQ1V1kjUTvb8fQT
 .  used memory: 123370 bytes
 .  executed operations: 0
[*] Zenroom operations completed.

> echo '{"val":1}' | ./zenroom-static encode.lua -
[*] Zenroom - crypto language restricted execution environment DECODE-0.4
 .  Copyright (C) 2017-2018 Dyne.org foundation
 .  reading CODE from stdin
 .  loaded file (10 bytes)
 .  using default configuration
 .  loading language extensions
 .  done loading all extensions
[!] lua error
[!] [string "{"val":1}..."]:1: unexpected symbol near '{'
[!] Error detected. Execution aborted.

Additionally to this I wanted to test this issue in my mac with the v0.3 but there is no available builds in Jenkins for Mac with this version, so I built my own artefacts checking out v0.3 tag, but the resulting executable was showing v0.2 version.

Thanks in advance.

Solution loading zenroom.wasm through webpack.

note: this is not a bug or issue in zenroom, but a general recommendation for anyone trying to make zenroom work in the browser with webpack

When using Zenroom in the browser, with a framework like Vue.js or React.js, .wasm files are parsed by webpack. This produces a cascade of compile errors, like:
failed to asynchronously prepare wasm: CompileError: wasm validation error: at offset 0: failed to match magic number
and
CompileError: AsyncCompile: Wasm decoding failed: expected magic word 00 61 73 6d, found 3c 21 44 4f @+0

Old way breaks with newer webpack

In the following: guide @puria demonstrated how to deal with this issue, but with more recent versions of webpack this method seems to break.

Solution:

webpack config

Following the recommendations in this guide by @dkirkman on how to load .wasm files, my webpack config contains the following rule for wasm files:

 module: {
    rules: [
      { 
        test: /\.wasm$/,
        type: 'javascript/auto',
        loader: 'file-loader',
        options: {
          name: '[name]-[hash].[ext]',         
        }        
      }
    ]
  },

No need to edit zenroom.js...

TL;DR:

leave zenroom.js untouched
copy (or symlink) zenroom.wasm to public/js

In the original guide, it was recommended to comment the locateFile line in zenroom.js. However, webpack seems to search for the file in public/js/. So, keep your zenroom.js untouched.

/* Suppose to comment this in original guide, but: no need to comment this. */
   wasmBinaryFile = locateFile(wasmBinaryFile);

... and move zenroom.wasm to public/js

and move your zenroom.wasm to public/js

AGPL seems a little bizzare for embedding

For a system that is meant to be embedded (especially one to replace cryptographic primitives) it seems a bizzare choice that makes it, well, much more difficult to embed.

Some kind of link exception would make sense, although it looks like (with the reference to "enterprise support" on the page, and the existence of a CLA) this is supposed to be dual licensed. But it doesn't appear obvious that is the case either.

Private information is leaked in Android and iOS

Running zenroom in Android or iOS, the results are printed in the log console. These are readable by 3rd parties.

Scalable argument passing

Playing with zenroom and its working out pretty well.

I think the "-a" for arguments is working out pretty well for simple scripts but it isn't easy to integrate with more complicated use cases.

My use case is as follows:

I have:

a lua file e.g. foo.lua
a json file of keys e.g. keys.json
a process emitting data

What I would like to do is something like

echo "{'temp' : 22.3}" | ./zenroom -a ./keys.json ./foo.lua

I would like the following to happen

zenroom loads foo.lua
get the keys from via the json.decode( arguments)
somehow get the data from the echo

Maybe a solution could be multiple arguments e.g

./zenroom -a ./keys.json -a $( echo "{'temp' : 22.3}" )  ./foo.lua

and make arguments an array of values e.g.

keys =  json.decode( arguements[0])
data =  json.decode( arguements[1])

Maybe it could be something else?

I do think that there should be a scalable way of passing data into zenroom.
Ideally I would like to use it as follows -

./zenroom ./foo.lua ./keys.json $( echo "{'temp' : 22.3}" )

My main reason would be one of familiarity.

Running zenroom_exec.js failes with error: Module.print is not a function

Hi @jaromil,

Last Wednesday we met at the Decode office of the gemeente, so I was excited to start working with Zenroom.
I downloaded the Zenroom-0.8.0-javascript package and tried to run 'zenroom_exec.js' with,
node zenroom_exec.js examples/hello.lua

but got an error saying,

 .  loading lua initialisation
/.../Zenroom-0.8.0-javascript/nodejs/zenroom.js:1
(function (exports, require, module, __filename, __dirname) { var Module=typeof Module!=="undefined"?Module:{};if(!Module.expectedDataFileDownloads){Module.expectedDataFileDownloads=0;Module.finishedDataFileDownloads=0}Module.expectedDataFileDownloads++;((function(){var loadPackage=(function(metadata){function runWithFS(){var fileData0=[];fileData0.push.apply(fileData0,[45,45,32,105,110,105,116,32,115,99,114,105,112,116,32,101,109,98,101,100,100,101,100,32,97,116,32,99,111,109,112,105,108,101,32,116,105,109,101,46,32,32,101,120,101,99,117,116,101,100,32,105,110,10,45,45,32,122,101,110,95,108,111,97,100,95,101,120,116,101,110,115,105,111,110,115,40,76,41,32,117,115,117,97,108,108,121,32,97,102,116,101,114,32,122,101,110,95,105,110,105,116,40,41,10,10,74,83,79,78,32,32,32,61,32,114,101,113,117,105,114,101,40,39,106,115,111,110,39,41,10,83,67,72,69,77,65,32,61,32,114,101,113,117,105,114,101,40,39,115,99,104,101,109,97,39,41,10,82,78,71,32,32

TypeError: Module.print is not a function
    at Array.<anonymous> (/.../Zenroom-0.8.0-javascript/nodejs/zenroom.js:1:1065157)
    at _emscripten_asm_const_ii (/.../Zenroom-0.8.0-javascript/nodejs/zenroom.js:1:1065763)
    at Array.Qc (/.../Zenroom-0.8.0-javascript/nodejs/zenroom.js:5:59039)
    at fh (/.../Zenroom-0.8.0-javascript/nodejs/zenroom.js:7:34701)
    at Mj (/.../Zenroom-0.8.0-javascript/nodejs/zenroom.js:6:53516)
    at hh (/.../Zenroom-0.8.0-javascript/nodejs/zenroom.js:7:35275)
    at Array.rg (/.../Zenroom-0.8.0-javascript/nodejs/zenroom.js:7:8220)
    at Object.pB [as dynCall_vii] (/.../Zenroom-0.8.0-javascript/nodejs/zenroom.js:13:42791)
    at invoke_vii (/.../Zenroom-0.8.0-javascript/nodejs/zenroom.js:1:1160577)
    at _g (/.../Zenroom-0.8.0-javascript/nodejs/zenroom.js:7:27894)

I did wanted to get it working so I changed a function call in 'nodejs/zenroom.js' from,
(function($0){Module.print(UTF8ToString($0))})

to,
(function($0){console.log(UTF8ToString($0))})

which after running node zenroom_exec.js examples/hello.lua again returned,

 .  loading lua initialisation
Hello World!
in hex: 48656c6c6f20576f726c6421
in base64: SGVsbG8gV29ybGQh
re-assign from hex, length 12
as string: Hello World!
in base64: SGVsbG8gV29ybGQh
re-assign from base64, length 12
as string: Hello World!
in hex: 48656c6c6f20576f726c6421
[*] Zenroom operations completed.
 .  Zenroom teardown.

Is there a fix so I can use the default zenroom.js?
Thanks :)

Running this on Mac OS 10.13.6 with Node v8.6.0

Messed up _zenroom.so file in PyPi package 1.0.12

Hello zenroom keepers!

I've been playing around with zenroom recently and stumbled upon an issue with with pypi package.

After trying to from zenroom import zenroom I got this traceback:

Traceback (most recent call last):
  File "/root/.local/lib/python3.6/site-packages/zenroom/zenroom_swig.py", line 26, in swig_import_helper
    pkg = __name__.rpartition('.')[0]
  File "/usr/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'zenroom._zenroom'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/root/.local/lib/python3.6/site-packages/zenroom/zenroom.py", line 5, in <module>
    from .zenroom_swig import zenroom_exec_tobuf, zencode_exec_tobuf
  File "/root/.local/lib/python3.6/site-packages/zenroom/zenroom_swig.py", line 32, in <module>
    _zenroom = swig_import_helper()
  File "/root/.local/lib/python3.6/site-packages/zenroom/zenroom_swig.py", line 26, in swig_import_helper
    pkg = __name__.rpartition('.')[0]
  File "/usr/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
ImportError: libpython3.7m.so.1.0: cannot open shared object file: No such file or directory

Which is strange since I have python 3.6.9 on that machine.
It turns out that not all _zenroom.so files are properly linked.

I went ran this ldd script in package directory.

for file in $(find | grep  .*_zenroom.so); do
    echo $(dirname $file) '->' $(ldd $file | grep -oE "libpython.{0,12}")
done

and got this:

./libs/Linux/3_6_2 -> libpython3.6m.so.1.0 libpython3.6m.so.1.0
./libs/Linux/3_6_5 -> libpython3.6m.so.1.0 libpython3.6m.so.1.0
./libs/Linux/3_7_2 -> libpython3.7m.so.1.0
./libs/Linux/3_5_0 -> libpython3.5m.so.1.0
./libs/Linux/3_6_4 -> libpython3.6m.so.1.0 libpython3.6m.so.1.0
./libs/Linux/3_5_1 -> libpython3.5m.so.1.0
./libs/Linux/3_6_6 -> libpython3.6m.so.1.0 libpython3.6m.so.1.0
./libs/Linux/3_7_0 -> libpython3.7m.so.1.0
./libs/Linux/3_5_2 -> libpython3.5m.so.1.0
./libs/Linux/3_5_4 -> libpython3.5m.so.1.0
./libs/Linux/3_6_3 -> libpython3.6m.so.1.0 libpython3.6m.so.1.0
./libs/Linux/3_6_9 -> libpython3.7m.so.1.0
./libs/Linux/3_7_4 -> libpython3.7m.so.1.0
./libs/Linux/3_5_3 -> libpython3.5m.so.1.0
./libs/Linux/3_6_7 -> libpython3.6m.so.1.0 libpython3.6m.so.1.0
./libs/Linux/3_6_1 -> libpython3.6m.so.1.0 libpython3.6m.so.1.0
./libs/Linux/3_7_3 -> libpython3.7m.so.1.0
./libs/Linux/3_5_5 -> libpython3.5m.so.1.0
./libs/Linux/3_6_8 -> libpython3.6m.so.1.0 libpython3.6m.so.1.0
./libs/Linux/3_6_0 -> libpython3.6m.so.1.0 libpython3.6m.so.1.0
./libs/Linux/3_7_1 -> libpython3.7m.so.1.0
./libs/Linux/3_5_6 -> libpython3.5m.so.1.0

If you look at 3_6_9 line you'll see the mismatch.

zencode_exec_tobuf crash in swift

zencode_exec_tobuf don't working in swift - ios app. Crashed...

Tried with following code:
// let outputSize: size_t = 1024 * 128
// let errorSize: size_t = 1024 * 128

// let z_output = UnsafePointer(Int8(malloc(outputSize * MemoryLayout.size)))
// let z_output = UnsafeMutableRawPointer(malloc(outputSize * MemoryLayout.size)).assumingMemoryBound(to: Int8.self)
// let z_error = UnsafeMutableRawPointer(malloc(outputSize * MemoryLayout.size)).assumingMemoryBound(to: Int8.self)

// let z_output = stringToUnsafeMutablePointer(message: ""+malloc(outputSize * MemoryLayout.size))

// let z_output = UnsafeMutablePointer.allocate(capacity: outputSize * MemoryLayout.size)
// let z_error = UnsafeMutablePointer.allocate(capacity: errorSize * MemoryLayout.size)
//
// zencode_exec_tobuf(stringToUnsafeMutablePointer(message: text), stringToUnsafeMutablePointer(message: ""), stringToUnsafeMutablePointer(message: ""), stringToUnsafeMutablePointer(message: ""), 1, z_output, outputSize, z_error, errorSize)

zenroom_exec working but I want to get the JSON Response from the static Lib.
zenroom-ios.a and zenroom.h attached.
zenroom_a_h.zip

used function:
func stringToUnsafeMutablePointer(message: String) -> UnsafeMutablePointer {
var messageCString = message.utf8CString
return messageCString.withUnsafeMutableBytes { mesUMRBP in
return mesUMRBP.baseAddress!.bindMemory(to: Int8.self, capacity: mesUMRBP.count)
}

Investigate adoption of an improved memory allocator

Eventually make support optional

benchmark impact on WASM and Redroom applications

Candidates:

mimalloc https://github.com/microsoft/mimalloc
jemalloc
ptmalloc
rpmalloc https://github.com/mjansson/rpmalloc

Figuring out current release version

Hello dear zenroomers!

I'm trying to make sense of release tags.
The VERSION file states 1.0.0
Python package is 1.0.12
The latest tag in git is 1.0.5 while latest github release is 0.8.1
Javascript is 1.1.1

Are these "bindings" folders different packages or not? Or they all
refer to the same thing? I'm asking because there is no clear distinction.

And the last question is -- can I assume that I should be able to build
zenroom bindings from the latest commit on the master branch?

Zenroom CLI stdin flexibility

I propose zenroom executables could have command line switches for stdin input type.
Input type could be script (Zencode/Lua), data or key.

It could also be useful for zenroom executables to have CLI options to read in file(s) for each type, with any stdin type appended to files or inline parameter CLI code read.

Return Zencode in stderr upon succesful execution

As a confirmation of the code executed, the Zencode should be returned in stderr also upon success.

Host applications need to parse the return code and not stderr to detect errors.

This needs to be explicit in the documentation about embedding Zenroom.

Thanks @puria for the advice

Implement ZKP credential revokation

Use this: Accumulator Based on Bilinear Maps and Efficient Revocation for Anonymous Credentials

Also implemented in Indy's anoncreds and 0011 Credential Revokation using tails and accumulators.

gmp missing

Hey,

Running make static gites me the following error -

mark@mark-XPS-13-9350:~/Desktop/git/zenroom$ make static
if ! [ -r /home/mark/Desktop/git/zenroom/build/musl/obj/musl-gcc ]; then mkdir -p /home/mark/Desktop/git/zenroom/build/musl && cd /home/mark/Desktop/git/zenroom/build/musl && CFLAGS="-Os -fstack-protector -static" /home/mark/Desktop/git/zenroom/lib/musl/configure; fi
make -j2 -C /home/mark/Desktop/git/zenroom/build/musl
make[1]: Entering directory '/home/mark/Desktop/git/zenroom/build/musl'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/home/mark/Desktop/git/zenroom/build/musl'
./build/apply-patches
mkdir -p /home/mark/Desktop/git/zenroom/build/jemalloc
if ! [ -r /home/mark/Desktop/git/zenroom/lib/jemalloc/configure ]; then cd /home/mark/Desktop/git/zenroom/lib/jemalloc &&  /home/mark/Desktop/git/zenroom/lib/jemalloc/autogen.sh; fi
if ! [ -r /home/mark/Desktop/git/zenroom/build/jemalloc/lib/libjemalloc.a ]; then cd /home/mark/Desktop/git/zenroom/build/jemalloc && CFLAGS="-Os -fstack-protector -static" CC=/home/mark/Desktop/git/zenroom/build/musl/obj/musl-gcc /home/mark/Desktop/git/zenroom/lib/jemalloc/configure --disable-cxx && make -C /home/mark/Desktop/git/zenroom/build/jemalloc; fi
if ! [ -r /home/mark/Desktop/git/zenroom/build/lua_sandbox/CMakeCache.txt ]; then mkdir -p /home/mark/Desktop/git/zenroom/build/lua_sandbox && cd /home/mark/Desktop/git/zenroom/build/lua_sandbox && CC=/home/mark/Desktop/git/zenroom/build/musl/obj/musl-gcc cmake /home/mark/Desktop/git/zenroom/lib/lua_sandbox -DCMAKE_C_FLAGS="-Os -fstack-protector -static" ; fi
if ! [ -r /home/mark/Desktop/git/zenroom/build/lua_sandbox/src/libluasandbox.a ]; then make -C /home/mark/Desktop/git/zenroom/build/lua_sandbox luasandbox; fi
cd /home/mark/Desktop/git/zenroom/lib/gmp && CFLAGS="-Os -fstack-protector -static" CC=/home/mark/Desktop/git/zenroom/build/musl/obj/musl-gcc ./configure --disable-shared
/bin/sh: 1: cd: can't cd to /home/mark/Desktop/git/zenroom/lib/gmp
Makefile:44: recipe for target 'gmp' failed
make: *** [gmp] Error 2

The path "./lib/gmp" is not in the repo - do we just need to add a submodule to the repo?

I'm not too sure if gmp mirrors to git? https://gmplib.org/devel/repo-usage.html

Pluggable RNG callback for certain platform ports

It should be possible to provide a callback to a custom external RNG. It is required by some platforms who don't support the standard linux/osx/win sources already implemented, for instance the unikernel Cortex port, but also others ports to different language bindings.

Perhaps this will be also a good occasion to formalise better the provision of callbacks for the print to stdout/stderr, as for instance those plugged by preprocessor's #defines for javascript/wasm.

Zenroom reading "my name" from data only

The fix didn't fix;

8103e1d

implement the noise protocol

Having the statemachine extension and compatible crypto it is easy to implement the noise protocol, whenever needed or within fun-time. It should be used for networked communication as it avoids some pitfalls and normalizes according to widespread practices.

https://noiseprotocol.org/noise.html

The networking code of course should be up to the caller and not in Zenroom.

Update build documentation with static targets

A reminder to:

Hi @puria, I can do that, but is seems to section on "static builds" is out of date as it mentions targets such as bootstrap, static, and check-static which I think dont exist anymore?

Originally posted by @fdeantoni in #94 (comment)

Go issue

Zenroom version: latest master
OS: Linux (ubuntu 18.04)
Go version: 1.10

When building the go bindings from Zenroom master, I now get a segfault when trying to use those generated bindings with our CGO wrapper.

I did a git bisect to try and isolate the commit that seems to have caused this issue, and the offending commit that bisect reported was this one: 89e271e

To replicate:

make sure you have a working Go environment
clone down the zenroom-go repo: https://github.com/DECODEproject/zenroom-go into the correct place on your $GOPATH
make sure the zenroom repo is pulled down and updated to latest HEAD of master, and that all submodules have been updated
build the go bindings using make linux-go from within the zenroom folder
copy the generated build/go/libzenroomgo.so into the lib folder inside the cloned zenroom-go repository
from within the zenroom-go folder run: make test

you should see a nasty segfault error.

If instead of zenroom HEAD, you instead checkout the commit before the one listed above (47be7ee ), then rebuild the bindings and copy across to the zenroom-go project, the tests should pass again.

This error might be the result of me doing something wrong with the CGO calls that attempt to pass data into and read back from the Zenroom VM, but thought I'd flag as something to investigate.

Profile and trace execution using DynamoRIO

See: http://dynamorio.org/
In particular, interesting clients:

The memory debugging tool Dr. Memory
The code coverage tool drcov
The memory tracing tool memtrace
The instruction tracing tool instrace
The basic block tracing tool bbbuf
The instruction counting tool inscount
The dynamic fuzz testing tool Dr. Fuzz

distance measurement algos

Integrate distance measurement algorithms, like hamming and levenshtein, to measure the differences between different arrays, useful to measure entropy, etc.

Reference implementation: https://github.com/doukremt/distance

Also interesting work on shifted-hamming (SHD), but uses CPU specific instructions (SSE3)
https://github.com/CMU-SAFARI/Shifted-Hamming-Distance

implement bech32 encoding

Bech32 has several advantages over Base58 and is the future preferred format for bitcoin

it is specified in BIP0173 https://github.com/bitcoin/bips/blob/master/bip-0173.mediawiki

reference implementation by Sipa: https://github.com/sipa/bech32/tree/master/ref/c

p.s. tangentially decide what to do with base58 which is right now bugged.

Implement a layer of proovability via large integer whooping

As recommended by Bruce Schneider & co in cryptographic engineering, we can shadow all basic arithmetic operation with a 'whoop' proof. whoop whoop!
p.s. many thanks @olabini amazing book recommendation <3

Lift 4KiB limit on JSON data

Report so far, needs investigation

   2. data json, is right now limited to 4kb which doesn't allow for us to
   add multiple options (I think actually I can not get more than 3). We can
   be working with only two options yes/no, then we lose doing optional
   information

Broken link

On your homepage https://zenroom.org/, under the heading 'Licensing', subheading 'Community
Open Source', the 'Docs for Developers' button links to 'https://zenroom.org/dev.zenroom.org' which I believe should be https://dev.zenroom.org

Catching errors from Python

Operating System: Linux (Ubuntu 18.04)
Python version: 3.6.6
Zenroom bindings: Built from commit: d2601b3

Hey again, I'm trying to figure out how I can detect an error when invoking Zenroom via the Python binding. I'm calling Zenroom using the following code:

import ctypes
_zenroom = ctypes.CDLL('./_zenroom.so')

script = b'print("hello world")'

stdout_buf = ctypes.create_string_buffer(b'\000', 1024)
stdout_len = ctypes.c_size_t(1024)
stderr_buf = ctypes.create_string_buffer(b'\000', 1024)
stderr_len = ctypes.c_size_t(1024)

_zenroom.zenroom_exec_tobuf(
    script,
    None,
    None,
    None,
    1,
    ctypes.byref(stdout_buf),
    stdout_len,
    ctypes.byref(stderr_buf),
    stderr_len,
)

print(stdout_buf.value)

So the above works fine if Zenroom is passed a valid script, however if I change things to pass in a script with some error, e.g.:

# missing close quote
script = b'print("hello world)'

and try running the above script again, I get the following string printed to stdout or stderr:

python3: [string "print("hello world?)"]:1: unfinished string near <eof>: Unknown error 280084567

and the script seems to immediately exit, without allowing me to do anything, i.e. I never reach the line print(stdout_buf.value) shown above.

I tried the following things:

wrapping the call to zenroom_exec_tobuf inside a try/except block, but there is no exception raised that I can catch
capturing the return int from zenroom_exec_tobuf, but when an error happens the script seems to exit before this point

Am I being dense - is this expected behaviour? I think what I'd expect is either to have zenroom_exec_tobuf throw some exception that I can catch, or to have it return a non-zero exit code which I can detect, and perhaps then throw an error of my own containing the output from stderr.

Issue with credentials printed out in Url64

With the script:

`
Rule output encoding url64

Rule input encoding url64

Scenario credential: read and validate the credentials

Scenario petition: create the petition

Given that I am known as 'Alice'

Given I have my valid 'credential keypair'

Given I have my valid 'credentials'

Given I have a valid 'verifier' inside 'MadHatter'

When I aggregate the verifiers

When I create the credential proof

When I create the petition 'More privacy for all!'

Then print the 'uid' as 'string' inside 'petition'

Then print 'uid' as 'string'

Then print data

Keys:

{ "Alice": { "credential_keypair": { "public": "BE8ERC16Xsdg3PaGDwLYgEoagKWC1Eb-88aXg09qhvTv_onrHigAR5bbtFMcyWF9_xOeiY9Xvrt1lcdpWmwbKc_FThYOKj8ezwPlkZ5a75karo-d762W5EzuXw-76gI7_w", "private": "StbYmlCs7YlIm3UmSpBoEwgdFRFaJuB2j4H10p7wRYY" }, "credentials": { "h": "BDokjEscxc9dE3MpPVFTKJ-WD1VRx3rcvHOyAlsQifwoukL9hQI1y9qfALfcS1_DqiCCvSg15FFev8kSW1yC-NxTuxCM5urU-PV5BeLQEKJSROntjYZfcBbm4kru7XI7UQ", "s": "BCXBxSbzILQoWYCBIeMYxt2-MKdLEqDIF-SmmzyXpEx6AMMutSn3PhPxBYEMfixVuQJuTe93FNfbgYFVL5RGVEZsSoLGB53zw3pWmWWOgCkPQy3D4ikjoQv3TQv_UEqmnQ" } }, "MadHatter": { "verifier": { "alpha": "Ax9PWJpMePSumkm7L4n1bNaFMazOkGUNvkZe_kWXe0ZZdkkjQYd4vE6LEOW1QvVdNzb8QWFINrgaWp3e-9Qci4Ay7x0HdHxwwqGld-bLUNME1wVLBtWAAZSsAcAWAfQESYeLaDCAm9qjRV_ngYSo6L_EIRhoJExRhJfsoiA40UGksnmHdwJnih_v25SHTQ-EK7cIDyjAa6dhxUrOzNzKguR6lC2Wjf2HHpkYBFW6FEHmka-hB700X6dWA8lsrO37", "beta": "D0VgCzzAFJlVwbqJYNvKvoa18CWyoNBK0136o495PR91FItBdlrd81S9GG7_KxwOMsJKPJUeGOlnGH6aZ_OuIprHCWd5ij5vXhb0BDxGlN2g2FfO6c7bC_FVEPW0_8n1DJS00TwG_9a6No5-ZSrheQm24K-FeRBnJE6Hm8-Qd8wMWb6Ea-pMxc8dYecUeQA8PbESZgFBqJ2pWoBghhumDYlVBVAxA-UNeOb3-WcMIImSr93059aQUJADKzs2VywQ" } } }

When executing, the first two chars of the credential keypair and the credentials, come out wrong:

"credential_keypair": {
"private": "StbYmlCs7YlIm3UmSpBoEwgdFRFaJuB2j4H10p7wRYY",
"public": "A08ERC16Xsdg3PaGDwLYgEoagKWC1Eb-88aXg09qhvTv_onrHigAR5bbtFMcyWF9_w"
}

And

"credentials": {
"h": "AzokjEscxc9dE3MpPVFTKJ-WD1VRx3rcvHOyAlsQifwoukL9hQI1y9qfALfcS1_Dqg",
"s": "AyXBxSbzILQoWYCBIeMYxt2-MKdLEqDIF-SmmzyXpEx6AMMutSn3PhPxBYEMfixVuQ"
}

Deterministic sorting of JSON de/coding

Current C library cjson doesn't sort encoded JSON on output resulting in non-deterministic serialization of tables. This poses a series of annoying problems to cryptographic operations.

And also MSGPACK is not deterministic!

JSON needs to be implemented to sort input/output and perhaps a different serialization library should be used in place of MSGPACK.

Possibly this JSON implementation is a good start, but needs to have implemented Zenroom's pimitive crypto types

http://regex.info/code/JSON.lua

Zencode line to check match within a list of strings/hashes

Zencode should be present get the value of an arbitrary key in the incoming data and then return error if that key is not contained inside a list of valid values. This can also imply hashing of values to avoid privacy-sensitive storage of lists.

(From usecase of credential-api in decidim, formulated with @Andrea-Tabbles)

Different array exports of octets

Add more functions similar to the octet:array()method to export the contents of octets in different array formats:

array of bytes (8bit)
array of words (16bit)
array of double-words (32bits)
array of quad-words (64bits, check support in native LUA Number type)
array of double-quad-words (128bits, check support in native LUA Number type)

Also include option for exporting the array as a big-endian or little-endian sequence.

Some lua modules are not loading (nil errors everywhere).

Hello Zenroomers!

I'm not having luck with making it work :)

When I run lua scripts with zenroom most of the time interpreters says things like, you just tried to call nil.
I've compiled zenroom with make linux in clean environment and also tried to compile on debian machine.
In both cases I reproduced this behavior...

Here is the example running octet.lua test.

zap$ zenroom test/octet.lua
[*] Zenroom v1.0.0+1ad133d - secure crypto language VM
 .  Zenroom is Copyright (C) 2017-2020 by the Dyne.org foundation
 .  For the original source code and documentation go to https://zenroom.org
 .  Zenroom is free software: you can redistribute it and/or modify
 .  it under the terms of the GNU Affero General Public License as
 .  published by the Free Software Foundation, either version 3 of the
 .  License, or (at your option) any later version.
 .  Zenroom is distributed in the hope that it will be useful,
 .  but WITHOUT ANY WARRANTY; without even the implied warranty of
 .  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 .  GNU Affero General Public License for more details.
 .  You should have received a copy of the GNU Affero General Public License
 .  along with this program.  If not, see http://www.gnu.org/licenses/
 .  using default configuration
 .  System memory manager in use
 .  ECDH curve is SECP256K1
 .  Memory in use: 291 KB
[*] reading Zencode from file: /home/zap/src/zenroom/test/octet.lua
 .  loaded file (3903 bytes)
= OCTET FIRST CLASS CITIZEN TESTS

== test octet copy
== test string import/export
=== compare octets
=== compare strings
=== compare hashes
== test base64 import/export
== test url64 import/export
== test base58 import/export
== test hex import/export
== test bin import/export
== ECP import/export
== JSON import/export
[!] [string "zencode_data"]:297: attempt to index a nil value (field 'encoding')

Here is another one

zap$ zenroom test/coconut_abc_zeta.lua
[*] Zenroom v1.0.0+1ad133d - secure crypto language VM
 .  Zenroom is Copyright (C) 2017-2020 by the Dyne.org foundation
 .  For the original source code and documentation go to https://zenroom.org
 .  Zenroom is free software: you can redistribute it and/or modify
 .  it under the terms of the GNU Affero General Public License as
 .  published by the Free Software Foundation, either version 3 of the
 .  License, or (at your option) any later version.
 .  Zenroom is distributed in the hope that it will be useful,
 .  but WITHOUT ANY WARRANTY; without even the implied warranty of
 .  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 .  GNU Affero General Public License for more details.
 .  You should have received a copy of the GNU Affero General Public License
 .  along with this program.  If not, see http://www.gnu.org/licenses/
 .  using default configuration
 .  System memory manager in use
 .  ECDH curve is SECP256K1
 .  Memory in use: 291 KB
[*] reading Zencode from file: /home/zap/src/zenroom/test/coconut_abc_zeta.lua
 .  loaded file (1962 bytes)

= ATTRIBUTE BASED CRYPTOGRAPHY TEST (ZKP + zeta/UID)

[!] [string "print()..."]:16: attempt to index a nil value (global 'ABC')

The same thing happens when I compile zenrroom as library and call zenroom_exec from python using using ctypes.
For example if I try to call zenroom_exec("print( JSON.encode(map(ECDH.keygen(), hex)) )") I get following:

Python 3.8.2 (default, Jan  1 1970, 00:00:01) 
[GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> python.el: native completion setup loaded
>>>  .  System memory manager in use
 .  ECDH curve is SECP256K1
 .  Memory in use: 309 KB
/home/zap/src/zenroom/bindings/python3/venv/bin/python3: [string "Scenario simple: Encrypt a message with the p..."]:1: syntax error near 'simple': Unknown error 2080776751

Process Python exited abnormally with code 136
Python 3.8.2 (default, Jan  1 1970, 00:00:01) 
[GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> python.el: native completion setup loaded
>>>  .  System memory manager in use
 .  ECDH curve is SECP256K1
 .  Memory in use: 309 KB
/home/zap/src/zenroom/bindings/python3/venv/bin/python3: [string "print( JSON.encode(map(ECDH.keygen(), hex)) )..."]:1: attempt to call a nil value (global 'map'): Unknown error 401211951

Process Python exited abnormally with code 184

Note: I'm not using python bindings provided by zenroom here.

I've tried to debug this with gdb but I have little understanding of how project is structured and I getting lost in the call stack...

It would be great If you could guide me in this situation or at least tell whether you can reproduce it on commit "1ad133deabcc98b99335bb199eb5ff5529a546da".

Zenroom's npm package is published with incompatible version of core-js

The package.json in the javascript bindings of GitHub mentions as dependencies:

  "dependencies": {
    "@babel/node": "^7.6.3",
    "core-js": "^3.1.4",
    "regenerator-runtime": "*"
  },

the published package on npm has:

  "dependencies": {
    "core-js": "^3.6.5"
  }

when doing npm install Zenroom and running it with the 3.6.5 version of core-js from shadow-cljs (a cljs build tool) I see this error:

The required JS dependency "path" is not available, it was required by "node_modules/zenroom/dist/lib/zenroom.js".

Searched in: (...)/node_modules

You probably need to run:
  npm install path

when running npm install [email protected] regenerator-runtime (the dependencies mentioned on GitHub) this error disappears and Zenroom works.

So the working dependencies seem to be the ones mentioned on GitHub, and not those actually published with the package.

post-quantum crypto functions

Drafting a plan to include code that provides post-quantum strength for basic functionalities as diffie-hellman sessions and signatures. Interesting so far:

Picnic for signatures https://github.com/Microsoft/Picnic/blob/master/picnic.h
Supersingular Isogeny Key Encapsulation Library (SIKE) for DH style de/coding https://github.com/Microsoft/PQCrypto-SIKE/blob/master/Optimized_Implementation/portable/SIKEp503/api.h

Also interesting Kelvin chain use of liboqs (includes also picnic and supersingular DH)

https://github.com/kelvinblockchain/libdap/tree/master/crypto

Fix SHA3 hashing to pass NIST vector tests

See test scripts here https://github.com/DECODEproject/Zenroom/tree/master/test/nist

Include a native Zenroom parser for RST vector files provided by NIST

A correct SHA3* implementation should then pass all tests launched in run.sh

Build documentation question

I checked the Build Instructions and tried to build the shared libs using make shared but that target does not seem to exist. The Makefile has other targets that do seem to work (.e.g. make osx) but these are not described in the above documentation though. Is there somewhere else I should look to create the static and/or shared libraries?

Release javascript bindings asap

as per dyne/restroom-mw#10 new zenroom features are requested in dependencies projects.

Friendly reminder to release ;)

Reduce incoming DATA to one single structure

Separation of incoming DATA and KEYS has generated quite some problems for developers implementing and confusing the arguments. We will remove it in favor of a single channel, so the host application will be responsible of flattening all incoming DATA into a single structure. This will be an API breaking change.

(evinced from the piloting of Zenroom implementations)

Implement BLS signature and SSS in Zenroom Lua

Reference implementation in milagro-c
apache/incubator-milagro-crypto-c#47

For readability and maintenance the new BLS implementation can be made in Lua script with a very small computational overhead in Zenroom.
Implementaiton: src/bls192.c.in
Example usage: examples/example_bls_sss_ZZZ.c.in

Fix chunk size mismatch in ARM builds

Bug occurring when running elgamal's keygen - to be investigated

   decode@decode03:~/zenroom/examples/elgamal$ ../../../zenroom-static
   keygen.lua
   [*] Zenroom v0.7.1 - crypto language restricted VM
    .  Copyright (C) 2017-2018 Dyne.org foundation
   [*] reading Zencode from file: keygen.lua
    .  loaded file (233 bytes)
    .  using default configuration
    .  loading lua initialisation
   [!] zen_load_string syntax error: init
   [!] init: size_t size mismatch in precompiled chunk
   Aborted

Documentation for Java

Check the doc are updated with the new Java bindings https://dev.zenroom.org/#/pages/java and adding the java targets to the https://dev.zenroom.org/#/pages/how-to-build

Verify stbsp_sprintf works well on armeabi-v7a (32bit)

This bug may or may not affect Zenroom (unverified)
nothings/stb#871

If problems arise on named platforms, investigate further.

Assess a pairing-friendly and safe curve implemented in Milagro for

To complete the ECP2 twisted curve and pairing operations we need a pairing-friendly curve, but none of those provided by Milagro seems to be "safe" according to https://safecurves.cr.yp.to/ (which may or may not be updated enough ATM)

So for now in Zenroom ECP we adopt a provisional use of pairing-friendly curve BLS383
which is however not to be confused with M-383.
According to https://tools.ietf.org/html/draft-ladd-safecurves-04

M383 is a curve over GF(2^383-187), formula y^2=x^3+2065150x^2+x,
	basepoint (12,
	           473762340189175399766054630037590257683961716725770372563038
	           9791524463565757299203154901655432096558642117242906494), order 2^380
	+ 166236275931373516105219794935542153308039234455761613271, cofactor
	8.

and https://eprint.iacr.org/2013/647 confirms this order.

Comparing orders extracted in ecp.order():hex() using zenroom:

BLS383 00000000000000000000000000000001002001800c00b809c04401c81698b381de05f095a120d3973b2099ebfebc0001
M-383  10000000000000000000000000000000000000000000000006C79673AC36BA6E7A32576F7B1B249E46BBC225BE9071D7

Passing arguments to a script

I would like to execute a lua script with one or many arguments using zenroom. Is this something that already exists?

If it doesn't exist I can see some options for implementing it -

Via a command line

Would the following command line construction make sense?

./zenroom-shared -c ${pwd}/test/decode-test.conf my-script.lua "foo" 22

{path to zenroom} -c {path to config file} {lua script} {arg-1} {arg-2} {arg-n...}

In this example the args supplied to the function are "foo" (a string) and 22 (a number - I don't know the lua number type I'm afraid)

Via some RPC-like mechanism

Alternatively would starting up an instance of zenroom and then pass a series of arguments via some
other mechanism (e.g. a socket, a file descriptor or a port exposed on the local machine) be valuable?

{path to zenroom} -c {path to config file} -p {port number to bind to} {lua script}

In this example I could imaging zenroom binding to a local port and waiting for input.
The data could then be sent to the zenroom instance and executed in strict order either completing or timing out.
The data could be either json of a known format or a line encoded format.

Something else??

I think there are tradeoffs with these ideas however I would prefer something like the second method as I would like to call the same script multiple times with different arguments but the first option would suffice for my needs at the moment.

Any thoughts?

Are there implementations of Zenroom for Microcontrollers

Hi.
I'm developing a device based on a SAMD21 Microcontroller.

I'm wondering whether any implementations of Zenroom were done to make it executable from a 32 bit hardware.

thanks in advance for the reply

Multiformats based metadata semantics for objects

The 1st citizen type (octet) should also carry a brief metadata byte corresponding to multiformats specification to provide semantic information on its contents.

[Android] zenroom crashes sometimes

2020-05-05 09:49:54.770 7414-7414/org.dhid.wallet A/libc: Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x1001d in tid 7414 (org.dhid.wallet), pid 7414 (org.dhid.wallet)
2020-05-05 09:49:54.819 7470-7470/? A/DEBUG: *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
2020-05-05 09:49:54.819 7470-7470/? A/DEBUG: Build fingerprint: 'google/sdk_gphone_x86_arm/generic_x86_arm:9/PSR1.180720.117/5875966:userdebug/dev-keys'
2020-05-05 09:49:54.819 7470-7470/? A/DEBUG: Revision: '0'
2020-05-05 09:49:54.819 7470-7470/? A/DEBUG: ABI: 'x86'
2020-05-05 09:49:54.819 7470-7470/? A/DEBUG: pid: 7414, tid: 7414, name: org.dhid.wallet  >>> org.dhid.wallet <<<
2020-05-05 09:49:54.819 7470-7470/? A/DEBUG: signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x1001d
2020-05-05 09:49:54.819 7470-7470/? A/DEBUG:     eax 20202e20  ebx 0001001d  ecx 0001021d  edx 00000200
2020-05-05 09:49:54.819 7470-7470/? A/DEBUG:     edi d3df9870  esi 1f1f2d1f
2020-05-05 09:49:54.819 7470-7470/? A/DEBUG:     ebp 0001001d  esp ff9d0500  eip d3d34db1
2020-05-05 09:49:54.819 7470-7470/? A/DEBUG: backtrace:
2020-05-05 09:49:54.819 7470-7470/? A/DEBUG:     #00 pc 0000edb1  /data/app/org.dhid.wallet-SeKliEKYNQIW_7U7dwdr9w==/lib/x86/libzenroom.so (z_vsprintfcb+465)

Support biometric-based generation of keys

Derive a private key from fingerprint biometric data, using a pair-minutiae feature extraction algorithm and adding a device based key for revocable transformation.

Reference paper: " A fingerprint based crypto-biometric system for secure communication" (https://arxiv.org/abs/1805.08399) has good explanation on how to make revokable keys that are usable in DH.

Reference implementations (incomplete):