clearmatics / libsnark

C++ library for zkSNARKs (forked from scipr-lab/libsnark)

libsnark

C++ library for zkSNARK proofs

(forked from https://github.com/scipr-lab/libff)

libsnark is a C++ library for zkSNARK proofs. The library was originally developed by SCIPR Lab and contributors (see AUTHORS file) and is released under the MIT License (see LICENSE file).

Overview

This library implements zkSNARK schemes, which are a cryptographic method for proving/verifying, in zero knowledge, the integrity of computations.

A computation can be expressed as an NP statement, in forms such as the following:

• "The C program foo, when executed, returns exit code 0 if given the input bar and some additional input qux."
• "The Boolean circuit foo is satisfiable by some input qux."
• "The arithmetic circuit foo accepts the partial assignment bar, when extended into some full assignment qux."
• "The set of constraints foo is satisfiable by the partial assignment bar, when extended into some full assignment qux."

A prover who knows the witness for the NP statement (i.e., a satisfying input/assignment) can produce a short proof attesting to the truth of the NP statement. This proof can be verified by anyone, and offers the following properties.

• Zero knowledge: the verifier learns nothing from the proof beside the truth of the statement (i.e., the value qux, in the above examples, remains secret).
• Succinctness: the proof is short and easy to verify.
• Non-interactivity: the proof is a string (i.e. it does not require back-and-forth interaction between the prover and the verifier).
• Soundness: the proof is computationally sound (i.e., it is infeasible to fake a proof of a false NP statement). Such a proof system is also called an argument.
• Proof of knowledge: the proof attests not just that the NP statement is true, but also that the prover knows why (e.g., knows a valid qux).

These properties are summarized by the zkSNARK acronym, which stands for Zero-Knowledge Succinct Non-interactive ARgument of Knowledge (though zkSNARKs are also knows as succinct non-interactive computationally-sound zero-knowledge proofs of knowledge). For formal definitions and theoretical discussions about these, see [BCCT12], [BCIOP13], and the references therein.

The libsnark library currently provides a C++ implementation of:

1. General-purpose proof systems:

   1. A preprocessing zkSNARK for the NP-complete language "R1CS" (Rank-1 Constraint Systems), which is a language that is similar to arithmetic circuit satisfiability.

      This zkSNARK construction follows, extends, and optimizes the approach described in [BCTV14a], itself an extension of [BCGTV13], following the approach of [GGPR13] and [BCIOP13]. (An alternative implementation of this approach is the Pinocchio system of [PGHR13].)

   2. A preprocessing SNARK for a language of arithmetic circuits, "BACS" (Bilinear Arithmetic Circuit Satisfiability). This simplifies the writing of NP statements when the additional flexibility of R1CS is not needed. Internally, it reduces to R1CS.

   3. A preprocessing SNARK for the language "USCS" (Unitary-Square Constraint Systems). This abstracts and implements the core contribution of [DFGK14]

   4. A preprocessing SNARK for a language of Boolean circuits, "TBCS" (Two-input Boolean Circuit Satisfiability). Internally, it reduces to USCS. This is much more efficient than going through R1CS.

   5. A simulation-extractable preprocessing SNARK for R1CS. This construction uses the approach described in [GM17]. For arithmetic circuits, it is slower than the [BCTV14a] approach, but produces shorter proofs.

   6. ADSNARK, a preprocessing SNARKs for proving statements on authenticated data, as described in [BBFR15].

   7. Proof-Carrying Data (PCD). This uses recursive composition of SNARKs, as explained in [BCCT13] and optimized in [BCTV14b].

2. Gadget libraries (gadgetlib1 and gadgetlib2) for constructing R1CS instances out of modular "gadget" classes.
3. Examples of applications that use the above proof systems to prove statements about:
   1. Several toy examples.
   2. Execution of TinyRAM machine code, as explained in [BCTV14a] and [BCGTV13]. (Such machine code can be obtained, e.g., by compiling from C.) This is easily adapted to any other Random Access Machine that satisfies a simple load-store interface.
   3. A scalable for TinyRAM using Proof-Carrying Data, as explained in [BCTV14b]
   4. Zero-knowldge cluster MapReduce, as explained in [CTV15].

See the above references for discussions of efficiency aspects that arise in practical use of such constructions, as well as security and trust considerations.

This scheme is a preprocessing zkSNARK (ppzkSNARK): before proofs can be created and verified, one needs to first decide on a size/circuit/system representing the NP statements to be proved, and run a generator algorithm to create corresponding public parameters (a long proving key and a short verification key).

Using the library involves the following high-level steps:

1. Express the statements to be proved as an R1CS (or any of the other languages above, such as arithmetic circuits, Boolean circuits, or TinyRAM). This is done by writing C++ code that constructs an R1CS, and linking this code together with libsnark
2. Use libsnark's generator algorithm to create the public parameters for this statement (once and for all).
3. Use libsnark's prover algorithm to create proofs of true statements about the satisfiability of the R1CS.
4. Use libsnark's verifier algorithm to check proofs for alleged statements.

The NP-complete language R1CS

The ppzkSNARK supports proving/verifying membership in a specific NP-complete language: R1CS (rank-1 constraint systems). An instance of the language is specified by a set of equations over a prime field F, and each equation looks like: < A, (1,X) > * < B , (1,X) > = < C, (1,X) > where A,B,C are vectors over F, and X is a vector of variables.

In particular, arithmetic (as well as boolean) circuits are easily reducible to this language by converting each gate into a rank-1 constraint. See [BCGTV13] Appendix E (and "System of Rank 1 Quadratic Equations") for more details about this.

Elliptic curve choices

The ppzkSNARK can be instantiated with different parameter choices, depending on which elliptic curve is used (see curves supported in libff)

Gadget libraries

The libsnark library currently provides two libraries for conveniently constructing R1CS instances out of reusable "gadgets". Both libraries provide a way to construct gadgets on other gadgets as well as additional explicit equations. In this way, complex R1CS instances can be built bottom up.

gadgetlib1

This is a low-level library which expose all features of the preprocessing zkSNARK for R1CS. Its design is based on templates (as does the ppzkSNARK code) to efficiently support working on multiple elliptic curves simultaneously. This library is used for most of the constraint-building in libsnark, both internal (reductions and Proof-Carrying Data) and examples applications.

gadgetlib2

This is an alternative library for constructing systems of polynomial equations and, in particular, also R1CS instances. It is better documented and easier to use than gadgetlib1, and its interface does not use templates. However, fewer useful gadgets are provided.

Security

The theoretical security of the underlying mathematical constructions, and the requisite assumptions, are analyzed in detailed in the aforementioned research papers.

This code is a research-quality proof of concept, and has not yet undergone extensive review or testing. It is thus not suitable, as is, for use in critical or production systems.

Known issues include the following:

• The ppzkSNARK's generator and prover exhibit data-dependent running times and memory usage. These form timing and cache-contention side channels, which may be an issue in some applications.

• Randomness is retrieved from /dev/urandom, but this should be changed to a carefully considered (depending on system and threat model) external, high-quality randomness source when creating long-term proving/verification keys.

Build instructions

Dependencies

The libsnark library relies on the following:

• C++ build environment
• CMake build infrastructure
• GMP for certain bit-integer arithmetic
• libprocps for reporting memory usage
• Fetched and compiled via Git submodules:
  • libff for finite fields and elliptic curves
  • libfqfft for fast polynomial evaluation and interpolation in various finite domains
  • ate-pairing for the BN128 elliptic curve
  • xbyak just-in-time assembler, for the BN128 elliptic curve
  • Subset of SUPERCOP for crypto primitives needed by ADSNARK

Furthermore, Doxygen is used to generate the documentation.

So far we have tested these only on Linux, though we have been able to make the libsnark work, with some features disabled (such as memory profiling or GTest tests), on Windows via Cygwin and on Mac OS X. See also the notes on portability below. (If you port libsnark to additional platforms, please let us know!)

Concretely, here are the requisite packages in some Linux distributions:

• On Ubuntu 20.04 LTS:

sudo apt install \
    build-essential \
    git \
    libboost-all-dev \
    cmake \
    libgmp3-dev \
    libssl-dev \
    libprocps-dev \
    pkg-config

• On Ubuntu 16.04 LTS:

sudo apt-get install \
    build-essential \
    cmake \
    git \
    libgmp3-dev \
    libprocps4-dev \
    libboost-all-dev \
    libssl-dev

• On Ubuntu 14.04 LTS:

sudo apt-get install \
    build-essential \
    cmake \
    git \
    libgmp3-dev \
    libprocps3-dev \
    libboost-all-dev \
    libssl-dev

• On Fedora 20 through 23:

sudo yum install \
    gcc-c++ \
    cmake \
    make \
    git \
    gmp-devel \
    procps-ng-devel

Building

Fetch dependencies from their GitHub repos:

git submodule update --init --recursive

Create the Makefile:

mkdir build && cd build && cmake ..

Then, to compile the library, tests, and profiling harness, run this within the build directory:

make

To create the Doxygen documentation, run:

# Make sure to install doxygen if not already
# e.g. on Ubuntu run: apt-get install doxygen graphviz
cd build && cmake .. -DGEN_DOC=ON && make docs

To compile and run the tests for this library, run:

make check

Using libsnark as a library

To develop an application that uses libsnark, it's recommended to use your own build system that incorporates libsnark and dependencies. If you're using CMake, add libsnark as a git submodule, and then add it as a subdirectory. Then, add snark as a library dependency to the appropriate rules.

To build and install the libsnark library:

DESTDIR=/install/path make install

This will install libsnark.a into /install/path/lib; so your application should be linked using -L/install/path/lib -lsnark. It also installs the requisite headers into /install/path/include; so your application should be compiled using -I/install/path/include.

In addition, unless you use WITH_SUPERCOP=OFF, libsnark_adsnark.a will be installed and should be linked in using -lsnark_adsnark.

When you use compile your application against libsnark, you must have the same conditional defines (#define FOO or g++ -DFOO) as when you compiled libsnark, due to the use of templates. One way to figure out the correct conditional defines is to look at build/libsnark/CMakeFiles/snark.dir/flags.make after running cmake. (Issue #21)

Building on Windows using Cygwin

Install Cygwin using the graphical installer, including the g++, libgmp, cmake, and git packages. Then disable the dependencies not easily supported under CygWin, using:

cmake -DWITH_PROCPS=OFF ..

Building on Mac OS X

On Mac OS X, install GMP from MacPorts (port install gmp). Then disable the dependencies not easily supported under OS X, using:

cmake -DWITH_PROCPS=OFF ..

MacPorts does not write its libraries into standard system folders, so you might need to explicitly provide the paths to the header files and libraries by appending CXXFLAGS=-I/opt/local/include LDFLAGS=-L/opt/local/lib to the line above.

Build options

Check CMakelists.txt for the whole list of build options. Some details are provided below.

• cmake -DCURVE=choice (where choice is one of: ALT_BN128, BN128, EDWARDS, MNT4, MNT6): Set the default curve to one of the above (see elliptic curve choices).

• cmake -DLOWMEM=ON: Limit the size of multi-exponentiation tables, for low-memory platforms.

• cmake -DWITH_PROCPS=OFF: Do not link against libprocps. This disables memory profiling.

• cmake -DWITH_SUPERCOP=OFF: Do not link against SUPERCOP for optimized crypto. The ADSNARK executables will not be built.

• cmake -DMULTICORE=ON: Enable parallelized execution of the ppzkSNARK generator and prover, using OpenMP. This will utilize all cores on the CPU for heavyweight parallelizable operations such as FFT and multiexponentiation. The default is single-core. To override the maximum number of cores used, set the environment variable OMP_NUM_THREADS at runtime (not compile time), e.g., OMP_NUM_THREADS=8 test_r1cs_sp_ppzkpc. It defaults to the autodetected number of cores, but on some devices, dynamic core management confused OpenMP's autodetection, so setting OMP_NUM_THREADS is necessary for full utilization.

• cmake -DUSE_PT_COMPRESSION=OFF: Do not use point compression. This gives much faster serialization times, at the expense of ~2x larger sizes for serialized keys and proofs.

• cmake -DMONTGOMERY_OUTPUT=ON: Serialize Fp elements as their Montgomery representations. If this option is disabled then Fp elements are serialized as their equivalence classes, which is slower but produces human-readable output.

• cmake -DBINARY_OUTPUT=ON: In serialization, output raw binary data (instead of decimal), which is smaller and faster.

• cmake -DPROFILE_OP_COUNTS=ON: Collect counts for field and curve operations inside static variables of the corresponding algebraic objects. This option works for all curves except bn128.

• cmake -DUSE_ASM=ON: Use architecture-specific assembly routines for F[p] arithmetic and heap in multi-exponentiation. (If disabled, use GMP's mpn_* routines instead.)

• cmake -DPERFORMANCE=ON: Enables compiler optimizations such as link-time optimization, and disables debugging aids. (On some distributions this causes a plugin needed to handle lto object link error and undefined references, which can be remedied by AR=gcc-ar make ....)

• cmake -DOPT_FLAGS=...: Set the C++ compiler optimization flags, overriding the default (e.g., -DOPT_FLAGS="-Os -march=i386").

• cmake -DDEPENDS_DIR=...: Sets the dependency installation directory to the provided absolute path (default: installs dependencies in the respective submodule directories)

• cmake -DUSE_LINKED_LIBRARIES=ON: Setting this flag enables CMake to include installed libfqfft and libff libraries. This will tell the compiler to ignore the libfqfft and libff dependencies provided in the depends folder.

Note: Not all combinations are tested together or supported by every part of the codebase.

Tutorials

libsnark includes a tutorial, and some usage examples, for the high-level API.

• libsnark/gadgetlib1/examples1 contains a simple example for constructing a constraint system using gadgetlib1.

• libsnark/gadgetlib2/examples contains a tutorial for using gadgetlib2 to express NP statements as constraint systems. It introduces basic terminology, design overview, and recommended programming style. It also shows how to invoke ppzkSNARKs on such constraint systems. The main file, tutorial.cpp, builds into a standalone executable.

• libsnark/zk_proof_systems/ppzksnark/r1cs_ppzksnark/profiling/profile_r1cs_ppzksnark.cpp constructs a simple constraint system and runs the ppzksnark. See below for how to run it.

Executing profiling example

The command

libsnark/zk_proof_systems/ppzksnark/r1cs_ppzksnark/profiling/profile_r1cs_ppzksnark 1000 10 Fr

exercises the ppzkSNARK (first generator, then prover, then verifier) on an R1CS instance with 1000 equations and an input consisting of 10 field elements.

(If you get the error zmInit ERR:can't protect, see the discussion above.)

The command

libsnark/zk_proof_systems/ppzksnark/r1cs_ppzksnark/profiling/profile_r1cs_ppzksnark 1000 10 bytes

does the same but now the input consists of 10 bytes.

Portability

libsnark is written in fairly standard C++11.

However, having been developed on Linux on x86-64 CPUs, libsnark has some limitations with respect to portability. Specifically:

1. libsnark's algebraic data structures assume little-endian byte order.

2. Profiling routines use clock_gettime and readproc calls, which are Linux-specific.

3. Random-number generation is done by reading from /dev/urandom, which is specific to Unix-like systems.

4. libsnark binary serialization routines (see BINARY_OUTPUT above) assume a fixed machine word size (i.e. sizeof(mp_limb_t) for GMP's limb data type). Objects serialized in binary on a 64-bit system cannot be de-serialized on a 32-bit system, and vice versa. (The decimal serialization routines have no such limitation.)

5. libsnark requires a C++ compiler with good C++11 support. It has been tested with g++ 4.7 and newer, and clang 3.4 and newer.

6. On x86-64, we by default use highly optimized assembly implementations for some operations (see USE_ASM above). On other architectures we fall back to a portable C++ implementation, which is slower.

7. The ate-pairing library, require by the BN128 curve, can be compiled only on i686 and x86-64. (On other platforms, use other -DCURVE=... choices.)

8. The SUPERCOP library, required by ADSNARK, can be compiled only on i686 and x86-64. (On other platforms, use -DWITH_SUPERCOP=OFF.)

Tested configurations include:

• Debian jessie with g++ 4.7 on x86-64
• Debian jessie with clang 3.4 on x86-64
• Fedora 20/21 with g++ 4.8.2/4.9.2 on x86-64
• Fedora 21 with g++ 4.9.2 on x86-32, for non-BN128 curves (-DWITH_SUPERCOP=OFF)
• Ubuntu 14.04 LTS with g++ 4.8 on x86-64
• Ubuntu 14.04 LTS with g++ 4.8 on x86-32, for non-BN128 curves (-DWITH_SUPERCOP=OFF)
• Ubuntu 15.04/16.04 LTS with g++ 4.9.2/5.3.1 on ARM AArch32/AArch64, for non-BN128 curve choices
• Debian wheezy with g++ 4.7 on ARM little endian (Debian armel port) inside QEMU, for EDWARDS and ALT_BN128 curve choices
• Windows 7 with g++ 4.8.3 under Cygwin 1.7.30 on x86-64 for EDWARDS and ALT_BN128 curve choices (-DWITH_PROCPS=OFF and GTestdisabled)
• Mac OS X 10.9.4 (Mavericks) with Apple LLVM version 5.1 (based on LLVM 3.4svn) on x86-64 (-DWITH_PROCPS=OFF and GTest disabled)

Directory structure

The directory structure of the libsnark library is as follows:

• libsnark: main C++ source code, containing the following modules:
  • common: miscellaneous utilities
  • gadgetlib1: gadgetlib1, a library to construct R1CS instances
    • gadgets: basic gadgets for gadgetlib1
  • gadgetlib2: gadgetlib2, a library to construct R1CS instances
  • relations: interfaces for expressing statement (relations between instances and witnesses) as various NP-complete languages
    • constraint_satisfaction_problems: R1CS and USCS languages
    • circuit_satisfaction_problems: Boolean and arithmetic circuit satisfiability languages
    • ram_computations: RAM computation languages
  • zk_proof_systems: interfaces and implementations of the proof systems
  • reductions: reductions between languages (used internally, but contains many examples of building constraints)
• depends: external dependencies which are automatically fetched and compiled (overridable by cmake -DDEPENDS_DIR=...)

Some of these module directories have the following subdirectories:

• ...
  • examples: example code and tutorials for this module
  • tests: unit tests for this module

In particular, the top-level API examples are at libsnark/r1cs_ppzksnark/examples/ and libsnark/gadgetlib2/examples/.

Further considerations

Multiexponentiation window size

The ppzkSNARK's generator has to solve a fixed-base multi-exponentiation problem. We use a window-based method in which the optimal window size depends on the size of the multiexponentiation instance and the platform.

On our benchmarking platform (a 3.40 GHz Intel Core i7-4770 CPU), we have computed for each curve optimal windows, provided as fixed_base_exp_window_table initialization sequences, for each curve; see X_init.cpp for X=edwards,bn128,alt_bn128.

Performance on other platforms may not be optimal (but probably not be far off). Future releases of the libsnark library will include a tool that generates optimal window sizes.

References

[BBFR15] ADSNARK: nearly practical and privacy-preserving proofs on authenticated data , Michael Backes, Manuel Barbosa, Dario Fiore, Raphael M. Reischuk, IEEE Symposium on Security and Privacy (Oakland) 2015

[BCCT12] From extractable collision resistance to succinct non-Interactive arguments of knowledge, and back again , Nir Bitansky, Ran Canetti, Alessandro Chiesa, Eran Tromer, Innovations in Computer Science (ITCS) 2012

[BCCT13] Recursive composition and bootstrapping for SNARKs and proof-carrying data Nir Bitansky, Ran Canetti, Alessandro Chiesa, Eran Tromer, Symposium on Theory of Computing (STOC) 13

[BCGTV13] SNARKs for C: Verifying Program Executions Succinctly and in Zero Knowledge , Eli Ben-Sasson, Alessandro Chiesa, Daniel Genkin, Eran Tromer, Madars Virza, CRYPTO 2013

[BCIOP13] Succinct non-interactive arguments via linear interactive Proofs , Nir Bitansky, Alessandro Chiesa, Yuval Ishai, Rafail Ostrovsky, Omer Paneth, Theory of Cryptography Conference 2013

[BCTV14a] Succinct non-interactive zero knowledge for a von Neumann architecture , Eli Ben-Sasson, Alessandro Chiesa, Eran Tromer, Madars Virza, USENIX Security 2014

[BCTV14b] Scalable succinct non-interactive arguments via cycles of elliptic curves , Eli Ben-Sasson, Alessandro Chiesa, Eran Tromer, Madars Virza, CRYPTO 2014

[CTV15] Cluster computing in zero knowledge , Alessandro Chiesa, Eran Tromer, Madars Virza, Eurocrypt 2015

[DFGK14] Square span programs with applications to succinct NIZK arguments , George Danezis, Cedric Fournet, Jens Groth, Markulf Kohlweiss, ASIACCS 2014

[GM17] Snarky Signatures: Minimal Signatures of Knowledge from Simulation-Extractable SNARKs , Jens Groth and Mary Maller, IACR-CRYPTO-2017

[GGPR13] Quadratic span programs and succinct NIZKs without PCPs , Rosario Gennaro, Craig Gentry, Bryan Parno, Mariana Raykova, EUROCRYPT 2013

[ate-pairing] High-Speed Software Implementation of the Optimal Ate Pairing over Barreto-Naehrig Curves , MITSUNARI Shigeo, TERUYA Tadanori

[PGHR13] Pinocchio: Nearly Practical Verifiable Computation , Bryan Parno, Craig Gentry, Jon Howell, Mariana Raykova, IEEE Symposium on Security and Privacy (Oakland) 2013

libsnark's Issues

Plonk: update inputs to plonk_srs_derive_from_usrs(usrs, matrix, perm, pi_indices)

• Modify plonk_srs_derive_from_usrs to accept the following inputs

• usrs
• gates_matrix
• wire_permutation
• public_input_indices

• (optional) Remove the circuit_t structure

Plonk: helper function plonk_prepare_matrix to build gates matrix

Implement helper function plonk_prepare_matrix to:

1. Prepare a gates matrix that has the top $L$ rows corresponding to $L$ Plonk public input gates (for a circuit with $L$ public inputs)

The caller will then manually:

1. Fill in the remaining rows from $L+1$ to $N$ (for a circuit with $N$ gates) of the gates matrix with the remaining Plonk gates describing the circuit (add, multiply, multiply-by-constant, dummy)

2. Generate the wire permutation (copy constraints) including the public inputs

3. Pass the constructed gates matrix, together with the wire permutation and public input indices as an input to function plonk_srs_derive_from_usrs (see Issue #88)

Rename `fp6_gadgets` to explicit the field tower used

See: https://github.com/clearmatics/libsnark/blob/master/libsnark/gadgetlib1/gadgets/fields/fp6_gadgets.hpp

Rename these gadgets to explicit the fact that Fp6 is built as "2 over 3" for MNT6.
E.g. Fp6_variable -> Fp6_2over3_variable etc.

Remove redundancy in filenames

Currently, and across most our projects , we have lots of redundancy in our file names. In may cases, we have things like:
gadgetlib1//pairing/mnt/mnt_pairing_params.hpp. The mnt emphasized here: mnt/**mnt**_pairing_params.hpp is not necessary. We could simply have things like gadgetlib1//pairing/mnt/pairing_params.hpp, gadgetlib1//pairing/bw6_761_bls12_377/pairing_params.hpp etc etc. In that sense, we can talk about the pairing_params.hpp file from the given pairing group folder.
The example above highlight some of the redundancy we have in file names across the different code bases:

• Zeth
• Zecale
• Libsnark/ff/fqfft

To action on this ticket, we need to potentially restructure the code, then rename the files, fix the #include directives and modify the #ifndef/define directives (with associated comment at the end of the files)

Plonk: move print_vector to libff

(bullet 5): Move the print_vector function https://github.com/clearmatics/libsnark/blob/plonk/libsnark/zk_proof_systems/plonk/utils.tcc#L19-L25 to libff/.../field_utils.hpp. (Nice print functions in libff clearmatics/libff#70)

Plonk: do not store the Lagrange basis L_basis explicitly and keep domain once

(bullet 2) Do not store the Lagrange basis L_basis explicitly (e.g. as part of the srs class and circuit_t struct). Instead compute it on-the-fly as needed. For the purpose store the domain instead (obtained through libfqfft::get_evaluation_domain<Field>()) in the srs and circuit_t and pass this as input to functions that use the L_basis parameter.

(bullet 3) (related to 2): create the domain once (using libfqfft::get_evaluation_domain<Field>()) and carry it around as necessary. See #49 (comment)

Plonk: add public input indices to the srs

• Add public input indices in the srs
• Move public input polynomial out of the srs
• Add public input values as an additional input to the verifier
• Remove circuit_t struct
• Add code to automatically transpose the gates matrix (as opposed to hard-code the transposed version as it is now)

Name test files consistently

Most of the test files are named test_<filename>.cpp, nevertheless https://github.com/clearmatics/libsnark/blob/develop/libsnark/gadgetlib1/tests/gadgetlib1_test.cpp is named <filename>_test.cpp which contrasts with the rest. This is inconsistent, and this inconsistency is ever more striking on #29 where a list of test files are added in the gadgetlib1/tests folder and named with a test_ prefix.

We need to bring consistency in how we name test files to have clean includes and a systematic way of creating tests. While the overwhelming set of test files is using the test_ prefix in libff and libsnark, the _test suffix is used in:

• Zeth (e.g. see: https://github.com/clearmatics/zeth/tree/master/libzeth/tests/serialization)
• Libfqfft (see: https://github.com/clearmatics/libfqfft/tree/master/libfqfft/tests)
• Zecale (e.g. see: https://github.com/clearmatics/zecale/tree/master/libzecale/tests/aggregator)

As such, I am in favor to adopt the <filename>_test.cpp naming convention for all test files across all projects to keep naming consistent and systematic and have similar looking include directives.

Remove domain from plonk srs?

Plonk implementation:

• check where domain is created / used
• can we create once and use in multiple places?

PRs:

• #70

Inconsistent asserts and default annotation parameters

Various places in the code (e.g. pb_variable_array) provide a default annotation_prefix parameter of "", but then assert if this is passed in. e.g. https://github.com/clearmatics/libsnark/blob/master/libsnark/gadgetlib1/pb_variable.tcc#L29.

Switch all tests to gtest

Some test code still uses manual function calls and assert to perform tests (which also generates a lot of "unused variable" warnings. Switch to gtest and use the ASSERT_... family of calls.

Anemoi: parameters struct for all curves of interest

Plonk: code cleanup

• Have explicit struct members of the a,b,c polynomials currently stored as a single vector W_polys_blinded_at_secret_g1[a,b,c] in the plonk_proof class. Similarly, have explicit struct members for the q_L, q_R, q_O, q_M, a_C selector polynomials currently stored as a single vector Q_polys in the circuit struct.

• In all header files: leave only functions that are externally visible to calling code. If a function is only used "internally" in the implementation then just leave it in the .tcc (and not in the .hpp), potentially in the internal namespace (to hide it from other code). See also discussion: #49 (comment)

• Make sure that all functions do not rely on the caller for allocating the correct sizes of the return parameters. Make the latter to be allocated inside the functions that compute them. See also #49 (comment)

Anemoi: small issues

To be fixed in base branch anemoi-hash-r1cs:

• remove all printfs (#65 (comment))
• use of test_pb_verify_circuit #65 (comment)
• use gtest framework (#65 (comment))

To be fixed in separate branches:

• remove duplicate gadgets (#65 (comment))
• static assert or specialised implementations for valid values of NumStateColumns_L (#65 (comment))

AnemoiJive R1CS gadget

Implement R1CS gadget for AnemoiJive. This is the Jive mode of operation of the Anemoi permutation [1]. It effectively hashes $m$ input elements into a single element ( $m$ to $1$ compression). Specifically designed to be used in Merkle tree applications. See Sect. 3.2 in [1] for the full description and [2] for some 3rd party applications using $3$ to $1$ compression for Merkle trees in TurboPlonk.

[1] https://eprint.iacr.org/2022/840
[2] https://eprint.iacr.org/2022/1487

AnemoiSponge R1CS gadget

Implement R1CS gadget for AnemoiSponge. This is the Sponge mode of operation of the Anemoi permutation [1]. It transforms Anemoi into a general purpose hash function like SHA3/Keccak, MimC, etc. with arbitrary size input and fixed but variable size output. See Sect. 3.1 in [1] for the full description.

[1] https://eprint.iacr.org/2022/840

CI check for memory errors

Some of the tests which use the r1cs_example and create and verify a zksnark fail intermittently on mac. Re-running the same executable about 10 times, the tests fail approx 8 times.

This smells very much like a memory error of some kind, so it would be great to start running valigrind on the CI.

Why to build your own library?

Why to build your own library, when there is already a C++-based libsnark-alike PLONK-featuring implmenetaion by =nil; Foundation: https://github.com/NilFoundation/crypto3, (in here more precisely: https://github.com/NilFoundation/crypto3-zk)?

Plonk circuit builder interface

Builder class to abstract the generation of constraints and permutations.

• Consider how this should look
• Check existing interfaces

Plonk: error handling

Issue description (bullet 8): When throwing an exception, it should not be up to the function that throws it to decide how it should be handled. Catch and handle must be done by the caller instead, who may wish to handle it in some way other than exiting the process, like retrying. See #49 (comment) , #49 (comment) , #49 (comment)

Anemoi circuit to run all rounds

Plonk: compute srs from matrix and permutation (depends on #86, #87, #88, #89)

• Generalise the function plonk_circuit_description_from_example to compute all polynomials from matrix + permutation
• Use this to make plonk_srs_derive_from_usrs accept constraint matrix and permutation.

Note: this issue depends on #86, #87, #88

Plonk: hard-code constants k1, k2 in plonk_srs_derive_from_usrs

In plonk_srs_derive_from_usrs hard-code the values for the constants $k_1,k_2$ as opposed to extracting them from the example class.

Anemoi: single anemoi pemutation round gadget

Plonk transcript_hasher implementation with SHA3, BLAKE, etc.

• Create a transcript_hasher implementation using standard hash function such as SHA3, BLAKE, etc. which can be used on any curve. Since it does not need special constants, the data can just serialize to binary and feed to the hash function.

• Set the default value of the Plonk transcript_hasher template parameter (on the Plonk prover and verifier) to the SHA3 (or BLAKE) hasher.

See #61 (comment)

Faster compuation of permutation polynomials

The current Plonk implementation enumerates the entire set $H' = H \cup k_1 \cdot H \cup k_2 \cdot H$, permutes the set, and then does an iFFT to compute the permutation polynomials (See srs.tcc and related comments).

@vesselinux I think instead we can compute the differences between, say $S_{\sigma_1} - S_{ID_1} = S_{\sigma_1}(X) - X$ as an array of values in the evaluation space. For each $i$ we want to subtract $\omega^i L_i(X)$ from $S_{ID_1}$ (to get a zero at $\omega^i$) and add $\sigma^{\star}(j) L_i(X)$. Therefore, if we create an array with $i$-th entry as $\omega^i - \sigma^{\star}(i)$ (which only needs to be computed for entries which are actually affected by the permutation) we can take the iFFT of this to get $S_{\sigma_1} - S_{ID_1}$. Adding a $1$ in the $X$ position (index 1) should then yield $S_{\sigma_1}$.

For $S_{\sigma_2}$ and $S_{\sigma_3}$ we need to use $k_1$ and $k_2$ instead of $1$ at the end.

I may have missed some detail here, but I think we can use this as an approach to avoid having to enumerate the entire set $H'$ and manually permute it.

Plonk: have explicit members for the commitmemnt and selector polynomials

A possible change to consider for Plonk:

Have explicit struct members of the a,b,c polynomials currently stored as a single vector W_polys_blinded_at_secret_g1[a,b,c] in the plonk_proof class. Similarly, have explicit struct members for the q_L, q_R, q_O, q_M, a_C selector polynomials currently stored as a single vector Q_polys in the circuit struct.

Plonk: list of suggested changes

1. Have explicit struct members of the a,b,c polynomials currently stored as a single vector W_polys_blinded_at_secret_g1[a,b,c] in the plonk_proof class. Similarly, have explicit struct members for the q_L, q_R, q_O, q_M, a_C selector polynomials currently stored as a single vector Q_polys in the circuit struct.

2. Do not store the Lagrange basis L_basis explicitly (e.g. as part of the srs class and circuit_t struct). Instead compute it on-the-fly as needed. For the purpose store the domain instead (obtained through libfqfft::get_evaluation_domain<Field>()) in the srs and circuit_t and pass this as input to functions that use the L_basis parameter.

3. Related to 2.: create the domain once (using libfqfft::get_evaluation_domain<Field>()) and carry it around as necessary. See #49 (comment)

4. In all header files: leave only functions that are externally visible to calling code. If a function is only used "internally" in the implementation then just leave it in the .tcc (and not in the .hpp), potentially in the internal namespace (to hide it from other code). See also discussion: #49 (comment)

5. Move the print_vector function https://github.com/clearmatics/libsnark/blob/plonk/libsnark/zk_proof_systems/plonk/utils.tcc#L19-L25 to libff/.../field_utils.hpp. (Nice print functions in libff clearmatics/libff#70)

6. Move the plonk main directory from zk_proof_systems/ to zk_proof_systems/ppzksnark/ since technically Plonk is a pre-processed zk-SNARK (ppzksnark).

7. Make sure that all functions do not rely on the caller for allocating the correct sizes of the return parameters. Make the latter to be allocated inside the functions that compute them. See also #49 (comment)

8. When throwing an exception, it should not be up to the function that throws it to decide how it should be handled. Catch and handle must be done by the caller instead, who may wish to handle it in some way other than exiting the process, like retrying. See #49 (comment) , #49 (comment) , #49 (comment)

9. Break down the long functions plonk_prover<ppT>::round_five() and plonk_prover<ppT>::round_three() into several sub-functions for the purposes of unit testing and readability (see next). See #49 (comment)

10. Have a separate unit test under tests/ to verify the value of r_poly, W_zeta and W_zeta_omega in plonk_prover<ppT>::round_five(). See #49 (comment) and #49 (comment)

11. Compute the challenges alpha, beta, gamma, zeta, nu, u using a hash function rather than passing them as pre-computed inputs as part of the transcript_hash_t structure. The latter is done now for the purposes of unit testing. In the long run we might want something like a transcript_hasher which can accept field and group elements as they are available, and then output a digest when required. See discussion at #49 (comment) and #49 (comment)

Add all miller loop gadgets for BW6-761/BLS12-377 gadgets

Check libsnark/gadgetlib1/gadgets/curves/bw6_761_bls12_377/... and add all required miller loops (e.g. ee/e) from libsnark/gadgetlib1/gadgets/pairing/pairing_params.hpp.

Better distinction between "gadget" and "variable" types

Objects such as G1_variable or r1cs_gg_ppzksnark_verification_key_variable are simply containers of logical groups of pb_variables and pb_linear_combinations. However, many of them still inherit from gadget (which in turn implies that they generate conditions), simply in order to maintain a reference to the protoboard.

These "variable" objects could inherit gadgets privately, since they only use the base class internally to access this->pb. Alternatively (preferably) we may wish to define a new interface for such objects, whereby they only have set (or generate_r1cs_witness) methods.

Potentially, we could also slim-down the variable objects by passing in the pb reference rather than redundantly storing it with every "variable" instance. e.g. the verification key variables hold the pb ref, as do each of their G1_variable and G2_variable members. (Note, a similar argument about the pb ref could also be made for regular gadgets).

Implemenation of Anemoi (algebraic hash function) gadget

Depends on #96

Plonk: transcript hasher

It could be useful to have the following interface for transcript hashing:

template<typename ppT> class transcript_hasher
{
public:
   void add_element(const libff::Fr<ppT> &element);
   void add_element(const libff::G1<ppT> &element);
   void add_element(const libff::G2<ppT> &element);

   libff::Fr<ppT> get_hash();
};

• #61

Consistent member variable naming

We currently have:

• member_variable in most old code
• member_variable_ in a few bits of old code
• _member_variable used in most new code

Determine a policy (e.g. _member_variable for any new or edited code?) and apply it to all related repos:

• clearmatics/zeth
• clearmatics/zecale
• clearmatics/libsnark
• clearmatics/libff
• clearmatics/libfqfft

Plonk: tests with real values

a sample circuit in terms of constraints + permutation

Plonk: setting k1 and k2

• can we precompute for each curve, assuming domains are power-of-2 size
• check how this is done in other implementations

Anemoi: NumStateColumns static parameter

Test simple Plonk circuit on all curves

Add unit tests for the simple quadratic residue circuit for all curves. See PR #107 and Issue #108.

Test simple Plonk circuit on Edwards and BN128 curves

Add unit tests for the simple quadratic residue circuit for the Edwards and BN128 curves. Issue #103 addresses this for all curves except Edwards and BN128. For the latter two, the following compilation errors are generated:

Edwards:

libsnark/depends/libff/libff/algebra/fields/bigint.tcc:36: libff::bigint<n>::bigint(const char*) [with long int n = 3]: Assertion limbs_written <= n failed.

BN128:

libsnark/depends/libff/libff/algebra/fields/field_serialization.tcc:133:51: error: ‘const class mie::Fp’ has no member named ‘coeffs’
133 |             typename std::decay<decltype(field_el.coeffs[0])>::type;

By the message, the BN128 error (serialization) may be related to the fact that when we are adding an element or a curve point to the hasher buffer, we first convert it into a string and then copy the string as a sequence of uint8_t elements at the end of the buffer. Possible solution to try would be to use the specific functions which can serialize as bytes in libff (see serialization files in the algebra dir).

Plonk: list of suggested optimizations

1. Generate the sets $H$, $k1 H$, $k2 H$ directly in the vector H_gen rather than first as the 2D vector omega[1, k1, k2][...] and then copying the latter into H_gen in plonk_roots_of_unity_omega_to_subgroup_H Update: done by introducing function plonk_multiplicative_subgroups_H_k1H_k2H .

2. TBD