btq-ag / keelung

The Keelung Language

Keelung

Keelung is a domain-specific language (DSL) embedded in Haskell that is designed for creating zero-knowledge proofs (ZKPs). With Keelung, developers can create high-quality and reliable ZKPs with ease, even without expertise in cryptography.

The Keelung Compiler

Keelung programs are compiled with the Keelung compiler.

Documentation

Check out our GitBook for more information.

Language Reference

The language reference is now available on Hackage.

Standard Library

The standard library is available here.

Discord

Find us on Discord!

Codespaces

Get started using Keelung in GitHub codespaces by navigating to Code -> Codespaces in Keelung Examples!

Roadmap

Current (v0.21.0)

• A more precise method of reference counting on unsigned integers
• Renaming of primitive datatype conversions
• Tons of bug fixes and stability improvements

Upcoming (v0.22.0)

• Operators for slicing and joining unsigned integers
• Internal: improvement of representation of polynomials with our new precise reference counter
• Internal: polynomial insertion is now 9x faster

Future

• Snarkjs/Circom Integration: Snarkjs/Circom is currently the most popular ZK toolchain. We aim to tap into this ecosystem by integrating Keelung with Snarkjs/Circom.
• Standard Library Optimization: We plan to heavily optimize ciphers and hash functions such as AES and Blake2.
• PLONKish Backend Compilation Support: Supporting protocols like PLONK would allow Keelung programs to efficiently manage complex privacy-preserving computations, significantly enhancing their scalability and security across various applications.
• Quantitative Information Flow Analysis: We are working with Prof. Kohei Suenaga of Kyoto University to enable privacy leakage analysis of Keelung programs.
• DSL Refinement: We are transitioning statement operators (e.g. performDivMod, which is only allowed in the Comp monad) to pure operators to make these operations more versatile and easily integrated into different parts of our system.
• Performance Improvement: We aim to speed up the compilation of operators on commonly used types such as Bytes, Word16, Word32, and Word64.
• Benchmarking CI/CD: We plan to include benchmarking in the CI/CD pipeline.

keelung's Issues

Cannot build with Cabal

Building with cabal build would result in the following error message:

Warning: Unknown/unsupported 'ghc' version detected (Cabal 3.6.2.0 supports
'ghc' version < 9.4): /opt/homebrew/bin/ghc is version 9.4.4
Resolving dependencies...
cabal: Could not resolve dependencies:
[__0] trying: keelung-0.9.1.0 (user goal)
[__1] trying: galois-field-1.0.2 (dependency of keelung)
[__2] next goal: protolude (dependency of galois-field)
[__2] rejecting: protolude-0.3.3 (conflict: galois-field => protolude>=0.2 &&
<0.3)
[__2] skipping: protolude-0.3.2, protolude-0.3.1, protolude-0.3.0 (has the
same characteristics that caused the previous version to fail: excluded by
constraint '>=0.2 && <0.3' from 'galois-field')
[__2] trying: protolude-0.2.4
[__3] next goal: base (dependency of keelung)
[__3] rejecting: base-4.17.0.0/installed-4.17.0.0 (conflict: protolude =>
base>=4.6 && <4.16)
[__3] skipping: base-4.17.0.0, base-4.16.4.0, base-4.16.3.0, base-4.16.2.0,
base-4.16.1.0, base-4.16.0.0 (has the same characteristics that caused the
previous version to fail: excluded by constraint '>=4.6 && <4.16' from
'protolude')
[__3] rejecting: base-4.15.1.0, base-4.15.0.0, base-4.14.3.0, base-4.14.2.0,
base-4.14.1.0, base-4.14.0.0, base-4.13.0.0, base-4.12.0.0, base-4.11.1.0,
base-4.11.0.0, base-4.10.1.0, base-4.10.0.0, base-4.9.1.0, base-4.9.0.0,
base-4.8.2.0, base-4.8.1.0, base-4.8.0.0, base-4.7.0.2, base-4.7.0.1,
base-4.7.0.0, base-4.6.0.1, base-4.6.0.0, base-4.5.1.0, base-4.5.0.0,
base-4.4.1.0, base-4.4.0.0, base-4.3.1.0, base-4.3.0.0, base-4.2.0.2,
base-4.2.0.1, base-4.2.0.0, base-4.1.0.0, base-4.0.0.0, base-3.0.3.2,
base-3.0.3.1 (constraint from non-upgradeable package requires installed
instance)
[__3] fail (backjumping, conflict set: base, keelung, protolude)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: base, protolude, galois-field, keelung
Try running with --minimize-conflict-set to improve the error message.

Probable cause:

• Reference to our fork of galois-field not available to Cabal
• GHC version not specified

Add `Traversable` constraint to `toArray` for convenience

keelung/src/Keelung/Monad.hs

Line 184 in b108bc7

toArray :: Referable t => [Val t n] -> Comp n (Val ('Arr t) n)

While writing hash functions, to be safe, I would prefer to use fix sized array instead of list.
wondering if it would be better and possible to add a Traversable constraint to toArray ?

toArray :: (Referable t, Traversable f) => f (Val t n) -> Comp n (Val ('Arr t) n)

so that I can write something like

toW64Arr :: Array Int Word64 -> Comp n (Val ('Arr W64) n)
toW64Arr = mapM fromWord64 >=> toArray

since Array Int is an instance of Traversable

Apparent variable substitution

g :: Comp (UInt 5)
g = do
  a <- input Private
  b <- input Private
  c <- reuse $ a * b
  return $ c .&. 5

yields

Right R1CS {
  Constriant (31): 
    Ordinary constraints (11):

      $0 + 1552511030102430251236801561344621993261920897571225600$18 = 0
      1552511030102430251236801561344621993261920897571225600$1 = 0
      $2 + 1552511030102430251236801561344621993261920897571225600$20 = 0
      1552511030102430251236801561344621993261920897571225600$3 = 0
      1552511030102430251236801561344621993261920897571225600$4 = 0
      1552511030102430251236801561344621993261920897571225600$18 + $23 = 0
      1552511030102430251236801561344621993261920897571225600$19 + $24 = 0
      1552511030102430251236801561344621993261920897571225600$20 + $25 = 0
      1552511030102430251236801561344621993261920897571225600$21 + $26 = 0
      1552511030102430251236801561344621993261920897571225600$22 + $27 = 0
      $16 * $17 = $34

    Boolean constraints (15):

      $0 = $0 * $0
        ...
      $4 = $4 * $4
      $6 = $6 * $6
        ...
      $15 = $15 * $15

    Binary representation constraints (5):

      $5 = $0 + 2$1 + ... + 2^4$4
      $16 = $6 + 2$7 + ... + 2^4$10
      $17 = $11 + 2$12 + ... + 2^4$15
      $33 = $18 + 2$19 + ... + 2^4$22
      $34 = $23 + 2$24 + ... + 2^9$32

  Variables (35):

    Output variables: $0 ... $5
    Private Input variables: $6 ... $17

}

, which is correct but there are apparent variable substitutions: $0 = $18 = $23, $2 = $20 = $24, etc.

Doing this at some stage of the optimization pipeline, preferably recursively, would result in much less constraints.

A mininal example is:

h :: Comp (UInt 5)
h = input Private

Right R1CS {
  Constriant (18): 
    Ordinary constraints (6):

      $0 + 1552511030102430251236801561344621993261920897571225600$6 = 0
      $1 + 1552511030102430251236801561344621993261920897571225600$7 = 0
      $2 + 1552511030102430251236801561344621993261920897571225600$8 = 0
      $3 + 1552511030102430251236801561344621993261920897571225600$9 = 0
      $4 + 1552511030102430251236801561344621993261920897571225600$10 = 0
      $5 + 1552511030102430251236801561344621993261920897571225600$11 = 0

    Boolean constraints (10):

      $0 = $0 * $0
        ...
      $4 = $4 * $4
      $6 = $6 * $6
        ...
      $10 = $10 * $10

    Binary representation constraints (2):

      $5 = $0 + 2$1 + ... + 2^4$4
      $11 = $6 + 2$7 + ... + 2^4$10

  Variables (12):

    Output variables: $0 ... $5
    Private Input variables: $6 ... $11

}

Also, it would be better to print p - 1 as -1, as older versions did.

By doing the operations above, it will be easier for humans to verify the correctness of the generated R1CS circuit.

Right shift not terminating

@ilway25 found that, right shift would cause both compile and interpret to be non-terminating.

program :: UInt 4
program = do
    x <- inputUInt Public
    return (x .>>. 3)

Replacing x .>>. 3 with x .<<. (-3) somehow works.

Other distribution channel for aurora-prover besides Docker

Hi, is there a way to compile or obtain the aurora-prover binary without using docker?

`toArray` should should copy elements instead of just moving them

@potsrevennil discovered that the array xs in the following example would only have one element.

dupArray :: Comp GF181 (Val 'Num GF181)
dupArray = do
  x <- input 
  xs <- toArray [x, x]
  access xs 1 

Tutorial Doesn't Load Types

Following your tutorial: https://btq.gitbook.io/keelung/getting-started/hello-world

I've loaded the tutorial module, and the moment I try to define the echo function, I get the following error:

How do I resolve this?

Serialization/deserialization problem when compiling `BLAKE2b.test`

Compiling BLAKE2b.test would result in:

ghci> compile BLAKE2b.test
"too few bytes\nFrom:\tdemandInput\n\n"

Serialization/deserialization went wrong.

`verify` is broken

The verify interface to call Aurora prover is for now unusable, e.g., after generating a proof successfully:

ghci> prove gf181 echo [] [1]

verify encounters such error:

ghci> verify
Generated parameter file at: aurora/proof
vector
*** Exception: readCreateProcess: aurora_verify "--r1cs_filepath" "aurora/circuit.jsonl" "--input_filepath" "aurora/witness.jsonl" "--parameter_filepath" "aurora/parameter.json" "--proof_filepath" "aurora/proof" (exit 1): failed

Aurora verifier only outputs a string vector.

Cannot build with stack

Hi, I'm running into an issue when trying to build keelung-examples with stack.

└─[$] <git:(main)> stack build
[1 of 2] Compiling Main             ( /Users/chris/.stack/setup-exe-src/setup-SvXsv1f_.hs, /Users/chris/.stack/setup-exe-src/setup-SvXsv1f_.o )
[2 of 2] Compiling StackSetupShim   ( /Users/chris/.stack/setup-exe-src/setup-shim-SvXsv1f_.hs, /Users/chris/.stack/setup-exe-src/setup-shim-SvXsv1f_.o )
Linking /Users/chris/.stack/setup-exe-cache/x86_64-osx/tmp-Cabal-simple_SvXsv1f__3.6.3.0_ghc-9.2.5 ...
ld64.lld: error: unknown argument '--gc-sections'
clang: error: linker command failed with exit code 1 (use -v to see invocation)
`gcc' failed in phase `Linker'. (Exit code: 1)

Error: [S-6374]
       While building simple Setup.hs (scroll up to its section to see the error) using:
       /Users/chris/.stack/programs/x86_64-osx/ghc-9.2.5/bin/ghc-9.2.5 -rtsopts -threaded -clear-package-db -global-package-db -hide-all-packages -package base -main-is StackSetupShim.mainOverride -package Cabal-3.6.3.0 /Users/chris/.stack/setup-exe-src/setup-SvXsv1f_.hs /Users/chris/.stack/setup-exe-src/setup-shim-SvXsv1f_.hs -o /Users/chris/.stack/setup-exe-cache/x86_64-osx/tmp-Cabal-simple_SvXsv1f__3.6.3.0_ghc-9.2.5
       Process exited with code: ExitFailure 1 

Missing Encode Instance for Either?

I'm curious why there isn't an Encode instance for Either, given that there is a GEncode instance, e.g

instance (GEncode a, GEncode b) => GEncode (a :+: b) where
  gencode (L1 x) = gencode x
  gencode (R1 x) = gencode x

https://github.com/btq-ag/keelung/blob/7a6f43a3b01e625e4cf08022a699b39f59fa695f/src/Keelung/Syntax/Encode.hs#L133-L135C25

Reconsider module names (`Syntax.Encode.Syntax` etc.)

It's confusing to have Keelung.Syntax, Keelung.Syntax.Encode, Keelung.Syntax.Encode.Syntax at the same time.
Representations of syntax to be passed to the compiler can be IR.

`access` constructs subarray with incorrect length

As pointed out by @ilway25, access should construct a subarray with its length from readHeap (addr, i).

keelung/src/Keelung/Monad.hs

Line 246 in 3d77697

access (Ref (Array elemType len addr)) i = Ref . Array elemType len . snd <$> readHeap (addr, i)

Some array indexing problem

https://github.com/btq-ag/keelung-examples/blob/31c0189ed67976497a84e2dee5a39610fe0530bd/src/BLAKE2b.hs#L235-L246
explodes if we change 3 to 4 in the following code.

forM_ [0 .. 3] $ \i -> do
    -- Select message mixing schedule for this round.
    -- BLAKE2b uses 12 rounds, while 'sigma' has only 10 entries.
    let indices = sigma !! i

Programs returning arrays get compiled into rubbish

goofy :: Comp GF181 (Val ('Arr 'Num) GF181)
goofy = do 
  x <- input 
  toArray [x, 2 * x]

If we compile the program above we'd get:

Right R1CS {
  R1C constraints (4):

    -1 + $0 = 0
    -1 + $1 = 0
    -2 + $2 = 0

  number of variables: 3
  input  variables: $0
  output variables: [1,2]
}

Allow users to use any field they like

Due to some reason in the early days of development, the choice of field is currently limited to just GF181, BN128, and B64.

keelung/src/Keelung/Field.hs

Lines 50 to 57 in 863b4e8

	-- | Field types provided by the compiler
	data FieldType
	= -- | Binary field of 64 bits
	B64
	| -- | Prime field of order 181
	GF181
	| -- | Barreto-Naehrig curve of 128 bits
	BN128

I believe that the blocking issue has been resolved, so the users should be allowed to use any field they like.