protolambda / go-kzg

FFT, data-recovery and KZG commitments, a.k.a. Kate commitments, in Go - *super experimental*

KZG and FFT utils

This repo is super experimental.

This is an implementation in Go, initially aimed at chunkification and extension of data, and building/verifying KZG proofs for the output data. The KZG proofs, or Kate proofs, are built on top of BLS12-381.

Part of a low-latency data-availability sampling network prototype for Eth2 Phase 1. See https://github.com/protolambda/eth2-das

Code is based on:

• KZG Data availability code by Dankrad
• Verkle and FFT code by Dankrad and Vitalik
• Reed solomon erasure code recovery with FFTs by Vitalik
• FFT explainer by Vitalik
• Kate explainer by Dankrad
• Kate amortized paper by Dankrad and Dmitry

Features:

• (I)FFT on F_r
• (I)FFT on G1
• Specialized FFT for extension of F_r data
• KZG
  • commitments
  • generate/verify proof for single point
  • generate/verify proofs for multiple points
  • generate/verify proofs for all points, using FK20
  • generate/verify proofs for ranges (cosets) of points, using FK20
• Data recovery: given an arbitrary subset of data (at least half), recover the rest
• Optimized for Data-availability usage
• Change Bignum / BLS with build tags.

BLS

Currently supported BLS implementations: Herumi BLS and Kilic BLS (default).

Field elements (Fr)

The BLS curve order is used for the modulo math, different libraries could be used to provide this functionality. Note: some of these libraries do not have full BLS functionality, only Bignum / uint256. The KZG code will be excluded when compiling with a non-BLS build tag.

Build tag options:

• (no build tags, default): Use Kilic BLS library. Previously used by bignum_kilic build tag. kilic/bls12-381
• -tags bignum_hbls: use Herumi BLS library. herumi/bls-eth-go-binary
• -tags bignum_hol256: Use the uint256 code that Geth uses, holiman/uint256
• -tags bignum_pure: Use the native Go Bignum implementation.

Benchmarks

See BENCH.md for benchmarks of FFT, FFT in G1, FFT-extension, zero polynomials, and sample recovery.

License

MIT, see LICENSE file.

go-kzg's Issues

kilic multiplication isn't reentrant

We found another thread-safety issue with Kilic, described here. In short, it seems that if MulG1 is called concurrently, the two contexts will interact and give an incorrect results.

The following test exposes the problem: 10 threads are running concurrently, each performing a simple EC point multiplication with MulG1, and the result will be sent over a channel. The results are compared with a value calculated in a serial execution context, and they differ (see the failed CI build).

func TestConcurrentMulG1(t *testing.T) {
	var fr bls.Fr
	bls.AsFr(&fr, 2)
	expected := new(bls.G1Point)
	bls.MulG1(expected, &bls.GenG1, &fr)

	threads := 10
	ch := make(chan *bls.G1Point)
	builder := func() {
		var fr bls.Fr
		bls.AsFr(&fr, 2)
		dst := new(bls.G1Point)
		bls.MulG1(dst, &bls.GenG1, &fr)
		ch <- dst
	}

	for i := 0; i < threads; i++ {
		go builder()
	}

	for i := 0; i < threads; i++ {
		res := <-ch
		if res.String() != expected.String() {
			t.Error("Incorrect fr")
		}
	}
}

Allow non unit strides in DASFFTExtension()

Hardcoding the value 1 for the initial stride here is unnecessarily restrictive:

	fs.dASFFTExtension(vals, 1)

See benjaminion/c-kzg#10

Why are there 4096 secret g2 points in the trusted setup?

Is there a reason there are so many secret g2 points in the embedded trusted setup? There are 4096 (which matches the number of secret g1 points) but I believe only 65 are required. This is what C-KZG-4844 uses in their trusted setup. The secret g1 points & the first 65 secret g2 points do match C-KZG-4844. The number of points seems intentional, as there's this check:

go-kzg/kzg.go

Lines 22 to 24 in c91cee5

	if len(secretG1) != len(secretG2) {
	panic("secret list lengths don't match")
	}

Performance benchmarking

Thank you for designing the library.

We are looking into using this library in our project.

Is there are a benchmarking tool ? If we use this library as a Merkle tree replacement, how many bytes per second could we process, say, on a typical PC? How much time does it take for a single update?