crate-crypto / rust-verkle Goto Github PK

• One of these is where the user passes in a key which has not been included in the proof
• Another is where the keys and values do not match in length

It might be worthwhile to benchmark the difference in performance for these two structures:

Variation 1:

enum Node {
Leaf(Leaf),
Hashed(Hashed),
Internal(Box),
}

Variation2:

Box<[Option<Box>; YOUR_LEN]>

Variation 2 is closer to what the Golang impl is doing since interfaces are Boxed

Here:

WIDTH_10 should be renamed to WIDTH

This is a tracking issue for the following tasks:

• impl SqLite as a key value and make the database optional
• One can also impl Sqlite in verkle repo and use tables
• Possibly add https://github.com/nomic-io/merk as a backend

This should match the format implemented in go-ipa . The following draft document specifies the rough rationale: https://hackmd.io/q60-8MnaT8SY5NhenKD0Pg

consolidate dummy_setup and other methods which setup a dummy srs

I am not yet sure if the error is with rust-verkle or go-verkle, I'm brain-dumping here.

I have a running testnet in which geth produces blocks, and I am verifying them with rust-verkle to make sure that the codebases are fully compatible.

I produce the following proof:

00000000090000000808080a0808080a0804000000bdab5fe8197dfb538e43e93b36bee12dc4b13a59bb88c96e9594b368e139c881a059443990023f4517419b00e286b52c979471c10034969ebb7ab86f7b6892d5c8f712f666f0daab3ac325f329f078f932afff9c7c970be57befaf4c6021576734854706d259fe6cb0e9fa626a8fb708fb03654aca32d003ca4078a07d7ae09d9f2b622162addd376bc0a7dcf01b8caf1e55cd10ab26f3d8f178ba3142183a43e7d77e98b9a74bf6b082682af1a0fd0d94d095c8f5cd97dbaf3c74ab32eaafbd06b9f6a3e317f60800768c7bd92482fa3b51f6cb9456fa9aa5e24a62a4c49faa13851e163c021e1eb096798cdc18ae42c637e8f96c075b627beb18013515f14984d3e47ea1df829209b850a0964e124b4da2605e881e2ba2f5e09a45672e6ca838c5ccd1a2ceb5d5ba629a0bbc8b9e1f6300debb4e6cb495c5faabe61afeb40744253e682305b5822d56ec0fb0d55338dc4fdf49803c3550a8294ebe366fec53a4cd218d32337b6c7c8b3d1d30920e9cc9aea232064ba63aba36c8a4a6e3daa07eb548b47406c09e2b619b6b9f02725ccd01c3e2ae60a461cdbe8033415d610e215070c9927f21a4b0f0c6cdf9122b2d69727d51cc6e9ae6fbcefe97936b87dbd312abd0e84942309a19d0da051f6faf1b953bbfd6b8452737b0e1d64a2f329be67a86bc4759c8430c625e21e5fa8d7f4a229c02cc8115d4f51960d8eae6fb9947169fa5ae5b3a9fbe2783bcb51fdbfbf9a828a469e08c9832d5f647848df3483c34f50e682fa271bad4e866ce04d8dc196a4135c51a534191d835998f6a6c70974a2e6444c9f85b8f314c510c929cc700ab58e7544babb1360cc2a5a7438e25a4b97603284fb60925486ffd7a8885e9bfe6ecfc0dc00ed9eddedfe96e78e60db0bec41442c042c826326c760e9f3cf6dff85b8a1cf319b9e984ef0c96ee8253e6e9762dc4d466d615304ec658d7397ddf2f6bf97a52ef70905c953be6746a18

Note that there are 9 "extension status" descriptors.

The associated (keys, values) are shown below. There are 10 different stems, but two stems use the same proof of absence (path 0x69...). All the blocks that don't verify seem to have the same feature (there's 500+ of them so far, only checked a dozen).

      [
        088bcd6531bcfdd7d4529b689d415af18485f3e5f7f08d50d0d3fc6f95e01500,
        "",
      ],
      [
        088bcd6531bcfdd7d4529b689d415af18485f3e5f7f08d50d0d3fc6f95e01502,
        "",
      ],
      [
        088bcd6531bcfdd7d4529b689d415af18485f3e5f7f08d50d0d3fc6f95e01503,
        "",
      ],
      [
        1be5b2c8a376ad34ba9e5917ef6f84de1d342758d5e147f7b9d05b90a2cd6700,
        "",
      ],
      [
        1be5b2c8a376ad34ba9e5917ef6f84de1d342758d5e147f7b9d05b90a2cd6701,
        "",
      ],
      [
        1be5b2c8a376ad34ba9e5917ef6f84de1d342758d5e147f7b9d05b90a2cd6702,
        "",
      ],
      [
        1be5b2c8a376ad34ba9e5917ef6f84de1d342758d5e147f7b9d05b90a2cd6703,
        "",
      ],
      [
        1be5b2c8a376ad34ba9e5917ef6f84de1d342758d5e147f7b9d05b90a2cd6704,
        "",
      ],
      [
        212e8cb758fb84a7ae1715f444bdb71c4c9e5db400a88cdee03338b91c5e8f79,
        "",
      ],
      [
        6766d007d8fd90ea45b2ac9027ff04fa57e49527f11010a12a73f58ffa580800,
        0000000000000000000000000000000000000000000000000000000000000000,
      ],
      [
        6766d007d8fd90ea45b2ac9027ff04fa57e49527f11010a12a73f58ffa580801,
        e89208cfa015d5191e0200000000000000000000000000000000000000000000,
      ],
      [
        6766d007d8fd90ea45b2ac9027ff04fa57e49527f11010a12a73f58ffa580802,
        3300000000000000000000000000000000000000000000000000000000000000,
      ],
      [
        6766d007d8fd90ea45b2ac9027ff04fa57e49527f11010a12a73f58ffa580803,
        c5d2460186f7233c927e7db2dcc703c0e500b653ca82273b7bfad8045d85a470,
      ],
      [
        6766d007d8fd90ea45b2ac9027ff04fa57e49527f11010a12a73f58ffa580804,
        0000000000000000000000000000000000000000000000000000000000000000,
      ],
      [
        694dc6ea427eea992996e5dfa6992eb0434d7e305574cf74f45226538e34a800,
        "",
      ],
      [
        694dc6ea427eea992996e5dfa6992eb0434d7e305574cf74f45226538e34a802,
        "",
      ],
      [
        694dc6ea427eea992996e5dfa6992eb0434d7e305574cf74f45226538e34a803,
        "",
      ],
      [
        695921dca3b16c5cc850e94cdd63f573c467669e89cec88935d03474d6bdf900,
        "",
      ],
      [
        695921dca3b16c5cc850e94cdd63f573c467669e89cec88935d03474d6bdf901,
        "",
      ],
      [
        695921dca3b16c5cc850e94cdd63f573c467669e89cec88935d03474d6bdf902,
        "",
      ],
      [
        695921dca3b16c5cc850e94cdd63f573c467669e89cec88935d03474d6bdf903,
        "",
      ],
      [
        695921dca3b16c5cc850e94cdd63f573c467669e89cec88935d03474d6bdf904,
        "",
      ],
      [
        6d3edc4c95095f56c4019b8aa097cfd9e3f7095966788bc497e488d78f29fa00,
        "",
      ],
      [
        6d3edc4c95095f56c4019b8aa097cfd9e3f7095966788bc497e488d78f29fa01,
        "",
      ],
      [
        6d3edc4c95095f56c4019b8aa097cfd9e3f7095966788bc497e488d78f29fa02,
        "",
      ],
      [
        6d3edc4c95095f56c4019b8aa097cfd9e3f7095966788bc497e488d78f29fa03,
        "",
      ],
      [
        6d3edc4c95095f56c4019b8aa097cfd9e3f7095966788bc497e488d78f29fa04,
        "",
      ],
      [
        8cbef1db2d70a48f58505953351092c37f903bcf78b0aff05b1d544609fb8900,
        "",
      ],
      [
        8cbef1db2d70a48f58505953351092c37f903bcf78b0aff05b1d544609fb8901,
        "",
      ],
      [
        8cbef1db2d70a48f58505953351092c37f903bcf78b0aff05b1d544609fb8902,
        "",
      ],
      [
        8cbef1db2d70a48f58505953351092c37f903bcf78b0aff05b1d544609fb8903,
        "",
      ],
      [
        8cbef1db2d70a48f58505953351092c37f903bcf78b0aff05b1d544609fb8904,
        "",
      ],
      [
        9f2a59ea98d7cb610eff49447571e1610188937ce9266c6b4ded1b6ee37ecd00,
        0000000000000000000000000000000000000000000000000000000000000000,
      ],
      [
        9f2a59ea98d7cb610eff49447571e1610188937ce9266c6b4ded1b6ee37ecd01,
        e06a0ec345e83437e43b00000000000000000000000000000000000000000000,
      ],
      [
        9f2a59ea98d7cb610eff49447571e1610188937ce9266c6b4ded1b6ee37ecd02,
        0200000000000000000000000000000000000000000000000000000000000000,
      ],
      [
        9f2a59ea98d7cb610eff49447571e1610188937ce9266c6b4ded1b6ee37ecd03,
        c5d2460186f7233c927e7db2dcc703c0e500b653ca82273b7bfad8045d85a470,
      ],
      [
        f1dd4962bea2321c889520858b4784bcf8f21181312ce2d7396244bd493e1d00,
        "",
      ],
      [
        f1dd4962bea2321c889520858b4784bcf8f21181312ce2d7396244bd493e1d01,
        "",
      ],
      [
        f1dd4962bea2321c889520858b4784bcf8f21181312ce2d7396244bd493e1d02,
        "",
      ],
      [
        f1dd4962bea2321c889520858b4784bcf8f21181312ce2d7396244bd493e1d03,
        "",
      ],
      [
        f1dd4962bea2321c889520858b4784bcf8f21181312ce2d7396244bd493e1d04,
        "",
      ],
    ],

It's quite hard to see why the verification in rust-verkle fails, would you have some insights on how to figure out if this is indeed the problem?

Currently there are numerous unsafe unwrap()s everywhere, the the codebase should be refactor like such

• - Remove all assert conditions, and instead return a result with an error matching what condition check failed. (So debugging isn't 64 != 39, but invalid root length, etc)
• - Use contextual specific errors for internal functions (ie SerializationError on internal conversion functions) then on public facing errors wrap them in VerkleError with an all encompassing error. (This saves on passing an enum on every return in memory, and only for public functions where it is needed)
• - Refactor use of .try_into().unwrap() with either safety comments, or more robust conversions
• - Handle Option returns to either return a Result or safety comments if assumptions can be made

Requires a modification to insert, such that we return all of the update delta values which we then use to update the commitment

Add a transcript struct that just hashes the inputs, to match the python implementation.

It seems we can solely rely on std::io allowing us to remove the ark_std: dependency.

Currently, KZG10 is generic, however, Verkle Trie is only planned to work with Bls12-381

Since the commitment scheme is homomorphic, we never need to recompute it

Given a proof, it is possible to update the commitment using only the components in the proof

Since it's quite trivial to compute group_to_field(commitment), we may not need to save it in the database

This can be useful when the tree is partially filled, and one wants to add a batch of keys to the trie

The VerkleCommitment struct can be replaced with Option<kzg10::Commitment> .

This will avoid the usage of into_repr and as_repr

Currently the implementation, always uses multi scalar multiplication. This is inefficient if for example, we are just adding a single leaf

Continuing my investigation for #48 I have found another error in the verification.

The panic message is no entry found for key and the location is, /rust-verkle/verkle-trie/src/proof/verifier.rs:207:37

Proof:

0300000072d308302f37eb372570431b23ca3fc7cc855b4325f4cb4f2af79cefe34bf7a2323ef19c876997a1e14eddc3f2a2ef55b1ee9252ab06471b78c4930e1f90a7bd4c3f752d09cb3d5b1f7dc594a260c03a7982c54639f0b68d50686b62d50b00000012101110101209091010080f000000f65af722896840956ce43fde8e34972fb03633ec035cd1785ff8f263a8e3e4c7b4d9bcf98de967e9031e91d35c3a7133904e145f69e5c130b3ccbeac20710c0483c83d96e13589c180778a5d237d5b4fa751704c49908dfaca4bfb0bcf155f9d1f28bf2b14ed760e9542bbc9f1cd31157413ae4ba9da60f54baff4e7069707055d82f34c08f53cddf80799918d793ac9787f99ac7df626456a74795986d45eee2229c094f5ec8bb95fbb03c2d5359c6ef3f5c8f0c9691a8f0b2b248aef93f1471acc106b9b4e67fb5eae8dc3e6ac6d5b13afab843549f276b2c4b728dfdcc397612ec643f355305380356fa0ae20137d8eb85c97422091771bfa1e7fa911a70dd171aeccced3f46425cc2f0741016e296141911b95448be1b7023df32ddf5b0231fe60c2577efe3fcf50b9a4d488e6330054864ef8e0e9052cb772855f37f290c43fd9f13ab6f97bd39ad3f69f8d3b5ea1e1686f6cbc33879730fbc4464e22c75d204ae30b7ba8f12f36547996d96edab72cf8b54b5b253570217bc22cd57416f9bcfb9ff60dedcc6fcc3518e5e29e53799a4e154df515f500d171b5bbcf35ce9c286fb81eab42daee4f9f9931e59aceedc17bbb6adca9902b3bdffc035f2006f9328970202baa809217b11253117d2ea01f54d11a1bac77fe6b949dadc75ac33c2dbe870f9fd13f1fdeccde90272c8a5862dde7cb52d98c0fb10356c1b576e50e17849254dc3a36327d826d0b4dc158566dc8d0a30b0de6708c00e5cb99a461a7f9c2cd084d48f489ff46873ef6f22a734e0cc9d2dbb2dcd6468eef0ed51c8efa98de6b6da0c3a3281f2a4a623912654ac9c0899278b07b0f7f91369f214332e7eba73f454471995e422029e2756087873066f6be55eca972d30c19a1dc58afb0437ce2a9d67a2798ff80e2ce2789d422d17487a3458112c5bfe4efdb3afc5534286613676e318dc61624320dbfc952004f4d3e33fb615624f9aca016f51cbe93e4c12425fdbca48513d88220394531d0b5143eebdca4cb80ab94e4ca16c9d72035c0657cc3d931329a5cd4786ba65073e4d51b78e3998fc9c1b5e13aa510b020286dd29e929324def28a9f717c28a2672db392903975c0fef84c4159f599612075550baa680cebabe99d85d676d43227c78d6e2eb841f3d4abe2035ed6a0eb199a11f5b7718258224c69f5425cd7a0e66445d747c2cd95562b2a3de540a14b491a61eef00322fe7f025afa0c19947297f0b5cd9335381dd7aad0f0765ccbf13acd54cc964ba72439e779b9b357fcbca81f3ad6cce8dfd46ebc68e5228f15727f14b67fa34ca2d525b5b30325ef94258e2e9629685c0bab3259773fb4d5c101435d562173d153506461d72f0c94146030935deb07e89d27392a79cfc2cc36aec336788489a3b6e85689f3b4c55ba2efb980b04a56f363afa559c99a621ebdca3fb5887bb31363989b38086c294be492d524d09dba56a294e8846106ba81f011

(key, value) set:

The proof doesn't seem able to reconstruct the tree, it fails when trying to read path [105, 89] from commitments_by_path. The address is absent, and the depth is 2 in the proof.

The explorer is able to rebuild that tree (albeit also with a bug, subtree 0xe4... is missing from the picture), the corresponding key is absent and the depth is 2.

This will remove the need for IFFT in verkle trie.

To do this, we initially need two functions:

• a function to divide two polynomials in lagrange basis, given that one of the evaluation points is zero.
• a function to divide two polynomials in lagrange basis, where no points are zero
• a function to evaluate a polynomial in lagrange basis, where the point is not in the domain.

database.rs

• - We need a populate cache method that populates the cache from storage
• - Think of a better name than ReadOnlyHigherDb, WriteOnlyHigherDb
• - We could auto implement update methods which assert that there was a previous value they would just wrap the insert methods and check for None
• - We can probably combine add_stem_as_branch_child and insert_branch into a single method called insert_branch_child
• - Maybe we can return BranchChild, as the previous data could have been a stem or branch_meta then we can leave it upto the caller on how to deal with it on insert_branch
• - Add a range query for the default database in verkle_db (x2)

default.rs

• - Branch length check assumes that the cache is populated on startup from disk
• - Get Stem Children from cache assumes that the cache is populated on startup from disk

memory_db

• - Maybe change branch_table to use BChild and also include the index in the key (Vec, u8)
• - get_branch_child could return the BranchChild instead and leave the caller to do what they want with it

meta.rs

• - Explain where length check number comes from
• - Use more accurate error msg when length check fails

Currently, the code is at best research quality.

Once the IPA has been merged, all of it will be refactored to be specific for the ETH use case. If possible without much performance degradation, over a generic width, however these are quite troublesome and a fixed width leads to cleaner code.

        BranchMeta {
            commitment,
            hash_commitment,
        }

Why do we need store hash_commitment, can compressed commitment (also 32 bytes) instead of it?

Failing code

   use tempfile::tempdir;
        let temp_dir = tempdir().unwrap();

        let db = MemoryDb::new();
        let mut trie = Trie::new(TestConfig::new(db));

        let tree_key_version: [u8; 32] = [
            121, 85, 7, 198, 131, 230, 143, 90, 165, 129, 173, 81, 186, 89, 19, 191, 13, 107, 197,
            120, 243, 229, 224, 183, 72, 25, 6, 8, 210, 159, 31, 0,
        ];

        let tree_key_nonce: [u8; 32] = [
            121, 85, 7, 198, 131, 230, 143, 90, 165, 129, 173, 81, 186, 89, 19, 191, 13, 107, 197,
            120, 243, 229, 224, 183, 72, 25, 6, 8, 210, 159, 31, 2,
        ];

        let value_0: [u8; 32] = [
            0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
            0, 0, 0,
        ];

        trie.insert_single(tree_key_version, value_0);
        trie.insert_single(tree_key_nonce, value_0);
        let val = trie.get(tree_key_nonce).unwrap();

The problem of finding tree_key_nonce, only happens when tree_key_version is in the same stem and is also set to 0. ie if we change tree_key_version to be in another stem, tree_key_nonce can be found

https://eprint.iacr.org/2022/864.pdf

lib.rs

• - Specify the parts in the code where we assume VERKLE_WIDTH = 256
• - Expose type markers in verkle-trie, so we can ensure that none of them overlap
• - With the type markers in this crate. In particular, this crate uses 2, while verkle-trie uses 1 and 0

utils.rs

• - This pads to 255*16, but we really only need it to be aligned

When using KZG, we referred to the reference string as a SRS, but now that we are using IPA, it should be renamed to CRS

Commits are contained in the branch aggregate_all_witness

A test run of the latest branch has produced the following proof:

00000000020000000a0a05000000538d6beebb32756be0b22c51ae45a9ed96f0299a2fd01f9ef6887946c26fffd25ce9c5ef692a444a65a087ce1cb90a59599bd1249c1fd787eb56787949a8f1e837a9007019834bef8e2c9e2ae868e33370b5a6723bab8a571d0da40046da856f6b90812ef10962d894a520a03813da890084b726e7396b585645ace2dc6c00d75a1fe87728a033caa1184f1349f1a1e70909d9eef21f470499f62f0ebbe387972ff2c70d324e1038f8f1114cd16839c864b8275f684839b90505f04c47875e030998a9eb0ca0419b918a372deeb31fc2b516319705527008f42a81afa21779f2047f388b70fd0e7573b092733bb9a438bb7636bbebcc8036fc2adba4c64acee25da097d74048cb5640f890a6d7ab823af9f181aef6a2d2d08811f1cec209ca4908dcaccb7b429eaf65ae41b5bb949adf038faa2fd391de7afdef986a9d13f5b21f0d2b472cdd14e0e082b5fc413ec6d9e857146883e98f2606641b78788f295424426a6813f6ceec2d741104aca5955005bab47948001fba26d6eee66cb1c89235dd504e3a8b70e21732657fee59aa7a239c25723e5732f44e779a244c2264d973559b6472a4adb461d476636561bd7a40e922af445ae800736f150f21b4adae627a0149d2740f318f2fa8ff7cba047d21fb5c486d9d03b4d55b5439de1f0f3821597a0b55df1848c880a001b66628ad11dd21ac30dfb9d6f6b112dd796415a307ce68a2f3adb6dc7c61c98febe9ad213d4d5dfe3ae52ac11d854cd055d992be0ba6e062e08d0e7e0b2fa59a2e28d504479585ca344be49e250526abf9827e3a5a97ca0df8efaf267e67aff682dfd47bf799d40e560dc7b3c6f2bd9548c5403e26cfb99557eb2ed741d8615db898cdf0a35216caf9dc4e789721bff60b43f13a4a2a26d7fb26f384f405818b37d0cbc7b7a6bec97d5a5be7442edb688af42fe30487bc85e5236f4deff81ada20bc092ae5ddf555691e17742989d34ad0939c29237bc3e6e5d95cdd5b38d1fe6c2758a799cd683ce3756107e3569788d695a909,

Using my rust-verkle-based tool to verify it produces the following output.

> cargo run -- -f ~/Downloads/block_14.rlp -p 36ec9df63867e06e1f46b0b4e28c8f7aeca256c53ab3bda067da9c02db388f15
    Finished dev [unoptimized + debuginfo] target(s) in 0.06s
     Running `target/debug/verkle-block-sample -f /home/gballet/Downloads/block_14.rlp -p 36ec9df63867e06e1f46b0b4e28c8f7aeca256c53ab3bda067da9c02db388f15`
de-serialized block:
- parent hash: 95b6b3cd2c6a21ac21cc031bde5ff2a56d0471b773cf3044442d88195a8e0bc6
- storage root: 620e50aebae6d4f4aaf81fafe056a2ceb74a6472aba092dea16c182d8fac3fcc
- block number: 0e
- key, value list:
	744f493648c83c5ede1726a0cfbe36d3830fd5b64a820b79ca77fe1593352600 => 0000000000000000000000000000000000000000000000000000000000000000
	744f493648c83c5ede1726a0cfbe36d3830fd5b64a820b79ca77fe1593352601 => 0000000000000000000000000000000000000000000000000000000000000000
	744f493648c83c5ede1726a0cfbe36d3830fd5b64a820b79ca77fe1593352602 => 0100000000000000000000000000000000000000000000000000000000000000
	744f493648c83c5ede1726a0cfbe36d3830fd5b64a820b79ca77fe1593352603 => f8811a5ee0d54eca4880eaee7b102eae4b3963ff343f50a024c0fd3d367cb8cc
	744f493648c83c5ede1726a0cfbe36d3830fd5b64a820b79ca77fe1593352604 => 5001000000000000000000000000000000000000000000000000000000000000
	744f493648c83c5ede1726a0cfbe36d3830fd5b64a820b79ca77fe1593352640 is absent
	744f493648c83c5ede1726a0cfbe36d3830fd5b64a820b79ca77fe1593352680 => 00608060405234801561001057600080fd5b50600436106100365760003560e0
	744f493648c83c5ede1726a0cfbe36d3830fd5b64a820b79ca77fe1593352681 => 001c80632e64cec11461003b5780636057361d14610059575b600080fd5b6100
	744f493648c83c5ede1726a0cfbe36d3830fd5b64a820b79ca77fe1593352682 => 0143610075565b60405161005091906100d9565b60405180910390f35b610073
	744f493648c83c5ede1726a0cfbe36d3830fd5b64a820b79ca77fe1593352683 => 00600480360381019061006e919061009d565b61007e565b005b600080549050
	744f493648c83c5ede1726a0cfbe36d3830fd5b64a820b79ca77fe1593352684 => 0090565b8060008190555050565b60008135905061009781610103565b929150
	744f493648c83c5ede1726a0cfbe36d3830fd5b64a820b79ca77fe1593352685 => 0050565b6000602082840312156100b3576100b26100fe565b5b60006100c184
	744f493648c83c5ede1726a0cfbe36d3830fd5b64a820b79ca77fe1593352686 => 00828501610088565b91505092915050565b6100d3816100f4565b8252505056
	744f493648c83c5ede1726a0cfbe36d3830fd5b64a820b79ca77fe1593352687 => 005b60006020820190506100ee60008301846100ca565b92915050565b600081
	744f493648c83c5ede1726a0cfbe36d3830fd5b64a820b79ca77fe1593352688 => 009050919050565b600080fd5b61010c816100f4565b811461011757600080fd
	744f493648c83c5ede1726a0cfbe36d3830fd5b64a820b79ca77fe1593352689 => 005b5056fea2646970667358221220404e37f487a89a932dca5e77faaf6ca2de
	9661ae0db10ecdb9bea3ef0c5fb46bb233cb6ed7404b77e7b0732512ecc60100 => 0000000000000000000000000000000000000000000000000000000000000000
	9661ae0db10ecdb9bea3ef0c5fb46bb233cb6ed7404b77e7b0732512ecc60101 => 2ce182cf5cb40833cb3f00000000000000000000000000000000000000000000
	9661ae0db10ecdb9bea3ef0c5fb46bb233cb6ed7404b77e7b0732512ecc60102 => 0200000000000000000000000000000000000000000000000000000000000000
	9661ae0db10ecdb9bea3ef0c5fb46bb233cb6ed7404b77e7b0732512ecc60103 => c5d2460186f7233c927e7db2dcc703c0e500b653ca82273b7bfad8045d85a470
	9661ae0db10ecdb9bea3ef0c5fb46bb233cb6ed7404b77e7b0732512ecc60104 => 0000000000000000000000000000000000000000000000000000000000000000
proof=Verkle proof:
 * verification hints: 1 1 Present Present 
 * commitments: 538d6beebb32756be0b22c51ae45a9ed96f0299a2fd01f9ef6887946c26fffd2 5ce9c5ef692a444a65a087ce1cb90a59599bd1249c1fd787eb56787949a8f1e8 37a9007019834bef8e2c9e2ae868e33370b5a6723bab8a571d0da40046da856f 6b90812ef10962d894a520a03813da890084b726e7396b585645ace2dc6c00d7 5a1fe87728a033caa1184f1349f1a1e70909d9eef21f470499f62f0ebbe38797 
thread 'main' panicked at 'the proof didn't check', src/main.rs:91:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

i.e. the proof didn't verify. I now have the same tool written in Go, and the proof checks.

The real bug is that rust fails to verify where even go manages to do it. Unacceptable 😠

Picture of the reconstructed tree:

To be fair, for the go code to verify this block, I had to fix an issue, maybe rust-verkle has the same? The issue was caused by GetProofItems returning 0 as a yi instead of ignoring it because it was not present.

When serialising a variable length of depths and extension status, we do not need to encode the number of depths/extension status, because this matches the number of stems.We could therefore ask the user to provide the number of stems.

This however, means we will need a method which iterates over all of the keys and figures out how many stems we have. This is a linear time operation and also makes for a less than desirable UX. Instead, we can encode the length using a u32 which takes up an extra 4 bytes in the proof, a u16 may also possibly be sufficient, but has not been confirmed, so will stick with a u32.

Modify KZG to use Precomputed points when committing inside of open_multipoint

Update to #49 and #48.

Proof:

00000000040000000a0a08080400000064dfc427f68e431244fae8433636dac858e017b777fa580123e17e412d52280671dbf2a761349bbb017ad625d1eef5c204109ab5bcf55054c10636d8f32fab6b087ad43bd1039a39ec46c5b66d8938f068f063e2ec82202d856eeb4a9d48771445d081501b91696e187c70d693ff3caddbe629ec33a134cc66a0c46c95c1c39510349e03b2caa23e82f0d5e0fedf963c223f1093ee7542a04b7f3373496b47f74b4cbaf2d2f1bafc25dbcd20f53d777e587bc1e557ba5d452a207b7617c112e84da7165ccb186272b071d4ea63373b0a8749f55ec68991b180395ed32c0102172c22106bb370a7def8822347e6872b37c564e663a74fc75db47ed9bb46e5d25a6f7ed2fb65c120e31d115f72872e96bcfeddf4688c6e7db61471f87e55ef871d183a23897e9d5b2791c639f5b88062a9b0951a9177ae0e6e44ca83d23b423e6432e3cdf6bef6795b85c43c3dec87122831fdb4e6d49873248771b956c7282be51889380fb7d0a386720a4cc6584e9e7811aef735a599d0631d2617db8b77e5d41d1e0b30ee956e299f9f1a9687c601dc52308270f7b152eede120e69fb3f40dc4f0d8d7439c3b115ba96921f7e30726727ba5ee320d659cc9cca33d7b2ca5c3b6d17d954fe88fd482410f947afef80453332d2c5bce5d3c533b270317c5c53ad54a7fa2c3c9611bccf44aa48e47affee6e4b480ed271fbd0738bee1827bfcac23a92a812bd70a427a701225dca3c150578f00151dd6cc45d50109b05b418bab7707c3c61b441671cf4d59b2e9cf3bc93d0f5c2a7b54940fd105b5df0ace79eeb3dde9a3946fe54872e13c3a59ef44b2ad672965dfff30f630408de78f4c3399d528361de40f1c2369ccad78fb571dc6ef504994e721efcf632fe316ff1e271c363a3f3b127196af15c50d5c6612ab1bfc82aae15a767f4b2d7903341b762428d9f5b17569c6997eb137bf1b4674772e9b76011c69db468e69c993c70c3afef02

Keys:

    [
      [
        5f7452bfe7d9397974506432eaac93dc856e39cb375ebc3ba355fd3590b31a00,
        0000000000000000000000000000000000000000000000000000000000000000,
      ],
      [
        5f7452bfe7d9397974506432eaac93dc856e39cb375ebc3ba355fd3590b31a01,
        41831c33dafdb76b050000000000000000000000000000000000000000000000,
      ],
      [
        5f7452bfe7d9397974506432eaac93dc856e39cb375ebc3ba355fd3590b31a02,
        4300000000000000000000000000000000000000000000000000000000000000,
      ],
      [
        5f7452bfe7d9397974506432eaac93dc856e39cb375ebc3ba355fd3590b31a03,
        c5d2460186f7233c927e7db2dcc703c0e500b653ca82273b7bfad8045d85a470,
      ],
      [
        5f7452bfe7d9397974506432eaac93dc856e39cb375ebc3ba355fd3590b31a04,
        0000000000000000000000000000000000000000000000000000000000000000,
      ],
      [
        9661ae0db10ecdb9bea3ef0c5fb46bb233cb6ed7404b77e7b0732512ecc60100,
        0000000000000000000000000000000000000000000000000000000000000000,
      ],
      [
        9661ae0db10ecdb9bea3ef0c5fb46bb233cb6ed7404b77e7b0732512ecc60101,
        4a6126f36ac715120f4000000000000000000000000000000000000000000000,
      ],
      [
        9661ae0db10ecdb9bea3ef0c5fb46bb233cb6ed7404b77e7b0732512ecc60102,
        0100000000000000000000000000000000000000000000000000000000000000,
      ],
      [
        9661ae0db10ecdb9bea3ef0c5fb46bb233cb6ed7404b77e7b0732512ecc60103,
        c5d2460186f7233c927e7db2dcc703c0e500b653ca82273b7bfad8045d85a470,
      ],
      [
        ee390d5f79a75e40f22942a639ffa7b982e3de2b3e8927919210cf252355df00,
        "",
      ],
      [
        ee390d5f79a75e40f22942a639ffa7b982e3de2b3e8927919210cf252355df01,
        "",
      ],
      [
        ee390d5f79a75e40f22942a639ffa7b982e3de2b3e8927919210cf252355df02,
        "",
      ],
      [
        ee390d5f79a75e40f22942a639ffa7b982e3de2b3e8927919210cf252355df03,
        "",
      ],
      [
        ee390d5f79a75e40f22942a639ffa7b982e3de2b3e8927919210cf252355df04,
        "",
      ],
      [
        ee9707ef2846473a6fe4e10781a3b8b225ea950a7c89fb88f01c3544a41dfb00,
        "",
      ],
      [
        ee9707ef2846473a6fe4e10781a3b8b225ea950a7c89fb88f01c3544a41dfb01,
        "",
      ],
      [
        ee9707ef2846473a6fe4e10781a3b8b225ea950a7c89fb88f01c3544a41dfb02,
        "",
      ],
      [
        ee9707ef2846473a6fe4e10781a3b8b225ea950a7c89fb88f01c3544a41dfb03,
        "",
      ],
      [
        ee9707ef2846473a6fe4e10781a3b8b225ea950a7c89fb88f01c3544a41dfb04,
        "",
      ],
      [
        ef866912039f3bf5768ce2af85a24615841fee4139b76bd83bc3d2dc0bc70800,
        "",
      ],
      [
        ef866912039f3bf5768ce2af85a24615841fee4139b76bd83bc3d2dc0bc70801,
        "",
      ],
      [
        ef866912039f3bf5768ce2af85a24615841fee4139b76bd83bc3d2dc0bc70802,
        "",
      ],
      [
        ef866912039f3bf5768ce2af85a24615841fee4139b76bd83bc3d2dc0bc70803,
        "",
      ],
      [
        ef866912039f3bf5768ce2af85a24615841fee4139b76bd83bc3d2dc0bc70804,
        "",
      ],
    ],

The root has no 0xee child, and this block contains two stems whose absence is proven by a None extension status: ee390d5f79a75e40f22942a639ffa7b982e3de2b3e8927919210cf252355df and ee9707ef2846473a6fe4e10781a3b8b225ea950a7c89fb88f01c3544a41dfb.

Top Level for the Verkle Trie Crate

commiter.rs

• - We can make this &[Fr;256] since we have committed to 256, this would force the caller

config.rs

• - Refactor Functions to return VerkleError and consider API

from_to_bytes.rs

• - Refactor to_bytes conversions to use fixed-length types

errors.rs

• - Split out Error enum to contained grouped specific errors

lib.rs

• - Return Result when creating a Proof
• - Group to Field proof document
• - Possible optimisation. This means we never allocate for paths
• - Find out if get is ever needed

Inner nodes can have variable lengths up to 31 bytes. Currently, we are using Vec for this, however we could use SmallVec which would not create a heap allocation until the size goes over 31 bytes.

The downside is that with enough SmallVec's, we may run into stack overflow issues. Also note that accessing the stack will not always be faster than accessing the heap, see https://stackoverflow.com/a/24057744 . Hence, we should benchmark the difference in pathological and non-pathological cases.

What is the purpose or limitation for value is only 32 bytes?

   pub type Value = [u8; 32];

    /// Inserts a single value
    /// This method will implicitly compute the new root
    fn insert_single(&mut self, key: Key, value: Value) {
        self.insert(vec![(key, value)].into_iter())
    }

The verkle-trie module currently cannot be compiled to WASM because it depends on the rocksdb bindings which are from C (if I'm reading the logs correctly). I'm not sure if there's any direct way around this unless there is a pure Rust version of rocksdb that could be compiled to WASM as well. Would it be possible to separate out just the verkle verifier code into a standalone module that isn't dependent on rocksdb so it could be used in the browser?

By adding the below to dependencies

[dependencies]
...
wasm-bindgen = "0.2"
getrandom = { version = "0.2", features = ["js"] }

and then

[lib]
crate-type = ["cdylib", "rlib"]

I was able to get to the point of the rocksdb failure that generates the below error by running wasm-pack build --target web in the root of the verkle-trie package.

error: failed to run custom build command for `librocksdb-sys v6.20.3`

Caused by:
  process didn't exit successfully: `/home/jim/development/rust-verkle/target/release/build/librocksdb-sys-80b89031e3c367eb/build-script-build` (exit status: 101)
  --- stderr
  rocksdb/include/rocksdb/c.h:65:10: fatal error: 'stdarg.h' file not found
  rocksdb/include/rocksdb/c.h:65:10: fatal error: 'stdarg.h' file not found, err: true
  thread 'main' panicked at 'unable to generate rocksdb bindings: ()', /home/jim/.cargo/registry/src/github.com-1ecc6299db9ec823/librocksdb-sys-6.20.3/build.rs:44:10
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace