paritytech / grandpa-bridge-gadget

A Bridge Gadget to Grandpa Finality.

Note that the BEEFY core components have been move to the Substrate repository and this repo has been marked as read-only.

BEEFY

BEEFY (Bridge Efficiency Enabling Finality Yielder) is a secondary protocol running along GRANDPA Finality to support efficient bridging with non-Substrate blockchains, currently mainly ETH mainnet.

It can be thought of as an (optional) Bridge-specific Gadget to the GRANDPA Finality protocol. The Protocol piggybacks on many assumptions provided by GRANDPA, and is required to be built on top of it to work correctly.

🚧 BEEFY is currently under construction - a hardhat is recommended beyond this point 🚧

Contents

• Build
• Documentation
• Project Layout
• BEEFY Key
• Running BEEFY

Build

To get up and running you need both stable and nightly Rust. Rust nightly is used to build the Web Assembly (WASM) runtime for the node. You can configure the WASM support as so:

rustup install nightly
rustup target add wasm32-unknown-unknown --toolchain nightly

Once this is configured you can build and test the repo as follows:

git clone https://https://github.com/paritytech/grandpa-bridge-gadget.git
cd grandpa-bridge-gadget
cargo build --all
cargo test --all

If you need more information about setting up your development environment Substrate's Getting Started page is a good resource.

Documentation

The best way to get going with BEEFY is by reading the walkthrough document! This document puts BEEFY into context and provides motivation for why this project has been started. In addition to that the current status as well as a preliminary roadmap is presented.

BEEFY brainstorming is a collection of early notes and ideas, still worth checking out.

Project Layout

This project is an auxiliary repository for BEEFY utility, testing and example code. The BEEFY core components have been added to Substrate.

This repository is also used to keep track of BEEFY related issues, i.e. even issues regarding BEEFY core functionality, which is now part of Substrate, should be filed against this repository and not Substrate itself.

├── beefy-cli         // BEEFY utilities and testing aids
│  └── ...
├── beefy-node        // A Substrate node running the BEEFY gadget
│  └──  ...
├── docs              // Documentation
│  └──  ...

BEEFY Key

The current cryptographic scheme used by BEEFY is ecdsa. This is different from other schemes like sr25519 and ed25519 which are commonly used in Substrate configurations for other pallets (BABE, GRANDPA, AuRa, etc). The most noticeable difference is that an ecdsa public key is 33 bytes long, instead of 32 bytes for a sr25519 based public key. So, a BEEFY key sticks out among the other public keys a bit.

For other crypto (using the default Substrate configuration) the AccountId (32-bytes) matches the PublicKey, but note that it's not the case for BEEFY. As a consequence of this, you can not convert the AccountId raw bytes into a BEEFY PublicKey.

The easiest way to generate or view hex-encoded or SS58-encoded BEEFY Public Key is by using the Subkey tool. Generate a BEEFY key using the following command

subkey generate --scheme ecdsa

The output will look something like

Secret phrase `sunset anxiety liberty mention dwarf actress advice stove peasant olive kite rebuild` is account:
  Secret seed:       0x9f844e21444683c8fcf558c4c11231a14ed9dea6f09a8cc505604368ef204a61
  Public key (hex):  0x02d69740c3bbfbdbb365886c8270c4aafd17cbffb2e04ecef581e6dced5aded2cd
  Public key (SS58): KW7n1vMENCBLQpbT5FWtmYWHNvEyGjSrNL4JE32mDds3xnXTf
  Account ID:        0x295509ae9a9b04ade5f1756b5f58f4161cf57037b4543eac37b3b555644f6aed
  SS58 Address:      5Czu5hudL79ETnQt6GAkVJHGhDQ6Qv3VWq54zN1CPKzKzYGu

In case your BEEFY keys are using the wrong cryptographic scheme, you will see an invalid public key format message at node startup. Basically something like

...
2021-05-28 12:37:51  [Relaychain] Invalid BEEFY PublicKey format!
...

Running BEEFY

Currently the easiest way to run BEEFY is to use a 3-node local testnet using beefy-node. We will call those nodes Alice, Bob and Charlie. Each node will use the built-in development account with the same name, i.e. node Alice will use the Alice development account and so on. Each of the three accounts has been configured as an initial authority at genesis. So, we are using three validators for our testnet.

Alice is our bootnode is is started like so:

$ RUST_LOG=beefy=trace ./target/debug/beefy-node --tmp --alice

Bob is started like so:

RUST_LOG=beefy=trace ./target/debug/beefy-node --tmp --bob

Charlie is started like so:

RUST_LOG=beefy=trace ./target/debug/beefy-node --tmp --charlie

Note that the examples above use an ephemeral DB due to the --tmp CLI option. If you want a persistent DB, use --/tmp/[node-name] instead. Replace node-name with the actual node name (e.g. alice) in order to assure separate dirctories for the DB.

grandpa-bridge-gadget's Issues

BEEFY audit

It's not clear how and when to audit the code, but it has to be done before we deploy to production.

Dependabot can't parse your Cargo.toml

Dependabot couldn't parse the Cargo.toml found at /pallet/Cargo.toml.

The error Dependabot encountered was:

Dependabot::DependencyFileNotParseable

View the update logs.

Expose data-to-sign as Header Digest

BEEFY should support different ways on getting the Commitment payload - one of them might be just reading a particular digest item from the header.
The advantage of this versus querying the state is that the latter might be pruned already if we follow behind GRANDPA (if that's needed it would either require running an archive node or state-pinning implemented).

For Polkadot case we might want to expose the MMR root hash in the header digest.

Register BEEFY key type as a known key type

Currently the BEEFY key type is defined in the beefy-primitives crate only. It has not been registered with the known key type registry in sp_core::crypto::key_types.

While this may not be strictly necessary from a technical point of view, it should still be done for the sake of consistency and completeness.

MMR leaf should contain block number

...to make sure that the validator set proof is for (the) most recent block.

Every time we import BEEFY signed commitment we should be able to receive a MMR proof of the next validator set. However MMR allows us to prove any past block as well as the recent one.
We should make sure that the validator set we are getting is actually the most recent one, that will be valid for the next session id.

Alternatively maybe it's better to include validator set id in both BEEFY signed commitment and the MMR.

Track BEEFY validator set

Get the client code to a point where it signs some arbitrary data.
This most likely includes having the gadget to track BEEFY validator set somehow.

Dependabot can't parse your Cargo.toml

Dependabot couldn't parse the Cargo.toml found at /pallet/Cargo.toml.

The error Dependabot encountered was:

Dependabot::DependencyFileNotParseable

View the update logs.

Integrate BEEFY with Rococo

Alongside with MMR integration

Expose BEEFY prometheus metrics.

After a "production" deployment of BEEFY (for instance on Westend / Rococo) it will be really nice to collect some metrics and get some insight of what happens inside the BEEFY gadget / gossip.
I'd see what metrics we expose for GRANDPA and copy that, but what would be interesting from the top of my head:

1. Current Validator Set Id
2. Round state: starting / concluding
3. Number of valid signed commitments received per round (with required threshold information)
4. Number of incoming / outgoing gossip messages / commitments per round.

Finalisation catch-up

Currently we piggy back on GRANDPA, meaning that if GRANDPA is lagging behind (i.e. there are uncompleted rounds), BEEFY will not make progress as well.

From BEEFY light client perspective it's enough to have at least one round concluded through out the session.
We need to make sure that BEEFY does conclude at least one round per session, because if we dont'BEEFY-only light clients will actually be unable to make progress (Verify authorityh hand off)

Add BEEFY error type

Introduce BEEFY error type for unified error handling.

The BEEFY error type should allow to convert dependency specific errors into BEEFY errors.

Improve block-selection strategy

Currently we have a basic next-power-of-two block selection strategy, but we should rather make sure that every session change (#14) and validator set change (#9) block is always included - i.e. we shouldn't be skipping these.

BEEFY gadget should recognize `BeefyApi` updates

Currently, the BEEFY gadget is using a somewhat crude approach in order to detect the availability of an on-chain BEEFY pallet. One situation which would cause some mayhem, for example, is the removal of the BEEFY pallet as part of a runtime update.

The canonical approach followed by the BEEFY gadget should actually be to follow runtime updates and explicitly check for the availability of the BeefyApi. This would require basically two things:

• catch and process frame_system::pallet::Event::CodeUpdated events
• calling either sp_api::ProvideRuntimeApi::runtime_api()::has_api() or has_api_with() and verify that the new runtime provides the BeefyApi.

Use CryptoStore instead of SyncCryptoStore

beefy-gadget is still using SyncCryptoStore which is deprecated.

sp_keystore::CryptoStore should be used instead. Also the signature of the key store methods has been changed recently Make keystore return None when a key doesn't exist. This change leads to a quite convoluted result handling in the current code which should be improved.

Beefy gadget/client should allow for delayed Beefy pallet initialization

From #95

The assumption here is that the beefy pallet is part of the runtime (pretty much since genesis), however existing changes will most likely upgrade to beefy pallet, so I think we should be handling this error a bit more gracefuly.

What I mean is that BEEFY will be introduced as a runtime upgrade to chains like Kusama, Polkadot and maybe even Rococo.
This means that the node (and the gadget code) needs to handle a case where the pallet is not present gracefuly. The client will be upgraded first and after that will the runtime get the pallet. This means that the gadget needs to be able to enable itself when it detect that the pallet becomes active in the runtime.

Integrate BEEFY & MMR with Westend

Make sure there is a Runtime API or RPC to generate MMR proofs

Parachain headers MUST NOT contain validator set changes in the digest

Currently Substrate spits out the full list of babe validators in the header digest. For parachains and in the bridges context it will get prohibitively expensive to proof anything about such headers, so we should make sure that cumulus does not do that.

CC @andresilva @bkchr

Add validator_set() runtime API

Currently the BeefyAPI has an authorities() function which will return the current BEEFY validator set. In order to address #95, the idea is to add a validator_set_id() function. Both of those functions together basically return the fields of a beefy_primitives::ValidatorSet.

Instead of having two functions, a single validator_set() runtime API function should be exposed, which returns a ValidatorSet right away. This would also align with the BEEFY AuthoritiesChange digest, which contains a ValidatorSet as well.

Initialize BeefyWorker validator set id for out-of-sync BEEFY gadget/client

From #95

So this obviously works for now, but what if the client is not fully in sync with the network? I think we need a bit more sophisticated mechanism, where we actually update the validator set id when new blocks are imported / we get synced up.

Currently the BeefyWorker validator set id is always initialized by using BeefyApi::validator_set(). This assumes the client is synced with the network and that the beefy_pallet session handler has seen all relevant on_new_sesseion() calls.

As per the above comment, this approach is not sufficient for an out-of-sync client.

Improve BEEFY rounds lifecycle.

Currently BEEFY is just starting a new round according to the voting rule (next-power-of-two), regardless of how many previous rounds are still running. We should have a more defined round lifecycle with clear rules for when we start a new round (rather than running all in parallel wildly), and also what happens to old rounds in the face of new rounds being concluded.

Also see the related issue Improve block-selection strategy

BEEFY sync verification

During the sync process we should verify incoming BEEFY justifications that we get from the peer.
We need to plug into block import pipeline, similar to how GRANDPA is doing that.

GRANDPA keys change every session, but BEEFY keys shouldn't

Currently session is 4h on polkadot, and that's the cadence of GRANDPA keys changing.
The validator set however only changes every era (epoch), which is 4 sessions (around 16h).

We should make sure that BEEFY keys stay the same across the entire epoch (i.e. to have one set transition block per epoch, not per era).

CC @andresilva

Introduce BEEFY bootstrap digest

BEEFY pallet should be able to emit a bootstrap/reset digest. This digest would cause clients with a BEEFY gadget to reset their state to State::New. State::New in turn will trigger BEEFY gadget to re-initialize itself.

Allow hasher configuration before signing with app-crypto

Substrate's app-crypto crate allows you to choose from different cryptography and generates application-specific Public, Signature, etc types.

In BEEFY we currently use ECDSA (over secp256k1) to make it the most compatible with Ethereum.

However, where we sign the Commitment (or rather the encoding of it) using SyncCryptoStore::sign_with here, it would eventually end up here.

This means that whatever message we are signing (in our case it's the Commitment) is going to be unconditionally blake2_256 hashed before being signed.

Ideally for maximum Ethereum compatibility we would prefer this hash to be keccak256 instead.

A solution is to either make the hasher somehow configurable in Substrate/CryptoStore or fall back to some other mechanism that will allow us to configure how the message is hashed before signing.

Merkle root of signatures is not needed in the interactive protocol

It seems that providing a merkle root of signatures in the interactive protocol doesn't really add any value, Alistair proposed a different scheme where we just provide one signature + public key with the commitment, details to follow.

Validate signatures for votes received from peers

BeefyWorker::handle_vote() should validate vote signatures before processing them.

Change `is_set_transition_block` to an enum

Initially we're going to implement BEEFY key's changes per every session (see #9), so we need a differentiator between set change (bumps validator_set_id and contains new keys) and key change (the commitment contains new keys in some form).

We should change the field to be an enum, which is is also going to be more future proof.

Track current and next authorities in BEEFY pallet

We will be announcing the upcoming authority set in advance, therefore we should update the pallet to also track what the next (already announced) authorities will be.

Additionally, we should store the authorities under a known storage key. This will make it easier to fetch storage proofs in the future since there is currently no easy way to expose the storage key of regular substrate storage fields (i.e. declared with decl_storage).

Generate Merkle Proof for Parachain Heads

We currently compute Merkle Root for Paraheads using keccak256, but there is no easy way to generate proofs (that will be needed for BEEFY light client to verify anything on the parachain).

Use bitfield in `SignedCommitment` for encoded-size optimisation.

Currently SignedCommitment uses Vec<Option<Signature>>> for the list of signatures. This vector is expected to contain at least 2/3+1 Some values anyway.

It's possible to optimize the storage though, and save 7 bits (1byte for enum variant vs 1 bit) per every signature in SCALE encoded form (minus alignment) by using a different scheme of:

signatures_from: BitField,
signatures: Vec<Signature>,

Every set bit in signatures_from signifies the index of validator that the next signature in signatures vec is for.

I think this should be implemented as custom SCALE encoding and we should keep the current struct format for backward compatibility (code not over-the-wire).

Dependabot can't parse your Cargo.toml

Dependabot couldn't parse the Cargo.toml found at /pallet/Cargo.toml.

The error Dependabot encountered was:

Dependabot::DependencyFileNotParseable

View the update logs.

Make BEEFY gadget `Client` trait public

Make the BEEFY gadget Client trait public and adjust client code accordingly.

BEEFY sync protocol

Currently the node has to be on-line to receive and store BEEFY justifications. If it falls behind it can rely on syncing the justification data along with the blocks (see #17), but there is no guarantee that we will get all of them (depends on who do we sync from).

Syncing only BEEFY justifications is also useful for Super Light Clients (see paritytech/parity-bridges-common#323 - beyond Solidity) - we could for instance sync Substrate Connect with this (see also paritytech/substrate#1208).

We should probably introduce:

• BEEFY justification gossip
• Sync protocol only for essential BEEFY justifications (session change blocks, era change blocks).

Support multiple crypto at the same time

For now we want secp256k1, but in the near future we might want to collect two signatures of:
secp256k1 and BLS for instance. Later on we might add some ZK stuff.

The protocol should be ready to upgrade to include a new crypto. Gossip messages should contain ALL required signatures.

Different crypto will require different start blocks and also might move at different cadence (for instance ZK stuff, since it's heavy might only be possible once per epoch).

Consolidate BeefyWorker trait bounds

The S and P::Signature trait bounds for BeefyWorker are redundant.

It should be possible to remove the S trait bound.

Merkle trie of BEEFY authority keys should be made of uncompressed public keys

Substrate represents secp256k1 public keys in their compressed format (33 bytes). Reverting the compression requires solving the curve equation (modulo p), which might be prohibetevely expensive on Ethereum.

Instead we could either store ETH addresses in the merkle tree (keccak_256(uncompressed_public)[20..32]) or simply use uncompressed public keys (65 bytes).

Converting to an ETH address should be fairly straight forward to do on the Ethereum side.

BEEFY equivocation processing

BEEFY must deal with equivocations. That includes at least

• detect equivocations
• report equivocations
• initiate slashing

Update README.md

The readme file is a bit out of date, we know have a really good intro that landed in #47 and it might be worth to link to this instead and leave README.md for a more technical documentation of the repository (building, running, organisation, etc).

Add concrete (polkadot) BEEFY signature tests

Example

	#[test]
	fn beefy_test() {
		let signature = hex!(
			"c557781c98f8ba8f8025accf037b83e4861101712e7cf316474f7d2f05f1894f55ea99db6a735d004e226b3aa5d3537b301bfedf1b4a095110eb7afc05b0563700"
			// "bdb33475db0157a6f93d12b72c6ed78e3ec6fd5f09cdf79d01da11603fa594bd73240f9823995c71ee6b62062603cca9fd7184d0c249e821f928db010af671a100"
		).to_vec();
		let signature: BeefySignature = signature.try_into().unwrap();

		let payload = H256::from_slice(&hex!(
			"a148e08efa4936efd7fb46109a917ee24ded43e230625da90a123929e1f9b8a5"
		));
		let block_number: BlockNumber = 1366;
		let validator_set_id = 0;
		let commitment = beefy_primitives::Commitment {
			payload,
			block_number,
			validator_set_id,
		};
		let message = commitment.encode();

		println!("Message: {}", hex::encode(&message));
		println!("Hashed: {}", hex::encode(sp_core::blake2_256(&message)));

		let untyped: <BeefySignature as sp_application_crypto::AppKey>::UntypedGeneric = signature.into();
		let public = untyped.recover(message).unwrap();
		println!("Public: {:?}", public);
		let public = secp256k1::PublicKey::parse_compressed(&public.0).unwrap();
		println!("Uncompressed: {:?}", hex::encode(public.serialize()));

		println!("Keccak256: {:?}", hex::encode(
			sp_core::keccak_256(&public.serialize()[1..])
		));

                // TODO: some assertions
	}

Currently it's hard to get concrete test data for signed commitments that would make it easy to compare with Ethereum implementation.
I imagine this would be added to the light client tests.

BEEFY justifications (SignedCommitments) should be stored in the DB

Storing them as justifications will expose them via getBlock RPC - we return all kinds of justifications we have for the block (BABE, GRANDPA, BEEFY, etc).

Custom RPC for BEEFY Justifications

Similar to GRANDPA RPC, should support:

• onDemand retrieval - get the SignedCommitment from DB (see #17) if there is any
• subscription - receive SignedCommitments as soon as they are available

Make sure the services works with no-validators

It should simply receive gossip messages, but don't do any voting.

BEEFY end-to-end test

Currently we don't really have a proper integration test of BEEFY and the entire node (i.e. we don't know if it works; see #80)

It would be good to run a super high level integration test where:

1. We spawn two nodes (alice & bob)
2. We subscribe to BEEFY justifications
3. We wait for the first few justifications to be received and consider the test finished.

Enact validator set change starting with the next round/block

Right now, if the validator_set_id changes, we use this new set id for the signed commitment of the current round/block already. The new set id should be enacted starting with the next round/block, however.

This is related to Improve BEEFY rounds livecycle

Track best BEEFY finalized block

Currently, BEEFY tracks the best GRANDPA finalized block in best_finalized_block. Instead what we need to track is the best block 'finalized' by BEEFY. That would be the block_number in the most recent BEEFY signed commitment.

• For a BEEFY gadget in State::New, wait for a SignedCommitment notification received through gossip.
• Update best_finalized_block with SignedCommitment.block_number.
• For each SignedCommitment notification received, update best_finalized_block again, if SignedCommitment.block_number is more recent.

Gossip will periodically send out SignedCommitments to allow for initial signed commitment discovery (first item in the list above).

Determine validator_set_id at BeefyWorker startup

BeefyWorker needs to be initialized with the current valid validator_set_id. Right now this id is hard coded to the genesis validator set id '0' (zero).

Follow substrate master

For external teams it would be easier if the code followed substrate master branch to included newest features (like MMR stuff).

Customize payload extraction

Currently we extract the MMR root hash from the header digest, but long term it would be better to read the storage directly.
We need to do it right away, to prevent the state from being pruned by when GRANDPA finalises the block.

MMR non-peak nodes pruning

It's still not implemented, and we need to do it before any production deployment.