t3rn / t3rn

Composable smart contract hosting with fail-safe interoperable execution

Composable smart contract hosting with interoperable, multi-chain execution.

t3rn is a hosting platform for smart contracts, that enables trustless, multi-chain execution and composable collaboration.

t3rn renders smart contracts blockchain agnostic, meaning they can instantly execute on multiple blockchains. The smart contracts can be uploaded as they are and the t3rn protocol will host and execute them across different blockchains, breaking the barrier to serving users across industries and blockchain platforms.

Interoperability plugin - integrate with Gateway

The protocol works well with Parachains regardless of whether they support smart contracts or not. It's designed to be highly compatible with different blockchain architectures and easy integration using one of three kind of Gateway: Intrinsic Programmable, Extrinsic Programmable or Transaction-Only.

t3rn emphasizes the existent decentralised-solutions and allows multiple blockchains to collaborate on the same contracts repository. By re-using the whole decentralised application blocks, fosters building the decentralised solutions with the freedom to operate on multiple chains. t3rn facilitates building interoperable solutions in familiar smart contract languages like Solidity, !ink or directly in Web Assembly. Smart contracts or Modules for Runtime which are hosted on a decentralised execution platform, Circuit and can be executed on multiple integrated blockchains. These smart contracts are shared and are being collaboratively added by community.

Learn more about Gateways and how to integrate with t3rn.

Multi-chain execution platform - execute on Circuit

Gateway Circuit shares the context of the overall Interoperable Transaction and passes it over to the Gateways. The context contains all of the necessary data base on the Parachains can decide whether to not proceed with the execution. Gateway Circuit has an access to all of the ongoing Interoperable Transactions and before the execution is started the circuit checks if there is no two transactions changing the same account at the same time.

Work on the Circuit Prototype is currently in progress.

On-chain contracts repository - share composable contracts.

Each successful compilation of Composable Smart Contracts is immediately available for the network to use. The on-chain contracts hosting can be compared with decentralised package manager created by the community of t3rn developers.

All the newly created code for interoperable programming is automatically shared with other developers to reuse: • projects can easily collaborate by sharing and re-using the business logic • developers can contribute code for free or expect remuneration per usage. This opens up a way for developers of earning money for writing the Open Source code.

Smart contracts can be written in familiar languages like !ink, Solidity, WebAssembly. Existent smart contracts can be uploaded as they are, no rewriting required.

Learn more about writing composable contracts in our SDK.

Motivation

Creating safe solutions operating and synchronizing multiple blockchains comes with new challenges.

Synchronisation

Multiple blockchains means that accounts and the storage allocated by them is located on different chains. Without the overarching synchronisation mechanism there is no guarantee that the state of accounts won't change while the interoperable transaction is executed. t3rn offers the interoperable execution protocol performed by Circuit, which manages the multi-chain transactions.

Non-reversible

Once a transaction is applied on a blockchain it's non-reversible, which constitutes a problem for transactions only considered useful if they succeed on multiple blockchains simultaneously. t3rn introduces multiple phases to execution of interoperable transactions and implements the safety mechanisms which are able to revert the execution on affected chains in case the overall interoperable transaction fails.

Complexity

Designing interoperable solutions is complex and requires developers and users to operate on multiple blockchains, possibly creating multiple different accounts, acquiring multiple currencies, providing a way for different blockchains to communicate. t3rn brings that complexity down and offers services and packages that facilitate interoperable execution securely.

Repository setup

Follow the steps below if you're interested building and running the Circuit on your local machine:

git clone https://github.com/t3rn/t3rn
git checkout origin/development
cargo build --release
cargo run --bin t0rn-collator

Git Config

We have a way of utilizing the build system (cargo) to reason about dependencies. However, this requires some changes to your configs. Ensure you have your SSH key added to github.

~/.gitconfig

[url "[email protected]:"]
  insteadOf = https://github.com/

~/.cargo/config

[net]
git-fetch-with-cli = true

Running benchmarks

1. Build the circuit node with runtime-benchmarks feature:

cargo build --release --features with-standalone-runtime,runtime-benchmarks

2. Run the command to execute the benchmarks for desirable pallet:

./target/release/t0rn-collator benchmark --chain dev --execution wasm --wasm-execution compiled --pallet pallet_you_want --extrinsic '*' --steps 50 --repeat 20 --raw --template=./benchmarking/frame-weight-template.hbs --output .

3. After executing following commands a file called pallet_you_want.rs will be generated. The file contains weights for the desirable pallet, copy that file into the pallets src directory, and rename it to weights.rs.

This concludes the process of generating weights for the pallet.

License

---

Copyright 2020-2023 t3rn Ltd.

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

---

t3rn's Issues

Unit tests remaining functionalities of Circuit

Description

Cover following scenarios with unit tests

Contracts registry [Beqa]

• purge fails (contract missing + non-root user case)
• add_new_contract fails (contract already exists + non-root user case)

Execution delivery

• register gateway fail and success; 4 different tests for each header and block size combination [Maciej]
• submit_step_confirmation fail and success; 4 different tests for each header and block size combination [Maciej]
• dry_run_single_contract; other success paths, other failure paths [Devanshu]
• preload_bunch_of_contracts [Devanshu]
• run_single_contract [Devanshu]
• pre_run_bunch_until_break; depends on above two functions [Devanshu]
• retrieve_gateway_pointer [Beqa]
• retrieve_gateway_protocol [Beqa]
• dry_run_whole_xtx [Beqa]

Fix subxt to work with latest Circuit and SDK

Description

Update the subxt to the latest version.
Try using subxt to submit encoded and signed extrinsic to the Circuit

Refactor 3rd party deps and Volatile VM

Acceptance criteria

• Fork/move frontier and parity-bridges-common from to t3rn GitHub account
• Update the dependencies across the project to reflect the changes above

Demonstrate the composable contracts execution via Gateways

Description

Extend SDK with demonstration showcasing execution of composable contracts with t3rn & tools to build, deploy and execute your own ones.

Acceptance Criteria

The example must be exhaustive.

• builds & deploys several scattered contracts (components) to demo-runtime
• demonstrate that chains without Contracts pallet can also be integrated into the execution circuit
• several composable smart contracts connected within one transaction facilitated via runtime + contracts gateways
  • !ink + WebAssembly text format in a single contract.

Create tools signing & dispatching calls with execution to Gateways

Description

Provide functionality for signing and sending external transactions into Runtime & Contracts Gateways. The transaction orders multi-phase execution of WASM code at Gateway (calls multistep_call).

After successful execution event is emitted on the Gateway which can be watched and retrieved back.

Extend library paritytech/substrate-subxt to be compatible with Gateways.

Acceptance Criteria

• sign multistep_call calls for both Runtime & Contracts Gateways
• dispatch multistep_call calls to Runtime & Contracts Gateways
• watch for multistep_call after-execution events at Runtime & Contracts Gateways

Refactor authority selection

Description

circuit/execution-delivery is currently storing the keypair of validators indirectly using pallet-im-online, however, the current implementation isn't functional and only selects the first authority available

Expected behaviour accepts the escrow account's id (either T::AccountId or T::MultiLocation, see connected issue) and selects connected AuthorityId.

// let submitter = 
	local_keys.binary_search(&escrow_account. special_convert() )
	.ok().map(|location| local_keys[location].clone())
	.ok_or("Can't match")?;

let submitter = local_keys[0].clone();

Acceptance Criteria

• Select AuthorityId connected with an escrow account.
• The selection of T::AuthorityId need to be tested and mocked authorities inserted for the sake of testing.

Upgrade Gateways Pallets to FRAMEv2 - Subtask #31

Description

Upgrade all Gateways-relevant pallets and runtimes to FRAMEv2

Acceptance Criteria

• pallet-runtime-gateway sources
• pallet-runtime-gateway tests
  - [ ] pallet-contracts-gateway sources
  - [ ] pallet-contracts-gateway tests
• demo: pallet-flipper
• demo: pallet-weights
• demo-runtime

Use live networks metadata in RPC Tests

Current RPC tests use a dummy locally stored mock for metadata from XDNS to discover the correct indices for modules and methods called on remote targets. That won't work in production where RPC tests are meant to execute on mutiiple live targets (t3rn devnet, chachacha, westend, kusama etc) and each time the latest metadata of those networks must be collected.
Create a similar test helper function to create_test_stuffed_gateway_protocol - fill_gateway_protocol_with_dev_metadata
that instead of providing mock metadata, fetched the latest metadata from the remote client.

Acceptance Criteria

• create new test util that fetches metadata from latest remote runtimes
• use the remote target's metadata in RPC tests

Make signing compatible with default Substrate networks

Description

Make outbound message signing compatible with default Substrate networks.

Correct or extend the current message_assembly/signer.rs implementation and bring it to the point where messages can be singed on-chain on by the Circuit authorities and successfully validated on the default development Substrate runtime.

Make sure messages wrapping calls are successful against the author_submitExtrinsic jsonrpc on the development node.

In order to do that use the standard UncheckedExtrinsic from sp-core in message_assembly/signer.rs.

Additionally, introduce the support for multi signatures at Circuit - that's what the current message_assembly/signer.rs is compatible with.

Populate that information in Gateway registrar - XDNS - introduce the field has_multisigand choose the signing method multisig vs default sr25519 signature at the point of creating messages.

How to arrive to encoded extrinsic?

1. In substrate_gateway_protocol use produce_signed_payload to produce a signed payload to CircuitOutboundMessage - since refactor at PR #67 called ExtraMessagePayload.
2. For the purpose of this task introduce additional param to produce_signed_paylod with multi signature / single sign - in case of single sign either directly use UncheckedExtrinsic here or extend signer.rs

//ToDo: Accept additional argument to differentiate between mutlisig and single sig produced by submitter / submitters.
pub fn produce_signed_payload(
   &self,
   namespace: &'staticstr,
   name: &'staticstr,
   arguments: Vec<Vec<u8>>,
) -> Result<ExtraMessagePayload, &'staticstr> {
    let call_bytes = compose_call!(self.assembly.metadata,namespace,name,arguments).encode();

//TODO: use a proper nonce
let extrinsic = self
        .assembly
        .assemble_signed_tx_offline(call_bytes.clone(), 0)?;

let signature = extrinsic
        .signature
        .clone()
        .expect("Signature of extrinsic should be valid if assemble_signed_tx was successfull")
        .1;

Ok(ExtraMessagePayload {
      signer: self.assembly.submitter.to_raw_vec(),
			module_name: namespace.encode(),
			method_name: name.encode(),
			call_bytes: call_bytes.clone(),
			signature: signature.encode(),
			extra: GenericExtra::new(Era::Immortal,0).encode(),
			tx_signed: extrinsic.encode(),
			custom_payload: None,
	})
}

Acceptance Criteria

• Extend message_assembly/signer.rs with support to standard UncheckedExtrinsic from sp-core.
• Make sure message_assembly/signer.rs produces compatible single-signed messages against author_submitExtrinsic jsonrpc on the development node
• Implement multi-signatures signing on Circuit and make sure that message_assembly/signer.rs produces compatible messages against author_submitExtrinsic jsonrpc on the development node
• in XDNS introduce the field has_multisigand choose the signing method multisig vs default sr25519 signature at the point of creating messages.

Refactor Gateway project structure

This task is about refactoring the project structure for demo-gateway as per @vedhavyas' draft.

https://hackmd.vedhavyas.dev/s/RVRfRJVoR

gateway/
    frontend/
    demo-node/ "I think we can name the node so that it can deployed along side circuit for our dev. We could have multiple gateway nodes based on their Gateway types"
    pallets/
        contracts-gateway
        runtime-gateway
    escrow-engine/
    rpc/
        src/
        tests/
    tests/ "integration tests"

Acceptance criteria

• Gateway project structure is as above
• Compiles / tests pass successfully

Provide handy API fetch_contracts to access on-chain contracts - Subtask #32

Description

Implement the search for contracts.

As part of the previous work ok #32 there has already been implemented inserting and getting contracts registry entries in the storage already. This task is about fetching them in a handy way using with query parameters.

Additionally, add smart contract metadata and make it browsable.

• Bonus - consider Runtime API implementing text search on code_txt

Acceptance Criteria

• Implement runtime API for on-chain contracts.
• Adds metadata to contracts entry (based on ETH)
• Implement fetch_contracts API entry and provide useful handlers to query data
  • by the author (has an address) → like an address
  • by id
  • by contract info - refcount (vvm tracked) - how many different contracts use it
  • by additional metadata ( tags + name + ... + standards )
• Cover with unit tests using Runtime RPC
• Bonus - fuzzy search by text on code - consider bringing back the regex library since Runtime API can use std lib

Implement transaction-only and track the Circuit chain on Programmable Gateways

Acceptance Criteria

• Implement state tracker at Programmable Gateways following Circuit’s recent state.
• Authorize active-only validators (for now Authorities) to act as an escrow account.
• Implement support for misbehaviour and evidence collection by fisherman.
• Implement a transaction-only external gateway for Substrate-based nodes.
• [Bonus] Implement transaction-only external gateway for Polkadot network
• [Bonus] Implement transaction-only external gateway for BTC network
• [Bonus] Implement transaction-only external gateway for ETH network.
• Update integration tests checking all of the API functionalities of all Gateways.

Implement Proof of Stake Circuit

Description

Implement a Circuit that works on decentralised Proof of Stake premises and demonstrates it on Testnet with the participation of independent community members taking active roles of validators/execution agents.

Acceptance Criteria

• Register validators upon receiving their bonding stake and implement their bonded stake slashing.
• Implement Golden Ticket selection of the next execution agent using BABE algorithm.
• Support on-chain voting for validators and implement it as part of the selection algorithm.
• Implement staking in foreign currencies and seamless exchange between them during interoperable execution.
• [Bonus] Implement necessary features for block collators as preparation for a Parachain slot.

Unit test Circuit inbound protocol with signed payload

Description

Write unit tests for SubstrateInboundProtocol that produce CircuitOutboundMessage as a result of their methods calls (e.g. by a call to substrate_gateway_protocol::transfer
Narrow the methods from inbound protocol to cover 6 following methods:

pub trait GatewayInboundProtocol {

	fn get_storage(
			&self,
			key: Vec<u8>,
			gateway_type: GatewayType
	) -> CircuitOutboundMessage;
	
	fn set_storage(
	    &self,
			key: Vec<u8>,
			value: Option<Vec<u8>>,
			gateway_type: GatewayType,
	) -> CircuitOutboundMessage;
	
	fn call(
	    &self,
			module_name: Vec<u8>,
			fn_name: Vec<u8>,
			data: Vec<u8>,
			escrow_account: Vec<u8>,
			requester: Vec<u8>,
			to: Vec<u8>,
			value: u128,
			gas: u64,
			gateway_type: GatewayType,
			return_value: Option<Vec<u8>>,
	) -> CircuitOutboundMessage;
	
	fn call_escrow(
	    &self,
			module_name: &str,
			fn_name: &str,
			data: Vec<u8>,
			to: Vec<u8>,
			value: u128,
			gas: u64,
			gateway_type: GatewayType,
			return_value: Option<Vec<u8>>,
	) -> CircuitOutboundMessage;
	
	fn custom_call_escrow(
	    &self,
			module_name: &str,
			fn_name: &str,
			data: Vec<u8>,
			to: Vec<u8>,
			value: u128,
			gas: u64,
			gateway_type: GatewayType,
			return_value: Option<Vec<u8>>,
	) -> CircuitOutboundMessage;
	
	fn transfer_escrow(
	    &self,
			escrow_account: Vec<u8>,
			requester: Vec<u8>,
			to: Vec<u8>,
			value: Vec<u8>,
			transfers: &mut Vec<TransferEntry>,
			gateway_type: GatewayType,
	) -> CircuitOutboundMessage;

}

Inside tests, the first call substrate_gateway_protocol methods with arguments and match the arguments in produced outbound_messages.

Acceptance Criteria

• Cover all 6 substrate_gateway_protocol methods specified above with unit tests

Integrate Multi Finality Verifier with Xdns to provide fresh data about active Gateways

Description

Each time the latest finality proof and block header is received by pallet-multi-finality-verifier, update data about the last successful delivery to pallet-xdns, which then will be able to select the best available based on that TTL updates.

• update_ttl

  With the goal to alert Circuit that connection to a requested gateway would likely fail, since the pallet-xdns hasn't received any update from it.

  pub fn update_ttl(last_finalised: u64, gateway_id: ChainId, gateway_type?) {
  
  		let xdns_record_id = look_up_gateway_type()?;
  
  	  <XDNSRegistry<T>>::mutate(&xdns_record_id, xdns_record {
  			 last_finalised,
  			 timestamp: <T as system::Config>::current_block()
  		});
  }

• best_available

  If more than one gateways are available (internal + external) and dispatching headers, suggest the best available gateway upon the Circuit's request - use internal → external → tx-only

  pub fn best_available(gateway_pointer) -> Result<XdnsRecord, Err> {
  
  	 fn calc_ttl() {
  		//		 last_finalised,
  	        //		 timestamp: <T as system::Config>::current_block()
  		// T::MaxAcceptableTTLMissing()? 
  	  }
      if (internal is available) -> calculate_ttl()? 
      if (ext is available) -> calculate_ttl()? 
      if (tx-only is available) -> calculate_ttl()? 
  
  	  <XDNSRegistry<T>>::mutate(&xdns_record_id, xdns_record {
  			 last_finalised,
  			 timestamp: <T as system::Config>::current_block()
  		});
  }

• update_cost * (bonus)

  With the goal to alert Circuit that connection to a requested gateway would likely fail, since the pallet-xdns hasn't received any update from it.

  pub fn update_cost(cost_per_weight: u64, recent_total_price, gateway_id: ChainId, gateway_type?) {
  
  		let xdns_record_id = look_up_gateway_type()?;
  
  	  <XDNSRegistry<T>>::mutate(&xdns_record_id, xdns_record {
  			 average_price = new_updaed_avg(recent_total_price),
  			 n_price_records += 1,
  			 cost_per_weight = new_updaed_avg(cost_per_weight) 
  		});
  }

Acceptance Criteria

• pallet-multi-finality-verifier updates TTL of pallet-xdns for gateways after each verified finality proof and headers:
  • after each finality proof received (end of pub fn submit_finality_proof)`
• Use best_available by Circuit when looking for a gateway to interact with
• pallet-xdns implements update_ttl and best_available based on delivered data
• bonus) Circuit updates the XDNS record with the recent fees statistics after successfully confirmed step

Benchmark on-chain contracts repository

Description

Benchmark contracts registry

Acceptance Criteria

• Following Circut's benchmark suite, setup a benchmark suite for pallet-contracts-repo
• Benchmark adding new smart contracts of 3 different sizes
• Benchmark fetching smart contracts with different query parameters
• Implement weights for pallet-contracts-repository following on the benchmarking results

Allow access to runtime modules via attached binaries

Description

Allow modules to be executed via Gateway as host functions, in order to support non-standard functionalities hosted by parachains that do not include Contracts pallet but have functionalities on runtime modules they want to share.

Acceptance Criteria

Runtime methods can be accessed via calls from the Wasm binaries that gateway executes. Runtime devs should connect the handler for dispatching calls to runtime methods easily, have an access to the origin, input data and charge fees.
There should be a new trait for the runtime modules that want to extend the functionalities of standards by dispatches to the local runtime.

Additional Information

Initially, it can be done by mimicking the behaviour of regular contracts calls. As a handler in runtime wasm (versatile VM) call will be dispatched to the handler that redirects the info about module name, method name and input and passes the output back. By doing so, the packages can inherit the validation (prepare step) after contracts. In the future, the standard extension trait should give the possibility of extending regular standards with the new callable operations but this means that the validation cannot inherit after contracts prepare step anymore.

Refactor exec composable at Gateway into separately-callable methods - Subtask #31

Description

Split the current exec_composable function at Internal Runtime Gateway that invokes different scenarios based on provided arguments into separate methods: set_storage, get_storage, call_dirty, call_escrow, call_static, transfer_dirty, transfer_escrow

See Current gateway protocol implementation for reference

Acceptance Criteria

• refactors gateway protocol as separately-callable pallet methods set_storage, get_storage, call_dirty, call_escrow, call_static, transfer_dirty, transfer_escrow

Create a script that properly calculates and sets weights based on benchmark results

Dockerize Circuit node

Acceptance criteria

• Create a docker image that builds and executes the Circuit node based on the latest development branch
• Trigger running and publishing this docker image when a PR is merged to the development branch

Prepare runtime node demo and Docker images

Description

• Create a docker image + launch scripts for node-tiny with following pallets installed:
  • pallet-contracts-gateway
  • pallet-runtime-gateway
  • demo-storage
• Rename node-tiny to demo-runtime and remove node-full for now as costly to maintain
• Extend functional tests to cover dockerised node-tiny and test execution of storage_runtime_demo.wat contract + check on emitted events.

Implement Relayer depending on Parity Bridge Messages pallet to access Internal Gateways

Description

The main difference to parity bridges common is that in our case, the proofs of message delivery can't be accessed directly from the runtime's storage, because we can't guarantee the synchronous communication between two pallets installed in both networks (pallet-bridge-messages). This is only possible in our case of internal-programmable-gateway

For Internal Substrate Gateways we can fully re-use the synchronous implementation of parity bridge relayers and only integrate them with Gateways

Probably makes sense to do alongside Circuit's integration tests.

Acceptance Criteria

• Run and write bash script for Circuit + Demo Gateway Runtime + relayer setup
• Implement and test relaying messages from Circuit to Gateway using standard parity-bridge relayer implementation.
  • It can be done by extending scripts + extending relayers implementation into adding one custom for gateway (submit execution)
• ⚠️ Add dependency to pallet-bridge-messages to Gateway's pallet!

Implement post-run as the confirm side effects

Description

Validate inclusion and correctness of execution for side effects and its values provided as a result of candidate for the delivered side effects results.

Implement remaining features to Circuit prototype

Description

implement a Circuit that implements all of the protocol features outlined in the whitepaper except any network-related ones, therefore still works as on semi-decentralised PoA premises. Authorities validate the execution via multiple integrated gateways and produce publicly verifiable proofs of that execution.

Acceptance Criteria

• Authorities guarantee the execution correctness by re-validating the execution and producing verifiable proofs.
• Implement a transaction pool for execution requests and order the requests by execution fees.
• Implement fishermen dispatching proofs of misbehaviour and execute on them by ensuring the execution of x2 transactions worth.
• Expose API and register new gateways and blockchains dynamically. Work out and implement a pricing model for running active gateways and contracts on Circuit (note that as collecting witness from a gateway grows the witness collider size).
• Form multi-signature for the support of transaction-only external gateway based on fast Multiparty Threshold ECDSA.

Produce signed outbound messages by authorised authorities on Circuit

Description

Messages can now be signed using AuthorityId.
Use the newly implemented compose_call and compose_extrinsics_offline macros to implement the method produce_signed_messages that is used by substrate outbound protocol in order to construct the messages which are ready to be dispatched onto gateways.

there is a commented out function body already, the task is about connecting it with the most recent AuthorityId structures.

pub fn produce_signed_payload(
        &self,
        namespace: Vec<u8>,
        name: Vec<u8>,
        _arguments: Vec<Vec<u8>>,
    ) -> MessagePayload {
        // let call_bytes = compose_call!(name_str, name_str, arguments).to_vec();
        // let tx = compose_call!(namespace, name);

        MessagePayload::Signed {
            // ToDo: get public key from AuthorityId instead
            signer: vec![],
            module_name: namespace,
            method_name: name,
            call_bytes: vec![],
            signature: vec![],
            extra: vec![],
        }
    }

Additionally, inspect the substrate_gateway_protocol code to look for references to submitter and insert the assembly.submitter_pair.public() when needed.

Acceptance Criteria

• produced_signed_message method is completed and functional
• produced_signed_message method is covered with unit tests
• substrate_gateway_protocol uses signer's account id from assembly.submitter_pair.public() when needed

Execution fees shared between all actors including developers on the pay-per-use model

Description

Share execution fees with developers to on-chain contracts repository who have decided to be remunerated when their contracts are used. "Miners" fees will be shared between all system actors: developers, relayers, liquidity providers, validators and collators.

Acceptance Criteria

• Account on-chain rewards for remunerated contracts execution per contracts usage.
• Excel on contract standards and implement urgently missing contracts where necessary (responding to the community needs).

Create Circuit scaffold as a new Substrate Pallet

Description

Create Substrate Pallet for Circuit maintainers. Implement methods for code execution which receives composable smart contracts and execution schedule. Integrate dispatch & watch functionalities from subxt in order to exchange messages with Gateways.

Acceptance Criteria

• implement new Circuit pallet with composable_execute method
• decode execution_schedule from JSON; support multiple execution types: transfer-only, volatile-execution, static-call, side-effects call, side-effects execution
• set up unit tests for Circuit
• create new FRAME-based Substrate node demo-circuit-runtime from template with Circuit pallet installed
• set up integration tests for Circuit - dispatch composable smart contracts over API of demo-circuit-runtime

Refactor Circuit project structure

This task is about refactoring the project structure for circuit as per @vedhavyas' draft found below:

https://hackmd.vedhavyas.dev/s/RVRfRJVoR

circuit/
    pallets/
        contracts-registry/
        execution-delivery/
        xdns
    primitives/
    rpc/
        src/
        tests/
    test-utils/
    runtime/
    node/

Acceptance criteria

• Circuit project structure looks as above
• Project compiles / tests pass successfully

Implement standards - versatile Wasm VM that integrates with runtime

Description

Provide a pallet that can be integrated with any parachain (with and without smart contracts) and by sandboxing the execution of attached to calls Wasm binaries (packages) can access shared resources of runtime of that parachain.

Standards also referred to as versatile Wasm VM is the equivalent of Contract pallet external execution context to work in multiple phases with the use of Escrow Account.

Acceptance Criteria

Following host functions are integrated into a parachain's runtime and can be accessed via the gateway:

fn gas(amount: u32);
fn get_storage(key: &StorageKey) -> Option<Vec<u8>>;
fn set_storage(key: StorageKey, value: Option<Vec<u8>>);
fn transfer(to: &AccountIdOf, value: BalanceOf, gas_meter: u64) -> Result<(), DispatchError>;
/// Call to runtime.
fn call( to: &AccountIdOf, value: BalanceOf, gas_meter: u64, input_data: Vec<u8>) -> ExecResult;
/// Returns a reference to the account id of the caller.
fn caller() -> &AccountIdOf;
/// Returns a reference to the account id of the current contract.
fn address() -> &AccountIdOf;
fn balance() -> BalanceOf;
fn value_transferred() -> BalanceOf;
fn now() -> &MomentOf;
fn minimum_balance() -> BalanceOf;
fn tombstone_deposit() -> BalanceOf;
fn random(subject: &[u8]) -> SeedOf;
fn deposit_event(topics: Vec<TopicOf>, data: Vec<u8>);
fn set_rent_allowance(rent_allowance: BalanceOf);
fn rent_allowance() -> BalanceOf;
fn block_number() -> BlockNumberOf;
fn max_value_size() -> u32;
fn get_weight_price(weight: Weight) -> BalanceOf;

Additional Information

Versatile VM should come as an additional pallet, as many actors of the system will integrate with it - parachains without smart contracts, circuit validators, foreign blockchains architectures.

Include Contracts Registry RPC to Circuit's RPC

Description

Right now, Contracts Registry has an RPC handler as well as a runtime API, but not included in the overall Circuit RPC. This task is a followup to #71 and should fix the existing TODOs by adding Contracts Registry RPC endpoints to the Circuit RPC and testing the provided functionality.

Acceptance Criteria

• Circuit RPC includes the Contracts Registry RPC as part of its client.
• Exposed endpoints are tested.

Receive relayed blocks and relevant messages at Circuit

Description

Relay blocks and relevant messages using Parity Bridges.

Acceptance Criteria

• Circuit implements one end of a Bridge
• Gateways implement the other end of a Bridge
• Register additional bridge between Circuit and Polkadot that collects block headers
• Register additional bridge between Circuit and Ethereum that collects block headers
• Bridges can be dynamically instantiated (currently only one bridge supported at Parity Bridges implementation

Collect contracts in on-chain repository

Description

Set up the on-chain composable contracts repository.

Acceptance Criteria

• Store successfully executed contracts in on-chain storage.
• Provide handy API fetch_contracts for access.
• Store state of underlying contracts independently for each requester.
• Set up an off-chain repository for composable contracts open for contributions.

Refactor CI pipeline to boost compilation speeds

The current pipeline has a lot of steps, which, even though they use a cache, can take a significant time to actually finish.

So this ticket is about optimizing and refactoring the current pipeline in order to make it faster.

Acceptance criteria

• Move format step to a pre-commit hook
• Base off a Docker image with dependencies pre-fetched and compiled
• Investigate using cargo chef to cache dependencies on Docker image
• Migrate all steps of the current CI pipeline to the new one
• Use this image as a base for the Circuit node Docker image

Validate on-gateway execution by Circuit

Description

Execution performed via Gateway must be validated by Circuit. The circuit must be delivered with:

• witness: pre and post-execution block headers from foreign blockchain
• execution stamp: execution after-effects contained in execution stamp
  Assuming above are collected and valid, Circuit can now re-execute the smart contract locally using versatile-vm and compare results with collected execution stamp.

Acceptance Criteria

• collect execution stamp
• collect witness, block headers, from FRAME-based Substrate node and establish the transaction containing execution stamp is indeed included on that chain
• implement validation

Additional Information

Validators authorizing execution stamps that are proven to be incorrect are not penalized yet for their misbehaviour. That comes with the next development milestones alongside with Proof of Stake-like system implementation.

Unit test dispatch of CircuitOutboundMessages locally using Runtime RPC

Implement a new unit test suite that would simulate the full path of CircuitOutboundMessage being created (e.g. by a call to substrate_gateway_protocol::transfer) and then dispatches those messages into a local endpoint - Runtime RPC. This goes pretty close to the use-case scenario when the CircuitOutboundMessage is being relayed and dispatched into foreign runtime's RPC endpoint.

It is important to check both Write and ReadOnly messages since they're using different methods to construct payload. Write corresponds with signing messages, where it's essential to test that the origin will be extracted and validated correctly.

Below there is an example test using runtime_api_client:

let child_info = ChildInfo::new_default(STORAGE_KEY);
	let client = Arc::new(substrate_test_runtime_client::TestClientBuilder::new()
		.add_child_storage(&child_info, "key", vec![42_u8])
		.build());
	let genesis_hash = client.genesis_hash();
	let (_client, child) = new_full(client, SubscriptionManager::new(Arc::new(TaskExecutor)));
	let child_key = prefixed_storage_key();
	let key = StorageKey(b"key".to_vec());

Read-only messages can be tested using direct calls to modules, like assert_ok!(Balances::transfer(Origin::signed(3), multi, 5));

For now, narrow the methods from inbound protocol to cover to:

pub trait GatewayInboundProtocol {

	fn get_storage(
			&self,
			key: Vec<u8>,
			gateway_type: GatewayType
	) -> CircuitOutboundMessage;
	
	fn set_storage(
	    &self,
			key: Vec<u8>,
			value: Option<Vec<u8>>,
			gateway_type: GatewayType,
	) -> CircuitOutboundMessage;
	
	fn call_escrow(
	    &self,
			module_name: &str,
			fn_name: &str,
			data: Vec<u8>,
			to: Vec<u8>,
			value: u128,
			gas: u64,
			gateway_type: GatewayType,
			return_value: Option<Vec<u8>>,
	) -> CircuitOutboundMessage;
	
	fn custom_call_escrow(
	    &self,
			module_name: &str,
			fn_name: &str,
			data: Vec<u8>,
			to: Vec<u8>,
			value: u128,
			gas: u64,
			gateway_type: GatewayType,
			return_value: Option<Vec<u8>>,
	) -> CircuitOutboundMessage;
	
}

Acceptance Criteria

• Cover all substrate_gateway_protocol methods specified above with unit tests
• Implement a good technique to unit test creating messages and dispatching them into local endpoints using Runtime RPC.
• Check results of dispatching CircuitOutboundMessage produced with 6 methods listed above against Runtime RPC.
• Submitter signing the messages is correctly recognised on receiving endpoint of Runtime RPC checking the origin

Relayer: Add ethereum light client pallet from snowbridge into circuit runtime

Description

We need ethereum light client in circuit for the message confirmation

Acceptance Criteria

Ethereum light client is added to runtime and block headers can be set through an extrinsinc

Receive and process execution requests from API

Description

Receive execution requests from API and process them according to specified IO schedule.

Acceptance Criteria

• Create a new runtime module responsible for processing Circuit requests (a new Pallet). It will become the main Circuit engine and execution orchestrator.
• New Engine Pallet implements the composable_exec method for RPC API.
• Analyze received IO Schedule using regular expressions and define & document rules of passing following execution steps and phases
• New Engine Pallet is connected to Circuit node and installable as a new Pallet
• Keep the context for the interoperable transaction
• Use off-chain workers client to dispatch the calls to external runtimes via gateways
• Analyze the response from gateways and temporarily assume it’s enough to validate the transaction’s security.

Relayers: Include beefy gadget as part of the circuit node

Description

We need to include beefy gadget as part of the circuit node so that we can use it for the light client on ethereum.

Acceptance Criteria

Circuit node will also start giving beefy justifications once it is started

Relayer: Add MMR Beefy pallet

Description

Add beefy mmr pallet that is required for the relayers

Bring XDNS to a serviceable state

Description

Populate test data of the default Substrate Runtime (after Circuit) to XDNS, covering the full path of a new network being registered at Circuit.

Make sure that send over jsonrpc CircuitOutbound messages reach the default runtime's metadata - test signed and unsigned messages via author_submitExtrinisc jsonrpc on the default metadata of the Circuit.

Metadata, for now, force push update after metadata for network changes?

Mainly manual testing.

Consider exposing Runtime RPC?

Acceptance Criteria

• Add proper Circuit and Gateway metadata to XDNS
• Add pallet XDNS unit tests
• Implement missing functionality to make sure pallet XDNS is serviceable
• Make sure Execution Delivery properly receives XDNS data when queried

Fix SDK cargo-contracts to work with latest Substrate version

Expected behavior

Fix cargo-contracts https://github.com/t3rn/cargo-contract/tree/ce23c44282d39a228a4cd24b9f8e4ed02755e38f

Update Programmable Gateways for substrate-based nodes

Description

Support all execution types available via Gateway specified in the whitepaper and release independently external and internal gateways for Substrate-based nodes.

Acceptance Criteria

• Upgrade current implementation to FRAMEv2 ↪️ Subtask #66
• Support all execution types available via Gateway specified in the whitepaper, i.e. exec-escrowed, transfer-escrowed, call-static, call-escrowed, call-dirty, transfer-dirty ↪️ Subtask #31
• Release independent extrinsic and intrinsic Programmable Gateways for Substrate-based runtimes.
• Write integration tests checking all of the API functionalities of all Gateways.

Benchmark Circuit's Execution Requests

Description

Introduce benchmarks for the Circuit based on Substrate standard benchmarking suite.

Acceptance Criteria

• benchmark suite runs all of the benchmarks on 3 different request sizes with 1, 3 and 10 components for each execution
• benchmark decompose_io_schedule
• assess whether the single consolidated Xtx per execution run isn't too heavy even for Prototype 1.
• benchmark dry_run (execution with new smart contracts unseen in on-chain repo)
• benchmark pre_run
• benchmark post_run
• Implement weights for execution Circuit call with the cost suggestion

Implement basic Benchmark suite at Circuit

Description

Introduce Benchmark test suite that could serve as a template and be used in all our pallets - Circuit, contracts registry etc. Install with a basic call at Circuit

Acceptance Criteria

• implement Circuit's basic benchmark suite
• install basic suite in Circuit

Readme typo fix

"Learn more about writing comosable contracts in our SDK."

Kindly fix the typo "comosable" on the readme file

Add benchmarks and weights for pallet XDNS

Pallet XDNS currently has dummy weights for functions under pallet::call, the issue is about benchmarking the pallet and generating relevant weights for every function in it.

Acceptance criteria

• benchmark add_new_xdns_record
• benchmark update_ttl
• benchmark purge_xdns_record
• benchmark best_available

Demonstrate calls to runtime with a simple storage service

Description

Demonstrate + Test calls to runtime and versatile-wasm functions by creating a new contract (stored in fixtures) that accesses and calls the native runtime functions of a simple storage service, e.g. one from recipies.
Integrate with the operating fees on that parachain and charge requester accordingly (fees metering needs to be added into versatile-wasm).

Acceptance Criteria

• contract calls the host function of runtime parachain (store_value, complex_calculations, double).
• contract accesses storage of runtime parachain.
• requester is charged accordingly to operative fee strategy
• contract deposits event after successful execution (that later can be verified by functional tests)
• Simple storage runtime + versatile-wasm pallets are bundled as an independent docker image.

Additional Information

Rename gateway pallets and cleanup directories

Description

• Rename pallet-escrow-gateway to pallet-contracts-gateway & move to a separate directory
• Rename pallet-gateway-balances to pallet-runtime-gateway & move to a separate directory
• Move separate repo pallet-contracts-copy to escrow-contracts-wrapper
  • Trait and all available public variables need to be passed from original pallet-contracts
• Delete commons as it's already replaced by pallet-escrow-engine
• Release pallets to the independent repo & setup node-tiny + node-full
• Change version of all pallets to 0.3.0 (as of the end of the 3rd milestone)

Change pre-run to on-chain contracts executor

Description

Convert current pre-run step of Volatile VM into on-chain execution that accepts smart contract input and necessary off-chain data (output of off-chain execution), validates it against the storage root from 1-way bridge and proceeds with the on-chain contracts execution on VVM if successful.

Output of the execution is emitted in an on-chain event list of side effects to be dispatched onto target chains.

Acceptance Criteria

• VVM::onchain_executiion produces a map of side effects for all available VM ops

Adapt compiler for !ink contracts to be compatible with Gateways

Description

Adapt paritytech/cargo-contract to execute !ink contracts on both Runtime & Contracts Gateways.

Acceptance Criteria

• command to execute WASM contract at Runtime Gateway & return the result
• command to execute WASM contract at Contracts Gateway & return the result