celo-org / optics-monorepo

🔮 OPTimistic Interchain Communication

Optics

OPTimistic Interchain Communication

Overview

Optics is a cross-chain communication system. It handles passing raw buffers between blockchains cheaply, and with minimal fuss. Like IBC and other cross-chain communication systems, Optics creates channels between chains, and then passes its messages over the channel. Once a channel is established, any application on the chain can use it to send messages to any other chain.

Compared to IBC and PoS light client based cross-chain communication, Optics has weaker security guarantees, and a longer latency period. However, Optics may be implemented on any smart contract chain, with no bespoke light client engineering. Because it does not run a light client, Optics does not spend extra gas verifying remote chain block headers.

In other words, Optics is designed to prioritize:

• Cost: No header verification or state management.
• Speed of implementation: Requires only simple smart contracts, no complex cryptography.
• Ease of use: Simple interface for maintaining xApp connections.

You can read more about Optics' architecture at Celo's main documentation site or within the docs folder of this repository.

Integrating with Optics

Optics establishes communication channels with other chains, but it's up to xApp (pronounced "zap", and short for "cross-chain applications") developers to use those. This repo provides a standard pattern for integrating Optics channels, and ensuring that communication is safe and secure.

Integrations require a few key components:

• A Home and any number of Replica contracts deployed on the chain already. These contracts manage Optics communication channels. and will be used by the xApp to send and receive messages.

• A XAppConnectionManager (in solidity/optics-core/contracts). This contract connects the xApp to Optics by allowing the xApp admin to enroll new Home and Replica contracts. Enrolling and unenrolling channels is the primary way to ensure that your xApp handles messages correctly. xApps may deploy their own connection manager, or share one with other xApps.

• A Message library. Optics sends raw byte arrays between chains. The xApp must define a message specification that can be serialized for sending, and deserialized for handling on the remote chain

• A Router contract. The router translates between the Optics cross-chain message format, and the local chain's call contract. It also implements the business logic of the xApp. It exposes the user-facing interface, handles messages coming in from other chains, and dispatches messages being sent to other chains.

Solidity developers interested in implementing their own Message library and Router contract should check out the optics-xapps package. It contains several example xApps.

You can find current testnet deploy configurations in the rust/config/ directory. These deployments happen frequently and are unstable. Please feel free to try out integrations using the deployed contracts in the LATEST config.

It is Strongly Recommended that xApp admins run a watcher daemon to maintain their XAppConnectionManager and guard from fraud. Please see the documentation in the rust/ directory and the Optics architecture documentation for more details.

Working on Optics

Commit signature verification

Commits (and tags) for this repo require signature verification. If you'd like to contribute to Optics, make sure that your commits are signed locally.

Pre-commit hooks

Set up your pre-commit hook:

echo "./pre-commit.sh" > .git/hooks/pre-commit
chmod +x .git/hooks/pre-commit

Note: In the event you need to bypass the pre-commit hooks, pass the --no-verify flag to your git commit command

Solidity

1. Install dependencies

   cd solidity/optics-core
   npm i
   cd ../optics-xapps
   npm i

2. Setup your .env files

   cd typescript/optics-deploy
   touch .env && cat .env.example > .env
   cd ../../solidity/optics-core
   touch .env && cat .env.example > .env
   cd ../optics-xapps
   touch .env && cat .env.example > .env

   Then, add values to the keys in the newly created .env files.

3. Install jq

   brew install jq

     OR  

   sudo apt-get install jq

4. Install solhint

   npm install -g solhint
   // to check it is installed:
   solhint --version

Rust

• install rustup
• see rust/README.md

Building Agent Images

There exists a docker build for the agent binaries. These docker images are used for deploying the agents in a production environment.

cd rust
./build.sh <image_tag>
./release.sh <image_tag>

What is Optics?

We present Optics — a system for sending messages between consensus systems without paying header validation costs by creating the illusion of cross-chain communication. Similar to an atomic swap, Optics uses non-global protocol validation to simulate cross-chain communication. Optics can carry arbitrary messages (raw byte vectors), uses a single-producer multi-consumer model, and has protocol overhead sublinear in the number of messages being sent.

Key Points

System sketch:

1. A "home" chain commits messages in a merkle tree
2. A bonded "updater" attests to the commitment
3. The home chain ensures the attestation is accurate, and slashes if not
4. Attested updates are replayed on any number of "replica" chains, after a time delay

As a result, one of the following is always true:

1. All replicas have a valid commitment to messages from the home chain
2. Failure was published before processing, and the updater can be slashed on the home chain

This guarantee, although weaker than header-chain validation, is still likely acceptable for most applications.

Summary

Optics is a new strategy for simulating cross-chain communication without validating headers. The goal is to create a single short piece of state (a 32-byte hash) that can be updated regularly. This hash represents a merkle tree containing a set of cross-chain messages being sent by a single chain (the "home" chain for the Optics system). Contracts on the home chain can submit messages, which are put into a merkle tree (the "message tree"). The message tree's root may be transferred to any number of "replica" chains.

Rather than proving validity of the commitment, we put a delay on message receipt, and ensure that failures are publicly visible. This ensures that participants in the protocol have a chance to react to failures before the failure can harm them. Which is to say, rather than preventing the inclusion of bad messages, Optics guarantees that message recipients are aware of the inclusion, and have a chance to refuse to process them.

To produce this effect, the home chain designates a single "updater." The updater places a bond ensuring her good behavior. She is responsible for producing signed attestations of the new message tree root. The home chain accepts and validates these attestations. It ensures that they extend a previous attestation, and contain a valid new root of the message set. These attestations are then sent to each replica.

The replica accepts an update attestation signed by the updater, and puts it in a pending state. After a timeout, it accepts the update from that attestation and stores a new local root. Because this root contains a commitment of all messages sent by the home chain, these messages can be proven (using the replica's root) and then dispatched to contracts on the replica chain.

The timeout on new updates to the replica serves two purposes:

1. It ensures that any misbehavior by the updater is published in advance of message processing. This guarantees that data necessary for home chain slashing is available for all faults.
2. It gives message recipients a chance to opt-out of message processing for the update. If an incorrect update is published, recipients always have the information necessary to take defensive measures before any messages can be processed.

Deploy Procedure

The contract addresses of each deploy can be found in rust/config. The latest deploy will be at rust/config/[latest timestamp] with bridge contracts within that same folder under /bridge/[latest timestamp].

The agents are setup to point at 2 environments at a time: deployment and staging.

When agents are deployed to point at a new environment, they cease to point at the old ones. We do not continue to operate off-chain agents on old contract deploys. Contracts not supported by the agents will cease to function (i.e. messages will not be relayed between chains).

Off-chain agents are not automatically re-deployed when new contract deploys are merged. Auto-redeploys will be implemented at some future date.

The Optics team will maintain a document here that will specify the contracts supported by the rust agents.

optics-monorepo's Issues

Update Replica.sol to keep queue of pending updates

Goal is to allow pipelining. Invalidating one should invalidate all successive ones

replace references to slip44 with "domain"

We ought not rely on slip44. We know there will be some 4 byte ID but don't want to nail it down yet

We'll call this ID the "domain" ID. as in sending messages between domains

Exhaustive tests

• shared tests for rust and solidity merkles
• shared tests for rust and solidity message format
• more

Extend optics_core::traits::Common with update lookup methods

async fn signed_update_by_old_root(&self, old_root:: H256) -> Result<Option<SignedUpdate>, ChainCommunicationError>

Should query chain events for an Update event with the correct old root, and return it (if any exists)

Update (and clarify understanding of) off-chain agent descriptions

• fill in Processor description
• Update others so level of detail is consistent between Notion and GitHub

fix: disallow resetting already-proven messages

• right now any actor is allowed to reset a processed message from Message.Processed back to Message.Pending by just calling prove on an already-processed message
• fix: require message in prove() to have status Message.None

Error in new dev setup

The new lib folder relies on the json artifacts.

It is loaded by hardhat before any task, including compile.

Therefore if the artifacts don't exist, hardhat errors during compilation, and the artifacts can't be generated

Figure out how to fix this

Write tests for the Merkle contract and rust implementation

Create a series of JSON test vectors that can be read from both the solidity and rust tests
These should test

• that the merkle tree root generation is identical for a large set of leaves
• that proofs generated in rust can be processed by solidity

Generate more documentation, specification, and prose description

More materials for explaining what Optics is and how people are expected to interact with it

feature: create struct to sync watcher with list of all roots

Processor

A rust daemon using optics-base

• maintain a local merkle tree with ALL leaves
• observe >= 1 replica
• wait for confirmations with new batch
• generate proofs for messages when the root changes
• dispatch proofs to replicas (if configured)
• dispatches messages for processing (if configured)

Rust double_update does not compile with new solidity ABI

introduced in #38

Need to update optics_base::abis to match the new solidity call

Relayer

A rust daemon using optics-base:

• observe Home for signed updates
• forward signed updates to the replica contracts
• poll replica contracts for updates that can be confirmed
• cause updates to be confirmed (call confirm)

Add `update_abis.sh` functionality to the `npm compile` script.

currently rust/optics-base/update_abis.sh must be manually run to update rust bindings whenever the contract ABI changes. We should ensure that npm run compile ALWAYS does this.

Add a prove_and_process function to the replica trait

• in optics_core::traits::replica add a prove_and_process function.
  • It should match the Solidity declaration and the existing prove and process functions
  • provide a default implementation in the trait that calls
• in optics_base::abis override the prove_and_process function to use Solidity's proveAndProcess instead of the default

here's the default implementation. it goes in the trait declaration

    /// WRITE A DOCSTRING
    async fn prove_and_process(
        &self,
        message: &Message,
        index: u32,
        proof: [H256; 32],
    ) -> Result<TxOutcome, ChainCommunicationError> {
        // Write a function using `prove` and `process`
    }

deduplicate workspace dependencies

https://github.com/rust-lang/rfcs/blob/master/text/2906-cargo-workspace-deduplicate.md

Write tests for the Common contracts

These should check

• checksig behavior
• double update processing
• domain has calculation (compare to rust if possible)

Create a struct that syncs a prover from the Home chain

pub struct OpticsProofSync {
    prover: optics_core::accumulator::Prover,
    home: Arc<Box<dyn Home>>
}
 
impl OpticsProofSync {
    // poll the chain for events
    // construct the prover from the list of events
    // handle large event lists
    // tokio::spawn a task that polls the chain for updates regularly
    pub async fn from_home(home: Arc<Box<dyn Home>>) -> Result<Self> {...}

    // pass through the rest of the Prover api
}

Update proveAndProcess function to take 3 parameters, excluding leaf

the proveAndProcess function will be easier to read if it only takes three params — message, proof, index — and then passes in keccak256(message) into the prove function

feature: add fraudulent update detection to watcher

• currently, watcher only checks for double updates
• need to cross-check Replica updates with list of all roots from home to ensure that updater is not signing fake data and only submitting it to the Replica
• listen for Update events from Replica and ensure they exist in Home event logs (i.e. signed_update_by_new_root(replica_root) exists)
• if root doesn't exist halt Home and somehow fail Replicas...

Add new Kathy chat generators

• Static message
• List of static messages
• Random messages

feature: create struct that persists watcher roots list to disk

• takes watcher's history map and persists data to disk
• bootup should load this data from disk if it exists
• bootup root collection and disk persistence should both go on new WatcherSync struct

Should come after #75

Write test for IncrementalMerkle in rust

Configure Signers to be Optional

Right now, it expects 1 signer, but it should accept 0 or 1 signers.

Steps to reproduce:
Run cargo run --example example from /rust/optics-base

Also, update readme to include that cargo run --example example command :)

Research upgradability and governance patterns

feature: create WatcherSync struct to sync watcher with history of all past updates

• get list of all previous Update events emitted from Home and Replica
• fill watcher's history map using list

Comes before #84

test: expect Dispatch events to be emitted in a new Home "enqueues message" test

Setup pre-commit hook

• run prettier
• do linting
• run tests / coverage (?)

governance design questions

• upgradability of contracts
  • home
  • replica upgrades
• future voting?
• social reset mechanisms
• code for "big red button" to disconnect a replica of a faulty home
  • who controls these buttons?
  • how do we ensure reliability?
  • how do we rotate them?
• what does "on-chain governance" mean in a cross-chain environment?

unhappy case design questions

• Halting and Restarting on Home

  • halt trigger: how do we trigger the "halted" state --> Watcher proves fraud on Replica
  • halt behavior: what happens when we enter "halted" state --> Updater is slashed; rewards go to Watcher; messages can still be submitted to Home; signed messages aren't accepted
  • gov process: elect a new Updater to replace the fraudulent one
  • restart trigger: when a new Updater has been elected
  • restart behavior: resume signing updates for all the messages

• Halting and Restarting on Replica

  • halt trigger: someone triggers "halt" after observing the Home contract has been halted
  • gov question: who can trigger a halt on the Replica? --
    • fraud can't be proven on the Replica chain, so whoever pushes the "big red button" must be trusted
    • what's the incentive to reliably un-enroll fraudulent Replicas?
      - [ ] 🔨 require the button-pusher to stake on Home, slash them via governance if they don't report to Replica
      - [ ] 🥕 reward the button-pusher for reporting with rewards on the Replica chain
  • halt behavior: un-enroll the Replica for the fraudulent Home from the UsingOptics contract registry
  • _wait for the new Updater to be elected on the old contract
    • possibly flag certain messages as fraudulent (?)
    • possibly flush the current queue of messages (?)
    • possibly deploy an entirely new Replica?_
  • restart trigger: someone triggers restart after observing Updater has been elected on Home
  • gov question: how do we decide who has the authority to re-enroll the Replica after an election?
  • restart behavior: set the new Updater public key on the Replica and re-enroll it with UsingOptics contract

Tactical tasks

• Watcher reporting Updater fraud on Home
  • slash the Updater's stake - custodied in some contract (where ?)
  • reward the Watcher for reporting fraud (give them the stake? give them governance tokens?)
  • could the stake be taken over as part of a community governable fund, perhaps 👁️

Move `.abi.json` files to a shared location

Currently we are reproducing these in rust/optics-base/src/abis and in solidity/lib. We should move them to a shared location if possible

hardhat tasks for basic actions

hardhat task documentation

Basic hardhat tasks for deploying and manually calling important contract functions.

• Deploy Home
• Deploy Replica
• Enqueue message
• Prove message
• Process message
• Submit Update
• Submit Double Update

design / research: on-chain governance patterns

• review existing patterns for governing contract upgrades on-chain
  • Uniswap
  • Celo
  • Compound
  • Fei
  • yearn.finance (multisig)
  • Dharma (multisig)
• governance token patterns
  • initial token distribution - who receives tokens ? when? how?
  • voting system - how are tokens used for voting?

Research existing governance systems
Describe design choices & tradeoffs — pros & cons
What do we like about them? What do we not like?

updater selection design questions

Need designs for:

• updater rotation
  • when is an updater rotated?
  • why?
  • how do we updater handle failure?
  • can we have round robin updates?
• replica notification
  • destination message that all replicas process?
• bond management
  • how is bond size set?
  • how is the bond custodied
• can we have synchrony assumptions?
  • should we embed home chain timestamp in messages?

docs: add natspec documentation to core contracts

Natspec documentation for clarification and easier understanding for later bridges newcomers. For now, this would include Common, Home, Replica, and maybe Sortition.

fix: re-add settings and can_confirm to Relayer

Somehow deleted during conflict resolution...

Write tests for the Home contract

Should check

• halt on fail
  • set the contract to failed, and ensure that notFailed modifier correctly causes errors
• double updates
• update acceptance
• improper updates
• suggest update behavior
• root calculation and the root queue
  • now: push N messages, log out the root, queue, branch, and messages
  • later: test the solidity against the rust using shared vectors
• write a JS util function to sign updates (@prestwich)

Increase test coverage of Common and Replica contracts

• Add tests to Replica for proving messages (use merkleTestCases.json for cases)
• Add test cases for message formatting helpers (ensure byte indexing is correct)

Write tests for the Queue contract

Should test the the queue is FIFO and doesn't allow out of bounds access

design a token bridge

Below is outdated. See code in #71

high level sketch of functionality

• Vault contract
  • unescrows native tokens (or creates non-native ones)
  • escrows tokens (or destroys non-native ones)
  • registers sender and receiver contracts
• Sender contract
  • dispatches messages to the Home contract
  • may send to any destination
• Receiver contract
  • manages a set of create2 addresses for non-native tokens
  • processes messages from a single local replica

big question:
how are tokens recognized based on their route?
- suppose it goes A-> B-> C -> A
- how do we know what to unescrow?

Updater

A rust daemon using optics-base:

• connect to the home chain only
• sign update attestations
• submit them to the home chain
• config: escalate gas middleware
• prevent double-update middleware

Slash the Updater if they do not produce an update after a certain period of time

On the Home.sol contract

• add a function that can be called by anyone that will slash the Updater's stake if a certain amount of time has passed since the a message has been added & an update has not occurred (timestamp should be timestamp of oldest message that has not also been updated)

Update documentation

Other updates?

Watcher

A rust daemon using optics-base

• connect to home
• connect to any number of replicas
• cache all updates
• check for fraud (double/improper updates)
• submit fraud to home chain
• config: emergency stop on any replica

Relayer Config

Relayer is missing config file

Move test case .json files to root vectors folder

Set up linting and coverage

• eslint
• solidity coverage? (may be broken)
• rust cargo test
• precommit hooks for running tests and linters?
• circle CI?

Refactor contract tests to use new lib/index.js classes and utility functions

• use submitSignedUpdate and submitDoubleUpdate
• use Replica and Home classes

Distinguish between full and partial messages

When a message is submitted to the home, it's just a destination, recipient, and message. The Home adds the sequence, origin, and sender attributes. Right now, the rust doesn't have a representation of the Message. just

• Come up with cute names for "message" and "stamped + addressed message"
• Refactor optics-core and optics-base to have 2 separate structs for these

Write tests for the Replica Contracts

These should check

• optimistic root acceptance
• the new root queue
• batched confirmations
• update signature checking
• halt on failure
• double updates
• merkle proving
• processing a proven message

design / research: upgradability patterns for contracts

• review existing upgradability patterns
  • OpenZeppelin
  • avoid the Diamond Standard
• review existing upgradable contract implementations + design choices
  • Compound
  • Uniswap
  • Dharma
  • Celo
• security concerns?