chainsafe / forest

🌲 Rust Filecoin Node Implementation

Forest is a Filecoin node written in Rust. With Forest, you can:

• Send and receive FIL (at your own risk - Forest is experimental),
• validate the Filecoin blockchain,
• generate blockchain snapshots.

While less feature-complete than the canonical Filecoin node, Lotus, Forest aims to be the faster and easier-to-use alternative.

Questions

Have questions? Feel free to post them in Forest Q&A!

Run with Docker

No need to install Rust toolchain or other dependencies, you will need only Docker - works on Linux, macOS and Windows.

# daemon
❯ docker run --init -it --rm ghcr.io/chainsafe/forest:latest --help
# cli
❯ docker run --init -it --rm --entrypoint forest-cli ghcr.io/chainsafe/forest:latest --help

Next, run a Forest node in a CLI window. E.g. Run calibration network

Thereafter, in another terminal, you will be able to use the forest-cli binary directly by launching bash in the forest container:

docker exec -it forest /bin/bash

For more in-depth usage and sample use cases, please refer to the Forest Docker documentation in the Forest Book. Keep in mind that the latest tag is the latest stable release. If you want to use the current development build, use the edge tag.

Dependencies

• Rust (toolchain version is specified in rust-toolchain.toml)

Install rustup

• OS Base-Devel/Build-Essential
• Clang compiler

Ubuntu (20.04)

sudo apt install build-essential clang

Archlinux

sudo pacman -S base-devel clang

Fedora (36)

sudo dnf install -y clang-devel

Alpine

apk add git curl make gcc clang clang-dev musl-dev

Installation

# Clone repository
git clone --recursive https://github.com/chainsafe/forest
cd forest

# Install binary to $HOME/.cargo/bin
make install

# Run the node on mainnet
forest

To create release binaries, checkout the latest tag and compile with the release feature.

git checkout $TAG
make build # make debug build of forest daemon and cli
# or
make release # make release build of forest daemon and cli
# or
make install # install forest daemon and cli

Config

Keystore

To encrypt the keystore while in headless mode, set the FOREST_KEYSTORE_PHRASE environmental variable. Otherwise, skip the encryption (not recommended in production environments) with --encrypt-keystore false.

Network

Run the node with custom config and bootnodes

forest --config /path/to/your_config.toml

Example of config options available:

data_dir = "<directory for all chain and networking data>"
genesis_file = "<relative file path of genesis car file>"

[network]
listening_multiaddr = "<multiaddress>"
bootstrap_peers = ["<multiaddress>"]

Example of a multiaddress: "/ip4/54.186.82.90/tcp/1347/p2p/12D3K1oWKNF7vNFEhnvB45E9mw2B5z6t419W3ziZPLdUDVnLLKGs"

Configuration sources

Forest will look for config files in the following order and priority:

• Paths passed to the command line via the --config flag.
• The environment variable FOREST_CONFIG_PATH, if no config was passed through command line arguments.
• If none of the above are found, Forest will look in the systems default configuration directory ($XDG_CONFIG_HOME on Linux systems).
• After all locations are exhausted and a config file is not found, a default configuration is assumed and used.

Logging

The Forest logger uses Rust's log filtering options with the RUST_LOG environment variable. For example:

RUST_LOG="debug,forest_libp2p::service=info" forest

Will show all debug logs by default, but the forest_libp2p::service logs will be limited to info

Forest can also send telemetry to the endpoint of a Loki instance or a Loki agent (see Grafana Cloud). Use --loki to enable it and --loki-endpoint to specify the interface and the port.

Testing

First, install the nextest test runner.

cargo install cargo-nextest --locked

# To run base tests
cargo nextest run # use `make test-release` for longer compilation but faster execution

# To run all tests and all features enabled
make test-all

Chain synchronization checks are run after every merge to main. This code is maintained in a separate repository - Forest IaC.

Linters

The project uses exhaustively a set of linters to keep the codebase clean and secure in an automated fashion. While the CI will have them installed, if you want to run them yourself before submitting a PR (recommended), you should install a few of them.

# You can install those linters also with other package managers or by manually grabbing the binaries from the projects' repositories.

# Rust code linter
rustup component add clippy

# Rust code formatter
rustup component add rustfmt

# TOML linter
cargo install taplo-cli --locked

# Scanning dependencies for security vulnerabilities
cargo install cargo-audit

# Spellcheck
cargo install cargo-spellcheck

After everything is installed, you can run make lint-all.

Joining the testnet

Select the builtin calibnet configuration with the --chain option. The --auto-download-snapshot will ensure that a snapshot is downloaded if needed without any prompts.

./target/release/forest --chain calibnet --auto-download-snapshot

Interacting with Forest via CLI

When the Forest daemon is started, an admin token will be displayed (alternatively, use --save-token <token> flag to save it on disk). You will need this for commands that require a higher level of authorization (like a password). Forest, as mentioned above, uses multiaddresses for networking. This is no different in the CLI. To set the host and the port to use, if not using the default port or using a remote host, set the FULLNODE_API_INFO environment variable. This is also where you can set a token for authentication.

FULLNODE_API_INFO="<token goes here>:/ip4/<host>/tcp/<port>/http

Note that if a token is not present in the FULLNODE_API_INFO env variable, the colon is removed.

Forest developers will prepend this variable to CLI commands over using export on Linux or its equivalent on Windows. This will look like the following:

FULLNODE_API_INFO="..." forest-cli auth api-info -p admin

The admin token can also be set using --token flag.

forest-cli --token <ADMIN_TOKEN>

Forest executable organization

The binaries in the Forest repository are organized into the following categories:

Binary	Role	Command example
forest	Forest daemon, used to connect to the Filecoin network	forest --chain calibnet --encrypt-keystore false
forest-cli	Human-friendly wrappers around the Filecoin JSON-RPC API	forest-cli info show
forest-tool	Handle tasks not involving the Forest daemon	forest-tool snapshot fetch

Detaching Forest process

You can detach Forest process via the --detach flag so that it runs in the background:

./target/release/forest --detach

The command will block until the detached Forest process has started its RPC server, allowing you to chain some RPC command immediately after.

Forest snapshot links

• calibration network
• main network

Documentation

• Forest Book
• rust doc

Contributing

• Check out our contribution guidelines: CONTRIBUTING.md

ChainSafe Security Policy

Reporting a Security Bug

We take all security issues seriously, if you believe you have found a security issue within a ChainSafe project please notify us immediately. If an issue is confirmed, we will take all necessary precautions to ensure a statement and patch release is made in a timely manner.

Please email a description of the flaw and any related information (e.g. reproduction steps, version) to security at chainsafe dot io.

License

Forest is dual licensed under MIT + Apache 2.0.

forest's Issues

Finish Block CIDs

We can finish the production of a CID for a block once we have CBOR encoding complete. We should compare our encodings to the output of the go-ipld-cbor implementation.

The process can be seen here: https://github.com/filecoin-project/go-filecoin/blob/5ba3b703dc44de3fe647ccc2a3ae62e78d3b47d3/internal/pkg/block/block.go#L65-L88

Basic Blockchain Data Structures

The aim here is to implement as much of the underlying data structures as we can.

Implement BLS aggregate signature verification

Individual signature verification was done in #40 but the aggregate signature verification needs to be implemented when the block's aggregate BLS signature needs to be verified. Should be pretty straightforward to expand logic done in that PR to verify an aggregate but if anyone picks this up before me I can give an example of how to do this.

The reason this isn't being done now is because it is not clear the exact usage or data inputs to be verified. It can be inferred or kept very basic, but since there is no need for this now it will be left as an issue.

Signature verification

We need to add the ability to verify BLS and Secp256k1 signatures.

There are stubs for this in vm/src/message/signature.rs as of #31 , however it might make sense to consolidate crypto operations.

Compute Block Cid

We can start to implement the production of a CID for a block once we have CBOR encoding complete.

The process can be seen here: https://github.com/filecoin-project/go-filecoin/blob/5ba3b703dc44de3fe647ccc2a3ae62e78d3b47d3/internal/pkg/block/block.go#L65-L88

Message Types

The concept of a message has several related types: Signed/Unsigned messages, receipts, input/outputs. This can probably be implemented almost entirely, with the exception of some types that don't exist yet. Many of these types are really aliases, we might want to decide collectively how heavily we want to use aliases.

Spec: https://filecoin-project.github.io/specs/#systems__filecoin_vm__sysactors

Entry Point

We will eventually need a main entry point for the system. This should initialize a Node and start all of its components. This will likely like in the root crate.

Implement Basic TipSet Manager

Defined in the spec as:

The Chain Tips Manager is a subcomponent of Filecoin consensus that is technically up to the implementer, but since the pseudocode in previous sections reference it, it is documented here for clarity.

The Chain Tips Manager is responsible for tracking all live tips of the Filecoin blockchain, and tracking what the current ‘best’ tipset is.

Required:

• fn ticket_from_round // returns the ticket that is at round 'r' in the chain behind 'head'
• fn tipset_from_round // returns the tipset that contains round 'r'
• fn get_best_tipset // returns the best known tipset
• fn add_losing_ticket // adds the losing ticket to the chaintips manager so that blocks can be mined on top of it

I don't exactly know where this functionality lives within go-filecoin but the spec contains details.

Cache Block CIDs

Since we commonly require the CID of a block we can cache it inside the Block instead of generating it on demand. It may also be sensible to cache the CBOR encoding as well, which would also save work when producing the CID.

Inspiration: https://github.com/filecoin-project/go-filecoin/blob/b5dbbc9021c94bac943d18413b5e00844889725b/internal/pkg/block/block.go#L65-L88

Setup and investigate consistent cbor encoding/decoding

Discover how we can consistently encode and decode structs to cbor, and find out if this cbor encoded data would be transfered over the wire to other client implementations to have to match the cbor format for all types.

If a generic encoding and decoding cannot be used, explore standardizing this in a library to be used within the codebase.

Node Components

Required components of a basic node.

Add CLI option for config file

The cli should accept a --config option which points to a .toml file

Add Basic Gossipsub Protocol

TBD

Spec Changelog -- [Last Updated] 11/28/19 09:00am

NOTES:
- This list begins for PR's merged on November 26th, 2019
- The list is based on date merged into specs, not necessarily the order you see on github.
- Every PR will be listed, even if they don't directly matter to Ferret

Glossary

✅ - Changes have been made to Ferret
❌ - Changes have not been made to Ferret

---

On the Horizon

pr #644 - Specify the order of tipset message execution, including implicit messages.

• Changes to gas calculations in the vm_interpreter
• Runtime actors changed

---

pr #656 - Check Pledge Collateral Requirement at PoSt Submission and Pledge Slashing

• Storage Power consensus has major structural changes

---

pr #666 - Add CurrReceiver() to runtime, remove unneeded CurrMethodNum

• Small changes to the runtime

---

Merged

November 28

❌ pr #662 - Specify the order of tipset message execution, including implicit messages.

• Descriptions for tipsets and vm interpreter has changed quite a bit
• Making messages, rewads, gas, and tipsets have changed a lot in the interpretere

---

November 27

❌ pr #658 - Keep time using an int64

• Filecoin Clock no longer uses a string to keep track of time
• Block structure slightly changed because of clock
• Tipset affected by change to clock

---

November 26

❌ pr #652 - Pull Tipset up to Blockchain/Blocks/Tipset

• Blockchain/blocks
• Tipset

Attempt To Bootstrap Filecoin Devnet

Once we have a basic libp2p implementation we should check it against a devnet.

Devnet info: https://docs.filecoin.io/go-filecoin-tutorial/Devnets.html#table-of-contents
Bootnode list: http://user.kittyhawk.wtf:9082/nodes

Address Type

Spec https://filecoin-project.github.io/specs/#systems__filecoin_vm__actor__address

Note: this should be related to state tree

Implement Basic Tipset

This is simply a set of blocks. The blocks must be stored in order of their corresponding ticket. There seems to be a notion of keys in the Go-flecoin implementation, and a separate CID set in Lotus.

Go-filecoin: https://github.com/filecoin-project/go-filecoin/blob/8345455565fb429ebc2eb7c1e3af03c5f7a92b22/internal/pkg/block/tipset.go

Lotus: https://github.com/filecoin-project/lotus/blob/4f9947a27b974512477342ce092381c40a5e8ac9/chain/types/tipset.go

Basic CLI

Once we have an entrypoint, we can start to define a basic CLI for our application. This should support any existing fields required by the other components of the system (eg. bootnodes).

This may require an additional "global" config struct to support parsing of all CLI flags before constructing the sub-configurations required for each system component.

A suggested library is: https://github.com/clap-rs/clap

Add Logger

We should choose and add a library and replace any println!'s we might have.

Node Clock

The clock struture has been finalized. It provides time as well as tracking epochs. The spec calls for multiple formats of UTC time to be provided, this seems a bit unnecessary.

According to the spec we also need to query an NTP server at an interval. This is likely out of scope initially, but we should consider it in the design and track it with further issues.

Spec: https://filecoin-project.github.io/specs/#systems__filecoin_nodes__clock

Setup RewardActor framework

Block reward and token vesting actor

http://localhost:1313/#systems__filecoin_vm__sysactors__reward_actor

basic setup, won't have functional tests until other vm components are built

Add Messages and Receipts to Block

Once implemented we can define the use of these in Block further.

Support Bootstrapping

We should support a user provided set of bootnodes to initialize the connection to the network.

This can be assumed to be provided as a part of the node configuration (#4). This should probably also be configurable through our CLI.

Setup CronActor framework

Actor to run functions at every epoch

http://localhost:1313/#systems__filecoin_vm__sysactors__cron_actor

basic setup, won't have functional tests until other vm components are built

Setup InitActor framework

Actor to initialize new actors

ref: http://localhost:1313/#systems__filecoin_vm__sysactors__init_actor

issue will be to just initialize the framework and connect it with the other implemented pieces

Add Block Topic

The pubsub topic for blocks appears to be /fil/blocks, although Go-Filecoin does seem to append a network name. We can probably do without network name for the time being.

The topic for messages is /fil/msgs, however we don't really need this at this time.

Lotus: https://github.com/filecoin-project/lotus/blob/53de5f804911f04391d7265412fcd3d059316c0c/node/modules/services.go#L59

Go-Fil: https://github.com/filecoin-project/go-filecoin/blob/e81d26c47cc3f205d433ff3ea408c60364bec6dd/internal/pkg/net/topics.go#L7

Setup Network Node Configuration

Trivially, the Network Node will require some sort of config. This will likely contain: host, port, protocol prefix, identity key (or path to), and others.

This might be an opportunity to consider how we propagate config options from the CLI.

Runtime implementation

Framework will be setup in #65 but it will be functionally unusable until a struct that implements the Runtime trait is created which will include all data needed to perform this functionality.

ref: http://localhost:1313/#vm-runtime-implementation

Can use VMContext as used in spec or some other name for the struct needed to implement these

Create TOML Config

User configs will be provided via a TOML file, through the cli flag --config.

The current requirements for this are mostly related to libp2p config (host, port, protocol ID, ... (?)).

Research Clock Sync

The nodes clock is expected to be sync'd every hour using an NTP server (pool.ntp.org recommended). More research is required into how we can do this.

http://localhost:1313/#clock-requirements (e9b36c7)

Initial VM Components

Initial implementation of the types VM depends on.

Initial Libp2p Node

We need to establish basic functionality for a libp2p node that provides all services we require (Kad DHT, Gossipsub, ...)

Initial Node Structure

We should add a containing struct for the root level Node. The entrypoint to our application would create an instance of this.

It might be useful to allow other types of node to be defined ontop of a base type (eg. Chain Verifier, Relayer Node).

Implement Basic Block and Header

The block/header structure appears to be well defined. We are missing some types, these should be stubbed (add type or use existing). We can probably open issues for some missing types.

Spec: https://filecoin-project.github.io/specs/#systems__filecoin_blockchain__struct__block
Go-filecoin: https://github.com/filecoin-project/go-filecoin/blob/b91042205bd3ad804c4a523310e562e143687b75/internal/pkg/block/block.go
Lotus: https://github.com/filecoin-project/lotus/blob/4f9947a27b974512477342ce092381c40a5e8ac9/chain/types/blockheader.go

Additional System Actor Types

Current spec:

- There are two system actors required for VM processing:
        CronActor - runs critical functions at every epoch
        InitActor - initializes new actors
- There is one more VM level actor:
        AccountActor - for user accounts.

Account Actor is further defined here: https://filecoin-project.github.io/specs/#systems__filecoin_vm__sysactors__account_actor

TBC...

Implement ChainEpoch type and epoch_at_time method

Is very basic, just need an u64 type (or whatever is decided) either wrapped or aliased (depends on how it's needed) just to have consistent types and allows it to be changed if needed in future.

epoch_at_time function implemented as well, which is a function on ChainEpochClock type, there is a TODO already in the code where it needs to be implemented and is very straightforward in the spec.

Also noticed the get_time method and this doesn't seem like this would ever be used and if so, the name should be changed to get_genesis_time(..) or something to not be ambiguous.

State Tree Interface

it seems some aspects of the state tree are unclear at this time. However, we could start an interface for the VM to interact with. Once finalized we can implement this interface using the underlying storage.

Clock seperation

We should consider adding different types of clocks:

• Testing clock
  • Has functionality to go forward, and backwards (time manipulation).
  • ?
• production clock
  • Used in testnets and production
  • Does not have time manipulation

Create Ticket Type

Contains VRF (VDF?) output.

Need ability to compare Tickets based on specification.

Store Libp2p ID

We should allow for persisting a libp2p ID between restarts.

Setup basic VM Interpreter

Create framework for VM interpreter for message invocation. This will handle the messages passed into the VM and perform the state transition. Relies on #11 but can be stubbed out before

ref: http://localhost:1313/#systems__filecoin_vm__interpreter or https://filecoin-project.github.io/specs/#systems__filecoin_vm__interpreter

Add Kademlia DHT Discovery

Our primary discovery mechanism in the libp2p node should be the Kademlia protocol.

Docs: https://docs.rs/libp2p-kad/0.13.1/libp2p_kad/

Create and setup makefile

Once usage and interactions start getting set up, would be nice to have a makefile to run workflows such as building, linting, testing, cleaning with any config (logging or otherwise)

I would suggest maybe leaving this until we have a consistent way to run the node, but if anyone is eager this would still be nice to have now.

Implement Basic SyncManager

We require some mechanism to manage queues, incoming/outgoing targets, and syncing states.

In Lotus this handled by the SyncManager:

• SyncManager

In Go-Filecoin I believe this is managed by the Dispatcher:

• Dispatcher

SyncManager:

• Define required sync manager fields and types
• Implement SyncManager struct
• Implement SyncManager constructor

Setup encoding package for hashing and serialization

Standardization of traits and errors from anywhere in the code. Will include basic utility functions which are generally used, but specific cbor encoding implementations will come with #35

Setup AccountActor framework

Actor which resembles user accounts

http://localhost:1313/#systems__filecoin_vm__sysactors__account_actor

basic setup, won't have functional tests until other vm components are built

Complete EpochAtTime

The spec seems to have a bit more definition now: http://localhost:1313/#systems__filecoin_nodes__clock

Basic Actor

An initial actor type should be implemented for use by the VM. It migth be worth considering that there a some "special" system actors

Spec: https://filecoin-project.github.io/specs/#systems__filecoin_vm__actor__address

Setup Networking Config

Enable a libp2p start command from the CLI, and add additional libp2p fields to the Config. Should consider moving the Default networking config impl to the CLI

Set up runtime framework

docs ref: https://filecoin-project.github.io/specs/#systems__filecoin_vm__runtime or http://localhost:1313/#systems__filecoin_vm__runtime

Issue to create trait interface for the runtime to be able to be used inside the interpreter for the state transition.