This repository contains a Rust implementation of the following distributed oracle agreement protocols.
- Delphi AAA protocol
- FIN ACS protocol [1]
- Abraham et al. AAA protocol [2]
The repository uses the libchatter networking library available here. This code has been written as a research prototype and has not been vetted for security. Therefore, this repository can contain serious security vulnerabilities. Please use at your own risk.
Please consider citing our paper if you use this artifact.
Delphi: Efficient Asynchronous Approximate Agreement for Distributed Oracles
Akhil Bandarupalli, Adithya Bhat, Saurabh Bagchi, Aniket Kate, Chen-Da Liu-Zhang, and Michael K. Reiter
To appear at 54th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), 2024.
The repository also contains a dataset containing values of prominent cryptocurrencies polled from 12 cryptocurrency exchanges. Details are available in the dataset
folder.
We describe the steps to run this artifact.
-
This artifact has been run and tested on
x86_64
andx64
architectures. However, we are unaware of any issues that would prevent this artifact from running onx86
architectures. -
This artifact has been run and tested on Ubuntu
20.04.5 LTS
OS and Raspbian Linux version released on2023-02-21
, both of which follow the Debian distro. However, we are unaware of any issues that would prevent this artifact from running on Fedora distros like CentOS and Red Hat Linux.
The repository uses the Cargo
build tool. The compatibility between dependencies has been tested for Rust version 1.63
.
- Run the set of following commands to install the toolchain required to compile code written in Rust and create binary executable files.
$ sudo apt-get update
$ sudo apt-get -y upgrade
$ sudo apt-get -y autoremove
$ sudo apt-get -y install build-essential
$ sudo apt-get -y install cmake
$ sudo apt-get -y install curl
# Install rust (non-interactive)
$ curl --proto "=https" --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
$ source $HOME/.cargo/env
$ rustup install 1.63.0
$ rustup override set 1.63.0
- Build the repository using the following command. The command should be run in the directory containing the
Cargo.toml
file.
$ cargo build --release
$ mkdir logs
- Next, generate configuration files for nodes in the system using the following command. Make sure to create the directory (in this example,
testdata/hyb_4/
) before running this command.
$ ./target/release/genconfig --base_port 8500 --client_base_port 7000 --client_run_port 9000 --NumNodes 4 --blocksize 100 --delay 100 --target testdata/hyb_4/ --local true
- After generating the configuration files, run the script
appxcon-test.sh
in the scripts folder with the following command line arguments. This command starts Delphi with four nodes.
$ ./scripts/appxcon-test.sh {epsilon} {rho} {Delta} testdata/hyb_4/syncer
-
Substitute desired values of
$\epsilon,\rho_0,\Delta$ . Example values include$\epsilon=1,\rho_0=10,\Delta=100000$ . The script randomly assigns input values$v_i$ to each node. This logic can be changed to make nodes start with custom input values. -
The outputs are logged into the
syncer.log
file in logs directory. The outputs of each node are printed in a JSON format, along with the amount of time the node took to terminate the protocol. -
Running the FIN ACS protocol requires additional configuration. FIN uses BLS threshold signatures to generate common coins necessary for proposal election and Binary Byzantine Agreement. This setup includes a master public key in the
pub
file,$n$ partial secret keys (one for each node) assec0,...,sec3
files, and the$n$ partial public keys aspub0,...,pub3
files. We utilized thecrypto_blstrs
library in the apss repository to generate these keys. We pregenerated these files for$n=16,64,112,160$ in the benchmark folder, in zip filestkeys-{n}.tar.gz
. After generating these files, place them in the configuration directory (testdata/hyb_4
in this example) and run the following command (We already performed this step and have these files ready intestdata/hyb_4
folder).
# Kill previous processes running on these ports
$ sudo lsof -ti:7000-7015 | xargs kill -9
$ ./scripts/fin-test.sh testdata/hyb_4/syncer
- Similarly, Abraham et al.'s Approximate Agreement protocol can be run using the following command.
# Kill previous processes running on these ports
$ sudo lsof -ti:7000-7015 | xargs kill -9
$ ./scripts/abraham-test.sh {epsilon} {delta} {Delta} testdata/hyb_4/syncer
The parameters {epsilon} and {delta} must be equal in this context to yield Abraham et al.'s protocol. {Delta} must be set to be equal to the difference between honest inputs M-m
. Example configuration run includes the following command.
$ ./scripts/abraham-test.sh 2 2 20 testdata/hyb_4/syncer
We utilize the code in the Narwhal repository to execute code in AWS. This repository uses fabric
to spawn AWS instances, install Rust, and build the repository on individual machines. Please refer to the benchmark
directory for more instructions about reproducing the results in the paper.
We described detailed instructions to reproduce the results in the paper in the Raspberry-Pi device testbed in the benchmark/raspberry-pi
directory.
Each node runs as an independent process, which communicates with other nodes through sockets. Apart from the syncer
. The syncer
is responsible for measuring latency of completion. It reliably measures the system's latency by issuing START
and STOP
commands to all nodes. The nodes begin executing the protocol only after the syncer
verifies that all nodes are online, and issues the START
command by sending a message to all nodes. Further, the nodes send a TERMINATED
message to the syncer
once they terminate the protocol. The syncer
records both start and termination times of all processes, which allows it to accurately measure the latency of each protocol.
The artifact uses multiple Rust libraries for various functionalities. We give a list of all dependencies used by the artifact in the Cargo.lock
file. Cargo
automatically manages these dependencies and fetches the specified versions from the crates.io
repository manager.
The artifact is organized into the following modules of code.
-
The
config
directory contains code pertaining to configuring each node in the distributed system. Each node requires information about port to use, network addresses of other nodes, symmetric keys to establish pairwise authenticated channels between nodes, and protocol specific configuration parameters like values of$\epsilon,\Delta,\rho$ . Code related to managing and parsing these parameters is in theconfig
directory. This library has been borrowed from thelibchatter
(https://github.com/libdist-rs/libchatter-rs) repository. -
The
crypto
directory contains code that manages the pairwise authenticated channels between nodes. Mainly, nodes use Message Authentication Codes (MACs) for message authentication. This repo manages the required secret keys and mechanisms for generating MACs. This library has been borrowed from thelibchatter
(https://github.com/libdist-rs/libchatter-rs) repository. -
The
crypto_blstrs
directory contains code that enables nodes to toss common coins from BLS threshold signatures. This library has been borrowed from theapss
(https://github.com/ISTA-SPiDerS/apss) repository. -
The
types
directory governs the message serialization and deserialization. Each message sent between nodes is serialized into bytecode to be sent over the network. Upon receiving a message, each node deserializes the received bytecode into the required message type after receiving. This library has been written on top of the library fromlibchatter
(https://github.com/libdist-rs/libchatter-rs) repository. -
Networking: This repository uses the
libnet-rs
(https://github.com/libdist-rs/libnet-rs) networking library. Similar libraries include networking library from thenarwhal
(https://github.com/MystenLabs/sui/tree/main/narwhal/) repository. The nodes use thetcp
protocol to send messages to each other. -
The
tools
directory consists of code that generates configuration files for nodes. This library has been borrowed from thelibchatter
(https://github.com/libdist-rs/libchatter-rs) repository. -
The
consensus
directory contains the implementations of various protocols. Primarily, it contains implementations of Abraham et al.'s approximate agreement protocol in thehyb_appxcon
subdirectory,delphi
protocol in thedelphi
subdirectory, and FIN protocol infin
subdirectory. Each protocol contains acontext.rs
file, which contains a function namedspawn
from where the protocol's execution starts. This function is called by thenode
library in thenode
folder. This library contains amain.rs
file, which spawns an instance of a node running the respective protocol by invoking thespawn
function.
[1] Abraham, Ittai, Yonatan Amit, and Danny Dolev. "Optimal resilience asynchronous approximate agreement." In Principles of Distributed Systems: 8th International Conference, OPODIS 2004, Grenoble, France, December 15-17, 2004, Revised Selected Papers 8, pp. 229-239. Springer Berlin Heidelberg, 2005.
[2] Duan, Sisi, Xin Wang, and Haibin Zhang. "Fin: Practical signature-free asynchronous common subset in constant time." In Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security, pp. 815-829. 2023.