tsumida / hotstuff-consensus

hotstuff-consensus

Intro

hotstuff-consensus is a Rust implementation of https://dl.acm.org/doi/10.1145/3293611.3331591. This project is under development. Since it's buggy, more tests and refactoring are needed.

Arch

• HotStuff Proxy: Network event handler and deliverer. Currently the proxy use tarpc to implement reliable event singlecast and broadcast.
• Machine: Implemetation of event-driven hotstuff algorithm.
• Pacemaker: Maintains liveness. Pacemaker consists of 3 components:
  • Timer: Setting and cancelling view timer.
  • RoudRobin Elector: mapping view to leader.
  • Branch synchronizer: collecting all missing proposals. In current implemetation it works when it receive a proposal with an unknown proposal.justify.node and then require leader deliver all missing blocks.
• Signaturer: Verifying, signing and combining partial signatures. This module is stateless and used by Machine, Pacemaker and HotStuffStorage.
• HotStuff Storage: Shared in-memory storage with a persistent backend(currently based on MySQL). Pacemaker and Machine access HotStuff Storage byLivenessStorage and SafetyStorage respectively.

hotstuff-consensus's Issues

Investigation: tonic and grpc

tonic

https://github.com/hyperium/tonic

proto-buf

https://developers.google.com/protocol-buffers/docs/overview

grpc

https://www.grpc.io/

usage & demo

• demo

commit_height didn't increase after committing proposal

Detailed description:
Machine.storage.commit_height is 0 and doesn't increase after committing proposal.

Steps to reproduce:

#[test]
fn test_consecutive_commit(){
    // init_node <- a1 <- a2 <- a3 <- a4 <- a5 <- a6 <- a7
    //                      
    //  a4 is committed and a5 is locked
    let n = 4;
    let f = 1;
    let leader = 0;
    let testee = 1;
    let mut mhs = MockHotStuff::new(n);

    mhs.specify_leader(leader)
        .specify_testee(testee)
        .init();

    mhs.load_continue_chain(vec![
        format!("a1"), 
        format!("a2"), 
        format!("a3"), 
        format!("a4"), 
        format!("a5"), 
        format!("a6"), 
        format!("a7"), 
    ]);

    mhs.check_with_expected_state(&ExpectedState::CommittedBeforeHeight(4));  // failed here
} 

Expected behavior:
mhs.check_with_expected_state(&ExpectedState::CommittedBeforeHeight(4)); pass.
Nodes with height <= 4 are all committed.

Environment:

• Rust 1.47

Additional context:

Three way for arch of hotstuff-consensus

Way 1: Machine as Pacemaker's field

Like:

struct Pacemaker{
    m: Machine<HotStuffStorage>, 
}

goodness

• reuse existing code
• no mutex.

shortcomings

• Pacemaker can't process other events until Machine return . Since most of the events needs stablization, this change should have little impact on performance.
• The size of async statemachine increases (I guess).
• medium change.

todo

• remove AsyncMachineWrapper
• may need to separate flusher
• exposing storage to pacemaker for processing Timeout events.

Way 2: PM & M stay running independently

HotStuffStorage needs a wrapper like Arc<Mutex<HotStuffStorage>.

shortcomings

• One mutex for all. Each method needs aquire mutex first.
• Medium change.

todo

• impl LivenessStorage & SafetyStorage for Arc<Mutex>.

Way 3: Split HotStuffStorage into 2 part

Separate HotStuffStorage into SafetyState and LivenessState, both of them are protected by Arc<Mutex<..>. InMemState ， shared by both hss, M and PM, saves states like qc-high, locked_node.
Also there are different ways to organize them:

		HSS  ------> Arc<Mutex<SafetyState>>     <---------------- M
		|		↓
		|  -----> Arc<Mutex<InMemState>>
		| 		↑
		------------> Arc<Mutex<LivenessState>>  <--------------- PM


		HSS -------> Arc<Mutex<SafetyState(InMemState)>>     <---- M
		|		↑
		|  	       	↑
		| 		↑
		------------> Arc<Mutex<LivenessState>>  	<-------- PM

shortcomings

• More mutex, some method need to aquire two mutex.
• Big change.

todo

• Split HotStuffStorage into different parts and modify implementation.