Strange situation after leader outage about nuraft

[greensky00]

Hi @alesapin
Seems like the leader election was successfully done quickly (node 3 became the leader within 1 second).

Node 3:

2021.02.04 22:45:54.578642 [ 63 ] {} <Warning> RaftInstance: Server is elected as leader for term 2

Node 2:

2021.02.04 22:45:54.577455 [ 22 ] {} <Information> RaftInstance:   FOLLOWER

The message that Node 3 was continuously sending was heartbeat; in Raft, there is no separation between heartbeat and append_entries request (please refer to paper, Figure 2 at page 4). It sent the log 152 (the last log it had at that time) at first, and then Node 2 accepted and stored it. After that, Node 3 sent an empty message as a heartbeat, with the info of the last log index and committed log index, which was 152 too.

You can see the Node 3's log that it sent log 152,

2021.02.04 22:45:54.579249 [ 63 ] {} <Notice> RaftInstance: append_entries for 2 with LastLogIndex=151, LastLogTerm=1, EntriesLength=1, CommitIndex=151, Term=2, peer_last_sent_idx 0

and after that, it sent an empty message (EntriesLength=0), with the last log/commit index info.

2021.02.04 22:45:54.581362 [ 31 ] {} <Notice> RaftInstance: append_entries for 2 with LastLogIndex=152, LastLogTerm=2, EntriesLength=0, CommitIndex=152, Term=2, peer_last_sent_idx 152

10 minutes later, Node 3 received a new client request, and appended log 153.

2021.02.04 22:55:59.558464 [ 22 ] {} <Notice> RaftInstance: Receive a client_request message from 0 with LastLogIndex=0, LastLogTerm=0, EntriesLength=1, CommitIndex=0 and Term=0
2021.02.04 22:55:59.558599 [ 22 ] {} <Notice> RaftInstance: append at log_idx 153

So everything looks normal (at least in Raft's side). The thing is that Node 3 didn't receive any client request for 10 minutes.

[alesapin]

Thanks!