Comments (4)
@ongardie Should a server wait until receiving an AppendEntries RPC (to receive the commitIndex
) from the current leader before applying commands in the log? Or should the committed index be saved to disk?
from raft.
To make sure everyone's clear, persisting commitIndex isn't needed for safety, since a new leader can always figure this out again with help from a quorum, and then tells everyone else through AppendEntries RPCs. I assume the question is when/whether it's beneficial to persist the commitIndex to disk. You can certainly do it, and as xiangli points out, you can do so asynchronously or periodically. But here are a couple reasons why you may not want to:
- It's extra code.
- If you're booting a server, hopefully it's in the minority of your cluster that's not needed for availability.
- If you're booting a server, it's probably already experienced significant downtime. If you delay applying log commands even by a few minutes, it wouldn't necessarily add significantly more downtime.
- If you need to read the log from a magnetic disk, you're lucky if you can get 100MB/s. Applying the commands should be much faster (a few hundred MB/s probably), so Amdahl's law says you can't expect much gain by overlapping the two. A good SSD might change this, though.
So I guess the most super-optimized implementations would do this, but I probably wouldn't bother.
from raft.
@ongardie I understand that it's not needed for safety. I guess my biggest concern was if the entire cluster goes down and then reboots then every node is waiting for an AE. I suppose it could wait for an election timeout before trying to replay the log. That way you wouldn't get delayed by replaying first if it's not needed.
from raft.
fixed.
from raft.
Related Issues (20)
- Improve test coverage
- Data loss / corruption due to reading partial data from log HOT 2
- Heartbeat lost blocks join requests HOT 2
- Broken initialization after loading from a snapshot HOT 1
- any discussion forum? HOT 1
- server.Start/Stop/Init cannot be called simultaneously
- Windows: log compaction problems HOT 2
- last activity of killed leader is not updated
- Any plan to integrate with different log facilities? HOT 1
- Build broken due to protobuf changes HOT 10
- [question] What is the recommended way to kick unreachable nodes?
- Link in README to 'In Search of an Understandable Consensus Algorithm' is dead HOT 3
- server loads old snapshots HOT 6
- Locking while taking a snapshot HOT 8
- raftd sample cluster reaches un-startable state: panic: assertion failed: leader.elected.at.same.term.345 HOT 2
- chore: use code in github.com instead of code.google.com HOT 4
- panic: assertion failed: leader.elected.at.same.term.283 HOT 2
- Please link to alternative Raft implementations HOT 2
- goraft seems not deal with read consistency problem HOT 3
- Getting package error
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from raft.