raft: relax fsync requirements about cockroach

[blathers-crl]

cc @cockroachdb/replication

[pav-kv]

There are probably some nuances here.

The replica can't be fully "inactive" while it learns the state. Consider a scenario in which all 3/3 replicas restarted simultaneously. They should be able to learn even though there is no active leader, and no replica transitioned to voter yet. One way to make it work is that all replicas are learners and readable by other replicas (kind of "read-only voter"), so that everyone can bootstrap from a quorum. This scenario is when the risk of losing state is the highest: if there is no quorum with up-to-date state, everyone will bootstrap stale. This risk exists today too, if fsync is disabled.

How does this work with config changes? What if a starting up replica regressed to a pre-config-change state? Config changes modify quorums, and this should be reflected in the recovery procedure. Potentially, config changes are one of few cases when durability guarantees must be stronger.