Semi-Sync Replication breaks if initial binlogs have been purged about mariadb-operator

[svenpecher]

Actually my issue here seems to pretty much the same as the unsolved but closed issue #141

[svenpecher]

As I understand the ResetSlavePos happens during ConfigureReplica which seems to be executed during the following procedures:

1. reconcilePodReady
2. reconcileReplicas
3. connectReplicasToNewPrimary
4. changeCurrentPrimaryToReplica

For 3. and 4. it seems right to reset gtid_slave_pos.

For 1. and 2. this is a bit unexpected: I would assume that here each replica has a certain replication state (gtid_slave_pos), which gets lost here. From my understanding here it rather makes sense to keep the current position without overwriting it.

@mmontes11 what do you think about that?

[mmontes11]

Hey there @svenpecher ! Thanks for bringing this up and your detailed analysis.

This is indeed related to the following issue:

• #141

We will be implementing some replication enhancements in this release, which include choosing the furthest advanced replica in the primary switchover, see:

• #193
• #209

Ideally, I would like to avoid resetting gtid_slave_pos where possible to minimize this kind of situations like the one you report. Your suggestion makes sense, it's like a middleground, so I will have it in mind by the time I'm implementing this.

To give you some context, this reset with timeout was introduced because we've faced situations where the replication was stucked waiting for replicas be synced. In this step specifically:

mariadb-operator/pkg/controller/replication/switchover.go

Line 54 in 6ba1c80

reconcile: r.waitForReplicaSync,

Switching to the furthest advanced replica might help with this? Maybe we don't need to wait for the furthest advanced replica to be sync with primary?

Anyway avoid the reset it in the steps 1. and 2. does sound sensible.