Fix MV dtests with tablets enabled about scylladb

[nyh]

As I suspected, test_repair_mv (and probably the other repair tests listed above) assumes that because they used RF=3 N=3, one can read from the view right after finishing to write to the base and be able to read everything. With tablets, this assumption is no longer correct, so the test sometimes, depending on timing, fails (it also sometimes succeeds, unsurprisingly).

The fix can to either enable synchronous_updates on the view in this test, or to add the missing "eventually".
It's easier for me to just add synchronous_updates - and it will also be less risky in the sense that I might miss one of the reads that needs an "eventually".

The downside of synchronous_updates is that will not work on Cassandra, but I don't think we ever cared and don't need to start caring now (these tests would have failed on Cassandra too because of the missing eventually).

UPDATE: The state of this test is even worse than I hoped. It didn't just assume that view updates are synchronous - even stranger, it assumed that it's possible to write with the default CL (1) and then read with the default CL (1). I don't know how this ever worked. Maybe it is more likely to work on overloaded test machines than on mine. I think I'll need to add eventually() instead of fixing every weirdness in this test separately.

UPDATE#2: Even with eventually, the test sometimes fails. I'm not sure why :-( Need to investigate further.

[mykaul]

Isn't synchronous view a different feature that needs to be tested, and 'eventually' is the 'default' mode we have and need to support?

[nyh]

Isn't synchronous view a different feature that needs to be tested, and 'eventually' is the 'default' mode we have and need to support?

Yes, sort of. In theory, this test was supposed to be about repair, so could have used other features freely, and moreover the part of the test which fails is the setup - it's not even the real part of the test. But you're right that it's better to not depend on synchronous view updates if we don't have to. I won't be using it anyway, I switched to using eventually.

[nyh]

The test_repair_mv failure turns out to be cause by a known (but previously undocumented in an issue) implication of the disabling of self-pairing that we did with MV+tablets (commit 300e549). I created an issue to describe this implication and its possible solution in the future - #17043. But in the short term, the test can and should be fixed to not do CL=1 reads where they are not guaranteed to be enough. I'll send a fix for that shortly.