Compacting interrupts reading/writing? about sonnerie

[njaard]

Fixed in commit 322e423

[ettom]

Similarly, doing

while :; do sonnerie -d db read % || break; done

on a database with about a 1000 rows and then running

sonnerie -d db compact

will result in the loop stopping with Error: Os { code: 2, kind: NotFound, message: "No such file or directory" }.

[njaard]

What operating system are you on?

[ettom]

Arch Linux. I also tried this on a custom LFS-like embedded Linux system and got the same results. sonnerie v0.8.5 and ext4 for both.

[ettom]

Interesting, this doesn't seem to happen on a Linux Mint 21 machine with ext4. On that machine the compaction loop just says disregarding "db/tx.*", it is zero length on every iteration.

[ettom]

Interesting, this doesn't seem to happen on a Linux Mint 21 machine with ext4. On that machine the compaction loop just says disregarding "db/tx.*", it is zero length on every iteration.

Scratch that, it still happens, but a lot less often. Easy to catch it with

for i in $(seq 1 100000); do echo $i; echo "test $i $i" | sonnerie -d db add --format u || break; done

[njaard]

It seems like there's a race condition in the time between the completion of read_dir and the actual opening of the files. If a compaction completes in between those two actions, then some files that were previously found via the read_dir will have been deleted by the compaction.

If that happens, then I think the correct action is to try the read_dir again from the beginning. You can't simply ignore the missing files, because your read_dir necessarily will not have opened the file they'll have been compacted into.