Support backup of files changing during backup time about rdiff-backup

[afoster]

Hi @honzasram

According to the rdiff-backup FAQ:

If rdiff-backup fails with the message "AssertionError: Bad index order," it could be because the files in a directory have changed while rdiff-backup is running. Possible ways of dealing with this situation include implementing filesystem snapshots using the volume manager, excluding the offending directory, or suspending the process that is changing the directory. After the text "Bad index order", the error messge will indicate which files have caused the problem.

I've also heard that it may be an indication of a failing disk or partially corrupted filesystem.

I think the best approach is probably to post on the mailing list until we can determine if it's a bug. I'll keep this issue open for a bit.

• Andy

[andrewufrank]

the same bad index order error occured with my backup when i was trying to add a new increment. the initial backup completed without error message on stdout/stderror and session statistics did not indicate any problem. the next backup (or --check-destination-dir) did then reveal the bad index order error, which seem not to be recoverable. i have run fsck with -cc to check the disk completely - it is no likely that this is hardware problem of the disk; it is, however, possible, that a file was changed while the backup was running (which should not cause a serious error and easily recoverable).
i consider this a damaging problem:

• errors in backup must be detected during the backup and reported then (assuming the error was during the completed backup process)

OR

• errors in a new backup can be thrown out by unwinding the new backup.

OR

• there is a published procedure to recover from the "Bad index order" error.

the problem is serious because i end with an incomplete backup which is not usable anymore, detected only later.
i run debian jessie on armhf with kernel 4.9. and the current rdiff-back from debian 1.2.8-7).

i hope you can indicate a method to recover from 'bad index order' and publish it, in case I or others encounter the same problem again.
thank you for picking up maintenance of rdiff-backup - it is still one of the most reliable backup systems.
andrew

[ericzolf]

This is a current limitation of rdiff-backup that it can't handle well files changing while they are backed-up. We can look into this but that might not be trivial and I would consider this as an enhancement.

[ottok]

Hello @honzasram!

The code base has been rewritten for Python 3 and there will be no further releases for the 1.x series in Python 2 anymore. Could you please help out and test again with the latest development version (https://github.com/rdiff-backup/rdiff-backup#installing-the-latest-development-version) if you can still reproduce this?

With your help, the upcoming 2.0.0 release could be improved and this reported issue eventually solved in future 2.x releases.

[honzasram]

Hi @ottok,

Unfortunately I abandon that solution and resolved it different way.
Exact setup was backup server with mounted SMB disk and running locally. That way, users could easily change data during backup. Since it was order of TB, it took long time to process and users could interfere with backup process. At least that was my understanding of problem.

Cheers.

[ericzolf]

OK, then let's close this bug as we wouldn't solve it any time soon anyway.