Bug: Synced file is removed when updated on the remote while being processed by onedrive about onedrive

[abraunegg]

@jtomkiew
Potentially resolved with onedrive v2.5.0-rc2-4-ge1e35fa

Please can you test onedrive v2.5.0-rc2

The instructions to do so can be found here: #2710

[abraunegg]

@jtomkiew
Essentially what you are calling out here is a potential race condition with v2.4.25. Unfortunately for you and where client development is at, this issue will not be fixed in v2.4.25. I will not look at it for resolution.

The resolution path however for you is to upgrade to v2.5.0-rc1. Please read in detail the client architecture documentation for v2.5.0 to evaluate if your use case above is taken care of, as this version has been a 100% total re-write including all logic handling.

The specific client architecture documentation can be found here: https://github.com/abraunegg/onedrive/blob/onedrive-v2.5.0-release-candidate-1/docs/client-architecture.md

Post reading this document, what I suggest is that you upgrade your client to v2.5.0-rc1 and re-perform your testing using this client version.

To upgrade to v2.5.0-rc1 will require you to manually compile the client for your system. Please read:

• #2662
• #2661

Please perform all your testing again with v2.5.0-rc1.

[abraunegg]

@jtomkiew
Thanks - will investigate further once I complete the items I am working on at present. It should be resolved for RC2 or GM.

[abraunegg]

@jtomkiew
FYI - now that v2.5.0-rc2 is out, I need to work out what is going on with the Docker builds (something has changed) and then I will be looking into this issue.

[abraunegg]

@jtomkiew

I was wondering if it is possible to get file version information from the API to determine if we need to duplicate local file or not (i.e. if file has the same hash and modification date as one of the previous versions, then we can just overwrite local file), but this is just some food for thought.

The client already does this taking into account timestamp, size and hash of the file, before determining if the local file needs to be preserved.

Closing this issue as original issue is resolved.

[jtomkiew]

Thanks @abraunegg, I'll verify this on 2.5.0-rc1 and let you know.

[abraunegg]

@jtomkiew
Any update on your testing and/or investigations?

[jtomkiew]

I just did the test (sorry for the delay) and it seems to behave the same way in this scenario:

No user or system config file found, using application defaults
Using 'user' configuration path for application state data: /home/user/.config/onedrive
Using IPv4 and IPv6 (if configured) for all network operations
Checking Application Version ...
Attempting to initialise the OneDrive API ...
Configuring Global Azure AD Endpoints
The OneDrive API was initialised successfully
Opening the item database ...
Application Version:  onedrive v2.5.0-rc1-36-g0f012b9
Account Type:         personal
Default Drive ID:     567e5894635bb906
Default Root ID:      567E5894635BB906!101
Remaining Free Space: 5.00 GB (5368666674 bytes)
Sync Engine Initialised with new Onedrive API instance
All application operations will be performed in the configured local 'sync_dir' directory: /home/user/OneDrive
Fetching /delta response from the OneDrive API for Drive ID: 567e5894635bb906
Processing API Response Bundle: 1 - Quantity of 'changes|items' in this bundle to process: 5
Finished processing /delta JSON response from the OneDrive API
Processing 4 applicable changes and items received from Microsoft OneDrive
Processing OneDrive JSON item batch [1/1] to ensure consistent local state
Number of items to download from OneDrive: 1
ERROR: File download size mis-match. Increase logging verbosity to determine why.
ERROR: File download hash mis-match. Increase logging verbosity to determine why.
INFO: Potentially add --disable-download-validation to work around this issue but downloaded data integrity cannot be guaranteed.
Removing file Documents/Obsidian/TheFile.md due to failed integrity checks
Downloading file: Documents/Obsidian/TheFile.md ... failed!
Performing a database consistency and integrity check on locally stored data
Processing DB entries for this Drive ID: 567e5894635bb906
Processing: ~/OneDrive

# snip #

Processing: Documents/Obsidian/TheFile.md
The file has been deleted locally
Deleting item from OneDrive: Documents/Obsidian/TheFile.md
Scanning the local file system '~/OneDrive' for new data to upload
Performing a last examination of the most recent online data within Microsoft OneDrive to complete the reconciliation process
Fetching /delta response from the OneDrive API for Drive ID: 567e5894635bb906
Processing API Response Bundle: 1 - Quantity of 'changes|items' in this bundle to process: 5
Finished processing /delta JSON response from the OneDrive API
Processing 3 applicable changes and items received from Microsoft OneDrive
Processing OneDrive JSON item batch [1/1] to ensure consistent local state

Failed items to download from OneDrive: 1
Failed to download: Documents/Obsidian/TheFile.md

Sync with Microsoft OneDrive has completed, however there are items that failed to sync.
To fix any download failures you may need to perform a --resync to ensure this system is correctly synced with your Microsoft OneDrive Account

Waiting for all internal threads to complete before exiting application

[jtomkiew]

@abraunegg
It does fix the issue indeed - one small side effect is that it will duplicate the local file on the next sync (The local item is out-of-sync with OneDrive, renaming to preserve existing file and prevent local data loss), but this is fine to manage as a manual clean up.

I was wondering if it is possible to get file version information from the API to determine if we need to duplicate local file or not (i.e. if file has the same hash and modification date as one of the previous versions, then we can just overwrite local file), but this is just some food for thought.

Feel free to close this issue at your leisure. Thanks!

[abraunegg]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.