Comments (5)
I'm trying this small change to DiffLists. The issue here is that comparing mod times between OSX files and Android device files doesn't line up. I'm having to munge the times to get the comparison to skip files already copied over previously. I tried snapping to minutes and then adding 16. So how can this sync ever work, if the timestamps don't have even a remote correspondence. I tried with --times and without.
if a_item[0] == b_item[0]:
# if a.mtime is greater, then want in a_only to do the copy
if a_item[1].st_mtime > b_item[1].st_mtime:
a_only.append(a_item)
else:
both.append((a_item[0], a_item[1], b_item[1]))
a_revlist.pop()
b_revlist.pop()
Modification timestamps seems to be an underlying flaw in the Android file system.
https://issuetracker.google.com/issues/36930892#c128
And some circular references back to that unresolved issue, but more of a description of the problem.
https://www.xda-developers.com/diving-into-sdcardfs-how-googles-fuse-replacement-will-reduce-io-overhead/
It's clear from --times still being broken in the release buld that timestamp comparisons in adb-sync were never tested/used.
from adb-sync.
Also seems like there's an "adb shell stat -c "%y" filename" command that might return more precise timings than "ls -al" which only shows timings to minutes. But this would have to get called for each filename. At least here the seconds are available.
2020-11-25 00:16:05.000000000
There's also "ls -ll" to get more accurate timestamps so they don't need to be rounded, so I'm going to try using that.
adb shell ls -all
drwxrwx--x 14 u0_a184 sdcard_rw 3488 2020-11-25 08:57:14.000000000 -0800 .
from adb-sync.
I rewrote the timestamp parsing to use microseconds by cropping the nanos returned by ls -ll, and also using the UTC timestamp returned. I stoped using --times since it only sets low resolution timings via "adb touch" and it's slow, and the "adb push" uses to emulate copy already preserves modstamps.
Then I added the modstamp tests on non-directories mentioned above, and fixed the "both" logic to skip copies off a newer modstamp. This all finally got adb-sync to sync files properly to the device.
from adb-sync.
Since it's an underpinning of adb-sync, I looked into the behavior of adb push by itself on and Android 9 device and macOS. My understanding is that macOS emulates the Android file system where Linux/Win have native implementations.
Looks like "adb push" if you pull the cable to the device in the middle of any push leaves a partial file with the current modstamp set as well. This means you have a partial copy of the file on the device, and since adb sync only checks modstamps, then it breaks our builds.
Also if you push, touch a file, push again and cancel the push then the file is deleted. Wasn't expecting that either. Thought that the file system would still have that original file. push and cancel seems to not leave a file on the device either.
Considering also adding a size check since the "ls -ll" command includes those too. At least it may be a checksum if there is correspondence between file sizes, and they're not thrown off by pagesize on the two file systems.
from adb-sync.
We ended up dumping adb-sync except for excruciatingly slow deletes that are done one file at a time even if all the files under a folder are deleted. The general idea is as follows, and this is crazy fast and more safe than adb-sync.
# First use "adb sync" to copy newer files to the device.
"adb push --sync folder/* /sdcard/Android/data/com.appdomain/files/"
# Now use python script "adb-sync" to delete files not present in current build. no-clobber so that it doesn't copy any files. Strip this python script down to just handling deletes.
"adb-sync --delete --no-clobber folder/ "/sdcard/Android/data/com.appdomain/files/"
from adb-sync.
Related Issues (20)
- error running it. stacktrace. HOT 1
- Colon not supported in folder name HOT 8
- Hard-coded location of python3 HOT 2
- TypeError with latest rev HOT 1
- Problems with parameters: a bytes-like object is required, not 'str' HOT 3
- adb-sync seems to close standard input? HOT 3
- anyway to --reverse and --delete?
- Problems with encoding: UnicodeDecodeError: 'utf-8' codec can't decode byte: invalid start byte HOT 6
- Is this project still active? HOT 2
- windows 10 HOT 8
- adb-sync doesn't work correctly with symlinks
- Crashes when trying to copy files with question marks in the name.
- Copying from external storage HOT 4
- When the file length is the same, it will be considered as no difference HOT 1
- two way sync breaks when filename contains spaces
- Scanning and diffing takes a long time for large directories HOT 2
- "bad date" from "touch" when using -t option HOT 1
- Ignore unreadable files
- Use my fork HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from adb-sync.