handyman5 / acd_fuse Goto Github PK

I mount ACD unlimited but it runs out of space. Using "df" shows 100% used. No more copy is possible.

Firstly, thanks for this tool. I've just got my cloud drive mounted on my fedora system.

I had a couple of issues

1. (not your concern) pyacd has configs for www.amazon.com and www.amazon.jp, so I had to add www.amazon.co.uk to pyacd.init.py, and set ubid
   ln 100. amazon_domain="www.amazon.co.uk"
   ln 135 "www.amazon.co.uk": "ubid-acbuk",

2. edit session dictionary object to 'accept_terms' and set username in acd
   ln 81-2 self.session.dict['agreed_with_terms']=True
   self.session.username = self.email

3. unable to dump session to file as session dictionary is not serializable.
   I commented out ln 83 to allow the cloud drive to mount

Would be interested in knowing if you've got a fix for (3)

Thanks again, Ian

Below is the error from json. Mounting proceeds if caching is commented out (ln 81-82)

Trying to login from cached sessionfile /tmp/acd_fuse/sessionfile
Cached session failed; trying auth login
Traceback (most recent call last):
  File "/home/ian/bin/acd", line 464, in <module>
    fs.main()
  File "/home/ian/bin/acd", line 83, in main
    json.dump(self.session.__dict__, sessfile)
  File "/usr/lib64/python2.7/json/__init__.py", line 181, in dump
    for chunk in iterable:
  File "/usr/lib64/python2.7/json/encoder.py", line 428, in _iterencode
    for chunk in _iterencode_dict(o, _current_indent_level):
  File "/usr/lib64/python2.7/json/encoder.py", line 402, in _iterencode_dict
    for chunk in chunks:
  File "/usr/lib64/python2.7/json/encoder.py", line 436, in _iterencode
    o = _default(o)
  File "/usr/lib64/python2.7/json/encoder.py", line 178, in default
    raise TypeError(repr(o) + " is not JSON serializable")
TypeError: <pyacd.auth.PicklableCookieJar[Cookie(version=0, 
etc etc etc
]> is not JSON serializable

Just found this awesome project! Getting this error when trying to mount. Fresh install of both python-fuse and acd. Any ideas on what I'm doing wrong?

user@host:/opt/acd# ./acd /mnt/acd/ -
Trying to login from cached sessionfile /tmp/acd_fuse/sessionfile
Login successful; starting filesystem
fuse: mount failed: Invalid argument
Traceback (most recent call last):
File "./acd", line 464, in
fs.main()
File "./acd", line 89, in main
fuse.Fuse.main(self)
File "/usr/lib/python2.7/dist-packages/fuse.py", line 757, in main
main(**d)
fuse.FuseError: filesystem initialization failed

Trying to login from cached sessionfile /tmp/acd_fuse/sessionfile
Cached session failed; trying auth login
Traceback (most recent call last):
File "./acd", line 468, in
fs.main()
File "./acd", line 83, in main
self.session = pyacd.login(self.email, self.password)
File "pyacd/pyacd/auth.py", line 73, in login
raise pyacd.PyAmazonCloudDriveError("Login failed.")
pyacd.exception.PyAmazonCloudDriveError: PyAmazonCloudDriveError: Login failed.

Unfortunately mounting my amazon drive doesn't work.
I cloned this repo with "git clone https://github.com/handyman5/acd_fuse.git --recursive". Working directory looks like it should, but when executing "./acd /mnt/amazonDrive/ -o email=XXXXX,password=XXXXX" it returns this exception.

I just installed fuse and attempted to run ./acd

Following is the console output:
Traceback (most recent call last):
File "./acd", line 9, in
import fuse
ImportError: No module named fuse

Fuse was installed using latest version on Ubuntu 15.04 x64 using ./configure, make, make install

ACD was downloaded and executed as ./acd

Am I missing something?

I got this trying today's source. Do I have to add acd_fuse as a device in ACD somewhere?

markt@host:/usr/local/src/acd/acd_fuse$ sudo ./acd /mnt/acd -oemail=[email protected],password=hackme
Trying to login from cached sessionfile /tmp/acd_fuse/sessionfile
Cached session failed; trying auth login
Login successful; starting filesystem
Traceback (most recent call last):
File "./acd", line 464, in
fs.main()
File "./acd", line 87, in main
self.download = self.api.can_device_download()
File "build/bdist.linux-x86_64/egg/pyacd/api.py", line 146, in can_device_download
File "build/bdist.linux-x86_64/egg/pyacd/connection.py", line 32, in do_get
File "build/bdist.linux-x86_64/egg/pyacd/connection.py", line 85, in _do_request
pyacd.exception.PyAmazonCloudDriveError: PyAmazonCloudDriveError: https://www.amazon.com/clouddrive/api/?deviceId.deviceType=ubid&ContentType=JSON&deviceId.deviceSerialNumber=184-1273477-9808429&Operation=canDeviceDownload&customerId=++%3A+%270%27%2C%0A++++++++++++++++++++++++++++++++campaignId++%3A+%27PRICING_2015%27%2C+%0A++++++++++++++++++++++++++++++++deviceClass+%3A+%27web%27%0A++++++++++++++++++++++++++++%7D%29%2C%0A++++++++++++++++beforeSend++%3A+function%28xhr%29+%7B%0A++++++++++++++++++++xhr.setRequestHeader%28%0A++++++++++++++++++++xhr.setRequestHeader%28%27x-Cookie%27%2C+%27session-id-time%3D2082787201l%3B+skin%3Dnoskin%3B+wl-uid%3D189-4177884-0612365%3B+x-wl-uid%3D1xwiGhsD%2FKjYx9veILAEhqDbbhjjQkS%2BM98qnuyGwnUSvL3Y7us40F%2BewBuqI2HfPm%2BhHYoBetDk%3D%3B+nav2%3D%3B+session-id%3D189-4177884-0612365%3B+ubid-main%3D184-1273477-9808429%27%29%3B%0A++++++++++++++++%7D%2C%0A++++++++++++++++xhrFields+++%3A+%7B+withCredentials%3A+true+%7D%2C%0A++++++++++++++++complete++++%3A+cdClose%0A++++++++++++%7D%29%3B%0A++++++++%7D%0A%0A++++++++function+triggerClose%28%29+%7B%0A++++++++++++cdAcknowledge%28%29%3B%0A++++++++%7D%0A++++%3C%2Fscript&_=1427824663 {'X-amzn-sessionid': '189-4177884-0612365'} 403:Forbidden

I have a top level directory called 'Backups' in ACD that I created a few hours ago. For some reason it isn't showing up via acd_fuse. I only see Documents, Music, Pictures, RecycleBin and Videos. Is there some hard coding of the top level directory structure?

Additionally - and I'm thinking its related - I have a bunch of pictures under the Pictures directory and it appears empty via acd_fuse.

Hi, great work creating this FUSE! Seems to work pretty well so far w/ a tweak or two.

The main problem I ran into is that pyacd sessions seem to cause things to break w/ JSON serialization. I just switched it to pickle and it worked fine. Happy to submit a pull request if you want?

Trying to login from cached sessionfile /tmp/acd_fuse/sessionfile
Cached session failed; trying auth login
Traceback (most recent call last):
  File "./acd", line 459, in <module>
    fs.main()
  File "./acd", line 80, in main
    json.dump(self.session.__dict__, sessfile)
  File "/usr/lib/python2.7/json/__init__.py", line 189, in dump
    for chunk in iterable:
  File "/usr/lib/python2.7/json/encoder.py", line 434, in _iterencode
    for chunk in _iterencode_dict(o, _current_indent_level):
  File "/usr/lib/python2.7/json/encoder.py", line 408, in _iterencode_dict
    for chunk in chunks:
  File "/usr/lib/python2.7/json/encoder.py", line 442, in _iterencode
    o = _default(o)
  File "/usr/lib/python2.7/json/encoder.py", line 184, in default
    raise TypeError(repr(o) + " is not JSON serializable")
TypeError: <pyacd.auth.PicklableCookieJar[Cookie(version=0, name='at-main', 
...
rest={}, rfc2109=False)]> is not JSON serializable

I think there may be a step missing in the usage/install guide. I'm on Ubuntu 14.04 I've installed python-fuse but when I navigate into the folder I extracted the downloaded zip into and try my first run of acd I get:

Traceback (most recent call last):
 File "./acd", line 25, in <module>
    import pyacd
ImportError: No module named pyacd

I am able to successfully log in and mount my Amazon Cloud Drive as a FUSE filesystem. However, browsing the file system and the Amazon Cloud Drive web interface yield different results. When the service first allowed the unlimited trial, I used the Mac client to upload a video to test the speed. I can see that video in the web interface, but it appears nowhere in the file system.

I am also having nothing but problems trying to upload files to my Amazon Cloud Drive by copying them to the file system. I've tried rsync and cp. I've tried directories and single files. The mounted file system shows the copied files, but they never appear in the Amazon Cloud Drive web interface. My network activity doesn't change after I cp a file from my local drive to the ACD FUSE drive, so it doesn't appear to upload, even though the /tmp/acd_fuse directory gets full/populated.

Anyone else having similar issues?

Trying to login from cached sessionfile /var/folders/px/1k8yck6x2tdgcwjbjxcjwfw00000gn/T/acd_fuse/sessionfile
Cached session failed; trying auth login
Login successful; starting filesystem
fuse: unknown option `big_writes'
Traceback (most recent call last):
File "./acd", line 468, in
fs.main()
File "./acd", line 90, in main
fuse.Fuse.main(self)
File "/Users/eric/.virtualenvs/acd/lib/python2.7/site-packages/fuse.py", line 754, in main
main(**d)
fuse.FuseError: filesystem initialization failed
mount_osxfusefs: failed to mount /Users/eric/.virtualenvs/acd/acd_fuse/mounter@/dev/osxfuse0: Socket is not connected

I do have osxfuse installed and can do sshfs mounts properly.

I want to use amazon clouddrive for backing up a whole lvm snapshot. Therefore I need to disable the local cache of acd_fuse because caching doesn't make sense. Is it possible to disable the cache?

The transferring speed is generally great but I experience very slow cd performance. It takes around 7 seconds to complete each task which I think too long... considering I can transfer 100 MB file with average 5 MB/s.

Is it really a problem or you guys experience the same ? or anyone has anything to suggest ?

I tested using DigitalOcean VPS.

Output:

Trying to login from cached sessionfile /tmp/acd_fuse/sessionfile
Cached session failed; trying auth login
email****

password******

Traceback (most recent call last):
File "./acd", line 473, in
fs.main()
File "./acd", line 88, in main
self.session = pyacd.login(self.email, self.password)
File "/usr/lib64/python2.7/site-packages/amazon_cloud_drive-0.0.6-py2.7.egg/pyacd/auth.py", line 45, in login
IndexError: list index out of range

Files debug.log and sessionfile was created but has zero size

It's my first run of acd_fuse.

Hi,

On a Mac, I get the following error:

Login successful; starting filesystem
Traceback (most recent call last):
File "./acd", line 468, in
fs.main()
File "./acd", line 88, in main
self.download = self.api.can_device_download()
File "/Users/eric/.virtualenvs/acd/acd_fuse-master/pyacd/api.py", line 146, in can_device_download
resp_json=json.loads(pyacd.do_get(end_point))
File "/Users/eric/.virtualenvs/acd/acd_fuse-master/pyacd/connection.py", line 32, in do_get
return _do_request(req,None)
File "/Users/eric/.virtualenvs/acd/acd_fuse-master/pyacd/connection.py", line 85, in _do_request
e.msg
pyacd.exception.PyAmazonCloudDriveError: PyAmazonCloudDriveError: https://www.amazon.com/clouddrive/api/?deviceId.deviceType=ubid&ContentType=JSON&deviceId.deviceSerialNumber=182-4412315-0887523&Operation=canDeviceDownload&customerId=++%3A+%270%27%2C%0A++++++++++++++++++++++++++++++++campaignId++%3A+%27PRICING_2015%27%2C+%0A++++++++++++++++++++++++++++++++deviceClass+%3A+%27web%27%0A++++++++++++++++++++++++++++%7D%29%2C%0A++++++++++++++++beforeSend++%3A+function%28xhr%29+%7B%0A++++++++++++++++++++xhr.setRequestHeader%28%0A++++++++++++++++++++xhr.setRequestHeader%28%27x-Cookie%27%2C+%27session-id-time%3D2082787201l%3B+skin%3Dnoskin%3B+wl-uid%3D183-7137901-4456421%3B+x-wl-uid%3D1DS5eA18TMo%2FLgMPuEb5b0x5gf0IhWuSa6qDhwfy9oA6eaYNNPuEldItmJ5cRsFskVYhFNOnY0iI%3D%3B+nav2%3D%3B+session-id%3D183-7137901-4456421%3B+session-token%3D0vYDfu6VsSaTZQas8l9POkF%2BGNjHtTzIKySBVvC0b6QfjOeD%2BI5e9yL4CZJAIwp76ONLpRBcLCMPS5r5R0xI2o4AtU9y1bw6oXEsUmVIbqEpWVg36I6CeTDdtkRA5B%2BQRXJ%2BzgvhEkDd0v7l%2B0XjcPoF71UAcPJb8fLFYtqdvxWq%2FW0dj6cIvzcB2i4frVDnCxOTN0ZAG8bCjbMo14ehKxRaIOl1cFIAhRaddXuITLEJTs0inp3HT1YGJwnH28pc%3B+ubid-main%3D182-4439373-0889730%27%29%3B%0A++++++++++++++++%7D%2C%0A++++++++++++++++xhrFields+++%3A+%7B+withCredentials%3A+true+%7D%2C%0A++++++++++++++++complete++++%3A+cdClose%0A++++++++++++%7D%29%3B%0A++++++++%7D%0A%0A++++++++function+triggerClose%28%29+%7B%0A++++++++++++cdAcknowledge%28%29%3B%0A++++++++%7D%0A++++%3C%2Fscript&_=1428281490 {'X-amzn-sessionid': '183-7123567-4476578'} 403:Forbidden

At this point I'm back in the command line. Any ideas?

I can't get this to work with my german Amazon account (amazon.de), any help would be appreciated.

Am I doing something wrong?

user@host:/usr/local/src/acd/acd_fuse$ sudo ./acd /mnt/acd -oemail=[email protected],password=hackme
Trying to login from cached sessionfile /tmp/acd_fuse/sessionfile
Traceback (most recent call last):
File "./acd", line 464, in
fs.main()
File "./acd", line 74, in main
if not self.session.is_valid() or not self.session.is_logined():
AttributeError: 'Session' object has no attribute 'is_valid'

Hi,

I tried acd for a bit and found that if I'm doing a transmission of a large file, the fuse system will cause the entire system hanging for a very long time.

Even ls and df don't work.

I did touch 999. What debug gives me:

LOOKUP /999
getattr /999
PyAmazonCloudDriveError: https://www.amazon.com/clouddrive/api/?ContentType=JSON&populatePath=true&path=%2F999&Operation=getInfoByPath&customerId=*&_=1431015393 {'X-amzn-sessionid': '*'} 
   unique: 19, error: -2 (No such file or directory), outsize: 16
unique: 20, opcode: CREATE (35), nodeid: 1, insize: 60, pid: 24873
create flags: 0x8841 /999 0100664 umask=0002
   create[0] flags: 0x8841 /999
getattr /999
   NODEID: 4
   unique: 20, success, outsize: 160
unique: 21, opcode: FLUSH (25), nodeid: 4, insize: 64, pid: 24873
flush[0]
   unique: 21, error: -2 (No such file or directory), outsize: 16
unique: 22, opcode: SETATTR (4), nodeid: 4, insize: 128, pid: 24873
utime /999 1431015394 1431015394
getattr /999
   unique: 22, success, outsize: 120
unique: 23, opcode: FLUSH (25), nodeid: 4, insize: 64, pid: 24873
flush[0]
   unique: 23, error: -2 (No such file or directory), outsize: 16
unique: 24, opcode: RELEASE (18), nodeid: 4, insize: 64, pid: 0
release[0] flags: 0x8801
   unique: 24, success, outsize: 16

On the CLI I see: touch: failed to close ‘999’: No such file or directory.

Happens on all files. When I remount the mount, it appears again. Who has an idea? I took the latest version from git :)

Are there any plans to allow a mounted filesystem to be used by other users beside the one who mounted it in the first place?
Also is there a way to include this to /etc/fstab to make it available after system boot?

While doing some testing around the caching behavior I noticed that file content on read() is some HTML from the cloud drive page or API rather than the actual file content. File metadata looks correct via ls, however.

Files being written appear to be uploaded correctly, however if I clear my cache directory and re-mount the files I just wrote out correctly will be retrieved corrupted.

Note: I am presently using the fork of pyacd referenced in this Issue: #24

It took me a short while to figure out what was going wrong, but my password has an ! in it. This then gives the error message:
bash: !,: event not found

I worked out that I needed ! but it would be better to have a different way of entering the password to avoid this - or a mention on the "usage" section

Hi
Firstly thanks for updating this!

I've been running tests on two different systems in separate locations with the same issue where I get ~100% CPU use from the python2 acd thread and throughput is maxed out at 200KB/s.

I'm measuring throughput rate using pv -r amazondrive/remotefile > ./localfile

I'm presuming that the low rate is linked to the high CPU? Is this usual performance?
Cheers

This was working just fine yesterday, now all of a sudden I cannot login / mount successfully. Tried on both my Arch and Ubuntu boxes. I removed my email, but I did check that it was correct. My password is also correct. Whats odd, is it seems to login just fine: Login successful; starting filesystem

Trying to login from cached sessionfile /tmp/acd_fuse/sessionfile
Cached session failed; trying auth login
Login successful; starting filesystem
Traceback (most recent call last):
  File "./acd", line 471, in <module>
    fs.main()
  File "./acd", line 90, in main
    self.download = self.api.can_device_download()
  File "/home/dh4/acd_fuse/pyacd/pyacd/api.py", line 134, in can_device_download
    raise pyacd.PyAmazonCloudDriveError("Not logined %s"%session)
pyacd.exception.PyAmazonCloudDriveError: PyAmazonCloudDriveError: Not logined <Session: username: [MY EMAIL], customer_id: None, agreed_with_terms: True>

I get this when trying to create a new file using touch.

user@host:/mnt/acd/testing123# touch test123.txt
touch: failed to close ‘test123.txt’: No such file or directory

If I run fuse with debug on (-d option),
./acd /mnt/acd -oemail='',password='',umask=002,allow_other -d
I get the below. Note the 400:Bad Request when looking up info on the new files being created.

unique: 691, opcode: LOOKUP (1), nodeid: 5, insize: 52, pid: 7083
LOOKUP /testing123/test123.txt
getattr /testing123/test123.txt
PyAmazonCloudDriveError: https://www.amazon.com/clouddrive/api/?ContentType=JSON&populatePath=true&path=%2Ftesting123%2Ftest123.txt&Operation=getInfoByPath&customerId=&_= {'X-amzn-sessionid': ''} 400:Bad Request
unique: 691, error: -2 (No such file or directory), outsize: 16
unique: 692, opcode: CREATE (35), nodeid: 5, insize: 68, pid: 7083
create flags: 0x8841 /testing123/test123.txt 0100644 umask=0022
create[0] flags: 0x8841 /testing123/test123.txt
getattr /testing123/test123.txt
NODEID: 8
unique: 692, success, outsize: 160
unique: 693, opcode: FLUSH (25), nodeid: 8, insize: 64, pid: 7083
flush[0]
unique: 693, error: -2 (No such file or directory), outsize: 16
unique: 694, opcode: SETATTR (4), nodeid: 8, insize: 128, pid: 7083
utime /testing123/test123.txt 1428960503 1428960503
getattr /testing123/test123.txt
unique: 694, success, outsize: 120
unique: 695, opcode: FLUSH (25), nodeid: 8, insize: 64, pid: 7083
flush[0]
unique: 695, error: -2 (No such file or directory), outsize: 16
unique: 696, opcode: RELEASE (18), nodeid: 8, insize: 64, pid: 0
release[0] flags: 0x8801
unique: 696, success, outsize: 16