wk8 / brive Goto Github PK

View Code? Open in Web Editor NEW

34.0 34.0 6.0 111 KB

Google Drive Backup app

License: The Unlicense

Python 98.29% Smarty 1.71%

brive's People

Contributors

Stargazers

Watchers

Forkers

richardtrip skion procule thlc arcan1s barnumd

brive's Issues

Missing required configuration parameter: google_app_email

When I try to run brive I get the following error

python /brive/brive.py -v --keep-dirs --owned-only --keep-on-crash --preferred-formats docx -u [email protected]
Traceback (most recent call last):
  File "/brive/brive.py", line 167, in <module>
    main()
  File "/brive/brive.py", line 116, in main
    client = Client(args.keep_dirs, args.streaming_http)
  File "/brive/client.py", line 122, in __init__
    self._creds = Credentials(self._http)
  File "/brive/client.py", line 78, in __init__
    not_null=True)
  File "/brive/configuration.py", line 52, in get
    return [instance.get(name, **kwargs) for name in args]
  File "/brive/configuration.py", line 45, in get
    .format(name)
Exception: Missing required configuration parameter: google_app_email

As far as I can tell there is no google_app_email parameter in the settings.yml.tpl example file.

If it's reffering to the yaml google: app: email: - I have that filled in with the [email protected]

Continue on Error

Doing some debugging...have a case where a specific document in the cloud crashes Brive on a specific user...made me think of this option as a possibility.

### Incorrect p12 file and/or password ###

Hey everyone.
Checked Instructions few times, and I can not understand where is the problem. I am continuing to receive the same error message.
Anyone can help?
Thanks!

### Incorrect p12 file and/or password ###

Only download files owned by user

If a file is shared amongst multiple users in a domain Brive will download the same file from each person. It'd be great if there was a setting to only downloaded files owned by the user.

Save files in original format only

Will be great if there is a option to save files in original format as there are in google drive.
Now when I run the script it's save one document in different formats csv,ods,pdf,tsv,xlsx and zip

Crash on filenames containing a slash (/)

[ 2013-01-17 T 03:47:45 Z ] Processing coral.lopez's doc "Dove $1 Coupon Post 4/18/12" (id: 1ihSUjeE7-J0IJzPVYES9MaQD8ZZrVy5u8rB6udQdvmY)
[ 2013-01-17 T 03:47:45 Z ] Starting download from https://docs.google.com/feeds/download/documents/export/Export?id=1ihSUjeE7-J0IJzPVYES9MaQD8ZZrVy5u8rB6udQdvmY&exportFormat=html
[ 2013-01-17 T 03:47:46 Z ] Starting download from https://docs.google.com/feeds/download/documents/export/Export?id=1ihSUjeE7-J0IJzPVYES9MaQD8ZZrVy5u8rB6udQdvmY&exportFormat=rtf
[ 2013-01-17 T 03:47:47 Z ] Starting download from https://docs.google.com/feeds/download/documents/export/Export?id=1ihSUjeE7-J0IJzPVYES9MaQD8ZZrVy5u8rB6udQdvmY&exportFormat=odt
[ 2013-01-17 T 03:47:47 Z ] Starting download from https://docs.google.com/feeds/download/documents/export/Export?id=1ihSUjeE7-J0IJzPVYES9MaQD8ZZrVy5u8rB6udQdvmY&exportFormat=pdf
[ 2013-01-17 T 03:47:49 Z ] Starting download from https://docs.google.com/feeds/download/documents/export/Export?id=1ihSUjeE7-J0IJzPVYES9MaQD8ZZrVy5u8rB6udQdvmY&exportFormat=docx
[ 2013-01-17 T 03:47:49 Z ] Starting download from https://docs.google.com/feeds/download/documents/export/Export?id=1ihSUjeE7-J0IJzPVYES9MaQD8ZZrVy5u8rB6udQdvmY&exportFormat=txt
[ 2013-01-17 T 03:47:50 Z ] Saving coral.lopez's doc "Dove $1 Coupon Post 4/18/12" (id: 1ihSUjeE7-J0IJzPVYES9MaQD8ZZrVy5u8rB6udQdvmY)
[ 2013-01-17 T 03:47:50 Z ] Unexpected shutdown, deleting /home/davidneudorfer/google_docs_backup/2013-01-17T034048Z/ folder
### Unexpected error when saving coral.lopez's documents (doc id: 1ihSUjeE7-J0IJzPVYES9MaQD8ZZrVy5u8rB6udQdvmY) ###
Traceback (most recent call last):
  File "brive.py", line 114, in <module>
    main()
  File "brive.py", line 92, in main
    user.save_documents(backend)
  File "/home/davidneudorfer/Brive/model.py", line 79, in save_documents
    self._save_single_document(backend, document)
  File "/home/davidneudorfer/Brive/model.py", line 105, in _save_single_document
    backend.save(self, document)
  File "/home/davidneudorfer/Brive/backend.py", line 78, in save
    f = open(path, 'w')
IOError: [Errno 2] No such file or directory: u'/home/davidneudorfer/google_docs_backup/2013-01-17T034048Z/coral.lopez/Dove $1 Coupon Post 4/18/12_1ihSUjeE7-J0IJzPVYES9MaQD8ZZrVy5u8rB6udQdvmY.pdf'

Resume feature?

It would be great to have a resume feature, or at least a continue from the last current user. Every time I launch Brive it starts from the beginning.

I've tried to run brive a few times and always when I get to a specific file it just seems to hang. Whats the best way to get you the information you need to debug this? It's a 14 page google docs document with some inline images but nothing exceptional. Also I'd like to echo issue #9 having it keep going past this file would be awesome. We don't have any suspended users so I doubt it's that. Thanks

File names with forward slashes create error

I'm running brive on Windows. I've noticed that if a google doc has a forward slash in the name that Brive will choke on the file with an error like No such file or directory. I'm assuming it's interpreting the / as a folder. So it take the first half of the file as a 'folder name' and the last half (after the /) as the file name. I might suggest you ignore them or replace them with other characters.

Suspended Google Users Crash Brive

oauth2client.client.AccessTokenRefreshError: invalid_request

Suggestion: Save files timestamp

Have you ever thought about saving the files using the Google Drive time and date? I can see this being useful in several ways.

If it sounds like a good idea, I might look into it.

Unexpected error when saving

Google Hangout Sketches crashes Brive

python brive.py -d -u [email protected]
[ 2013-01-17 T 03:17:47 Z ] Configuration loaded
No handlers could be found for logger "oauth2client.util"
[ 2013-01-17 T 03:17:48 Z ] App's credentials valid
[ 2013-01-17 T 03:17:48 Z ] Client loaded
[ 2013-01-17 T 03:17:48 Z ] SimpleBackend loaded
[ 2013-01-17 T 03:17:48 Z ] Processing docs for aaron.farley
[ 2013-01-17 T 03:17:48 Z ] Authorizing client for aaron.farley
[ 2013-01-17 T 03:17:50 Z ] Retrieving page # 1 of docs : found 11 documents
[ 2013-01-17 T 03:17:50 Z ] Processing aaron.farley's doc "Hangout sketchpad: 1/8/13 6:14 PM" (id: 1WI2m76_d5n_cNJJzTPwX7uBQKl9flzW7zJJc7KhvIA4)
[ 2013-01-17 T 03:17:50 Z ] Fetching contents for doc id 1WI2m76_d5n_cNJJzTPwX7uBQKl9flzW7zJJc7KhvIA4
[ 2013-01-17 T 03:17:50 Z ] Found extension .png for document id 1WI2m76_d5n_cNJJzTPwX7uBQKl9flzW7zJJc7KhvIA4
[ 2013-01-17 T 03:17:50 Z ] Found extension .pdf for document id 1WI2m76_d5n_cNJJzTPwX7uBQKl9flzW7zJJc7KhvIA4
[ 2013-01-17 T 03:17:50 Z ] Found extension .jpeg for document id 1WI2m76_d5n_cNJJzTPwX7uBQKl9flzW7zJJc7KhvIA4
[ 2013-01-17 T 03:17:50 Z ] Found extension .svg for document id 1WI2m76_d5n_cNJJzTPwX7uBQKl9flzW7zJJc7KhvIA4
[ 2013-01-17 T 03:17:50 Z ] Starting download from https://docs.google.com/feeds/download/drawings/Export?id=1WI2m76_d5n_cNJJzTPwX7uBQKl9flzW7zJJc7KhvIA4&exportFormat=png
[ 2013-01-17 T 03:17:51 Z ] Checking download integrity for doc id 1WI2m76_d5n_cNJJzTPwX7uBQKl9flzW7zJJc7KhvIA4
[ 2013-01-17 T 03:17:51 Z ] Starting download from https://docs.google.com/feeds/download/drawings/Export?id=1WI2m76_d5n_cNJJzTPwX7uBQKl9flzW7zJJc7KhvIA4&exportFormat=pdf
[ 2013-01-17 T 03:17:51 Z ] Checking download integrity for doc id 1WI2m76_d5n_cNJJzTPwX7uBQKl9flzW7zJJc7KhvIA4
[ 2013-01-17 T 03:17:51 Z ] Starting download from https://docs.google.com/feeds/download/drawings/Export?id=1WI2m76_d5n_cNJJzTPwX7uBQKl9flzW7zJJc7KhvIA4&exportFormat=svg
[ 2013-01-17 T 03:17:51 Z ] Checking download integrity for doc id 1WI2m76_d5n_cNJJzTPwX7uBQKl9flzW7zJJc7KhvIA4
[ 2013-01-17 T 03:17:51 Z ] Starting download from https://docs.google.com/feeds/download/drawings/Export?id=1WI2m76_d5n_cNJJzTPwX7uBQKl9flzW7zJJc7KhvIA4&exportFormat=jpeg
[ 2013-01-17 T 03:17:52 Z ] Checking download integrity for doc id 1WI2m76_d5n_cNJJzTPwX7uBQKl9flzW7zJJc7KhvIA4
[ 2013-01-17 T 03:17:52 Z ] Saving aaron.farley's doc "Hangout sketchpad: 1/8/13 6:14 PM" (id: 1WI2m76_d5n_cNJJzTPwX7uBQKl9flzW7zJJc7KhvIA4)
[ 2013-01-17 T 03:17:52 Z ] Writing aaron.farley's Hangout sketchpad: 1/8/13 6:14 PM to /home/davidneudorfer/google_docs_backup/2013-01-17T031748Z/aaron.farley/Hangout sketchpad: 1/8/13 6:14 PM_1WI2m76_d5n_cNJJzTPwX7uBQKl9flzW7zJJc7KhvIA4.pdf
[ 2013-01-17 T 03:17:52 Z ] Unexpected shutdown, deleting /home/davidneudorfer/google_docs_backup/2013-01-17T031748Z/ folder
### Unexpected error when saving aaron.farley's documents (doc id: 1WI2m76_d5n_cNJJzTPwX7uBQKl9flzW7zJJc7KhvIA4) ###
Traceback (most recent call last):
  File "brive.py", line 114, in <module>
    main()
  File "brive.py", line 92, in main
    user.save_documents(backend)
  File "/home/davidneudorfer/Brive/model.py", line 79, in save_documents
    self._save_single_document(backend, document)
  File "/home/davidneudorfer/Brive/model.py", line 105, in _save_single_document
    backend.save(self, document)
  File "/home/davidneudorfer/Brive/backend.py", line 78, in save
    f = open(path, 'w')
IOError: [Errno 2] No such file or directory: u'/home/davidneudorfer/google_docs_backup/2013-01-17T031748Z/aaron.farley/Hangout sketchpad: 1/8/13 6:14 PM_1WI2m76_d5n_cNJJzTPwX7uBQKl9flzW7zJJc7KhvIA4.pdf'

Brive doesn't recognize Google Drive folders

Interactive is a folder in this users Google Drive folder

[ 2013-01-17 T 03:35:34 Z ] Processing aaron.farley's doc "Interactive" (id: 0BxdWo-U0wQS-bExMRXl4Sm0wQ1U)
[ 2013-01-17 T 03:35:34 Z ] No suitable download URL for document id 0BxdWo-U0wQS-bExMRXl4Sm0wQ1U
[ 2013-01-17 T 03:35:34 Z ] No suitable download URL for document id 0BxdWo-U0wQS-bExMRXl4Sm0wQ1U
[ 2013-01-17 T 03:35:34 Z ] Couldn't retrieve any version of document id 0BxdWo-U0wQS-bExMRXl4Sm0wQ1U (title: Interactive) (Traceback (most recent call last): [('brive.py', 114, '<module>', 'main()'), ('brive.py', 92, 'main', 'user.save_documents(backend)'), ('/home/davidneudorfer/Brive/model.py', 66, 'save_documents', 'document.fetch_contents(self._client)'), ('/home/davidneudorfer/Brive/model.py', 163, 'fetch_contents', 'self._do_fetch_contents(client)'), ('/home/davidneudorfer/Brive/model.py', 188, '_do_fetch_contents', 'self._do_fetch_contents(client, True, banned_urls)'), ('/home/davidneudorfer/Brive/model.py', 184, '_do_fetch_contents', "+ u'{} (title: {})'.format(self.id, self.title))"), ('/home/davidneudorfer/Brive/utils.py', 61, 'error', 'sys.stderr.write(Log._timestamped_string(*args, **ttstr_kwargs))'), ('/home/davidneudorfer/Brive/utils.py', 70, '_timestamped_string', "+ str(traceback.extract_stack()) + ' )'")] )
[ 2013-01-17 T 03:35:34 Z ] Saving aaron.farley's doc "Interactive" (id: 0BxdWo-U0wQS-bExMRXl4Sm0wQ1U)

Suggestion: Save multiple into just one folder

I'm trying to figure out if this is a bad thing.

I have about 5000 Google accounts I'd like to download their own Google drive files.

If I do one run of all of the accounts, one bad file will crash the backup of the remaining accounts. Plus it will take days.

Right now I have a BASH script that allows me to backup all of the accounts (one at a time) but 15 accounts simultaneously. (I can actually max out our company's internet pipe this way.) However I then wind up with 5000 backup folders, each having only one account. I then wrote another script the rsyncs all of the folders into one "combined" folder.

If I were to add a command line parm to make the destination folder NOT add the timestamped subfolder, would this create issues? Since I'm using a ZFS filesystem with snapshotting, I don't really need the timestamped folder names.

wk8 / brive Goto Github PK

brive's People

Contributors

Stargazers

Watchers

Forkers

brive's Issues

Recommend Projects

Recommend Topics

Recommend Org

Jobs