GithubHelp home page GithubHelp logo

pydrive's Introduction

Deprecated

This project is deprecated and no longer maintained. No further changes will be made.

In one of the PyDrive issues, we learned about the PyDrive2 fork of PyDrive. Forks are permitted under PyDrive's license, and we hope that such forks will be useful for the needs of PyDrive users. The PyDrive team makes no endorsement or support promises of any particular fork, but we're excited to see the open source license being a vehicle for new project development.

PyDrive

PyDrive is a wrapper library of google-api-python-client that simplifies many common Google Drive API tasks.

Project Info

Features of PyDrive

  • Simplifies OAuth2.0 into just few lines with flexible settings.
  • Wraps Google Drive API into classes of each resource to make your program more object-oriented.
  • Helps common operations else than API calls, such as content fetching and pagination control.

How to install

You can install PyDrive with regular pip command.

$ pip install PyDrive

To install the current development version from GitHub, use:

$  pip install git+https://github.com/googleworkspace/PyDrive.git#egg=PyDrive

OAuth made easy

Download client_secrets.json from Google API Console and OAuth2.0 is done in two lines. You can customize behavior of OAuth2 in one settings file settings.yaml.

from pydrive.auth import GoogleAuth
from pydrive.drive import GoogleDrive

gauth = GoogleAuth()
gauth.LocalWebserverAuth()

drive = GoogleDrive(gauth)

File management made easy

Upload/update the file with one method. PyDrive will do it in the most efficient way.

file1 = drive.CreateFile({'title': 'Hello.txt'})
file1.SetContentString('Hello')
file1.Upload() # Files.insert()

file1['title'] = 'HelloWorld.txt'  # Change title of the file
file1.Upload() # Files.patch()

content = file1.GetContentString()  # 'Hello'
file1.SetContentString(content+' World!')  # 'Hello World!'
file1.Upload() # Files.update()

file2 = drive.CreateFile()
file2.SetContentFile('hello.png')
file2.Upload()
print('Created file %s with mimeType %s' % (file2['title'],
file2['mimeType']))
# Created file hello.png with mimeType image/png

file3 = drive.CreateFile({'id': file2['id']})
print('Downloading file %s from Google Drive' % file3['title']) # 'hello.png'
file3.GetContentFile('world.png')  # Save Drive file as a local file

# or download Google Docs files in an export format provided.
# downloading a docs document as an html file:
docsfile.GetContentFile('test.html', mimetype='text/html')

File listing pagination made easy

PyDrive handles file listing pagination for you.

# Auto-iterate through all files that matches this query
file_list = drive.ListFile({'q': "'root' in parents"}).GetList()
for file1 in file_list:
    print('title: {}, id: {}'.format(file1['title'], file1['id']))

# Paginate file lists by specifying number of max results
for file_list in drive.ListFile({'maxResults': 10}):
    print('Received {} files from Files.list()'.format(len(file_list))) # <= 10
    for file1 in file_list:
        print('title: {}, id: {}'.format(file1['title'], file1['id']))

Concurrent access made easy

All calls made are thread-safe. The underlying implementation in the google-api-client library is not thread-safe, which means that every request has to re-authenticate an http object. You can avoid this overhead by creating your own http object for each thread and re-use it for every call.

This can be done as follows:

# Create httplib.Http() object.
http = drive.auth.Get_Http_Object()

# Create file object to upload.
file_obj = drive.CreateFile()
file_obj['title'] = "file name"

# Upload the file and pass the http object into the call to Upload.
file_obj.Upload(param={"http": http})

You can specify the http-object in every access method which takes a param parameter.

Note: This is not an official Google product.

pydrive's People

Contributors

aliafshar avatar arpit1997 avatar casyfill avatar cmiksche avatar efiop avatar femtotrader avatar fjodor42 avatar grant avatar hschauhan avatar jeffyehtw avatar jgwak avatar kiaragrouwstra avatar lawben avatar mansiag avatar pferate avatar piperchester avatar rikkiprince avatar rnabel avatar sfujiwara avatar sqrrrl avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

pydrive's Issues

Unable to access "permissions" dictionary key for file resource.

I'm not having trouble authenticating with google drive.

When I try to access the dictionary key for a google drive file called 'permissions' I receive an error:

Authentication successful.
title: [document name snipped], id: [id of file snipped]
permissions:


Traceback (most recent call last):
  File "quickstart.py", line 18, in <module>
    permissions = file1['permissions']
  File "/home/myusername/.local/lib/python2.7/site-packages/pydrive/files.py", line 113, in __getitem__
    raise KeyError(e)
KeyError: KeyError('permissions',)

I'm using this code based on the sample documentation code:

from pprint import pprint
from pydrive.auth import GoogleAuth

gauth = GoogleAuth()
gauth.LocalWebserverAuth() # Creates local webserver and auto handles authentication

from pydrive.drive import GoogleDrive
drive = GoogleDrive(gauth)

file_list = drive.ListFile({'q': "'root' in parents and trashed=false"}).GetList()
for file1 in file_list:
    print 'title: %s, id: %s\npermissions:\n\n' % (file1['title'], file1['id'])
    permissions = file1['permissions']
    pprint(permissions)
    print '\n\n'

I'm able to access any of the other google drive API keys in the documentation including ownerName and userPermission, but not permissions. Any idea why?

Where does the google store the files uploaded

I am using pyDrive with Google service account.
All is working fine and i can upload the docs and retrive the list like this

title: Hello.txt, id: 0BxuVfEvSgcvkWnJKdHVLT0JldUk
title: Hello.txt, id: 0BxuVfEvSgcvkcVllLVRFendHSnM
title: My New Text Document, id: 0BxuVfEvSgcvkbWpFejZGYlZlZkE
title: My New Text Document, id: 0BxuVfEvSgcvkVEtPNWl5dDhLUFk
title: My New Text Document, id: 0BxuVfEvSgcvkSV8tMlQ3RjFaSk0
title: My New Text Document, id: 0BxuVfEvSgcvkSHRoRXcwQ0pFZEE
title: My New Text Document, id: 0BxuVfEvSgcvkUjBfYldablpld1U
title: How to get started with Drive, id: 0BxuVfEvSgcvkc3RhcnRlcl9maWxl
title: My New Text Document, id: 0BxuVfEvSgcvkRGw5cWpfZ2FyOTA

But when i go to my google drive then i don't any of those files and in my drive account i have 10k files , neither i see those in above list.

I want to know that in which account does it store docs.

I created the service account via API console my my main Google account

Cant' list folder, only files

See the code below:

from pydrive.auth import GoogleAuth
from pydrive.drive import GoogleDrive

gauth = GoogleAuth()
gauth.CommandLineAuth()
drive = GoogleDrive(gauth)

file_list = drive.ListFile({'q': "'root' in parents and trashed=false"}).GetList()
for gfile in file_list:
    print(gfile['title'])

returns

2014_07_20_02_30_espritco.7z
2014_07_19_02_30_espritco.7z
2014_07_18_20_02_espritco.7z

While it should return

2014_07_20_02_30_espritco.7z
2014_07_19_02_30_espritco.7z
2014_07_18_20_02_espritco.7z
backup

Where backup is a folder.

Any idea ?

How to use with refresh tokens in a webserver?

I can get any token for any user and access its Google Drive, but I don't know how to rebuild the drive object in a later session. Can I do this if I get the refresh token and store it in my database? How?

Issues while installing it with pip

When I tried installing pydrive via pip, it installed python 2.x version, I was infact hoping for python 3. Further when I tried pip install git+https://github.com/googledrive/PyDrive.git, it still resulted in python 2 version. I want the python 3 version in my application. Let me know if I am doing something wrong here.

Ubuntu OS 14.04
Virtual Environment created with python 3.4

Please create tests files in a subdirectory

When I run the test suite, I get lots of test files created in my drive. To make it easier to clean up after that, can you please make the testsuite create them not in root, but in a subdirectory with some unique name, like pydrive-test-files?

An example of catching exception of revoked access_token and refresh it.

I have successfully stored the access_token and refresh_token in credentials.json file. Now i need a basic example of handling exceptions being thrown if access_token needed to be regenerated. I have done this so far, correct me where i am wrong or all good:

from pydrive.auth import GoogleAuth, AuthenticationError, RefreshError
from pydrive.drive import GoogleDrive

gauth = GoogleAuth() # or authentication exceptions are thrown here?

tries = 0
while True
    try: 
        drive = GoogleDrive(gauth) # handling exceptions here at the moment
                break
    except AuthenticationError:
        if tries >= 5:
            raise
        else:
            tries += 1
            # refresh token?
            try:
                gauth.Refresh()
            except RefreshError:
                # log error
                raise

file_list = drive.ListFile({'q': "'root' in parents"}).GetList()
for file1 in file_list:
    print 'title: %s, id: %s' % (file1['title'], file1['id'])

ApiRequestError

I did these steps in http://pythonhosted.org/PyDrive/quickstart.html#authentication

and when i try to upload new file by using these lines + giving access permission via the browser:

from pydrive.drive import GoogleDrive
from pydrive.auth import GoogleAuth


gauth = GoogleAuth()
gauth.LocalWebserverAuth() # Creates local webserver and auto handles authentication
drive = GoogleDrive(gauth)

file1 = drive.CreateFile({'title': 'Hello.txt'})  # Create GoogleDriveFile instance with title 'Hello.txt'
file1.SetContentString('Hello World!') # Set content of the file from given string
file1.Upload()

i get this error

ApiRequestError                           Traceback (most recent call last)
<ipython-input-12-c183c71d3424> in <module>()
----> 1 file2.Upload()

/Library/Python/2.7/site-packages/pydrive/files.pyc in Upload(self, param)
    214         self._FilesPatch(param=param)
    215     else:
--> 216       self._FilesInsert(param=param)
    217 
    218   @LoadAuth

/Library/Python/2.7/site-packages/pydrive/auth.pyc in _decorated(self, *args, **kwargs)
     52     if self.auth.service is None:  # Check if drive api is built.
     53       self.auth.Authorize()
---> 54     return decoratee(self, *args, **kwargs)
     55   return _decorated
     56 

/Library/Python/2.7/site-packages/pydrive/files.pyc in _FilesInsert(self, param)
    232       metadata = self.auth.service.files().insert(**param).execute()
    233     except errors.HttpError, error:
--> 234       raise ApiRequestError(error)
    235     else:
    236       self.uploaded = True

ApiRequestError: <HttpError 403 when requesting https://www.googleapis.com/upload/drive/v2/files?uploadType=multipart&alt=json returned "Access Not Configured">

Can't download files.

Hi I get the following issue:

Authentication successful.
(u'Hello.txt', u'application/vnd.google-apps.document')
Traceback (most recent call last):
  File "quickstart.py", line 18, in <module>
    file2.GetContentFile(file1['title'],file1["mimeType"])
  File "/Library/Python/2.7/site-packages/pydrive/files.py", line 167, in GetContentFile
    self.FetchContent(mimetype)
  File "/Library/Python/2.7/site-packages/pydrive/files.py", line 36, in _decorated
    return decoratee(self, *args, **kwargs)
  File "/Library/Python/2.7/site-packages/pydrive/files.py", line 210, in FetchContent
    'No downloadLink/exportLinks for mimetype found in metadata')
pydrive.files.FileNotDownloadableError: No downloadLink/exportLinks for mimetype found in metadata

Here is how my code looks like now:

from pydrive.auth import GoogleAuth

gauth = GoogleAuth()
gauth.LocalWebserverAuth() # Creates local webserver and auto handles authentication.

from pydrive.drive import GoogleDrive

# Create GoogleDrive instance with authenticated GoogleAuth instance.
drive = GoogleDrive(gauth)
# Auto-iterate through all files that matches this query
file_list = drive.ListFile({'q': "'root' in parents and trashed=false"}).GetList()
for file1 in file_list:
  file2 = drive.CreateFile({'id': file1['id']})
  print(file1["title"], file1["mimeType"])
  file2.GetContentFile(file1['title']) #download the file

I am new to python so I am assuming that i did something wrong even if I use examples from the doc.

QuickStart instructions incompatible with Python 3

Running the quickstart.py code:

from pydrive.auth import GoogleAuth

gauth = GoogleAuth()
gauth.LocalWebserverAuth() # Creates local webserver and auto handles authentication

results in a syntax error.

I guess I didn't see anywhere that PyDrive wasn't Py3K compatible :-(

file "/usr/lib/python2.6/site-packages/PyDrive-1.0.0-py2.6.egg/pydrive/auth.py", line 14, in <module> from oauth2client.tools import ClientRedirectHandler

Hello I´m getting this error with pydrive https://pypi.python.org/pypi/PyDrive , the json key is the same directory than script (a.py).

[[email protected] user]# pip install PyDrive
Requirement already satisfied (use --upgrade to upgrade): PyDrive in /usr/lib/python2.6/site-packages/PyDrive-1.0.0-py2.6.egg
Requirement already satisfied (use --upgrade to upgrade): google-api-python-client>=1.2 in /usr/lib/python2.6/site-packages (from PyDrive)
Requirement already satisfied (use --upgrade to upgrade): PyYAML>=3.0 in /usr/lib64/python2.6/site-packages (from PyDrive)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8 in /usr/lib/python2.6/site-packages (from google-api-python-client>=1.2->PyDrive)
Cleaning up...
[[email protected] user]# vi a.py
[[email protected] user]# cat a.py
from pydrive.auth import GoogleAuth
from pydrive.drive import GoogleDrive

gauth = GoogleAuth()
gauth.LocalWebserverAuth()

drive = GoogleDrive(gauth)
[[email protected] user]# python a.py
Traceback (most recent call last):
File "a.py", line 1, in
from pydrive.auth import GoogleAuth
File "/usr/lib/python2.6/site-packages/PyDrive-1.0.0-py2.6.egg/pydrive/auth.py", line 14, in
from oauth2client.tools import ClientRedirectHandler
File "/usr/lib/python2.6/site-packages/oauth2client/tools.py", line 27, in
import argparse
ImportError: No module named argparse
[[email protected] user]#
Any idea about this?

Seems no maintenance? out of date.

Most of time it raises: FileNotDownloadableError: No downloadLink/exportLinks for mimetype found in metadata due to the <class 'pydrive.files.GoogleDriveFile'> , the format of GoogleDriveFile class seems changed a lot, currently no work, at least can't download file to local.
Below is the trace:

---------------------------------------------------------------------------
FileNotDownloadableError                  Traceback (most recent call last)
 in ()
      1 for file1 in file_list:
----> 2     file1.GetContentString()
      3

/Library/Python/2.7/site-packages/pydrive/files.pyc in GetContentString(self)
    154     """
    155     if self.content is None or type(self.content) is not io.BytesIO:
--> 156       self.FetchContent()
    157     return self.content.getvalue().decode('utf-8')
    158

/Library/Python/2.7/site-packages/pydrive/files.pyc in _decorated(self, *args, **kwargs)
     34     if not self.uploaded:
     35       self.FetchMetadata()
---> 36     return decoratee(self, *args, **kwargs)
     37   return _decorated
     38

/Library/Python/2.7/site-packages/pydrive/files.pyc in FetchContent(self, mimetype)
    208
    209     raise FileNotDownloadableError(
--> 210         'No downloadLink/exportLinks for mimetype found in metadata')
    211
    212   def Upload(self, param=None):

FileNotDownloadableError: No downloadLink/exportLinks for mimetype found in metadata

Error on large files

I cannot use this in uploading a file larger than 11GB.
File "/PyDrive-Backup/drive.py", line 17, in
file_or_folder.Upload()
File "build/bdist.linux-x86_64/egg/pydrive/files.py", line 225, in Upload
File "build/bdist.linux-x86_64/egg/pydrive/auth.py", line 54, in _decorated
File "build/bdist.linux-x86_64/egg/pydrive/files.py", line 241, in _FilesInsert
File "/usr/local/lib/python2.7/site-packages/googleapiclient/discovery.py", line 750, in method
payload = media_upload.getbytes(0, media_upload.size())
File "/usr/local/lib/python2.7/site-packages/googleapiclient/http.py", line 357, in getbytes
return self._fd.read(length)
MemoryError

Can you please fix this?

File permissions not changing

When I try to change permissions of same file to allow sharing by updating its metadata. The file is uploaded

drive = GoogleDrive(gauth)
file1 = drive.CreateFile({'title': 'hello.txt', 'shareable':True,  'userPermission': [{'kind':'drive#permission', 'type': 'anyone', 'value': 'anyone','role': 'reader'}]})
file1.SetContentFile('hello.txt')
file1.Upload()
print(file1['userPermission'])

However, when I print the userPermission, it doesn't seem to have set the permissions as I have intended.

I get:

{u'kind': u'drive#permission', u'etag': u'"TEeH8_qJkyJwjXXXXXXXXXXX/JoK_XXXXXXXXXXXXXXXXX"', u'role': u'owner', u'type': u'user', u'id': u'me', u'selfLink': u'https://www.googleapis.com/drive/v2/files/0B7vnOfySXXXXXXXXXXXXXX/permissions/me'}

Quickstart docs incorrect

Hi there. I'm trying to get started with PyDrive, but got stuck on the very first set of instructions on http://pythonhosted.org/PyDrive/quickstart.html. I presume that Google has changed the UI for us; probably the quickstart docs just need a slight refresh:

  1. The URL is now different; it redirects to another UI.
  2. After making a new project, there is no "Services menu".
  3. After figuring out how to translate this, I figured out how to enable Drive API, but then there is no "API Access" menu.

The UI is therefore confusing compared to the instructions. Once I figure out about credentials, there are several options, none of which seems to correspond to the local server hack that you suggest. There is no place at all to make a "OAuth2.0 client ID". There is "OAuth client ID" and "Service account key".

Going through with "Service account key" (which sounds like what I'm interested in; a script running on unix to access drive) seems straightforward, and makes a json file, but not as per the QuickStart.

I was able to go into "OAuth client ID" but there is no "more options" or hostname settings.

Then the url http://localhost:8080/ is not valid (as mentioned in another issue).

THEN, the file created isn't called client_secret.json; it's got a giant url and key in it.

Push to PyPi

I see that this code is now compatible with python3. But it hasn't been pushed to pypi since.
Could the code be pushed so those of us who are up to date can use this?

OAuth 2.0 Error

I followed your tutorial with python and the browser is opening the URL, which throws the error:

Error: invalid_client

no application name

Request Details
response_type=code
scope=https://www.googleapis.com/auth/drive
access_type=offline
redirect_uri=http://localhost:8080/
client_id=xxxxxxxxxx---------I removed it---------------x.apps.googleusercont

How to get a refresh_token?

Hi,

I'd like to get the refresh_token for the first time only so the app doesn't require subsequent authentication prompts (as this will be a script running in the background).

It works the first time but after the token expires it gives an error:

 gauth.Refresh()
  File "/Library/Python/2.7/site-packages/pydrive/auth.py", line 369, in Refresh
    raise RefreshError('No refresh_token found.'
pydrive.auth.RefreshError: No refresh_token found.Please set access_type of OAuth to offline.

This is the code handling the authentication:

# Sort out the authentication...
gauth = GoogleAuth()

# Try to load saved client credentials
gauth.LoadCredentialsFile(os.path.dirname(os.path.realpath(__file__)) + os.sep + "credentials.json")

if gauth.credentials is None:
    # Authenticate if they're not there
    gauth.LocalWebserverAuth()
elif gauth.access_token_expired:
    # Refresh them if expired
    gauth.Refresh()
else:
    # Initialize the saved creds
    gauth.Authorize()

Then I have a settings.yaml:

client_config_backend: settings
client_config:
  client_id: xxx
  client_secret: xxx

save_credentials: True
save_credentials_backend: file
save_credentials_file: <path_to_save_credentials.json>

get_refresh_token: True

oauth_scope:
  - https://www.googleapis.com/auth/drive.file
  - https://www.googleapis.com/auth/drive.install

I believe client_secrets.json is not required if you have settings.yaml, is that correct?

Any ideas why refresh_token is always NULL in the response credentials.json?

Exception: AttributeError: 'GoogleDriveFileList' object has no attribute 'auth_method'

I'm in PythonAnywhere.com cloud service running a Python3.5 environment.
I successfully cloned and installed this repo in this env.
Following the example, I successfully authorized, as none exception was thrown at my face, using a local credential.json file.
Unfortunately, the listing files example raised me the exception "AttributeError: 'GoogleDriveFileList' object has no attribute 'auth_method'". Googling about it I couldn't find any useful information.
Bellow is the full stack:

AttributeErrorTraceback (most recent call last)
<ipython-input-15-f4a2609ca408> in <module>()
      1 # Auto-iterate through all files that matches this query
----> 2 file_list = drive.ListFile({'q': "'root' in parents"}).GetList()
      3 for file1 in file_list:
      4   print('title: %s, id: %s' % (file1['title'], file1['id']))

/home/srodriguex/MYENV/lib/python3.5/site-packages/PyDrive-1.1.2-py3.5.egg/pydrive/apiattr.py in GetList(self)
    160       self['maxResults'] = 1000
    161       result = []
--> 162       for x in self:
    163         result.extend(x)
    164       del self['maxResults']

/home/srodriguex/MYENV/lib/python3.5/site-packages/PyDrive-1.1.2-py3.5.egg/pydrive/apiattr.py in __next__(self)
    144     if 'pageToken' in self and self['pageToken'] is None:
    145       raise StopIteration
--> 146     result = self._GetList()
    147     self['pageToken'] = self.metadata.get('nextPageToken')
    148     return result

/home/srodriguex/MYENV/lib/python3.5/site-packages/PyDrive-1.1.2-py3.5.egg/pydrive/auth.py in _decorated(self, *args, **kwargs)
     53     # Re-create access token if it expired.
     54     if self.auth.access_token_expired:
---> 55       if self.auth_method == 'service':
     56         self.auth.ServiceAuth()
     57       else:

AttributeError: 'GoogleDriveFileList' object has no attribute 'auth_method'

Installing pydrive through Pip causes an error

When importing PyDrive, after installing with the command sudo -H pip install -t "/usr/local/lib/python2.7/dist-packages" PyDrive, the import fails when attempting to import a PyDrive dependency, httplib2.

This issue has probably risen due to not porting PyDrive to Python3...

Wouldn't it just be viable to use lib2to3 to convert it all to Py3?

should close file after upload

Even though the comment of SetContentFile mentions that the content will be closed in upload, but it does not happen.

Two tests fail with python2.7

When running the testsuite, I get:

======================================================================
FAIL: test_01_LocalWebserverAuthWithClientConfigFromFile (pydrive.test.test_oauth.GoogleAuthTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dmitry/upstream/PyDrive/pydrive/test/test_oauth.py", line 19, in test_01_LocalWebserverAuthWithClientConfigFromFile
    self.CheckCredentialsFile('credentials/1.dat')
  File "/home/dmitry/upstream/PyDrive/pydrive/test/test_oauth.py", line 71, in CheckCredentialsFile
    self.assertEqual(ga.access_token_expired, no_file)
AssertionError: True != False

======================================================================
FAIL: test_02_LocalWebserverAuthWithClientConfigFromSettings (pydrive.test.test_oauth.GoogleAuthTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dmitry/upstream/PyDrive/pydrive/test/test_oauth.py", line 30, in test_02_LocalWebserverAuthWithClientConfigFromSettings
    self.CheckCredentialsFile('credentials/2.dat')
  File "/home/dmitry/upstream/PyDrive/pydrive/test/test_oauth.py", line 71, in CheckCredentialsFile
    self.assertEqual(ga.access_token_expired, no_file)
AssertionError: True != False

Other tests pass successfully.
When running the testsuite I have some auth prompts which I handle quickly, so this should not be a problem.

Uploading a huge file

Currently, I am using PyDrive to upload my backup to google drive.

Is there a special thing to do with this library to upload a huge file to Google Drive (around 5gb). In the Google Drive API documentation, it says that we must use the Resumable upload ? https://developers.google.com/drive/web/manage-uploads

My problem is that when I try to send a huge file, the script executes without any errors. However, if I do this with a small file around 100mb, everything works perfectly fine...

My code is the following:

def upload(self, backupFile, backupFileName):

json_data=open(os.path.join(__location__, 'client_secrets.json'))

data = json.load(json_data)

"""Email of the Service Account"""
SERVICE_ACCOUNT_EMAIL = data['client_email']

"""Path to the Service Account's Private Key file"""
SERVICE_ACCOUNT_PKCS12_FILE_PATH = os.path.join(__location__, 'key.p12')

f = file(SERVICE_ACCOUNT_PKCS12_FILE_PATH, 'rb')
key = f.read()
f.close()

credentials = SignedJwtAssertionCredentials(SERVICE_ACCOUNT_EMAIL, key,
    scope='https://www.googleapis.com/auth/drive', sub='email')
http = httplib2.Http()
credentials.authorize(http)

gauth = GoogleAuth()
gauth.credentials = credentials

drive = GoogleDrive(gauth)

file1 = drive.CreateFile({'title': backupFileName, "parents" : [{"id":"0B7FoN03AUUdZVlNETEtWLS1VTzQ"}]} )  # Create GoogleDriveFile instance with title 'Hello.txt'

file1.SetContentFile(backupFile);
file1.Upload()

When I try to send a large file, no errors are returned whatsoever. The python script simply ends without anything shown...

Cannot authenticate

Hi there,

I am having issues authenticated via PyDrive.

When I execute the fairly basic code:

settings_file = os.path.join(os.path.dirname(os.path.realpath(__file__)), 'settings.yaml')
gauth = GoogleAuth(settings_file=settings_file)
gauth.CommandLineAuth()

drive = GoogleDrive(gauth)

and follow the output url, and grant permissions (via the web browser), I keep seeing redirect_uri_mismatch followed by the description The redirect URI in the request: urn:ietf:wg:oauth:2.0:oob did not match a registered redirect URI.

I'm not sure what setting to change here. My settings.yaml file contains:

client_config_backend: settings
client_config:
  client_id: <CLIENT_ID>
  client_secret: <SECRET>
  redirect_uri: http://locahost:8080/

save_credentials: True
save_credentials_backend: file
save_credentials_file: drive_credentials.json

get_refresh_token: True

oauth_scope:
  - https://www.googleapis.com/auth/drive.file
  - https://www.googleapis.com/auth/drive.instal

And the Redirect Uri's and Javascript Origins are both set to http://localhost:8080/ in my Google API Console.

Does anyone have any ideas what is going on here?

large file download fails with OverflowError

On my 32-bit linux machine, files over 2GB fail to download. Memory usage while my test script runs gets very high, suggesting the entire download is being cached in memory; I think the download should be streamed to disk instead.

To use the script, upload a large file called bigvid.avi to google drive and put client_secrets.json in the working directory.

$ ./test.py 
bigvid.avi
Traceback (most recent call last):
  File "./test.py", line 17, in <module>
    f.GetContentFile('/tmp/bigvid-from-pydrive.avi')
  File "/usr/local/lib/python2.7/dist-packages/pydrive/files.py", line 167, in GetContentFile
    self.FetchContent(mimetype)
  File "/usr/local/lib/python2.7/dist-packages/pydrive/files.py", line 36, in _decorated
    return decoratee(self, *args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/pydrive/files.py", line 198, in FetchContent
    self.content = io.BytesIO(self._DownloadFromUrl(download_url))
  File "/usr/local/lib/python2.7/dist-packages/pydrive/auth.py", line 54, in _decorated
    return decoratee(self, *args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/pydrive/files.py", line 313, in _DownloadFromUrl
    resp, content = self.auth.service._http.request(url)
  File "/usr/local/lib/python2.7/dist-packages/oauth2client/util.py", line 135, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/oauth2client/client.py", line 547, in new_request
    redirections, connection_type)
  File "/usr/local/lib/python2.7/dist-packages/httplib2/__init__.py", line 1593, in request
    (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
  File "/usr/local/lib/python2.7/dist-packages/httplib2/__init__.py", line 1335, in _request
    (response, content) = self._conn_request(conn, request_uri, method, body, headers)
  File "/usr/local/lib/python2.7/dist-packages/httplib2/__init__.py", line 1318, in _conn_request
    content = response.read()
  File "/usr/lib/python2.7/httplib.py", line 541, in read
    return self._read_chunked(amt)
  File "/usr/lib/python2.7/httplib.py", line 624, in _read_chunked
    return ''.join(value)
OverflowError: join() result is too long for a Python string
$ cat test.py
#!/usr/bin/env python

from pydrive.auth import GoogleAuth
from pydrive.drive import GoogleDrive

gauth=GoogleAuth()
if not gauth.LoadCredentialsFile("auth.txt") :
    gauth.CommandLineAuth()
    gauth.SaveCredentialsFile("auth.txt")

drive=GoogleDrive(gauth)

filelist=drive.ListFile({'q': "title='bigvid.avi'"}).GetList()
for f in filelist:
    print f['title'];
    f.GetContentFile('/tmp/bigvid-from-pydrive.avi')

$ ls -l /tmp/big*
ls: cannot access /tmp/big*: No such file or directory

Exception raised on shutdown

When I close the python program, the following message pops up multiple times:

Exception ignored in: <bound method ApiAttribute.__del__ of 
    <pydrive.apiattr.ApiAttribute object at 0x7f819e8dabe0>>
TypeError: __del__() missing 1 required positional argument: 'obj'

I am running this code off the master, not PyPI, in Python 3.4.

Any ideas what is causing this error?

Documentation for Upload() unclear

I can successfully and repeatedly authenticate and upoad small files, but as soon as I start uploading larger files (~200Mb), PyDrive returns MemoryError.
I would like to pass a resumable=true parameter but it's unclear how to do so and the PyDrive documentation provides no examples that I've found.
I've tried backup.Upload({"resumable": "true"}) and variants but get errors such as TypeError: Got an unexpected keyword argument "resumable"
(running Upload() with no parameters works fine for files of a few kb)
A simple example of passing any parameter to Upload() would be very helpful.

List files in folder

It would be really useful to be able to request the list of all files within a folder specified by an ID, rather than just the result of a query. The API supports this through drive.children.list.

Bug in pydrive.settings

When scope is not defined in settings.yaml, for instance if you want to use only the default scope, initialization fails with "NoneType is not iterable" error.

The error is in line 176. Maybe you would like to fix it.

Cheers,

Jindra

auth.py behind a proxy

I am having problems with oauth2client

Created new window in existing browser session.
Traceback (most recent call last):
File "quickstart.py", line 5, in
gauth.LocalWebserverAuth()
File "/usr/local/lib/python2.7/dist-packages/pydrive/auth.py", line 79, in _decorated
self.Auth(code)
File "/usr/local/lib/python2.7/dist-packages/pydrive/auth.py", line 394, in Auth
self.Authenticate(code)
File "/usr/local/lib/python2.7/dist-packages/pydrive/auth.py", line 407, in Authenticate
self.credentials = self.flow.step2_exchange(code)
File "/usr/local/lib/python2.7/dist-packages/oauth2client/util.py", line 132, in positional_wrapper
return wrapped(_args, *_kwargs)
File "/usr/local/lib/python2.7/dist-packages/oauth2client/client.py", line 1283, in step2_exchange
headers=headers)
File "/usr/local/lib/python2.7/dist-packages/httplib2/init.py", line 1570, in request
(response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
File "/usr/local/lib/python2.7/dist-packages/httplib2/init.py", line 1317, in _request
(response, content) = self._conn_request(conn, request_uri, method, body, headers)
File "/usr/local/lib/python2.7/dist-packages/httplib2/init.py", line 1258, in _conn_request
raise ServerNotFoundError("Unable to find the server at %s" % conn.host)
httplib2.ServerNotFoundError: Unable to find the server at accounts.google.com

Googling on this problem it seems that the solution is to pass a http object to oauth2client to handle such requests as in :

https://groups.google.com/forum/#!msg/google-api-python-client/BoRZ6dnIKW0/1EOCXKCbskIJ

It seems that this should be done at auth.py at 407 in Authenticate, was hoping someone might be able to supply a patch, thanks!

Get file by id

Google Drive API doesn't support querying by id, just making a straight request to /file/:id (or something like it). Is it possible to add this functionality? Would it be difficult?

PyDrive not getting refresh token

I set the correct option in the settings.yaml file, and verified if it was being correctly parsed by checking if this condition was being met:

if self.settings.get('get_refresh_token'):
    self.flow.params.update({'access_type': 'offline'})

And it was. So, the "access_type" param is being correctly set (is it?). But I don't know why my credentials object has a None value for refresh_token.

authentication

Somehow I don't want to give permission each time I start my code. Is there a way to do this?

Upload to directory

Is there a supported way to upload to a given directory instead of the root?

Can't upload pdf/png files

Hi,

I'm having problems uploading files besides .txt files (or at least pdf/png I haven't tried other types).

Executing the following code enables me to upload .txt files:

from pydrive.auth import GoogleAuth
from pydrive.drive import GoogleDrive

gauth = GoogleAuth();
gauth.LoadCredentialsFile('/path/to/credfile/Usr_cred.txt');
gauth.Authorize();
drive = GoogleDrive(garth);

new_path_file = '/path/to/file/';
new_file = 'file.txt';
new_full_path = ''.join((new_path_file,new_file));
file_to_upload = drive.CreateFile({'title': new_file});
file_to_upload.SetContentFile(new_full_path);
file_to_upload.Upload();

However trying to upload a pdf or png file using the same code throws an encode error in python http.client

//anaconda/lib/python3.4/http/client.py in _send_request(self, method, url, body, headers)
   1123             # RFC 2616 Section 3.7.1 says that text default has a
   1124             # default charset of iso-8859-1.
-> 1125             body = body.encode('iso-8859-1')
   1126         self.endheaders(body)
   1127 

UnicodeEncodeError: 'latin-1' codec can't encode character '\ufffd' in position 259: ordinal not in range(256)

\ufffd is in unicode a replacement character and manually encoding with uft-8 works. Http an utf-8 doesn't work (as far as I know). The full error traceback is:

---------------------------------------------------------------------------
UnicodeEncodeError                        Traceback (most recent call last)
<ipython-input-38-1c5545d22e97> in <module>()
----> 1 file4.Upload()

//anaconda/lib/python3.4/site-packages/pydrive/files.py in Upload(self, param)
    223         self._FilesPatch(param=param)
    224     else:
--> 225       self._FilesInsert(param=param)
    226 
    227   @LoadAuth

//anaconda/lib/python3.4/site-packages/pydrive/auth.py in _decorated(self, *args, **kwargs)
     53     if self.auth.service is None:  # Check if drive api is built.
     54       self.auth.Authorize()
---> 55     return decoratee(self, *args, **kwargs)
     56   return _decorated
     57 

//anaconda/lib/python3.4/site-packages/pydrive/files.py in _FilesInsert(self, param)
    239       if self.dirty['content']:
    240         param['media_body'] = self._BuildMediaBody()
--> 241       metadata = self.auth.service.files().insert(**param).execute()
    242     except errors.HttpError as error:
    243       raise ApiRequestError(error)

//anaconda/lib/python3.4/site-packages/oauth2client/util.py in positional_wrapper(*args, **kwargs)
    138                 else:  # IGNORE
    139                     pass
--> 140             return wrapped(*args, **kwargs)
    141         return positional_wrapper
    142 

//anaconda/lib/python3.4/site-packages/googleapiclient/http.py in execute(self, http, num_retries)
    720 
    721       resp, content = http.request(str(self.uri), method=str(self.method),
--> 722                                    body=self.body, headers=self.headers)
    723       if resp.status < 500:
    724         break

//anaconda/lib/python3.4/site-packages/oauth2client/client.py in new_request(uri, method, body, headers, redirections, connection_type)
    594             resp, content = request_orig(uri, method, body,
    595                                          clean_headers(headers),
--> 596                                          redirections, connection_type)
    597 
    598             # A stored token may expire between the time it is retrieved and

//anaconda/lib/python3.4/site-packages/httplib2/__init__.py in request(self, uri, method, body, headers, redirections, connection_type)
   1312                     content = b""
   1313                 else:
-> 1314                     (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
   1315         except Exception as e:
   1316             if self.force_exception_to_status_code:

//anaconda/lib/python3.4/site-packages/httplib2/__init__.py in _request(self, conn, host, absolute_uri, request_uri, method, body, headers, redirections, cachekey)
   1062             auth.request(method, request_uri, headers, body)
   1063 
-> 1064         (response, content) = self._conn_request(conn, request_uri, method, body, headers)
   1065 
   1066         if auth:

//anaconda/lib/python3.4/site-packages/httplib2/__init__.py in _conn_request(self, conn, request_uri, method, body, headers)
    986                 if conn.sock is None:
    987                     conn.connect()
--> 988                 conn.request(method, request_uri, body, headers)
    989             except socket.timeout:
    990                 conn.close()

//anaconda/lib/python3.4/http/client.py in request(self, method, url, body, headers)
   1086     def request(self, method, url, body=None, headers={}):
   1087         """Send a complete request to the server."""
-> 1088         self._send_request(method, url, body, headers)
   1089 
   1090     def _set_content_length(self, body):

//anaconda/lib/python3.4/http/client.py in _send_request(self, method, url, body, headers)
   1123             # RFC 2616 Section 3.7.1 says that text default has a
   1124             # default charset of iso-8859-1.
-> 1125             body = body.encode('iso-8859-1')
   1126         self.endheaders(body)
   1127 

UnicodeEncodeError: 'latin-1' codec can't encode character '\ufffd' in position 259: ordinal not in range(256)

Any suggestions?

Christian

thread safety

Hi,

Is PyDrive thread-safe (or at least the Upload method)? Or is there a way to create an independent copy of the GoogleDrive instance? I tried to deepcopy the drive and also to create a new drive with the same GoogleAuth object. But I get a BadStatusLine: '' Error when trying to create and upload a new file with it.

Thx,
joe

Losing file content during upload()

Hi All,

This is also an open question on stack, but after no real response there I thought I ought to take it to the source.

I currently have a 34x22 .xlsx spreadsheet. I am downloading it via pydrive, filling in some of the blank values, and uploading the file back via pydrive. When I upload the file back, all cells with formulas are blank (any cell that starts with =). I have a local copy of the file I want to upload, and it looks fine so I'm pretty sure the issue must be with pydrive.

My code:

def upload_r1masterfile(filename='temp.xlsx'):
        """
        Upload a given file to drive as our master file
        :param filename: name of local file to upload
        :return:
        """
        # Get the file we want
        master_file = find_r1masterfile()
        try:
            master_file.SetContentFile(filename)
            master_file.Upload()
            print 'Master file updated. ' + str(datetime.datetime.now())
        except Exception, e:
            print "Warning: Something wrong with file R1 Master File."
            print str(e)
            return e

The only hint I have is that if I add the param={'convert': True}tag to Upload, then there is no loss. However, that means I am now working in google sheets format, and I would rather not do that. Not only because it's not the performed format to work with here, but also because if I try to master_file.GetContentFile(filename) I get the error: No downloadLink/exportLinks for mimetype found in metadata

Any hints? Is there another attribute on upload that I am not aware of?

I haven't found any clues in the documentation, so my recommendation would be to add something describing possible parameters in the Upload function, as I found that only after some googling and no where in the documentation.

Thanks!

AuthenticationError while accessing PyDrive from Google App Engine App

Hello Developers,

I am using PyDrive in my Google App Engine App locally. I have added the pydrive in src/lib/pydrive also its dependency src/lib/PyYAML-3.10.

Now I am using it from my views.py file

I have included

from pydrive.auth import GoogleAuth
from pydrive.drive import GoogleDrive

at the top and then I wrote a view

@app.route('/oauth-authorized/')
def federated_sign():
  gauth = GoogleAuth('test1.yaml')
  print gauth
  gauth.LocalWebserverAuth()
  drive = GoogleDrive(gauth)
  print "Done Authentication"
  file1 = drive.CreateFile({'title': 'Hello.txt'})  # Create GoogleDriveFile instance with title 'Hello.txt'
  file1.SetContentString('Hello World!') # Set content of the file from given string
  file1.Upload()

I am getting Error at gauth.LocalWebserverAuth() name AuthenticationError and this message on Terminal

Failed to start a local webserver. Please check your firewall settings and locally running programs that may be blocking or using configured ports. Default ports are 8080 and 8090.

My Redirect URI and JavaScript Origins are
Redirect URIs: http://localhost:8080/oauth-authorized/ ,
http://localhost:8080/ ,
http://localhost:5000/oauth-authorized/ ,

JavaScript origins: http://localhost:8080/, http://localhost:5000/

I followed the Error and found it is in pydrive/auth.py line 69

Here is the snippet that might causing the error

in ChechAuth class

if self.credentials is None:
      code = decoratee(self, *args, **kwargs)

I am able to use the test applications provided in the API. Please let me know Where i am doing wrong

Authenticating with existing access token?

I have a Google authentication flow in another part of my web application, so I am already getting users' access tokens. Is there a way to create a GoogleDrive client using this existing access token?

No tests to verify large downloads work.

All files are downloaded into io.BytesIO objects which store all contents in RAM and could result in files larger than the available space resulting in a crash.

Byte order marks are set at the start of my docs

It seems that pydrive or google puts BOMs at the start of my files which renders them useless for me .

I hope you guys know how to remove them. I alrdy did research on that and couldnt fix it with command line tools. Maybe there is a option which i can set to remove BOM.

Maybe u can add an option for removing them in future

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.