space-physics / pyzenodo3

Simple, clean pure Python 3 Zenodo API (upload, download).

PyZenodo

Pure Python wrapper for Zenodo REST API.

Allows upload / download of data from Zenodo.

Install

pip install pyzenodo3

Latest development

git clone https://github.com/scivision/pyzenodo3

pip install -e pyzenodo3

Usage

Here are several examples of using Zenodo from Python 3. All of them assume you have first:

import pyzenodo3

zen = pyzenodo3.Zenodo()

Upload file to Zenodo

0. Get a Zenodo deposit:write API Token. This token must remain private, NOT uploaded to GitHub, etc.!

1. create a simple text file mymeta.ini containing title, author etc. (see the example meta.ini in this repo)

2. upload file to Zenodo (myApiToken is the cut-n-pasted Zenodo API text token)

   python pyzenodo3.upload myApiToken mymeta.ini myfile.zip --use-sandbox

Note the --use-sandbox is to avoid making junk uploads while testing out. Once you're sure things are working as intended, not using that flag uploads to "real" Zenodo permanently.

Find Zenodo record by Github repo

Rec = zen.find_record__by_github_repo('scivision/lowtran')

This Zenodo Record contains the metadata that can be further manipulated in a simple class containing the data in dictionaries, with a few future helper methods.

Find Zenodo records by Github username

Recs = zen.search('scivision')

Recs is a list of Zenodo Records for the GitHub username queried, as in the example above.

Notes

• We don't use deposit:publish API token to keep a human-in-the-loop in case of hacking of sensor nodes.

pyzenodo3's Issues

Support using sandbox API

Hello, thanks for this lib.

Zenodo has a sandbox site and API at https://sandbox.zenodo.org. Would be useful to be able switch to that while testing things out.

ConnectionError "WSAECONNRESET" when uploading a 1GB ZIP:

When trying to upload a 1GB ZIP to Zenodo using the UploadZenodo.py file, I get this:

runfile('C:/Users/user/Dropbox/TMP/ZENODO/UploadZenodo.py', wdir='C:/Users/user/Dropbox/TMP/ZENODO')
writing JSON metadata to meta.json
Traceback (most recent call last):

  File "C:\Users\user\Anaconda3\envs\tfgpu\lib\site-packages\urllib3\contrib\pyopenssl.py", line 340, in _send_until_done
    return self.connection.send(data)

  File "C:\Users\user\Anaconda3\envs\tfgpu\lib\site-packages\OpenSSL\SSL.py", line 1757, in send
    self._raise_ssl_error(self._ssl, result)

  File "C:\Users\user\Anaconda3\envs\tfgpu\lib\site-packages\OpenSSL\SSL.py", line 1663, in _raise_ssl_error
    raise SysCallError(errno, errorcode.get(errno))

SysCallError: (10054, 'WSAECONNRESET')


During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "C:\Users\user\Anaconda3\envs\tfgpu\lib\site-packages\urllib3\connectionpool.py", line 672, in urlopen
    chunked=chunked,

  File "C:\Users\user\Anaconda3\envs\tfgpu\lib\site-packages\urllib3\connectionpool.py", line 387, in _make_request
    conn.request(method, url, **httplib_request_kw)

  File "C:\Users\user\Anaconda3\envs\tfgpu\lib\http\client.py", line 1262, in request
    self._send_request(method, url, body, headers, encode_chunked)

  File "C:\Users\user\Anaconda3\envs\tfgpu\lib\http\client.py", line 1308, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)

  File "C:\Users\user\Anaconda3\envs\tfgpu\lib\http\client.py", line 1257, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)

  File "C:\Users\user\Anaconda3\envs\tfgpu\lib\http\client.py", line 1075, in _send_output
    self.send(chunk)

  File "C:\Users\user\Anaconda3\envs\tfgpu\lib\http\client.py", line 996, in send
    self.sock.sendall(data)

  File "C:\Users\user\Anaconda3\envs\tfgpu\lib\site-packages\urllib3\contrib\pyopenssl.py", line 352, in sendall
    data[total_sent : total_sent + SSL_WRITE_BLOCKSIZE]

  File "C:\Users\user\Anaconda3\envs\tfgpu\lib\site-packages\urllib3\contrib\pyopenssl.py", line 346, in _send_until_done
    raise SocketError(str(e))

OSError: (10054, 'WSAECONNRESET')


During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "C:\Users\user\Anaconda3\envs\tfgpu\lib\site-packages\requests\adapters.py", line 449, in send
    timeout=timeout

  File "C:\Users\user\Anaconda3\envs\tfgpu\lib\site-packages\urllib3\connectionpool.py", line 720, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]

  File "C:\Users\user\Anaconda3\envs\tfgpu\lib\site-packages\urllib3\util\retry.py", line 400, in increment
    raise six.reraise(type(error), error, _stacktrace)

  File "C:\Users\user\Anaconda3\envs\tfgpu\lib\site-packages\urllib3\packages\six.py", line 734, in reraise
    raise value.with_traceback(tb)

  File "C:\Users\user\Anaconda3\envs\tfgpu\lib\site-packages\urllib3\connectionpool.py", line 672, in urlopen
    chunked=chunked,

  File "C:\Users\user\Anaconda3\envs\tfgpu\lib\site-packages\urllib3\connectionpool.py", line 387, in _make_request
    conn.request(method, url, **httplib_request_kw)

  File "C:\Users\user\Anaconda3\envs\tfgpu\lib\http\client.py", line 1262, in request
    self._send_request(method, url, body, headers, encode_chunked)

  File "C:\Users\user\Anaconda3\envs\tfgpu\lib\http\client.py", line 1308, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)

  File "C:\Users\user\Anaconda3\envs\tfgpu\lib\http\client.py", line 1257, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)

  File "C:\Users\user\Anaconda3\envs\tfgpu\lib\http\client.py", line 1075, in _send_output
    self.send(chunk)

  File "C:\Users\user\Anaconda3\envs\tfgpu\lib\http\client.py", line 996, in send
    self.sock.sendall(data)

  File "C:\Users\user\Anaconda3\envs\tfgpu\lib\site-packages\urllib3\contrib\pyopenssl.py", line 352, in sendall
    data[total_sent : total_sent + SSL_WRITE_BLOCKSIZE]

  File "C:\Users\user\Anaconda3\envs\tfgpu\lib\site-packages\urllib3\contrib\pyopenssl.py", line 346, in _send_until_done
    raise SocketError(str(e))

ProtocolError: ('Connection aborted.', OSError("(10054, 'WSAECONNRESET')",))


During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "C:\Users\user\Dropbox\TMP\ZENODO\UploadZenodo.py", line 30, in <module>
    zup.upload(metafn,data_file, access_token, base_url=base_url)

  File "C:\Users\user\Anaconda3\envs\tfgpu\lib\site-packages\pyzenodo3\upload.py", line 133, in upload
    upload_data(token, datafn, depid, base_url)

  File "C:\Users\user\Anaconda3\envs\tfgpu\lib\site-packages\pyzenodo3\upload.py", line 101, in upload_data
    files={"file": datafn.open("rb")},

  File "C:\Users\user\Anaconda3\envs\tfgpu\lib\site-packages\requests\api.py", line 119, in post
    return request('post', url, data=data, json=json, **kwargs)

  File "C:\Users\user\Anaconda3\envs\tfgpu\lib\site-packages\requests\api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)

  File "C:\Users\user\Anaconda3\envs\tfgpu\lib\site-packages\requests\sessions.py", line 530, in request
    resp = self.send(prep, **send_kwargs)

  File "C:\Users\user\Anaconda3\envs\tfgpu\lib\site-packages\requests\sessions.py", line 643, in send
    r = adapter.send(request, **kwargs)

  File "C:\Users\user\Anaconda3\envs\tfgpu\lib\site-packages\requests\adapters.py", line 498, in send
    raise ConnectionError(err, request=request)

ConnectionError: ('Connection aborted.', OSError("(10054, 'WSAECONNRESET')",))

I see that a new data entry in Zenodo is created, but there is no data uploaded. Is this a firewall problem?

Thanks for your help!

Meta info upload doesn't work

Seems like it's currently commented out in upload.py and un commenting it causes errors.

Uploading to existing Zenodo REPO

Great project! :-)
Is it possible to use the records to push/upload data to an existing not published repo using this package? How would I do that?

Download all files from record

Hi,

The Readme says we can download from Zenodo, how can I do that with pyzenodo please? Preferably, I'd like to download all files in the same time.

Thanks,

Missing license file in pypi-tar-archive

Hey,

i tried to install your library via pip, but i get the following message: error: [Errno 2] No such file or directory: 'LICENSE'. This file is missing in your tar archive on pypi.org.

Also i had to install the package wheel manually, so maybe you have a missing dependency, too?

Thank you and have a good day.