jovianhq / opendatasets Goto Github PK

View Code? Open in Web Editor NEW

321.0 321.0 139.0 26.56 MB

A Python library for downloading datasets from Kaggle, Google Drive, and other online sources.

License: MIT License

Python 99.20% Shell 0.80%

data-science datasets machine-learning python

opendatasets's People

Contributors

Stargazers

Watchers

Forkers

indianvalantine kushaljana vishal-7 jcgit2018 sir1989 amanullachishty mrzombie69232 pratik20002 jaisejohnson shakilanwar saurabh1barasiya sushilrajeeva vedantmadane rajasagar ak0029 agatha-l faysalmiah55 lippdas abobo555 msoftware fake-kira iamvazu swaileh edmontdants pranuh kashy64 aly202012 sanawat777 yohannes-didana butterfly2012010 karthy257 vspatilim1 trentinos rustydigg918 saritassamil mahal97 rubenmccarty praveenkumar5051 sban84 1rdas cybergeekgyan akshatjn1313 shushlav suryatmodulus in6days ldruizsan abhishek-99ks siddhu1132 sriram1010 madhankumarnagaraji mustafasamy28 jmwas keepithunnyt coubanao prachi9127-code hgsnkzm admariner faisalshahbaz kritij19 mhbsheikh sandhyaammasi jishnukoliyadan j84m9 sabrinsingh qfizik kshitijahuja17 pjc510 kcpele farhadkazemzadeh dinu2881 esha0910 kongshu2212 srijandeb karthi-govindharaju ch-bhavani zeekliviu jobaerlisedu azadmehedi lancelot82 hlxavier macmullensantiago ojoagomoses favourogboi rishi128166 kiliyo kamalshkeir khambro12 funshoelias ingle-santosh jedaiahredden leecvz dilpatrai andregulyi muhammadanas0716 michalboc salma-gaam trellixvulnteam theomegalightning maheshg8plrq2p raun1997

opendatasets's Issues

Downloading datasets behind network Proxies fail due to timeout errors

For users behind network proxies, the following example in the main README.md fails due to timeout errors:

$ python
Python 3.9.18 (main, Sep 11 2023, 13:41:44) 
[GCC 11.2.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import opendatasets as od
>>> dataset_url = 'https://www.kaggle.com/tunguz/us-elections-dataset'
>>> od.download(dataset_url)
Please provide your Kaggle credentials to download this dataset. Learn more: http://bit.ly/kaggle-creds
Your Kaggle username: ****
Your Kaggle Key: 
2024-01-12 06:45:08,854 WARNING Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f1a5408e490>: Failed to establish a new connection: [Errno 110] Connection timed out')': /api/v1/datasets/download/tunguz/us-elections-dataset?datasetVersionNumber=None

However if KAGGLE_PROXY environment variable is properly set, the example works for users behind network proxy as well:

Here's the code snippet that makes this work:

import os
if 'https_proxy' in os.environ.keys():
    os.environ['KAGGLE_PROXY'] = os.environ['https_proxy']
elif 'HTTPS_PROXY' in os.environ.keys():
    os.environ['KAGGLE_PROXY'] = os.environ['HTTPS_PROXY']
else:
    os.environ['KAGGLE_PROXY'] = ''

import opendatasets as od
dataset_url = 'https://www.kaggle.com/tunguz/us-elections-dataset'
od.download(dataset_url)

and here's the sample run behind network proxy:

python
Python 3.9.18 (main, Sep 11 2023, 13:41:44) 
[GCC 11.2.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> if 'https_proxy' in os.environ.keys():
...     os.environ['KAGGLE_PROXY'] = os.environ['https_proxy']
... elif 'HTTPS_PROXY' in os.environ.keys():
...     os.environ['KAGGLE_PROXY'] = os.environ['HTTPS_PROXY']
... else:
...     os.environ['KAGGLE_PROXY'] = ''
... 
>>> import opendatasets as od
>>> dataset_url = 'https://www.kaggle.com/tunguz/us-elections-dataset'
>>> od.download(dataset_url)
Please provide your Kaggle credentials to download this dataset. Learn more: http://bit.ly/kaggle-creds
Your Kaggle username: ****
Your Kaggle Key: 
Downloading us-elections-dataset.zip to ./us-elections-dataset
  0%|                                                                        | 0.00/133k [00:00<?, ?B/s]
100%|████████████████████████████████████████████████████████████████| 133k/133k [00:00<00:00, 6.49MB/s]

I was planning to submit a PR to fix the issue but I see that the last time this repo updated was over 2 years ago.

Can't import opendatasets on Colab

Trying to import the library on Colab gives this error:


AttributeError: module 'urllib' has no attribute 'urlopen'

Clean up minimal example of README

You can clean up the example from the README from:

import opendatasets as od
dataset_url = 'https://www.kaggle.com/tunguz/us-elections-dataset'
od.download('https://www.kaggle.com/tunguz/us-elections-dataset')

to:

import opendatasets as od
od.download('https://www.kaggle.com/tunguz/us-elections-dataset')

or:

import opendatasets as od
dataset_url = 'https://www.kaggle.com/tunguz/us-elections-dataset'
od.download(dataset_url)

Kaggle JSON (in different place)

aakashns

This is a useful repository. Thank you for sharing.

FYI: Digging into the code, the method "read_kaggle_creds()" looks for the kaggle.json at the current directory, "./kaggle.json", but most of the time, the kaggle.json is stored at "~/.kaggle/kaggle.json"

Thanks.

Return the output path

Return an object, that contains the data target directory and datafiles.
I would use it to load the data into the notebook.

ApiException: (404) Reason: Not Found

I am using Google Colab where I am downloading this dataset : https://www.kaggle.com/datasets/hereisburak/pins-face-recognition
using opendatasets library

I've uploaded my kaggle.json into the files section of Google Colab as well but when I am doing od.download(url) I am getting following error :

I always do this same process but this time it is not working
So what could be the reason and how to resolve it ?

versions :
Python : 3.7.13
opendatasets : 0.1.20

jovianhq / opendatasets Goto Github PK

opendatasets's People

Contributors

Stargazers

Watchers

Forkers

opendatasets's Issues

Downloading datasets behind network Proxies fail due to timeout errors

Can't import opendatasets on Colab

Clean up minimal example of README

Kaggle JSON (in different place)

Return the output path

ApiException: (404) Reason: Not Found

Opendataset

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent

Jobs