Comments (6)
Actually not sure. There are differences (possibly on purpose) between
https://github.com/dask/adlfs/blob/master/requirements.txt
and
https://github.com/dask/adlfs/blob/master/setup.py#L22L27
i.e. requirements.txt
has two extra packages:
pytest>=5.0,<6.0
requests>=2.22.0,<3.0
from adlfs.
pytest can certainly be moved to dev-requirements.txt
from adlfs.
What about requests? Perhaps it is installed with one of the other packages. I'll check
from adlfs.
Tested creating requirements.txt as:
azure-core>=1.5.0
azure-datalake-store>=0.0.46,<0.1
azure-identity
azure-storage-blob>=12.0.0,<12.4.0
fsspec>=0.8.0
msrestazure
and ran:
conda create -n test_env python=3.8
conda activate test_env
pip install -r requirements.txt
which installs all package below. It brings in requests 2.24.0
which fits the pinned version requirement of >=2.22.0,<3.0
. Is pinning it at <3.0
superfluous because the latest release is 2.24.0
https://github.com/psf/requests/releases. Given the adlfs
works for the latest version of requests
does it need to be pinned?.
(test_env) C:\Users\131416\Desktop>conda list
# packages in environment at C:\Users\131416\AppData\Local\Continuum\anaconda3\envs\test_env:
#
# Name Version Build Channel
adal 1.2.4 pypi_0 pypi
azure-core 1.8.0 pypi_0 pypi
azure-datalake-store 0.0.49 pypi_0 pypi
azure-identity 1.4.0 pypi_0 pypi
azure-storage-blob 12.3.2 pypi_0 pypi
ca-certificates 2020.6.24 0
certifi 2020.6.20 py38_0
cffi 1.14.2 pypi_0 pypi
chardet 3.0.4 pypi_0 pypi
cryptography 3.0 pypi_0 pypi
fsspec 0.8.0 pypi_0 pypi
idna 2.10 pypi_0 pypi
isodate 0.6.0 pypi_0 pypi
msal 1.4.3 pypi_0 pypi
msal-extensions 0.2.2 pypi_0 pypi
msrest 0.6.18 pypi_0 pypi
msrestazure 0.6.4 pypi_0 pypi
oauthlib 3.1.0 pypi_0 pypi
openssl 1.1.1g he774522_1
pip 20.2.2 py38_0
portalocker 1.7.1 pypi_0 pypi
pycparser 2.20 pypi_0 pypi
pyjwt 1.7.1 pypi_0 pypi
python 3.8.5 he1778fa_0
python-dateutil 2.8.1 pypi_0 pypi
pywin32 228 pypi_0 pypi
requests 2.24.0 pypi_0 pypi
requests-oauthlib 1.3.0 pypi_0 pypi
setuptools 49.6.0 py38_0
six 1.15.0 pypi_0 pypi
sqlite 3.33.0 h2a8f88b_0
urllib3 1.25.10 pypi_0 pypi
vc 14.1 h0510ff6_4
vs2015_runtime 14.16.27012 hf0eaf9b_3
wheel 0.34.2 py38_0
wincertstore 0.2 py38_0
zlib 1.2.11 h62dcd97_4
from adlfs.
I've updated setup.py to pull from requirements.txt in commit db9dd79 and pulled it into master. The pin to requests<3.0 exists due to a bias that any major change in the requests package should be validated, but you raise a fair question. Will open another issue to track that conversation.
from adlfs.
Believe this can be reopened via #101
from adlfs.
Related Issues (20)
- ls does not work with protocol prefix HOT 1
- `'AzureBlobFile' object has no attribute 'forced'` in `__del__` on bad file mode
- AzureBlobFileSystem to support CertificateCredential HOT 2
- Support for storage accounts in the URL (Azure Data Lake Storage Gen2 URI support) HOT 2
- Support DataLakeServiceClient for Azure Gen 2 Storage HOT 5
- Maybe fsspec can't working well in Azure CN
- Question: wasb[s] protocol support
- Encryption-in-transit support
- Specifying one ffspec url to one azure blob in duckdb read_parquet causes an ls scan of the entire container
- Cannot read parquet from Azure when archive file exists
- `AzureBlobFileSystem.ls(path)` returns inconsistent results HOT 2
- DefaultAzureCredential() not being used with anon=False keyword passed in storage_options HOT 2
- New release? HOT 1
- Concurrent reads/writes cause BlobNotFound
- TypeError: unsupported operand type(s) for -: 'NoneType' and 'int' HOT 6
- Support for MS Fabric OneLake
- support az cli config ~/.azure/config HOT 1
- UserWarning: Failed to fetch container properties for CONTAINER_NAME. Assume it exists already HOT 1
- "sdk_moniker" key error HOT 9
- Avoid private APIs from azure.storage HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from adlfs.