rkhwaja / fs.onedrivefs Goto Github PK

Like fs.googledrivefs

If you pass None for the token, it gets all the way down to the network call before throwing, for example, DirectoryExpected from listdir.

Proper solution is to pass the tokens themselves into the constructor and not expose the oauthlib format

pynacl has a dependency on cffi, which doesn't have an M1 wheel.
Waiting for pyca/pynacl#654

Need wheel for cffi and maybe for lazy-object-proxy, pynacl, pyyaml, typed_ast and wrapt

See rclone/rclone#557

It's an option to override the virus checking on downloads

So that we don't have to manually update the read only token

For calling from async http servers it's easier to call async

Since OAuth2Session that is used in this library supports different types of OAuth2 clients it would be convenient to be able to use the client that works with the type of token grant flow that is set up with the authentication against MS Graph API.

Currently the Authorzation Code Grant is the only one supported, since the default OAuth2 client is WebApplicationClient.

We have a different setup and the refreshing of tokens doesn't seem to work due to this.

So my request is to add support for this.

Does anyone know if there is, or plans to, support fsspec so we can do one liner read/write data frames to/from onedrive?

see https://docs.pyfilesystem.org/en/latest/extension.html#opener

For example, https://travis-ci.org/rkhwaja/fs.onedrivefs/jobs/631726534

Upgrading to 1.1.3 makes pkg_resources.require("anything") raise a DistributionNotFound error.

Repro:

mkdir onedrivefs113
cd onedrivefs113
python -m venv venv
. venv/bin/activate
pip install fs.onedrivefs==1.1.3
python
>>> import pkg_resources
>>> pkg_resources.require("fs.onedrivefs")

Output:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "venv/lib/python3.9/site-packages/pkg_resources/__init__.py", line 891, in require
    needed = self.resolve(parse_requirements(requirements))
  File "venv/lib/python3.9/site-packages/pkg_resources/__init__.py", line 777, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'fs.onedrivefs' distribution was not found and is required by the application

Doing the same with 1.1.1:

>>> import pkg_resources
>>> pkg_resources.require("fs.onedrivefs")
[fs.onedrivefs 1.1.1 (venv/lib/python3.9/site-packages), requests-oauthlib 1.3.1 (venv/lib/python3.9/site-packages), requests 2.28.1 (venv/lib/python3.9/site-packages), fs 2.4.16 (venv/lib/python3.9/site-packages), requests 2.28.1 (venv/lib/python3.9/site-packages), oauthlib 3.2.2 (venv/lib/python3.9/site-packages), certifi 2022.9.24 (venv/lib/python3.9/site-packages), urllib3 1.26.12 (venv/lib/python3.9/site-packages), idna 3.4 (venv/lib/python3.9/site-packages), charset-normalizer 2.1.1 (venv/lib/python3.9/site-packages), six 1.16.0 (venv/lib/python3.9/site-packages), setuptools 60.5.0 (venv/lib/python3.9/site-packages), appdirs 1.4.4 (venv/lib/python3.9/site-packages)]

@rkhwaja I can't see any differences and the CI workflow does not include any poetry build command so it's unclear how you build these wheels to get repeatable builds. I do notice though that you have built v1.1.1 and v1.1.3 using different poetry versions and building with poetry 1.1.x instead of 1.0.x is changing some things. For example, the lock file no longer contains platform markers so if you build with poetry build you will be fine, but if you do a poetry export the platform markers etc will be gone in the requirements file. There are also some strange duplication going on in the supported python version metadata in the new wheel.

I was able to help out with the PR for the fix in 1.1.2 but here I can't help out since the build command is not contained in the CI workflow.

It might be that the copy isn't finished even though it says it is

https://docs.microsoft.com/en-us/onedrive/developer/rest-api/concepts/using-webhooks?view=odsp-graph-online

OneDrive has a feature where they will convert the file format before downloading. Some of the pyfilesystem methods support an options dictionary which you could use to pass the format.

https://docs.microsoft.com/en-us/onedrive/developer/rest-api/api/driveitem_get_content_format?view=odsp-graph-online

https://techcommunity.microsoft.com/t5/azure-active-directory-identity/update-your-applications-to-use-microsoft-authentication-library/ba-p/1257363

Have to switch to using the Graph API and authentication

AssertionError: Unexpected status: {'operation': 'itemCopy', 'percentageComplete': 100.0, 'resourceId': 'xxxxxxxxxxxxxxxx!xxxxx', 'status': 'completed'}

Supposedly need it for download conversion to jpg, for instance.

https://stackoverflow.com/a/50610600/3573

To lessen memory usage for big files

I think we can use FileIO (optionally) instead of BytesIO

When the log level is set to info the logs are filled with information from fs.onedrivefs that is irrelevant in most cases. The logs are too chatty/verbose and it would be really nice if it could be possible to reduce the logging level for all calls in OneDriveFS to debug instead if info.

pyfilesystem side documentation is here

Use this Graph API endpoint

It will allow much faster copy - at the moment it downloads and uploads the data