GithubHelp home page GithubHelp logo

webdav-client-python-3's Introduction

Python WebDAV Client 3

Build Status Quality Gate Status Coverage PyPI PyPI - Python Version

Package webdavclient3 based on https://github.com/designerror/webdav-client-python but uses requests instead of PyCURL. It provides easy way to work with WebDAV-servers.

Installation

$ pip install webdavclient3

Sample Usage

from webdav3.client import Client
options = {
 'webdav_hostname': "https://webdav.server.ru",
 'webdav_login':    "login",
 'webdav_password': "password"
}
client = Client(options)
client.verify = False # To not check SSL certificates (Default = True)
client.session.proxies(...) # To set proxy directly into the session (Optional)
client.session.auth(...) # To set proxy auth directly into the session (Optional)
client.execute_request("mkdir", 'directory_name')

Webdav API

Webdav API is a set of webdav actions of work with cloud storage. This set includes the following actions: check, free, info, list, mkdir, clean, copy, move, download, upload, publish and unpublish.

Configuring the client

Required key is host name or IP address of the WevDAV-server with param name webdav_hostname.
For authentication in WebDAV server use webdav_login, webdav_password.
For an anonymous login do not specify auth properties.

from webdav3.client import Client

options = {
 'webdav_hostname': "https://webdav.server.ru",
 'webdav_login':    "login",
 'webdav_password': "password"
}
client = Client(options)

If your server does not support HEAD method or there are other reasons to override default WebDAV methods for actions use a dictionary option webdav_override_methods. The key should be in the following list: check, free, info, list, mkdir, clean, copy, move, download, upload, publish and unpublish. The value should a string name of WebDAV method, for example GET.

from webdav3.client import Client

options = {
 'webdav_hostname': "https://webdav.server.ru",
 'webdav_login':    "login",
 'webdav_password': "password",
 'webdav_override_methods': {
            'check': 'GET'
        }

}
client = Client(options)

For configuring a requests timeout you can use an option webdav_timeout with int value in seconds, by default the timeout is set to 30 seconds.

from webdav3.client import Client

options = {
 'webdav_hostname': "https://webdav.server.ru",
 'webdav_login':    "login",
 'webdav_password': "password",
 'webdav_timeout': 30
}
client = Client(options)

When a proxy server you need to specify settings to connect through it.

from webdav3.client import Client
options = {
 'webdav_hostname': "https://webdav.server.ru",
 'webdav_login':    "w_login",
 'webdav_password': "w_password", 
 'proxy_hostname':  "http://127.0.0.1:8080",
 'proxy_login':     "p_login",
 'proxy_password':  "p_password"
}
client = Client(options)

If you want to use the certificate path to certificate and private key is defined as follows:

from webdav3.client import Client
options = {
 'webdav_hostname': "https://webdav.server.ru",
 'webdav_login':    "w_login",
 'webdav_password': "w_password",
 'cert_path':       "/etc/ssl/certs/certificate.crt",
 'key_path':        "/etc/ssl/private/certificate.key"
}
client = Client(options)

Or you want to limit the speed or turn on verbose mode:

options = {
 ...
 'recv_speed' : 3000000,
 'send_speed' : 3000000,
 'verbose'    : True
}
client = Client(options)

recv_speed: rate limit data download speed in Bytes per second. Defaults to unlimited speed.
send_speed: rate limit data upload speed in Bytes per second. Defaults to unlimited speed.
verbose: set verbose mode on/off. By default verbose mode is off.

Also if your server does not support check it is possible to disable it:

options = {
 ...
 'disable_check': True
}
client = Client(options)

By default, checking of remote resources is enabled.

For configuring chunk size of content downloading use chunk_size param, by default it is 65536

options = {
 ...
 'chunk_size': 65536
}
client = Client(options)

Synchronous methods

# Checking existence of the resource

client.check("dir1/file1")
client.check("dir1")
# Get information about the resource

client.info("dir1/file1")
client.info("dir1/")
# Check free space

free_size = client.free()
# Get a list of resources

files1 = client.list()
files2 = client.list("dir1")
files3 = client.list("dir1", get_info=True) # returns a list of dictionaries with files details
# Create directory

client.mkdir("dir1/dir2")
# Delete resource

client.clean("dir1/dir2")
# Copy resource

client.copy(remote_path_from="dir1/file1", remote_path_to="dir2/file1")
client.copy(remote_path_from="dir2", remote_path_to="dir3")
# Move resource

client.move(remote_path_from="dir1/file1", remote_path_to="dir2/file1")
client.move(remote_path_from="dir2", remote_path_to="dir3")
# Download a resource

client.download_sync(remote_path="dir1/file1", local_path="~/Downloads/file1")
client.download_sync(remote_path="dir1/dir2/", local_path="~/Downloads/dir2/")
# Upload resource

client.upload_sync(remote_path="dir1/file1", local_path="~/Documents/file1")
client.upload_sync(remote_path="dir1/dir2/", local_path="~/Documents/dir2/")
# Publish the resource

link = client.publish("dir1/file1")
link = client.publish("dir2")
# Unpublish resource

client.unpublish("dir1/file1")
client.unpublish("dir2")
# Exception handling

from webdav3.client import WebDavException
try:
...
except WebDavException as exception:
...
# Get the missing files

client.pull(remote_directory='dir1', local_directory='~/Documents/dir1')
# Send missing files

client.push(remote_directory='dir1', local_directory='~/Documents/dir1')

Asynchronous methods

# Load resource

kwargs = {
 'remote_path': "dir1/file1",
 'local_path':  "~/Downloads/file1",
 'callback':    callback
}
client.download_async(**kwargs)

kwargs = {
 'remote_path': "dir1/dir2/",
 'local_path':  "~/Downloads/dir2/",
 'callback':    callback
}
client.download_async(**kwargs)
# Unload resource

kwargs = {
 'remote_path': "dir1/file1",
 'local_path':  "~/Downloads/file1",
 'callback':    callback
}
client.upload_async(**kwargs)

kwargs = {
 'remote_path': "dir1/dir2/",
 'local_path':  "~/Downloads/dir2/",
 'callback':    callback
}
client.upload_async(**kwargs)

Resource API

Resource API using the concept of OOP that enables cloud-level resources.

# Get a resource

res1 = client.resource("dir1/file1")
# Work with the resource

res1.rename("file2")
res1.move("dir1/file2")
res1.copy("dir2/file1")
info = res1.info()
res1.read_from(buffer)
res1.read(local_path="~/Documents/file1")
res1.read_async(local_path="~/Documents/file1", callback)
res1.write_to(buffer)
res1.write(local_path="~/Downloads/file1")
res1.write_async(local_path="~/Downloads/file1", callback)

For Contributors

Prepare development environment

  1. Install docker on your development machine
  2. Start WebDAV server for testing by following commands from the project's root folder or change path to conf dir in second command to correct:
docker pull bytemark/webdav
docker run -d --name webdav -e AUTH_TYPE=Basic -e USERNAME=alice -e PASSWORD=secret1234 -v conf:/usr/local/apache2/conf -p 8585:80 bytemark/webdav

Code convention

Please check your code according PEP8 Style guides.

Run tests

  1. Check that webdav container is started on your local machine
  2. Execute following command in the project's root folder:
python -m unittest discover -s tests

Prepare a Pull Request

Please use this check list before creating PR:

  1. You code should be formatted according PEP8
  2. All tests should successfully pass
  3. Your changes shouldn't change previous default behaviour, exclude defects
  4. All changes are covered by tests

webdav-client-python-3's People

Contributors

a-guzhin avatar a1ezzz avatar adclose avatar ajordanbbn avatar appetito avatar bboehmke avatar danielloader avatar dzhuang avatar ezhov-evgeny avatar f100024 avatar huangganggui avatar james-pcdr avatar janlo avatar jesperhakansson avatar jorgeajimenezl avatar liuliqiu avatar matrixx567 avatar mephinet avatar michael-o avatar nuwang avatar philipprieth avatar rpcmoritz avatar skeen avatar www avatar xmaples avatar yeus avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

webdav-client-python-3's Issues

Cannot ignore self signed certificates

Hi,

the verify option does not work because and option request is sent in the constructor thus there is no way to set the verify to false before that.

Client.move() не обрабатывает кириллицу при работе с NextCloud

При работе с сервером NextCloud, если в названии файла или папки есть кириллица, то Client.move() или Сlient.clean() падают с ошибкой UnicodeEncodeError: 'latin-1' codec can't encode characters in position 52-55: ordinal not in range(256). При этом другие функции (.mkdir(), .info(), .list(), .download_file(), .upload_file()) работают нормально

download_sync appends last remote_path element causing Remote resource error

I find that the last element of the path is appended and causes a Remote resource error.

Example code and response:

# Imports, requires `pip install webdavclient3`
from webdav3.client import Client

# Set options
options = {
    'webdav_hostname':
'https://owncloud.local/remote.php/dav/files/username',
    'webdav_login'   : 'username',
    'webdav_password': 'token'
}

# Set client
client = Client(options)

# List root
client.list("/")
[
 'username/',
 'BOXER/',
 'Documents/'
]
# Download
client.download_sync(remote_path="BOXER/", local_path="~/BOXER-local")
---------------------------------------------------------------------------
RemoteResourceNotFound                    Traceback (most recent call last)
<ipython-input-47-2d7b3e02ed57> in <module>
     21
     22 # Download
---> 23 client.download_sync(remote_path="BOXER/",
local_path="~/BOXER-local")

/opt/conda/lib/python3.7/site-packages/webdav3/client.py in
download_sync(self, remote_path, local_path, callback)
    395         :param callback: the callback which will be invoked when
downloading is complete.
    396         """
--> 397         self.download(local_path=local_path,
remote_path=remote_path)
    398         if callback:
    399             callback()

/opt/conda/lib/python3.7/site-packages/webdav3/client.py in download(self,
remote_path, local_path, progress)
    338         urn = Urn(remote_path)
    339         if self.is_dir(urn.path()):
--> 340             self.download_directory(local_path=local_path,
remote_path=remote_path, progress=progress)
    341         else:
    342             self.download_file(local_path=local_path,
remote_path=remote_path, progress=progress)

/opt/conda/lib/python3.7/site-packages/webdav3/client.py in
download_directory(self, remote_path, local_path, progress)
    362             _remote_path =
"{parent}{name}".format(parent=urn.path(), name=resource_name)
    363             _local_path = os.path.join(local_path, resource_name)
--> 364             self.download(local_path=_local_path,
remote_path=_remote_path, progress=progress)
    365
    366     @wrap_connection_error

/opt/conda/lib/python3.7/site-packages/webdav3/client.py in download(self,
remote_path, local_path, progress)
    337         """
    338         urn = Urn(remote_path)
--> 339         if self.is_dir(urn.path()):
    340             self.download_directory(local_path=local_path,
remote_path=remote_path, progress=progress)
    341         else:

/opt/conda/lib/python3.7/site-packages/webdav3/client.py in _wrapper(self,
*args, **kw)
     68         log.debug("Requesting %s(%s, %s)", fn, args, kw)
     69         try:
---> 70             res = fn(self, *args, **kw)
     71         except requests.ConnectionError:
     72             raise NoConnection(self.webdav.hostname)

/opt/conda/lib/python3.7/site-packages/webdav3/client.py in is_dir(self,
remote_path)
    607         urn = Urn(remote_path)
    608         parent_urn = Urn(urn.parent())
--> 609         self._check_remote_resource(remote_path, urn)
    610
    611         response = self.execute_request(action='info',
path=parent_urn.quote())

/opt/conda/lib/python3.7/site-packages/webdav3/client.py in
_check_remote_resource(self, remote_path, urn)
    595     def _check_remote_resource(self, remote_path, urn):
    596         if not self.check(urn.path()) and not
self.check(Urn(remote_path, directory=True).path()):
--> 597             raise RemoteResourceNotFound(remote_path)
    598
    599     @wrap_connection_error

RemoteResourceNotFound: Remote resource: /BOXER/BOXER/ not found

Add option to avoid HTTP request in Client constructor

Since this commit, the Client constructor performs a request to the WebDAV server on construction.

This is pretty annoying as it now breaks a lot of unit tests on my side where I used dumb URLs for wrapper object, and I am sure it will impact other users as well.

Would it be possible to add a constructor argument to bypass this?

Content type property support

Currently, when calling list() or info() the client does not attempt to parse the <d:getcontenttype> property.

This is odd, because it is part of the standard.

It woud be very useful if it could be parsed and returned in the result.

Is there any other way I can access this property in the response?

Not work mkdir

Directories are not created on any service via mkdir. I managed to create a directory when using upload_dir on yandex, but I got an error and the files did not load.

>>> client.mkdir("fdvfdv") Traceback (most recent call last): File "<stdin>", line 1, in <module> File "C:\Program Files (x86)\Python38-32\lib\site-packages\webdav3\client.py", line 69, in _wrapper res = fn(self, *args, **kw) File "C:\Program Files (x86)\Python38-32\lib\site-packages\webdav3\client.py", line 306, in mkdir raise RemoteParentNotFound(directory_urn.path()) webdav3.exceptions.RemoteParentNotFound: Remote parent for: /fdvfdv/ not found

>>> client.execute_request("mkdir", "33") Traceback (most recent call last): File "<stdin>", line 1, in <module> File "C:\Program Files (x86)\Python38-32\lib\site-packages\webdav3\client.py", line 233, in execute_request raise ResponseErrorCode(url=self.get_url(path), code=response.status_code, message=response.content) webdav3.exceptions.ResponseErrorCode: Request to https://dav.box.com/dav33 failed with code 403 and message: b'<?xml version="1.0" encoding="utf-8"?>\n<d:error xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns">\n <s:exception>Sabre_DAV_Exception_Forbidden</s:exception>\n <s:message>Permission denied to create node</s:message>\n</d:error>\n'

Not all webdav servers implement HEAD on folders

Stackoverflow is not in unison about the question if webdav servers must implement the HEAD request on folders: https://stackoverflow.com/questions/16578345/should-webdav-server-support-head-on-folders

Anyway, the webdav server in my Synology NAS does not implement HEAD on folders. It answers with status code 404 on the root folder and status code 403 on nested folders. If I do a PROPFIND on the same path it gives me information about the path.

(See https://gist.github.com/Phaiax/1b186336eb3c6d457705a1564ed112a4 for some curl tests
Edit: I tried it all with trailing / and without, the results were the same.)

This has the effect that I can't list folders because list() first checks if the folder exists:

        if directory_urn.path() != Client.root and not self.check(directory_urn.path()):
            raise RemoteResourceNotFound(directory_urn.path())

Check uses the HEAD method on the folder, and that fails.

I propose that list() should call is_dir() instead of check(). Also, the info action should also set Depth=0 instead of Depth=1 in the header because the directory listing is not needed in the cases where the info action is used. Thirdly, document that not all servers support check() on folders, even if the RFC says otherwise.

Thanks, Daniel

Question about parallel operation

If I want to upload or download massive files, typically 100k.
And I would like to make sure it only spawn at most 5 concurrent requests at the same time(from the server side aspect).
What is the recommendation of this implementation?
Can I set maximum concurrent requests number?
And if I use python Threading to multi-thread it,
should they share one client instance(and use download_async) or spawn for each thread worker(and use download_sync)?
Thank you

RemoteResourceNotFound for info, download, rename...

while using the webdav-client for python 3 I experience an issue very similar to CloudPolis#18 and CloudPolis#22.

I can list contents of a directory and check if a file exists but cannot get info or perform any resource operations (rename, move, etc). Uploading goes without any issues.

I have tried setting the root to /remote.php/webdav as suggested in the CloudPolis solutions but that seems to limit the number of folders I can see. It does reveal the info.

The webdav server I am trying to connect to called Stack and is a service of TransIP stack

The version of the client is webdavclient3==0.12.

Feature: Authorise with bearer token instead of basic auth

Hey this is a great library and been using it extensively in scripts, but I would like to propose the option to authorise using a header, Authorization: Bearer $token instead of Authorization: Basic $base64auth.

Would this be feasible?

Set `timeout` from input

Hello! I am using webdav-client-python-3 to talk to a WebDAV server - it's working really great! However, the hardcoded timeout value of 30 sec used in the HTTP requests is too short for my use case. Would it be possible to add the time out value as an optional setting to initialise the client? Thanks in advance!

Problem with initial connect in release 0.14

I have the following problem, I'm connecting to a propietary WebDAV service, but the service is in a path that wasn't root and root directory answers a 404 error.

In the 0.14 release the client in the __init__ function make a test connection to the root

response = self.execute_request('options', '')

In my special case, it raises a NotFound error when connect, so maybe is possibly to give a root path or a config to not test the connection at start.

Thanks!

Use connection pooling for metadata queries

When listing directories, it looks like connection pooling isn't being used.
Since I am listing a lot of directories, I am cycling through the ephemeral port list around 20 times, before doing anything else.
I would probably significantly reduce time and load on both ends, if I was able to reuse a single connection, instead of spawning 200000.

I believe the issue why connection pooling isn't being used automatically (as advertised by requests/urllib3) is that the connections created are always streaming, even if it's just a PROPFIND, which doesn't really benefit from the streaming.

My proposed solution would be to make the streaming flag in the request conditional upon the method being called. stream=True for actual up/downloads, and stream=False for list

getting access to the response?

I am curious why methods like upload_to, upload, etc only return None instead of returning the response. I am happy to write a patch to return the response if it would be acceptable.

My use case: I want to check the return code to ensure it is a clean 200 after doing an upload.

Thanks for the library!

Extend upload functions with "headers_ext" parameter?

Hi,

Problem:
I used this library to upload files into a Nextcloud instance, but was unhappy with the fact, that the "creation-date" and the "last modified-date" was set by Nextcloud to the "current date". In my case I lost a great deal of information.

After some research I found that this is not a problem of the webdav client, but the server and as there is no "generic" way, all servers use slightly different approaches to overcome this problem.

Solution
For Own/NextCloud it's simply an additional header added to the "upload" request ("X-OC-CTime: " plus int unix timestamp for "creation time" and "X-OC-MTime: " plus int unix timestamp for "modification time").

As the "execute_request" function has the ability to "take" additional headers ("headers_ext"), it was quite simple to provide the required "header" via the "upload" function to "execute_request".

Question
Is it possible to extend the API for all (or at least for "upload") to contain the "header_ext" parameter, as this would give users a better flexability?

For my use-case only uploading of files requires this extension, but for other use-cases it might be required in other api calls as well.

Regards

Angus

List return rootpath

When files is:
a
-b
-c
list('a') will return ['a', 'b', 'c']
When files is:
a
-a
-b
-c
list('a') will return ['a', 'b', 'c', 'a']

It makes me confused. i expect that it don't return the parent dir name.

Client fails when the webdav_root is actually needed to quote

Methods of Client, .list, .info, .download*, .pull, .push, ..., etc., fail when the webdav_root is actually needed to quote. For example, we have a webdav_root='/remote.php/webdav/steph(2021)', in which the parentheses are actually needed to quote, so we get a quoted webdav_root, '/remote.php/webdav/steph%282021%29', and next, Client.list(path) fails to exclude the target path itself, Client.info fails to find resources, ....

This issue can be easily reproduced with any version (until now latest 3.14.6) of client, a nextcloud server, the webdav_root mentioned above.

Here is a screenshot about the issue:
quote_root
The true structure of files in the remote is:

steph(2021)/
└── file1.txt

"mkdir" Problem

client.mkdir('test1/test2/test3')

webdav3.exceptions.RemoteParentNotFound: Remote parent for: test1/test2/test3 not found

Unexpected/wrong behaviour in combination with nextcloud

Is this library compatiple with nextcloud? I'm experiencing very odd+wrong behaviour.

In the code below I first check for existing and non-existing directories and files. Afterwards I create a new directory and sync a single file.

Based on that test, every tested function doesn't work as I expect it to based on the readme,

  • check returns always true for files and always false for directories. Except if the directory doesn' exist. In that case it returns true.
  • mkdir returns true, indicating that the directory was created, but thats not the case (check with the nextcloud webview)
  • upload_sync doesn't upload the file but also raises no error.

`

def issue():
  c = {
      'webdav_hostname': config.nextcloud_url,
      'webdav_login':    config.username,
      'webdav_password': config.password
  }
  client = Client(c)

  # exists
  print('Check existing: expect 3x True')
  print(client.check('Photos'))
  print(client.check('Nextcloud.png'))
  print(client.check('Photos/Frog.jpg'))

  #doesn't exist
  print('Check non-existing: expect 3x False')
  print(client.check('test_Documents')) 
  print(client.check('test3.pdf')) 
  print(client.check('test_Photos/Frog.jpg'))

  # create
  r = client.mkdir('test_Documents')
  r2 = client.check('test_Documents')
  print('Add new dir: expect 2x True')
  print(r, r2)

  client.upload_sync(remote_path='test3.pdf', local_path='../test3.pdf')
  r3 = client.check('test3.pdf')
  print('Check sync file: expect True')
  print(r3)

  print('Check new dir, synced file')
  client = Client(c)
  r = client.check('test_Documents')
  r2 = client.check('test3.pdf')
  print('expect 2x True')
  print(r, r2)

Output:

Check existing: expect 3x True
False
True
True
Check non-existing: expect 3x False
True
True
True
Add new dir: expect 2x True
True True
Check sync file: expect True
True
Check new dir + synced file: expect 2x True
False True

Verify credentials

Hi,

When I tried to download a file using the code below, I got a RemoteResourceNotFound error:

from webdav3.client import Client
webdav = Client({
    "webdav_hostname": "XXXX",
    "webdav_login":  "XXXX",
    "webdav_password":  "XXXX"
})
webdav.download_sync(remote_path="some filename", local_path="some path")
Traceback (most recent call last):
[..]
  File "/home/bauer/conda/lib/python3.7/site-packages/webdav3/client.py", line 409, in download_sync
    self.download(local_path=local_path, remote_path=remote_path)
  File "/home/bauer/conda/lib/python3.7/site-packages/webdav3/client.py", line 349, in download
    if self.is_dir(urn.path()):
  File "/home/bauer/conda/lib/python3.7/site-packages/webdav3/client.py", line 66, in _wrapper
    res = fn(self, *args, **kw)
  File "/home/bauer/conda/lib/python3.7/site-packages/webdav3/client.py", line 622, in is_dir
    self._check_remote_resource(remote_path, urn)
  File "/home/bauer/conda/lib/python3.7/site-packages/webdav3/client.py", line 610, in _check_remote_resource
    raise RemoteResourceNotFound(remote_path)
webdav3.exceptions.RemoteResourceNotFound: Remote resource: /HUUD55DU.zip not found

After some debugging I realized that the source of this error was not a missing file, but wrong credentials. More specifically the password was not correct. I think the error is misleading here, since it does not point to an authentication problem. Additionally, I wonder if there is a way to verify login credentials before making any other request? I didn't find one.

ordering of directory contents

Hi again,
If I do something like this

client = Client(options)
all_files = client.list()

and then iterate through it, the ordering seems random. Is there a way to have the list method return it in a specific order (timestamp, filename, etc.) or is this not implemented (or even out of scope of your module)?
thanks!

list files recursive

would be nice to have an option in the function to get the file list recursively should be as simple as adding a header
Depth = infinity

just a thought..
Aaron

code 400, message Bad request syntax ('0') on my Server

Hi,
I have a djangodav webdav server
https://github.com/MnogoByte/djangodav

I upload files with this client.
That results in the following console log:

[19/Feb/2021 14:27:24] "HEAD /auth/webdav/24/input/ HTTP/1.1" 200 0
[19/Feb/2021 14:27:26] "PUT /auth/webdav/24/input/Freilauf.CATPart HTTP/1.1" 201 0
[19/Feb/2021 14:27:26] code 400, message Bad request syntax ('0')
[19/Feb/2021 14:27:26] "0" 400 -

the upload works perfectly. But what does the Bad request message mean?
Does this clients is sending a bad request in not the corryt syntax, or are the requests correct and the problem is, that djangodav is just expecting something different?

Extend requests with custom http headers

We had an issue due to a strict mod_security that suddenly started to drop all requests. Changing the user-agent seems to fix the issue.
Current webdav-client-python-3 implementation support custom headers for all commands with the headers_ext parameter, however this is not used by the main methods as list, info and so on. From our point of view, the better would be to set headers_ext at client connection.

specify remote timezone

When we calls client.push, the client compare the local time and remote time.
Sometimes, the remote server's timezone is not the same with local timezone. It maybe cause some misunderstood.

if local_resource_name in remote_resource_names and not self.is_local_more_recent(local_path, remote_path):

Maybe we can introduce a new variable, specify the remote timezone params.

options = {
 'webdav_hostname': "https://webdav.server.ru",
 'webdav_login':    "login",
 'webdav_password': "password",
 'remote_timezone': 'Asia/Shanghai', # new variable
}
client = Client(options)

In the self.is_local_more_recent function, we use the timezone param:

def is_local_more_recent(self, local_path, remote_path):

try:
        remote_info = self.info(remote_path)
        remote_last_mod_date = remote_info['modified']
        remote_last_mod_date = dateutil_parser.parse(remote_last_mod_date)
        # change the timezone
        if self.remote_timezone:
            remote_last_mod_date = remote_last_mod_date.astimezone(tz.gettz(self.remote_timezone)) 
        remote_last_mod_date_unix_ts = int(remote_last_mod_date.timestamp())
except Exception:
        pass

handle webdav_hostname tailing slash

Hi, I am new to webdav, and I just spent hours to get client.list() to work, then I just find out that I have a tailing slash in config['webdav_hostname'] that makes me failed. It has a tailing slash because the URL from my service provider has it and I didn't think it matters until now. Should the library have the ability to handle tailing slash?

download file get an empty file

HI, when I download file from webdav server with, an empty file downloaded by download_sync.

It is very difficult to reappear. And I have a question about https://github.com/ezhov-evgeny/webdav-client-python-3/blob/09ae1414ddd331775686ebe55a625e732ef78b14/webdav3/client.py#L195 , Is it only when response.status_code == 507, == 404 , == 405, >= 400 raise exceptions? Is it possible response other status_code when download file(GET)? If other status_code come, https://github.com/ezhov-evgeny/webdav-client-python-3/blob/09ae1414ddd331775686ebe55a625e732ef78b14/webdav3/client.py#L399 it may create an empty file but not write anything. I don't know if that's the reason why I get an empty file.

3.14.5 import WebDavException fails

Hi,
it seems with version 3.14.5, I can't import WebDavException anymore. It worked with 3.14.4.
Anything changed in this area?
Thanks!

Traceback (most recent call last):
  File "/Users/jojo/.venvs/discodos/bin/discosync", line 6, in <module>
    from discodos.cmd.sync import main
  File "/Users/jojo/git/discodos/discodos/cmd/sync.py", line 21, in <module>
    from webdav3.client import WebDavException
ImportError: cannot import name 'WebDavException' from 'webdav3.client' (/Users/jojo/.venvs/discodos/lib/python3.7/site-packages/webdav3/client.py)

Abort connection. Upload not work.

Python 3.8.0 (tags/v3.8.0:fa919fd, Oct 14 2019, 19:21:23) [MSC v.1916 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from webdav3.client import Client
>>> options = {
...  'webdav_hostname': "https://webdav.yandex.ru",
...  'webdav_login':    "my_login",
...  'webdav_password': "my_pass"
... }
>>> client = Client(options)
>>> client.free()
10737418240
>>> client.list()
['33/', '55.txt']
>>> client.upload_sync(remote_path="33/123.txt", local_path="C:\\Users\\USER\\Desktop\\123.txt")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\webdav3\client.py", line 497, in upload_sync
    self.upload(local_path=local_path, remote_path=remote_path)
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\webdav3\client.py", line 433, in upload
    self.upload_file(local_path=local_path, remote_path=remote_path)
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\webdav3\client.py", line 68, in _wrapper
    res = fn(self, *args, **kw)
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\webdav3\client.py", line 484, in upload_file
    raise RemoteParentNotFound(urn.path())
webdav3.exceptions.RemoteParentNotFound: Remote parent for: /33/123.txt not found
>>> client.upload_sync(remote_path="33\\123.txt", local_path="C:\\Users\\USER\\Desktop\\123.txt")
Traceback (most recent call last):
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\urllib3\connectionpool.py", line 665, in urlopen
    httplib_response = self._make_request(
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\urllib3\connectionpool.py", line 376, in _make_request
    self._validate_conn(conn)
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\urllib3\connectionpool.py", line 994, in _validate_conn
    conn.connect()
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\urllib3\connection.py", line 386, in connect
    self.sock = ssl_wrap_socket(
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\urllib3\util\ssl_.py", line 370, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "C:\Program Files (x86)\Python38-32\lib\ssl.py", line 500, in wrap_socket
    return self.sslsocket_class._create(
  File "C:\Program Files (x86)\Python38-32\lib\ssl.py", line 1040, in _create
    self.do_handshake()
  File "C:\Program Files (x86)\Python38-32\lib\ssl.py", line 1309, in do_handshake
    self._sslobj.do_handshake()
ConnectionResetError: [WinError 10054] Удаленный хост принудительно разорвал существующее подключение

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\requests\adapters.py", line 439, in send
    resp = conn.urlopen(
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\urllib3\connectionpool.py", line 719, in urlopen
    retries = retries.increment(
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\urllib3\util\retry.py", line 400, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\urllib3\packages\six.py", line 734, in reraise
    raise value.with_traceback(tb)
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\urllib3\connectionpool.py", line 665, in urlopen
    httplib_response = self._make_request(
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\urllib3\connectionpool.py", line 376, in _make_request
    self._validate_conn(conn)
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\urllib3\connectionpool.py", line 994, in _validate_conn
    conn.connect()
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\urllib3\connection.py", line 386, in connect
    self.sock = ssl_wrap_socket(
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\urllib3\util\ssl_.py", line 370, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "C:\Program Files (x86)\Python38-32\lib\ssl.py", line 500, in wrap_socket
    return self.sslsocket_class._create(
  File "C:\Program Files (x86)\Python38-32\lib\ssl.py", line 1040, in _create
    self.do_handshake()
  File "C:\Program Files (x86)\Python38-32\lib\ssl.py", line 1309, in do_handshake
    self._sslobj.do_handshake()
urllib3.exceptions.ProtocolError: ('Connection aborted.', ConnectionResetError(10054, 'Удаленный хост принудительно разорвал существующее подключение', None, 10054, None))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\webdav3\client.py", line 68, in _wrapper
    res = fn(self, *args, **kw)
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\webdav3\client.py", line 280, in check
    response = self.execute_request(action='check', path=urn.quote())
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\webdav3\client.py", line 162, in execute_request
    response = self.session.request(
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\requests\sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\requests\sessions.py", line 668, in send
    history = [resp for resp in gen] if allow_redirects else []
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\requests\sessions.py", line 668, in <listcomp>
    history = [resp for resp in gen] if allow_redirects else []
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\requests\sessions.py", line 239, in resolve_redirects
    resp = self.send(
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\requests\sessions.py", line 646, in send
    r = adapter.send(request, **kwargs)
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\requests\adapters.py", line 498, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(10054, 'Удаленный хост принудительно разорвал существующее подключение', None, 10054, None))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\webdav3\client.py", line 497, in upload_sync
    self.upload(local_path=local_path, remote_path=remote_path)
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\webdav3\client.py", line 433, in upload
    self.upload_file(local_path=local_path, remote_path=remote_path)
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\webdav3\client.py", line 68, in _wrapper
    res = fn(self, *args, **kw)
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\webdav3\client.py", line 483, in upload_file
    if not self.check(urn.parent()):
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\webdav3\client.py", line 70, in _wrapper
    raise NoConnection(self.webdav.hostname)
webdav3.exceptions.NoConnection: Not connection with https://webdav.yandex.ru
>>> client.upload_sync(remote_path="123.txt", local_path="C:\\Users\\USER\\Desktop\\123.txt")
Traceback (most recent call last):
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\urllib3\connectionpool.py", line 665, in urlopen
    httplib_response = self._make_request(
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\urllib3\connectionpool.py", line 376, in _make_request
    self._validate_conn(conn)
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\urllib3\connectionpool.py", line 994, in _validate_conn
    conn.connect()
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\urllib3\connection.py", line 386, in connect
    self.sock = ssl_wrap_socket(
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\urllib3\util\ssl_.py", line 370, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "C:\Program Files (x86)\Python38-32\lib\ssl.py", line 500, in wrap_socket
    return self.sslsocket_class._create(
  File "C:\Program Files (x86)\Python38-32\lib\ssl.py", line 1040, in _create
    self.do_handshake()
  File "C:\Program Files (x86)\Python38-32\lib\ssl.py", line 1309, in do_handshake
    self._sslobj.do_handshake()
ConnectionResetError: [WinError 10054] Удаленный хост принудительно разорвал существующее подключение

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\requests\adapters.py", line 439, in send
    resp = conn.urlopen(
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\urllib3\connectionpool.py", line 719, in urlopen
    retries = retries.increment(
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\urllib3\util\retry.py", line 400, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\urllib3\packages\six.py", line 734, in reraise
    raise value.with_traceback(tb)
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\urllib3\connectionpool.py", line 665, in urlopen
    httplib_response = self._make_request(
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\urllib3\connectionpool.py", line 376, in _make_request
    self._validate_conn(conn)
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\urllib3\connectionpool.py", line 994, in _validate_conn
    conn.connect()
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\urllib3\connection.py", line 386, in connect
    self.sock = ssl_wrap_socket(
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\urllib3\util\ssl_.py", line 370, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "C:\Program Files (x86)\Python38-32\lib\ssl.py", line 500, in wrap_socket
    return self.sslsocket_class._create(
  File "C:\Program Files (x86)\Python38-32\lib\ssl.py", line 1040, in _create
    self.do_handshake()
  File "C:\Program Files (x86)\Python38-32\lib\ssl.py", line 1309, in do_handshake
    self._sslobj.do_handshake()
urllib3.exceptions.ProtocolError: ('Connection aborted.', ConnectionResetError(10054, 'Удаленный хост принудительно разорвал существующее подключение', None, 10054, None))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\webdav3\client.py", line 68, in _wrapper
    res = fn(self, *args, **kw)
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\webdav3\client.py", line 280, in check
    response = self.execute_request(action='check', path=urn.quote())
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\webdav3\client.py", line 162, in execute_request
    response = self.session.request(
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\requests\sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\requests\sessions.py", line 668, in send
    history = [resp for resp in gen] if allow_redirects else []
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\requests\sessions.py", line 668, in <listcomp>
    history = [resp for resp in gen] if allow_redirects else []
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\requests\sessions.py", line 239, in resolve_redirects
    resp = self.send(
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\requests\sessions.py", line 646, in send
    r = adapter.send(request, **kwargs)
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\requests\adapters.py", line 498, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(10054, 'Удаленный хост принудительно разорвал существующее подключение', None, 10054, None))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\webdav3\client.py", line 497, in upload_sync
    self.upload(local_path=local_path, remote_path=remote_path)
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\webdav3\client.py", line 433, in upload
    self.upload_file(local_path=local_path, remote_path=remote_path)
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\webdav3\client.py", line 68, in _wrapper
    res = fn(self, *args, **kw)
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\webdav3\client.py", line 483, in upload_file
    if not self.check(urn.parent()):
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\webdav3\client.py", line 70, in _wrapper
    raise NoConnection(self.webdav.hostname)
webdav3.exceptions.NoConnection: Not connection with https://webdav.yandex.ru

Error with download() and download_sync() in 3.14.6

Steps to reproduce

  1. docker run -it python:3.9-slim-buster bash
  2. pip3 install webdavclient3==3.14.6
  3. run script:
    from webdav3.client import Client
    options = { 'webdav_hostname': HOSTNAME, 'webdav_login': LOGIN, 'webdav_password': PASSWORD, 'webdav_root': DAVROOT, 'disable_check': True }
    client = Client(options)
    client.download(remote_path=TESTFILE,local_path="./test.txt")

Error

Traceback (most recent call last):
File "//webdav-test.py", line 13, in
client.download(remote_path=TESTFILE,local_path="./test.txt")
File "/usr/local/lib/python3.9/site-packages/webdav3/client.py", line 397, in download
self.download_file(local_path=local_path, remote_path=remote_path, progress=progress,
File "/usr/local/lib/python3.9/site-packages/webdav3/client.py", line 67, in _wrapper
res = fn(self, *args, **kw)
File "/usr/local/lib/python3.9/site-packages/webdav3/client.py", line 458, in download_file
total = int(response.headers['content-length'])
File "/usr/local/lib/python3.9/site-packages/requests/structures.py", line 54, in getitem
return self._store[key.lower()][1]
KeyError: 'content-length'

Current workaround
Force downgrade to 3.14.5: pip3 install webdavclient3==3.14.5

Is webdavclient3 v0.13 multiprocess safe ?

I just upgrade to version 0.13 with no code modified.
raise error
ERROR - name 'hostname' is not defined

When I downgrade to 0.12. everything is okay. I am using webdavclient3 in multiprocessing.pool. Is webdavclient3 v0.13 multiprocess safe ?

Downloaded tar.gz files are automatically "extracted"

Hi,

I have a question regarding the download method which calls def download_file(self, remote_path, local_path, progress=None):. When I download a tar.gz file from a WebDAV server, the file is much bigger than on the WebDAV server. I realized that the file getting extracted during the download process. I read that this can happen if you are not using the .read method:

f.write(response.raw) instead of f.write(response.raw.read())

Is this a normal behavior? Is it possible to download a file without getting extracted?

Thanks in advance!

Don't solicit GET againt webserver root

This snippet:

if self.session.auth:
self.session.request(method="GET", url=self.webdav.hostname, verify=self.verify, timeout=self.timeout) # (Re)Authenticates against the proxy

is issuing request which our server will reject with 403. I don't know which proxy is meant in this spot, but either remove this unrequested roundtrip or add an option to disable this.

This spills logs w/o any reason:

2021-04-30 11:42:06,358 [documents-job-worker_2] ERROR requests_gssapi.gssapi_: handle_other(): Mutual authentication unavailable on 403 response
2021-04-30 11:42:06,417 [documents-job-worker_2] ERROR requests_gssapi.gssapi_: handle_other(): Mutual authentication unavailable on 403 response
2021-04-30 11:42:06,547 [documents-job-worker_2] ERROR requests_gssapi.gssapi_: handle_other(): Mutual authentication unavailable on 403 response
2021-04-30 11:42:06,661 [documents-job-worker_2] ERROR requests_gssapi.gssapi_: handle_other(): Mutual authentication unavailable on 403 response
2021-04-30 11:42:06,800 [documents-job-worker_2] ERROR requests_gssapi.gssapi_: handle_other(): Mutual authentication unavailable on 403 response
2021-04-30 11:42:06,859 [documents-job-worker_2] ERROR requests_gssapi.gssapi_: handle_other(): Mutual authentication unavailable on 403 response
2021-04-30 11:42:06,997 [documents-job-worker_2] ERROR requests_gssapi.gssapi_: handle_other(): Mutual authentication unavailable on 403 response
2021-04-30 11:42:07,055 [documents-job-worker_2] ERROR requests_gssapi.gssapi_: handle_other(): Mutual authentication unavailable on 403 response
2021-04-30 11:42:07,188 [documents-job-worker_2] ERROR requests_gssapi.gssapi_: handle_other(): Mutual authentication unavailable on 403 response

download_from() incorrectly handles gzipped response

To reproduce, try creating an empty file on a server that supports gzip encoding, then download it using download_from:

from io import BytesIO
buf = BytesIO()
client.download_from(buf, 'http://example.com/empty-file')
print(buf.getvalue())

Instead of empty content (b''), I'm getting b'\x1f\x8b\x08\x00\x00\x00\x00\x00\x00\x03\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00', which is gzipped empty content.

It looks like the method uses response.raw, which does not decode gzip. A demonstration (https://httpbin.org/gzip should reply with a JSON body):

>>> import requests
>>> response = requests.get('https://httpbin.org/gzip', stream=True)
>>> response.raw.read()
b'\x1f\x8b\x08\x00\xf7\xa7\xa1^\x02\xff= ...

Min. upload speed limit

Is there a possibility to set minimal upload speed and return exception, when speed it too low

Provide on a per-request basis to set authentication

I am using this module along with pythongssapi/requests-gssapi, unfortunately the module is not threadsafe and I have to perform my requests with a ThreadPoolExecutor. Interleaving threads break requests. My basic code passes auth always to the session request now. Unfortunately, this module does not allow that. Or if you want to use alternative credentials this is not possible.

My request is to pass this through and if auth is None use the one from options/session else this one.

Errors during simultaneous connections

Situation:
Need to put single file into multiple accounts using webdav.

Implementation:
Connecting to accounts using threads (3-4). Each thread create client = Client(options)
and running small download to test connections. There are many clients so each
thread is doing same operation but for different client (limit on 3-4 threads not saturate network link.)

Problem:
CopyThread::__copy_file [Thread-2]!(['Test_Client01_2020-01.zip'])
CopyThread::__copy_file [Thread-1]!(['Test_Client02_2020-01.zip'])
CopyThread::__copy_file [Thread-3]!(['Test_Client05_2020-01.zip'])
CopyThread::__copy_file [Thread-1]!(['Test_Client03_2020-01.zip'])
CopyThread::__copy_file [Thread-2]!(['Test_Client04_2020-01.zip'])
except CopyThread::__process_package [Thread-2]! Remote resource: / not found
CopyThread::__copy_file [Thread-3]!(['Test_Client07_2020-01.zip'])
CopyThread::__copy_file [Thread-1]!(['Test_Client08_2020-01.zip'])
except CopyThread::__process_package [Thread-1]! Remote resource: / not found
.......

My quick solution:
Move {root, session, timeout verify} from static class Client variables into local object variables
so they in init function {self.root, self.session, self.timeout, self.verify}
This seems to solve my problem.

The "proxy_hostname" argument to Client is ignored

Thanks for everyone's work on this project, it's very helpful.

Issue: The "proxy_hostname" argument to Client is ignored.

Expected behaviour: Specifying the "proxy_hostname" when creating the client would cause connections to use the specified proxy.

Simple test case:

  1. Create a client with a proxy specified. The proxy should not exist. Attempt a connection to the WebDAV server. This connection should fail. It will succeed.
from webdav3.exceptions import *
from webdav3.client import Client

server = ""
login_name = ""
password = ""
root = "/"

_webdav_client = Client(options={
                                 "webdav_hostname": server,
                                 "webdav_login": login_name,
                                 "webdav_password": password,
                                 "proxy_hostname": "localhost:8080"
                                  })
# _webdav_client.session.proxies = 'localhost:8080'  <<-- This also doesn't work
_webdav_client.verify = False
root_dir = _webdav_client.list(str(root), get_info=True)
print(root_dir)

# There is a successful response - 
[{'created': None, 'name': None, 'size': '', 'modified': 'Sun, 24 Jan 2021 20:24:17 GMT', 'etag': '"600dd771d443e"', 'isdir': True, 'path': '/somestuff'
....}}

Rerun the test with the environment variables that the Requests library uses. This should fail because the proxy does not exist. It does fail.

import os
from webdav3.exceptions import *
from webdav3.client import Client

server = ""
login_name = ""
password = ""
root = "/"

os.environ["HTTP_PROXY"] = "localhost:8080"
os.environ["HTTPS_PROXY"] = "localhost:8080"

_webdav_client = Client(options={
                                 "webdav_hostname": server,
                                 "webdav_login": login_name,
                                 "webdav_password": password,
                                 "proxy_hostname": "localhost:8080"
                                  })
_webdav_client.verify = False
root_dir = _webdav_client.list(str(root), get_info=True)
print(root_dir)

# An exception result -
...   
raise NoConnection(self.webdav.hostname)
webdav3.exceptions.NoConnection: No connection with https://myserver

To test the success case start a proxy -

$ mitmproxy

Rerun the two tests above. You would expect to see traffic through the proxy in both cases but there won't be in the first test, there will be in the second.

Potential cause -

It looks to me like the Client class is ignoring the proxy options as is the WebDAVSettings class..

I also don't see a way for the requests session to include a proxy option,

if self.session.auth:
self.session.request(method="GET", url=self.webdav.hostname, verify=self.verify, timeout=self.timeout) # (Re)Authenticates against the proxy
response = self.session.request(
method=self.requests[action],
url=self.get_url(path),
auth=(self.webdav.login, self.webdav.password) if (not self.webdav.token and not self.session.auth)
and (self.webdav.login and self.webdav.password) else None,
headers=self.get_headers(action, headers_ext),
timeout=self.timeout,
cert=(self.webdav.cert_path, self.webdav.key_path) if (self.webdav.cert_path and self.webdav.key_path) else None,
data=data,
stream=True,
verify=self.verify
. I would expect to see something along the lines of

self.session.proxies.update(proxy_hostname)

Client.is_dir performance issue

When parent directory has too many files more than one thousand, Client.is_dir will be very slowly.
I check the source code, is_dir execute request with action info and parent path and depth 1, and response will return all files in parent directory.
I try execute request with remote path and depth 0, and it works.

push method fails in windows enviroments, due to trailing backslash

The push method does not remove the trailing '' from the end of windows directroy names returned by listdir before formatting the remote path. This results in an invalid character error being returned by the webdav server.

To reporduce:

  • On a windows machine, make a directory containing at least 1 other directory.
  • Use client.push with local_directory as your new top level directory.

This will return something like:

webdav3.exceptions.ResponseErrorCode: Request to https://your-webdav-domain.example/remote.php/dav/files/your-user/test_folder1/test_folder2%5C/ failed with code 400 and message: b'<?xml version="1.0" encoding="utf-8"?>\n<d:error xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns" xmlns:o="http://owncloud.org/ns">\n  <s:exception>OCA\\DAV\\Connector\\Sabre\\Exception\\InvalidPath</s:exception>\n  <s:message>File name contains at least one invalid character</s:message>\n  <o:retry xmlns:o="o:">false</o:retry>\n  <o:reason xmlns:o="o:">File name contains at least one invalid character</o:reason>\n</d:error>\n'

ValueError: <class 'webdav3.exceptions.RemoteResourceNotFound'>:

Few weeks ago I developed an app using the module and worked flawless, but from a few days ago, any time I try to download any .vcf file from contacts, it shows that error.
The weird thing is that I can list the files perfectly.
This is the client initialization method:
` def get_client(self):

    options = {
        'webdav_hostname': self.env.user.company_id.nc_url,
        'webdav_login': self.env.user.company_id.nc_user,
        'webdav_password': self.env.user.company_id.nc_passwd
    }
    client = Client(options)
    client.verify = False
    return client`

Consider using <d:getcontenttype> to calculate is_dir

Currently, Urn marks itself as a directory if and only if its href ends with a '/'. However, this may not apply to all webdav implementations. For example, the webdav provider I'm using currently (link, chinese website) returns <d:response><d:href> never ends with a /. Instead, it marks an item a directory using <d:response><d:propstat><d:prop><d:getcontenttype> with text httpd/unix-directory. See demo response below.

<d:response>
    <d:href>/dav/(censored)</d:href>
    <d:propstat>
        <d:prop>
            <d:getlastmodified>Mon, 24 Feb 2020 02:29:02 GMT</d:getlastmodified>
            <d:getcontentlength>0</d:getcontentlength>
            <d:getetag />
            <d:owner>[email protected]</d:owner>
            <d:current-user-privilege-set>
                <d:privilege>
                    <d:read />
                </d:privilege>
                <d:privilege>
                    <d:write />
                </d:privilege>
                <d:privilege>
                    <d:all />
                </d:privilege>
                <d:privilege>
                    <d:read_acl />
                </d:privilege>
                <d:privilege>
                    <d:write_acl />
                </d:privilege>
            </d:current-user-privilege-set>
            <d:getcontenttype>httpd/unix-directory</d:getcontenttype>
            <d:displayname>(censored)</d:displayname>
            <d:resourcetype>
                <d:collection />
            </d:resourcetype>
        </d:prop>
        <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
</d:response>

I'm new to webdav protocol and not familier with its standard implementation. If it is a wrong implementation from the webdav provider, please let me known and I'll contact the site's custom. Many thanks.

Failed to dowload files due to KeyError about Content-Length

KeyError will be raised if missing content-length http header in the response from the remote server. That happens when communicating to nextcloud servers (and/or maybe other servers) for some cases on downloading files.
Here is a screenshot about the issue
keyerror

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.