Comments (4)
Some new test for versoin 0.8.1
with in-memory storage:
CPython 3.6
STOR (client -> server) 366.06 MB/sec
RETR (server -> client) 516.45 MB/sec
200 concurrent clients (connect, login) 0.12 secs
STOR (1 file with 200 idle clients) 406.52 MB/sec
RETR (1 file with 200 idle clients) 518.47 MB/sec
200 concurrent clients (RETR 10.0M file) 10.41 secs
200 concurrent clients (STOR 10.0M file) 13.45 secs
200 concurrent clients (QUIT) 0.03 secs
CPython 3.6 + uvloop
STOR (client -> server) 418.34 MB/sec
RETR (server -> client) 626.34 MB/sec
200 concurrent clients (connect, login) 0.09 secs
STOR (1 file with 200 idle clients) 418.53 MB/sec
RETR (1 file with 200 idle clients) 627.51 MB/sec
200 concurrent clients (RETR 10.0M file) 2.08 secs
200 concurrent clients (STOR 10.0M file) 5.41 secs
200 concurrent clients (QUIT) 0.03 secs
PyPy 3.5-5.9.0
STOR (client -> server) 262.79 MB/sec
RETR (server -> client) 863.06 MB/sec
200 concurrent clients (connect, login) 0.14 secs
STOR (1 file with 200 idle clients) 268.81 MB/sec
RETR (1 file with 200 idle clients) 890.49 MB/sec
200 concurrent clients (RETR 10.0M file) 1.56 secs
200 concurrent clients (STOR 10.0M file) 5.93 secs
200 concurrent clients (QUIT) 0.04 secs
from aioftp.
@iselind
First of all, pyftpdlib
is asynchronous, but it can't be used with asyncio
right now (but they have uncompleted PR). pyftpdlib
also use many things to speed up work. aioftp
was started as "missing asyncio ftp client" library and more later server side was implemented. It use many high-level asyncio abstractions, so bottle neck is somewhere there.
I will try to explain benchmark results.
STOR/RETR: aioftp
bottleneck here is disk io. Definitely, pyftpdlib
use memory cache and/or some optimizations, since you can't write to disk faster, than it limit. So, there is some "cheating", pyftpdlib
said, that everything is done, your file is on disk, but actually they are storing foreground.
LOGIN/QUIT: only high-level streams overhead here. Timings are pretty similar.
Here is benchmark result for in-memory storage:
STOR (client -> server) 433.46 MB/sec
RETR (server -> client) 493.69 MB/sec
200 concurrent clients (connect, login) 0.17 secs
STOR (1 file with 200 idle clients) 430.04 MB/sec
RETR (1 file with 200 idle clients) 517.03 MB/sec
200 concurrent clients (RETR 10.0M file) 10.74 secs
200 concurrent clients (STOR 10.0M file) 13.60 secs
200 concurrent clients (QUIT) 0.04 secs
This numbers shows aioftp
python bottleneck, price for using high-level asyncio api and many abstraction layers.
from aioftp.
Thanks a lot for your quick reply.
Because pyftpdlib cheat by using a memory cache, then the data isn't secure on the drive when the call completes. If the computer suffer a power outage here i'll lose data, sounds like a bad implementation. aioftp doesn't have this flaw i guess?
I'm looking for a ftp client, preferably usable in a asyncio context. I guess pyftpdlib isn't such a great choice as it seems to be a server only. Are there any other FTP clients than aioftp that you know of? Google only show aioftp when i search 'python asyncio ftp client'.
from aioftp.
aioftp doesn't have this flaw i guess?
Yes.
pyftpdlib cheat
This is "cheat" for benchmark. For users caching is very good thing, since you can get your data faster (from cache).
Are there any other FTP clients than aioftp that you know of?
Unfortunately, I don't know any other asyncio ftp client. That was the main reason to make aioftp
library.
from aioftp.
Related Issues (20)
- advanced-download-and-upload-abort-restart Question HOT 8
- test_pasv_connection_pasv_forced_response_address fails on FreeBSD HOT 5
- Please tag release revisions in git HOT 6
- Asynchronous permission check HOT 2
- pytest-asyncio mode strict lets tests fail
- Client.download tries to parse my file's contents HOT 7
- Multiplex download HOT 4
- Im not sure but downloads are bugging HOT 2
- Pytest Asyncio 0.17.2: Deprecation Warning HOT 2
- aioftp.errors.PathIsNotAbsolute: E:\PythonPrj\6xx\serverFile\recvFiles\客户端30 HOT 2
- get_passive_connection aborts immediately if epsv server response is 550 Permission denied HOT 2
- Deadlock upon calling `await client.abort()` HOT 5
- Add type hints to aioftp HOT 29
- time format err HOT 1
- cannot download file from filezilla server HOT 3
- [0.21.4] client.list returns Value Error because date format does not match on Windows HOT 1
- Let's put `aioftp` under the `aio-libs` org on PyPI HOT 6
- Support python version older than 3.11 HOT 1
- `[Errno 111] Connect call failed` with simple stat code HOT 2
- [feature-request] Client HTTP(S) proxy support (CONNECT)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from aioftp.