GithubHelp home page GithubHelp logo

veeso / suppaftp Goto Github PK

View Code? Open in Web Editor NEW
97.0 6.0 25.0 985 KB

a super FTP/FTPS client library for Rust with support for both passive and active mode

License: Apache License 2.0

Rust 99.76% Shell 0.24%
ftp ftp-client ftps async asynchronous rust rust-lang rust-library rust-crate rust-ftp

suppaftp's Introduction

SuppaFTP

logo

~ A super FTP/FTPS client library for Rust ~

Documentation ยท Crates.io

Developed by veeso and Matt McCoy

Current version: 5.3.0 (06/01/2024)

License-Apache-2.0/MIT Repo stars Downloads counter Latest version Ko-fi conventional-commits

Lib-CI Cli-bin-ci Coveralls Docs



Introduction ๐Ÿ‘‹

SuppaFTP is the main FTP/FTPS client library for Rust, with both support for sync/async programming and for all the FTP protocol features. It is a fork of the original ftp library "rust-ftp", but since the original library is currently unmaintained, I decided to keep working on this library by myself. Currently, I consider myself as the only maintainer of this project, indeed I've already added some features to the library and improved it with better error handling and test units.

Main differences between SuppaFTP and rust-ftp ๐Ÿค”

  • Replaced OpenSSL with native-tls or rustls as you prefer ๐Ÿ”’
  • Added methods to work with streams (e.g. put_with_stream) โฌ‡๏ธ
  • suppaftp supports Active mode
  • Added get_welcome_msg method ๐Ÿ‘‹
  • Supports for both sync/async rust ๐Ÿ•™
  • Supports for more commands ๐ŸŒŸ
    • ABOR
    • APPE
    • REST
    • EPSV
    • EPRT
  • Some extra features, such as the LIST command output parser
  • Implementation of RFC 2428
  • Implementationb of RFC 2389
  • Removed deprecated statements โšฐ๏ธ
  • Better error handling ๐Ÿ›
  • Added test units keeping an eye on code coverage ๐Ÿ‘€

Get started ๐Ÿ

To get started, first add suppaftp to your dependencies:

suppaftp = "^5.3.0"

Features

SSL/TLS Support

If you want to enable support for FTPS, you must enable the native-tls or rustls feature in your cargo dependencies, based on the TLS provider you prefer.

suppaftp = { version = "^5.3.0", features = ["native-tls"] }
# or
suppaftp = { version = "^5.3.0", features = ["rustls"] }

๐Ÿ’ก If you don't know what to choose, native-tls should be preferred for compatibility reasons. โ— If you want to link libssl statically, enable feature native-tls-vendored

Async support

If you want to enable async support, you must enable async feature in your cargo dependencies.

suppaftp = { version = "^5.2.0", features = ["async"] }

โš ๏ธ If you want to enable both native-tls and async you must use the async-native-tls feature โš ๏ธ โš ๏ธ If you want to enable both rustls and async you must use the async-rustls feature โš ๏ธ โ— If you want to link libssl statically, enable feature async-native-tls-vendored

Deprecated methods

If you want to enable deprecated methods of FTPS, please enable the deprecated feature in your cargo dependencies.

This feature enables these methods:

  • connect_secure_implicit(): used to connect via implicit FTPS

Logging

By default, the library will log if there is any log crate consumer on the user implementation. Logging can be if preferred, disabled via the no-log feature.

Examples ๐Ÿ“š

use std::str;
use std::io::Cursor;
use suppaftp::FtpStream;

fn main() {
    // Create a connection to an FTP server and authenticate to it.
    let mut ftp_stream = FtpStream::connect("127.0.0.1:21").unwrap();
    let _ = ftp_stream.login("username", "password").unwrap();

    // Get the current directory that the client will be reading from and writing to.
    println!("Current directory: {}", ftp_stream.pwd().unwrap());

    // Change into a new directory, relative to the one we are currently in.
    let _ = ftp_stream.cwd("test_data").unwrap();

    // Retrieve (GET) a file from the FTP server in the current working directory.
    let data = ftp_stream.retr_as_buffer("ftpext-charter.txt").unwrap();
    println!("Read file with contents\n{}\n", str::from_utf8(&data.into_inner()).unwrap());

    // Store (PUT) a file from the client to the current working directory of the server.
    let mut reader = Cursor::new("Hello from the Rust \"ftp\" crate!".as_bytes());
    let _ = ftp_stream.put_file("greeting.txt", &mut reader);
    println!("Successfully wrote greeting.txt");

    // Terminate the connection to the server.
    let _ = ftp_stream.quit();
}

Ftp with TLS (native-tls)

use suppaftp::{NativeTlsFtpStream, NativeTlsConnector};
use suppaftp::native_tls::{TlsConnector, TlsStream};

fn main() {
    let ftp_stream = NativeTlsFtpStream::connect("test.rebex.net:21").unwrap();
    // Switch to the secure mode
    let mut ftp_stream = ftp_stream.into_secure(NativeTlsConnector::from(TlsConnector::new().unwrap()), "test.rebex.net").unwrap();
    ftp_stream.login("demo", "password").unwrap();
    // Do other secret stuff
    assert!(ftp_stream.quit().is_ok());
}

Ftp with TLS (rustls)

use std::str;
use std::io::Cursor;
use std::sync::Arc;
use suppaftp::{RustlsFtpStream, RustlsConnector};
use suppaftp::rustls::ClientConfig;

fn main() {
    let mut root_store = rustls::RootCertStore::empty();
    root_store.add_server_trust_anchors(webpki_roots::TLS_SERVER_ROOTS.0.iter().map(|ta| {
        rustls::OwnedTrustAnchor::from_subject_spki_name_constraints(
            ta.subject,
            ta.spki,
            ta.name_constraints,
        )
    }));
    let config = ClientConfig::builder()
        .with_safe_defaults()
        .with_root_certificates(root_store)
        .with_no_client_auth();
    // Create a connection to an FTP server and authenticate to it.
    let config = Arc::new(rustls_config());
    let mut ftp_stream = RustlsFtpStream::connect("test.rebex.net:21")
        .unwrap()
        .into_secure(RustlsConnector::from(Arc::clone(&config)), "test.rebex.net")
        .unwrap();
    // Terminate the connection to the server.
    let _ = ftp_stream.quit();
}

Going Async

use suppaftp::{AsyncNativeTlsFtpStream, AsyncNativeTlsConnector};
use suppaftp::async_native_tls::{TlsConnector, TlsStream};
let ftp_stream = AsyncNativeFtpStream::connect("test.rebex.net:21").await.unwrap();
// Switch to the secure mode
let mut ftp_stream = ftp_stream.into_secure(AsyncNativeTlsConnector::from(TlsConnector::new()), "test.rebex.net").await.unwrap();
ftp_stream.login("demo", "password").await.unwrap();
// Do other secret stuff
assert!(ftp_stream.quit().await.is_ok());

Built-in CLI client ๐Ÿ–ฅ๏ธ

SuppaFTP comes also with a built-in command-line FTP client. This CLI application provides all the commands to interact with a remote FTP server and supports FTPS too. You can also use it as a reference to implement your project. You can find it in the cli/ directory.

You can simply install as any other rust application via Cargo:

cargo install suppaftp-cli
suppaftp --version

Support the developer โ˜•

If you like SuppaFTP, please consider a little donation ๐Ÿฅณ

ko-fi PayPal


Changelog โŒ›

View Changelog here


License ๐Ÿ“œ

Licensed under either of

at your option.


Contribution ๐Ÿค

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

If you want to contribute to this project, please read the Contributing guide first ๐Ÿ™‚.

suppaftp's People

Contributors

17dec avatar benaryorg avatar bombless avatar brunoarueira avatar clonejo avatar daa84 avatar dbrgn avatar devbydav avatar emberian avatar fitztrev avatar imgbotapp avatar johnscience avatar little-dude avatar luxed avatar matt2xu avatar mattnenterprise avatar mss1451 avatar newfla avatar rrevenantt avatar rye avatar senden9 avatar veeso avatar workanator avatar xuanwo avatar yackushevas avatar yakov-bakhmatov avatar zsck avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

suppaftp's Issues

[Feature Request] - Set a timeout on FTP connect

Description

In my application I want to set a custom timeout on the FTP connection (connect function).

Changes

  • An additional associated function on FtpStream that allows a custom timeout to be set.

Implementation

There is an associated function on TcpStream called connect_timeout that would provide the functionality needed to implement this feature.

[BUG] - abort can be called without passing ownership to data_stream

Description

I had trouble with using ImplAsyncFtpStream::abort() correctly. Apparently it is possible to only pass a mutable reference for data_stream instead of full ownership, so the drop does not work.

Steps to reproduce

let _ = env_logger::builder().is_test(true).try_init();
use suppaftp::{AsyncFtpStream, Mode};
let mut ftp_stream = AsyncFtpStream::connect("127.0.0.1:1234").await?;
ftp_stream.login("anonymous", "anonymous").await?;
ftp_stream.set_mode(Mode::ExtendedPassive);
let mut stream = ftp_stream.retr_as_stream("foo").await?;
// only pass &mut:
ftp_stream.abort(&mut stream).await?;
// this code will never actually be reached:
loop {
    dbg!();
}

Expected behaviour

abort() does not block indefinitely, or rather, it should be impossible to pass a reference here.

Environment

  • OS: Arch
  • Architecture: x86_64
  • Rust version: 1.76.0-beta.1
  • library version: v5.2.2
  • Protocol used: FTP
  • Remote server version and name: vsftpd 3.0.3

Additional information

Last lines from trace log:

[2024-01-03T00:50:29Z DEBUG suppaftp::async_ftp] Aborting active file transfer
[2024-01-03T00:50:29Z TRACE suppaftp::async_ftp] CC OUT: ABOR
[2024-01-03T00:50:29Z TRACE suppaftp::async_ftp] dropped stream
[2024-01-03T00:50:29Z TRACE polling::epoll] modify: epoll_fd=10, fd=9, ev=Event { key: 0, readable: true, writable: false }

[Feature Request] - Support server timezones other than UTC

Description

SuppaFtp::FtpStream::mdtm assumes the server timezone is UTC. I used this hack to change it to a different time zone afterwards:

fn fix_timezone(datetime_wrong_tz: &DateTime<Utc>, actual_tz: Tz) -> chrono::LocalResult<DateTime<Tz>> {
    datetime_wrong_tz.naive_utc().and_local_timezone(actual_tz)
}

But I think there should be a way to either directly return a chrono::NativeDateTime, a second method that does so, or a way to provide server time zone.

[BUG] - list parser: SyntaxError on name that starts with 2 numbers

Description

A file called 01 1234 foo.mp3 breaks the file listing parser.

Reducing the file name to one number makes the example below work.

Steps to reproduce

let file: File = dbg!(File::from_str(
    "-r--r--r--    1 23        23         1234567 Jan 1  2000 01 1234 foo.mp3",
))
.ok()
.unwrap();
assert_eq!(file.name, "01 1234 foo.mp3");

File::from_str() returns SyntaxError.

Environment

  • OS: Arch
  • Architecture x86_64
  • Rust version rustc 1.76.0-beta.1 (0e09125c6 2023-12-21)
  • library version: v5.3.0 a0537a2
  • Protocol used: FTP
  • Remote server version and name: vsftpd

[BUG] - Active mode commands timeout

Description

ftp.ncbi.nih.gov uses active mode. list command after login times out.

Steps to reproduce

use suppaftp::FtpStream;

fn main() {
    let parent_url = "ftp.ncbi.nlm.nih.gov:21";
    let ftp_stream = FtpStream::connect(parent_url).unwrap();
    let mut ftp_stream = ftp_stream.active_mode(std::time::Duration::new(60, 0));
    ftp_stream.login("anonymous", "").unwrap();
    ftp_stream.cwd("/snp/latest_release/VCF").unwrap();
    let filenames_r = ftp_stream.list(None);
    println!("@ filenames r={:?}", filenames_r);
}

Expected behaviour

filenames_r being a Vec of filenames

Environment

  • OS: MacOS on M3 Max
  • Architecture: Apple Silicon
  • Rust version: 1.75.0
  • library version: 5.3.0
  • Protocol used: No TLS
  • Remote server version and name: N/A

Additional information

N/A

[Feature Request] Allow interchangeable use of stream types by exposing ImplFtpStream

Description

I'd like to do something like

pub fn connect(addr: &str, user: &str, password: &str, secure: bool) -> FtpResult<ImplFtpStream> {
    let mut ftp_stream = match secure {
        false => init_plain_stream(addr)?,
        true => init_secure_stream(addr)?,
    };
    ftp_stream.login(user, password)?;
    Ok(ftp_stream)
}

where init_plain_stream() and init_secure_stream() respectively return FtpStream and NativeTlsFtpStream using the connect()/into_secure() calls outlined in the documentation.

Basically my desire is to establish the connection once - going through the appropriate steps to secure it, or not - and then use it as a generic stream afterwards. In terms of function signatures I feel like I should be able to achieve this by using the ImplFtpStream struct that's common across connection types. But this is currently set to private.

Is what I'm thinking likely to work? And if so, would it be possible to make the generic ImplFtpStream struct public?

By the way, if there are other ways to do this I'd be quite happy to reconsider my approach! I'm fairly new to Rust. (Also I'm aware that the match statement in my example code won't work as is anyway since the types of the arms need to match, but that's solvable).

Changes

Make ImplFtpStream public?

[QUESTION] - Potential timeout? Advice on correct usage.

Hello, thanks for the great crate!

I am hoping you can help me with a strange issue some of my users are experiencing. I am essentially just looping through a list of files to download. Some users have reported that the download will begin, but eventually will stall out and not continue. In the attached logs, you can see the downloads start, but the user reported waiting for about 15 minutes with no response (the files are not large, and it is all on a LAN, so should not be network issue). I am doing so in batches of 10 via tokio::sync::Semaphore, but even when I was just looping through one by one, this issue seemed to pop up here and there.

Am I doing anything wrong in my code? At one point I was using the standard ftp_stream.retr_as_buffer method, but I switched to ftp_stream.retr_as_stream to see if that would help, but it does not seem to have made a difference. As you can tell from the logs, I never make it out of ap_ftp_copy by Ok or Err.

Any advice is welcome!

Code

//////////////////////////////////////////////////////////////////
        info!("Copying POS from AP...");
        progress_bar.set_message(format!("{} (Copying POS from AP...)", date_string.clone()));
        let pos_ap_path = Path::new(&parent_path).join("pos_ap");

        let permits = Arc::new(Semaphore::new(10));
        let futures = date.pos.ap.clone().into_iter().map(|pos| {
            let permits = Arc::clone(&permits);
            let pos_ap_path_clone = pos_ap_path.clone();
            let pos_clone: String = pos.clone();
            async move {
                let permit = match permits.acquire().await {
                    Ok(permit) => permit,
                    Err(e) => {
                        return Err(CommandError {
                            message: format!(
                                "Could not acquire permit to copy {}",
                                pos_clone.clone()
                            ),
                            cause: e.to_string(),
                        })
                    }
                };
                info!("[pos_ap] Copying {}", pos_clone);
                match ap_ftp_copy(pos_clone.as_str(), &pos_ap_path_clone).await {
                    Ok(message) => {                
                        info!("[pos_ap] Successfully copied {}", pos_clone);
                        drop(permit);
                        progress_bar.inc(1);
                        Ok(message)
                    }
                    Err(e) => return Err(e),
                }
            }
        });

        let results = future::join_all(futures).await;
        for result in results {
            match result {
                Ok(message) => debug!("{}", message),
                Err(e) => return Err(e),
            }
        }
//////////////////////////////////////////////////////////////////

pub async fn ap_ftp_copy(file: &str, dest: &PathBuf) -> Result<String, CommandError> {
    if async_std::fs::metadata(&dest).await.is_err() {
        debug!(" mkdir: {:?}", dest);
        match async_std::fs::create_dir_all(&dest).await {
            Ok(()) => (),
            Err(e) => {
                return Err(CommandError {
                    message: format!("Problem creating dirs for {:?}", dest),
                    cause: e.to_string(),
                })
            }
        };
    }

    let mut new_file = match async_std::fs::OpenOptions::new()
        .write(true)
        .create(true)
        .truncate(true)
        .open(dest.join(file))
        .await
    {
        Ok(new_file) => new_file,
        Err(e) => {
            return Err(CommandError {
                message: format!("Problem creating file for {}", file),
                cause: e.to_string(),
            })
        }
    };

    let mut ftp_stream = match ap_ftp::connect_async().await {
        Ok(ftp_stream_async) => ftp_stream_async,
        Err(e) => return Err(e),
    };

    let mut file_stream = match ftp_stream.retr_as_stream(file).await {
        Ok(file_stream) => file_stream,
        Err(e) => {
            return Err(CommandError {
                message: format!("Problem creating stream for file {}", file),
                cause: e.to_string(),
            })
        }
    };

    let copy_response = match async_std::io::copy(&mut file_stream, &mut new_file).await {
        Ok(_bytes) => {
            format!("Successfully copied file: {}", file)
        }
        Err(e) => {
            return Err(CommandError {
                message: format!("Problem writing file {} from stream", file),
                cause: e.to_string(),
            })
        }
    };

    match ftp_stream.finalize_retr_stream(file_stream).await {
        Ok(()) => (),
        Err(e) => {
            return Err(CommandError {
                message: format!("Problem finalizing stream for file {}", file),
                cause: e.to_string(),
            })
        }
    }
        
    match ftp_stream.quit().await {
        Ok(()) => Ok(copy_response),
        Err(e) => {
            warn!("Could not close FTP session in discover: {}", e.to_string());
            return Ok(copy_response);
        }
    }
}

Log

ap_ftp.log

[Feature Request] - Allow using rustls instead of native-tls

Description

First of all: thank you continuing the rust-ftp crate! I looked for useful FTP crates for my package manager, that are also still maintained, and this was the only one that fits.
My small issue is that I want to use rustls (with the system CAs) when it comes to TLS. Reqwest allows me to do that, but it's obviously for HTTP and not FTP.
I'd appreciate if this gets implemented, though I don't really know how much effort it is to do so ^^'

Changes

The following changes to the application are expected

  • Add a new feature, e.g. rustls (and async-rustls), which allows using rustls instead of native-tls.

[BUG] - Async TLS example fails to compile

Description

The async example in README fails to compile, specifically the line:

let mut ftp_stream = ftp_stream.into_secure(AsyncNativeTlsConnector::from(TlsConnector::new()), "test.rebex.net").await.unwrap();

Error message:

error[E0271]: type mismatch resolving `<AsyncNativeTlsConnector as AsyncTlsConnector>::Stream == AsyncNoTlsStream`
   --> test/src/main.rs:8:49
    |
8   | ...am.into_secure(AsyncNativeTlsConnector::from(TlsConnector::new()), "...
    |       ----------- ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected `AsyncNoTlsStream`, found `AsyncNativeTlsStream`
    |       |
    |       required by a bound introduced by this call
    |
note: required by a bound in `ImplAsyncFtpStream::<T>::into_secure`
   --> /home/michaelyang/.cargo/registry/src/index.crates.io-6f17d22bba15001f/suppaftp-5.1.2/src/async_ftp/mod.rs:122:47
    |
122 | ... AsyncTlsConnector<Stream = T> + 'static,
    |                       ^^^^^^^^^^ required by this bound in `ImplAsyncFtpStream::<T>::into_secure`

Investigation

I did some digging around in the source file, and it appears that the problem stems from the fact that ImplAsyncFtpStream::into_secure requires the bound on T for the parameter tls_connector:

tls_connector: impl AsyncTlsConnector<Stream = T> + 'static

which suggests that the stream before and after switching to secure mode must be the same, and hence why AsyncFtpStream with T = AsyncNoTlsStream failed to be promoted to a secure ftp stream with T = AsyncNativeTlsStream.

On the other hand, the into_secure documentation provides the correct example:

let mut ctx = TlsConnector::new();
let mut ftp_stream = ImplAsyncFtpStream::connect("127.0.0.1:21").await.unwrap();
let mut ftp_stream = ftp_stream.into_secure(ctx, "localhost").await.unwrap();

Where the original ftp_stream has an indeterminate generic type T that is subsequently inferred in the following line.

Hence, which of the two (either promotion from insecure stream or direct construction from ImplAsyncFtpStream) examples is the desired behaviour?

Environment

  • OS: Linux 6.2.6
  • Architecture x86_64
  • Rust version 1.70.0
  • library version 5.1.2

Damaged uploaded image

Description

I tried to upload the image, but I found that the uploaded image was damaged, what did I do wrong?
A clear and concise description of what the bug is.

Steps to reproduce

Steps to reproduce the bug you encountered
ๅ›พ็‰‡

Expected behaviour

A clear and concise description of what you expected to happen.

Environment

  • OS: [Windows 11]
  • Architecture [x86_64]
  • Rust 1.67.1
  • library 5.0.0
  • Protocol used
  • Remote server version and name: Ubuntu 22.04.2 LTS x86_64

Additional information

There was no problem uploading the text, but the uploaded image was corrupted.

Problem setting up TLS -> bad server or bad client?

I'm connecting to an FTP server on an industrial device. In the course of setting up TLS, the following exchange takes place:

[DEBUG suppaftp::sync_ftp] Connecting to server
[DEBUG suppaftp::sync_ftp] Established connection with server
[DEBUG suppaftp::sync_ftp] Reading server response...
[TRACE suppaftp::sync_ftp] CC IN: [50, 50, 48, 32, 86, 120, 87, 111, 114, 107, 115, 32, 70, 84, 80, 32, 115, 101, 114, 118, 101, 114, 32, 40, 86, 120, 87, 111, 114, 107, 115, 32, 86, 120, 87, 111, 114, 107, 115, 53, 46, 53, 46, 49, 41, 32, 114, 101, 97, 100, 121, 46, 13, 10]
[TRACE suppaftp::sync_ftp] Code parsed from response: service ready for new user (220)
[DEBUG suppaftp::sync_ftp] Server READY; response: Some("220 VxWorks FTP server (VxWorks VxWorks5.5.1) ready.")
[DEBUG suppaftp::sync_ftp] Initializing TLS auth
[TRACE suppaftp::sync_ftp] CC OUT: AUTH TLS
[TRACE suppaftp::sync_ftp] CC IN: [50, 48, 48, 32, 65, 85, 84, 72, 32, 99, 111, 109, 109, 97, 110, 100, 32, 111, 107, 59, 32, 115, 116, 97, 114, 116, 105, 110, 103, 32, 83, 83, 76, 32, 99, 111, 110, 110, 101, 99, 116, 105, 111, 110, 46, 46, 46, 13, 10]
[TRACE suppaftp::sync_ftp] Code parsed from response: command okay (200)

Unfortunately at that point the suppaftp client raises UnexpectedResponse, since it expects Status::AuthOk (234) rather than Status::CommandOk (200) - based on https://github.com/veeso/suppaftp/blob/main/suppaftp/src/sync_ftp/mod.rs#L141.

Not quite sure what to make of that. It's entirely possible that the server is just not spec-compliant. But it would be good to find a way to establish the connection nonetheless. FWIW, wget connects fine and doesnโ€™t seem to complain. Any thoughts? Maybe Iโ€™m just connecting improperly in some way?

[BUG] - Fails to RETR large files

Description

Using the CLI (or the lib), I get corrupted files when trying to retrieve them (using RETR).

Steps to reproduce

I tried to get the same file using the suppaftp-cli and the GNU ftp client.
The GNU ftp client correctly retrieves the file as it is the same file size as the original one. I can run the command gunzip -d ftp.csv.gz without an issue.
When I try to do the same with suppaftp, gunzip tells me that the file is corrupted.

โฏ ls -l ftp.csv.gz suppa.csv.gz 
-rw-r--r-- 1 matthew_deville matthew_deville 11509311 Nov 10 11:44 ftp.csv.gz
-rw-r--r-- 1 matthew_deville matthew_deville 11552314 Nov 10 11:34 suppa.csv.gz

Expected behaviour

The file retrieved should be identical to the original one.

Environment

  • OS: Manjaro Linux (Arch linux)
  • Architecture: x86_64
  • Rust version: rustc 1.65.0 (897e37553 2022-11-02)
  • library version: 4.5.2

[BUG] - The file.log upload to ftp server. then lost \n line breaks

The Upload code of Core

fn test() {
  let mut buffer = Vec::new();
  let mut f = fs::File::open(src).unwrap();
  let mut reader = f
    .read_to_end(&mut buffer)
    .map(|_| Cursor::new(buffer))
    .map_err(FtpError::ConnectionError)?;
  match upload_type {
    StorageUploadType::Append => {
      ftp.append_file(&fname, &mut reader)?;
    }
    StorageUploadType::Replace => {
      ftp.put_file(&fname, &mut reader)?;
    }
    StorageUploadType::Once => {
      if ftp.size(&fname).is_err() {
        ftp.put_file(&fname, &mut reader)?;
      } else {
        /* ๅทฒ็ปๅญ˜ๅœจ */
      }
    }
  };
}

[Feature Request] - support of FTP servers running behind a NAT/proxy

When an FTP server is running behind a NAT, it returns its private network IP address in response to the PASV command.

In this case, received address should be replaced with the address of the remote peer of the existing connection.

This is implemented, for example, in the Apache Commons Net library: https://github.com/apache/commons-net/blob/b1f6d1af65c67804524a616ce35d8b31f6e750df/src/main/java/org/apache/commons/net/ftp/FTPClient.java#L376

See my patch:

https://github.com/yakov-bakhmatov/suppaftp/tree/pasv-nat

[Feature Request] - Implement EPSV (Extended Passive Mode)

Description

I'm trying to upload a file to an FTP server over a TLS-encrypted IPv6 connection. The upload fails with the following error message:

Error: Invalid response: [425] 425 You cannot use PASV on IPv6 connections. Use EPSV instead.

The server is powered by Pure-FTPd.

PASV works for IPv4 only, therefore it's not possible to use passive mode over IPv6 at the moment.

More info: https://www.jscape.com/blog/what-is-the-ftp/s-epsv-command-and-when-do-you-use-it

Changes

Suppaftp should use EPSV instead of PASV if required or requested by the server.

Implementation

EPSV is defined in RFC2428: https://www.rfc-editor.org/rfc/rfc2428#section-3

[Feature Request] - Support MLST and MLSD

Description

The MLST and MLSD commands allow getting more detailed metadata about a file.

Changes

Add something like this:

fn mlsd(pathname: Option<&str>) -> Vec<FileInfo>{

}

fn mlst(pathname: &str) -> FileInfo {

}

struct FileInfo {
    pathname: String,
    size: Option<usize>,
    created: Option<DateTime>,
    modified: Option<DateTime>,
    type: Option<TypeFact>,
    unique: Option<String>,
    perm: Option<String>,
    lang: Option<String>,
    media_type: Option<String>,
    charset: Option<String>,
    custom: HashMap<String, String>
}

enum TypeFact {
    File,
    Directory,
    CurrentDirectory,
    ParentDirectory
}

Implementation

Should be fairly straightforward

[QUESTION] - Reading files when using AsyncFtpStream

Hi,

I'm using AsyncFtpStream - I've been able to transfer the write, metadata, list methods etc alright, but ftp_stream.retr is causing issues. I'm not sure if I'm missing something simple or if it's an actual issue.

Since retr_as_buffer doesn't exist for the Async FTP Stream it'd be good to have an example in the docs for retr when using the async feature

Thanks

[Feature Request] - Build the docs with doc_cfg

Description

Some of the features of suppaftp are behind #[cfg]s , yet they are not displayed in the official documentation because the project doesn't utilize #[doc_cfg]s.

Changes

Display conditionally compiled items on docs.rs.

Implementation

You can keep the library stable while allowing the documentation to be built using the combination of --cfg, #[package.metadata], #[cfg_attr(...)], and #[cfg_doc(...)]. syn is a good example.

[Feature Request] - Timeouts for active data connection acceptance

Description

When active data connections are configured to be used, currently FtpStream waits for an incoming connection indefinitely, while it may never happen, leading to a program hanging. Would be nice if it was possible to set a timeout.

Changes

The following changes to the application are expected

  • A data connection acceptance timeout setting and a handling of that setting.
  • Perhaps a similar one for passive connections: currently there is the connect_timeout function for control channel connection, but data channel in passive mode always uses plain connect.

Implementation

Once a setting is introduced, select or something along those lines can be called with a timeout, only calling accept when there is something to accept. For passive mode, TcpStream::connect_timeout can be used, as for the control channel.

[Feature Request] - Implementation of SITE command

Hey there,

I'm currently working on a project that involves interacting through FTP with a server that emulates a mainframe. I've been looking to use SuppaFTP, but unfortunately I need to use the SITE command to accomplish my desired goal. I do not believe that SuppaFTP currently supports this command.

Is the inclusion of the SITE command by chance something that would be considered worthwhile for SuppaFTP?

Description

The SITE FTP command allows a user to send commands specific to the server that the user is interacting with. My understanding is that it is considered to be a standard part of FTP.

Changes

While I am not yet especially familiar with SuppaFTP, looking through the documentation leads me to believe that the following would be added:

  • The struct ImplFtpStream would have a new function called site. This function would likely take in a string representing a server-specific command as an argument. It would use that string to attempt to execute the SITE command on the server, i.e. SITE {COMMAND}.
  • A similar function would be added to the struct ImplAsyncFtpStream.

[SECURITY] - SSL Stream is not properly closed

Description

Severity:

  • critical
  • high
  • medium
  • low

When using SSL data stream, the channel is not properly closed as specified in RFC2246 7.2.1 https://www.ietf.org/rfc/rfc2246.txt.
This may lead to a truncation attack.

Additional information

It may be enough to implement the stream shut down as already done by @devbydav.

if let DataStream::Ssl(ssl_stream) = stream.get_mut() {
            ssl_stream
                .shutdown()
                .map_err(FtpError::ConnectionError)?;
}

[Feature Request] - Better composability

Description

As I was working on a commercial project using suppaftp, I faced various challenges:

  • lack of support for Drop on ftp- and data- streams,
  • lack of Send, which is a WIP`,
  • lack of integration with serde,
  • lack of support for implementing retry logic in case of connectivity issues,
  • private visibility of important traits, notably TlsStream,
  • rigidity of the APIs, notably ImplFtpStream::list returns a FtpResult<Vec<String>> instead of an iterator over results,
  • lack of an iterator over std::list::File on a remote server,
  • lack of RemotePathBuf and RemotePath (though it goes beyond the scope of your crate),
  • lack of a canonical method for file synchronization (think of comparing the state of the remote server against an "index" and applying changes to the local replica of the file hierarchy).

Thanks to your crate, I was spared from a lot of work, so I thank you for that. However, the library right now is fairly low level and does not offer many conveniences that one could expect.

For now, my biggest annoyance is the ImplFtpStream::list method, which probably shouldn't have existed in this form.

When implementing an iterator over remote entries, I have to deal with a Vec<Vec<Result<MyCustomDirEntryType, MyCustomWrapperAroundParseError>>.

And due to the nature of the iterator, I ended up facing the limitations of the NLL borrow checker (danielhenrymantilla/polonius-the-crab.rs#11) and I had to deal with the absence of entry API on vector until the pain became so severe that I ended up implementing entry API for Vec as a stack (link).

Without your library, I would have to write FTP client code myself, so again, thank you.

Changes

The most important changes are the following:

  • Add a method on FTP streams that returns an iterator over results to allow users to choose the container.
  • Add a feature that would enable serialization and deserialization of suppaftp::list::File, suppaftp::list::ParseError, and suppaftp::types::FtpError.

[Feature Request] - Expose DataStream to users

Description

retr_as_stream returns DataStream<T> but we can't write this type out.

Changes

The following changes to the application are expected

  • Make DataStream<T> public, so that we can return FtpReader<DataStream<XxxStream>> to users.

[BUG] - put_file does not work with Async SSL Native TLS

Description

If I upload a file using async mode and native TLS, I get the following error:

Err(UnexpectedResponse(Response { status: TransferAborted, body: [52, 50, 54, 32, 70, 97, 105, 108, 117, 114, 101, 32, 114, 101, 97, 100, 105, 110, 103, 32, 110, 101, 116, 119, 111, 114, 107, 32, 115, 116, 114, 101, 97, 109, 46, 13, 10] }))

which translates as:

>>> ''.join(map(chr,[52, 50, 54, 32, 70, 97, 105, 108, 117, 114, 101, 32, 114, 101, 97, 100, 105, 110, 103, 32, 110, 101, 116, 119, 111, 114, 107, 32, 115, 116, 114, 101, 97, 109, 46, 13, 10]))
'426 Failure reading network stream.\r\n'

If I look at the logs on the server, I see:

Sun Mar 24 09:57:00 2024 [pid 27668] [xxx] FTP command: Client "censored", "PASV"
Sun Mar 24 09:57:00 2024 [pid 27668] [xxx] FTP response: Client "censored "227 Entering Passive Mode (213,32,70,64,158,39)."
Sun Mar 24 09:57:00 2024 [pid 27668] [xxx] FTP command: Client "censored", "STOR Welcome.md"
Sun Mar 24 09:57:00 2024 [pid 27668] [xxx] FTP response: Client "censored", "150 Ok to send data."
Sun Mar 24 09:57:00 2024 [pid 27667] [xxx] DEBUG: Client "censored", "DATA connection terminated without SSL shutdown. Buggy client! Integrity of upload cannot be asserted."

Seems the SSL connection isn't properly terminated. Need an async drop :)

If I disable SSL or switch to the sync version, it works fine.

Steps to reproduce

let mut ftp_stream = AsyncNativeTlsFtpStream::connect(&config.remote_origin).await.unwrap_or_else(|err| {
        panic!("{err}");
    });
let mut ftp_stream = ftp_stream
        .into_secure(
            ctx,
            &config.remote_domain,
        ).await
        .unwrap();
    ftp_stream
        .login(
            &config.remote_user,
            &config.remote_password,
        ).await
        .unwrap();

let mut file = tokio::fs::OpenOptions::new()
            .read(true)
            .open(local_filename.clone())
            .await
            .unwrap().compat();
        let bytes_written = ftp_stream.put_file(parts[parts.len() - 1], &mut file).await;
println!("wrote {bytes_written:?} bytes");

Expected behaviour

The file to be uploaded normally.

Environment

  • OS: Manjaro
  • arch: x64
  • version: Git
  • protocol: FTPS
  • Remote: vsftpd

Additional information

Might need a better shutdown for the data stream than drop. Drop probably can't work because it can't be async and shutting down the SSL connection requires an async frame.

[Feature Request] - Make function `connect_with_stream` public

I'm working on an embedded system and more than one connection interface is used, so if I could have access to the connect_with_stream function, I could manipulate things more easily.

Description

Leaving function connect_with_stream public would make it easier to handle more specific connections.

Changes

  • Make the connect_with_stream function public

[Feature Request] - Support active mode

Description

suppaftp currently supports passive mode only, which is the most commonly used. Sometimes active mode is needed, for example with servers behind a firewall that doesn't open port for data connections.

I have already started implementing it because I needed it, but it would be great to add the feature to suppaftp! I will be happy to make any changes to fit in.

Changes

Support active mode

Which ports to open on the firewall?

I'm using for the first time today this crate and I'm super happy.

On my local machine everything works.

On production machine it does not.

On production machine there is a firewall and I don't know which which ports to be open.

Let's say I'm using the passive mode (by default) I think I need to open the port 21 at least.

And in fact it works partially: it creates directories an files but do not send any data.

What other port do I need?

[BUG] - Ftp with TLS (native-tls) Not working

Description

When copying the example from https://github.com/veeso/suppaftp#ftp-with-tls-native-tls the first unwrap errors with status: NotLoggedIn

Steps to reproduce

  1. cargo new ftps_test
  2. cd ftps_test
  3. cargo add suppaftp -F native-tls
  4. Copy example to main.rs
  5. cargo run

This gives me:

% cargo run
warning: unused import: `TlsStream`
 --> src/main.rs:1:42
  |
1 | use suppaftp::native_tls::{TlsConnector, TlsStream};
  |                                          ^^^^^^^^^
  |
  = note: `#[warn(unused_imports)]` on by default

warning: `ftps_test` (bin "ftps_test") generated 1 warning (run `cargo fix --bin "ftps_test"` to apply 1 suggestion)
    Finished dev [unoptimized + debuginfo] target(s) in 0.02s
     Running `target/debug/ftps_test`
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: UnexpectedResponse(Response { status: NotLoggedIn, body: [53, 51, 48, 32, 78, 111, 116, 32, 108, 111, 103, 103, 101, 100, 32, 105, 110, 46, 13, 10] })', src/main.rs:12:10
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Expected behaviour

I would assume I could expect the unwraps and the assert to not give an error

Environment

  • OS: Arch Linux
  • Architecture: x86
  • Rust version: 1.70
  • library version: 5.2.0
  • Protocol used: FTPS
  • Remote server version and name: test.rebex.net (not sure what they are running)

Additional information

It could very well be me that does something wrong

[BUG] - POSIX setgid/setuid/sticky bits not supported

Description

When using the LIST command, the setgid, setuid, and sticky bits (rendered over the execute bits as s/S, and t/T) are not supported.

Steps to reproduce

Have a directory on a remote server with one of the aforementioned bytes set on one of the directory entries.

$ ls -l
drwsrwsrwt ...

Try to use the list binding. This will result in a SyntaxError.

Expected behaviour

I expect the parser to accept this input.

Resolution

I plan to contribute a PR momentarily which will add basic support for these bits.

[SECURITY] - upgrade chrono->time dependency

Description

Chrono version used by this crate is really old and its using a really old time crate with a security issue (possible segmentation fault).
see
time-rs/time#293
chronotope/chrono#499

Severity:

  • critical
  • high
  • medium
  • low

A clear and concise description of the security vulnerability.

Additional information

Add any other context about the problem here.

[QUESTION] - Does suppaftp support an explicit proxy?

Here, we need to explciity use an HTTP proxy for http, https and ftp. It can be specified through the *_proxy family of environment variables. Does suppaftp support such a configuration? The CLI tool called lftp does but I'm interested in embedding this into a Rust CLI app.

[QUESTION] - Can't connect, no TLS

I can't make connection.
I use FTP server in Docker.
Here is the code:

        let host_addr = SocketAddr::V4(SocketAddrV4::new(host, port));
        println!("host_addr: {}", host_addr);
        TcpStream::connect(host_addr).unwrap();
        println!("Tcp stream connected");
        let mut ftp_stream = AsyncFtpStream::connect_timeout(host_addr, Duration::from_secs(5)).await.unwrap();

This produces something like:

host_addr: 127.0.0.1:32793
Tcp stream connected
thread 'test_upload' panicked:
called Result::unwrap() on an Err value: BadResponse

I tried to connect to the server using desktop client - worked without issue.
What can I do to debug the issue and fix it?

[QUESTION] - How to use cli-command

How to use cli-command?
As the suppaftp has been installed by 'cargo install suppaftp --features="secure cli-bin"', what should I do to test the cli-command?
I use suppaftp, but command not found!

Implicit SSL example

Hello developers, I recently used your crate and found it to be very interesting. Everything just works. However, I cannot seems to find way to connect when using implicit ssl settings. Filezilla seems to be working fine. Whenever I tried to connect it via the explicit method, it hanged at ftpstream::connect. Maybe this isn't the right way to connect. Could you show me how to do it? Thanks

[BUG] - expected `AsyncNoTlsStream`, found `AsyncNativeTlsStream`

Description

On a pet project I'm working on, I've set to use async-native-tls feature and when I use into_secure with AsyncNativeTlsConnector::from(TlsConnector::new()), similar to the README, I got the error from title of this bug.

Steps to reproduce

Steps to reproduce the bug you encountered:

use std::error::Error;
use suppaftp::{AsyncFtpStream, AsyncNativeTlsConnector};
use suppaftp::async_native_tls::{TlsConnector, TlsStream};

#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
    // Create a connection to an FTP server and authenticate to it.
    let mut ftp_stream = AsyncFtpStream::connect("localhost:21").await.unwrap();

    let mut connector = TlsConnector::new();

    let mut ftp_stream =
        ftp_stream
        .into_secure(AsyncNativeTlsConnector::from(connector), "localhost")
        .await
        .unwrap();

    Ok(())
}

Probably the bug is about this:

pub type AsyncFtpStream = ImplAsyncFtpStream<AsyncNoTlsStream>;

Missing to generate a type for async-native-tls feature.

Expected behaviour

Using the above code must work as expected and should compile at least.

Environment

  • OS: Darwin
  • Architecture x86_64
  • Rust version: 1.72.0
  • library version: 5.2.0
  • Protocol used: ftp
  • Remote server version and name: vsftpd 3.0

Additional information

None

[QUESTION] - How to properly use resume_transfer

Hi,

Thank you for your work on this crate!

I'm using it to create a custom ftp client with a priority queue of files to upload/download, and I'm facing difficulties to restart a file transfer from a given point.
I call the resume_transfer function, but then when I call either put_file or retr_as_stream, it restarts from the beginning each time, acting like there was no call to resume_transfer at all.

async fn restart_upload(mut ftp_stream: AsyncFtpStream, mut file: &async_std::fs::File, destination_filename: String) {
    let uploaded_bytes = ftp_stream.size(destination_filename.clone()).await
        .expect("No file at the given path on the FTP server");

    let _ = ftp_stream.resume_transfer(uploaded_bytes).await.expect("Resume failed"); 
    let uploaded_bytes = ftp_stream.put_file(destination_filename, &mut file).await.expect("Failed to upload the file");
                                 // Act as if there was no call to resume_transfer
}

For the upload, I managed to find a workaround with append_file and File trait Seek :

async fn restart_upload(mut ftp_stream: AsyncFtpStream, mut file: &async_std::fs::File, destination_filename: String) {
    let uploaded_bytes = ftp_stream.size(destination_filename.clone()).await
        .expect("No file at the given path on the FTP server");

    // Move the cursor after "uploaded_bytes" from the start
    let _seeked_bytes = file.seek(SeekFrom::Start(uploaded_bytes as u64)).await
        .expect("Could not move the cursor to given position");

    ftp_stream.append_file(destination_filename.as_str(), &mut file).await.expect("Failed to upload the file");
}

But I can't do the same thing with the DataStream I get back from the retr_as_stream call. At the moment, I am doing a full re-download and throwing away the part I already have with io::sink, but it's not optimal at all :/

async fn restart_download(mut ftp_stream: AsyncFtpStream, local_file_path: String, wanted_filename: String) {
    let mut file = async_std::fs::OpenOptions::new()
        .append(true)        
        .open(local_file_path.as_str()).await.expect("Could not open file");
    let file_size = file.metadata().await.unwrap().len();

    let _ = ftp_stream.transfer_type(Binary).await.expect("Could not set transfert type to binary");
    let _ = ftp_stream.resume_transfer(file_size as usize).await.expect("Resume failed");

    let mut data_stream = ftp_stream.retr_as_stream(wanted_filename).await
        .expect("Could not find the given file on remote server");
    
    // Discard already downloaded bytes. Imply a full re-download
    async_std::io::copy(&mut data_stream.by_ref().take(file_size), &mut io::sink()).await.expect("Could not skip the bytes");

    // Read the rest
    async_std::io::copy(&mut data_stream, &mut file).await.expect("Error during file downloading");

    ftp_stream.finalize_retr_stream(data_stream).await.unwrap();
}

Am I missing something ?

[QUESTION] - Is there a plan to support sftp

Hi, I'm looking for a ftp library, and find your crate is easily to use! Thanks!

Since it doesn't mentioned about sftp, I think it doesn't support it for now, just wonder is there any plan to support it?

[BUG] - features are not additive

Description

Features are not additive, for example, if a dependency uses the 'async' functionality then it breaks the sync functionality

Steps to reproduce

Enable the async feature and try using the sync API

Expected behaviour

Features should be purely additive, they should not break API's otherwise.

Environment

  • OS: [e.g. GNU/Linux Debian 10] Debian 11
  • Architecture [Arm, x86_64, ...] x86_64
  • Rust version 1.67.1
  • library version 4.7.0
  • Protocol used FTP
  • Remote server version and name Not a clue, can't get it to compile because different deps are using it in different ways

Additional information

Features should only ever be additive. One library that's being used uses it with the sync API, another is using it via the async, yet the async runner here is tokio and on initial testing it shows there's some issues with things because it appears to be hardcoded to async-std (which is not well used in the ecosystem at this time)?

[QUESTION] - 522 SSL connection failed

Hello

I am trying to play around with this FTPS server https://dlptest.com/ftp-test/
But when I try to list dir or downloa a file I get the following error:

Invalid response: [0] 522 SSL connection failed; session reuse required: see require_ssl_reuse option in vsftpd.conf man page

I get that it is a server error, but using FileZilla it seem to ignore or some how work around this error, is that possible to do with SuppaFTP as well?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.