ahmedsoliman / netperf

A network performance measurement tool written in Rust

netperf

A network (TCP-only) performance measurement tool written in Rust, inspired by iperf3's original code.

All basic features are implemented. Key differences from iperf3:

• Uses a different control protocol (not compatible with iperf3's servers or clients)
• Multi-threaded, parallel streams (-P) will be executed on different threads.
• Design simulates realworld server/client applications in terms of work scheduling.

Installation

cargo install --locked netperf

Usage

On one node you run netperf in server mode:

netperf -s

On a client node, you need to connect to that server (you will need an addressable IP address IPv6 is supported).

netperf -c ::1

By default, the test will use a single stream (client sends and server receives). You can control the number of parallel streams with -P and the direction of traffic with -R/--bidir

Current Limitations

• Does not support configuring MSS, Congestion control algorithm.
• No UDP/STCP support.
• Does not collect extra stats like retransmits, cwnd, etc. (contributions are appreciated)

License

Licensed under either of Apache License, Version 2.0 or MIT license at your option. Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in this crate by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

netperf's Issues

Emit `[SUM]` output with -P on each iteration

iperf output:

[  5]   2.00-2.78   sec  2.19 GBytes  24.1 Gbits/sec    0   1.31 MBytes
[  7]   2.00-2.78   sec  2.19 GBytes  24.1 Gbits/sec    1   1.37 MBytes
[SUM]   2.00-2.78   sec  4.38 GBytes  48.2 Gbits/sec    1
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-2.78   sec  7.98 GBytes  24.6 Gbits/sec    0             sender
[  5]   0.00-2.78   sec  0.00 Bytes  0.00 bits/sec                  receiver
[  7]   0.00-2.78   sec  7.98 GBytes  24.6 Gbits/sec    1             sender
[  7]   0.00-2.78   sec  0.00 Bytes  0.00 bits/sec                  receiver
[SUM]   0.00-2.78   sec  16.0 GBytes  49.3 Gbits/sec    1             sender
[SUM]   0.00-2.78   sec  0.00 Bytes  0.00 bits/sec                  receiver

netperf output:

[  0]   6.00..7.00 sec  5.30 GiB   45.52 Gbits/sec        sender
[  1]   7.00..8.00 sec  5.37 GiB   46.15 Gbits/sec        sender
[  0]   7.00..8.00 sec  5.23 GiB   44.90 Gbits/sec        sender
[  1]   8.00..9.00 sec  4.86 GiB   41.76 Gbits/sec        sender
[  0]   8.00..9.00 sec  4.96 GiB   42.57 Gbits/sec        sender
- - - - - - - - - - - - - - - - - - - - - - - - - - - - -
ID   Interval          Transfer      Bitrate
[  0]   0.00..10.00 sec  52.25 GiB   44.88 Gbits/sec        sender
[  0]   0.00..10.00 sec  52.03 GiB   44.69 Gbits/sec        receiver
[  1]   0.00..10.00 sec  51.24 GiB   44.01 Gbits/sec        sender
[  1]   0.00..10.00 sec  51.01 GiB   43.81 Gbits/sec        receiver

[SUM]   0.00..10.00 sec  103.49 GiB   88.89 Gbits/sec        sender
[SUM]   0.00..10.00 sec  103.04 GiB   88.51 Gbits/sec        receiver

You can see netperf only emits the SUM on completion, rather than with each report. Its nice seeing the sum on each report to get a feel for the results before completion.

Test aborted between NAS and pc (windows)

Server

NAS Synology
Linux
aarch64-unknown-linux-musl

[2021-04-05T14:58:25Z TRACE mio::poll] registering event source with poller: token=Token(1), interests=READABLE | WRITABLE
[2021-04-05T14:58:25Z INFO  netperf::server] Accepted connection from [::ffff:192.168.1.28]:62535
[2021-04-05T14:58:25Z DEBUG netperf::common::net_utils] Received a control message (77 bytes) in 1 iterations
[2021-04-05T14:58:25Z DEBUG netperf::server] Hello received: 9c1c401d-ed25-4498-addb-fa51a0351d90
[2021-04-05T14:58:25Z DEBUG netperf::common::net_utils] Sent: 31 bytes
[2021-04-05T14:58:25Z DEBUG netperf::common::net_utils] Received a control message (200 bytes) in 1 iterations
[2021-04-05T14:58:25Z INFO  netperf::server] [[::ffff:192.168.1.28]:62535] Test Created
[2021-04-05T14:58:25Z DEBUG netperf::controller] Controller has started
[2021-04-05T14:58:25Z DEBUG netperf::controller] Controller state: TestStart
[2021-04-05T14:58:25Z DEBUG netperf::controller] Test is initialising
[2021-04-05T14:58:25Z DEBUG netperf::common::net_utils] Sent: 102 bytes
[2021-04-05T14:58:25Z DEBUG netperf::controller] Controller state: CreateStreams { cookie: "9c1c401d-ed25-4498-addb-fa51a0351d90" }
[2021-04-05T14:58:25Z DEBUG netperf::controller] Waiting for data streams to be connected
[2021-04-05T14:58:25Z TRACE mio::poll] registering event source with poller: token=Token(2), interests=READABLE | WRITABLE
[2021-04-05T14:58:25Z DEBUG netperf::common::net_utils] Received a control message (77 bytes) in 1 iterations
[2021-04-05T14:58:25Z DEBUG netperf::common::net_utils] Sent: 31 bytes
[2021-04-05T14:58:25Z DEBUG netperf::common::net_utils] Sent: 44 bytes
[2021-04-05T14:58:25Z DEBUG netperf::controller] Controller state: Running
[2021-04-05T14:58:25Z DEBUG netperf::controller] Streams have been created, starting the load test
[ ID]   Interval          Transfer      Bitrate
[2021-04-05T14:58:25Z DEBUG netperf::controller] Broadcasting to streams StartLoad
[2021-04-05T14:58:25Z DEBUG netperf::common::stream_worker] Data stream 0 created (receiving), waiting for the StartLoad signal!
[2021-04-05T14:58:25Z WARN  netperf::common::stream_worker] Stream 0's connection has been closed.
[2021-04-05T14:58:25Z TRACE mio::poll] deregistering event source from poller
[2021-04-05T14:58:25Z DEBUG netperf::common::net_utils] Sent: 52 bytes
[2021-04-05T14:58:25Z DEBUG netperf::controller] Controller state: ExchangeResults
[2021-04-05T14:58:25Z DEBUG netperf::controller] Broadcasting to streams Terminate
[2021-04-05T14:58:25Z TRACE mio::poll] registering event source with poller: token=Token(16777218), interests=READABLE | WRITABLE
[2021-04-05T14:58:25Z INFO  netperf::server] Test already in-flight, rejecting connection from [::ffff:192.168.1.28]:62537
[2021-04-05T14:58:25Z DEBUG netperf::common::net_utils] Sent: 55 bytes
[2021-04-05T14:58:25Z TRACE mio::poll] deregistering event source from poller
[2021-04-05T14:58:25Z TRACE mio::poll] deregistering event source from poller
[2021-04-05T14:58:25Z DEBUG netperf::server] Controller aborted: unexpected end of file
Test aborted!

Client

Windows
target = x86_64-pc-windows-gnu

[2021-04-05T14:58:27Z INFO  netperf] LogLevel: TRACE
[2021-04-05T14:58:27Z TRACE mio::poll] registering event source with poller: token=Token(0), interests=READABLE | WRITABLE
Connecting to (192.168.1.23:7559) ...Connected!
[2021-04-05T14:58:27Z DEBUG netperf::common::net_utils] Sent: 81 bytes
[2021-04-05T14:58:27Z DEBUG netperf::client] Hello sent!
[2021-04-05T14:58:27Z DEBUG netperf::common::net_utils] Received a control message (27 bytes) in 1 iterations
[2021-04-05T14:58:27Z DEBUG netperf::client] Welcome received!
[2021-04-05T14:58:27Z DEBUG netperf::common::net_utils] Sent: 204 bytes
[2021-04-05T14:58:27Z DEBUG netperf::client] Params sent!
[2021-04-05T14:58:27Z DEBUG netperf::controller] Controller has started
[2021-04-05T14:58:27Z DEBUG netperf::controller] Controller state: TestStart
[2021-04-05T14:58:27Z DEBUG netperf::common::net_utils] Received a control message (98 bytes) in 1 iterations
[2021-04-05T14:58:27Z DEBUG netperf::controller] Controller state: CreateStreams { cookie: "9c1c401d-ed25-4498-addb-fa51a0351d90" }
[2021-04-05T14:58:27Z DEBUG netperf::controller] We should create data streams now
[2021-04-05T14:58:27Z DEBUG netperf::controller] Opening a data stream to (192.168.1.23:7559) ...
[2021-04-05T14:58:27Z TRACE mio::poll] registering event source with poller: token=Token(1), interests=READABLE | WRITABLE
[2021-04-05T14:58:27Z DEBUG netperf::common::net_utils] Sent: 81 bytes
[2021-04-05T14:58:27Z DEBUG netperf::common::net_utils] Received a control message (27 bytes) in 1 iterations
[2021-04-05T14:58:27Z DEBUG netperf::controller] Data stream created to 192.168.1.23:7559
[2021-04-05T14:58:27Z TRACE mio::poll] deregistering event source from poller
[2021-04-05T14:58:27Z WARN  netperf::controller] Stream 0 has terminated with an error, we cannot fetch its total stream stats. This means that the results might be partial
[2021-04-05T14:58:27Z DEBUG netperf::controller] Controller state: CreateStreams { cookie: "9c1c401d-ed25-4498-addb-fa51a0351d90" }
[2021-04-05T14:58:27Z DEBUG netperf::controller] We should create data streams now
[2021-04-05T14:58:27Z DEBUG netperf::controller] Opening a data stream to (192.168.1.23:7559) ...
[2021-04-05T14:58:27Z TRACE mio::poll] registering event source with poller: token=Token(16777217), interests=READABLE | WRITABLE
[2021-04-05T14:58:27Z DEBUG netperf::common::net_utils] Sent: 81 bytes
[2021-04-05T14:58:27Z DEBUG netperf::common::net_utils] Received a control message (51 bytes) in 1 iterations
[2021-04-05T14:58:27Z TRACE mio::poll] deregistering event source from poller
[2021-04-05T14:58:27Z TRACE mio::poll] deregistering event source from poller
Error: Access denied: Test already in-flight 

`netperf -1` panics

$ RUST_BACKTRACE=1 netperf -1
thread 'main' panicked at 'called `Option::unwrap()` on a `None` value', /usr/local/google/home/howardjohn/.cargo/registry/src/github.com-1ecc6299db9ec823/netperf-0.2.3/src/client.rs:15:56
stack backtrace:
   0: rust_begin_unwind
             at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panicking.rs:584:5
   1: core::panicking::panic_fmt
             at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/panicking.rs:142:14
   2: core::panicking::panic
             at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/panicking.rs:48:5
   3: netperf::main::{{closure}}
   4: std::thread::local::LocalKey<T>::with
   5: tokio::park::thread::CachedParkThread::block_on
   6: tokio::runtime::scheduler::multi_thread::MultiThread::block_on
   7: tokio::runtime::Runtime::block_on
   8: netperf::main
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

A cursory look seems to suggest that one_off is just a flag but not implemented, so it ends up hitting the wrong code path

UDP support

Hi Ahmed,

I"m wondering whether UDP is in the plans for this crate.

Thanks!

Summary calculations in bidirectional streams are incorrect

For a connection where we have ~14MBits/sec upstream and 43MBits/sec downstream, netperf shows the result like following:

- - - - - - - - - - - - - - - - - - - - - - - - - - - - -
ID   Interval          Transfer      Bitrate
[1]   0.00..10.00 sec  50.93 MiB   42.72 Mbits/sec        receiver
[1]   0.00..10.01 sec  51.74 MiB   43.36 Mbits/sec        sender
[0]   0.00..10.00 sec  17.58 MiB   14.74 Mbits/sec        sender
[0]   0.00..10.00 sec  17.45 MiB   14.64 Mbits/sec        receiver

[SUM]   0.00..10.01 sec  69.31 MiB   58.08 Mbits/sec        sender
[SUM]   0.00..10.00 sec  68.37 MiB   57.35 Mbits/sec        receiver

These totals are incorrect since we are adding (the local sender + the remote sender) and same for receiver.

Improve use as a library

Huge fan of this tool - thanks for providing it!

I was interested in additionally using it as a library. This sort of works - I can easily import it and run a client and server. However, a few tweaks would make it a bit more ergonomic to use.

• Opts: probably could make the client not require server params and vis-versa; neither should need to depend on verbose
• Server: allow a way to bind to port 0 (random) then get the real port
• Server: expose when bound to port. Currently its hard to do run_server() and then know when its safe to run the client
• sub-second granularity of test runs would be nice
• return a TestResults from run_client for programmatic access to the results