Track: port RustScan to tokio (PR #856)

#944 · open · 1 comments

View on GitHub ↗

bee-san

Tracking issue for the async-runtime port proposed in #856 ("port to tokio again" by @vrtgs). #856 is a large architecture change, so it stays open and isn't part of the current round of PR merges. This issue records what it does and what it would need before it can land. ## Why it matters RustScan's scanner, DNS resolution and tests run on `async-std`. Upstream has discontinued `async-std` and recommends migrating away (see the async-std README/CHANGELOG and RUSTSEC-2025-0052). So moving to a maintained runtime is worth doing eventually. ## What #856 changes - **Size:** one commit, +966/−1218 across 13 files (about 1.2k of that churn is `Cargo.lock`). There's no PR description and no benchmark numbers. - **Runtime:** replaces `async-std`/`futures` with `tokio` (`rt-multi-thread`, `net`, `fs`, `macros`). It also adds `tokio-par-stream`, `futures-lite`, `futures-util`, `either` and `blackrock2`. - **Port randomisation:** deletes the LCG `src/port_strategy/range_iterator.rs` and switches to `blackrock2`. It also reshapes `PortStrategy` (`src/port_strategy/mod.rs` → `src/port_strategy.rs`). - **DNS/address parsing:** becomes async (`TokioAsyncResolver`, `tokio::net::lookup_host`, async file reads). `parse_addresses` turns into an `async fn`, and `parse_address` now takes a `Cow<str>` and returns an iterator. - **Scanner:** rewrites `Scanner::run`, the socket iterator and the TCP/UDP connect paths on top of tokio. - **Library API:** `rustscan::address`, `rustscan::scanner` and `rustscan::port_strategy` all change in breaking ways, so this needs a semver-major release (or at least a clearly announced minor one). ## Conflicts / staleness - `CONFLICTING` with `master`. It was branched before several dependency bumps (`rand` 0.8, `dirs` 5, `toml` 0.8, `colored` 2, `subprocess`), and nearly every file it touches has changed since. - It edits `tests/timelimits.rs`, which no longer exists. - It turns the address tests into `#[tokio::test]`, including ones that resolve `google.com`, invalid hostnames and host files. Those network/DNS tests were removed on purpose in #942/#943, so they must not come back. ## Risks - **Scanning hot path:** concurrency, batching and timeout behaviour can shift in subtle ways, including fd limits and the batch-size/ulimit logic. - **Performance:** speed is RustScan's main feature, and no comparison has been posted. - **New dependencies:** `blackrock2` and `tokio-par-stream` are small, little-used crates. They'd need a supply-chain review, or replacing with well-known alternatives. - **Port order:** replacing the LCG changes the order (and the distribution) of randomised scans. - **Embedders:** library users have to adopt tokio and the new async signatures. - **Open PRs:** it will conflict with every PR in flight that touches the scanner or address parsing. ## What would be needed to land it 1. Rebase onto current `master`, with no network/DNS/script-executing tests (keep the #942 policy). 2. Split it into reviewable steps, e.g. (a) swap the runtime and keep the public API, (b) async resolver, (c) any port-randomisation change as its own PR (or keep the existing LCG iterator). 3. Justify or drop each new dependency, and keep `cargo tree` growth minimal. 4. Post before/after scan-speed numbers from a controlled environment. 5. Document the library API migration and decide on the version bump. 6. CI green on Linux, macOS and Windows. PR: #856

Comments

bee-san

#951 ports RustScan to tokio, following the plan above. All CI is green, including the new benchmark. 1. **Rebased on `master`, no network tests.** Unit tests still never open sockets (#942). Scanning and speed are measured by a new opt-in workflow, `.github/workflows/scan-benchmark.yml`. It builds the PR and its base and scans listeners it opens on 127.0.0.1 inside the runner, plus dropped ports and a network-namespace host 5 ms away. It runs on Linux x64/arm64, macOS and Windows, and fails on wrong results or clear slowdowns. 2. **Split into reviewable steps.** First a like-for-like runtime swap that keeps the public signatures, then separate perf/fix commits. The resolver stays synchronous for now (it runs before the scan runtime exists, so it never blocks the scan), and the LCG port order is unchanged. 3. **No new dependencies.** tokio was already in the tree through hickory-resolver. `Cargo.lock` loses 28 packages (async-std and its runtime) and gains none. `tokio-par-stream`, `blackrock2`, `either` and `futures-lite` from #856 aren't needed. 4. **Before/after numbers** (medians over interleaved CI runs; full tables in the PR): - Linux x64: TCP and UDP sweeps 39–74% faster, first open port in 6 ms instead of 185 ms, CPU time about halved, peak RSS and descriptors equal or lower, filtered/timeout scenarios unchanged. - arm64 and macOS show similar gains. - Windows sweeps are bound by its connect retries: throughput −0.3% to +12%, first open port 57–97% sooner. 5. **Library API.** The signatures are unchanged, but `Scanner::run` must now be polled from a tokio runtime. That's documented in the crate docs; I'd ship it as a minor release with a release note (or 3.0). 6. **CI green on Linux, macOS and Windows.** Tests and benchmark both pass. The first benchmark round also caught two tokio-specific problems, both fixed in the PR. The single-threaded scan could starve tokio's I/O driver during long polls; on Windows this made one build miss every open port with `-b 10000 -t 500`. And tokio's UDP `recv` never sees ICMP "port unreachable" errors on Linux. Supersedes #856. Thanks @vrtgs, who is credited as co-author of the port commit.