tokio-rs / tokio-uring

An io_uring backed runtime for Rust

tokio-uring

This crate provides io-uring for Tokio by exposing a new Runtime that is compatible with Tokio but also can drive io-uring-backed resources. Any library that works with Tokio also works with tokio-uring. The crate provides new resource types that work with io-uring.

API Docs | Chat

Getting started

Using tokio-uring requires starting a [tokio-uring] runtime. This runtime internally manages the main Tokio runtime and a io-uring driver.

In your Cargo.toml:

[dependencies]
tokio-uring = { version = "0.4.0" }

In your main.rs:

use tokio_uring::fs::File;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    tokio_uring::start(async {
        // Open a file
        let file = File::open("hello.txt").await?;

        let buf = vec![0; 4096];
        // Read some data, the buffer is passed by ownership and
        // submitted to the kernel. When the operation completes,
        // we get the buffer back.
        let (res, buf) = file.read_at(buf, 0).await;
        let n = res?;

        // Display the contents
        println!("{:?}", &buf[..n]);

        Ok(())
    })
}

Requirements

tokio-uring requires a very recent linux kernel. (Not even all kernels with io_uring support will work) In particular 5.4.0 does not work (This is standard on Ubuntu 20.4). However 5.11.0 (the ubuntu hwe image) does work.

Project status

The tokio-uring project is still very young. Currently, we are focusing on supporting filesystem and network operations. Eventually, we will add safe APIs for all io-uring compatible operations.

License

This project is licensed under the MIT license.

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in tokio-uring by you, shall be licensed as MIT, without any additional terms or conditions.

tokio-uring's Issues

a bug?

hello
I want to test performance on mmap,BIO,AIO,and iouring(by tokio-uring) by write 1KB data to 1G files 1024 times . then i write the code bellow.when program running i find something unexpected.

1. too many threads created and destroyed,maybe 20+ thousands.(my test server is 8c16g,5.15.10-1.el7.elrepo.x86_64)
2. some file succeed ,some not,no exception throwed,program is still alive,but no worker threads work now .many file are only 794MB and not growing, the others are 1024MB. bug here?
3. the writing speed is much more slowly than tranditional io(maybe all AIO model slowly the same).

use std::thread;
use std::time::Duration;
use std::time::Instant;
use std::{path::PathBuf, sync::Arc};
use tokio::sync::mpsc;
use tokio::sync::mpsc::UnboundedSender;
use tokio::sync::Mutex;
use tokio::time::sleep;
use tokio_uring::fs::{File, OpenOptions};
fn main() {
    tokio_uring::start(async {
        let (tx, mut rx) = mpsc::unbounded_channel::<()>();
        let txx = Arc::new(tx);
        let count = Instant::now();
        for i in 0..1024 {
            let t = txx.clone();
            tokio_uring::spawn(create(i, t));
        }

        drop(txx);
        while let Some(_) = rx.recv().await {}
        println!("{:?}", count.elapsed());
    });
}

async fn create(idx: i32, tx: Arc<UnboundedSender<()>>) {
    let path = PathBuf::from(r"/data2/testuring/".to_owned() + idx.to_string().as_str());
    let mut file = OpenOptions::new()
        .read(true)
        .write(true)
        .create(true)
        .open(&path)
        .await
        .unwrap();
    // file.set_len(1024 * 1024 * 1024).await.unwrap();
    println!("create file={} {:?}", idx, thread::current().id());
    let x=b"0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef";
    let y=&x[..];
    let size_1_g = 1024 * 1024 * 1024;
    let mut idx = 0;
    while idx < size_1_g {
        let (_, _) = file.write_at(y, idx).await;
        idx = idx + 1024;
    }
    println!("end file={:?} {:?}", &path, thread::current().id());
    file.close().await.unwrap();
    drop(tx);
}

Reconsider API design

Currently, our API just adapts the existing tokio APIs to be completion-based. While this makes the library easier to use, it raises a number of problems for the maintenance of these APIs.

io_uring has a large number of useful flags which can be applied to operations, and more continue to be added. We cannot effectively make use of many of these flags (e.g event linking flags, buffer preallocation flags, etc.) without adding more and more functions with larger signature or issuing frequent breaking changes to said functions as new useful flags come along, which might continue to occur for some time.

I propose that we move to a builder API for operations. A builder pattern would allow operations to be constructed, with decisions like buffer strategy selection and operation linking made before the operation is dispatched to yield a future. This would offer a tremendous benefit especially to power users of io_uring, who may be trying to take advantage of various features of the API to improve performance or reliability.

Add missing filesystem operations

It would be great if we could add the most important functions from tokio::fs/std::fs to this crate. As far as I can tell tokio-uring only exposes fs::File right now and none of the ones below are supported yet.

Directories:

• create_dir
• read_dir
• remove_dir
• remove_dir_all

Files:

• metadata
• rename
• copy
• remove_file

Links (I don't really need these but would be great to have):

• symlink_metadata
• soft_link
• hard_link
• read_link

This should be fairly straightforward to implement, right? I can try to take a stab at some of these once I find some time.

Could add a udp server demo with using IORING_FEAT_FAST_POLL and IOSQE_BUFFER_SELECT ?

Test failure: Os { code: 9, kind: Uncategorized, message: "Bad file descriptor" }

Upon checking out the code and running cargo test I observe:

---- too_many_submissions stdout ----
thread 'too_many_submissions' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 9, kind: Uncategorized, message: "Bad file descriptor" }', tests/driver.rs:84:56
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- complete_ops_on_drop stdout ----
thread 'complete_ops_on_drop' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 9, kind: Uncategorized, message: "Bad file descriptor" }', tests/driver.rs:53:56

problem with explicit_close test

The explicit_close test sometimes fails because by the time it asserts the raw fd it is left holding is no longer valid, the kernel has opened a new file for one of the other tests and reused the raw fd value; so the raw fd is valid after all, just for a different file.

Not sure yet the best way to work around this race between one test and the other tests.

Manually, it is possible to run them with a single thread and then this test passes without any modifications.

cargo test --test fs_file -- --test-threads=1

Found an env variable can be set in .config/cargo.toml for the local project and with that the test threads can be set to 1.
Will send a PR when the existing PR is resolved.

Allow access for Runtime struct?

Hi

I'm trying to benchmark tokio-uring and different implementations. As criterion requires Runtime struct to run async code, I'd like to ask for changing Runtime struct visibility.

Is it valid change or any other viable solution to achieve it?

Thanks

Use of IORING_REGISTER_PBUF_RING, introduced in kernel 5.19

An issue to track this crate's use of the register op IORING_REGISTER_PBUF_RING.

First described here io_uring: add support for ring mapped supplied buffers

Hangs on `drop` of `Runtime` if `TcpListener` has an outstanding `accept` op

Since there is no accept with a timeout I've used a standard timeout operation to implement a graceful shutdown. It kind of worked, but lead to a hang at the end of tokio_uring::start scope.

Support Builder Tyes

Something which has been mentioned several times, but I don't think has a dedicated issue?

problematic use of try_join in network examples

There are two examples, both related to networking, that use the tokio try_join macro to weave a bound socket that will accept with a connection attempt to that socket. One for the TcpListener, and one for the UnixListener.

At least in the case of TcpListener, there are times on my 5.15 kernel where this fails.

The original symptom was the try_join would just hang, but only when what should have been an innocuous change to the on_thread_park was made. My best guess at the moment was the different compile caused a reordering of what was attempted in the try_join first. But without further digging into try_join, my guess may be wrong.

After a println was added to the on_thread_park callback, the text from the println was not printed but a useful panic from within the try_join did manifest itself. The panic said

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 97, kind: Uncategorized, message: "Address family not supported by protocol" }'

And this is the same error I've seen before when a TCP connection is attempted on my kernel before a server has called accept on the socket.

So, aside from some questions about why try_join was hung in one case, and why a different compile would make it succeed at times, it seems a more clinical approach to providing a simple example is called for - to avoid this problem entirely.

A PR that uses spawn for the accept and a one shot channel for getting the resulting TCPStream back to the main thread is forthcoming. For consistency, I'll make the same change for the UnixListener example.

Close on Drop Semantics

What do we think about our current semantics regarding close-on-drop?

I'm not sure if asynchronously pushing a close op is actually the best approach on drop. It complicates our code a surprising amount, as we need to be able to handle the case here where we drop outside of a runtime context, and thus we need to do a bit of working around this with try_submit and other hacks.

Personally, I think that we should change the semantics to synchronously close the fd (like tokio does) on drop and allow users to explicitly invoke an async close operation if they want to close an fd with uring.

`driver::op::submit_with()` panic

Pr #144 can trigger This panic.

It can be reliably triggered if:

• The concurrency factor submitting operations is greater than the submission queue size.
• .setup_sqpoll() is specified on the ring builder

I speculate this can occur under other conditions, but the above is an easy way to fill the submission queue without draining it (i.e, if the ring poll thread is not scheduled whilst simultaneously having loads of things adding entries)

potential errors from sys::io_uring_enter usually being ignored

tokio-uring gets submissions passed to the kernel with its calls to uring.submit() (ultimately getting sys::io_uring_enter called) which it does in two places.

The most common way to get submit called is to have the on_thread_park callback invoked when the current thread has no more active work to do. Under normal operation, the thread would have handled some uring results and loaded some new uring submissions.

tokio-uring/src/runtime.rs

Lines 53 to 57 in 4026539

	.on_thread_park(|| {
	CURRENT.with(|x| {
	let _ = RefCell::borrow_mut(x).uring.submit();
	});
	})

The other, less common mechanism for issuing a submit call, is for the application to create so many submissions while the current thread is continuously doing work that the submission queue becomes full and the tokio-uring driver is forced to submit what is there to make room for another submission.

tokio-uring/src/driver/op.rs

Lines 71 to 74 in 4026539

	// If the submission queue is full, flush it to the kernel
	if inner.uring.submission().is_full() {
	inner.submit()?;
	}

But errors that the call to sys::io_uring_enter may return are ignored by the common route, the first one shown. And if the submission queue were filled up, the error returned that the caller does see, via the second route shown, aren't directly related to its submission, but rather indicate a problem with the uring or its use in general.

Here are the current documented io_uring_enter system call errors:

These are the errors returned by io_uring_enter() system call.

       EAGAIN The kernel was unable to allocate memory for the request, or
              otherwise ran out of resources to handle it. The application should
              wait for some completions and try again.

       EBADF  fd is not a valid file descriptor.

       EBADFD fd is a valid file descriptor, but the io_uring ring is not in the
              right state (enabled). See io_uring_register(2) for details on how
              to enable the ring.

       EBADR  At least one CQE was dropped even with the IORING_FEAT_NODROP
              feature, and there are no otherwise available CQEs. This clears the
              error state and so with no other changes the next call to
              io_uring_setup(2) will not have this error. This error should be
              extremely rare and indicates the machine is running critically low
              on memory and. It may be reasonable for the application to
              terminate running unless it is able to safely handle any CQE being
              lost.

       EBUSY  If the IORING_FEAT_NODROP feature flag is set, then EBUSY will be
              returned if there were overflow entries, IORING_ENTER_GETEVENTS
              flag is set and not all of the overflow entries were able to be
              flushed to the CQ ring.

              Without IORING_FEAT_NODROP the application is attempting to
              overcommit the number of requests it can have pending. The
              application should wait for some completions and try again. May
              occur if the application tries to queue more requests than we have
              room for in the CQ ring, or if the application attempts to wait for
              more events without having reaped the ones already present in the
              CQ ring.

       EINVAL Some bits in the flags argument are invalid.

       EFAULT An invalid user space address was specified for the sig argument.

       ENXIO  The io_uring instance is in the process of being torn down.

       EOPNOTSUPP
              fd does not refer to an io_uring instance.

       EINTR  The operation was interrupted by a delivery of a signal before it
              could complete; see signal(7).  Can happen while waiting for events
              with IORING_ENTER_GETEVENTS.

They relate to interesting reasons for a problem. Without any corrective action, at a minimum, the uring should be shut down and all further uring operations invalidated. At a maximum, the error should be reported and the current thread panicked.

In the async world of tokio-uring, what would a mechanism look like that didn't want to drop any errors from io_uring_enter and how would they be delivered back to the caller?

Should the default be to panic in the on_park_enter callback? Should the user be given the chance to register a callback for errors encountered by the on_park_enter callback? If so, should the same mechanism be built into the second submit route, so the two can be handled in a uniformed manor, freeing the caller of a singular submission from having any error handling?

Thread starvation

I saw #78 and I triggered flashbacks to an issue I ran into when implementing my own io_uring runtime a year ago.

I had 1k frontend requests evenly split between threads and all blocked (parked) on 1 backend fetch. When the backend fetch finishes, the thread that issued it is woken and can process its frontend requests. However, the requests bound to the other threads will stay blocked on iouring until they get randomly woken by unrelated network activity. This can cause significant latency spikes depending on the traffic load.

Basically, once a thread calls submit_and_wait, there's no way to unpark it for non-IO work.

My solution was to was create an eventfd for each thread. Right before getting parked, the thread queues up a read on the eventfd and submits all of the tasks to uring. In order to unpark a thread, another thread writes to the eventfd using its local uring. There's some atomics sprinkled in to avoid unnecessary wakeups.

My repo is currently private at the moment because my Rust sucks but I can make it public if the reference would be useful.

Some `Arc`ed buffer for parallel (but disjoint) IO?

I'm trying to figure out how can I have a one big Vec<u8>-like buffer that I could split into disjoint parts and have parallel uring operations on each part. From what I can tell right now slice consumes the Vec so this will not work.

I guess this might require some Arc-like functionality, where I could hand over Vec to the runtime, get a slice for buf[..], and then be able to call slice.split(index) to split the slice? Under the hood runtime would keep track of how many slices were handed over, and calling some slice.into_inner() on the last one would get the original Vec back?

I will have to do without it for now, but I think it's a useful use case to consider when designing the API.

SharedFd doesn't call wake()

There's some plumbing in shared_fd.rs for tracking a waker but I don't see a call to wake(). There's a good chance a call to file.close().await would hang if the SharedFd was being shared with something else at the time.

I was going to submit a PR for this, but I took the route of supporting multiple close calls, so keeping a list of wakers, and it was causing lots of changes to the file and then I still hadn't figured out how to let multiple calls block on a close().await but already have the RC strong count not count them ... and I decided to punt on that and just log this issue. Maybe someone closer to the original code will see how to fix it up with minimal changes.

If you do want a File to be clonable and closable multiple times, let me know and I'll finish what I had started. Otherwise the fix can wait for a single call to close to be supported.

This isn't a blocker for my work so no great rush. I just ran into it while reading through more of the code base.

CI improvements

When adding new IO_URING ops, we often want access to a bleeding edge, or even unreleased kernel to properly test.

KVM would be one approach, but there is a long standing github actions issue regarding support for that. actions/runner-images#183

I'm not aware of other approaches, but I'd love to get some suggestions to avoid things like #175 again

[Bug] Wrong `Slice::deref/deref_mut` implementation

Here's a minimal reproducible example:

let v = Vec::with_capacity(32);
let slice = v.slice(..);
let x = &*slice;
// raise error 'range end index 32 out of range for slice of length 0'

The reason is that Slice creation uses bytes_total(), Slice::deref uses bytes_init().
The module level deref/deref_mut or the Slice::deref/deref_mut implementation, I'm not sure which one should be modified.

tokio-uring/src/buf/io_buf.rs

Lines 79 to 83 in 2c9fbea

	let end = match range.end_bound() {
	Bound::Included(&n) => n.checked_add(1).expect("out of range"),
	Bound::Excluded(&n) => n,
	Bound::Unbounded => self.bytes_total(),
	};

tokio-uring/src/buf/mod.rs

Lines 16 to 20 in 2c9fbea

	pub(crate) fn deref(buf: &impl IoBuf) -> &[u8] {
	// Safety: the `IoBuf` trait is marked as unsafe and is expected to be
	// implemented correctly.
	unsafe { std::slice::from_raw_parts(buf.stable_ptr(), buf.bytes_init()) }
	}

tokio-uring/src/buf/slice.rs

Lines 126 to 132 in 2c9fbea

	impl<T: IoBuf> ops::Deref for Slice<T> {
	type Target = [u8];
	fn deref(&self) -> &[u8] {
	&super::deref(&self.buf)[self.begin..self.end]
	}
	}

Implement local_addr for TcpListener and UnixListener

I don't believe io_uring can be used to get the local_addr from a socket that is TCP listening or Unix listening, but the ability would be useful nonetheless.

Is it okay to provide blocking method calls to return the local_addr for the two types?

If so, new unit tests would not need to hardcode port numbers in their TCP tests. Existing unit tests could also be modified.

Random panics at closing file descriptors

I observe random panics when running the following code:

async fn compute_tokio_uring() {
    use tokio::task::JoinHandle;
    use tokio_uring::fs::File;

    let handles: Vec<JoinHandle<_>> = (0..1_000)
        .map(|_| {
            tokio_uring::spawn(async {
                let buffer = vec![0; 10];
                let dev_urandom = File::open("/dev/urandom").await.unwrap();
                let (res, buffer) = dev_urandom.read_at(buffer, 0).await;
                let _read_n = res.unwrap();
                let _ = dev_urandom.close().await;
                let dev_null = File::open("/dev/null").await.unwrap();
                let _ = dev_null.write_at(buffer, 0).await;
                let _ = dev_null.close().await;
            })
        })
        .collect();
    for handle in handles {
        handle.await.unwrap();
    }
}

fn main() {
    tokio_uring::start(async {
        for _ in 0..10 {
            compute_tokio_uring().await;
        }
        for _ in 0..10 {
            let before = std::time::Instant::now();
            for _ in 0..1_000 {
                compute_tokio_uring().await;
            }
            let elapsed = before.elapsed();
            println!(
                "[tokio_uring] {:?} total, {:?} avg per iteration",
                elapsed,
                elapsed / 1_000
            );
        }
    })
}

Typical output:

[tokio_uring] 3.140604502s total, 3.140604ms avg per iteration
[tokio_uring] 3.138823337s total, 3.138823ms avg per iteration
[tokio_uring] 3.182483812s total, 3.182483ms avg per iteration
thread 'main' panicked at 'already borrowed: BorrowMutError', /home/mcs/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-uring-0.1.0/src/driver/op.rs:151:37
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread 'main' panicked at 'already borrowed: BorrowMutError', /home/mcs/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-uring-0.1.0/src/driver/op.rs:151:37
stack backtrace:
   0:     0x560a89614b66 - std::backtrace_rs::backtrace::libunwind::trace::h0f5cd2ee8b0d7274
                               at /rustc/32c9b7b091534f6d80e7e85da0cd425acb6c9a79/library/std/src/../../backtrace/src/backtrace/libunwind.rs:90:5
   1:     0x560a89614b66 - std::backtrace_rs::backtrace::trace_unsynchronized::h06905b5aeda069a1
                               at /rustc/32c9b7b091534f6d80e7e85da0cd425acb6c9a79/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
   2:     0x560a89614b66 - std::sys_common::backtrace::_print_fmt::h4fe4c7c875072f30
                               at /rustc/32c9b7b091534f6d80e7e85da0cd425acb6c9a79/library/std/src/sys_common/backtrace.rs:67:5
   3:     0x560a89614b66 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::hcc0746f004a9b7ef
                               at /rustc/32c9b7b091534f6d80e7e85da0cd425acb6c9a79/library/std/src/sys_common/backtrace.rs:46:22
   4:     0x560a895fdcec - core::fmt::write::h9a6d9c74526a6c1b
                               at /rustc/32c9b7b091534f6d80e7e85da0cd425acb6c9a79/library/core/src/fmt/mod.rs:1115:17
   5:     0x560a89612972 - std::io::Write::write_fmt::h23dab4cc9ce72ee2
                               at /rustc/32c9b7b091534f6d80e7e85da0cd425acb6c9a79/library/std/src/io/mod.rs:1665:15
   6:     0x560a89613b58 - std::sys_common::backtrace::_print::h173dc702502d65d2
                               at /rustc/32c9b7b091534f6d80e7e85da0cd425acb6c9a79/library/std/src/sys_common/backtrace.rs:49:5
   7:     0x560a89613b58 - std::sys_common::backtrace::print::h61bd27c4742ba817
                               at /rustc/32c9b7b091534f6d80e7e85da0cd425acb6c9a79/library/std/src/sys_common/backtrace.rs:36:9
   8:     0x560a89613b58 - std::panicking::default_hook::{{closure}}::hcaae87f0495ae613
                               at /rustc/32c9b7b091534f6d80e7e85da0cd425acb6c9a79/library/std/src/panicking.rs:208:50
   9:     0x560a89612584 - std::panicking::default_hook::h0538e728ee080db0
                               at /rustc/32c9b7b091534f6d80e7e85da0cd425acb6c9a79/library/std/src/panicking.rs:225:9
  10:     0x560a89612584 - std::panicking::rust_panic_with_hook::h3039e236b6ca482c
                               at /rustc/32c9b7b091534f6d80e7e85da0cd425acb6c9a79/library/std/src/panicking.rs:622:17
  11:     0x560a8962d7f8 - std::panicking::begin_panic_handler::{{closure}}::h884fbab544ffd91c
                               at /rustc/32c9b7b091534f6d80e7e85da0cd425acb6c9a79/library/std/src/panicking.rs:519:13
  12:     0x560a8962d76e - std::sys_common::backtrace::__rust_end_short_backtrace::hdaf2e18ba3d91210
                               at /rustc/32c9b7b091534f6d80e7e85da0cd425acb6c9a79/library/std/src/sys_common/backtrace.rs:141:18
  13:     0x560a8962d71d - rust_begin_unwind
                               at /rustc/32c9b7b091534f6d80e7e85da0cd425acb6c9a79/library/std/src/panicking.rs:515:5
  14:     0x560a895fceb0 - core::panicking::panic_fmt::hcf5f6d96e1dd7099
                               at /rustc/32c9b7b091534f6d80e7e85da0cd425acb6c9a79/library/core/src/panicking.rs:92:14
  15:     0x560a895fe1e2 - core::result::unwrap_failed::he898b02f57993c42
                               at /rustc/32c9b7b091534f6d80e7e85da0cd425acb6c9a79/library/core/src/result.rs:1599:5
  16:     0x560a8964d1a7 - core::ptr::drop_in_place<tokio_uring::driver::op::Op<tokio_uring::driver::close::Close>>::hfcf27dfadca700b2
  17:     0x560a8964cdde - tokio_uring::driver::shared_fd::Inner::submit_close_op::h3f8bddef42430763
  18:     0x560a895ef99e - core::ptr::drop_in_place<tokio_uring::fs::file::File>::h362afd75267e2033
  19:     0x560a895f4a25 - tokio::runtime::task::raw::poll::h085f63b72b0ae836
  20:     0x560a895f8583 - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h3c0c4df751b39350
  21:     0x560a895fac6a - probing::main::h251d5f151cb99f6d
  22:     0x560a895ec1b3 - std::sys_common::backtrace::__rust_begin_short_backtrace::h7ba9bcd7fa98fc5f
  23:     0x560a895fc247 - main
  24:     0x7f946e667565 - __libc_start_main
  25:     0x560a895ec08e - _start
  26:                0x0 - <unknown>
thread panicked while panicking. aborting.

After panic while panicking such process hangs forever, even kill -9 cannot terminate it.

Sometimes the result is slightly different:

...
thread panicked while panicking. aborting.
Illegal instruction (core dumped)

Tested on the following systems:

Linux 5.11.0-25-generic #27-Ubuntu SMP x86_64 x86_64 x86_64 GNU/Linux
Linux 5.8.0-1038-aws #40~20.04.1-Ubuntu SMP Thu Jun 17 13:25:28 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Rust version

rustc 1.55.0-nightly (027187094 2021-07-22)

and also on the previous rust nightly (2021-07-21)

How many system calls happen underneath if I were to broadcast to bunch of connected clients of my server?

I am new to this library and I am trying to figure out how does this library help batching syscalls for network I/O. my use case is I am trying to use this library to broadcast a stream of messages m1,m2, m3....to all the connected sockets of my server. so if there are 1000 clients connected to my server I want to broadcast each message to all 1000 and I am trying to accomplish this is in as few syscalls as possible because called syscall.send on every socket descriptor seems to be the bottleneck of my server to scale. Latency is very important for my app.

Below are my questions/comments

does calling ring.write_at makes a syscall? I am assuming the answer is no since it returns a Future however I assume calling ring.write_at.await? will make a syscall. so if I need to batch I need to call ring.write_at for each individual message I want to broadcast and add then call ring.submit_all? but then what I do with each each individual future I get from calling ring.write_at?
How do I handle backpressure in my app if the submission queue is full? what I do? just keep retrying forever?

How to combine the multi-thread mode of Tokio

Tokio-uring uses a single thread. If I want to use hyper and tokio-uring to implement a file server service, how to combine tokio's multi-thread runtime to improve the overall QPS?

Multishot operations

io_uring now has multiple multishot operations, including IORING_OP_ACCEPT and IORING_OP_POLL_ADD. We should add support for streaming back completions for such operations.

Benchmark vs monoio?

https://github.com/bytedance/monoio/blob/master/docs/en/benchmark.md
This new framework seems to be disruptive.
Note however:

In the case of a single core and very few connections, Monoio's latency will be higher than Tokio, resulting in lower throughput than Tokio. This latency difference is due to the difference between io-uring and epoll.
So obviously the best of both worlds would be a meta-algorithm that dynamically (or at least once at runtime) determine whether to use epoll Vs io_uring (based e.g on the dynamic number of connections)

io_uring setup parameters

Currently, tokio_uring::start calls runtime::Runtime::new which calls the io_uring::IoUring::new with an entries count value of 256. But there are many settings besides the entries value that one may want to set when creating the kernel io_uring device.

The io_uring crate provides a builder API for setting those parameters that then get passed to the kernel's io_uring setup; but there is no API in tokio_uring to use that builder API for the parameters.

How should the tokio_uring support use of the io_uring builder api for setup parameters?

Can it reexport the io-uring Builder and pass that to another version of the tokio_uring::start (perhaps start_builder(b: io_uring::Builder)? Or should it make its own Builder that copies all the methods from the io-uring Builder, or should it provide a few mixes of parameter settings that are thought to be more safe or more tested or more useful than the rest?

possibly deferring work in the creation of an operation

Currently, some actual uring work is done when an Op<T> is created in the submit_with function, as seen here.

tokio-uring/src/driver/op.rs

Lines 61 to 91 in 913aa14

	pub(super) fn submit_with<F>(data: T, f: F) -> io::Result<Op<T>>
	where
	F: FnOnce(&mut T) -> squeue::Entry,
	{
	driver::CURRENT.with(|inner_rc| {
	let mut inner_ref = inner_rc.borrow_mut();
	let inner = &mut *inner_ref;
	// If the submission queue is full, flush it to the kernel
	if inner.uring.submission().is_full() {
	inner.submit()?;
	}
	// Create the operation
	let mut op = Op::new(data, inner, inner_rc);
	// Configure the SQE
	let sqe = f(op.data.as_mut().unwrap()).user_data(op.index as _);
	{
	let mut sq = inner.uring.submission();
	// Push the new operation
	if unsafe { sq.push(&sqe).is_err() } {
	unimplemented!("when is this hit?");
	}
	}
	Ok(op)
	})
	}

So a uring submission queue entry is created and added to the actual uring that is shared with the kernel when submit_with is called. It is only waiting for a response that is accomplished with an await on the returned result.

Given the discussion in #115, I wanted to ask if it is really desirable to do this work at creation time?

An alternative would be to return a Future that would first perform the submission and then await the response.

I ask because I can see the design working either way, and this seems as good a place as any to document why the crate design in this regards is what it is.

Having read through what the try_join! and select! macros are doing and seeing how a good crate will have documented what is cancel safe, this seemed like the right time to ask and get the reason down in writing.

I don't see how we can make the uring operations cancel safe anyway, operations where the kernel has begun reading data seem inherently not cancel safe if waiting for the result is canceled (in select! terminology). So I think I like the design the way it is, but wondering what others see are the tradeoffs.

Add `custom_flags` to OpenOptions

I'd like to open my device with custom_flags as our device requires O_NONBLOCK flag. Is it reasonable to add the field that user can touch it? (I wonder why it have not existed in the first place.)

If then, I will make a small PR to do.

Runtime will not start

I tried switching my application to using this lib for the primary reactor as shown in the example, but on startup it produces:

 thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 38, kind: Unsupported, message: "Function not implemented" }', /root/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-uring-0.1.0/src/lib.rs:131:42

Build target is MUSL, using clux/muslrust as a base container for the build.
Execution environment is Arch Linux, happens with liburing installed and without, liburing is:

Name            : liburing
Version         : 2.0-1
Description     : Linux-native io_uring I/O access library
Architecture    : x86_64
URL             : https://git.kernel.dk/cgit/liburing/
Licenses        : LGPL2.1  MIT
Groups          : None
Provides        : liburing.so=2-64
Depends On      : glibc
Optional Deps   : None
Required By     : None
Optional For    : None
Conflicts With  : None
Replaces        : None
Installed Size  : 74.58 KiB
Packager        : David Runge <[email protected]>
Build Date      : Thu 11 Mar 2021 09:08:44 PM UTC
Install Date    : Fri 20 Aug 2021 10:28:32 PM UTC
Install Reason  : Explicitly installed
Install Script  : No

rt: Hang on too small completion queue

Similar to #145, and again exercised by #144. If there size of the completion queue is smaller than the number of concurrent writers to the submission queue, the runtime just hangs.

Probably not commiting?

write_all

I had wanted a write_all for tokio_uring TcpStream. Locally I added it as a method but would you prefer a public trait like the AsyncWriteExt trait in tokio? I'm confused by the tokio std::io::Write trait also have a write_all method. So maybe a tokio_uring::io::Write trait is actually what is called for?

Or as a POC, is it acceptable to just add the write_all directly to the TcpStream and UnixStream?

Structured Concurrency Support

disclaimer: I am async noob, and I've just had my first morning cup of tea, so I maybe talking nonsense :-)

So, today I was writing some tokio code which looked like this:

pub async fn run_service(token: CancellationToken) -> Result<()> {
    let server = tokio::spawn(run_server(token.clone()));
    let client = tokio::spawn(run_client(token.clone()));
    server.await.unwrap()?;
    client.await.unwrap()?;
    Ok(())
}

async fn run_server(token: CancellationToken) -> Result<()> { ... }
async fn run_client(token: CancellationToken) -> Result<()> { ... }

Essentially, I am running two bits of concurrent work.

This code made me mildly uncomfortable -- if the server panics (or, if I add some other early-return logic), the client will be left running, violating the structured concurrency.

This isn't easy to fix: I can add let guard = token.clone().drop_guard(), but that requires client to cooperate and is not strictly structured. I can also add some code to call JoinHandle::abort, but that is annoying to write and is still not strictly structured. In general, I think it is well-understood that what I want to do here is impossible without some kind of language support for async drop.

But than it dawned on me that I can just do

pub async fn run_service(token: CancellationToken) -> Result<()> {
    let server = run_server(token.clone());
    let client = run_client(token.clone());
    let (server_res, client_res) = tokio::join!(server, client);
    server_res.or(client_res)
}

this is strictly structured and has almost exactly the semantics I want (the exact semantics would require select! shenanigans).

It's very curious that, what is impossible with spawn, falls out of join! naturally. I think that's just how the world works -- with spawn, the task we want to structurally cancel may be, at this moment, running on a different thread, so we physically can't preempt it. With join!, select!, we know that both "tasks" run on the same thread, so, if during execution of a task we decide that we need to kill another one, we can just do that, as we know that that other task isn't currently running.

Which finally brings me to this issue :) It seems to me that async structured concurrency is actually possible for single-threaded executors. As tokio-uring is aimed at thread-per-core architecture, it seems that it might give a shot at supporting structured concurrency as well!

cc tokio-rs/tokio#1879

possible Op poll simplification

Look at this simplication I found possible in the src/driver/read.rs

    pub(crate) async fn read(self) -> BufResult<usize, T> {
        let complete = self.await;

        // Convert the operation result to `usize`
        let res = complete.result.map(|v| v as usize);
        // Recover the buffer
        let mut buf = complete.data.buf;

        // If the operation was successful, advance the initialized cursor.
        if let Ok(n) = res {
            // Safety: the kernel wrote `n` bytes to the buffer.
            unsafe {
                buf.set_init(n);
            }
        }

        (res, buf)
    }

    /* old
    pub(crate) async fn read(mut self) -> BufResult<usize, T> {
        crate::future::poll_fn(move |cx| self.poll_i(cx)).await
    }

    fn poll_i(&mut self, cx: &mut Context<'_>) -> Poll<BufResult<usize, T>> {
        use std::future::Future;
        use std::pin::Pin;

        let complete = ready!(Pin::new(self).poll(cx));

        // Convert the operation result to `usize`
        let res = complete.result.map(|v| v as usize);
        // Recover the buffer
        let mut buf = complete.data.buf;

        // If the operation was successful, advance the initialized cursor.
        if let Ok(n) = res {
            // Safety: the kernel wrote `n` bytes to the buffer.
            unsafe {
                buf.set_init(n);
            }
        }

        Poll::Ready((res, buf))
    }
    */

It compiles and runs. Is fewer lines of code, one function instead of three, probably less cpu and less stack. Maybe less heap.

Any other driver operations that followed the earlier model might be easily swapped for this too.

Is anything important being lost with this?

Writting at the offset ignores the offset (when `.append(true)` is used)

Either I'm not understanding or something is off.

The documentation says:

Write a buffer into this file at the specified offset, returning how many bytes were written.

use std::time::Duration;

fn main() {
    tokio_uring::start(async {
        let mut buf = vec![3; 1];

        let file = tokio_uring::fs::OpenOptions::new()
            .append(true)
            .create(true)
            .open("file")
            .await
            .unwrap();
        loop {
            let (res, ret_buf) = file.write_at(buf, 1).await;
            buf = ret_buf;

            res.unwrap();

            tokio::time::sleep(Duration::from_secs(1)).await;
        }
    });
}

Since the offset given is fixed (1), then if write_at is supposed to write at the absolute offset, then one would expect this file to stay at 2 byte size, and third and further iterations would just keep overwriting the byte.

However, I see the file grow:

> hexdump -C file
00000000  03 03 03 03 03 03 03 03  03 03 03 03              |............|
0000000c

For reference:

> cat Cargo.toml 
[package]
name = "ioring-test"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
tokio-uring = "0.3.0"
tokio = { version = "1.19.2", features = ["macros", "rt", "rt-multi-thread", "signal", "io-util", "net", "sync", "time"] }

JoinHandle not completing when too many tasks or opened files present

Hi, probably I'm doing something wrong while trying the 0.3.0 tokio_uring.

I build a release from the small benchmark code sample in #35 and run it on a cloud Clear Linux OS with a kernel at 5.18.9 but only see it complete successfully when a maximum of 512 urandom tasks are spawned before awaiting each of their handles. When I run with the original 1000 tasks, usually the 513th handle.await.unwrap() doesn't return. The 'ulimit -n' had been 1024 so I changed that to 2048 and it may have made a small difference. Sometimes a few more task handles complete. And when I add print statements, the entire benchmark sometimes runs, sometimes still hangs.

I understand the task/code that is being benchmarked is almost throwaway code, there are no explicit checks for errors, it opens /dev/urandom, reads 10 bytes, closes it, opens /dev/null, writes the 10 bytes, and closes it. But I'm under the impression each task should still complete if any of the steps failed.

I don't think it's as relevant, but just in case, I'll add that I'm cross compiling from Darwin with the x86_64-unknown-linux-musl target and it is being run on a single vCPU machine running Clear Linux OS. And it's no surprise that the cpu is pegged at 100% utilization until the hang.

Is it obvious that I'm doing something wrong or is there an avenue I can follow to debug this further?

Here is the code again.

const HCOUNT : usize = 1_000;
// const HCOUNT : usize = 512;
// const HCOUNT : usize = 800;

const DEBUG : bool = false;

async fn compute_tokio_uring() {
    use tokio::task::JoinHandle;
    use tokio_uring::fs::File;

    let handles: Vec<JoinHandle<_>> = (0..HCOUNT)
        .map(|_| {
            tokio_uring::spawn(async {
                let buffer = vec![0; 10];
                let dev_urandom = File::open("/dev/urandom").await.unwrap();
                let (res, buffer) = dev_urandom.read_at(buffer, 0).await;
                let _read_n = res.unwrap();
                let _ = dev_urandom.close().await;
                let dev_null = File::open("/dev/null").await.unwrap();
                let _ = dev_null.write_at(buffer, 0).await;
                let _ = dev_null.close().await;
            })
        })
        .collect();
    if DEBUG {
        println!("A");
    }
    let mut count = 0;
    for handle in handles {
        if DEBUG {
            println!("B count {}", count);
        }
        count += 1;
        handle.await.unwrap();  // <--- This is the line that appears to hang, often on the 513th invocation.
    }
    if DEBUG {
        println!("C");
    }
}

fn main() {
    tokio_uring::start(async {
        for _ in 0..10 {
            compute_tokio_uring().await;
        }
        for _ in 0..3 {
            let before = std::time::Instant::now();
            for _ in 0..1_000 {
                compute_tokio_uring().await;
            }
            let elapsed = before.elapsed();
            println!(
                "[tokio_uring] {:?} total, {:?} avg per iteration",
                elapsed,
                elapsed / 1_000
            );
        }
    })
}

Writing to a file with Slice causes runtime panic

I'm still working on a minimal example that reproduces this issue, but I'm encountering a panic when trying to write to a file using a Slice<Vec<u8>> instead of a Vec<u8>:

// This is fine:
let body = req_buf[body_off..].to_vec(); // make a copy
// Panics if we instead use
// let body = req_buf.slice(body_off..);
match file.write_at(body, 0).await {

The stack trace is a doozie, but it seems like the key issue relates to the fact that Slice uses vec.capacity() to find the end bound.

thread 'main' panicked at 'range end index 4096 out of range for slice of length 176', library/core/src/slice/index.rs:73:5
stack backtrace:
   0: rust_begin_unwind
             at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/std/src/panicking.rs:584:5
   1: core::panicking::panic_fmt
             at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/core/src/panicking.rs:142:14
   2: core::slice::index::slice_end_index_len_fail_rt
             at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/core/src/slice/index.rs:73:5
   3: core::ops::function::FnOnce::call_once
             at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/core/src/ops/function.rs:248:5
   4: core::intrinsics::const_eval_select
             at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/core/src/intrinsics.rs:2370:5
   5: core::slice::index::slice_end_index_len_fail
             at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/core/src/slice/index.rs:67:9
   6: <core::ops::range::Range<usize> as core::slice::index::SliceIndex<[T]>>::index
             at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/core/src/slice/index.rs:304:13
   7: core::slice::index::<impl core::ops::index::Index<I> for [T]>::index
             at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/core/src/slice/index.rs:18:9
   8: <tokio_uring::buf::slice::Slice<T> as core::ops::deref::Deref>::deref
             at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-uring-0.3.0/src/buf/slice.rs:130:10
   9: <tokio_uring::buf::slice::Slice<T> as tokio_uring::buf::io_buf::IoBuf>::stable_ptr
             at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-uring-0.3.0/src/buf/slice.rs:142:9
  10: tokio_uring::driver::write::<impl tokio_uring::driver::op::Op<tokio_uring::driver::write::Write<T>>>::write_at::{{closure}}
             at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-uring-0.3.0/src/driver/write.rs:31:27
  11: tokio_uring::driver::op::Op<T>::submit_with::{{closure}}
             at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-uring-0.3.0/src/driver/op.rs:80:23
  12: scoped_tls::ScopedKey<T>::with
             at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/scoped-tls-1.0.0/src/lib.rs:171:13
  13: tokio_uring::driver::op::Op<T>::submit_with
             at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-uring-0.3.0/src/driver/op.rs:67:9
  14: tokio_uring::driver::write::<impl tokio_uring::driver::op::Op<tokio_uring::driver::write::Write<T>>>::write_at
             at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-uring-0.3.0/src/driver/write.rs:24:9
  15: tokio_uring::fs::file::File::write_at::{{closure}}
             at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-uring-0.3.0/src/fs/file.rs:221:18
  16: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
             at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/core/src/future/mod.rs:91:19
  17: elden::handle_request::{{closure}}
             at ./src/main.rs:75:41
  18: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
             at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/core/src/future/mod.rs:91:19
  19: elden::main::{{closure}}::{{closure}}
             at ./src/main.rs:118:46
  20: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
             at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/core/src/future/mod.rs:91:19
  21: tokio::runtime::task::core::CoreStage<T>::poll::{{closure}}
             at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/runtime/task/core.rs:161:17
  22: tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut
             at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/loom/std/unsafe_cell.rs:14:9
  23: tokio::runtime::task::core::CoreStage<T>::poll
             at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/runtime/task/core.rs:151:13
  24: tokio::runtime::task::harness::poll_future::{{closure}}
             at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/runtime/task/harness.rs:467:19
  25: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/core/src/panic/unwind_safe.rs:271:9
  26: std::panicking::try::do_call
             at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/std/src/panicking.rs:492:40
  27: __rust_try
  28: std::panicking::try
             at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/std/src/panicking.rs:456:19
  29: std::panic::catch_unwind
             at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/std/src/panic.rs:137:14
  30: tokio::runtime::task::harness::poll_future
             at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/runtime/task/harness.rs:455:18
  31: tokio::runtime::task::harness::Harness<T,S>::poll_inner
             at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/runtime/task/harness.rs:103:27
  32: tokio::runtime::task::harness::Harness<T,S>::poll
             at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/runtime/task/harness.rs:57:15
  33: tokio::runtime::task::raw::poll
             at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/runtime/task/raw.rs:128:5
  34: tokio::runtime::task::raw::RawTask::poll
             at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/runtime/task/raw.rs:80:18
  35: tokio::runtime::task::LocalNotified<S>::run
             at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/runtime/task/mod.rs:347:9
  36: tokio::task::local::LocalSet::tick::{{closure}}
             at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/task/local.rs:523:54
  37: tokio::coop::with_budget::{{closure}}
             at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/coop.rs:102:9
  38: std::thread::local::LocalKey<T>::try_with
             at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/std/src/thread/local.rs:442:16
  39: std::thread::local::LocalKey<T>::with
             at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/std/src/thread/local.rs:418:9
  40: tokio::coop::with_budget
             at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/coop.rs:95:5
  41: tokio::coop::budget
             at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/coop.rs:72:5
  42: tokio::task::local::LocalSet::tick
             at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/task/local.rs:523:31
  43: <tokio::task::local::RunUntil<T> as core::future::future::Future>::poll::{{closure}}
             at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/task/local.rs:648:16
  44: tokio::macros::scoped_tls::ScopedKey<T>::set
             at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/macros/scoped_tls.rs:61:9
  45: tokio::task::local::LocalSet::with
             at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/task/local.rs:561:9
  46: <tokio::task::local::RunUntil<T> as core::future::future::Future>::poll
             at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/task/local.rs:634:9
  47: tokio::task::local::LocalSet::run_until::{{closure}}
             at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/task/local.rs:507:18
  48: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
             at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/core/src/future/mod.rs:91:19
  49: <core::pin::Pin<P> as core::future::future::Future>::poll
             at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/core/src/future/future.rs:124:9
  50: tokio::runtime::basic_scheduler::CoreGuard::block_on::{{closure}}::{{closure}}::{{closure}}
             at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/runtime/basic_scheduler.rs:498:48
  51: tokio::coop::with_budget::{{closure}}
             at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/coop.rs:102:9
  52: std::thread::local::LocalKey<T>::try_with
             at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/std/src/thread/local.rs:442:16
  53: std::thread::local::LocalKey<T>::with
             at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/std/src/thread/local.rs:418:9
  54: tokio::coop::with_budget
             at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/coop.rs:95:5
  55: tokio::coop::budget
             at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/coop.rs:72:5
  56: tokio::runtime::basic_scheduler::CoreGuard::block_on::{{closure}}::{{closure}}
             at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/runtime/basic_scheduler.rs:498:25
  57: tokio::runtime::basic_scheduler::Context::enter
             at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/runtime/basic_scheduler.rs:356:19
  58: tokio::runtime::basic_scheduler::CoreGuard::block_on::{{closure}}
             at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/runtime/basic_scheduler.rs:497:36
  59: tokio::runtime::basic_scheduler::CoreGuard::enter::{{closure}}
             at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/runtime/basic_scheduler.rs:555:57
  60: tokio::macros::scoped_tls::ScopedKey<T>::set
             at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/macros/scoped_tls.rs:61:9
  61: tokio::runtime::basic_scheduler::CoreGuard::enter
             at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/runtime/basic_scheduler.rs:555:27
  62: tokio::runtime::basic_scheduler::CoreGuard::block_on
             at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/runtime/basic_scheduler.rs:488:9
  63: tokio::runtime::basic_scheduler::BasicScheduler::block_on
             at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/runtime/basic_scheduler.rs:168:24
  64: tokio::runtime::Runtime::block_on
             at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/runtime/mod.rs:475:46
  65: tokio_uring::runtime::Runtime::block_on::{{closure}}
             at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-uring-0.3.0/src/runtime.rs:84:13
  66: tokio_uring::driver::Driver::with::{{closure}}
             at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-uring-0.3.0/src/driver/mod.rs:75:37
  67: scoped_tls::ScopedKey<T>::set
             at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/scoped-tls-1.0.0/src/lib.rs:137:9
  68: tokio_uring::driver::Driver::with
             at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-uring-0.3.0/src/driver/mod.rs:75:9
  69: tokio_uring::runtime::Runtime::block_on
             at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-uring-0.3.0/src/runtime.rs:71:9
  70: tokio_uring::start
             at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-uring-0.3.0/src/lib.rs:144:5
  71: elden::main
             at ./src/main.rs:111:5
  72: core::ops::function::FnOnce::call_once
             at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/core/src/ops/function.rs:248:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
thread 'main' panicked at 'already borrowed: BorrowMutError', /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-uring-0.3.0/src/driver/op.rs:157:37
stack backtrace:
   0:     0xaaaaba0dca04 - std::backtrace_rs::backtrace::libunwind::trace::h5f4e5af8964d1816
                               at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/std/src/../../backtrace/src/backtrace/libunwind.rs:93:5
   1:     0xaaaaba0dca04 - std::backtrace_rs::backtrace::trace_unsynchronized::hff9fb2d12fae4f37
                               at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
   2:     0xaaaaba0dca04 - std::sys_common::backtrace::_print_fmt::hd445ffe454fd13c4
                               at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/std/src/sys_common/backtrace.rs:66:5
   3:     0xaaaaba0dca04 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h3eec229346eb0c4f
                               at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/std/src/sys_common/backtrace.rs:45:22
   4:     0xaaaaba0f515c - core::fmt::write::h6bdf752cee6f4182
                               at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/core/src/fmt/mod.rs:1194:17
   5:     0xaaaaba0d97f8 - std::io::Write::write_fmt::hc58904dd95d679cb
                               at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/std/src/io/mod.rs:1655:15
   6:     0xaaaaba0de0e8 - std::sys_common::backtrace::_print::h694fda7a19eb57c3
                               at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/std/src/sys_common/backtrace.rs:48:5
   7:     0xaaaaba0de0e8 - std::sys_common::backtrace::print::h6ad345ece32a144b
                               at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/std/src/sys_common/backtrace.rs:35:9
   8:     0xaaaaba0de0e8 - std::panicking::default_hook::{{closure}}::hec3010953ede1ef2
                               at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/std/src/panicking.rs:295:22
   9:     0xaaaaba0ddd4c - std::panicking::default_hook::h5a61c89d4eb19199
                               at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/std/src/panicking.rs:314:9
  10:     0xaaaaba0de63c - std::panicking::rust_panic_with_hook::hf1cb2b67c8bfb325
                               at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/std/src/panicking.rs:698:17
  11:     0xaaaaba0de51c - std::panicking::begin_panic_handler::{{closure}}::hd4eb83047b921aef
                               at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/std/src/panicking.rs:588:13
  12:     0xaaaaba0dceac - std::sys_common::backtrace::__rust_end_short_backtrace::h5bcb024f13df9013
                               at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/std/src/sys_common/backtrace.rs:138:18
  13:     0xaaaaba0de268 - rust_begin_unwind
                               at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/std/src/panicking.rs:584:5
  14:     0xaaaaba02c18c - core::panicking::panic_fmt::h196577eb09dbd8c1
                               at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/core/src/panicking.rs:142:14
  15:     0xaaaaba02c1e8 - core::result::unwrap_failed::h014e56b1e5c77c78
                               at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/core/src/result.rs:1785:5
  16:     0xaaaaba05fee0 - core::result::Result<T,E>::expect::h34ca48b878241bed
                               at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/core/src/result.rs:1035:23
  17:     0xaaaaba065c14 - core::cell::RefCell<T>::borrow_mut::hd6f443ed64720939
                               at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/core/src/cell.rs:947:9
  18:     0xaaaaba045f14 - <tokio_uring::driver::op::Op<T> as core::ops::drop::Drop>::drop::hdd70cdf72f8c9cae
                               at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-uring-0.3.0/src/driver/op.rs:157:25
  19:     0xaaaaba043b18 - core::ptr::drop_in_place<tokio_uring::driver::op::Op<tokio_uring::driver::write::Write<tokio_uring::buf::slice::Slice<alloc::vec::Vec<u8>>>>>::h488eb886897d8f6a
                               at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/core/src/ptr/mod.rs:486:1
  20:     0xaaaaba02cde8 - tokio_uring::driver::op::Op<T>::submit_with::{{closure}}::h0f2fc61dfff43ab8
                               at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-uring-0.3.0/src/driver/op.rs:98:9
  21:     0xaaaaba04a750 - scoped_tls::ScopedKey<T>::with::h854276079b6938c7
                               at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/scoped-tls-1.0.0/src/lib.rs:171:13
  22:     0xaaaaba02ca7c - tokio_uring::driver::op::Op<T>::submit_with::h3f0854acde00da52
                               at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-uring-0.3.0/src/driver/op.rs:67:9
  23:     0xaaaaba02e304 - tokio_uring::driver::write::<impl tokio_uring::driver::op::Op<tokio_uring::driver::write::Write<T>>>::write_at::h67f4e65f03596e7e
                               at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-uring-0.3.0/src/driver/write.rs:24:9
  24:     0xaaaaba03248c - tokio_uring::fs::file::File::write_at::{{closure}}::h8ac5b56696d98145
                               at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-uring-0.3.0/src/fs/file.rs:221:18
  25:     0xaaaaba0353a0 - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h868b9db0e52fc4b5
                               at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/core/src/future/mod.rs:91:19
  26:     0xaaaaba03a120 - elden::handle_request::{{closure}}::he3d4f0e44424bdb4
                               at /home/meyer/elden/src/main.rs:75:41
  27:     0xaaaaba0356d0 - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::ha8c4e7921f7303d7
                               at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/core/src/future/mod.rs:91:19
  28:     0xaaaaba03b008 - elden::main::{{closure}}::{{closure}}::h58522f913d6b51c0
                               at /home/meyer/elden/src/main.rs:118:46
  29:     0xaaaaba035588 - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h9ba0cedc3ce0652e
                               at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/core/src/future/mod.rs:91:19
  30:     0xaaaaba04ea1c - tokio::runtime::task::core::CoreStage<T>::poll::{{closure}}::h1c164cc1ae3baccd
                               at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/runtime/task/core.rs:161:17
  31:     0xaaaaba0465f8 - tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut::h89f1f31225a9d4c5
                               at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/loom/std/unsafe_cell.rs:14:9
  32:     0xaaaaba04e930 - tokio::runtime::task::core::CoreStage<T>::poll::h3a4a2c01486a7116
                               at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/runtime/task/core.rs:151:13
  33:     0xaaaaba048704 - tokio::runtime::task::harness::poll_future::{{closure}}::ha910796037831a6e
                               at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/runtime/task/harness.rs:467:19
  34:     0xaaaaba040c8c - <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::he8614346735a93f4
                               at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/core/src/panic/unwind_safe.rs:271:9
  35:     0xaaaaba041324 - std::panicking::try::do_call::h4ec33e1324faaf7f
                               at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/std/src/panicking.rs:492:40
  36:     0xaaaaba041790 - __rust_try
  37:     0xaaaaba04113c - std::panicking::try::h90002e9a624813d2
                               at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/std/src/panicking.rs:456:19
  38:     0xaaaaba049468 - std::panic::catch_unwind::h006cf8aa6129ad4c
                               at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/std/src/panic.rs:137:14
  39:     0xaaaaba048574 - tokio::runtime::task::harness::poll_future::h9143f15639fc8a29
                               at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/runtime/task/harness.rs:455:18
  40:     0xaaaaba0488d0 - tokio::runtime::task::harness::Harness<T,S>::poll_inner::h1f89e5c4efce7199
                               at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/runtime/task/harness.rs:103:27
  41:     0xaaaaba048e60 - tokio::runtime::task::harness::Harness<T,S>::poll::h0f6d055afe6239c6
                               at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/runtime/task/harness.rs:57:15
  42:     0xaaaaba04eca0 - tokio::runtime::task::raw::poll::h1dfb9c2308a352ae
                               at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/runtime/task/raw.rs:128:5
  43:     0xaaaaba09f4c4 - tokio::runtime::task::raw::RawTask::poll::h9d87190112677124
                               at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/runtime/task/raw.rs:80:18
  44:     0xaaaaba09be5c - tokio::runtime::task::LocalNotified<S>::run::h483750eb909e357f
                               at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/runtime/task/mod.rs:347:9
  45:     0xaaaaba09c418 - tokio::task::local::LocalSet::tick::{{closure}}::h315a4ea404498395
                               at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/task/local.rs:523:54
  46:     0xaaaaba06f16c - tokio::coop::with_budget::{{closure}}::h08f6ff761fd6268c
                               at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/coop.rs:102:9
  47:     0xaaaaba085b4c - std::thread::local::LocalKey<T>::try_with::hc9f79b73add73096
                               at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/std/src/thread/local.rs:442:16
  48:     0xaaaaba084778 - std::thread::local::LocalKey<T>::with::h488dbba5bda5853f
                               at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/std/src/thread/local.rs:418:9
  49:     0xaaaaba09c3c0 - tokio::coop::with_budget::h97e45ba93975be15
                               at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/coop.rs:95:5
  50:     0xaaaaba09c3c0 - tokio::coop::budget::hda38821cedc52005
                               at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/coop.rs:72:5
  51:     0xaaaaba09c3c0 - tokio::task::local::LocalSet::tick::h59730777a25748f7
                               at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/task/local.rs:523:31
  52:     0xaaaaba04b5a8 - <tokio::task::local::RunUntil<T> as core::future::future::Future>::poll::{{closure}}::h8ababc96ffc58823
                               at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/task/local.rs:648:16
  53:     0xaaaaba03db64 - tokio::macros::scoped_tls::ScopedKey<T>::set::h78f32aeeeec746d4
                               at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/macros/scoped_tls.rs:61:9
  54:     0xaaaaba04b0f4 - tokio::task::local::LocalSet::with::hd350aa2c9a4c20b9
                               at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/task/local.rs:561:9
  55:     0xaaaaba04b3c4 - <tokio::task::local::RunUntil<T> as core::future::future::Future>::poll::h35b08a3f0e3cee6c
                               at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/task/local.rs:634:9
  56:     0xaaaaba04b2e8 - tokio::task::local::LocalSet::run_until::{{closure}}::hc14315eee29c543c
                               at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/task/local.rs:507:18
  57:     0xaaaaba0352e8 - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h83225e0ba0f6198d
                               at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/core/src/future/mod.rs:91:19
  58:     0xaaaaba03f9c4 - <core::pin::Pin<P> as core::future::future::Future>::poll::h18c41c26b19e6d10
                               at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/core/src/future/future.rs:124:9
  59:     0xaaaaba04e344 - tokio::runtime::basic_scheduler::CoreGuard::block_on::{{closure}}::{{closure}}::{{closure}}::h18d11ba520667d7c
                               at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/runtime/basic_scheduler.rs:498:48
  60:     0xaaaaba04b81c - tokio::coop::with_budget::{{closure}}::h2ccab5fff0f848bc
                               at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/coop.rs:102:9
  61:     0xaaaaba030eec - std::thread::local::LocalKey<T>::try_with::h5e050bed0b7b98e7
                               at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/std/src/thread/local.rs:442:16
  62:     0xaaaaba030a7c - std::thread::local::LocalKey<T>::with::hfd70af18e6cf595c
                               at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/std/src/thread/local.rs:418:9
  63:     0xaaaaba04e2ac - tokio::coop::with_budget::h2dce3d6f117eff73
                               at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/coop.rs:95:5
  64:     0xaaaaba04e2ac - tokio::coop::budget::h0c83f89bf7fceb3f
                               at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/coop.rs:72:5
  65:     0xaaaaba04e2ac - tokio::runtime::basic_scheduler::CoreGuard::block_on::{{closure}}::{{closure}}::hce112ff72ffb511c
                               at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/runtime/basic_scheduler.rs:498:25
  66:     0xaaaaba04d3cc - tokio::runtime::basic_scheduler::Context::enter::h3db1e8ea30676ea2
                               at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/runtime/basic_scheduler.rs:356:19
  67:     0xaaaaba04ddc4 - tokio::runtime::basic_scheduler::CoreGuard::block_on::{{closure}}::h9fb0cad747f1fa23
                               at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/runtime/basic_scheduler.rs:497:36
  68:     0xaaaaba04dbd0 - tokio::runtime::basic_scheduler::CoreGuard::enter::{{closure}}::h3231df3dff1402df
                               at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/runtime/basic_scheduler.rs:555:57
  69:     0xaaaaba03da68 - tokio::macros::scoped_tls::ScopedKey<T>::set::h405d2ce31df10aac
                               at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/macros/scoped_tls.rs:61:9
  70:     0xaaaaba04da10 - tokio::runtime::basic_scheduler::CoreGuard::enter::h2227ebfa54b17793
                               at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/runtime/basic_scheduler.rs:555:27
  71:     0xaaaaba04dc14 - tokio::runtime::basic_scheduler::CoreGuard::block_on::h1d49f6f9d2da2838
                               at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/runtime/basic_scheduler.rs:488:9
  72:     0xaaaaba04ce78 - tokio::runtime::basic_scheduler::BasicScheduler::block_on::hfe940522e42feadd
                               at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/runtime/basic_scheduler.rs:168:24
  73:     0xaaaaba03e548 - tokio::runtime::Runtime::block_on::hc63a354dd4b15f3b
                               at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/runtime/mod.rs:475:46
  74:     0xaaaaba036540 - tokio_uring::runtime::Runtime::block_on::{{closure}}::h2aa8eb5cfc6005e1
                               at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-uring-0.3.0/src/runtime.rs:84:13
  75:     0xaaaaba03e290 - tokio_uring::driver::Driver::with::{{closure}}::h7388b9718f1cda07
                               at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-uring-0.3.0/src/driver/mod.rs:75:37
  76:     0xaaaaba04a514 - scoped_tls::ScopedKey<T>::set::hd86caf6295d12faa
                               at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/scoped-tls-1.0.0/src/lib.rs:137:9
  77:     0xaaaaba03e248 - tokio_uring::driver::Driver::with::h32a29a63dacdaae1
                               at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-uring-0.3.0/src/driver/mod.rs:75:9
  78:     0xaaaaba036380 - tokio_uring::runtime::Runtime::block_on::haf915596c76351d5
                               at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-uring-0.3.0/src/runtime.rs:71:9
  79:     0xaaaaba02c494 - tokio_uring::start::h0ec08ab939f2face
                               at /home/meyer/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-uring-0.3.0/src/lib.rs:144:5
  80:     0xaaaaba02c6b8 - elden::main::h18da81d24a9bfb9d
                               at /home/meyer/elden/src/main.rs:111:5
  81:     0xaaaaba042918 - core::ops::function::FnOnce::call_once::h3b14204d8a168b97
                               at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/core/src/ops/function.rs:248:5
  82:     0xaaaaba040de0 - std::sys_common::backtrace::__rust_begin_short_backtrace::hc0b7e4cc305566ab
                               at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/std/src/sys_common/backtrace.rs:122:18
  83:     0xaaaaba040d8c - std::rt::lang_start::{{closure}}::hd2d85a5b1a6324f5
                               at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/std/src/rt.rs:145:18
  84:     0xaaaaba0d662c - core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once::h51c034c845a79965
                               at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/core/src/ops/function.rs:280:13
  85:     0xaaaaba0d662c - std::panicking::try::do_call::h9a1efe426c546c17
                               at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/std/src/panicking.rs:492:40
  86:     0xaaaaba0d662c - std::panicking::try::hbe33f2675417f548
                               at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/std/src/panicking.rs:456:19
  87:     0xaaaaba0d662c - std::panic::catch_unwind::h79cbdc4f7dc37def
                               at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/std/src/panic.rs:137:14
  88:     0xaaaaba0d662c - std::rt::lang_start_internal::{{closure}}::h424483edd5e388c2
                               at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/std/src/rt.rs:128:48
  89:     0xaaaaba0d662c - std::panicking::try::do_call::h38f7685cbf617483
                               at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/std/src/panicking.rs:492:40
  90:     0xaaaaba0d662c - std::panicking::try::h813d7d482369b9f9
                               at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/std/src/panicking.rs:456:19
  91:     0xaaaaba0d662c - std::panic::catch_unwind::h124e83f7877d507e
                               at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/std/src/panic.rs:137:14
  92:     0xaaaaba0d662c - std::rt::lang_start_internal::h2ec8eb82ad9931b1
                               at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/std/src/rt.rs:128:20
  93:     0xaaaaba040d58 - std::rt::lang_start::h2417bc69a8e1183f
                               at /rustc/e745b4ddbd05026c75aae4506aef39fdfe1603c5/library/std/src/rt.rs:144:17
  94:     0xaaaaba02c720 - main
  95:     0xffff980f73fc - __libc_start_call_main
                               at ./csu/../sysdeps/nptl/libc_start_call_main.h:58:16
  96:     0xffff980f74cc - __libc_start_main_impl
                               at ./csu/../csu/libc-start.c:392:3
  97:     0xaaaaba02c330 - _start
  98:                0x0 - <unknown>
thread panicked while panicking. aborting.
Aborted (core dumped)

Support for Linking Ops

Io_uring supports linking ops: that is, imposing order on the execution of Ops within the io_uring execution. The documentation here give a reasonable overview https://unixism.net/loti/tutorial/link_liburing.html

This issue exists to both track the support, and also as a place to discuss approaches to the implementation

TcpListener.accept is not cancellation safe, an open fd can be leaked

It took some finagling but I was able to create a scenario where the Future created by a TcpListener.accept call was partially polled, but not to completion. In select! parlance, a test I wrote shows the Future is not cancellation safe because the side effect was that file descriptors were being assigned by the kernel and never closed (until the process was terminated).

This is what I had expected but only after seeing the more obvious case of the multi accept Stream being dropped while the kernel races to create file descriptors that the user land will never get to recognize - they are put into the cq and read out as cqe's that were being ignored.

So some cleanup function should be definable for the singleshot accept case and the multishot accept case where the ignored lifetime variant still has a way to reclaim resources in a synchronous way, based on the type of operation that is now being ignored.

I was going to work on the multishot case for this fd leak but after realizing it can affect the singleshot case too, and getting an example binary that can reproduce the singleshot case, planning a solution first for the more general singleshot case seems prudent.

Impl `IoBuf`/`IoBufMut` for `bytes::Bytes`/`BytesMut`

It will be very useful if implement IoBuf/IoBufMut for bytes::Bytes/BytesMut.
bytes can be an optional dependency.

explicit_close is a flaky test

thread 'explicit_close' panicked at 'Ok(14)', tests/fs_file.rs:152:16

I wonder why

File should implement Debug

Currently File does not implement the debug trait. This is annoying when attempting to automatically derive debug for structs containing files.

Generialise Buffer Types

Using io_uring requires ownership of the backing buffers for the duration of a Operation lifetime. This is currently enforced using the traits IoBuf and IoBufMut. This issue is suggesting dropping IoBuf and IoBufMut in favour of Pin. In conjunction with a slightly modified Slice struct, I believe Pin provides all the invariant we currently have, whilst allowing us to generalize over any input type (without newtyping everything)

I'd like to illustrate my proposal by considering the Op

// current
impl<T: IoBuf> Op<Write<T>> {
    pub(crate) fn write_at(fd: &SharedFd, buf:T, offset: u64) -> io::Result<Op<Write<T>>> {
        // ...
    }
}

// Pr #53
impl<T: IoBuf> Op<Write<T>> {
    pub(crate) fn write_at(fd: &SharedFd, buf: Slice<T>, offset: u64) -> io::Result<Op<Write<T>>> {
        // ...
    }
}

// Proposed
impl<T: Deref<Target = [u8]>> Op<Write<T>> {
    pub(crate) fn write_at(fd: &SharedFd, buf: Pin<Slice<T>>, offset: u64) -> io::Result<Op<Write<T>>> {
    }
}

A PR will be along shortly to give a concrete reference point, but I wanted to put this out there for discussion

Smell of Bug in File::read_at(...)

When performing this function, the result is an infinite loop, But I read in the document it said: read_at result returns number of bytes read But here it always returns 100

where is problem ??

 pub async fn fetch(&mut self) -> Result<Vec<String>, ()> {
            let mut queries = Vec::new();
            loop {
                let buf = vec![0; 100];
/*---> */    let (res, buf) = self.file.read_at(buf, 0).await;
                match res {
                    Err(_) => return Err(()),
/*---> */         Ok(n) => {
                        println!("==> {}", n);
                        if n == 0 {
                            break
                        }  

                        let string = unsafe {
                            String::from_utf8_unchecked(buf)
                        };
                        queries.push(string);
                                         
                    }
                };
            };
            
            Ok(queries)
        }

How is wrk-bench example meant to be exercised?

https://github.com/tokio-rs/tokio-uring/blob/master/examples/wrk-bench.rs is the program for running a TCP server that is running 16 accepts on the same socket. How is it meant to be exercised from a client? Is there a program for already doing this?

Thanks.

Sample not available

OS:
Ubuntu 20.04.2 LTS (Focal Fossa)
Linux pktserver 5.4.0-80-generic #90-Ubuntu SMP Fri Jul 9 22:49:44 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

dependencies：
tokio-uring = "0.1.0"

bug: Calling `block_on` twice causes a panic

See PR #159 for reproduction. I believe this was introduced bus was introduced in #142. It became possible to exercise it because of #148

net: server side panic after client send data in a timeouted connection

Hi, I'm trying to implement a "read TCP connection with timeout" function, so I want to make read() future returns immediately after the client hasn't sent data in 10s.

I found tokio::time::timeout should be useful and read-future can timeout and return with it, but it's strange the server-side should meet panic if the client sends more data.

reproduce step:

1. run server-side with https://github.com/zojw/tokio-uring/blob/reproduce_panic/examples/tcp_listener.rs
2. run nc 0.0.0.0 [port] to connect server
3. after 10s, the server-side should output "wait read timeout, and close connection???"
4. then input some words in nc terminal and enter

what expect to see

nc report error about connection's closed

what did we see instead

server-side panic with thread 'main' panicked at 'already borrowed: BorrowMutError', src/driver/op.rs:68:42

with stack:

rust_panic (@rust_panic:7)
rust_panic_with_hook (@std::panicking::rust_panic_with_hook::h3cf1e389ae2148cd:129)
{closure#0} (@std::panicking::begin_panic_handler::_$u7b$$u7b$closure$u7d$$u7d$::h0a5f32b79c6e3a95:52)
__rust_end_short_backtrace<std::panicking::begin_panic_handler::{closure_env#0}, !> (@std::sys_common::backtrace::__rust_end_short_backtrace::h6d640bca5165509c:9)
begin_panic_handler (@rust_begin_unwind:21)
panic_fmt (@core::panicking::panic_fmt::h1c34501c4b8544a1:13)
unwrap_failed (@core::result::unwrap_failed::ha9974bfbcd81656a:26)
expect<core::cell::RefMut<tokio_uring::driver::Inner>, core::cell::BorrowMutError> (@core::result::Result$LT$T$C$E$GT$::expect::hf295eccdaa9ab954:32)
borrow_mut<tokio_uring::driver::Inner> (@core::cell::RefCell$LT$T$GT$::borrow_mut::h5be30d736c2998d7:15)
{closure#0}<tokio_uring::driver::close::Close, tokio_uring::driver::close::{impl#0}::close::{closure_env#0}> (/home/robi/Code/rust/tokio-uring/src/driver/op.rs:68)
with<alloc::rc::Rc<core::cell::RefCell<tokio_uring::driver::Inner>>, tokio_uring::driver::op::{impl#0}::submit_with::{closure_env#0}<tokio_uring::driver::close::Close, tokio_uring::driver::close::{impl#0}::close::{closure_env#0}>, core::result::Result<tokio_uring::driver::op::Op<tokio_uring::driver::close::Close>, std::io::error::Error>> (/home/robi/.cargo/registry/src/mirrors.tuna.tsinghua.edu.cn-df7c3c540f42cdbd/scoped-tls-1.0.0/src/lib.rs:171)
submit_with<tokio_uring::driver::close::Close, tokio_uring::driver::close::{impl#0}::close::{closure_env#0}> (/home/robi/Code/rust/tokio-uring/src/driver/op.rs:67)
try_submit_with<tokio_uring::driver::close::Close, tokio_uring::driver::close::{impl#0}::close::{closure_env#0}> (/home/robi/Code/rust/tokio-uring/src/driver/op.rs:101)
close (/home/robi/Code/rust/tokio-uring/src/driver/close.rs:14)
submit_close_op (/home/robi/Code/rust/tokio-uring/src/driver/shared_fd.rs:77)
drop (/home/robi/Code/rust/tokio-uring/src/driver/shared_fd.rs:140)
drop_in_place<tokio_uring::driver::shared_fd::Inner> (@core::ptr::drop_in_place$LT$tokio_uring..driver..shared_fd..Inner$GT$::hb8a14c69a05e7e33:8)
drop<tokio_uring::driver::shared_fd::Inner> (@_$LT$alloc..rc..Rc$LT$T$GT$$u20$as$u20$core..ops..drop..Drop$GT$::drop::hb002e9fd80b7c14d:30)
drop_in_place<alloc::rc::Rc<tokio_uring::driver::shared_fd::Inner>> (@core::ptr::drop_in_place$LT$alloc..rc..Rc$LT$tokio_uring..driver..shared_fd..Inner$GT$$GT$::h222ed3da7126a159:6)
drop_in_place<tokio_uring::driver::shared_fd::SharedFd> (@core::ptr::drop_in_place$LT$tokio_uring..driver..shared_fd..SharedFd$GT$::hfccfc25a4a14ce2c:6)
drop_in_place<tokio_uring::driver::read::Read<alloc::vec::Vec<u8, alloc::alloc::Global>>> (@core::ptr::drop_in_place$LT$tokio_uring..driver..read..Read$LT$alloc..vec..Vec$LT$u8$GT$$GT$$GT$::h16b1649dabfe6007:8)
drop_in_place<core::option::Option<tokio_uring::driver::read::Read<alloc::vec::Vec<u8, alloc::alloc::Global>>>> (@core::ptr::drop_in_place$LT$core..option..Option$LT$tokio_uring..driver..read..Read$LT$alloc..vec..Vec$LT$u8$GT$$GT$$GT$$GT$::h9ff4ad4f264b10be:16)
drop_in_place<alloc::boxed::Box<dyn core::any::Any, alloc::alloc::Global>> (@core::ptr::drop_in_place$LT$alloc..boxed..Box$LT$dyn$u20$core..any..Any$GT$$GT$::h1aa706a56d43585d:11)
drop_in_place<tokio_uring::driver::op::Lifecycle> (@core::ptr::drop_in_place$LT$tokio_uring..driver..op..Lifecycle$GT$::h56bb8e50d8665171:31)
complete (/home/robi/Code/rust/tokio-uring/src/driver/op.rs:186)
complete (/home/robi/Code/rust/tokio-uring/src/driver/mod.rs:170)
tick (/home/robi/Code/rust/tokio-uring/src/driver/mod.rs:111)
tick (/home/robi/Code/rust/tokio-uring/src/driver/mod.rs:80)
{async_block#0}<core::future::from_generator::GenFuture<tcp_listener::main::{async_block_env#0}>> (/home/robi/Code/rust/tokio-uring/src/runtime.rs:82)
poll<tokio_uring::runtime::{impl#0}::block_on::{closure#0}::{async_block_env#0}<core::future::from_generator::GenFuture<tcp_listener::main::{async_block_env#0}>>> (@_$LT$core..future..from_generator..GenFuture$LT$T$GT$$u20$as$u20$core..future..future..Future$GT$::poll::h6ce2355388535dcc:21)
{closure#1}<core::future::from_generator::GenFuture<tcp_listener::main::{async_block_env#0}>> (/home/robi/Code/rust/tokio-uring/src/runtime.rs:92)
poll<(), tokio_uring::runtime::{impl#0}::block_on::{closure#0}::{closure_env#1}<core::future::from_generator::GenFuture<tcp_listener::main::{async_block_env#0}>>> (/home/robi/Code/rust/tokio-uring/src/future.rs:35)
{closure#0}<tokio_uring::future::PollFn<tokio_uring::runtime::{impl#0}::block_on::{closure#0}::{closure_env#1}<core::future::from_generator::GenFuture<tcp_listener::main::{async_block_env#0}>>>> (/home/robi/.cargo/registry/src/mirrors.tuna.tsinghua.edu.cn-df7c3c540f42cdbd/tokio-1.15.0/src/task/local.rs:642)
{closure#0}<core::task::poll::Poll<()>, tokio::task::local::{impl#5}::poll::{closure#0}::{closure_env#0}<tokio_uring::future::PollFn<tokio_uring::runtime::{impl#0}::block_on::{closure#0}::{closure_env#1}<core::future::from_generator::GenFuture<tcp_listener::main::{async_block_env#0}>>>>> (/home/robi/.cargo/registry/src/mirrors.tuna.tsinghua.edu.cn-df7c3c540f42cdbd/tokio-1.15.0/src/coop.rs:102)
try_with<core::cell::Cell<tokio::coop::Budget>, tokio::coop::with_budget::{closure_env#0}<core::task::poll::Poll<()>, tokio::task::local::{impl#5}::poll::{closure#0}::{closure_env#0}<tokio_uring::future::PollFn<tokio_uring::runtime::{impl#0}::block_on::{closure#0}::{closure_env#1}<core::future::from_generator::GenFuture<tcp_listener::main::{async_block_env#0}>>>>>, core::task::poll::Poll<()>> (@std::thread::local::LocalKey$LT$T$GT$::try_with::hfaec365b06919edb:55)
with<core::cell::Cell<tokio::coop::Budget>, tokio::coop::with_budget::{closure_env#0}<core::task::poll::Poll<()>, tokio::task::local::{impl#5}::poll::{closure#0}::{closure_env#0}<tokio_uring::future::PollFn<tokio_uring::runtime::{impl#0}::block_on::{closure#0}::{closure_env#1}<core::future::from_generator::GenFuture<tcp_listener::main::{async_block_env#0}>>>>>, core::task::poll::Poll<()>> (@std::thread::local::LocalKey$LT$T$GT$::with::hc75e59a32501a831:13)
tokio::coop::with_budget::h75dea0622c0d8d18 (/home/robi/.cargo/registry/src/mirrors.tuna.tsinghua.edu.cn-df7c3c540f42cdbd/tokio-1.15.0/src/coop.rs:95)
tokio::coop::budget::h1ee2fa0796c49b2a (/home/robi/.cargo/registry/src/mirrors.tuna.tsinghua.edu.cn-df7c3c540f42cdbd/tokio-1.15.0/src/coop.rs:72)
{closure#0}<tokio_uring::future::PollFn<tokio_uring::runtime::{impl#0}::block_on::{closure#0}::{closure_env#1}<core::future::from_generator::GenFuture<tcp_listener::main::{async_block_env#0}>>>> (/home/robi/.cargo/registry/src/mirrors.tuna.tsinghua.edu.cn-df7c3c540f42cdbd/tokio-1.15.0/src/task/local.rs:642)
set<tokio::task::local::Context, tokio::task::local::{impl#5}::poll::{closure_env#0}<tokio_uring::future::PollFn<tokio_uring::runtime::{impl#0}::block_on::{closure#0}::{closure_env#1}<core::future::from_generator::GenFuture<tcp_listener::main::{async_block_env#0}>>>>, core::task::poll::Poll<()>> (/home/robi/.cargo/registry/src/mirrors.tuna.tsinghua.edu.cn-df7c3c540f42cdbd/tokio-1.15.0/src/macros/scoped_tls.rs:61)
with<core::task::poll::Poll<()>, tokio::task::local::{impl#5}::poll::{closure_env#0}<tokio_uring::future::PollFn<tokio_uring::runtime::{impl#0}::block_on::{closure#0}::{closure_env#1}<core::future::from_generator::GenFuture<tcp_listener::main::{async_block_env#0}>>>>> (/home/robi/.cargo/registry/src/mirrors.tuna.tsinghua.edu.cn-df7c3c540f42cdbd/tokio-1.15.0/src/task/local.rs:559)
poll<tokio_uring::future::PollFn<tokio_uring::runtime::{impl#0}::block_on::{closure#0}::{closure_env#1}<core::future::from_generator::GenFuture<tcp_listener::main::{async_block_env#0}>>>> (/home/robi/.cargo/registry/src/mirrors.tuna.tsinghua.edu.cn-df7c3c540f42cdbd/tokio-1.15.0/src/task/local.rs:632)
{async_fn#0}<tokio_uring::future::PollFn<tokio_uring::runtime::{impl#0}::block_on::{closure#0}::{closure_env#1}<core::future::from_generator::GenFuture<tcp_listener::main::{async_block_env#0}>>>> (/home/robi/.cargo/registry/src/mirrors.tuna.tsinghua.edu.cn-df7c3c540f42cdbd/tokio-1.15.0/src/task/local.rs:505)
poll<tokio::task::local::{impl#0}::run_until::{async_fn_env#0}<tokio_uring::future::PollFn<tokio_uring::runtime::{impl#0}::block_on::{closure#0}::{closure_env#1}<core::future::from_generator::GenFuture<tcp_listener::main::{async_block_env#0}>>>>> (@_$LT$core..future..from_generator..GenFuture$LT$T$GT$$u20$as$u20$core..future..future..Future$GT$::poll::ha8e2c452ae0a69df:21)
poll<&mut core::future::from_generator::GenFuture<tokio::task::local::{impl#0}::run_until::{async_fn_env#0}<tokio_uring::future::PollFn<tokio_uring::runtime::{impl#0}::block_on::{closure#0}::{closure_env#1}<core::future::from_generator::GenFuture<tcp_listener::main::{async_block_env#0}>>>>>> (@_$LT$core..pin..Pin$LT$P$GT$$u20$as$u20$core..future..future..Future$GT$::poll::h12d9aa0d3dc38df1:18)
{closure#0}<tokio::runtime::driver::Driver, core::pin::Pin<&mut core::future::from_generator::GenFuture<tokio::task::local::{impl#0}::run_until::{async_fn_env#0}<tokio_uring::future::PollFn<tokio_uring::runtime::{impl#0}::block_on::{closure#0}::{closure_env#1}<core::future::from_generator::GenFuture<tcp_listener::main::{async_block_env#0}>>>>>>> (/home/robi/.cargo/registry/src/mirrors.tuna.tsinghua.edu.cn-df7c3c540f42cdbd/tokio-1.15.0/src/runtime/basic_scheduler.rs:240)
{closure#0}<core::task::poll::Poll<()>, tokio::runtime::basic_scheduler::{impl#2}::block_on::{closure#0}::{closure_env#0}<tokio::runtime::driver::Driver, core::pin::Pin<&mut core::future::from_generator::GenFuture<tokio::task::local::{impl#0}::run_until::{async_fn_env#0}<tokio_uring::future::PollFn<tokio_uring::runtime::{impl#0}::block_on::{closure#0}::{closure_env#1}<core::future::from_generator::GenFuture<tcp_listener::main::{async_block_env#0}>>>>>>>> (/home/robi/.cargo/registry/src/mirrors.tuna.tsinghua.edu.cn-df7c3c540f42cdbd/tokio-1.15.0/src/coop.rs:102)
try_with<core::cell::Cell<tokio::coop::Budget>, tokio::coop::with_budget::{closure_env#0}<core::task::poll::Poll<()>, tokio::runtime::basic_scheduler::{impl#2}::block_on::{closure#0}::{closure_env#0}<tokio::runtime::driver::Driver, core::pin::Pin<&mut core::future::from_generator::GenFuture<tokio::task::local::{impl#0}::run_until::{async_fn_env#0}<tokio_uring::future::PollFn<tokio_uring::runtime::{impl#0}::block_on::{closure#0}::{closure_env#1}<core::future::from_generator::GenFuture<tcp_listener::main::{async_block_env#0}>>>>>>>>, core::task::poll::Poll<()>> (@std::thread::local::LocalKey$LT$T$GT$::try_with::heb231fbe63433d36:55)
with<core::cell::Cell<tokio::coop::Budget>, tokio::coop::with_budget::{closure_env#0}<core::task::poll::Poll<()>, tokio::runtime::basic_scheduler::{impl#2}::block_on::{closure#0}::{closure_env#0}<tokio::runtime::driver::Driver, core::pin::Pin<&mut core::future::from_generator::GenFuture<tokio::task::local::{impl#0}::run_until::{async_fn_env#0}<tokio_uring::future::PollFn<tokio_uring::runtime::{impl#0}::block_on::{closure#0}::{closure_env#1}<core::future::from_generator::GenFuture<tcp_listener::main::{async_block_env#0}>>>>>>>>, core::task::poll::Poll<()>> (@std::thread::local::LocalKey$LT$T$GT$::with::hfb083861b8949336:13)
tokio::coop::with_budget::h10416856f6cf7971 (/home/robi/.cargo/registry/src/mirrors.tuna.tsinghua.edu.cn-df7c3c540f42cdbd/tokio-1.15.0/src/coop.rs:95)
tokio::coop::budget::hc890c64c8aa72bd1 (/home/robi/.cargo/registry/src/mirrors.tuna.tsinghua.edu.cn-df7c3c540f42cdbd/tokio-1.15.0/src/coop.rs:72)
{closure#0}<tokio::runtime::driver::Driver, core::pin::Pin<&mut core::future::from_generator::GenFuture<tokio::task::local::{impl#0}::run_until::{async_fn_env#0}<tokio_uring::future::PollFn<tokio_uring::runtime::{impl#0}::block_on::{closure#0}::{closure_env#1}<core::future::from_generator::GenFuture<tcp_listener::main::{async_block_env#0}>>>>>>> (/home/robi/.cargo/registry/src/mirrors.tuna.tsinghua.edu.cn-df7c3c540f42cdbd/tokio-1.15.0/src/runtime/basic_scheduler.rs:240)
{closure#0}<tokio::runtime::basic_scheduler::{impl#2}::block_on::{closure_env#0}<tokio::runtime::driver::Driver, core::pin::Pin<&mut core::future::from_generator::GenFuture<tokio::task::local::{impl#0}::run_until::{async_fn_env#0}<tokio_uring::future::PollFn<tokio_uring::runtime::{impl#0}::block_on::{closure#0}::{closure_env#1}<core::future::from_generator::GenFuture<tcp_listener::main::{async_block_env#0}>>>>>>>, (), tokio::runtime::driver::Driver> (/home/robi/.cargo/registry/src/mirrors.tuna.tsinghua.edu.cn-df7c3c540f42cdbd/tokio-1.15.0/src/runtime/basic_scheduler.rs:349)
set<tokio::runtime::basic_scheduler::Context, tokio::runtime::basic_scheduler::enter::{closure_env#0}<tokio::runtime::basic_scheduler::{impl#2}::block_on::{closure_env#0}<tokio::runtime::driver::Driver, core::pin::Pin<&mut core::future::from_generator::GenFuture<tokio::task::local::{impl#0}::run_until::{async_fn_env#0}<tokio_uring::future::PollFn<tokio_uring::runtime::{impl#0}::block_on::{closure#0}::{closure_env#1}<core::future::from_generator::GenFuture<tcp_listener::main::{async_block_env#0}>>>>>>>, (), tokio::runtime::driver::Driver>, ()> (/home/robi/.cargo/registry/src/mirrors.tuna.tsinghua.edu.cn-df7c3c540f42cdbd/tokio-1.15.0/src/macros/scoped_tls.rs:61)
enter<tokio::runtime::basic_scheduler::{impl#2}::block_on::{closure_env#0}<tokio::runtime::driver::Driver, core::pin::Pin<&mut core::future::from_generator::GenFuture<tokio::task::local::{impl#0}::run_until::{async_fn_env#0}<tokio_uring::future::PollFn<tokio_uring::runtime::{impl#0}::block_on::{closure#0}::{closure_env#1}<core::future::from_generator::GenFuture<tcp_listener::main::{async_block_env#0}>>>>>>>, (), tokio::runtime::driver::Driver> (/home/robi/.cargo/registry/src/mirrors.tuna.tsinghua.edu.cn-df7c3c540f42cdbd/tokio-1.15.0/src/runtime/basic_scheduler.rs:349)
block_on<tokio::runtime::driver::Driver, core::pin::Pin<&mut core::future::from_generator::GenFuture<tokio::task::local::{impl#0}::run_until::{async_fn_env#0}<tokio_uring::future::PollFn<tokio_uring::runtime::{impl#0}::block_on::{closure#0}::{closure_env#1}<core::future::from_generator::GenFuture<tcp_listener::main::{async_block_env#0}>>>>>>> (/home/robi/.cargo/registry/src/mirrors.tuna.tsinghua.edu.cn-df7c3c540f42cdbd/tokio-1.15.0/src/runtime/basic_scheduler.rs:230)
block_on<tokio::runtime::driver::Driver, core::pin::Pin<&mut core::future::from_generator::GenFuture<tokio::task::local::{impl#0}::run_until::{async_fn_env#0}<tokio_uring::future::PollFn<tokio_uring::runtime::{impl#0}::block_on::{closure#0}::{closure_env#1}<core::future::from_generator::GenFuture<tcp_listener::main::{async_block_env#0}>>>>>>> (/home/robi/.cargo/registry/src/mirrors.tuna.tsinghua.edu.cn-df7c3c540f42cdbd/tokio-1.15.0/src/runtime/basic_scheduler.rs:501)
block_on<tokio::runtime::driver::Driver, core::future::from_generator::GenFuture<tokio::task::local::{impl#0}::run_until::{async_fn_env#0}<tokio_uring::future::PollFn<tokio_uring::runtime::{impl#0}::block_on::{closure#0}::{closure_env#1}<core::future::from_generator::GenFuture<tcp_listener::main::{async_block_env#0}>>>>>> (/home/robi/.cargo/registry/src/mirrors.tuna.tsinghua.edu.cn-df7c3c540f42cdbd/tokio-1.15.0/src/runtime/basic_scheduler.rs:186)
block_on<core::future::from_generator::GenFuture<tokio::task::local::{impl#0}::run_until::{async_fn_env#0}<tokio_uring::future::PollFn<tokio_uring::runtime::{impl#0}::block_on::{closure#0}::{closure_env#1}<core::future::from_generator::GenFuture<tcp_listener::main::{async_block_env#0}>>>>>> (/home/robi/.cargo/registry/src/mirrors.tuna.tsinghua.edu.cn-df7c3c540f42cdbd/tokio-1.15.0/src/runtime/mod.rs:461)
{closure#0}<core::future::from_generator::GenFuture<tcp_listener::main::{async_block_env#0}>> (/home/robi/Code/rust/tokio-uring/src/runtime.rs:90)
{closure#0}<(), tokio_uring::runtime::{impl#0}::block_on::{closure_env#0}<core::future::from_generator::GenFuture<tcp_listener::main::{async_block_env#0}>>> (/home/robi/Code/rust/tokio-uring/src/driver/mod.rs:75)
set<alloc::rc::Rc<core::cell::RefCell<tokio_uring::driver::Inner>>, tokio_uring::driver::{impl#0}::with::{closure_env#0}<(), tokio_uring::runtime::{impl#0}::block_on::{closure_env#0}<core::future::from_generator::GenFuture<tcp_listener::main::{async_block_env#0}>>>, ()> (/home/robi/.cargo/registry/src/mirrors.tuna.tsinghua.edu.cn-df7c3c540f42cdbd/scoped-tls-1.0.0/src/lib.rs:137)
with<(), tokio_uring::runtime::{impl#0}::block_on::{closure_env#0}<core::future::from_generator::GenFuture<tcp_listener::main::{async_block_env#0}>>> (/home/robi/Code/rust/tokio-uring/src/driver/mod.rs:75)
block_on<core::future::from_generator::GenFuture<tcp_listener::main::{async_block_env#0}>> (/home/robi/Code/rust/tokio-uring/src/runtime.rs:77)
start<core::future::from_generator::GenFuture<tcp_listener::main::{async_block_env#0}>> (/home/robi/Code/rust/tokio-uring/src/lib.rs:144)
main (/home/robi/Code/rust/tokio-uring/examples/tcp_listener.rs:14)
call_once<fn(), ()> (@core::ops::function::FnOnce::call_once::h0bdead0eeb99adf8:6)
__rust_begin_short_backtrace<fn(), ()> (@std::sys_common::backtrace::__rust_begin_short_backtrace::hd1d69321906bc1f5:6)
{closure#0}<()> (@std::rt::lang_start::_$u7b$$u7b$closure$u7d$$u7d$::h5cc8ca0dca6d7b4a:7)
core::ops::function::impls::_$LT$impl$u20$core..ops..function..FnOnce$LT$A$GT$$u20$for$u20$$RF$F$GT$::call_once::hb0de949e057e3b17 (@std::rt::lang_start_internal::he0f2542e47c09e70:172)
std::panicking::try::do_call::hcf3b1fbd7249caa5 (@std::rt::lang_start_internal::he0f2542e47c09e70:170)
std::panicking::try::h3f0d88b1713d48ca (@std::rt::lang_start_internal::he0f2542e47c09e70:170)
std::panic::catch_unwind::hf9ae408bcd761f44 (@std::rt::lang_start_internal::he0f2542e47c09e70:170)
std::rt::lang_start_internal::_$u7b$$u7b$closure$u7d$$u7d$::hd83541db0ecf9e5b (@std::rt::lang_start_internal::he0f2542e47c09e70:170)
std::panicking::try::do_call::h9d93145031632300 (@std::rt::lang_start_internal::he0f2542e47c09e70:170)
std::panicking::try::h5c1ae104f0d6f1fd (@std::rt::lang_start_internal::he0f2542e47c09e70:170)
std::panic::catch_unwind::h932b68c0b8fd6df3 (@std::rt::lang_start_internal::he0f2542e47c09e70:170)
lang_start_internal (@std::rt::lang_start_internal::he0f2542e47c09e70:170)
lang_start<()> (@std::rt::lang_start::h328b1a37be837683:13)
main (@main:10)
__libc_start_main (@__libc_start_main:64)
_start (@_start:15)

Export the Runtime type

I want to write a library with multi-thread compatible async APIs. So I can't use tokio_uring::start internally or let the library user call it. My idea is using a thread local variable to store and get the runtime per thread. Export the tokio_uring::Runtime will be useful for me.

Impl `File::read_exact_at` and `File::write_all_at`

Same as std::io::Read::read_exact and std::io::Write::write_all.

Here's my simple solution:

impl File {
    pub async fn read_exact_at<T: IoBufMut>(&self, mut buf: T, mut pos: u64) -> BufResult<(), T> {
        let mut bytes_read = 0;
        let buf_len = buf.bytes_init();
        while bytes_read < buf_len {
            let slice = buf.slice(bytes_read..);
            let (res, slice) = self.read_at(slice, pos).await;
            buf = slice.into_inner();
            match res {
                Ok(0) => {
                    return (
                        Err(io::Error::new(
                            io::ErrorKind::UnexpectedEof,
                            "failed to fill whole buffer",
                        )),
                        buf,
                    )
                }
                Ok(n) => {
                    bytes_read += n;
                    pos += n as u64;
                }
                Err(ref e) if e.kind() == io::ErrorKind::Interrupted => {}
                Err(e) => return (Err(e), buf),
            };
        }

        (Ok(()), buf)
    }

    pub async fn write_all_at<T: IoBuf>(&self, mut buf: T, mut pos: u64) -> BufResult<(), T> {
        let mut bytes_written = 0;
        let buf_len = buf.bytes_init();
        while bytes_written < buf_len {
            let slice = buf.slice(bytes_written..);
            let (res, slice) = self.write_at(slice, pos).await;
            buf = slice.into_inner();
            match res {
                Ok(0) => {
                    return (
                        Err(io::Error::new(
                            io::ErrorKind::WriteZero,
                            "failed to write whole buffer",
                        )),
                        buf,
                    )
                }
                Ok(n) => {
                    bytes_written += n;
                    pos += n as u64;
                }
                Err(ref e) if e.kind() == io::ErrorKind::Interrupted => {}
                Err(e) => return (Err(e), buf),
            };
        }

        (Ok(()), buf)
    }
}

Use-after free in `Op::connect`

I was wondering why connecting to IPv4 addresses sometimes worked.. but IPv6 doesn't. Sometimes I got an errno 97 (address family not supported), sometimes it was just hanging... and adding a couple dbg! statements made it work?

All of this smells a lot like memory unsafety.. and it is!

Here, a Connect struct is created, moving a SockAddr into it:

tokio-uring/src/driver/connect.rs

Lines 17 to 30 in 1c0a2bf

	Op::submit_with(
	Connect {
	fd: fd.clone(),
	socket_addr,
	},
	|connect| {
	opcode::Connect::new(
	types::Fd(connect.fd.raw_fd()),
	connect.socket_addr.as_ptr(),
	connect.socket_addr.len(),
	)
	.build()
	},
	)

Then, in submit_with, the data is moved into an Op struct, the callback is called to actually configure the SQE from the given data...

tokio-uring/src/driver/op.rs

Lines 74 to 78 in 1c0a2bf

	// Create the operation
	let mut op = Op::new(data, inner, inner_rc);
	// Configure the SQE
	let sqe = f(op.data.as_mut().unwrap()).user_data(op.index as _);

And then op is dropped (and the data with it). We now have an op in the submission queue that refers to some userspace address that's been freed (somewhere on the stack, presumably). This probably works well in tests because immediately after attempting to connect, it sleeps. But we now have the kernel using some memory after we've freed it.

A potential fix would be to move the data into the Lifecycle::Submitted variant - but that of course raises the question of: should it be boxed? Probably not right? So there's only so much data we can stuff in there - maybe connect is one of the rare ops that needs something like this, so maybe it should itself contain an enum?

At any rate, that explains the random failures I've been getting.