smol-rs / futures-lite Goto Github PK

These functions are stable in the core library since rustc 1.47.0. see also rust-lang/rust#74328

futures-lite currently depends on parking 1.0.5, while async-io depends on parking 2.0.0 and on futures-lite; this means any crate using async-io pulls in two versions of parking. Please consider upgrading futures-lite to parking 2.0.0.

As I am working on a venerable dual core MacBook Air, I will always be mindful of compile times, trying my best to reduce them in all my crates.

Futures-lite seems like it could make a difference in prodash, so I tried to migrate. Right now, I would be missing the following capabilities to finish the transition:

• crosstermion
  • StreamExt::filter_map(…) (if I had to chose one, this would be it)
• prodash::tui::engine
  • stream::select_all(…)
  • StreamExt::map(…)
  • StreamExt::boxed(…) (see #1)

Even though I could add boxed* myself (or could have worked around it), I feel way less confident about contributing implementations for the above without having a conversation first and most certainly guidance.

Do you consider adding those at some point and/or would you be open to contributions for these?

Thanks a bunch!

Edit: I will get started on implementing these unless @stjepang gets to them more quickly.

Hi,

I ran into a strange issue where a test of mine was hanging. Interestingly, changing from an instance of Async<serialport::TTYPort> to Async<mio_serial::SerialStream> seems to solve the problem. At first, I thought that was significant, but now I think it may just be coincidental.

While trying to create a minimal reproduction, it looks like swapping the order of smol::spawn calls influences the outcome. That fact, combined with knowledge that the ReadHalf and WriteHalf types returned by smol::io::split wrap instances of Arc<Mutex<T>>, makes me strongly suspect this is a deadlock of some kind. That's reinforced by the fact that I'm seeing slightly different behavior between my minimal example and my real code. Moreover, I've now seen variation in the behavior from day to day. Finally, I should note that the problem goes away completely if I spawn two instances of smol::io::copy. However, as mentioned in smol-rs/smol#270, one of these streams can't be implemented with smol::io::copy.

Here is the reproduction code. It's specific to my system, because it communicates with my serial device. But it could be easily adapted to some other device.

When I run this code, the blocking_serialport and async_mio_serial tests pass, but the async_serialport test hangs. However, if I move the spawn of smol::io::copy to the indicated position, all tests pass.

use std::io::{Read, Write};
use std::os::fd::AsRawFd;
use std::os::unix::net::UnixStream;
use std::time::Duration;

use smol::io::{AsyncReadExt, AsyncWriteExt};
use smol::Async;

const REQUEST: [u8; 12] = [
    0xC0, 0x00, 0x01, 0x00, 0x52, 0x00, 0x01, 0x77, 0xa0, 0x35, 0xc4, 0xC0,
];
const RESPONSE: [u8; 12] = [
    0xC0, 0x00, 0x20, 0x04, 0x81, 0x00, 0x01, 0xf9, 0xf5, 0x94, 0xaf, 0xC0,
];

fn adapter(device: impl Read + Write + AsRawFd + Send + 'static) -> Async<UnixStream> {
    let (a, b) = UnixStream::pair().unwrap();
    let b = Async::new(b).unwrap();
    let device = Async::new(device).unwrap();
    let (mut b_reader, b_writer) = smol::io::split(b);
    let (device_reader, mut device_writer) = smol::io::split(device);
    smol::spawn(smol::io::copy(device_reader, b_writer)).detach();
    smol::spawn(async move {
        let mut buf = [0; 1024];
        loop {
            println!("Starting loop");
            let n = match b_reader.read(&mut buf).await {
                Ok(n) => n,
                Err(err) => {
                    println!("{err:?}");
                    continue;
                }
            };
            println!("Read {n} bytes");
            match device_writer.write_all(&buf[..n]).await {
                Ok(()) => {}
                Err(err) => {
                    println!("{err:?}");
                    continue;
                }
            };
            println!("Wrote {n} bytes");
        }
    })
    .detach();
    // FIXME: Moving the copy spawn here allows the test to pass
    //smol::spawn(smol::io::copy(device_reader, b_writer)).detach();
    Async::new(a).unwrap()
}

fn serialport_device() -> serialport::TTYPort {
    serialport::new("/dev/ttyUSB0", 230400)
        .timeout(Duration::from_secs(1))
        .open_native()
        .unwrap()
}

fn mio_serial_device() -> mio_serial::SerialStream {
    use mio_serial::SerialPortBuilderExt;

    mio_serial::new("/dev/ttyUSB0", 230400)
        .timeout(Duration::from_secs(1))
        .open_native_async()
        .unwrap()
}

fn test_blocking(mut device: impl Read + Write) {
    let mut buf = [0; RESPONSE.len()];
    device.write_all(&REQUEST).unwrap();
    device.read_exact(&mut buf).unwrap();
    assert_eq!(buf, RESPONSE);
}

fn test_async(device: impl Read + Write + AsRawFd + Send + 'static) {
    let mut socket = adapter(device);
    let mut buf = [0; RESPONSE.len()];
    smol::block_on(async {
        socket.write_all(&REQUEST).await.unwrap();
        socket.read_exact(&mut buf).await.unwrap();
    });
    assert_eq!(buf, RESPONSE);
}

#[test]
fn blocking_serialport() {
    let device = serialport_device();
    test_blocking(device);
}

#[test]
fn async_serialport() {
    let device = serialport_device();
    test_async(device);
}

#[test]
fn async_mio_serial() {
    let device = mio_serial_device();
    test_async(device);
}

I'm moving from futures to futures-lite but noticed this is missing StreamExt::for_each_concurrent.

Hello,

Would it be possible to add the following 2 to futures-lite?

https://docs.rs/futures-util/latest/futures_util/stream/trait.TryStreamExt.html#method.map_ok
https://docs.rs/futures-util/latest/futures_util/stream/trait.StreamExt.html#method.buffered

I'm trying to see if I can switch from futures-util and tokio to futures-lite and smol..

EDIT: I have to say though, futures-lite is taking 10-20x longer... maybe because I can't use buffered() here.

This is rather just a question: does this crate support future::Either (here) or something similar? Thanks.

I am surprised this is something not currently implemented for Stream, unless I am missing something. It would be a great addition.

I have a use case where I need a byte stream and async reader. I currently have my own implementation of this that works fine but I was wondering if other might find this useful. I could either add an AsyncRead impl to io::Bytes or create a separate type for this. I wanted feedback before submitting a patch. Thoughts?

How about adding a similar method?
https://docs.rs/futures/latest/futures/stream/trait.StreamExt.html#method.take_until

This can be convenient for a stream cancellation.

Hey,

first of all, great work! I don't think the smol project get's enough recognition for its amazing work.

I learned from #65 that you have an or function which is like select! except in that it requires both futures to have the same output type. Is there already a function like or which works for two futures with distinct output types and returns an Either enum?

If there is no such function as of yet I would like to try my hands at implementing it. I have thought of two issues:

1. what to name the function
2. this necessitates an Either struct we would either (ba dum tsss) need to implement ourselves or depend on an an (albeit small) either crate.

I think this would be a very useful and straight forward addition.

Not sure about the name, but I just found a use for this and it's incredibly small. Maybe you have a better idea for a name?

pub async fn waker() -> Waker {
    future::poll_fn(|c| Poll::Ready(c.waker().clone())).await
}

When trying to run the example as shown in the blog, it just hangs after it awaits the first Task.
Any idea what may be going on?

Is the parking crate strictly necessary when the standard library is available? According to the docs, std::thread::park has been available since 1.0 and looking at it's implementation, it seems very similar in the generic case to parking's. It does seem to have per-platform optimizations too. An equivalent Unparker can be made wrapping a std::thread::Thread using Thread::unpark.

To prevent being stuck at 0.1 forever, I am considering publishing 1.0 rather sooner than later. The crate is 90% done anyway and I don't expect there will be many changes - just a few remaining additions.

The only breaking change I might do is rename future::join() to future::zip(). Thoughts?

this has stopped me from using futures-lite a couple of times now. incredibly useful function.

In smol, Executor and LocalExecutor don't implement any common traits. So, it's impossible to create a common abstraction over both traits. This is a blocker for notgull/smol-hyper#2 (comment) and smol-rs/smol#292.

However I think that we should have an abstraction for executors in general. We put an emphasis on a diversity of executors like smolscale but we have no way of abstracting over them. Therefore I think we should have a trait for spawning futures, specifically in this crate.

A couple of potential strategies here:

• We could just bring in futures-task, which would fit the general ethos of this crate. However I think that we should avoid this, as futures-task doesn't really fit well with async-executor outside of the basics.
• Have our own Spawn trait that returns a Task trait that is a superset of Future.

@smol-rs/admins Thoughts?

Hi! The background is that I've been looking for a crate for implementing a Stream in an embedded system. I see that this crate does have a std feature, so is this crate intended to support no_std?

In terms of the current state of this:

• I tried running the tests with --no-default-features, and there were a lot of failures. Maybe some of the tests could be gated behind the std feature? It looks like there may be some pretty important functionality that requires std, though.
• The fastrand crate doesn't currently support nostd, so it would either need to be an optional dependency or we'd need to add nostd support to fastrand.

You seem to have forgotten to remove some debug prints from the poll_fill_buf function:
https://github.com/stjepang/futures-lite/blob/f5faa76ab6c20a0aa5f27d4d7fb4f5faf4a5f4ec/src/io.rs#L619
https://github.com/stjepang/futures-lite/blob/f5faa76ab6c20a0aa5f27d4d7fb4f5faf4a5f4ec/src/io.rs#L627
https://github.com/stjepang/futures-lite/blob/f5faa76ab6c20a0aa5f27d4d7fb4f5faf4a5f4ec/src/io.rs#L630

Now that Wakers can be constructed safely, we can probably eliminate the waker-fn dependency.

Maybe blocked on smol-rs/parking#17

Hello,

I am encountering the following issue:

use futures::io::{AsyncRead, Take, AsyncReadExt};

fn foo<T: AsyncRead>(s: &mut T) -> Take<&mut T> {
    s.take(5)
}

error[E0308]: mismatched types
 --> src/lib.rs:6:5
  |
5 | fn foo<T: AsyncRead>(s: &mut T) -> Take<&mut T> {
  |        - this type parameter
6 |     s.take(5)
  |     ^^^^^^^^^ expected `&mut T`, found type parameter `T`
  |
  = note: expected struct `futures::io::Take<&mut T>`
             found struct `futures::io::Take<T>`

I usually solves this by implementing the trait on &mut T, but this is not possible for AsyncReadExt (conflicting implementation).

Would you consider adding a helper function on Take, please, such:

impl<T> Take<T> {
    fn by_ref(&mut self) -> Take<&mut T> {...} 
}

Otherwise what workaround would you recommend, please ?

Since we have future::block_on() and stream::block_on(), it would be nice to also have io::block_on() for consistency.

I'm considering adding the following type aliases:

type BoxedAsyncRead = Pin<Box<dyn AsyncRead + Send + 'static>>;
type BoxedAsyncWrite = Pin<Box<dyn AsyncWrite + Send + 'static>>;

Then, AsyncReadExt and AsyncWriteExt could also have methods boxed_read() and boxed_write().

Would anyone else find this useful?

Recently I switched from futures and futures-util to futures-lite. For the most case this was okay.

The only thing I did miss was the fuse functionality.

I implemented it here https://github.com/plabayo/rama/blob/main/src/future.rs.
Could I implement it in the actual futures-lite codebase.
Or what do you suggest?

futures-lite implements the join function to join two futures. I'd love to have the join! macro to join more than two futures, if that'd be a reasonable addition to futures-lite.

thread 'sscale-wkr-c' panicked at 'assertion failed: `(left == right)`
  left: `669338`,
 right: `0`', /Users/shmendez/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-lite-1.12.0/src/io.rs:1700:13
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
[1]    39699 abort      cargo run --bin themelio-node -- --index-coins

Is this a known issue?

I just discovered the futures-lite crate and I thought "let me try to convert some trivial code that I have using futures to this lighterweight crate". And it's great because I trimmed 20 seconds of build time!

Unfortunately, the conversion wasn't as clean as I hoped because BoxedLocal currently seems to enforce a 'static lifetime... and I have a piece of code that was previously using LocalBoxFuture and requires a different lifetime. I was able to easily workaround this by redefining BoxedLocal to take a custom lifetime and then directly using Box::pin instead of FuturesExt::boxed_local(). See this commit.

But I think it'd be nice if the interface in futures-lite supported a custom lifetime on its own for both Boxed and BoxedLocal. Thoughts?

In futures there is stream::once which allows creating a stream from a future. I would argue it generic enough and simple enough to warrant it being functionality provided in this library.

Would a stream::future or similar named function be accepted with a PR?

https://github.com/smol-rs/futures-lite/actions/runs/6060417025/job/16444572020

Hey there,
so I was trying to read_line from a BufReader but it produces WouldBlock if there's no data available.
What is the proper way to force the thing to wait until the data is actually there?

        p.read_line(&mut msg).await.unwrap();

thread 'bash' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 11, kind: WouldBlock, message: "Resource temporarily unavailable" }', tests/repl_async.rs:15:37

I use BufReader on a async_fs::File

By the way is it normal? https://docs.rs/futures-io/0.3.15/futures_io/trait.AsyncBufRead.html#tymethod.poll_fill_buf

If no data is available for reading, the method returns Poll::Pending and arranges for the current task (via cx.waker().wake_by_ref()) to receive a notification when the object becomes readable or is closed.

I'm trying to use stream::Race to join a couple of timer-based streams that periodically emit events before terminating. Sadly, the current implementation of stream::Race will never terminate as it will always return Poll::Pending if no sub-stream polls Ready(Some(_)):

I'm happy to make a PR if this would be considered a bug, but I figured I'd check first since it could break users who rely on its non-termination.

The StreamExt::map(), StreamExt::filter_map() in the futures crate accepts async functions which resolve to an <T>/Option<T>, whereas the same methods in the futures-lite crate accept normal functions which are meant to return <T>/Option<T>. (there are other similar methods as well)

I don't know if this is by design or a mistake, but since this crate is supposed to be fully compatible with the futures crate, I think these methods should be changed accordingly.

https://docs.rs/futures-lite/1.11.1/futures_lite/macro.ready.html#examples

Lists an example usage of:

use futures_lite::future::FutureExt;
use futures_lite::ready;
use std::future::Future;
use std::pin::Pin;
use std::task::{Context, Poll};

// Polls two futures and sums their results.
fn poll_sum(
    cx: &mut Context<'_>,
    a: Pin<&mut impl Future<Output = i32>>,
    b: Pin<&mut impl Future<Output = i32>>,
) -> Poll<i32> {
    let x = ready!(a.poll(cx));
    let y = ready!(b.poll(cx));
    Poll::Ready(x + y)
}

I'm pretty sure this is incorrect because if a.poll() returns Ready but b.poll() returns Pending, a will get poll()ed twice, which isn't allowed: https://doc.rust-lang.org/std/future/trait.Future.html#panics

Hello,

Are you considering adding futures::TryStreamExt and futures::TryFutureExt?

I find the and_then, err_into and into_stream and other chaining utilities quiet convenient.
Or you recommend using futures for this on the long term ?

What do you think ?

Thanks for all your work on Smol and associated crates. I'm learning a lot from reading the code and enjoy using the small apis. While reading through I came across what I think is a bug in the implementation of race.

https://github.com/stjepang/futures-lite/blob/master/src/future.rs#L504-L517

        if fastrand::bool() {
            if let Poll::Ready(t) = this.future1.poll(cx) {
                return Poll::Ready(t);
            }
            if let Poll::Ready(t) = this.future2.poll(cx) {
                return Poll::Ready(t);
            }
        } else {
            if let Poll::Ready(t) = this.future1.poll(cx) {
                return Poll::Ready(t);
            }
            if let Poll::Ready(t) = this.future2.poll(cx) {
                return Poll::Ready(t);
            }

I might be missing a subtle point, but shouldn't that be:

        if fastrand::bool() {
            if let Poll::Ready(t) = this.future1.poll(cx) {
                return Poll::Ready(t);
            }
            if let Poll::Ready(t) = this.future2.poll(cx) {
                return Poll::Ready(t);
            }
        } else {
            if let Poll::Ready(t) = this.future2.poll(cx) {
                return Poll::Ready(t);
            }
            if let Poll::Ready(t) = this.future1.poll(cx) {
                return Poll::Ready(t);
            }

To switch the order the futures get polled in the else branch.

I don't seem to find an equivalent for select! and select_biased! in the smol ecosystem. If it exists, please point it out. If it doesn't exist, can I use the one in futures-util for now? Will it cause incompatibility to use only this macro and fuse() with FutureExt?

Just a heads up. Sticking with 1.8 is working for me.

error[E0596]: cannot borrow self as mutable, as it is not declared as mutable
--> C:\Users___.cargo\registry\src\github.com-1ecc6299db9ec823\futures-lite-1.10.0\src\io.rs:2819:24
|
2815 | self: Pin<&mut Self>,
| ---- help: consider changing this to be mutable: mut self
...
2819 | Pin::new(&mut *self.0.lock().unwrap()).poll_read(cx, buf)
| ^^^^ cannot borrow as mutable

error[E0596]: cannot borrow self as mutable, as it is not declared as mutable
--> C:\Users___.cargo\registry\src\github.com-1ecc6299db9ec823\futures-lite-1.10.0\src\io.rs:2827:24
|
2823 | self: Pin<&mut Self>,
| ---- help: consider changing this to be mutable: mut self
...
2827 | Pin::new(&mut *self.0.lock().unwrap()).poll_read_vectored(cx, bufs)
| ^^^^ cannot borrow as mutable

error[E0596]: cannot borrow self as mutable, as it is not declared as mutable
--> C:\Users___.cargo\registry\src\github.com-1ecc6299db9ec823\futures-lite-1.10.0\src\io.rs:2833:24
|
2832 | fn poll_write(self: Pin<&mut Self>, cx: &mut Context<'_>, buf: &[u8]) -> Poll<Result> {
| ---- help: consider changing this to be mutable: mut self
2833 | Pin::new(&mut *self.0.lock().unwrap()).poll_write(cx, buf)
| ^^^^ cannot borrow as mutable

error[E0596]: cannot borrow self as mutable, as it is not declared as mutable
--> C:\Users___.cargo\registry\src\github.com-1ecc6299db9ec823\futures-lite-1.10.0\src\io.rs:2837:24
|
2836 | fn poll_flush(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Result<()>> {
| ---- help: consider changing this to be mutable: mut self
2837 | Pin::new(&mut *self.0.lock().unwrap()).poll_flush(cx)
| ^^^^ cannot borrow as mutable

error[E0596]: cannot borrow self as mutable, as it is not declared as mutable
--> C:\Users___.cargo\registry\src\github.com-1ecc6299db9ec823\futures-lite-1.10.0\src\io.rs:2841:24
|
2840 | fn poll_close(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Result<()>> {
| ---- help: consider changing this to be mutable: mut self
2841 | Pin::new(&mut *self.0.lock().unwrap()).poll_close(cx)
| ^^^^ cannot borrow as mutable