Is IO prioritisation possible? about eio

[talex5]

Yes, using a separate multicore domain would work for this. However, Eio is currently lacking primitives for communicating easily between domains.

With the io-uring backend, there is an ioprio field that might be useful (I don't know - I've never used it). It's not exposed in the OCaml API at the moment, though. Apart from that, Linux will do the reads in whatever order it wants.

The easiest way is probably to have each reader (handling one client connection) push the client request onto an Eio.Stream and have a single fibre pulling the requests off and handling them, doing Fibre.yield after each request. Then when a high-priority internal message arrives, that doesn't go on the queue and is instead handled immediately. But that assumes that it's the processing of the requests that is slow, rather than merely receiving them.

[Cjen1]

The stream approach should work I think (given limited size streams), but would require that any rpc library to implement that kind of processing.

Is there any way to do prioritisation of the scheduling of fibres themselves?
(since in theory the read will not complete until the fibre is run again)

[talex5]

Is this still a problem?

being able to prioritise internal communications over external ones is quite important

I think we always prioritise internal comms over IO now. Both uring and luv backends keep a special "IO" task in the run queue to remind them to check for IO from time to time because otherwise doing Fiber.yield in a loop would block everything else.

[Cjen1]

After rewriting the IO layer, this is no longer an issue for me specifically.

What was occurring was that if client requests were coming in they would each start a new fiber waiting for the resolution of that rpc request.
The issue was that if client requests kept coming in the time between main loop runs would increase, effectively causing queuing at the client side.

I've now redone the IO layer such that it is one fiber per client which should prevent this from occurring and also I can chuck it into another domain to avoid this entirely.

I think that if there was some overarching scheduling prioritisation feature this would help (attaching each fiber some priority) but I also fully understand how much work that would entail and also that it would be difficult to avoid starvation etc.
Additionally if other solutions exist (such as popping the client communication into its own domain) then this should be a non-issue for now.