hadrieng2 / async-ops Goto Github PK

Every modern computer comes with a little GPU supercomputer inside. Given a little bit of hardware-specific driver setup, one can command all of these using a consistent asynchronous API, OpenCL. I should explore ways to interface this API in a consistent way.

The current inline callback executor has many issues, and is only suitable for special cases. As a common case, I would ultimately like callbacks to be processed asynchronously by a CPU-sized pool of threads, rather than synchronously by the single unlucky and arbitrarily selected thread who happens to have propagated a server status update to the client.

In this TBB-like design, it is desirable to use work-stealing dequeues in order to improve the scalability to many-core CPUs. At the same time, and unlike in TBB, deterministic work-stealing algorithms (rather than random selection) should be used in order to enable efficient CPU usage when no task is being processed.

Sometimes, the server carrying out asynchronous work is actually located on a remote machine, accessible over a network. This usage scenario should be considered as an extreme variant of the IPC scenario of #4, where each of the concerns outlined there is exacerbated. In addition, other specific challenges emerge:

• Networked computers may not be binary-compatible, requiring use of serialization.
• Network communications are out-of-order and unreliable. Trying to ignore this reality with reliable in-order protocols like TCP is just a way to take a performance hit in a vain attempt to ignore the underlying issues. A better approach is to build an UDP-based protocol that fits our needs.
• Network latencies are enormous. Blocking is less acceptable than ever.
• A single system can manage many network connections, so #2 is more critical than ever.

Clients often want to interact with sets of asynchronous operations, by waiting for the status of one or all of them to change in some way. We should provide a way to achieve this goal through something like the when_any/when_all future combinators of the C++ concurrency TS.

Some libraries such as Tokio use a coroutine-based model for asynchronous operations, where some monitoring code is executed on demand whenever the status of an asynchronous operation is polled or synchronized with. We should probably provide a backend which is compatible with this.

Once we have threaded asynchronous operations, an obvious thing to do is to start implementing user threading on top of a finite thread pool, like HPX and Intel TBB in the C++ world. Need to explore which libraries already exist for this purpose and how much they would fit our needs. Code should probably be heavily shared with #1 .

People are not going to use yet another abstraction if it's not fast enough. I really need to measure the performance of this library, profile, determine what's slow and improve upon it.

Sometimes, it makes sense to implement the client and the server of an asynchronous operation in different OS processes. Although shared-memory communication can still easily be used, the interprocess communication setting still has a number of unique points that should be carefully considered:

• Processes can crash independently and should tolerate each other's failure
• Processes live in separate address spaces, so pointers should be used sparingly in communication
• Processes are isolated from each other for a reason, security considerations should be examined
• Interprocess signaling is more complex and less efficient than thread signaling

An asynchronous API is not complete without proper IO support. For networked IO, we will probably want to use Tokio, which will require #3. For file IO, I still need to check which good libraries exist in Rust.