andersea / slurry Goto Github PK

Hi, I think slurry is good for my use case, but I cannot tell for sure, because there are not enough examples of using the various parts of the library in combination.

I would particularly like to know:

• the best way to have a pipeline take input from a Trio memory channel
• the best way to have a pipeline tap output into a Trio memory channel
• how to enable/disable sections of a pipeline on the fly

This is a list of prior art for slurry-like frameworks of note, for documentation purposes.

https://github.com/vxgmichel/aiostream

https://github.com/erdewit/eventkit

https://github.com/vodik/asyncitertools

The way broadcasting to taps and extensions works currently is not safe. It should support backpressure, but currently messages just pile up in a big queue, leading to possible memory leaks and other unpleasant issues. This needs a thorough rethink. There is a lot of discussion available on the issue of broadcasting in the trio community as well.

See for instance: python-trio/trio#987