Open ThroughStream for extension about stream

[kelunik]

If implementing new streams is complicated, maybe some work should be put into a simpler API that's promise based instead of event based? There could then be a new stream implementation, that makes these simpler implementations work with the existing stream API.

[mmoreram]

Don't misunderstand me. When I say complicated I mean that there are some specific parts that you should take into account once and again (event propagation, some checks before writing...). When I create a new class in my codebase I need to test it and testing the same feature for all my Streams transformers, even if it's the same, doesn't make sense IMO.

The fast and easy way of doing that would be to create by myself a copy of ThroughStream and doing these changes, but I think that would be interesting to make something available for everyone.

[WyriHaximus]

Take a look at these streams @mmoreram https://github.com/WyriHaximus/reactphp-stream-base64/tree/master/src If the only thing you need is data transformation that should show we'd you need

[kelunik]

@WyriHaximus Seems like https://github.com/WyriHaximus/reactphp-stream-base64/blob/af6d98b72850aaf220e6ec28d7c4c788973ed3b9/src/ReadableStreamBase64Encode.php#L56 pipes the original stream instead of the transformed stream?

[mmoreram]

@WyriHaximus but these examples are storing the whole data set in memory, right? At least until you close the stream?

[WyriHaximus]

@mmoreram only, only what they need to transform the next chunk of data, once they encode/decode that chunk it's released from memory again.

[mmoreram]

@WyriHaximus I've created this PR.

#152

TBH, I don't like so much the name "TransformerStream", but I didn't find a better one.
For the implementation, I took as reference the NDJson encoder. An abstraction of what a transformer should always seem a good thing for the final user.

[WyriHaximus]

@kelunik thanks just fixed them 👍

[clue]

@mmoreram Thanks for bringing up this discussion!

It's my understanding opening the ThroughStream for extension is not a good idea because it breaks the existing encapsulation. This class specifically exists for 1-on-1 mapping from input to output.

If you want to implement custom streaming behavior, you can always implement the DuplexStreamInterface. Depending on your specific use case, it might also be easier to implement just the ReadableStreamInterface or WritableStreamInterface. This allows you to implement any kind of custom behavior, such as data transformations, time-based forwarding and much more.

You can take a look at https://packanalyst.com/class?q=React\Stream\ReadableStreamInterface and others for some possible implementation ideas. It's my understanding this covers all possible use cases and implementing this interface shouldn't be too hard imho.

I hope this helps 👍

I'm closing this for now as it hasn't received any input in a while and I believe this has been answered. Please come back with more details if this problem persists and we can always reopen this 👍