Cannot read messages using the examples from readme file about rstream

[wrobell]

Actually, the example I was using was from PyPI:

await producer.publish('mystream', f'msg: {i}'.encode())

It works if I replace above line with

msg = AMQPMessage(body='hello: {}'.format(i))
await producer.publish('mystream', msg)

Looking at the changelog, the example was modified with message change the example to AMQP 1.0.

IMHO, it would be great to keep the first version of API as it is simpler.

Selection of protocol version probably shall be done differently, i.e. when creating producer.

[wrobell]

I have looked into the code a bit. We can send

• raw bytes (is that AMQP 0-9-1?)
• AMQP objects (this seems to be AMQP 1-0)
• mix of above (publish batch allows that)

Is mixing of messages by design? This does not seem to be supported by consumer?

[Gsantomaggio]

raw bytes (is that AMQP 0-9-1?)

No, you can store what you want. In general we decided to use amqp 1.0 to save the messages in this way all the clients are compatible.

But if you want you could use something else to save the messages.

Currently, all the stream clients support amqp 1.0 as default

[qweeze]

Sorry for long delay. @wrobell, thanks for reporting this!

The errors you're getting are most likely because you published raw (non AMQP-encoded) messages to the stream and then tried to consume them with decoder=amqp_decoder.

Indeed, mixing of differently encoded messages breaks consumer.
On the other hand, restricting publisher to use a single protocol doesn't seem good enough solution to me. Firstly, it doesn't really guarantee any data consistency in the stream - other publishers/applications can still write anything to the stream. So consumers will still have to deal with invalid messages.
Also, producer doesn't really "support" these protocols, it just treats anything that can be converted to bytes (basically anything with a __bytes__ method) as a valid message. I want to keep this feature since it makes extending library and using other encodings really easy.

I also think it would be helpful to wrap this error and make it more descriptive

[wrobell]

@qweeze @Gsantomaggio Thanks for the information. I studied the code and the documentation bit more and it all makes sense now.

I wonder if the API could be improved, though. For example if AMQP is the default, then we should not have to specify the decoder when receiving a message. But then how to design the publisher method, so it uses AMQP by default and keeps message format flexibility?

BTW. While trying to understand how things work, I wrote some scripts to publish data to RabbitMQ streams on my own. My concerns are reconnecting clients (i.e. on broker restart) and performance (it can be improved by using custom asyncio protocol). Now, also above API dilemma 🤔 I will pour my findings into this project https://gitlab.com/wrobell/rbfly. Please do not hesitate to steal the ideas from it.