upyun / cafex Goto Github PK

Hi and thanks for creating Cafex!

I'd like to have a consumer receive input from two or more topics to perform some stream joining, similar to how a samza job might be structured: (https://samza.apache.org/learn/documentation/0.9/container/state-management.html)

It looks like the underlying Kafka Fetch protocol supports multiple topics (https://github.com/upyun/cafex/blob/master/lib/cafex/protocol/fetch.ex#L10), but it is not surfaced in the Cafex consumer configuration API?

In the meantime, is the best alternative to start a consumer for each topic, and forward the messages to my own GenServer, which can maintain some state and produce the outputs?

Thanks again!

Hi,

I'd like to experiment Heroku's Kafka hosting, does that implementation can connect via TLS (the only way accepted by Heroku)? Thanks by advance.