This is a project created for the subject TAP at the university of Catania. The idea is to showcase a simple ETL pipeline using some of the most widely known technologies in the big data fields
How are you managing the fact that confluent-kafka's wheel does not include an install of librdkafka?
Is that necessary for the build? or is it just for the dev environment? currently, this is preventing the docker-compose up command from running successfully.
Error: (during ingestion's dockerfile pip install requirements): confluent_kafka.h:23:10: fatal error: librdkafka/rdkafka.h: No such file or directory