artso17 / mongodb-kafka-hw

Streaming Processing Using Apache Kafka and MongoDB (Source Data: Finhub) | DigitalSkola Homework

Streaming Processing Using Apache Kafka and MongoDB (Source Data: Finhub) | DigitalSkola Homework

Project Brief

The Project aimed to gain and prove my skills in Data Engineering that is streaming processing using Apache Kafka as message broker and streaming tool, MongoDB as Data Warehouse. In this case, Finhub is the data source retrieved through API.

Success Criteria

• Create and Run producer script with name producer_finhub.py that get data source using API
• Create and Run consumer script with name consumer_mongo that consume the data from Kafka and store it to MongoDB
• Simulate and Record the process

Result

• The project using python 3.8

mongo-kafka1.1.mp4