turtlecoder / lungko-2019a-de-project

Stock Trading with Apache Kafka Streams

Introduction

This project implements a Kafka Streams pipeline that processes a stock ticker stream to initiate buy or sell actions.

Motivation

Apache Kafka has grown in popularity over the past few years and has been widely adopted by many businesses. While it started out as a log messaging system, Kafka has grown to encompass more functionality, including the addition of Kafka Streams. Implementing a stream processing pipeline entirely in Kafka without adding technology overhead such as Apache Spark or Apache Storm presents itself as an attractive option.

Tools and Technologies Used

1. Apache Kafka (Streams DSL, Connect)
2. S3
3. Python
4. Dash
5. PostgreSQL

Data Flow