Stefen's Projects
Ce projet développe un modèle de type Retrieve-Augment-Generate (RAG) pour répondre aux questions en utilisant les données publiques des avis laissés sur Google pour des hôpitaux à Montréal
big data, the ability to extract, transform, and store data from the web into different storage systems like MongoDB, PostgreSQL, MinIO, and Elasticsearch is a crucial skill for developers and data scientists
Setting Up and Querying Elasticsearch with Python and Streamlit
reating a modern data pipeline using a combination of Terraform, AWS Lambda and S3, Snowflake, DBT, Mage AI, and Dash.
Building a Robust Data Pipeline: Integrating Proxy Rotation, Kafka, MongoDB, Redis, Logstash, Elasticsearch, and MinIO for Efficient Web Scraping
This tool migrates data from MongoDB collections to Elasticsearch indices. It's built using Rust and supports configurable migrations.
simple machine learning projet
Building a Modern Uber Eats Data Pipeline
This post will demonstrate the creation of a containerized data engineer environment using Docker Stacks.
This repository contains structured datasets in various categories
A library to access OpenStreetMap related services
Utilizing of open source technologies for the implementation of a data pipeline
Automatização de pequenos processos em python
convert_json_to_csv
Dynamic Snake Game: Unleashing Real-Time Streaming Analytics with Redis, Kafka, Flink, ClickHouse & Chart.js in an Online Snake Game via Flask API
The project aims to demonstrate how to work with real-time data using Kafka, KSQL, Elasticsearch, and Flask. It shows how to perform joins on Kafka topics, ingest data into Elasticsearch using Kafka Connect, and build a REST API to provide real-time metrics to end-users.
we are thrilled to announce our new PoC project aimed at providing a complete real-time extraction, transformation, and exposure architecture for the new provincial transportation systems.
In this rendition, Elastic and Kibana have been replaced with the powerful Splunk, MQTT has been swapped out for ActiveMQ, and instead of the traditional Kafka, we’ve integrated Confluent Cloud.
The goal of this dbt project is to analyze music streaming data to determine listening trends, user preferences and popular genres. The project consists of SQL templates, tests and macros to transform and verify the data
In this article, we'll walk through how to build a scalable ETL pipeline using Apache Airflow, Kafka, and Python, Mongo and Flask
Config files for my GitHub profile.
In this article, you will learn how to set up a real-time data processing and analytics environment using Docker, MySQL, Redpanda, MinIO, and Apache Spark.
Streaming Synthetic Sales Data Generator: Streaming sales data generator for Apache Kafka, written in Python
Developpement piloté par des test