elsayed91 / oot

Order Of The Template - a Python toolkit to parse YAML files and Jinja templates, resolve environment variables, and validate JSON schemas

Hello there! I'm Islam ✨✨

🔧 Technologies & Tools

🌱 About Me

Former manager and civil engineer transitioning to data engineering after discovering a passion for building scalable solutions.

🚀 Featured Projects

• NY Taxi Data & MLOps Pipeline: Automated data & MLOps pipeline leveraging Kubernetes and Apache Airflow. Integrates Spark, Kafka, and DBT with a focus on data quality. Tailors solutions for diverse user needs.
• Xbox Data Scraping & Analysis Pipeline: Automated data-driven project leveraging Python, Airflow, and GKE. Scrapes diverse data sources, providing insights into Xbox hardware and game data.

📦 Packages

• Easy Expectations: A python package that abstracts away the complexity of Great Expectations and allow for easy no-knowledge-required implementation for basic use cases.
• SchemaDiff: A python package that efficiently detects files with inconsistent schemas amidst thousands of files by reading the parquet files metadata.
• Order of The Template: A Python toolkit for parsing and processing YAML templates, capable of resolving Bash syntax environment variables and Jinja templating. It also offers schema validation functionality.