amazon-sagemaker-safe-deployment-pipeline

Safe blue/green deployment of Amazon SageMaker endpoints using AWS CodePipeline, CodeBuild and CodeDeploy.

auto-gpt

An experimental open-source attempt to make GPT-4 fully autonomous.

awesome-python

A curated list of awesome Python frameworks, libraries, software and resources

company-scraper

Scrapes Company,CEO and other details from a Linkedin URL

data-engineering-books

data-engineering-zoomcamp

Code for Data Engineer Zoomcamp course

dateandtimedimensionbuilders

Data warehousing date dimension and time dimension builders written in Python.

engineering-blogs

A curated list of engineering blogs

event-fetcher

This repository hosts a cloud-based data pipeline built on AWS. The pipeline is designed to scrape web data using a Python script, process the data, and store the results in a CSV file in an S3 bucket. The pipeline is triggered every day at midnight. It leverages several AWS services including EventBridge, Step Functions, Lambda, EC2, SSM

glue-code

AWS Glue Codes (Sample Repo)

gq-great-expectations

Great Expectations Data Quality Checks is a specialized repository designed to harness the capabilities of the great_expectations Python library. With a focus on ensuring data quality, this project provides robust tools and methodologies to validate and check data across various sources.

great-expectations-dq

This script performs a series of data quality checks and generates a data profiling report. It utilizes Great Expectations for data validation and ydata_profiling for data profiling. The script reads data from a PostgreSQL database, applies various quality checks, and outputs validation results and a data profile HTML report.

kachow

Config files for my GitHub profile.

kinesis-lambda

This repo consists of python lambdas for reading and writing an AWS kinesis stream.

lambda_layer_maker

This is a script to make create, update and ready AWS Lambda layers you have in your account.

nested-dataframe-flattener

Flattens a complex nested pyspark dataframe having a combination of struct and array types

postgres-mongo-migrator

This repository contains a Python script for migrating data from a PostgreSQL database to a MongoDB database. The script is designed to be robust and fault-tolerant, capable of handling large datasets.

python-code-snippets

Code snippets on how to use different features in Python 3

python-one-liners

This repository contains python one-liners obtained from various sources.

python-oneliners

Some of the Python One-Liners which I regularly use and feel saves a lot of time.

sqlglot

Python SQL Parser and Transpiler

teams-pipeline

Set of scripts and instructions for setting up a Microsoft Teams webhook pipeline with potential Azure functions integration. Includes automation scripts, database structures, and a comprehensive setup guide.