GithubHelp home page GithubHelp logo

πŸ‘‹ Hello, Welcome to my profile.

I'm a passionate Big Data Engineer with 4+ years of hands-on experience in designing and implementing robust data solutions. My goal is to leverage the power of data to drive meaningful insights and solve complex business challenges.

πŸ’‘ Expertise:
In my journey as a Big Data Engineer, I have honed my skills in:

πŸ”Ή Big Data Technologies: I have a strong command over Hadoop, Spark and their ecosystems. I specialize in building scalable data pipelines, processing large datasets, and optimizing performance for efficient data processing.

πŸ”Ή Programming Languages: I am proficient in Python, SQL and Spark, using them to develop data-centric applications, perform data analysis, and build machine learning models.

πŸ”Ή Data Warehousing: I have hands-on experience with data warehousing principles, including data modeling, ETL (Extract, Transform, Load) processes, and dimensional modeling. I am well-versed in designing and implementing data warehouses for improved data accessibility and reporting.

πŸ”Ή Database Management: I have a strong grasp of SQL and have worked extensively with both relational databases (MySQL, PostgreSQL) and NoSQL databases (MongoDB, Cassandra). I excel at writing complex queries, optimizing database performance, and ensuring data integrity.

πŸ”Ή Cloud Platforms: I am adept at working with cloud-based environments, particularly on AWS and Azure.

πŸ”Ή Data Visualization: I possess a keen eye for visualizing data insights and effectively communicating complex findings to stakeholders. I am skilled in using tools like Tableau and Power BI to create intuitive dashboards and reports.

βš’οΈ Skills:

πŸ§‘β€πŸ’» Programming Languages:
Python | SQL | Spark

⛓️ Distributed Framework:
Spark | Hadoop | Hive | Kafka | Sqoop

πŸ’Ύ Databases:
MySQL | MongoDB | Cassandra | HBase

🧬 Version Control:
Git | DVC

⏰ Workflow Management:
Airflow | Mage

☁️ AWS Services:
S3 | EC2 | EMR | RDS | Redshift | Glue | CloudWatch | ECS

☁️ Azure Services:
Data Factory | Databricks | Functions | Blob | Synapse | Delta Lake

πŸš€ MLOps:
Docker | Docker Compose | GitHub Actions | MLflow

πŸͺ„ ML Frameworks:
Pandas | Numpy | Sklearn | PySpark | Pytorch | Matplotlib | Seaborn | TFX

🌐 Socials:

LinkedIn Topmate Resume Portfolio

πŸ“Š GitHub Stats:



✍️ Random Dev Quote

πŸ˜‚ Random Dev Meme


Vishal Singh's Projects

3d_portfolio icon 3d_portfolio

Let's build a cool 3D website together! You'll learn how to make a portfolio with fun interactive parts, like a floating island and a fox that moves when you type. It'll allow you to show off your skills and get a job!

act icon act

Run your GitHub Actions locally πŸš€

aim icon aim

Aim πŸ’« β€” easy-to-use and performant open-source ML experiment tracker.

aiohttp icon aiohttp

Asynchronous HTTP client/server framework for asyncio and Python

airbyte icon airbyte

Data integration platform for ELT pipelines from APIs, databases & files to warehouses & lakes.

airflow icon airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

alibi-detect icon alibi-detect

Algorithms for outlier, adversarial and drift detection

amplication icon amplication

Amplication is an open‑source development tool. It helps you develop quality Node.js applications without spending time on repetitive coding tasks.

anomalib icon anomalib

An anomaly detection library comprising state-of-the-art algorithms and features such as experiment management, hyper-parameter optimization, and edge inference.

ansible icon ansible

Ansible is a radically simple IT automation platform that makes your applications and systems easier to deploy and maintain. Automate everything from code deployment to network configuration to cloud management, in a language that approaches plain English, using SSH, with no agents to install on remote systems. https://docs.ansible.com.

antares icon antares

A modern, fast and productivity driven SQL client with a focus in UX

anytype-ts icon anytype-ts

Official Anytype client for MacOS, Linux, and Windows

applied-ml icon applied-ml

πŸ“š Papers & tech blogs by companies sharing their work on data science & machine learning in production.

appwrite icon appwrite

Secure Backend Server for Web, Mobile & Flutter Developers πŸš€ AKA the 100% open-source Firebase alternative.

architecturepatternswithpython icon architecturepatternswithpython

This repository contains code from the guide Architecture Patterns with Python, Enabling Test-Driven Development, Domain-Driven Design, and Event-Driven Microservices.

argilla icon argilla

✨ Open-source tool for data-centric NLP. Argilla helps domain experts and data teams to build better NLP datasets in less time.

arrow icon arrow

🏹 Better dates & times for Python

arrow-1 icon arrow-1

Apache Arrow is a multi-language toolbox for accelerated data interchange and in-memory processing

askai icon askai

Your simple terminal helper - A CLI integration with OpenAI's GPT3

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.