soopertramp / ds-end-to-end-project

Data science end to end project

Hello I'm Pradeepchandra Reddy S C

👀 I’m interested in:

 Data Science and Machine learning, 
 particulary interested in Data Wrangling (Cleaning the data fascinates me), 
 Exploratory Data Analysis (Without this how can i Live)
 and Mostly in the field of Classical Machine Learning (For Now)

🌱 I’m currently learning and Working on:

 Machine learning Models and Working on some capstone projects.

💞️ I’m looking to collaborate on:

 Any data science and machine learning related projects

📫 How to reach me:

 Mail - [email protected]
 
 Twitter - @soopertramp07

 Instagram - @soopertramp07

✍️ Check Out My Articles:

🌐 Check Out My Portfolio:

📝 My Practice:

⬆️ It's Currently a Private Repository

📂 My Projects:

☎️ Connect with me:

🛠 Languages and Tools:

📖 Learning

• • • •

ds-end-to-end-project's Issues

database_v1&v2_creation

Create Issue
Create New branch
Clean the script
Add functions wherever necessary in utils
Use those functions in data base script
Then push the code
Then create database_v2 script
Then Push the code
Then create a pull request add you as reviewer
Then after you’ll give approval then I merge with main
Then delete the branch

PR Guidline

Here are few things to check before creating a PR, after merging PR, and when you are working on new features to add to your main (production) branch:

• First create an issue
• Create a new branch and start coding
• Push changes to the respective branch
• When done with all coding and ready to push to production -> Create a pull request (PR), add reviewers
• You can still continue to code and push changes to your branch after creating PR until the PR is merged by you
• Merge PR after approval
• Delete branch on PR
• Go to your local repo -> switch to main branch -> git pull origin main -> then delete the same branch

Data Science part of the Project

1. write sql query to merge 3 tables into a single big table - validate this with original csv file

2. figure out how to run sql script from within python file. upload_to_s3.py -> this file will read sql script and run the sql script

3. how to upload files to s3 - python function

4. how to read files from s3 - python function

Database Script

Create a database script:

• create database
• create tables in the database
• insert data into the tables