mortiniera / data_science_portfolio

Data Science Portfolio completed both in the academia and for self-learning.

Data Science Portfolio

This Portfolio contains diverse projects in Data Science completed both in the academia and in self-learning fashion.

Organisation

1. Machine Learning

• Telco Customer Churn Problem : Data analysis and Machine Learning to solve Telco customer churn. Blog article

• Boston Houses : Linear regression tutorial comparing from scratch numpy/pandas implementation against scikit-learn to predict the best price that a client can sell his house. Blog article

• Higgs Boson Prediction : Predict if a signature results from a Higgs Boson (signal) or some other process (background). All ML models are implementended from scratch using only Numpy.

• tweet Sentiment Classification : Positive/Negative sentiment analysis classification of tweets using traditional machine learning techniques.

2. Deep Learning

• Collaborative Image Inpainting : Use collaborative sampling scheme to improve the quality of the generated inpainted images in deep generative adversarial networks. (GANs)

• Image Classification : Transfer learning using InceptionV3 module for 3-class image classification in tensorflow/keras. Google Colab notebook

• Reinforcement Learning :Lunar Lander : Simple implementation of Reinforce Algorithm to teach an agent to land on the moon using policy gradient. (OpenAI Gym)

3. Systems, Databases and Data-Parallel Programming

• hogwild-spark : Parallel synchronous and asynchronous versions of SGD in Spark on Kubernetes for learning support-vector machines.

• hogwild-multicore : Multicore implementation of the sparse SVM problem

• Robust-Journey Planner : Public transport route planner based on CSA

4. Data Analysis and NLP

• brazilian-e-commerce : Data-driven strategic analysis of Olist, brazilian E-commerce platform.

• Fake News : Fake News : An average Citizen's guide

data_science_portfolio's Issues

Data Normalization is before splitting but should be after splitting

Hello.

I find your code very useful but I guess I found a mistake.
Data Normalization should be after splitting, is not it?
If you do that before, then the test set will know information of the whole dataset (something that we would like to hide) and the evaluation of the model would be very optimistic.

I found some more information here:

https://datascience.stackexchange.com/questions/54908/data-normalization-before-or-after-train-test-split

"Normalization across instances should be done after splitting the data between training and test set, using only the data from the training set".