iyyappana / machine-learning

Python machine learning applications in image processing and algorithm implementations including Expectation Maximization, Factor Analysis, Gaussian Mixture Model, OPTICS, DBSCAN, Random Forest, Decision Tree, Support Vector Machine, Independent Component Analysis, Latent Semantic Indexing, Principal Component Analysis, Singular Value Decomposition, K Nearest Neighbors, K Means, Naïve Bayes Mixture Model, Gaussian Discriminant Analysis, Newton Method, Gradient Descent

Machine Learning

Intro

Machine learning is so chic that every programmer even non-programmer starts to learn. After several months of online courses, everyone becomes self-proclaimed data scientist. The managers hold high hopes and deploy data scientists to machine learning this or that. In no time, people run into cul-de-sac, things don't work so well outside of the realm of iris dataset! If you have been to my other repositories like quant trading or graph theory, you must have seen me bashing reckless applications of machine learning. Stop selling AI snake oil! Don't get me wrong. I ain't no machine-learning-sceptic. I see great potential in machine learning but I am merely cynical to the current overstatement of artificial intelligence where it is frankly nowhere in sight.

The most popular supervised learning has very rigid requirement in both data quality and data quantity. Reinforcement learning is a drain on existing hardware. On the contrary, unsupervised learning is something I mess around frequently. It greatly boosts my work efficiency by dimension reduction, although I struggle to interpret the substantial meaning of the clustering pattern from time to time. In short, machine learning is no panacea. Its strongest suit is classification with discrete answers. When it comes to predicting stock price tomorrow or computing basic reproduction number yesterday, we still have to take the conventional path.

This repository is based upon the course material by Stanford University. Professor Andrew Ng may not teach the most comprehensive lectures but he has inspired millions to study data science. This repository attempts to replicate every algorithm mentioned in the course as well as the popular ones outside of the course. The experienced coders urge us not to reinvent the wheel but I firmly believe we never truly understand how a wheel works until we reinvent it. If you only learn OPTICS from some articles on towardsdatascience.com, you would've skipped DBSCAN since OPTICS does not require the key input ε. Well, by reinventing the wheels, you would come to senses that this is purely quid pro quoi. The introduction of new input ξ is crucial to determine the clustering. Yet, few people talk about it. In that sense, data modelling is not really scientific and will never be that way. Machine learning is a state of art where you fine tune the parameters to create discrete answers to the real-life problems. I sincerely hope this repository can help you see that.

---

Algorithms

Supervised

• Generative Learning (Gaussian Discriminant Analysis)

• Gradient Descent (Batch/Stochastic/Mini-batch)

• Naïve Bayes (Multivariate/Multinomial/TF-IDF/KL Divergence)

• Newton Method (Logistic Regression)

• Support Vector Machine (Binary/Multiclass/DAG/SMO)

• Tree-based Learning (Decision Tree/Random Forest)

• Instance-based Learning (K Nearest Neighbors)

Unsupervised

• Centroid-based Model (K Means)

• Density-based Model (DBSCAN/OPTICS)

• Distribution-based Model (Gaussian Mixture Model/Naïve Bayes Mixture Model)

• Expectation Maximization (Factor Analysis)

• Principal Component Analysis (Singular Value Decomposition/Low Rank Approximation/Latent Semantic Indexing)

• Signal Processing (Independent Component Analysis)

Applications

1. Reverse Engineering project

Creating a visualization from data is easy. In Tableau, it's only one click. What happens if you want to extract data from a visualization? A simple google search yields a few reverse engineering tools, yet they share the same malaise – they only work with single curve and require a lot of clicks. This project addresses these issues by incorporating unsupervised learning into image processing. Multiple curves are separated by different color channels with clustering techniques. Data can be easily extracted via computing coordinates of each pixel. A simple conversion from resolution scale to axis scale approximates the coordinates to the original spreadsheet. Voila, no more ridiculous subscription to Statista 😲

For more details, please refer to the read me page of a separate directory or machine learning section on my personal blog.