quyanh2005 / homemade-machine-learning

Understand and code some basic algorithms in machine learning from scratch

homemade-machine-learning

Implement some ML algorithms from scratch.

Content

1. Linear Regression

• Linear Regression
  • Model
  • Least Square Method
  • Test Your Model
• Gradient with Linear Regression
  • Gradient Formula
  • Implement
  • Apply Model

2. Logistic Regression

• Regression and Sigmoid
• Cost function and Gradient
  • Cost function
  • Partial derivative of $J(\theta)$
  • Update the weights
  • Implement gradient descent function
• Training model
• Test model
  • Your model
  • Sklearn model

3. Principle Component Analysis

• Why Need to Reduce Dimensionality?
• Main Approaches for Dimensionality Reduction
  • Projection
  • Manifold Learning
• Principle Compent Analysis
  • Preserving the Variance
  • Principal Components
  • Projecting Down to d Dimensions
• PCA in Sklearn

4. K-means Clustering

• What is k-means clustering?
• Pros and cons of k-means
• K-means algorithm and implementation
• K-means algorithm in sklearn

5. Decision Tree

• What is Decision Tree?
• Important Terminology related to Decision Trees
• How do Decision Trees work?
• ID3 Algorithm
  • Entropy
  • Infomation Gain
  • Split a node into branches
• Model from scratch and predict
• Decision Tree in sklearn and some notes
  • CART cost function for classification
  • CART cost function for regression
• Pros and cons

6. Support Vector Machines

• SVMs - Gradient Descent
  • What is SVMs?
  • Primal Support Vector Machine
    • Distance between Two Parallel Lines
    • Optimal Hyperplane
  • Hard Margin
  • Soft Margin
  • Solve SVMs by Gradient Descent
    • Hard Margin by Gradient Descent
    • Soft Margin by Gradient Descent
• SVMs - Lagrange Method
  • Review about Primal Support Vector Machine
  • Dual Support Vector Machine
  • Hard - Soft Margin
    • Implementation
    • Sklearn
  • Kernel SVM
    • Implementation
    • Sklearn
  • Visualization
  • References

7. Naive Bayes

• Gaussian Naive Bayes
  • Naive Bayes Rule
  • Gaussian Naive Bayes
  • Gaussian Naive Bayes Model from Scratch
  • Gaussian Naive Bayes Model in Sklearn
• Multinomial Naive Bayes
  • Multinomial Naive Bayes
  • Multinomial Naive Bayes Model from Scratch
  • Multinomial Naive Bayes Model in Sklearn
  • Multinomial Naive Bayes for Out of Vocabulary

8. Random Forest

• Bagging
• What is Random Forest?
• Important Features of Random Forest
• Important Hyperparameters
• Pseudo-code
• Implementation
• Advantages and Disadvantages
• References

9. Gradient Boosting

• Introduction
  • Definnition of Gradient Boosting
  • Overview of ensemble learning
• Theoretical foundations of Gradient Boosting
  • Introduction to decision trees
  • Mathematical formulation of Gradient Boosting
• Implementation
• Pros and cons

Other Algorithms

Besides the above algorithms, I have researched some other algorithms and will publish as soon as possible. A list of algorithms:

• SVD
• LDA
• DBSCAN
• ...

Contact

If you have any problems, please contact me:

• Email-1: [email protected]
• Email-2: [email protected]