roatienza / ml

Foundations of Machine Learning Course Materials.

Foundations of Machine Learning Course Materials.

This is the repository of my course in Foundations of Machine Learning (EE 298M/CoE 197M). Similar to my previous course in Deep Learning, I would like to strike a balance between theory and practical implementation of concepts. The course materials are still under development. Please expect occasional errors, missing parts, bugs, etc. Apologies in advance.

With some modifications, the theory part is based on Mathematics for Machine Learning book. It is freely available at this link.

Thank you! I greatly appreciate the authors for having the book available online. This is helpful for students who have limited resources.

As much as possible, code examples are written to better understand key concepts. The code examples in this course are in Jupyter Notebook. I tried using Google Colab but encountered errors in saving my notebooks. So, I switched to Jupyter Notebook. In theory, you can upload the notebooks here to Google Colab.

Roadmap

0. Why Machine Learning - Importance of Foundations of Machine Learning, Course Outline
   • Slides
1. ML Toolkit - Environment, Code Editor, Python, Numpy, Matplotlib, etc
   • Slides
2. Linear Algebra - Tensors, Operations, Basis, Rank, Spaces/Subspaces, Groups, Linear Mappings
   • Slides
   • Notebook (Code Examples):
     • Introduction to Numpy
     • Linear Regression
     • Basis Vectors Transformation
3. Analytic Geometry - Distance, Metric, Norm, Inner Product, Basis, Projection, Gram-Schmidt, Rotation
   • Slides
4. Matrix Decomposition - Eigenvalues, Eigenvector, Eigendecomposition, Spectral Theorem, Singular-Value Decomposition, Matrix Approximation
   • Slides
   • Notebook (Code Example):
     • Matrix Approximation
5. Vector Calculus - Learning, Taylor Series, Gradients, Jacobian, Backpropagation, Hessian
   • Slides
6. Probability Distributions - Distributions, Gaussian, Bayes, Sufficient Statistics, Exponential Family, Conjugacy, Transformation
   • Slides
7. Optimization - Gradient Descent, Stochastic Gradient Descent, Convex Optimization, Linear & Quadratic Programming, Convex Conjugate
   • Slides
8. Machine Learning Principles - Empirical Risk Minimization (ERM), Maximum Likelihood Estimation (MLE), Maximum A Posteriori (MAP), Intro to Information Theory, Directed Graphical Models
   • Slides
9. Linear Regression - MLE, MAP
   • Slides
   • Notebook (Code Example):
     • MLE and MAP
10. Principal Component Analysis - Low-dimensional Representation, Latent Variable Model

• Slides
• Notebook (Code Example):
  • MNIST PCA

11. Gaussian Mixture Models Responsibilities, Parameter Updates, Expectation Maximization

• Slides
• Notebook (Code Example):
  • GMM 2-component 2D

12. Support Vector Machines

Cheat Sheets

0. Numpy
1. Scikit-Learn
2. dplyr and tidyr
3. Neural Networks

Appreciation

If you find the materials in this repo useful, please give it a star or fork it.

Citation

If you find this work useful, please cite:

@misc{atienza2020ml,
  title={Foundations of Machine Learning},
  author={Atienza, Rowel},
  year={2020},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/roatienza/ml}},
}