htnani / machine-learning

Machine learning course materials

---

Machine Learning

This course provides an introduction to machine learning. Machine learning explores the study and construction of algorithms that can learn from and make predictions on data. Such algorithms operate by building a model from example inputs in order to make data-driven predictions or decisions, rather than following strictly static program instructions.

Topic categories include supervised, unsupervised, and reinforcement learning. Students will learn how to apply machine learning methods to solve problems in computer vision, natural language processing, classification, and prediction. Fundamental and current state-of-the-art methods including boosting and deep learning will be covered. Students will reinforce their learning of machine learning algorithms with hands-on tutorial oriented laboratory exercises using Jupyter Notebooks.

Prerequisites: MA-262 Probability and Statistics; programming maturity, and the ability to program in Python.

Helpful: CS3851 Algorithms, MA-383 Linear Algebra, Data Science.

ABET: Math/Science, Engineering Topics.

2-2-3 (class hours/week, laboratory hours/week, credits)

Lectures are augmented with hands-on tutorials using Jupyter Notebooks. Laboratory assignments will be completed using Python and related data science packages: NumPy, Pandas, ScipPy, StatsModels, Scikit-learn, Matplotlib, TensorFlow, Keras, PyTorch.

Prerequisites: MA-262 Probability and Statistics; programming maturity, and the ability to program in Python.

Helpful: CS3851 Algorithms, MA-383 Linear Algebra, Data Science.

ABET: Math/Science, Engineering Topics.

Outcomes:

• Understand the basic process of machine learning.
• Understand the concepts of learning theory, i.e., what is learnable, bias, variance, overfitting.
• Understand the concepts and application of supervised, unsupervised, semi-supervised, and reinforcement learning.
• The ability to analyze a data set including the ability to understand which data attributes (dimensions) affect the outcome.
• Understand the application of learned models to problems in classification, prediction, clustering, computer vision, and NLP.
• Understand deep learning concepts and architectures including representation learning Multi-layer Perceptrons, Convolutional Neural Networks, Recurrent Neural Networks, and Attention Mechanisms.
• The ability to assess the quality of predictions and inferences.
• The ability to apply methods to real world data sets.

References:

Hands-On Machine Learning with Scikit-Learn and TensorFlow Concepts, Tools, and Techniques to Build Intelligent Systems, Aurélien Géron. O'Reilly Media, 2017

Deep Learning with Python, François Chollet. Manning, 2017.

Deep Learning, Ian Goodfellow, Yoshua Bengio, and Aaron Courville. MIT Press, 2016.

[An Introduction to Statistical Learning: with Applications in R]. Gareth James, Daniela Witten, Trevor Hastie, Robert Tibshirani. 2015 Edition, Springer.](http://www-bcf.usc.edu/~gareth/ISL/ISLR%20Seventh%20Printing.pdf)

Python Data Science Handbook, Jake VanderPlas, O'Reilly.

Mining of Massive Datasets. Anand Rajaraman and Jeffrey David Ullman. http://infolab.stanford.edu/~ullman/mmds.html

---

Week 1: Intro to Machine Learning, Linear Regression

Lecture:

1. Introduction to Machine Learning

• Demonstrations

2. Linear Regression

Lab Notebooks:

• Jupyter Notebooks
• Python Machine Learning Environment
• Python Programming Submission required

Optional tutorial notebooks:

• Dates and Time
• Python Objects, Map, Lambda, and List Comprehensions
• Python Numpy Submission required
• Python Numpy Aggregates
• Pandas Data Manipulation
• Python Reading and Writing CSV files
• Data Visualization

Outcomes addressed in week 1:

• Understand the basic process of machine learning:
• Understand the concepts and application of supervised, unsupervised, semi-supervised, and reinforcement learning.

---

Week 2: Gradient Descent Learning, Logistic Regression

Lecture:

1. Gradient Descent

• Normal Equation Derivation

2. Logistic Regression

Lab Notebooks:

• Gradient Descent Learning Submission required

Optional exercise:

Online Machine Learning with Stochastic Gradient Descent

Outcomes addressed in week 2:

• Understand the concepts of learning theory, i.e., what is learnable, bias, variance, overfitting.
• Understand the concepts and application of supervised, unsupervised, semi-supervised, and reinforcement learning.

---

Week 3: Model Selection and Regularization, Decision Trees

Lecture:

1. Model selection and regularization

2. Decision Trees, Overfitting, Model Selection, Techniques to Improve Classifier Accuracy Slides

• Information Gain Calculation Spreadsheet

Lab Notebooks:

Introduction to Machine Learning with SciKit Learn.ipynb
Supervised Learning - Linear Regression Submission required
Supervised Learning - Logistic Regression Submission required
Generalized Linear Models Optional

Outcomes addressed in week 3:

• Understand the concepts of learning theory, i.e., what is learnable, bias, variance, overfitting.
• Understand the concepts and application of supervised, unsupervised, semi-supervised, and reinforcement learning.
• The ability to analyze a data set including the ability to understand which data attributes (dimensions) affect the outcome.

---

Week 4: Probabilistic Models

Lecture:

1. Validation, Bagging, Boosting, Random Forests

2. Naive Bayes Slides 87-102

3. Bayesian Networks optional

• BayesianNetworkSprinklerExample.xls

Lab Notebooks:

• Decision Trees with Scikit Learn Submission required
• Random Forests and Boosting
• XGBoost

Outcomes addressed in week 4:

• Understand the concepts of learning theory, i.e., what is learnable, bias, variance, overfitting.
• Understand the concepts and application of supervised, unsupervised, semi-supervised, and reinforcement learning.
• The ability to analyze a data set including the ability to understand which data attributes (dimensions) affect the outcome.
• Understand the application of learned models to problems in classification, prediction, clustering, computer vision, and NLP.

---

Week 5: Unsupervised Learning

Lecture:

1. K-Means Clustering
2. Hierarchical Agglomerative and Probabilistic Clustering

Lab Notebooks:

• Clustering Submission required

Outcomes addressed in week 5:

• Understand the concepts and application of supervised, unsupervised, semi-supervised, and reinforcement learning.
• The ability to analyze a data set including the ability to understand which data attributes (dimensions) affect the outcome.
• Understand the application of learned models to problems in classification, prediction, clustering, computer vision, and NLP.

---

Week 6: Midterm, Dimensionality Reduction

Lecture:

1. Dimensionality Reduction

2. Midterm exam

Lab Notebooks:

• SVD Hands-on only
• PCA Hands-on only

Outcomes addressed in week 6:

• Understand the concepts and application of supervised, unsupervised, semi-supervised, and reinforcement learning.
• The ability to analyze a data set including the ability to understand which data attributes (dimensions) affect the outcome.

---

Week 7: Introduction to Neural Networks and Deep Learning

Lecture:

1. Deep Learning Introduction 1

2. Deep Learning Introduction 2

3. Backpropagation

Lab Notebooks:

Option 1

• Introduction to TensorFlow Submission required
• Neural Network Fundamentals Submission required
  Option 2
• Introduction to TensorFlow Submission required
• Neural Network Fundamentals Submission required

Note: need to prune answers from notebooks

Outcomes addressed in week 8:

• Understand the concepts of learning theory, i.e., what is learnable, bias, variance, overfitting.
• Understand the concepts and application of supervised, unsupervised, semi-supervised, and reinforcement learning.
• Understand the application of learned models to problems in classification, prediction, clustering, computer vision, and NLP.
• Understand deep learning concepts and architectures including representation learning Multi-layer Perceptrons, Convolutional Neural Networks, Recurrent Neural Networks, and Attention Mechanisms.

Week 8: Deep Learning for Computer Vision

Lecture:

1. Deep Learning for Computer Vision

2. Convnets

Lab Notebooks:

Keras Intro Submission required
Image Classification Submission required

Note: need to prune answers from notebooks

Outcomes addressed in week 9:

• Understand the concepts of learning theory, i.e., what is learnable, bias, variance, overfitting.
• Understand the concepts and application of supervised, unsupervised, semi-supervised, and reinforcement learning.
• Understand the application of learned models to problems in classification, prediction, clustering, computer vision, and NLP.
• Understand deep learning concepts and architectures including representation learning Multi-layer Perceptrons, Convolutional Neural Networks, Recurrent Neural Networks, and Attention Mechanisms.

---

Week 9: Deep Learning for NLP

Lecture:

1. NLP Classification

2. Optional Convnets for Structured Prediction

3. NLP Translation

Lab Notebooks:

• NLP Classification
• NLP Translation

Note: need to prune answers from notebooks

Outcomes addressed in week 9:

• Understand the concepts of learning theory, i.e., what is learnable, bias, variance, overfitting.
• Understand the concepts and application of supervised, unsupervised, semi-supervised, and reinforcement learning.
• Understand the application of learned models to problems in classification, prediction, clustering, computer vision, and NLP.
• Understand deep learning concepts and architectures including representation learning Multi-layer Perceptrons, Convolutional Neural Networks, Recurrent Neural Networks, and Attention Mechanisms.

---

Week 10:  Generative Deep Learning

Lecture:

1. Future Challenges

2. Auto-encoder, Variational Auto-encoder, GANs TBD: probably too advanced

Lab Notebooks:

Complete assignments

Outcomes addressed in week 10:

• Understand the concepts of learning theory, i.e., what is learnable, bias, variance, overfitting.
• Understand the concepts and application of supervised, unsupervised, semi-supervised, and reinforcement learning.
• Understand the application of learned models to problems in classification, prediction, clustering, computer vision, and NLP.
• Understand deep learning concepts including representation learning.

---