robertou2 / stat453-deep-learning-ss20

STAT 453: Intro to Deep Learning @ UW-Madison (Spring 2020)

STAT 453: Introduction to Deep Learning and Generative Models

Course Website: http://pages.stat.wisc.edu/~sraschka/teaching/stat453-ss2020/

Topics Summary (Planned)

Below is a list of the topics I am planning to cover. Note that while these topics are numerated by lectures, note that some lectures are longer or shorter than others. Also, we may skip over certain topics in favor of others if time is a concern. While this section provides an overview of potential topics to be covered, the actual topics will be listed in the course calendar at the bottom of the course website.

Part 1: Introduction

• L01: Course overview, introduction to deep learning
• L02: The brief history of deep learning
• L03: Single-layer neural networks: The perceptron algorithm

Part 2: Mathematical and computational foundations

• L04: Linear algebra and calculus for deep learning
• L05: Parameter optimization with gradient descent
• L06: Automatic differentiation & PyTorch
• L06.5: Cloud Computing [ Recording ]

Part 3: Introduction to neural networks

• L07: Multinomial logistic regression
• L08: Multilayer perceptrons
• L09: Regularization [ Recording ]
• L10: Input normalization and weight initialization [ Recording Part 1/2 ] [ Recording Part 2/2 ]
• L11: Learning rates and advanced optimization algorithms [ Recording ]

Part 4: Deep learning for computer vision and language modeling

• L12: Introduction to convolutional neural networks 1 [ Recording ]
• L13: Introduction to convolutional neural networks 2 [ Recording 1/2 ] [ Recording 2/2 ]
• L 14: Introduction to recurrent neural networks 1 [ Recording ]
• Introduction to recurrent neural networks 2

Part 5: Deep generative models

• Autoencoders
• Autoregressive models
• Variational autoencoders
• Normalizing Flow Models
• Generative adversarial networks 1
• Generative adversarial networks 2
• Evaluating generative models

Part 6: Class projects and final exam

• Student project presentations 1
• Student project presentations 2
• Student project presentations 3
• Final exam
• Final report (online submission)