sjchoi86 / bayes-nn

Lecture notes on Bayesian deep learning

Understanding Bayesian Deep Learning

1. Elementary mathematics

• Set theory
• Measure theory
• Probability
• Random variable
• Random process
• Functional analysis (harmonic analysis)

2. Gaussian process

• Gaussian process
• Weight-space view
• Function-space view
• Gaussian process latent variable model

3. Bayesian neural netwrok

• Minimum description length
• Ensemble learning in Bayesian neural network
• Practical variational inference
• Bayes by backprop
• Summary of variational inference
• Dropout as a Bayesian approximation
• Stein variational gradient descent

4. Summary

• Measure thoery
• Probability
• Random variable
• Random process
• Gaussian process
• Functional Analysis
• Summary of variational inference
• Stein variational gradient descent

5. Uncertainty in Deep Learning

• Yarin Gal, Uncertainty in Deep Learning
• Anonymous, Bayesian Uncertainty Estimation for Batch Normalized Deep Networks
• Patrick McClure, Representing Inferential Uncertainty in Deep Neural Networks through Sampling
• Balaji Lakshminarayanan, Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles
• Alex Kendal, What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
• Gregory Kahn, Uncertainty-Aware Reinforcement Learning for Collision Avoidance
• Charles Richter, Safe Visual Navigation via Deep Learning and Novelty Detection
• Sungjoon Choi, Uncertainty-Aware Learning from Demonstration Using Mixture Density Networks with Sampling-Free Variance Modeling

bayes-nn's Issues

강의질문

1. p47에서 K가 hidden layer갯수라고 말씀하셨는데 강의노트에 number of hidden units in the hidden layer라고 했는데 노트에있는게 맞는건가요?

2. p48에서 F가 정확히 무엇을 의미하나요?

3. p51에서 Fi Fi^T + tau ^(-1)로 분산에서 왜 더한건가요..?