yihanjiang / representation_learning_course

Some Materials about SP 18 course

Representation_Learning_Course

Some Materials about SP 18 course

Module 1: Unsupervised Learning: Dimension Reductio and Tensor Methods

1. The first homework will be a comprehensive examples on dimension reduction. kernel PCA/CCA Renyi Correlation(ACE) IsoMap, tSNE. Some practical examples on that.

2. Latent is continuous and discrete, Mixture of Gaussian, Tensor Method package: tensorly (1) HMM on tensor method. (2) Mixture of Gaussian on tensor method. (3) EM convergence proof.

3. Modern Unsupervised Learning: Ganerative Models The 4th module will be on AE and GAN. AE: AE, VAE, AAE, WAE. GAN: WGAN, ConditionalGAN, DualGAN.

Muodule 2: Supervised Learning: Boosting Tree Methods

The second homework will be on using Kaggle's nuclear weapon: XGBoost and Random Forest.

1. Boosting(proof, and how it works,,), CART. Random Forest. XGBoost, as homework. also proofs.

2. Mixture of expert.

Module 3: Supervised Learning: Neural Network

The third homework wil be on basic neural networks, FC-NN, simple CNN, simple RNN.

Also something about optimizations as homework example.

Architecture: NN: CNN+ pooling, Inception v2,v3, v4.

BP & Architecture, loss function design, training algorithms. (1)Training(optimization), Rahul, SGD++, gradient flow. (2)Proof of representation learning: (tensor, landscape design(GLU). ) (3) Escaping saddle point. (sanjeev, jin chi, ) (4) generalization,

representation theorem of NN.

Homework:

practical: generative model: generate with NN, decode with NN. -> everywhere. figure out what it is.

Find the signature of the data, to decide the model.

Module 4: Modern Problems

Adversial Examples Interpretability Generalization Theory on Neural Networks Tensor/ Landscape Design Graph Neural Networks

Final Projects

The final projects will be open. Could be (1) bio-related problem with Deep Learning (2) theoritical problems with Deep Learning.