A collection of various deep learning architectures, models, and tips for TensorFlow and PyTorch in Jupyter Notebooks.
Traditional Machine Learning
Title |
Dataset |
Description |
Notebooks |
Perceptron |
2D toy data |
TBD |
![TensorFlow](https://camo.githubusercontent.com/0456829b900fc19fc10eabe32058e9941297c43dc8624d6bf448af883c89ada3/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f54656e736f722d466c6f77312e302d6f72616e6765) |
Logistic Regression |
2D toy data |
TBD |
![TensorFlow](https://camo.githubusercontent.com/0456829b900fc19fc10eabe32058e9941297c43dc8624d6bf448af883c89ada3/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f54656e736f722d466c6f77312e302d6f72616e6765) |
Softmax Regression (Multinomial Logistic Regression) |
MNIST |
TBD |
![TensorFlow](https://camo.githubusercontent.com/0456829b900fc19fc10eabe32058e9941297c43dc8624d6bf448af883c89ada3/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f54656e736f722d466c6f77312e302d6f72616e6765) |
Softmax Regression with MLxtend's plot_decision_regions on Iris |
Iris |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Title |
Dataset |
Description |
Notebooks |
Multilayer Perceptron |
MNIST |
TBD |
![TensorFlow](https://camo.githubusercontent.com/0456829b900fc19fc10eabe32058e9941297c43dc8624d6bf448af883c89ada3/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f54656e736f722d466c6f77312e302d6f72616e6765) |
Multilayer Perceptron with Dropout |
MNIST |
TBD |
![TensorFlow](https://camo.githubusercontent.com/0456829b900fc19fc10eabe32058e9941297c43dc8624d6bf448af883c89ada3/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f54656e736f722d466c6f77312e302d6f72616e6765) |
Multilayer Perceptron with Batch Normalization |
MNIST |
TBD |
![TensorFlow](https://camo.githubusercontent.com/0456829b900fc19fc10eabe32058e9941297c43dc8624d6bf448af883c89ada3/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f54656e736f722d466c6f77312e302d6f72616e6765) |
Multilayer Perceptron with Backpropagation from Scratch |
MNIST |
TBD |
![TensorFlow](https://camo.githubusercontent.com/0456829b900fc19fc10eabe32058e9941297c43dc8624d6bf448af883c89ada3/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f54656e736f722d466c6f77312e302d6f72616e6765) |
Convolutional Neural Networks
Title |
Dataset |
Description |
Notebooks |
Convolutional Neural Network |
TBD |
TBD |
![TensorFlow](https://camo.githubusercontent.com/0456829b900fc19fc10eabe32058e9941297c43dc8624d6bf448af883c89ada3/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f54656e736f722d466c6f77312e302d6f72616e6765) |
CNN with He Initialization |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Title |
Dataset |
Description |
Notebooks |
Replacing Fully-Connnected by Equivalent Convolutional Layers |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Title |
Dataset |
Description |
Notebooks |
AlexNet Trained on CIFAR-10 |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
AlexNet with Grouped Convolutions Trained on CIFAR-10 |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Title |
Description |
Daset |
Notebooks |
DenseNet-121 Digit Classifier Trained on MNIST |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
DenseNet-121 Image Classifier Trained on CIFAR-10 |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Title |
Dataset |
Description |
Notebooks |
"All Convolutionl Net" -- A Fully Convolutional Neural Network |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Title |
Dataset |
Description |
Notebooks |
LeNet-5 on MNIST |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
LeNet-5 on CIFAR-10 |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
LeNet-5 on QuickDraw |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Title |
Dataset |
Description |
Notebooks |
MobileNet-v2 on Cifar-10 |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
MobileNet-v3 small on Cifar-10 |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
MobileNet-v3 large on Cifar-10 |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
MobileNet-v3 large on MNIST via Embetter |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Title |
Dataset |
Description |
Notebooks |
Network in Network Trained on CIFAR-10 |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Title |
Dataset |
Description |
Notebooks |
Convolutional Neural Network VGG-16 Trained on CIFAR-10 |
TBD |
TBD |
![TensorFlow](https://camo.githubusercontent.com/0456829b900fc19fc10eabe32058e9941297c43dc8624d6bf448af883c89ada3/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f54656e736f722d466c6f77312e302d6f72616e6765) |
VGG-16 Smile Classifier |
CelebA |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
VGG-16 Dogs vs Cats Classifier |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Convolutional Neural Network VGG-19 |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Title |
Dataset |
Description |
Notebooks |
ResNet and Residual Blocks |
MNIST |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
ResNet-18 Digit Classifier |
MNIST |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
ResNet-18 Gender Classifier |
CelebA |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
ResNet-34 Digit Classifier |
MNIST |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
ResNet-34 Object Classifier |
QuickDraw |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
ResNet-34 Gender Classifier |
CelebA |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
ResNet-50 Digit Classifier |
MNIST |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
ResNet-50 Gender Classifier |
CelebA |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
ResNet-101 Gender Classifier |
CelebA |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
ResNet-101 |
CIFAR-10 |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
ResNet-152 Gender Classifier |
CelebA |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Title |
Dataset |
Description |
Notebooks |
Multilabel DistilBERT |
Jigsaw Toxic Comment Challenge |
DistilBERT classifier fine-tuning |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
DistilBERT as feature extractor |
IMDB movie review |
DistilBERT classifier with sklearn random forest and logistic regression |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
DistilBERT as feature extractor using embetter |
IMDB movie review |
DistilBERT classifier with sklearn random forest and logistic regression using the scikit-learn embetter library |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Fine-tune DistilBERT I |
IMDB movie review |
Fine-tune only the last 2 layers of DistilBERT classifier |
![PyTorch Lightning](https://camo.githubusercontent.com/ba467d39cf3f58b0e9bf88d3cb039f08a7e3792cd1ad5cd35b0a4ee390801306/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f5079546f7263682d4c696768746e696e672d626c756576696f6c6574) |
Fine-tune DistilBERT II |
IMDB movie review |
Fine-tune the whole DistilBERT classifier |
![PyTorch Lightning](https://camo.githubusercontent.com/ba467d39cf3f58b0e9bf88d3cb039f08a7e3792cd1ad5cd35b0a4ee390801306/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f5079546f7263682d4c696768746e696e672d626c756576696f6c6574) |
Ordinal Regression and Deep Learning
Please note that the following notebooks below provide reference implementations to use the respective methods. They are not performance benchmarks.
Title |
Dataset |
Description |
Notebooks |
BatchNorm before and after Activation for Network-in-Network CIFAR-10 Classifier |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Filter Response Normalization for Network-in-Network CIFAR-10 Classifier |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Title |
Dataset |
Description |
Notebooks |
Siamese Network with Multilayer Perceptrons |
TBD |
TBD |
![TensorFlow](https://camo.githubusercontent.com/0456829b900fc19fc10eabe32058e9941297c43dc8624d6bf448af883c89ada3/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f54656e736f722d466c6f77312e302d6f72616e6765) |
Fully-connected Autoencoders
Title |
Dataset |
Description |
Notebooks |
Autoencoder (MNIST) |
TBD |
TBD |
![TensorFlow](https://camo.githubusercontent.com/0456829b900fc19fc10eabe32058e9941297c43dc8624d6bf448af883c89ada3/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f54656e736f722d466c6f77312e302d6f72616e6765) |
Autoencoder (MNIST) + Scikit-Learn Random Forest Classifier |
TBD |
TBD |
![TensorFlow](https://camo.githubusercontent.com/0456829b900fc19fc10eabe32058e9941297c43dc8624d6bf448af883c89ada3/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f54656e736f722d466c6f77312e302d6f72616e6765) |
Convolutional Autoencoders
Title |
Dataset |
Description |
Notebooks |
Convolutional Autoencoder with Deconvolutions / Transposed Convolutions |
TBD |
TBD |
![TensorFlow](https://camo.githubusercontent.com/0456829b900fc19fc10eabe32058e9941297c43dc8624d6bf448af883c89ada3/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f54656e736f722d466c6f77312e302d6f72616e6765) |
Convolutional Autoencoder with Deconvolutions and Continuous Jaccard Distance |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Convolutional Autoencoder with Deconvolutions (without pooling operations) |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Convolutional Autoencoder with Nearest-neighbor Interpolation |
TBD |
TBD |
![TensorFlow](https://camo.githubusercontent.com/0456829b900fc19fc10eabe32058e9941297c43dc8624d6bf448af883c89ada3/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f54656e736f722d466c6f77312e302d6f72616e6765) |
Convolutional Autoencoder with Nearest-neighbor Interpolation -- Trained on CelebA |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Convolutional Autoencoder with Nearest-neighbor Interpolation -- Trained on Quickdraw |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Title |
Dataset |
Description |
Notebooks |
Variational Autoencoder |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Convolutional Variational Autoencoder |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Conditional Variational Autoencoders
Title |
Dataset |
Description |
Notebooks |
Conditional Variational Autoencoder (with labels in reconstruction loss) |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Conditional Variational Autoencoder (without labels in reconstruction loss) |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Convolutional Conditional Variational Autoencoder (with labels in reconstruction loss) |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Convolutional Conditional Variational Autoencoder (without labels in reconstruction loss) |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Generative Adversarial Networks (GANs)
Title |
Dataset |
Description |
Notebooks |
Fully Connected GAN on MNIST |
TBD |
TBD |
![TensorFlow](https://camo.githubusercontent.com/0456829b900fc19fc10eabe32058e9941297c43dc8624d6bf448af883c89ada3/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f54656e736f722d466c6f77312e302d6f72616e6765) |
Fully Connected Wasserstein GAN on MNIST |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Convolutional GAN on MNIST |
TBD |
TBD |
![TensorFlow](https://camo.githubusercontent.com/0456829b900fc19fc10eabe32058e9941297c43dc8624d6bf448af883c89ada3/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f54656e736f722d466c6f77312e302d6f72616e6765) |
Convolutional GAN on MNIST with Label Smoothing |
TBD |
TBD |
![TensorFlow](https://camo.githubusercontent.com/0456829b900fc19fc10eabe32058e9941297c43dc8624d6bf448af883c89ada3/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f54656e736f722d466c6f77312e302d6f72616e6765) |
Convolutional Wasserstein GAN on MNIST |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Deep Convolutional GAN (DCGAN) on Cats and Dogs Images |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Deep Convolutional GAN (DCGAN) on CelebA Face Images |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Graph Neural Networks (GNNs)
Title |
Dataset |
Description |
Notebooks |
Most Basic Graph Neural Network with Gaussian Filter on MNIST |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Basic Graph Neural Network with Edge Prediction on MNIST |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Basic Graph Neural Network with Spectral Graph Convolution on MNIST |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Recurrent Neural Networks (RNNs)
Many-to-one: Sentiment Analysis / Classification
Title |
Dataset |
Description |
Notebooks |
A simple single-layer RNN (IMDB) |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
A simple single-layer RNN with packed sequences to ignore padding characters (IMDB) |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
RNN with LSTM cells (IMDB) |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
RNN with LSTM cells (IMDB) and pre-trained GloVe word vectors |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
RNN with LSTM cells and Own Dataset in CSV Format (IMDB) |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
RNN with GRU cells (IMDB) |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Multilayer bi-directional RNN (IMDB) |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Bidirectional Multi-layer RNN with LSTM with Own Dataset in CSV Format (AG News) |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Many-to-Many / Sequence-to-Sequence
Title |
Dataset |
Description |
Notebooks |
A simple character RNN to generate new text (Charles Dickens) |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Title |
Dataset |
Description |
Notebooks |
Baseline CNN |
MNIST |
A simple baseline with traditional train/validation/test splits |
![PyTorch Lightning](https://camo.githubusercontent.com/ba467d39cf3f58b0e9bf88d3cb039f08a7e3792cd1ad5cd35b0a4ee390801306/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f5079546f7263682d4c696768746e696e672d626c756576696f6c6574) |
K-fold with pl_cross |
MNIST |
A 5-fold cross-validation run using the pl_cross library |
![PyTorch Lightning](https://camo.githubusercontent.com/ba467d39cf3f58b0e9bf88d3cb039f08a7e3792cd1ad5cd35b0a4ee390801306/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f5079546f7263682d4c696768746e696e672d626c756576696f6c6574) |
Title |
Dataset |
Description |
Notebooks |
Cyclical Learning Rate |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Annealing with Increasing the Batch Size (w. CIFAR-10 & AlexNet) |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Gradient Clipping (w. MLP on MNIST) |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Title |
Dataset |
Description |
Notebooks |
Transfer Learning Example (VGG16 pre-trained on ImageNet for Cifar-10) |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Visualization and Interpretation
Title |
Dataset |
Description |
Notebooks |
Vanilla Loss Gradient (wrt Inputs) Visualization (Based on a VGG16 Convolutional Neural Network for Kaggle's Cats and Dogs Images) |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Guided Backpropagation (Based on a VGG16 Convolutional Neural Network for Kaggle's Cats and Dogs Images) |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
PyTorch Workflows and Mechanics
PyTorch Lightning Examples
Title |
Dataset |
Description |
Notebooks |
MLP in Lightning with TensorBoard -- continue training the last model |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
MLP in Lightning with TensorBoard -- checkpointing best model |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Title |
Dataset |
Description |
Notebooks |
Custom Data Loader Example for PNG Files |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Using PyTorch Dataset Loading Utilities for Custom Datasets -- CSV files converted to HDF5 |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Using PyTorch Dataset Loading Utilities for Custom Datasets -- Face Images from CelebA |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Using PyTorch Dataset Loading Utilities for Custom Datasets -- Drawings from Quickdraw |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Using PyTorch Dataset Loading Utilities for Custom Datasets -- Drawings from the Street View House Number (SVHN) Dataset |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Using PyTorch Dataset Loading Utilities for Custom Datasets -- Asian Face Dataset (AFAD) |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Using PyTorch Dataset Loading Utilities for Custom Datasets -- Dating Historical Color Images |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Using PyTorch Dataset Loading Utilities for Custom Datasets -- Fashion MNIST |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Training and Preprocessing
Title |
Dataset |
Description |
Notebooks |
Generating Validation Set Splits |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Dataloading with Pinned Memory |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Standardizing Images |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Image Transformation Examples |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Char-RNN with Own Text File |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Sentiment Classification RNN with Own CSV File |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Improving Memory Efficiency
Title |
Dataset |
Description |
Notebooks |
Gradient Checkpointing Demo (Network-in-Network trained on CIFAR-10) |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Title |
Description |
Notebooks |
Using Multiple GPUs with DataParallel -- VGG-16 Gender Classifier on CelebA |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Distribute a Model Across Multiple GPUs with Pipeline Parallelism (VGG-16 Example) |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Title |
Dataset |
Description |
Notebooks |
PyTorch with and without Deterministic Behavior -- Runtime Benchmark |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Sequential API and hooks |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Weight Sharing Within a Layer |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Plotting Live Training Performance in Jupyter Notebooks with just Matplotlib |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
Title |
Dataset |
Description |
Notebooks |
Getting Gradients of an Intermediate Variable in PyTorch |
TBD |
TBD |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |
TensorFlow Workflows and Mechanics
Title |
Description |
Notebooks |
Chunking an Image Dataset for Minibatch Training using NumPy NPZ Archives |
TBD |
![TensorFlow](https://camo.githubusercontent.com/0456829b900fc19fc10eabe32058e9941297c43dc8624d6bf448af883c89ada3/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f54656e736f722d466c6f77312e302d6f72616e6765) |
Storing an Image Dataset for Minibatch Training using HDF5 |
TBD |
![TensorFlow](https://camo.githubusercontent.com/0456829b900fc19fc10eabe32058e9941297c43dc8624d6bf448af883c89ada3/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f54656e736f722d466c6f77312e302d6f72616e6765) |
Using Input Pipelines to Read Data from TFRecords Files |
TBD |
![TensorFlow](https://camo.githubusercontent.com/0456829b900fc19fc10eabe32058e9941297c43dc8624d6bf448af883c89ada3/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f54656e736f722d466c6f77312e302d6f72616e6765) |
Using Queue Runners to Feed Images Directly from Disk |
TBD |
![TensorFlow](https://camo.githubusercontent.com/0456829b900fc19fc10eabe32058e9941297c43dc8624d6bf448af883c89ada3/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f54656e736f722d466c6f77312e302d6f72616e6765) |
Using TensorFlow's Dataset API |
TBD |
![TensorFlow](https://camo.githubusercontent.com/0456829b900fc19fc10eabe32058e9941297c43dc8624d6bf448af883c89ada3/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f54656e736f722d466c6f77312e302d6f72616e6765) |
Training and Preprocessing
Title |
Dataset |
Description |
Notebooks |
Saving and Loading Trained Models -- from TensorFlow Checkpoint Files and NumPy NPZ Archives |
TBD |
TBD |
![TensorFlow](https://camo.githubusercontent.com/0456829b900fc19fc10eabe32058e9941297c43dc8624d6bf448af883c89ada3/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f54656e736f722d466c6f77312e302d6f72616e6765) |
Title |
Description |
Notebooks |
TorchMetrics |
How do we use it, and what's the difference between .update() and .forward()? |
![PyTorch](https://camo.githubusercontent.com/4291582a6c309b5956b2c64e3ccd250ce661274b6905134be7b9e3d9e92e7cd8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50792d546f7263682d726564) |