GithubHelp home page GithubHelp logo

cyc790 / awesome-deep-reinforcement-learning Goto Github PK

View Code? Open in Web Editor NEW

This project forked from jgvictores/awesome-deep-reinforcement-learning

0.0 1.0 0.0 3.27 MB

Curated list for Deep Reinforcement Learning (DRL): software frameworks, models, datasets, gyms, baselines...

awesome-deep-reinforcement-learning's Introduction

awesome-deep-reinforcement-learning

Curated list for Deep Reinforcement Learning (DRL): software frameworks, models, datasets, gyms, baselines...

To accomplish this, includes general Machine Learning (ML), Neural Networks (NN) and Deep Neural Networks (DNN) with many vision examples, and Reinforcement Learning (RL) with videogames/robotics examples. Some alternative Evolutionary Algorithms (EA) with similar objectives included too.

General Machine Learning (ML)

General ML Software Frameworks

General ML Books

  • Jake VanderPlas, "Python Data Science Handbook", 2017. safari

Neural Networks (NN) and Deep Neural Networks (DNN)

NN/DNN Software Frameworks

  • Overview: presentation (permalink).
  • Docker images with several pre-installed software frameworks: 1, 2, 3.
  • Projects to port trained models from one software framework to another: 1

Attempling to order software frameworks by current popularity:

NN/DNN Models

Image Object Segmentation, Localization, Detection Models

Overviews: 1, 2. Taxonomy: 1.

  • Mask R-CNN (2017), Kaiming He et Al; Facebook AI Research (FAIR); "Mask R-CNN"; arxiv. keras.
  • FCIS (2017). "Fully Convolutional Instance-aware Semantic Segmentation". arxiv. Coded in caffe but released in mxnet, port: chainer.
  • YOLO (2015). Joseph Redmond et Al; U Washington, Allen AI, FAIR; "You Only Look Once: Unified, Real-Time Object Detection"; arxiv. Variants: YOLO9000, YOLO v3... Darknet, ports: tensorflow.
  • SSD (2015). Wei Liu et Al; UNC, Zoox, Google, et Al; "SSD: Single Shot MultiBox Detector"; arxiv. caffe
  • U-Net (2015); Olaf Ronneberger et Al; "Convolutional Networks for Biomedical Image Segmentation"; arxiv. caffe.
  • OverFeat (2015). Pierre Sermanet, David Eigen, Xiang Zhang, Michael Mathieu, Rob Fergus, and Yann LeCun; NYU; "OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks"; arxiv.

Image Classification Models

Overviews: 1, 2, 3

  • EfficientNets (2019). Mingxing Tan and Quoc V. Le; Google; "EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks"; arxiv.
  • MobileNets (2017). Andrew Howard et Al; Google; "MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications"; arxiv.
  • DenseNets (2017). Gao Huang et Al; "Densely Connected Convolutional Networks"; arxiv. torch includes links to ports.
  • ResNet (2015). Kaiming He et Al; Microsoft Research; "Deep Residual Learning for Image Recognition"; arxiv. Introduces "Residual Blocks" via "Skip Connections" (some cite similarities with GRUs), and additionally uses heavy batch normalization. Variants: ResNet50, ResNet101, ResNet152 (correspond to number of layers). 25.5 million parameters.
  • VGGNet (Sept 2014). Karen Simonyan, Andrew Zisserman; Visual Geometry Group (Oxford); "Very Deep Convolutional Networks for Large-Scale Image Recognition"; arxiv. Input: 224x224x3. Conv/pool and fully connected. Variants: VGG11, VGG13, VGG16, VGG19 (correspond to number of layers); with batch normalization. 138 million parameters; trained on 4 Titan Black GPUs for 2-3 weeks.
  • GoogLeNet/InceptionV1 (Sept 2014). Christian Szegedy et Al; Google, UNC; "Going Deeper with Convolutions"; arxiv. 22 layer deep CNN. Only 4-7 million parameters, via smaller convs. A more aggressive cropping approach than that of Krizhevsky. Batch normalization, image distortions, RMSprop. Uses 9 novel "Inception modules" (at each layer of a traditional ConvNet, you have to make a choice of whether to have a pooling operation or a conv operation as well as the choice of filter size; an Inception module performa all these operations in parallel), and no fully connected. Trained on CPU (estimated as weeks via GPU) implemented in DistBelief (closed-source predecessor of TensorFlow). Variants (summary): v1, v2, v4, resnet v1, resnet v2; v9 (slides). Also see Xception (2017) paper.
  • NIN (2013). Min Lin et Al; NUSingapore; "Network In Network"; arxiv. Provides inspiration for GoogLeNet.
  • ZFNet (2013). Matthew D Zeiler and Rob Fergus; NYU; "Visualizing and Understanding Convolutional Networks"; doi, arxiv. Similar to AlexNet, with well-justified finer tuning and visualization (namely Deconvolutional Network).
  • AlexNet (2012). Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton; SuperVision (UToronto); "ImageNet Classification with Deep Convolutional Neural Networks"; doi. In 224x224 (227x227?) color patches (and their horizontal reflections) from 256x256 color images; 5 conv, maxpool, 3 full; ReLU; SVD with momentum; dropout and data augmentation. 60-61 million parameters, split into 2 pipelines to enable 5-6 day GTX 580 GPU training (while CPU data augmentation).
  • LeNet-5 (1998). Yann LeCun et Al; ATT now at Facebook AI Research; "Gradient-based learning applied to document recognition"; doi. In 32x32 grayscale; 7 layer (conv, pool, full...). 60 thousand parameters.

Graph/Manifold/Network Convolutional Models

Generative Models

Tutorial: pytorch

  • Auto-Regressive Generative Models: PixelRNN, PixelCNN++... ref
  • Deep Dream. caffe
  • Style Transfer:
    • Tutorial: tensorflow
    • Fujun Luan et Al (2018), "Deep Painterly Harmonization"; arxiv. torch+matlab
    • Deep Photo Style Transfer (2017). Fujun Luan et Al, "Deep Photo Style Transfer"; arxiv. torch+matlab
    • Neuralart (2015). Leon A. Gatys et Al; "A Neural Algorithm of Artistic Style"; arxiv. Uses base+style+target as inputs and optimizes for target via BFGS. tensorflow, torch, keras 1 2 3 4
  • GANs:
    • https://github.com/hindupuravinash/the-gan-zoo
    • BigGAN (2018); "Large Scale GAN Training for High Fidelity Natural Image Synthesis"; arxiv. pytorch
    • Terro Karas et Al (2018); NVIDIA; "Progressive Growing of GANs for Improved Quality, Stability, and Variation"; arxiv. tensorflow
    • CANs (2017). Ahmed Elgammal et Al; Berkeley; "CAN: Creative Adversarial Networks, Generating "Art" by Learning About Styles and Deviating from Style Norms"; arxiv. tensorflow
    • CycleGAN (2017). Jun-Yan Zhu et Al; Berkeley; "Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks". torch and migrated to pytorch.
    • DCGAN (2015). Alec Radford, Luke Metz, Soumith Chintala; Indico Research, Facebook AI Research; "Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks"; arxiv.
    • GAN (2014). Ian J. Goodfellow et Al; Université de Montréal; "Generative Adversarial Nets"; arxiv.
  • Audio synthesis
    • FTTNet (2018). Zeyu Jin et Al; "FFTNet: a Real-Time Speaker-Dependent Neural Vocoder". pytorch
    • WaveNet (2016). Aäron van den Oord et Al; DeepMind; "WaveNet: A Generative Model for Raw Audio"; arxiv. wikipedia.

Recurrent Models

Can be trained via Back Propagation Through Time (BPTT). Also see Connectionist Temporal Classification (CTC). Cells include: SimpleRNN (commonly has TanH activation as second derivative decays slowly to 0), Gated Recurrent Units (GRU), Long short-term memory (LSTM), ConvLSTM2D, LSTM with peephole connection; keras.

  • Recurrent Neural Networks (RNN).
  • Bidirectional RNN.
  • Stateful RNN.

Word Embedding Models

  • BERT
  • ELMo
  • GloVe (2014). Jeffrey Pennington et Al; Stanford; "GloVe: Global Vectors for Word Representation".
  • word2vec.

More Models

  • Regression Networks (essentialy same, remove last activation and use some loss such as MSE rather than binary/categorical cross-entropy).
  • Autoencoders (AE), Variational Autoencoders (VAE), Denoising Autoencoders.
    • Tutorials: keras, keras
    • Yunchen Pu et Al; "Variational Autoencoder for Deep Learning of Images, Labels and Captions"; arxiv.
  • Memory Networks. Use "Memory Units".
  • Capsule Networks. Use "Capsules". wikipedia
  • Echo-state networks.
  • Restricted Boltzmann Machine (RBM).
  • AutoML.

NN/DNN Datasets

Lists of lists before citing the classics:

Image, Sound, Multimedia...

  • MNIST: Handwritten digits, set of 70000 28x28 images, is a subset of a larger set available from NIST (and centered from its 32x32). Also see 2018's Kuzushiji-MNIST.
  • ImageNet: Project organized according to the WordNet hierarchy (22000 categories). Includes SIFT features, bounding boxes, attributes. Currently over 14 million images, 21841 cognitive synonyms (synsets) indexed, goal of +1000 images per synset.
    • ImageNet Large Visual Recognition Challenge (ILSVRC): Goal of 1000 categories using +100000 test images. E.g. LS-LOC
  • PASCAL VOC (Visual Object Classes)
  • COCO (Common Objects in Context): 2014, 2015, 2017. Includes classes and annotations.
  • CIFAR-10: 60000 32x32 colour images (selected from MIT TinyImages) in 10 classes, with 6000 images per class
  • CIFAR-100: 60000 32x32 colour images (selected from MIT TinyImages) in 100 classes containing 600 images per class, grouped into 20 superclasses
  • SVHN (Street View House Numbers)
  • HICO (Humans Interacting with Common Objects)
  • KIT Motion-Language: https://motion-annotation.humanoids.kit.edu/dataset
  • Sketches: Quick Draw
  • Driving: https://robotcar-dataset.robots.ox.ac.uk/datasets/
  • Robotics: iCubWorld; where iCWT: 200 domestic objects in 20 categories (11 categories also in ILSVRC, rest in ImageNet). Also muratkrty/iCub-camera-dataset.

Text

NN/DNN Benchmarks

NN/DNN Pretrained Models

NN/DNN Techniques Misc

  • Layers: Dense (aka Fully Connected), Convolutional (1D/2D/3D... keras, advanced: upsampling (e.g. in GANs), dilated causal (aka atrous)(e.g. in WaveNet)), Pooling (aka SubSampling)(1D/2D/3D)(Max, Average, Global Max, Global Average, Average with learnable weights per feature map... keras), Normalisation. Note: Keras implements activation functions, dropout, etc as layers.
  • Weight initialization: pretrained (see above section), zeros, ones, constant, normal random, uniform random, truncated normal, variance scaling, orthogonal, identity, normal/uniform as done by Yann LeCun, normal/uniform as done by Xavier Glorot, normal/uniform as done by Kaiming He. keras, StackExchange
  • Activation functions: Linear, Sigmoid, Hard Sigmoid, Logit, Hyperbolic tangent (TanH), SoftSign, Rectified Linear Unit (ReLU), Leaky ReLU (LeakyReLU or LReLU), Parametrized or Parametric ReLU (PReLU), Thresholded ReLU (Thresholded ReLU), Exponential Linear Unit (ELU), Scaled ELU (SELU), SoftPlus, SoftMax, Swish. wikipedia, keras, keras (advanced), ref.
  • Regularization techniques (reduce overfitting and/or control the complexity of model; may be applied to kernel (weight matrix), to bias vector, or to activity (activation of the layer output)): L1(lasso)/L2(ridge)/ElasticNet(L1/L2)/Maxnorm regularization (keras), dropout, batch and weight normalization, Local Response Normalisation (LRN), data augmentation (image distortions, scale jittering...), early stopping, gradient checking.
  • Optimizers: keras, ref
    • Gradient descent variants: Batch gradient descent, Stochastic gradient descent (SGD), Mini-batch gradient descent.
    • Gradient descent optimization algorithms: Momentum, Nesterov accelerated gradient, Adagrad, Adadelta, RMSprop, Adam, AdaMax, Nadam, AMSGrad, Eve.
    • Parallelizing and distributing SGD: Hogwild!, Downpour SGD, Delay-tolerant Algorithms for SGD, TensorFlow, Elastic Averaging SGD.
    • Additional strategies for optimizing SGD: Shuffling and Curriculum Learning, Batch normalization, Early Stopping, Gradient noise.
    • Broyden-Fletcher-Goldfarb-Shanno (BFGS)
    • Gradient-free: facebookresearch/nevergrad
  • Error/loss functions: keras
    • Accuracy used for classification problems: binary accuracy (mean accuracy rate across all predictions for binary classification problems), categorical accuracy (mean accuracy rate across all predictions for multiclass classification problems), sparse categorical accuracy (useful for sparse targets), top k categorical accuracy (success when the target class is within the top k predictions provided).
    • Error loss (measures the difference between the values predicted and the values actually observed, can be used for regression): mean square error (MSE), root square error (RMSE), mean absolute error (MAE), mean absolute percentage error (MAPE), mean squared logarithmic error (MSLE).
    • Hinge: hinge loss, squared hinge loss, categorical hinge.
    • Class loss, used to calculate the cross-entropy for classification problems: binary cross-entropy (binary classification), categorical cross-entropy (multi-class classification), sparse categorical cross-entropy. wikipedia
    • Logarithm of the hyperbolic cosine of the prediction error (logcosh), kullback leibler divergence, poisson, cosine proximity.
  • Metric functions: usually same type as error/loss functions, but used for evaluationg rather than training. keras
  • Cross-validation: hold-out, stratified k-fold. wikipedia.
  • Transfer learning. tensorflow, keras

NN/DNN Visualization and Explanation

Reinforcement Learning (RL) and Deep Reinforcement Learning (DRL)

RL/DRL Software Frameworks

Attempting to order by current popularity:

RL/DRL Gyms

Should be compatible with OpenAI Gym and also rllab (both mentioned above).

RL/DRL Baselines and Benchmarks

RL/DRL Techniques Misc

  • Batch: REINFORCE, Deep Q-Network (DQN), Expected-SARSA, True Online Temporal-Difference (TD), Double DQN, Truncated Natural Policy Gradient (TNPG), Trust Region Policy Optimization (TRPO), Reward-Weighted Regression, Relative Entropy Policy Search (REPS), Cross Entropy Method (CEM), Advantage-Actor-Critic (A2C), Asynchronous Advantage Actor-Critic (A3C), Actor-critic with Experience Replay (ACER), Actor Critic using Kronecker-Factored Trust Region (ACKTR), Generative Adversarial Imitation Learning (GAIL), Hindsight Experience Replay (HER), Proximal Policy Optimization (PPO, PPO1, PPO2), Ape-X Distributed Prioritized Experience Replay, Continuous DQN (CDQN or NAF), Dueling network DQN (Dueling DQN), Deep SARSA, Multi-Agent Deep Deterministic Policy Gradient (MADDPG).
  • Online: Deep Determisitc Policy Gradient (DDPG).
  • Experience Replay.

RL/DRL Books

Evolutionary Algorithms (EA)

Only accounting those with same objective as RL.

Misc Tools

Similar pages

awesome-deep-reinforcement-learning's People

Contributors

alicest9 avatar jgvictores avatar

Watchers

avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.