piojanu / gradientailab

Experiments with Deep Reinforcement Learning

---

What is it all about?

    In this project we want to explore reinforcement learning techniques in deep learning. Our baseline is experiment presented by Andrej Karpathy on his blog. You can find his post in pg_pong/etc directory. It's about lightweight Policy Gradient agent.
    Starting from reproduction of his work, we now go further developing better agents for reinforcement learning tasks we find interesting. Eventually, we will start whole new projects based on gained experience.
Along the road, articles/presentations for Gradient research circle and further should emerge.

Do you have your own project/idea or want to join one of our projects? Contact us now! Do you want to start working on AI? Read further!

How to start AI&RL journey?

    Do you want to start to work with Artificial Intelligence? Don't be scared! You just need some basic programming skills and great desire to learn and create incredible things :) We will find some tailor-made task for you. Contact us now!

Here you have some theoretical materials about DRL and ANNs to start with:

0. Of course some background in Artificial Neural Networks:
   • Great introduction to theory behind Neural Networks from 3Blue1Brown
   • Fantastic classes about "Convolutional Neural Networks for Visual Recognition":
     • Lectures videos
     • Notes, assignments etc.
1. Artificial Intelligence grounding:
   • Berkeley cs188 course (on edXedge):
     • Uncertainty and Utilities
     • MDP part I and MDP part II
     • Reinforcement Learning part I and Reinforcement Learning part II
   • Book Artificial Intelligence: A Modern Approach (3rd Edition) (do some exercises too for better understanding):
     • Ch. 16.1 - 16.3
     • Ch. 17.1 - 17.3
     • Ch. 21
2. Deep Reinforcement Learning:
   • Book Reinforcement Learning: An Introduction (2nd Edition Draft)
   • Andrej Karpathy great post about Deep Reinforcement Learning: Pong from Pixels
   • Courses:
     • Personally I would start with those lectures:
       • Reinforcement Learning
       • Policy Search for RL
       • Deep Reinforcement Learning
     • Great courses list (not only watch, but always do the assignments too!):
       • Deep RL Bootcamp
       • Reinforcement Learning Summer School
       • CS 294: Deep Reinforcement Learning

Repository organization

(Preferred) Directory tree

.
├── README.md (This file. Organization, targets, tasks, descritptions etc.)
├── etc (Other resources related to reinforcement learning in general e.g. papers)
└── <project name> 
    ├── README.md (Project description, organization, milestones etc.)
    ├── doc (Articles, presentations, experiments descriptions and results etc.)
    ├── etc (Other resources related to project e.g. papers, diagrams etc.)
    └── src (All experiments live here.)
        ├── checkpoints (Saved models etc.)
        ├── codebase    (Classes, helpers, utils etc.)
        ├── logs        (All the logging related files.)
        ├── out         (All side products of scripts that don't fit anywhere else.)
        ├── third_party (As `codebase` + scripts but from third party.)
        └── script1.py  (All scripts performing experiments live in `src`.)

Contribution guidelines

See CONTRIBUTING.md in root directory of this repo.

Directions to explore

• Stochastic Policy Gradients:

  Reading:

  • Deep Reinforcement Learning: Pong from Pixels

• Continuous control of roboarm and/or hovering rocket:

  Reading about continuous control:

  • Asynchronous Methods for Deep Reinforcement Learning
  • High-Dimensional Continuous Control Using Generalized Advantage Estimation
  • Continuous control with deep reinforcement learning

  Reading about learning robots:

  • Asymmetric Actor Critic for Image-Based Robot Learning
  • One-Shot Imitation Learning
  • Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World
  • Sim-to-Real Transfer of Robotic Control with Dynamics Randomization

• Learning from Human Preferences:

  Reading:

  • OpenAI Blog: Learning from Human Preferences
  • Learning from Human Preferences

• Planning and reasoning about the future:

  Reading about imagination augmentation:

  • Agents that imagine and plan
  • Learning model-based planning from scratch
  • Imagination-Augmented Agents for Deep Reinforcement Learning

  Reading about "Schema Networks":

  • General Game Playing with Schema Networks
  • Schema Networks: Zero-shot Transfer with a Generative Causal Model of Intuitive Physics

• Robofish in simulated aquarium:

  Reading:

  • Terrain-Adaptive Locomotion Skills Using Deep Reinforcement Learning

Contact us now

Administrator:

Piotr Januszewski
[email protected]

---

This is Gradient research circle project. Our website: http://gradient.eti.pg.gda.pl/
The truth is, we will send fully automated "Prometeusz" space cruise to the Solaris in colonization mission! Maybe..