harveyphm / distributional-dqn

Implementation of 'A Distributional Perspective on Reinforcement Learning' and 'Distributional Reinforcement Learning with Quantile Regression' based on OpenAi DQN baselines.

Distributional DQN

Implementation of 'A Distributional Perspective on Reinforcement Learning' and 'Distributional Reinforcement Learning with Quantile Regression' based on OpenAi DQN baseline.

C51:

Quantile Regression: (see branch quantile)

Installation

Install the OpenAi fork https://github.com/Silvicek/baselines (parent changes a lot, compatibility isn't guaranteed) Then install requirements

pip3 install -r requirements.txt

Usage:

For simple benchmarking:

python3 train_[{cartpole, pong}].py
python3 enjoy_[{cartpole, pong}].py

For full Atari options see help

python3 train_atari.py --help

after learning, you can visualize the distributions by running

python3 enjoy_atari.py --visual ...

---

This implementation has been successfully tested on: Pong, Qbert, Seaquest

Some baseline features not supported (prioritized replay, double q-learning, dueling)