zl-6956 / drl_implementation

A DRL implementation repo

DRL_Implementation

Current status: under development

Introduction

• This repository is a pytorch-based implementation of modern DRL algorithms, designed to be reusable for as many Gym-like training environments as possible
• The package is under active development, use the latest version with caution
• It is recommended to use the released version
• Understand more with the Wiki!
• Tested environments: Gym, Pybullet-gym, Pybullet-multigoal-gym
• My priority is on continuous action algorithms as I'm working on robotics

Installation

git clone https://github.com/IanYangChina/DRL_Implementation.git
cd DRL_Implementation
python -m pip install -r requirements.txt
python -m pip install .

Click here for example codes , to run the codes you will need to install Gym, Pybullet, or pybullet-multigoal-gym. See env installation links below. For more use cases, have a look at the drl_imp_test repo
From the project root, run python drl_implementation/examples/$SCTIPT_NAME.py

State-based

• DDPG - Deterministic, Continuous
• TD3 -Deterministic, Continuous
• SAC (Adaptive Temperature) - Stochastic, Continuous
• D4PG - Deterministic, Continuous -only work for torch 1.4.0, fix it later-

Image-based

• DQN - Deterministic, Discrete (LSTM network for Atari) -debugging-
• SAC-AE - Continuous -to be built-
• DrQ - Continuous -debugging-

Replay buffers

• Hindsight
• Prioritised

Tested Environments

• Pybullet Gym (Continuous)
• OpenAI Gym Mujoco Robotics Multigoal Environment (Continuous)
• Pybullet Multigoal Gym (OpenAI Robotics Multigoal Pybullet Migration) (Continuous)

Some result figures

Reference Papers: Algorithm

• DQN
• DoubleDQN
• LSTM network on raw Atari pixel observation
• DDPG
• TD3
• SAC (Adaptive Temperature)
• PER
• HER
• D4PG
• R2D2
• SAC-AE
• DrQ

Reference Papers: Implementation Matters

• Time limit
• SOTA PPO Hyperparameters (many applicable to other algorithms)
• SAC Temperature Auto-tuning