kyushik / drl

Repository for codes of 'Deep Reinforcement Learning'

DRL

Introduction

This repository is the codes for Deep Reinforcement Learning
I verified my codes with games. The games are made with pygame. I made the games or I modified them to apply for DRL.

Also, environments, which are made by Unity ML-agents, are in Unity_ML_Agent Repository

Performance of each algorithm that I implemented are as follows (verified with Breakout).

Youtube Link of Playing Breakout with NoisyNet DQN

I set up the DRL code as follows.

1. Human-level Control Through Deep Reinforcement Learning (DQN)
2. Deep Reinforcement Learning with Double Q-Learning (DDQN)
3. Prioritized Experience Replay (PER)
4. Dueling Network Architecture for Deep Reinforcement Learning (Dueling DQN)
5. Noisy Networks for Exploration (NoisyNet DQN)
6. A Distributional Perspective on Reinforcement Learning (C51)
7. Learning to Predict by the Methods of Temporal Differences (N-step DQN)
8. Deep Recurrent Q-Learning for Partially Observable MDPs (DRQN)
9. Distributional Reinforcement Learning with Quantile Regression(QR-DQN)
10. Implicit Quantile Networks for Distributional Reinforcement Learning(IQN)
11. Curiosity-driven Exploration by Self-supervised Prediction (ICM)
12. Exploration by Random Network Distillation(RND)

This is the PPT file for the description of DQN codes that I implemented.

The description of DRL algorithms

Description of algorithms link

---

Environment

Software

• Windows7 (64bit)
• Python 3.6.5
• Anaconda 4.2.0
• Tensorflow-gpu 1.12.0
• pygame 1.9.3
• opencv3 3.1.0

Hardware

• CPU: Intel(R) Core(TM) i7-4790K CPU @ 4.00GHZ

• GPU: GeForce GTX 1080TI

• Memory: 16GB

  
  

---

Implementation

The sample code for my environment is Environment_sample.py

# This is sample code for Deep Reinforcement Learning testing environment 

# Import modules
import sys 
import numpy as np
import random

# Import games
sys.path.append("DQN_GAMES/")

# add as game the one that you want to play!! 
import pong as game
import dot  
import dot_test  
import tetris  
import wormy
import breakout

# Get Number of action and name of game from the game code
Num_action = game.Return_Num_Action()
game_name = game.ReturnName()

# Get game state class from game code
game_state = game.GameState()

while True:
    # Choose random action
    action = np.zeros([Num_action])
    action[random.randint(0, Num_action - 1)] = 1.0

    # You can get next observation, reward and terminal after action
    observation_next, reward, terminal = game_state.frame_step(action)

You can run the game with random action using this code!!

Please check that you made the folder for saved variables

checkpoint = tf.train.get_checkpoint_state("saved_networks_DQN")

Then you are ready to implement the code! 😆

---

Games

Most of the games are made with python and pygame!

The codes are easy to understand and variables are easy to understand.

So fix the code as you want!! 😆

Description of the games

Specific description of games