alirezakazemipour / dqn-her

Implementation of the hindsight experience by DQN algorithm on the bit flip environment.

DQN-HER

This repository contains the Pytorch implementation of Deep Q-Networks with hindsight experience replay to solve the bit flip problem consisted of 25 bits.
The number of bits can be changed; 25 and more are considered to be slightly difficult challenges.

Dependencies

• numpy == 1.18.2
• torch == 1.2.0

Hyper-parameters

n_bits: Number of bits
lr: Learning rate
gamma: Discount factor
k_future: Number of episode for future hindsight strategy

Parameter	Value
n_bits	25
lr	1e-3
gamma	0.98
memory_size	1000000
batch_size	128
k_future	4

Result

According to the plot when the problem is solved, at the worst case scenario, it takes 12 bits to be flipped by the agent in order to achieve the real, desired goal.

Reference

1. Human-level control through deep reinforcement learning, Mnih et al., 2015
2. Hindsight Experience Replay, Andrychowicz et al., 2017