ifestus / rl Goto Github PK

Currently the action_from_dqn function does too many things.

1. One function for crafting the input
2. One function for getting the actions from the DQN

I'm missing an important pre-processing step from the paper. More specifically, I need to take the maximum pixel value between frame n-1 and frame n and recognize that as frame n.

processed_frame = maxFn(frame_n-1, frame_n))

or something...

Implement DQN as described in https://storage.googleapis.com/deepmind-media/dqn/DQNNaturePaper.pdf

The goal here is to get a better understanding of the paper, but also getting a feel for how estimating an action-value function can be done with a DNN.

Write code for TRPO as described in paper: https://arxiv.org/abs/1502.05477

Goal: Get a better understanding for TRPO and a better understanding of how to code estimation of policies using a DNN.

Testing against Atari is kind of slow. Ideally I'd like to have a <5-10 minute feedback time on changes to the model. In attempt to make this feedback a bit faster, create a python script to run classic problems.

run_environment.py should use the helpers.py functions.

take_step function logic control loop uses the done variable incorrectly. It should take note of the done value and pass that long to the next call as to not overwrite the last frame of every game.

There's a lot of functionality baked into run_environment.py that would do to be moved into another, more general format. In the spirit of completing #5 without causing myself emotional trauma from copying so much code, I'm going to move that functionality into it's own helper.py file.