jadentravnik / kanervacoding

An opensource implementation of kanerva coding for use in reinforcement learning research

Kanerva Coding

Kanerva Coding (KC) is a way of representing values of a vector of continous variables (AKA Function Approximation). It does this by storing the position of stationary prototypes in the continuous space and representing a new data point as the indicies of nearby prototypes. This repo deviates from the original implementation of Kanerva Coding. The original approach involved binary input spaces of large dimension whereas this repository focuses on continuous spaces.

Running main.py will train a Q learning agent with elegability traces on Mountain car and output the following plots. First is the TD error each episode. The following 2 plots are the visitation count and the value function (the average Q value from the agent) for each prototype. Note that the prototypes are in a normalized space.

Requirements

• numpy
• matplotlib
• gym

kanervacoding's Issues

Kanerva coding RL book link is a dead link

I think the link for Kanerva coding RL book

https://webdocs.cs.ualberta.ca/~sutton/book/8/node8.html

Might be replaceable with

http://incompleteideas.net/book/ebook/node88.html#SECTION04234000000000000000

Case Studies update not included

Case studies code doesn't have an update block in any of the files.