weidi-chang / sarsavsexpectedsarsa

An a bias-variance tradeoff of Sarsa vs. Expected Sarsa with experiments.

SarsaVsExpectedSarsa

An analysis of bias-variance tradeoff of Sarsa, Expected Sarsa, Double Sarsa, and Double Expected Sarsa with experiments.

Note that our main analysis is in the BiasVarianceTradeoff.ipynb

Supporting experiments were run in the other files in the directory.

Authors:

Peter Henderson Wei-Di Chang

Based on the following works:

Van Seijen, Harm, et al. "A theoretical and empirical analysis of Expected Sarsa." Adaptive Dynamic Programming and Reinforcement Learning, 2009. ADPRL'09. IEEE Symposium on. IEEE, 2009. Ganger, Michael, Ethan Duryea, and Wei Hu. "Double Sarsa and Double Expected Sarsa with Shallow and Deep Learning." Journal of Data Analysis and Information Processing 4.04 (2016): 159.