NOTE: Current TF-Agents pre-release is under active development and interfaces may change at any time. Feel free to provide feedback and comments.
The documentation, examples and tutorials will grow over the next few weeks.
Agents
Tutorials
Examples
Installation
Contributing
Principles
Citation
Disclaimer
In TF-Agents, the core elements of RL algorithms are implemented as Agents
.
An agent encompasses two main responsibilities: defining a Policy to interact
with the Environment, and how to learn/train that Policy from collected
experience.
Currently the following algorithms are available under TF-Agents:
- DQN: Human level control through deep reinforcement learning Mnih et al., 2015 https://deepmind.com/research/dqn/
- DDQN: Deep Reinforcement Learning with Double Q-learning Hasselt et al., 2015 https://arxiv.org/abs/1509.06461
- DDPG: Continuous control with deep reinforcement learning Lilicrap et al. https://arxiv.org/abs/1509.02971
- TD3: Addressing Function Approximation Error in Actor-Critic Methods Fujimoto et al. https://arxiv.org/abs/1802.09477.
- REINFORCE: Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning Williams http://www-anw.cs.umass.edu/~barto/courses/cs687/williams92simple.pdf
- PPO: Proximal Policy Optimization Algorithms Schulman et al. http://arxiv.org/abs/1707.06347
See tf_agents/colabs/
for tutorials on the major components provided.
End-to-end examples training agents can be found under each agent directory. e.g.:
To install the latest version, run the following:
# Installing with the `--upgrade` flag ensures you'll get the latest version.
pip install --user --upgrade tf-agents # depends on TensorFlow
TF-Agents depends on a recent stable release of
TensorFlow (pip package tensorflow
).
Note: Since TensorFlow is not included as a dependency of the TF-Agents
package (in setup.py
), you must explicitly install the TensorFlow
package (tensorflow
or tensorflow-gpu
). This allows us to maintain one
package instead of separate packages for CPU and GPU-enabled TensorFlow.
To force a Python 3-specific install, replace pip
with pip3
in the above
commands. For additional installation help, guidance installing prerequisites,
and (optionally) setting up virtual environments, see the TensorFlow
installation guide.
There are also nightly builds of TF-Agents under the pip package
tf-agents-nightly
, which requires you install on one of tf-nightly
and
tf-nightly-gpu
. Nightly builds include newer features, but may be less stable
than the versioned releases.
To install the nightly build version, run the following:
# Installing with the `--upgrade` flag ensures you'll get the latest version.
pip install --user --upgrade tf-agents-nightly # depends on TensorFlow
We're eager to collaborate with you! See CONTRIBUTING.md
for a guide on how to contribute. This project adheres to TensorFlow's
code of conduct. By participating, you are expected to
uphold this code.
This project adheres to Google's AI principles. By participating, using or contributing to this project you are expected to adhere to these principles.
If you use this code please cite it as:
@misc{TFAgents,
title = {{TF-Agents}: A library for Reinforcement Learning in TensorFlow},
author = "{Sergio Guadarrama, Anoop Korattikara, Oscar Ramirez,
Pablo Castro, Ethan Holly, Sam Fishman, Ke Wang, Katya Gonina,
Chris Harris, Vincent Vanhoucke, Eugene Brevdo}",
howpublished = {\url{https://github.com/tensorflow/agents}},
url = "https://github.com/tensorflow/agents",
year = 2018,
note = "[Online; accessed 30-November-2018]"
}
This is not an official Google product.