Author: Lucas Takara Date: 09/13/2019
The goal of this project is to train an agent to navigate and collect bananas in a squared world! It has to collect as many yellow bananas as possible while avoiding purple bananas.
This environment was developed on Unity Real-Time Development Platform and it has some peculiarities:
- A reward of +1 is provided for collecting a yellow banana
- A reward of -1 is provided for collecting a purple banana
The state space has 37 dimensions and contains the agent's velocity, along with ray-based perception of objects around the agent's forward direction. Given this information, the agent has to learn how to best select actions. Four discrete actions are available, corresponding to:
W
- move forward (Note: when playing the game, the agent will move forward, if you don't select a different action in time)S
- move backwardA
- move leftD
- move right
The task is episodic and in order to solve the environment, your agent must get an average score of +13 over 100 consecutive episodes.
Follow the instructions bellow to install the environment on your own machine:
If you haven't already, please follow the instructions in Udacity's DRLND GitHub repository to set up your Python environment. By following these instructions, you will install PyTorch
, ML-Agents toolkit
and a few more Python packages required to install the project.
To set up your Python environment, follow these instructions:
- For Linux or Mac users:
conda create --name drlnd python=3.6 source activate drlnd
- For Windows users:
conda create --name drlnd python=3.6 activate drlnd
OpenAI Gym is a toolkit for developing and comparing reinforcement learning algorithms. This is the gym
open-source library, which gives you access to standardized set of environments.
- Installation
You can perform a minimal install of the packaged version directly from PyPI:
pip install gym
4. Clone the repository (if you haven't already) and navigate to the python/
folder. then, install the dependencies.
git clone https://github.com/udacity/deep-reinforcement-learning.git
cd deep-reinforcement-learning/python
pip install .
5. Create an IPython kernel for the drlnd
environment
python -m ipykernel install --user --name drlnd --display-name "drlnd"
6. Install Numpy Library
NumPy is the fundamental package for array computing with Python. We will be using the 1.12.1 version on this project.
- Installation:
pip install numpy==1.12.1
7. Install Matplotlib Library
Matplotlib is a Python 2D plotting library which produces publication quality figures in a variety of hardcopy formats and interactive environments across platforms.
On your Command line, type:
conda install matplotlib=2.1.1
8. Install Unity Machine Learning Agents Package:
Unity Machine Learning Agents allows researchers and developers to transform games and simulations created using the Unity Editor into environments where intelligent agents can be trained using reinforcement learning, evolutionary strategies, or other machine learning methods through a simple to use Python API.
- Installation via pip:
pip install unityagents
9. Install Pytorch Library
Pytorch is an open source machine learning framework that accelerates the path from research prototyping to production deployment. It will be used for creating and training our artificial neural networks.
- For Mac users:
via conda:
conda install pytorch=0.4.1 cuda80 -c pytorch
- For Windows users:
Download the whl
file with the version 0.4.1 from the following html page:
https://download.pytorch.org/whl/cu80/torch_stable.html
Then, install the file with pip install [downloaded file]
- For Linux users:
Acess Pytorch
and download the whl
file and install it with
pip install [downloaded file]
Other packages that may be installed are listed on the requirements.txt
file.
10. Before running code in a notebook, change the kernel to match the drlnd
environment that we created by using the drop-down Kernel
menu.
- Linux: click here
- Mac OSX: click here
- Windows 32 bits: click here
- Windows 64 bits: click here
Once you've downloaded it, place the file in the p1_navigation/
folder in the DRLND GitHub repository, and unzip (or decompress) the file.
(For Windows users) Check out this link if you need help with determining if your computer is running a 32-bit version or 64-bit version of the Windows operating system.
For this project, we'll be using the Jupyter Notebook
software which comes along with the Anaconda Distribution.
- Linux: click here
- Mac OSX: click here
- Windows 32: click here
- Windows 64: click here
In order to connect python with the environment, which is already saved in the Workspace, please import a few libraries and instantiate an object providing the location of the environment (see example below):
import numpy as np
from unityagents import UnityEnvironment
env = UnityEnvironment(file_name="/Users/Desktop/Banana.app)
Open Navigation.ipynb
file and follow the given instructions for training and testing your agent!
Good Luck!!