farruxx / learn-bts-with-gp

Learning Behavior Trees with Genetic Programming in a probailistic state-machine simulator.

Learning Behavior Trees with Genetic Programming in unpredictable environments

This repository contains an implementation of a Genetic Programming (GP) algorithm that evolves Behavior Trees (BTs) to solve different mobile manipulation tasks. The tasks consist in picking (from 1 to 3) cubes from tables of the environment and placing them in a different placing table (see figure below). The BTs are executed in a high level state machine to make the problem computationally tractable, but they can be exported and run in a physical simulator (the code for the simulation in Gazebo with the learned BT can be provided upon request).

Notes on installation

After cloning the repository, run the following command to install the correct dpeendencies:

pip3 install -r requirements.txt

The Behavior Tree library is based on py-trees==0.6.8 (see documentation and repository), which is the version used by py-tree-ros (see repository) for the ROS distribution Melodic.

In particular, the following modifications have been made to the py_trees source code:

• the function pt.display.render_dot_tree has been modified to take as input parameter the path of the target folder to save the figure;
• the function pt.display.render_dot_tree has been modified to display ' ' (spaces) instead of * to distinguish nodes of the same type (e.g. for two Sequence nodes, the first one has name 'Sequence' and the second one has name 'Sequence ' instead of 'Sequence*').

Content

• behavior_tree.py is a class for handling string representations of behavior trees.

• behaviors.py contains the implementation of all behaviors used in the simulations.

• cost_function.py is used to compute the cost function, the costs are defined here.

• environment.py handles the scenarios configurations and executes the BT, returning the fitness score.

• genetic_programming.py implements the GP algorithm, with many possible settings.

• gp_bt_interface.py provides an interface between a GP algorithm and behavior tree functions.

• py_trees_interface.py provides an interface between py_trees(documentation and repository) and the string representation of the BTs.

• state_machine.py is an high-level simulator used to simulate the execution of the BTs. It is probabilistic as state transitions are regulated by the success probabilities of specific events.

• hash_table.py and logplot.py are utilities for data storage and visualization.

Task description

There configuration files .yaml contain the list of behaviors used in the different tasks discussed here. In particular, the evironment script is configured to run configurations of type BT_scenario_X.yaml, so it is necessary to make the following modifications:

'Effects of adding non-essential behaviors'

Execute the first scenario in the nominal settings described in the paper. This will create the logs for the Necessary behaviors. RenameBT_scenario_1_lowNoise.yaml as BT_scenario_1.yaml. This will create the logs for the 7 non-essential behaviors. RenameBT_scenario_1_highNoise.yaml as BT_scenario_1.yaml. This will create the logs for the 14 non-essential behaviors.

'Impact of the failure probability cost on the fitness function'

RenameBT_scenario_1_safe.yaml as BT_scenario_1.yaml and modify the following parameters for the State Machine in state_machine.py:

Parameter name	Value
fail_pick_probability	0.2
fail_place_probability	0.1
fail_tuck_probability	0.0
fail_localization_probability	0.2
fail_navigation_probability	0.0
drop_probability	0.2
lost_probability	0.4

This will create the logs for the Risky path. RenameBT_scenario_1_safe.yaml as BT_scenario_1.yaml and modify the value failure: float = 0.0 to 150.0 in cost_function.py. This will create the logs for the Safe path.

Run the simulation

Firstly, it is necessary to create the logs folder. The main.py script is already configured to run the 3 scenarios described in the paper, but the script can be easily modified to run solely the first scenario, for the scope of obtaining the same results as in the paper. The parameters for the BT execution (e.g. number of ticks, successes and failures) can be modified in py_trees_interface.py.

Testing

Some BTs can be tested, to understand how they interact with the state machine simulator and to see how the fitness is computed. To do so, run

$ pytest -s tests/test_fitness.py

from the repository root.

Media

The BT solving the task in the State Machine can be shown to solve the same task designed in a physical simulator, e.g. Gazebo. A video showing this transfer, can be found here.

Behavior Trees solving the first scenario

Behavior Trees solving the second scenario

Behavior Trees solving the third scenario

Hand-coded Behavior Trees solving the third scenario

It is interesting to see how the GP algorithm finds a more compact solution than the hand coded one (which is just three copies of the BT solving task 1 with cube related modifications), since the size of the tree is a factor in the fitness function. Note however, that sometimes these solutions are less human readable, especially for non experts in the field of Bts.