pgeon (pgeon-xai) is a Python package that produces explanations for opaque agents using Policy Graphs (PGs).

A Policy Graph is a means to obtain a representation of the behavior of an opaque agent, in the form of a directed graph. Discrete states are mapped to nodes and actions to edges.

• Install pgeon with pip: pip install pgeon-xai

or:

1. Download the pgeon/ folder and move it into the root directory of your project.
2. Install pgeon's requirements with pip install -r PATH_TO_PGEON_REQUIREMENTS.

See CONTRIBUTING.md for more details on how to install pgeon for development.

Given a Gymnasium environment and a discretizer, you can generate a PG to describe an opaque agent's behavior with fit().

from pgeon import PolicyGraph

pg = PolicyGraph(environment, discretizer)
pg = pg.fit(agent, num_episodes=1000)

There exist two PG-based policies. You can select one or the other with PGBasedPolicyMode.

from pgeon import PGBasedPolicy, PGBasedPolicyMode

greedy_policy = PGBasedPolicy(pg, mode=PGBasedPolicyMode.GREEDY)
stochastic_policy = PGBasedPolicy(pg, mode=PGBasedPolicyMode.STOCHASTIC)

# Passing the policy an observation to get an action
obs, _ = environment.reset()
action = greedy_policy.act(obs)

You can check examples/cartpole/demo.ipynb for a complete breakdown of pgeon's features.

To run the notebook yourself:

1. Download the entire repository.
2. Install pgeon's requirements with pip install -r requirements.txt.
3. Install an extra dependency, rllib, with pip install "ray[rllib]".
4. Open and execute examples/cartpole/demo.ipynb.

If you use the pgeon library, please cite:

Tormos, A., Gimenez-Abalos, V., Vázquez-Salceda, J., & Alvarez-Napagao, S. (2024, May). pgeon applied to Overcooked-AI to explain agents' behaviour. In Proceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems (pp. 2821-2823).

@inproceedings{tormos2024pgeon,
  title={pgeon applied to {Overcooked-AI} to explain agents' behaviour},
  author={Tormos, Adrian and Gimenez-Abalos, Victor and V{\'a}zquez-Salceda, Javier and Alvarez-Napagao, Sergio},
  booktitle={Proceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems},
  pages={2821--2823},
  year={2024}
}