roboeden / flatland-marl

A multi-agent reinforcement learning solution to Flatland3 challenge.

Flatland-MARL

This is a multi-agent reinforcement learning solution to Flatland3 challenge. The solution itself is elaborated on in our paper.

Jiang, Yuhao, Kunjie Zhang, Qimai Li, Jiaxin Chen, and Xiaolong Zhu. "Multi-Agent Path Finding via Tree LSTM." arXiv preprint arXiv:2210.12933 (2022).

Install Dependencies

Clone the repository.

$ git clone http://gitlab.parametrix.cn/parametrix/challenge/flatland-marl.git
$ cd flatland-marl

The code is tested with Python 3.7 and is expected to also work with higher versions of Python. If you are using conda, you can create a new environment with the following command (optional) :

$ conda create -n flatland-marl python=3.7
$ conda activate flatland-marl 

Install flatland

We found a bug in flatland environment that may lead to performance drop for RL solutions, so we cloned flatland-rl-3.0.15 and fixed the bug. The bug-free one is provided in folder flatland-rl/. Please install this version of flatland.

$ cd flatland-rl
$ pip install .
$ cd ..

Install other requirements

$ pip install -r requirements.txt

Build flatland_cutils

flatland_cutils is a feature parsing package designed to substitute the built-in flatland.envs.observations.TreeObsForRailEnv. Our feature parser is developed in c++ language, which is much faster than the built-in TreeObsForRailEnv.

$ cd flatland_cutils
$ pip install .
$ cd ..

For WSL2 users only

The game rendering relies on OpenGL. If you are wsl2 user, it is very likely that you don't have OpenGL installed. Please install it.

$ sudo apt-get update
$ sudo apt-get install freeglut3-dev

Usage

Quick demo

Run our solution in random environments:

$ cd solution/
$ python demo.py

In a terminal without GUI, you may disable real-time rendering and save the replay as a video.

$ python demo.py --no-render --save-video replay.mp4

Test as Flatland3 challenge round 2

Generate test cases

We provide a script to generate test cases with the same configuration as Flatland3 challenge round 2. The generation may take several minutes.

$ cd solution/
$ python debug-environments/generate_test_cases.py

Test in a specific case

$ python demo.py --env debug-environments/Test_3/Level_0.pkl

Run the whole Flatland3 challenge round 2

Install redis

The Flatland3 challenge is evaluated in Client/Server architecture, which relies on redis. Please go to https://redis.io/docs/getting-started/ and follow the instructions to install redis.

Install ffmpeg

We relies on ffmpeg to generate replay videos.

$ sudo apt-get install ffmpeg

Start flatland-evaluator

First, start redis-sever.

$ sudo service redis-server start

Then start the flatland-evaluator server.

$ redis-cli flushall
$ FLATLAND_OVERALL_TIMEOUT=999999 flatland-evaluator --tests ./debug-environments/ --shuffle False --disable_timeouts true

Open another terminal, and run our solution.

$ cd solution
$ python remote_test.py --save-videos

Replays are saved in solution/replay/.

Bibtex

If you use this repo in your research, please cite our paper.

@article{jiang2022multi,
  title={Multi-Agent Path Finding via Tree LSTM},
  author={Jiang, Yuhao and Zhang, Kunjie and Li, Qimai and Chen, Jiaxin and Zhu, Xiaolong},
  journal={arXiv preprint arXiv:2210.12933},
  year={2022}
}

flatland-marl's Issues

error when install flatland_cutils

Missing training scripts

@JYuhao88 @liqimai Would you please add the training scripts for predictability?

Error when installed the flatland environment

Hi, I am studyidng your interesting work. However, when I installed the flatland environment you provided, the following error occured. Could you help me deal with it? Thank you !

how can I train the Neural network？

I'm a rookie.I would love to be able to repeat the training process, but I can only run the provided network model on the local machine. I would like to try the process of training the neural network, can you give me a link to a simple tutorial? thank you。