CARL: Congestion-Aware Reinforcement Learning for Imitation-based Perturbations in Mixed Traffic Control
Paper accepted to IEEE CYBER 2024 (ArXiV)
This work was done on top of FLOW framework obtained on Jan 3, 2023.
Developed and tested on Ubuntu 18.04, Python 3.7.3
- Install Anaconda
- Setup SUMO versio 1.15.0
- Clone this repository
- Use the following commands to complete setup.
conda env create -f environment.yml
conda activate flow
python setup.py develop
pip install -U pip setuptools
pip install -r requirements.txt
- For the data processing (car-following filter), follow the Notebook: Car-Following trajectory analysis
- Follow the Notebook: Imitation Learning
- Follow the Notebook: Ring
If you want to use the trained CSCs, see Data
section below.
Go to the folder ring/Ours
and enter the command:
# Train our policy in Ring (at 5% penetration)
python train.py singleagent_ring
# Train our policy in Ring (at penetrations >5%)
python train.py multiagent_ring
All scripts related to this part are consolidated in the Notebook: Evaluate Ring
Replace the method name to be one of: ours, wu
python test_rllib.py [Location of trained policy] [checkpoint number] --method wu --gen_emission --num_rollouts [no_of_rollouts] --shock --render --length 260
For all (replace the method_name to be one of: bcm, lacc, piws, fs, idm)
python classic.py --method [method_name] --render --length 260 --num_rollouts [no_of_rollouts] --shock --gen_emission
For stability tests where a standard perturbation is applied by a leading HV, include --stability to the line above
To evaluate the generated rollouts into Safety and Efficiency metrics, replace the method name to be one of: idm
, bcm
, fs
, piws
, lacc
, wu
, ours
.
python eval_metrics.py --method [method_name] --num_rollouts [no_of_rollouts]
To add plots to the metrics, include --save_plots
For Stability plots
python eval_plots.py --method [method_name]
-
Trained Policies (at all penetration rates):
-
Trained CSC Models: HuggingFace
-
Experiment Data (including rollouts and data for plots): HuggingFace
@article{poudel2024carl,
title={CARL: Congestion-Aware Reinforcement Learning for Imitation-based Perturbations in Mixed Traffic Control},
author={Poudel, Bibek and Li, Weizi and Li, Suhai},
journal={arXiv preprint arXiv:2404.00796},
year={2024}
}