1ellejade / feddrivetest

FedDrive: Generalizing Federated Learning to Semantic Segmentation in Autonomous Driving

Official implementation of FedDrive: Generalizing Federated Learning to Semantic Segmentation in Autonomous Driving

by Lidia Fantauzzo^*,1, Eros Fanì^*,1, Debora Caldarola¹, Antonio Tavera¹, Fabio Cermelli^1,2, Marco Ciccone¹, Barbara Caputo¹.

Official Website: https://feddrive.github.io/.

Corresponding authors: [email protected], [email protected].

^*Equal contribution. ¹All the authors are supported by Politecnico di Torino, Turin, Italy. ²Fabio Cermelli is with Italian Institute of Technology, Genoa, Italy.

Citation

If you find our work relevant to your research, or use this code, please cite our IROS 2022 paper:

@inproceedings{feddrive2022,
  title={FedDrive: Generalizing Federated Learning to Semantic Segmentation in Autonomous Driving},
  author={Fantauzzo, Lidia and Fanì, Eros and Caldarola, Debora and Tavera, Antonio and Cermelli, Fabio and Ciccone, Marco and Caputo, Barbara},
  booktitle={Proceedings of the 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year={2022}
}

Summary

FedDrive is a new benchmark for the Semantic Segmentation task in a Federated Learning scenario for self-driving cars. It consists of three settings and two datasets, incorporating the real-world challenges of statistical heterogeneity and domain generalization. FedDrive is a benchmark of state-of-the-art algorithms and style transfer methods taken from the Federated Learning, Domain Generalization, and Domain Adaptation literature, whose objective is to improve the generalization ability and robustness statistical heterogeneity robustness of the model. We demonstrate that correctly handling normalization statistics is crucial to deal with the aforementioned challenges. Furthermore, style transfer dramatically improves performance when dealing with significant appearance shifts.

Setup

1. Clone this repository

2. Move to the root path of your local copy of the repository

3. Create the feddrive new conda virtual environment and activate it:

conda env create -f environment.yml
conda activate feddrive

4. Download the Cityscapes dataset from here. You may need a new account if you do not have one yet. Download the gtFine_trainvaltest.zip and leftImg8bit_trainvaltest.ziparchives

5. Extract the archives and move the gtFine and leftImg8bit folders in [local_repo_path]/data/cityscapes/data/

6. Ask for the IDDA V3 version of IDDA, available here

7. Extract the archive and move the IDDAsmall folder in [local_repo_path]/data/idda/data/

8. Make a new wandb account if you do not have one yet, and create a new wandb project.

9. In the configs folder, it is possible to find examples of config files for some of the experiments to replicate the results of the paper. Run one of the exemplar configs or a custom one:

./run.sh [path/to/config]

N.B. change the wandb_entity argument with the entity name of your wandb project.

N.B. always leave a blank new line at the end of the config. Otherwise, your last argument will be ignored.

Results

Cityscapes

Method	mIoU ± std (%)
FedAvg (uniform)	45.71 ± 0.37
FedAvg	43.85 ± 1.24
FedAvg + CFSI	41.50 ± 0.98
FedAvg + LAB	39.20 ± 1.37
SiloBN	44.20 ± 1.43
SiloBN + CFSI	40.48 ± 1.40
SiloBN + LAB	42.23 ± 1.23

IDDA

The seen and unseen columns refer to the results for the test client that contains images from the same training domains and to the test client that contains images from other domains, respectively. Results are reported in the form mIoU ± std (%).

Method	Setting	seen	unseen
FedAvg (uniform)	country	63.57 ± 0.60	49.74 ± 0.79
FedAvg	country	42.43 ± 1.78	40.01 ± 1.26
FedAvg + CFSI	country	54.70 ± 1.12	45.70 ± 1.73
FedAvg + LAB	country	56.59 ± 0.90	45.68 ± 1.04
FedBN	country	54.39	-
SiloBN	country	58.82 ± 2.93	45.32 ± 0.90
SiloBN + CFSI	country	61.22 ± 3.88	49.17 ± 1.01
SiloBN + LAB	country	64.32 ± 0.76	50.43 ± 0.63
FedAvg	rainy	62.72 ± 3.35	27.61 ± 2.80
FedAvg + CFSI	rainy	38.18 ± 1.40	26.75 ± 2.32
FedAvg + LAB	rainy	55.24 ± 1.65	31.05 ± 2.68
FedBN	rainy	56.45	-
SiloBN	rainy	62.48 ± 1.42	50.03 ± 0.79
SiloBN + CFSI	rainy	63.04 ± 0.31	50.54 ± 0.88
SiloBN + LAB	rainy	65.85 ± 0.91	53.99 ± 0.79