deepthi-jallepalli / cmpe-295a-master-project

Federated Learning for Object Detection in Autonomous Vehicles

Federated Learning for Object Detection in Autonomous Vehicles

---

Table of Contents

• Team Members
• Objective
• Dataset
• Data Conversion to YOLOv3 Format
• Data Split
• Evaluation Metrics
• Implementation
• Results
• Steps to Reproduce
• References

Team Members

• Deepthi Jallepalli
• Navya Chennagiri Ravikumar
• Poojitha Vurtur Badarinath
• Shravya Uchil

Objective

Object detection is extensively used in today's world of AI. It is one of the key features of Autonomous Driving Systems. The current object detection models are highly dependent on centralized data to train. The raising concern with this is data privacy. The data privacy issue can be addressed with the Federated Learning(FL) model proposed in this paper. FL architecture aims at preserving data privacy and improving performance by training the model with decentralized data. Object detection models are trained locally at each node on their proprietary dataset, and the resultant weights are securely aggregated at the global server to yield an improved model. Further, we draw the comparison of object detection performance for an FL versus traditional deep learning approach.

Dataset

The dataset used in this project is KITTI Dataset, a real-world computer vision benchmark designed specifically for autonomous driving.
The dataset consists of 7481 training images and 7518 test images. It consists of 8 classes: Car, Van, Truck, Pedestrian, Person sitting, Cyclist, Tram, and Misc. Additionally, few objects are not labelled, perhaps they were too far from the scanner. Hence, they are classified as DontCare.

Data Conversion to YOLOv3 Format

To run YOLO model with Darknet weights, KITTI labels need to be converted to YOLO format. A text file is generated which contains ground truth of the image in the following format:
<object-class> <x> <y> <width> <height>
where x, y, width, and height are relative to the image's width and height.
<object-class> has the image path prepended to it.

Data Split

Dataset is split into eight categories (DontCare excluded), ensuring that each image in a category contains at least one object belonging to that class. Further, these categories are distributed among four participating clients such that each client is deficit of a minimum of one class. Since KITTI dataset is imbalanced, in that 'Car' and 'Pedestrian' classes have more occurrences in comparison to other classes, all clients may perhaps have samples belonging to 'Car' and 'Pedestrian'.

Evaluation Metrics

Mean Average Precision (mAP) and Intersection over Union (IoU) are used as the evaluation metrics in this project. Intersection over Union is a ratio between the intersection and the union of the predicted boxes and the ground truth boxes. mAP compares the ground-truth bounding box to the detected box and returns a score. The higher the score, the more accurate the model is in its detections.

Implementation

• We implemented Object detection using YOLO framework in TensorFlow
• The Federated Learning set up incorporates a client-server model
• We have implemented weights aggregation using Federated Averaging Algorithm
• We are using connection-oriented TCP protocol to establish communication between the server and the clients. This protocol is implemented using Socket library available in Python.
• The weights file and client metadata are encrypted in the client node before transferring them via socket to the server. The server decrypts the files, performs secure aggregation of the weights, and encrypts them back before re sending to the client. We have implemented this using Python package called cryptography.

Results

We ran the Object Detection code individually for four clients over 5 epochs, repeated for 15 communication rounds. Below is the results obtained:

Client	Data Size	Epochs	Comm Rounds	mAP
C1	1615	5	15	68.5%
C2	1725	5	1	66.9%
C3	1662	5	15	64.7%
C4	1632	5	15	64.4%

Comparison between Deep Learning and Federated Learning Approaches:

Training Type	Data Size	Epochs	Total Loss	mAP
Deep Learning	6481	5	21.43	44.5%
Federated Learning (3 rounds)	6481	5	26.26	46.1%
Deep Learning	6481	10	16.30	65.2%
Federated Learning (3 rounds)	6481	10	8.5	63%

Steps to Reproduce

• requires TensorFlow with GPU (The code runs without GPU, but slow)
• download the dataset from http://www.cvlibs.net/datasets/kitti/eval_object.php?obj_benchmark
• install dependencies and download pretrained weights

pip install -r requirements.txt

# yolov3
wget -P model_data https://pjreddie.com/media/files/yolov3.weights

• perform data split

python dataset/data_split.py

• start FL training

# start server
python server_rec_weights.py

# start client
python client_send_weights.py

• perform detection

python detection_custom.py

References

• https://github.com/pythonlessons/TensorFlow-2.x-YOLOv3

cmpe-295a-master-project's Issues

Questions about data split

Thank you for providing the code.
I am trying to reproduce it with reference to Steps to Reproduce, but the data split is not working. Here is the error What should I do?

Traceback (most recent call last):
File "dataset/data_split.py", line 12, in
f = open('/home/014352130/fl_project/kitti_train_backup.txt')
FileNotFoundError: [Errno 2] No such file or directory: '/home/014352130/fl_project/kitti_train_backup.txt'

Questions about data split

Please let me know if there are a few things I don't understand.
Currently I have downloaded the dataset from the official kitti website. (The two files I downloaded are as follows. Is this correct?
on this site
・Download left color images of object data set (12 GB) (data_object_image_2.zip)
・Download training labels of object data set (5 MB) (data_object_label_2.zip))

Then I extracted them in the same directory.
The directory structure is as follows
.
├── testing
│ 　　└── image_2
└── training
　　 　├── image_2
　　　 └── label_2

How can I then convert the data to yolo and do the data split?
I checked the code in data_split.py, but I don't think it includes the conversion to yolo.
Could you please explain in as much detail as possible the steps for converting to yolo and data split?
Thank you in advance.