pramod627 / yolo-on-pynq-z2

This repository contains all the necessary material to implement a YOLOv3 object detection algorithm on the PYNQ-Z2 FPGA. There is a step-by-step tutorial associated so everyone can do it.

⚠️ Disclaimer

This repository is pretty much finished and with all the files necessary to develop de project. There is a step-by-step tutorial on Gitbook that you can read that explains the whole process.

---

YOLO-on-PYNQ-Z2

This project consists on implementing a YOLOv3 object detector on a PYNQ-Z2 PLD, making use of the Zynq-7020 to accelerate the inference and provide accurate results.

In the end, the user should be able to obtain the detection (bounding boxes, class and probability) of an input image. On average the inference time of the YOLOv3 on the PYNQ-Z2 is about 0,45 seconds!

Tutorial

• If you want to follow the whole development process to implement the YOLOv3 on the board, I recommend you check out my tutorial on gitbook! It's pretty in depth, but you should be able to toss out any excessive information. In case you miss any of the steps the Auxiliary files folder contains the necessary files for each part of the project.

• If you want, you can also follow the tutorials presented on each folder as they are more straight to the point and might assume you have previous knowledge.

• Finally, if you just want to execute the YOLOv3 on your PYNQ-Z2 you can download the SD card image here and follow the process on the Requirements chapter of the gitbook tutorial.

Project organization

This project involves 3 steps to get to the final result. The steps are in chronological order, so you first go to the DPU implementation folder and follow the instructions, then to the DNNDK folder and lastly to the Deployment folder. There is also a Metrics folder if you want to obtain the accuracy of your model. Here is a little description of the topics:

1. DPU implementation - Here we create a Hardware layout capable of processing various types of Neural Networks (including YOLO) called the DPU. The DPU is Xilinx's property (You can check the DPU Product Guide) and here we will adopt the best architecture for the PYNQ-Z2 limitations. Then we will create a SD card image with the DPU and other Software configurations.

2. DNNDK - The YOLO model will be compressed and optimized to fit the PYNQ-Z2 board as the DPU has it's limitations and can't use the raw model. To do this we will use Xilinx tool DNNDK v3.1. This program will allow quantization of the model and also compilation, resulting on a file that can communicate with the DPU telling it to form the compressed YOLO Network.

3. Deployment - This is the part where we will set up the board in order to see the results. The files will be organized and compiled resulting on an executable file that is able to run YOLOv3 inference on a desired image.

4. Metrics - You have the YOLOv3 model working on the PYNQ-Z2 and now you want to know the accuracy of the model. On this chapter we will go trough all the steps to do that evaluation so at the end you can see if the detections are accurate enough for your applications.

Development flow

So, how do we tie everything down to make the YOLOv3 object detector work on the PYNQ-Z2? It's simple, we design the DPU with the architecture that fits the PYNQ-Z2 limitations, include that information and the Operating System on the SD card so when we boot the board, the DPU is implemented on the Zynq-7020 chip and the OS is ready with the required packages. Then we compress the YOLOv3 model using the DNNDK v3.1 tool and join the resulting file with a c++ script that can interact with the DPU and do the post-processing of the image. Those files are the included on the OS and after proceeding with the board setup and start-up, we compile the code and execute the YOLO executable associating an image to get detections on.

Help

If you have any doubts or suggestions, express them on the issues tab. I will try to answer everything as soon as possible.

You can also contact me by email at: [email protected]

Acknowledgements

This project was made possible with the help of my professor adviser, my colleagues from the Engineering course and also other creators. First, I want to thank Wu-Tianze for the inspiration he gave me with his project. All of this wouldn't be possible without his work guidelines and his help. Thanks to Jin-Chen for his work on Aiotlab which was really helpful on this project, particularly the first part. Thanks to the community for all the information on the forums and all that shared curiosity. Finally, I want to Thank you for visiting this project!

Wu-Tianze's work: https://github.com/wutianze/dnndk-pynqz2

Jin-chen's work: http://www.aiotlab.org/teaching/fpga.html

PYNQ forum: https://discuss.pynq.io/

Feedback

If you find my work helpfull in any way consider these actions:

• Give the project a nice star ⭐
• Give some feedback on the GitBook tutorial 😃
• Contact me via e-mail 📬

---

Work to be done until product is finished:

• Write chapter regarding the DPU 3.0 development on Vivado;
• Write metrics chapter (if not lazy);
• Create README files for each folder on the repository, introducing everything and making a compressed tutorial;
• Update the repository to match the needs of someone willing to follow the whole process;
• Include a folder with extra files in case the user gets lost halfway through the process;
• Correct my grammar mistakes (probably);
• Create and add some interesting images, so the user's eyes get happy and fulfilled;
• Make a fancy introductory README file and don't forget to thank everyone at the end!;
• Add e-mail contact;