hipatil / alpr

This repo is created for SIH-2020 Grand Finale

Number Plate Detection

Table of content (Click Here to Expand)

• Number Plate Detection
  • Important Link
  • Initial Setup
  • How to Run the Pipeline
  • Dataset Info
  • Problem Statement
  • Pipeline
    • Vehicle Type Detection
      • Detected Classes:
    • Plate Detection
    • Alphanumeric Digits Detection
    • Convert to String
    • Data to Cloud
    • Web App Integration
    • Android App Integration
  • Contributors

Important Link

• PPT Link:
  • Main PPT: Google Slides
  • Day 2 PPT: Google Slides
• YouTube Video: Working Demo
• Business Plan: Google Docs
• WebApp: GitHub Repo
• Android App: GitHub Repo
• Sample Test results: Videos Folder

Initial Setup

1. Clone this repository git clone https://github.com/take2rohit/CB31_IvLabs_WIMDR.git
2. Change your directory cd CB31_IvLabs_WIMDR
3. Make your virtual environment either using virtualenv or conda virtualenv sih -p python3 (optional)
4. Activate the virtual environment source sih/bin/activate (optional)
5. Install all dependencies pip3 install -r requirements.txt
6. Make sure all the NVIDIA Drivers are installed for best performance
7. Download three weights and place them in CB31_IvLabs_WIMDR/weights/ folder.
   • OCR detection: Google Drive
   • Vehicle Detection: Google Drive
   • Number Plate: Google Drive

How to Run the Pipeline

1. Open the file yolov4_test.py locate call of main() function (At the end of the file).
2. Change your video file name in video_file variable.
3. Open terminal and run python3 yolov4_test.py
4. If you are not getting required output try changing hyper-parameters of code

Note:

• In case of RuntimeError: CUDA out of memory. please change variable use_cuda as use_cuda=False
• If the code not working try changing hyperparameters like window_size, frame_skipping, confidence, etc.

Dataset Info

Total size of all database = 10 GB+

• Manually labelled Vehicle and Plate Dataset: Link
• Manually labelled OCR dataset: Link
• Manually labelled OCR dataset (MixOrg): Link
• Manually labelled License Plate Dataset: Link
• Pre-labelled OpenImage Dataset: Link

Problem Statement

1. Localities face persistent threat of security due to illegal parking, theft and unregulated entry/exit.
2. We aim to automate the registration of entry/exits of every commute to ensure round the clock surveillance.
3. Database with registered entry can be used to prevent theft.
4. Can be extended for parking management in malls, shopping complexes, theatres, etc.

Problem Statement ID: CB31 Problem Statement Organizer: MixORG

Pipeline

1. Vehicle Type Detection: Classifying the vehicle, whether it is a car, bus, truck, etc.
2. Plate Detection: Segmenting the number plate.
3. Alphanumeric Digits Detection: Segmenting the number plate digits from the number plate.
4. Convert to String: Each digit is concatenated to form a string.
5. Data to Cloud: When the final string is obtained, it is pushed in the cloudbase.
6. Web App Integration: Vehicle information shall be visible on the web application to the admin. More information can be found here
7. Android App Integration: A user - friendly android app, rich in features is also developed. More information can be found here
8. Deployment: Integration of steps 1 to 6 for realtime deployment in different scenarios.

Vehicle Type Detection

First the image is acquired through a camera placed at an optimal position from where the vehicle and number plate are recognizable. Starting with vehicle type detection, it detect the type of the vehicle such as whether it is a car, truck, bicycle, etc. (YOLOv4 has been used, trained on COCO dataset for vehicles).

Detected Classes:

• Car
• Motorbike
• Truck
• Number Plate

Plate Detection

The step focusses on the detection of number plate, which was trained with some of our own dataset (gathered and annotated by us) as well as some from of Open Images Dataset. This was trained with YOLOv4 architecture. The results we got were not only robust but also accurate.

Alphanumeric Digits Detection

The number plate detected is now segmented and passed through another YOLOv4 architecture (multiclass classification), trained again on some of our own dataset as well as with some dataset from Kaggle. In order to incorporate the number plates with more than one line (such as on trucks and buses), we perform affine correction to make the plate symmetric about x and y axis (make it rectangular). Now, the plate is read from left to right and top to bottom.

Convert to String

The digits are read from the previous step and coverted to string.

Data to Cloud

The obtained string is then pushed to the firebase, which links the final web integration to record the dynamic changes. Each number plate is mapped to the number of visits made through a particular gate or society and also whether the vehicle is authorized to pass the gate or not.

Realtime Implementation: Google Drive

Web App Integration

The details about any activity of user's car will be updated on the web app. The administrator can monitor the traffic flow and prevent blacklisted vehicles from passing. For every entry/exit of vehicle, its snapshot will be processed and stored, for future reference.

Realtime Implementation: Google Drive

Android App Integration

A user - friendly android app has been created, so the user can register his/her vehicle and link with their car. The car can be registered easily by scanning a QR code maintained by the society. Other features include an interactive dashboard, UPI payment for parking charges, notice board for society, vigilance mode to send notification about any activity related to the user's vehicle and provision to add more than one vehicle per user.

Realtime Implementation: Google Drive

Contributors

• Rohit Lal - website
• Himanshu Patil - LinkedIn
• Khush Agrawal - website
• Rishesh Agarwal - LinkedIn
• Arihant Gaur - website
• Akshata Kinage - LinkedIn