Hand Gesture Recognition 🤙

Given an image containing a single hand 🖐, our system classifies the hand gesture into one of six digits from 0️⃣ to 5️⃣. It handles different lighting effects⚡ and different hand poses as much as possible.

A complete machine learning pipeline, composed of the following modules:

Preprocessing
Feature extraction
Classification

Wait for the classification stage to see our magical touch ✨

Preprocessing

This module aims to handle the variation in the images color, texture, size, orientation and lighting conditions. It bridges the gap between different hands of same gesture by applying the following steps:

Resize the image for faster processing
Convert the image to HSV
Extract S and V channels
Get the thresholds of hand and apply it in both channels
Bitwise AND the two thresholded images
Filling the empty regions
Resize region filled mask to be compatable with original image size
Remove padding
Horizontally flip the image, if required
Extract OCR of the image from the previous step

Note: We tried about 🔟 other methods. This was the most robust one🤖

Feature Extraction

This module aims to describe the image with a constant size feature vector. We tried various options to achieve this. Here they are:

Using OCR extracted from the previous stage
Histogram of Oriented Gradients
SIFT (failed)
ORB (failed)

Classification

This module includes our trained model, or should we say our trained models

We mainly depend on two classical ML algorithms [Support Vector Machines - Random Forest]

Loads 3 different models
- HOG descriptor, SVM
- HOG descriptor, RF
- OCR descriptor, SVM
Runs each of them on a single thread
Collects votes from them
Classifies the image according to the majority of votes

The model ouputs the HOG_SVM result in case a tie happens

How to run our model on your test set

# Clone this project
$ git clone https://github.com/{{YOUR_GITHUB_USERNAME}}/hand-gesture-recognition

# Access
$ cd Hand-Gesture-Recognition

# Install dependencies
$ pip install -r requirements.txt

# Go to the run directory
$ cd .\src\final

Put the expected.txt in the same directory

Put your images in the 'data' folder

It's better to label your images from 1 to n

# Go back to the root directory
$ cd ..\..\
# Run the following batch script
$ .\src\final\run.bat

# The results.txt and time.txt will be generated in the same directory
# The script will output the accuracy as well