rigvedrs / red-hen-project

All the code for my project done at Red Hen Labs | Creating a multi-modal solution for the Emile Male pipeline

Red-Hen-project

View the final pipeline here

Here I shall be keeping track of all the work I am doing in my project. The work done includes

• Data analysis and preprocessing for obtaining the COCO dataset
• Generating data using style transfer on the COCO using HPC and training the YOLO V8 pose detection model on it
• Implementing another art pose detection transformer and making use of it in the Pipeline
• Training a neural network to classify the poses obtained from the keypoints detected by the transformer
• Created and annotated over 2000 images for training the palm gesture detection model using YOLO
• Created another dataset of 1000 images for classifying and detecting the person in the images and further improving the caption generated by the pipeline.
• Trained YOLO V8 classification and object detection models on these datasets, and created a pipeline combining them to provide the final output.

Blog

Refer my blog to understand everything I have done in case you want to work further on this. You can view the blog for this here

You can also view my progress slides here to view how my journey progressed.

Proposed pipeline

This is the overall pipeline I had proposed for my project with Red Hen Labs. It consists of a palm pose detection module and a body pose detection module that would then classify the combined gesture accordingly.

The pipeline that was finally created is as follows:

Code Map:

The code for obtaining the style transfered data and training the YOLO V8 pose detection model can be found here

The Full Pipeline can be found here