vadev / handsign-tensorflow-gatsby

A simple ASL (American Sign Language) alphabet detection using TensorFlow and Handpose model

Handsign - ASL Hand Gesture Detection Using TensorFlow

Handsign is a simple AI-based hand gesture recognition that translates a hand pose into the American Sign Language (ASL) alphabet. Using Tensorflow JS and its Handpose preloaded model to detect the hand object and its parts. Handsign also uses an additional library called Fingerpose to classify certain of custom hand gestures based on the finger position.

#dohackathon #madewithTFJS

See Demo

Installation

1. Clone the repository

# copy the repo to your machine

git clone https://github.com/syauqy/handsign-tensorflow-gatsby.git

2. Start the project

# move to the project folder and install all dependencies

cd handsign-tensorflow-gatsby
yarn install

3. Run the project on your local machine

# run Gatsby

yarn develop

4. The project is live 🚀

Your project is live and running at http://localhost:8000

You can edit the core program at src/pages/app.js

What's inside the project

Extract the fingerpose data

uncomment the <pre> component

<Image h="150px" objectFit="cover" id='emojimage'/> 

// uncomment this
{/* <pre className="pose-data" color="white" style={{position: 'fixed', top: '150px', left: '10px'}} >Pose data</pre> */}

</Container>

uncomment the estimatedGestures data to change '.pose-data' innerHTML

// document.querySelector('.pose-data').innerHTML =JSON.stringify(estimatedGestures.poseData, null, 2);

the estimatedGestures data will render on your screen.

References & Libraries

• Tensorflow JS - A Library for ML in JS.

• Handpose - A lightweight ML pipeline consisting of two models: A palm detector and a hand-skeleton finger tracking model.

• Fingerpose - A pose classifier for hand landmarks detected by TensorFlow.js Handpose's model.

• Sign language illustration is created by Pelin Kahraman

If you want to learn more about Tensorflow JS and custom gesture handpose, please kindly check these amazing videos

• Machine Learning with TensorFlow in JavaScript - by Jason Lengstorf and Jason Mayes

• Building a Real Time Sign Language Detection App with React.JS and Tensorflow.JS - by Nicholas Renotte

Deploy to your site