lisajamhoury / body_classifier_posenet

Gesture Classifier Using Posenet, TensorFlow.js and ml5.js

🌈 Based on Daniel Shiffman's Color Classifier Example. 🌈 🙏 Thanks to Yining for Posenet + KNN Example.:pray:

Trains model based on recorded posenet data in JSON file. (Will upload the recording code soon).

User has two options:

1. Train model with training data in JSON file, then test based on live input using Posenet.

// to test on live input set testLive on line 18 to true
let testLive = true;

2. Train model with training data in JSON file, then test the model based on the recorded training data. Bad practice, but we're learning here!

// to test from recorded data set testLive  on line 18 to false
let testLive = false; 

The current poses are:

1. T pose: Stand with legs together and arms out in a T
2. Boss pose: Stand with legs together with hands on hips
3. X pose: Stand with legs wide apart and arms over head and wide apart so you look like an X

I included two recordings:

1. fullBodyLisa.json I recorded standing far away from camera. Legs are in full view.
2. closeBodyLisa.json I recorded from about two feet from camera. Legs are not in full view.

Results

It works! It's not so bad considering my data is not so good. Posenet had a lot of trouble catching my arms overhead in pose 3, and you can see that in the training and testing results. I'll test with better data once I have it.

Issues

1. I recorded data in my tiny apartment with bad light and it's not very good. Will record better data at ITP with more space. Maybe today!
2. I'm getting an identifier has already been used error at the top of my sketch.js file. It's not affecting anything, but I would like to figure out where it's coming from. Filed issue here.

What I Want To Do Next

1. Train from live data. I believe I just need to call model.fit each time a user adds data to live train it. Is this correct?
2. Add the recording functionality to repo
3. Record new data.

body_classifier_posenet's Issues

Posenet Keywords length.

Hi,

I am currently learning how to use the Posenet library. So, I am thinking if there's a way to get the length of a detected keyword. e.g say when the webcam detects the "rightArm" is it possible to get the length of that right arm?

Identifier 'testLive' has already been declared

I am getting this error on the first declared variable in my sketch.js file. It's not affecting performance, but I'm not sure where it's coming from.

Live training question

Train from live data. I believe I just need to call model.fit() each time a user adds data to live train it. Is this correct?

The nice thing about using KNN is that the "model" is just the training data itself, there is no additional training step. With the neural network classifier, however, you have to explicitly run fit() after collecting the training data. Ideally you would collect all the data from the user before running fit() you might run into some weird issues running it over and over again for each new data point. But I'd be curious to hear about what happens a variety of different ways!