lysdexic-audio / n4m-handpose

Wraps MediaPipe HandPose inside electron and serves the detected parts via MaxAPI.

lysdexic-audio 👋

I make sound and interaction design & composition tools. MaxMSP, C++, JS, Python etc.

For the last few years I've been full-time as a Senior C++ Audio Software Engineer at Aimi.fm - which takes most of my time. I continue to create and update repos here from time to time, please create issues for bug reports and feature requests.

🏢 Primary / work account: https://github.com/jcurtis-cc

---

Google Mediapipe projects

Max - jweb

• Pose (body) : https://github.com/lysdexic-audio/jweb-pose-landmarker
• Hand tracking: https://github.com/lysdexic-audio/jweb-hands-landmarker
• Hand gestures: https://github.com/lysdexic-audio/jweb-hands-gesture-recognizer
• Face tracking: https://github.com/lysdexic-audio/jweb-facemesh
• Face gestures: https://github.com/lysdexic-audio/jweb-face-landmarker
• Object detection: https://github.com/lysdexic-audio/jweb-object-detection

Max - node for max

• Handtracking : https://github.com/lysdexic-audio/n4m-handpose
• Facetracking : https://github.com/lysdexic-audio/n4m-facemesh

n4m-handpose's Issues

Download HandPose locally

Hello :)

Thanks to you, I am soon making a pre-release of my SP recognition tool!

Because I am going to use it during the summer in places where there is no internet, I was wondering if there was a way to download the model locally rather than fetching it from the web. I would dig into that after my master thesis defence, but perhaps you would know how to make it?

Also, just let me know if you could not solve the crashes and add gui parameters for some reason, I could try my best to gelp with that also!

Arthur

this browser does not support video capture,or this device does not have a camera

Big fan of this tool, Thanks for sharing, but cannot get to recognize camera anymore.

Summary or problem
After downloading modules i start the script. No user interface is visible and I receive this error after 1 minute: "this browser does not support video capture, or this device does not have a camera" I do not get this with facetracking tool which is working fine.

When I open 'developer tools' 1 error and 3 warnings : shown in order of appearance_

Warning:

1. Platform browser has already been set. Overwriting the platform with [object Object].
   Environment.setPlatform @ C:\Users\dante\Documents\Max 8\Packages\n4m-Bundler\n4m-handpose-master\n4m-handpose-master\node_modules@tensorflow\tfjs-core\dist\environment.js:38

2. (electron) Security Warning: webFrame.executeJavaScript was called without worldSafeExecuteJavaScript enabled. This is considered unsafe. worldSafeExecuteJavaScript will be enabled by default in Electron 12.
   electron/js2c/renderer_init.js:15

3. Electron Security Warning (Insecure Content-Security-Policy) This renderer process has either no Content Security electron/js2c/renderer_init.js:135 Electron Security Warning (Insecure Content-Security-Policy) This renderer process has either no Content Security. electron/js2c/renderer_init.js:135

4. ERROR: Uncaught (in promise) DOMExecption: Could not start video source - n4m-handpose_camera.js 445

5. WARNING: DevTools failed to load SourceMap: Could not load content for file:///C:/Users/dante/Documents/Max%208/Packages/n4m-Bundler/n4m-handpose-master/n4m-handpose-master/node_modules/ajv.js.map: _This warning occurs for every module (my guess) there are like a hundred all leading to the modules folder. _

system Windows 11, Max 8.1.11

Add GUI parameters / fix crashes mentionned in camera.js

Hello!

It would be really nice to get more of the GUI to change some confidence parameters and control more of the model.

In the camera.js file, I see that things were commented because of crashes. Maybe things could be taken from the 2020 example from tensorflow?

https://github.com/tensorflow/tfjs-models/blob/master/handpose/demo/index.js

Install npm

Hi,
Thanks for sharing this fantastic tool.
I want to try and work on it, but the installation is not working on my computer. (Install dependencies from npm. Wait until the box below says 'completed.')
I tried a few times and waited very long, and nothing happened. It can not download the files.
Is there another way to download the files and run the software on Max?

I am using max 8.5 on a MacBook Pro (macOS 12.5).

Greetings,
Ali

Intermittent crashes on MacOS (Mojave)

Cause uncertain. Seems to only occur when filling the max dictionary, with the gate to the dict closed crashes don't seem to occur.