zhongyi-zhou / gestureimt

Gesture-aware Interactive Machine Teaching with In-situ Object Annotations (UIST 22)

Hi there 👋

• 😃 I'm a postdoc at RIKEN
• 🔭 I study both HCI and CV, focusing on how these two disciplines can jointly contribute to future interactive systems
• 📧 email: zhongyi.zhou.work [at] gmail.com
• 💻 personal website: https://zhongyizhou.net/
• ✋ Be free to contact me for dicussions or collaborations!

gestureimt's Issues

Strong Delay and camera selection

I have the following two problems while testing the project with a Philips SPC 1330NC, Intel RealSense and AzureKinect. The graphics card is a RTX 2080 Ti and the CPU Intel Xeon with 3.1GHz.:

1.the frame rate of the camera is always 5-10 fps, which in itself would not be so bad. However, when I point my hand into the camera, a few minutes pass until I see my hand in the GUI. So it is a very strong delay. Do I have a buffer somewhere or is the problem somewhere else?

2.the project always seems to detect the first camera. Is there a way to select the camera yourself? Because it doesn't work for me or the mask of the object recognition doesn't work correctly when I rewrite the code.

In the file GestureIMT/tree/main/demo_app/public/assets/js/webrtc.js in line 23 your code looks like this : document.getElementById("webcam").srcObject = e.streams[0];

If I change it to the following, the mask doesn't work anymore but I can select the camera itself:
const stream = await navigator.mediaDevices.getUserMedia(constraints);
window.stream = stream; document.getElementById("webcam").srcObject = stream;

This is one of many examples I have tried.

I would appreciate any help especially for the first problem and thank you.