pntt3011 / mediapipe_face_iris_cpp

Real-time Face and Iris Landmarks Detection using C++

🐱 My GitHub Data

🏆 36 Contributions in the Year 2022

📦 1.1 MB Used in GitHub's Storage

🚫 Not Opted to Hire

📜 23 Public Repositories

🔑 29 Private Repositories

I Mostly Code in Python

Python                   12 repos            ███████░░░░░░░░░░░░░░░░░░   30.0% 
Kotlin                   8 repos             █████░░░░░░░░░░░░░░░░░░░░   20.0% 
Jupyter Notebook         5 repos             ███░░░░░░░░░░░░░░░░░░░░░░   12.5% 
C++                      5 repos             ███░░░░░░░░░░░░░░░░░░░░░░   12.5% 
JavaScript               5 repos             ███░░░░░░░░░░░░░░░░░░░░░░   12.5%

Timeline

Last Updated on 09/11/2022 18:47:21 UTC

mediapipe_face_iris_cpp's Issues

Crash during Debug

Hello.
I built your repo and Release works like a charm. But it crashes during Debug. Interpreter and its subgrapsh have lot of garbage. Did you encouner such issue?
Thanks in adnvance

How did you manage to get the models as tflite?

I wonder how you transformed the mediapipe models to tflite ones.

When I using still images to make a video, the result landmark coordinates are still changing

Hello, I tried to use still images to make a video, the result landmark coordinates are still changing (not stable).
(using video 1920x1080)
Is this correct?

And one more thing, I found it seems that left iris result landmark is not as correct as right one.
I am not sure about this point, and I will take more tests.

Thanks.

Question about depth estimation

I see in code comments that I can get depth estimation from the model output inside getEyeLandmarkAt by float _d = model->getOutputData(isIris)[index * 3 + 2];. Can you please say what kind of postprocessing needs to be done on this output? Seems there should be some postprocessing, because by printing the value I get also negtaive values.

Get the 3d coordinate output of the model

The output data seems to be different from python，where am i doing wrong?

Question about resolution settings causing inaccuracy

I tried to change the resolution to 1280*720 and found that it was no longer accurate. Is it because of the model? If yes, does it mean that I need to retrain the model?

In addition, there will be jitter, even if I stay in front of the camera motionless. Is there any good solution for this. so many Thanks!

How to adjust this code for hand detection?

Could someone please guide me as to what changes I should make to the files of this repository to get the hand detection solution running?

Change resolution

When I use the following statement to change the resolution of the camera (changing the resolution from the default 640 * 480 to 1280 * 720), why does the average frame rate increase instead?
cap.set(CV_CAP_PROP_FRAME_WIDTH, 1280);
cap.set(CV_CAP_PROP_FRAME_HEIGHT, 720);
1280*720：

640*480：

I hope you can answer, thank you!

Info request : What are the values output by the face_detector.tflite ?

The Regressor shape: (1, 896, 16) and Classifier shape: (1, 896, 1) values output by face_detector.tflite when there is only one facein the image are undocumented, or I could not find anywhere their meaning.

lets pretend thatthe classifier outputs a score for each of the 896 potential bounding boxes to have a face inside, then the highest value of the classifier is tied to 16 valuesof the regressor .
I could not find what these mean...for instance I got these values :

Classifier highes raw value: [1.8791088]
Regressor raw values related to the Classifier Highest raw value: [ -9.997869 5.4132013 97.73436 97.71959 -31.320122 -15.798918 13.305289 -12.395595 11.063564 11.03928 -12.75334 30.372728 -53.83273 -10.672908 34.493393 -3.749894 ]
what are their meaning? thanks for the info :)

About the ROI

The landmarks detected in this project are sometimes inaccurate and jittery.After reading the code, I think it is because the ROI here is inconsistent with the ROI croped by the original processing method of mediapipe.In addition to find the face area, I think it is necessary to do affine transformation to the area before feed it to landmark model.Also, mediapipe uses the landmark-refine model, is it possible to implement it in this project?

关键点准确度的问题

请问下，当在640480的分辨率时，关键点的位置看起来比较准确，当我设置成1280720时，关键点检测的位置就不太准确了，特别是脸部轮廓的位置，会超出脸部的范围，请问这个是什么原因呢？

How to make code work on FreeBSD/Linux?

I'm getting the error during cmake build:

c++: error: no such file or directory: '/MP'

windows_x86_64/libtensorflowlite.so.if.lib

hello!The download link given is the result of compilation for X86_Linux. Could you provide the download for X86_WIN

Do you have mediapipe_hand_landmark_cpp ?

How to extend this project for selfie segmentation?

Hi,
This repo is the only repo I could find to use mediapipe in a C++ application. Thanks for sharing it.

Can you please show the right way of adding a class for selfie segmentation and create a demo similar to the python example?

can this project be compiled to a .dll ?

This project is really good, but can you tell me how can i use the code in another project? in a dll way or other way? thank you very much

License

Hi, good job, thank you for that.
If you want to allow others to use it, you should consider adding a license to it.

https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/licensing-a-repository

Otherwise it won't be possible because of copyright laws.

I found that tf models you used in your code is not the same as official tf models

I don't know if you have done some operation to make your models have the same output as offcial models

Output vector not same as mediapipe

Firstly, very nice implementation. However, I am facing some issues understanding the output of your work. I ran through your face detection code, and the output of your model is (896,4). However, the original output from mediapipe is (896,16). Can you please tell me how to get the additional 4 keypoints associated with face please? Also, where should I make changes in your code if I want my output to be exactly similar to mediapipe. Thank you!

How easy to adopt your code for object detection.

I would like to reuse/generalize your code with ssdlite_object_detection.tflite. How would NUM_SIZES and AnchorOptions look like in this case (assuming that DETECTION_SIZE = 320, NUM_BOXES = 2034, NUM_COORD = 4)?

Please help with hand detection

Hi,

I would like to kindly ask you for help with hands detection using mediapipe and tensorflow.
I used your awesome project (forked here https://github.com/AdrianPeniak/mediapipe_face_iris_cpp.git) and port it to linux env.
However when I reuse your classes for hand detection using palm model I can't get good results (hands not detected, drawing random roi according code in repo). I understand that functions as getHighestScoreDetection() etc. must be rewritten, but I afraid that problem is somewhere before (e.g. image preprocessing).

Could you please take a look to my forked project?

Thanks for any advise.

Thanks

why the input image is flipped around y axis?

Hi, why you flip the input image from cam aroud y axis?

Linux system

Hello, can you send the original tensorflow Lite so file (applicable to Linux system), which was accidentally deleted after downloading. Thank you very much!

tensorflow 2.4.4 and bazel build //tensorflow:libtensorflow_cc.so in MacOs 12.1

bug ld: warning: dylib (/private/var/tmp/_bazel_also/c6fd75dad1daa641f8f9e0b5d840292d/execroot/org_tensorflow/bazel-out/darwin-opt/bin/tensorflow/libtensorflow_cc.so) was built for newer macOS version (12.1) than being linked (12.0)
ld: warning: dylib (/private/var/tmp/_bazel_also/c6fd75dad1daa641f8f9e0b5d840292d/execroot/org_tensorflow/bazel-out/darwin-opt/bin/tensorflow/libtensorflow_framework.2.dylib) was built for newer macOS version (12.1) than being linked (12.0)
Undefined symbols for architecture x86_64:

failed to allocate tensors for face_landmark_with_attension model

Thank you @pntt3011 for your good work.
I was trying to use the face_landmark_with_attension model to get more accurate eyelids points.

But it gets failed to allocate tensor. Can you please help/suggest resolving this issue?

error log:

INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
ERROR: Encountered unresolved custom op: Landmarks2TransformMatrix.
ERROR: Node number 192 (Landmarks2TransformMatrix) failed to prepare.

ERROR: Failed to apply the default TensorFlow Lite delegate indexed at 0.
Failed to allocate tensors.