thepowerfuldeez / facemesh.pytorch Goto Github PK

Hello, I would like to ask how to use the three-dimensional coordinates of the image. I see that the value range of the third dimension is all numbers less than zeros, I am not sure how to convert them
Thank you so much for your work.,you're so great.

Hi,
is there a method put full img as input into model and get the right position of detections mapped into ori img?

Hi, thank you for providing useful code.

I ran predict_on_batch.
Input size is 128x128
but I didn't work because of tensor size...

you comment that size must be 128 pixel but it didn't work
I got this error

could you provide 128x128 pixel model??
[email protected]

Interesting project. Is it possible to use facemesh to change face and make nose smaller? How can I do that?

Thanks for wonderful work. However, i would like to train this facemesh model with our own dataset which has 98 keypoints.
Could you please help me how we can start?
What type of loss function we should use as we don't know what mediapipe has used during their training.
Although you have already mentioned that we should as the BatchNorm layer for fine tuning. Could you help me which places we should use BatchNorm. In both FaceMeshBlock and FaceMesh block?

This is your result.

but the result from mediapipe is

obviously, your reproduce result is tooooooo bad!

I cannot find facemesh-lite.f16.tflite from https://github.com/google/mediapipe . Where can we download a facemesh model??

Cheers

firstly, thanks for your excellent work, but I found torch result is slightly different from the original tflite model:

tensorflow and torch version:

(Pdb) tf.__version__
'2.2.1'
(Pdb) torch.__version__
'1.5.1'

test code:

import os
import numpy as np

import tensorflow as tf
import torch
import torch.nn as nn

from facemesh import FaceMesh

import cv2
sample_img = cv2.imread("test.jpg")
sample_img_192 = cv2.resize(sample_img, (192, 192))
input_data = np.expand_dims(sample_img_192, axis=0).astype(np.float32) / 127.5 - 1.0

interpreter = tf.lite.Interpreter(model_path="facemesh-lite.f16.tflite")
interpreter.allocate_tensors()

net = FaceMesh()
net.load_weights("facemesh.pth")

# Get input and output tensors.
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()
input_shape = input_details[0]['shape']
# input_data = np.array(np.random.random_sample(input_shape), dtype=np.float32)

# tf inference
interpreter.set_tensor(input_details[0]['index'], input_data)
interpreter.invoke()
tf_coord_res = interpreter.get_tensor(output_details[0]['index'])

# torch inference
torch_output_data = net(torch.from_numpy(input_data.transpose(0, 3, 1, 2)))
torch_coord_res = torch_output_data[0].detach().numpy()

print(["torch", torch_coord_res[0]])
print(["tflite", tf_coord_res[0, 0, 0]])
print("diff %f" % (np.abs(torch_coord_res[0] - tf_coord_res[0, 0, 0]).mean()))

results:

#==>
['torch', array([ 94.1816  , 140.77983 , -14.322037, ..., 136.51678 ,  88.71278 ,
         6.525924], dtype=float32)]
['tflite', array([ 92.17496 , 139.39285 , -14.361812, ..., 134.26816 ,  87.34091 ,
         5.629876], dtype=float32)]
diff 1.167073

I think this may caused by the option conv2d("padding=same" ) different from tensorflow, have you fixed this problem or some advise?

thanks!

This model performs very well in my testing. Could you please share the dataset and the training code?