haixpham / end2end_au_speech Goto Github PK

OS: linux

Unable to execute ShapeUtils2.py , many of the functions and variables , require opengl library ,which is not imported , even if added ,shows errors

Hi @haixpham, thanks for making the code available for the community. I am working on reproducing your results as part of research work. You have provided a link for AU Labels for RAVDESS dataset. How would i create/acquire Labels.npy files, if i want to do train/inference on any arbitrary video/audio pair.

Thanks for this amazing work.
Do your recommend a specific configuration for better results? I am getting too bad predictions compared to the ground truth.
One more thing. Do the results of this repo represent your journal paper entitled "Learning Continuous Facial Actions from Speech for Real-time Animation"?

Thanks in advance

Hi haixpham,
I have some trouble when reproduce your code.
I found my features length are always not equal to the AU labels when using RAVDESS dataset .
if it is right ?
Waiting for your reply, thanks a lot .

Hi haixpham,
I have some trouble when reproduce your code.
How should I use a trained model to drive the face model? There is no details in your paper. And can I drive in real time?
Waiting for your reply, thanks a lot .

I am trying to implement your paper and use it for comparison with other methods i am trying. However since it is based on CNTK and we cant train it on colab because it requires Ubuntu-16.04 only i am unable to train the model.
Hence i would like to know if we could access the pretrained model.

As title and appreciate you!

Firstly, thank you very much for opening source code. How to extracted blendshapes curves from Generated video frames ? Or is there an intermediate parameter blendShape before generating the video frame?

What are the contents of this file?