GithubHelp home page GithubHelp logo

jixinya / eamm Goto Github PK

View Code? Open in Web Editor NEW
178.0 178.0 18.0 10.1 MB

Code for paper 'EAMM: One-Shot Emotional Talking Face via Audio-Based Emotion-Aware Motion Model'

License: MIT License

Python 100.00%

eamm's People

Contributors

jixinya avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

eamm's Issues

Dataset process

Hello! Thank you for sharing the code. Could you please also share dataset process? I want to train your model from scratch. Thank you

RuntimeError: Unable to open ./shape_predictor_68_face_landmarks.dat

run and error:

(EAMM) root@ubuntu-desktop:~/EAMM# python demo.py
Traceback (most recent call last):
  File "demo.py", line 44, in <module>
    predictor = dlib.shape_predictor('./shape_predictor_68_face_landmarks.dat')
RuntimeError: Unable to open ./shape_predictor_68_face_landmarks.dat

work directory:

(EAMM) root@ubuntu-desktop:~/EAMM# ls
3DDFA_V2         demo               LICENSE            modules          README.md         %SystemDrive%  venv
augmentation.py  demo.py            log                ops.py           requirements.txt  teaser-1.png
config           filter1.py         logger.py          process_data.py  run.py            test
dataset          frames_dataset.py  M003_template.npy  __pycache__      sync_batchnorm    train.py

log:

(EAMM) root@ubuntu-desktop:~/EAMM# tree log -L 2
log
├── 124_52000.pth.tar
├── 1-6000.pth.tar
└── 5-3000.pth.tar

What's going on? ./shape_predictor_68_face_landmarks.dat is missing or not submit?

code and pretrained model weights not match

hi xinya,

i found the AT_net2 use jaco_net = 'cnn', but the model weights you provide is 'gan', so configs and weights are not match, hope you can revise it soon, thx

duomin

Module Missing: sync_batchnorm

Hi,
First, thanks for the awesome repo. I am encountering the following error while trying to run the demo.py

Module sync_batchnorm not found
This is happening due to the call in Line-7 of modules/util.py (this line)

Can you kindly suggest a fix for this ?

Thanks,
Avisek

Missing VoxCeleb1_Cut/right.npy

Hello,

Thank you for your contribution. I'm trying to run training, but it seems like we're missing the videos at VoxCeleb1_Cut/right.npy:

FileNotFoundError: [Errno 2] No such file or directory: '/mnt/lustre/share_data/jixinya/VoxCeleb1_Cut/right.npy'   

Can you provide this file?

Errors in run.py

Hello, I found a few errors in run.py file:

  1. You have an extra space in from of train_part2 function on the last line of run.py
  2. ImportError: cannot import name 'get_logger' from 'modules.util', so basically there is no get_logger in modules.util. Could you fix it?

Strange eye shape

Hi! Thank you for sharing your code. When I run the test code, the right eye is much larger than the left eye in the result. Is that correct or some config parameters might be wrong? Can I fix this through finetuning it on my own dataset?

How to get pose_given for demo.py?

Thank you for the great work. I wanted to ask regarding the parameter "--pose_given" that is used in demo.py file, how do I get this for an image?

Errors in train.py

Hello, I found a few errors in train.py:

  1. Traceback (most recent call last):
    File "/fsx/nikitadrobyshev/EAMM/run.py", line 121, in
    train_part1(config, generator, discriminator, kp_detector, kp_detector_a,audio_feature, opt.checkpoint, opt.audio_checkpoint, log_dir, dataset, test_dataset,opt.device_ids, name)
    File "/fsx/nikitadrobyshev/EAMM/train.py", line 26, in train_part1
    optimizer_generator, optimizer_discriminator,
    NameError: name 'optimizer_generator' is not defined

  2. File "/fsx/nikitadrobyshev/EAMM/train.py", line 55, in train_part1
    generator_full = TrainPart1Model(kp_detector, kp_detector_a, audio_feature, generator, discriminator, train_params,device_ids)
    File "/fsx/nikitadrobyshev/EAMM/modules/model.py", line 164, in init
    super(TrainFullModel, self).init()
    NameError: name 'TrainFullModel' is not defined

Emotion Displacement Learner should be train for each emotion?

Hello,
I am trying to generate some videos using the pre-trained models, however even though I change the emotion the code seems to generate videos for the same emotion, therefore I was wondering if the Emotion Displacement Learner should be train for each emotion.
Thanks so much,
Jenifer

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.