Comments (16)
@duantiantian I am a bit confused for I just checked the code, it runs normally on my computer.
The error means no audio files can be found, and you just removed the part for loading audio files, then how can it work? ><
Hello, the problem is not code, the problem is command line
change from
python test_all.py --test_root ./0572_0019_0003/audio --test_type audio --test_audio_video_length 99 --test_resume_path ./checkpoints/101_DAVS_checkpoint.pth.tar
to
python test_all.py --test_root ./0572_0019_0003/audio --test_type audio --test_audio_video_length 98 --test_resume_path ./checkpoints/101_DAVS_checkpoint.pth.tar
Because there are only 98 mfcc bin files
from talking-face-generation-davs.
Hi, we recently found out that the audio part is not runnable on Pytorch version >=0.4.0, please check your environment to be the same as ours (Pytorch 0.2.0 with python 2.7).
from talking-face-generation-davs.
@sarah-zhu Sorry that there was a mistake in our testing data loading code, we have now fixed it. Thank you for pointing it out.
from talking-face-generation-davs.
@Hangz-nju-cuhk Thanks for fixing it! Is the audio part runnable on Pytorch version >=0.4.0 now?
from talking-face-generation-davs.
@sarah-zhu there could be a bug on version >= 0.4.0 that needs fixing. We recommend using Pytorch version < 0.4.0 for now. I will be working on releasing the new version for Pytorch 0.4.1 in a month.
from talking-face-generation-davs.
sorry, when I run the testing script to generate videos from audio use the modified code, I still face the following mistake:
Traceback (most recent call last):
File "test_all.py", line 34, in
test_folder = Test_VideoFolder(root=opt.test_root, A_path=A_path, config=opt)
File "/home/dtt/workspace/Talking-Face-Generation-DAVS/Dataloader/Test_load_audio.py", line 92, in init
self.vid = self.loader(self.root, self.A_path, config=self.config)
File "/home/dtt/workspace/Talking-Face-Generation-DAVS/Dataloader/Test_load_audio.py", line 67, in Test_Outside_Loader
raise ("mfccs = 0")
TypeError: exceptions must be old-style classes or derived from BaseException, not str
from talking-face-generation-davs.
when I remove the code
if require_audio: # ipdb.set_trace() k4 = 0 for mfcc_num in pair: # for s in range(-1,2): mfcc_path = os.path.join(path, str(mfcc_num) + '.bin') if os.path.exists(mfcc_path): mfcc = np.fromfile(mfcc_path) mfcc = mfcc.reshape(20, 12) mfcc_block[k4, 0, :, :] = mfcc k4 += 1 # else: # raise ("mfccs = 0")
in “Dataloader/Test_load_audio.py”, the code work!
from talking-face-generation-davs.
@duantiantian I am a bit confused for I just checked the code, it runs normally on my computer.
The error means no audio files can be found, and you just removed the part for loading audio files, then how can it work? ><
from talking-face-generation-davs.
@Hangz-nju-cuhk Thanks. I tested and it works on 0.4.0. Another problem: does this model also work for sentence level audio inputs?
from talking-face-generation-davs.
@sarah-zhu I am not sure what do you mean by "sentence-level audio inputs", maybe our kind of method is segment-level audio inputs? We did crop the mfcc by time shifts to let it align with images, so if this process can be realised online in the dataloader, then it is possible to work with a sentence level mfcc feature.
from talking-face-generation-davs.
@Hangz-nju-cuhk Sorry, I just comment out "else: raise ("mfccs = 0")", I thought after loading all the audio files, the code will raise an error rather break the for loop when the next audio file does not exist.
Thank you very much for your reply!!! ^-^
from talking-face-generation-davs.
@Hangz-nju-cuhk Hi, I tested your model on both LRW and Voxceleb datasets using provided mfcc extraction and face alignment code. However, I found the result is more accurate on Voxceleb dataset than on LRW. Is the provided model trained on Voxceleb dataset?
from talking-face-generation-davs.
@sarah-zhu I am glad that you did further experiments!
If I remember correctly, this model is trained on the training set of LRW only, so were the results provided in our paper and videos. However, we did further experiments to train our model on a small sample of Voxceleb (from the first 100 classes), and we did not find any difference in our testing results on Voxceleb or LRW comparing with before.
I think the main reason is that Voxceleb provides normally cleaner audio clips than LRW, and humans are less sensitive to long-term results in Voxceleb than shorter ones in LRW.
from talking-face-generation-davs.
I'm facing a similar issue. Using pytorch (0.2.0) and python 2.7. The only difference is that I'm not using CUDA. Can this work without CUDA?
from talking-face-generation-davs.
@Hangz-nju-cuhk getting this error using audio in google drive bin files
Traceback (most recent call last):
File "test_all.py", line 34, in
test_folder = Test_VideoFolder(root=opt.test_root, A_path=A_path, config=opt)
File "/content/Talking-Face-Generation-DAVS/Dataloader/Test_load_audio.py", line 92, in init
self.vid = self.loader(self.root, self.A_path, config=self.config)
File "/content/Talking-Face-Generation-DAVS/Dataloader/Test_load_audio.py", line 67, in Test_Outside_Loader
raise ("mfccs = 0")
TypeError: exceptions must be old-style classes or derived from BaseException, not str
from talking-face-generation-davs.
@ak9250 @duantiantian modifiy raise ("mfccs = 0")
to raise Exception("mfccs = 0")
can fix the bug
from talking-face-generation-davs.
Related Issues (20)
- Hello HOT 1
- No change HOT 5
- Run testing script result not as good as demo HOT 2
- result file HOT 2
- Pre-Processing Data HOT 5
- Code running without CUDA and pytorch version HOT 1
- What's the meaning of the parameter --test_audio_video_length? HOT 1
- Chinese characters are spoken faster than English words, will this model work on Chinese? HOT 4
- About preprocess HOT 3
- May I ask about post processing detail? HOT 1
- Problem with run model on CPU
- face_align HOT 1
- How to turn the output result of test_all into a video (image + audio) form
- How can I conviently convert audio files into .bin files
- 请教一下关于mouth.txt数据的处理过程
- Solution for ValueError: expected 2D or 3D input (got 4D input)
- pretrain checkpoing tarball looks like not one tar file
- question about computing contrastive loss
- Non-human Characters HOT 1
- During the reproduction process, No module named 'lws'
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from talking-face-generation-davs.