ksw0306 / clarinet Goto Github PK

Is pretrained model availible?

there is a lot of noise in the high frequency?? do u have any solution?

It is needed to roll back to the previous version. Then the changes were not saved, but the network worked, although in fact without "clone_as_averaged_model()". Later I will try to do PR on this issue

when trying to run synthesize_student, I get the following error:

/usr/local/lib/python3.5/dist-packages/torch/nn/functional.py:995: UserWarning: nn.functional.tanh is deprecated. Use torch.tanh instead.
warnings.warn("nn.functional.tanh is deprecated. Use torch.tanh instead.")
/usr/local/lib/python3.5/dist-packages/torch/nn/functional.py:1006: UserWarning: nn.functional.sigmoid is deprecated. Use torch.sigmoid instead.
warnings.warn("nn.functional.sigmoid is deprecated. Use torch.sigmoid instead.")
0.6159329414367676 seconds
Traceback (most recent call last):
File "synthesize_student.py", line 133, in
librosa.output.write_wav(wav_name, wav, sr=22050)
File "/usr/local/lib/python3.5/dist-packages/librosa/output.py", line 216, in write_wav
util.valid_audio(y, mono=False)
File "/usr/local/lib/python3.5/dist-packages/librosa/util/utils.py", line 157, in valid_audio
raise ParameterError('Audio buffer is not finite everywhere')
librosa.util.exceptions.ParameterError: Audio buffer is not finite everywhere

After further investigation, I found that most of the entries in wav are nans.

Any ideas what caused this? Does this work for anyone else?

BTW, I'm trying to severely overfit on my training examples to make sure that the model is capable of learning, so I'm training for a very long time, could this be the cause? My teacher is at step 395000 and my student is at step 9000

as i known, Clarinet is a end-to-end model(Text-to-Speech). But this model allows only the .npy file as the inputs. Can anyone use a sentence to synthesize ?
what's more , i wonder the function of the Clarinet. can it realize the multi-speakers synthetise? or just make synthesize results better?

Hi, Why is teacher model loaded when inference? Is it necessary? How to disable it?

Hi, I use my owner data to train teacher model and student model. The teacher model is normal; but the KL loss of student model becomes nan at about 50k step. The log information is like as follows:
Global Step : 67438, [55, 100] [Total Loss, KL Loss, Reg Loss, Frame Loss] : [ nan nan 2.301 nan]
100 Step Time : 143.40849208831787
Global Step : 67538, [55, 200] [Total Loss, KL Loss, Reg Loss, Frame Loss] : [ nan nan 2.3145 nan]
100 Step Time : 142.97523188591003
Global Step : 67638, [55, 300] [Total Loss, KL Loss, Reg Loss, Frame Loss] : [ nan nan 2.3899 nan]
100 Step Time : 142.70189571380615
Global Step : 67738, [55, 400] [Total Loss, KL Loss, Reg Loss, Frame Loss] : [ nan nan 2.2744 nan]
100 Step Time : 142.38775205612183
Global Step : 67838, [55, 500] [Total Loss, KL Loss, Reg Loss, Frame Loss] : [ nan nan 2.436 nan]
100 Step Time : 142.91834259033203
Global Step : 67938, [55, 600] [Total Loss, KL Loss, Reg Loss, Frame Loss] : [ nan nan 2.3337 nan]
100 Step Time : 142.9723343849182
Global Step : 68038, [55, 700] [Total Loss, KL Loss, Reg Loss, Frame Loss] : [ nan nan 2.5294 nan]
100 Step Time : 142.89931344985962
Global Step : 68138, [55, 800] [Total Loss, KL Loss, Reg Loss, Frame Loss] : [ nan nan 2.3567 nan]
100 Step Time : 143.14595890045166
Global Step : 68238, [55, 900] [Total Loss, KL Loss, Reg Loss, Frame Loss] : [ nan nan 2.3591 nan]
100 Step Time : 143.44508004188538
Global Step : 68338, [55, 1000] [Total Loss, KL Loss, Reg Loss, Frame Loss] : [ nan nan 2.2139 nan]
100 Step Time : 143.32597756385803
Global Step : 68438, [55, 1100] [Total Loss, KL Loss, Reg Loss, Frame Loss] : [ nan nan 2.402 nan]
100 Step Time : 142.90216040611267
Global Step : 68538, [55, 1200] [Total Loss, KL Loss, Reg Loss, Frame Loss] : [ nan nan 2.4041 nan]
100 Step Time : 142.8380832672119
55 Epoch Training Loss : nan
100 [Total, KL, Reg, Frame Loss] : [ nan nan 2.042 nan]