johnglover / sound-rnn Goto Github PK

I am currently trying to find the reason for the NaNs in the loss function. I found the guilty already, which are zero values in the norm variable inside the function pdf, which then causes divisions by zero. A no-brainer workaround would be to simply add a small value to both the variances and the norms, but maybe you have a better idea?

I'm getting these NaNs really soon (around the 500th iteration) in training when using the GPU (which might be due to the float16 representation) and mini-batches of 64 samples. I don't get them when using the fix I proposed, so it might be worth adding it to the repository. I can submit a PR if you think this fix is acceptable.

Hey, do you know if anyone has got this working with opencl/amd cards?

I managed to get char-rnn/word-rnn working by changing a single line that gets the program to use the proper tensor type. Do you think a similar fix might be possible?

I'm wondering if you could give me any insight to how and where CUDA is being used here and if you know of any way I might be able to replace it with opencl. I am a RNN/ML noob, so anything helps!

Thanks.

After running the sine example described in the README with -gpu_id 0 (so running with CUDA), attempts to generate samples with that model generates the error unknown Torch class <torch.CudaTensor>.

I tried adding require 'cunn' and require 'cutorch' to sample.lua, but that generates another error about invalid arguments to addmm.

Running Arch Linux 4.9.11-1 on x64 laptop. GPU is GTX 1060.