Problem with the dataset MP-20 about cdvae

[txie-93]

I think this is due to training instability, which we also see during our experiments. My experience is that you can restart the training by loading the latest checkpoint. The training can usually continue. You can decrease the learning rate if it becomes too unstable.

[ubikpt]

Thank you very much for all your answers ! Now that you talk about that, I think I also got this error once with the carbon dataset. At that time, I attributed it to my experiments (python library upgrades and some changes in your code to try to eliminate warnings). About the learning rate, how can I decrease it ? I found the parameter lr in conf/optim/default.yaml, reduced it to half, but still crashing ...