loss about mprnet

[swz30]

Hi @ldlshizhu

There is no need to use two GPUs for a batch size of 1.

I would suggest trying a low initial learning rate, e.g., 1.5e-4

[ldlshizhu]

Hello, thank you for your patience!
Your suggestion is very effective for me. After changing the learning rate to 1.5e-4, the loss was about 100 in the first epoch and dropped to 70 in the 69th epoch, but it suddenly became 800,000 in the 70th epoch! I don't know what happened…

[swz30]

Hi @ldlshizhu

If you run codes with the default settings, there should not be an issue.

Try even lower learning rate: 1e-4.

[ldlshizhu]

Hi @ldlshizhu

If you run codes with the default settings, there should not be an issue.

Try even lower learning rate: 1e-4.

Thank you very much! I'll try again!