4uiiurz1 / keras-cosine-annealing Goto Github PK

As per my understanding of the paper, the authors use 'Tcur' which denotes the number of epochs since the last restart. But I see that you have used epoch instead of it.

Is there any specific reason for using epoch directly?

I see that the epochs in the upward cosine curve don't make any significant impact on the scheduler. Instead, once the eta_min is reached, the Tcur variable could be restarted.

Hi,
this implementation changes the LR after each epoch.
It should do this after each batch.
What do you think?