A question about parameter assignment in the training process about ssl_eln

[kinux98]

We have observed that training is more stable when the parameters of the Student network are initialized with the parameters of the Teacher network at the beginning of every epoch in the second stage.
For every iteration within one epoch, the parameters of the Teacher network are updated with an exponential moving average, following standard mean-teacher framework

P.S.
In the usual way using EMA, that is, the mean-teacher framework, it does not initialize the parameters of the Teacher network to those of the Student network at the beginning of every epoch.

[zjuxiaobaiq]

We have observed that training is more stable when the parameters of the Teacher network are initialized with the parameters of the Student network at the beginning of every epoch in the second stage. For every iteration within one epoch, the parameters of the Teacher network are updated with an exponential moving average, following standard mean-teacher framework

P.S. In the usual way using EMA, that is, the mean-teacher framework, it does not initialize the parameters of the Teacher network to those of the Student network at the beginning of every epoch.

Do i get it wrong? From the code, it means the parameters of the Student network are initialized with the parameters of the Teacher network ，contrary to what you said?

[kinux98]

You are right. The parameters of the Student network are initialized with the parameters of the Teacher network at the every beginning of the epoch.