Question about EMA about softteacher

[MendelXu]

See

SoftTeacher/ssod/utils/hooks/mean_teacher.py

Line 37 in 5be1762

def before_train_iter(self, runner):

[songbae]

See

SoftTeacher/ssod/utils/hooks/mean_teacher.py

Line 37 in 5be1762

def before_train_iter(self, runner):

sry for keep asking , another question is EMA only updates before_train_iter? and interval other parameters are fixed? like interval, momentum ?

[MendelXu]

Yes, if you want to adjust these parameters, maybe you can write an extension.
I implement a simple multi-step momentum adjustment but have not tested it:

SoftTeacher/ssod/utils/hooks/mean_teacher.py

Line 52 in 5be1762

def after_train_iter(self, runner):

[songbae]

Yes, if you want to adjust these parameters, maybe you can write an extension. I implement a simple multi-step momentum adjustment but have not tested it:

SoftTeacher/ssod/utils/hooks/mean_teacher.py

Line 52 in 5be1762

def after_train_iter(self, runner):

thanks! !!

[lhphanto]

Sorry to came back on this old thread.

I see for EMA update, the momentum itself is dynamically decayed.

This is a little bit different from for example https://www.tensorflow.org/api_docs/python/tf/train/ExponentialMovingAverage

Just wonder if dynamically decaying the momentum was found to be very helpful? Or it was relatively minor improvment?

Thanks!

[MendelXu]

No. The implementation to decay the momentum is not used in our experiments. It is just an option provided for the user.