Comments (3)
We have observed that training is more stable when the parameters of the Student network are initialized with the parameters of the Teacher network at the beginning of every epoch in the second stage.
For every iteration within one epoch, the parameters of the Teacher network are updated with an exponential moving average, following standard mean-teacher framework
P.S.
In the usual way using EMA, that is, the mean-teacher framework, it does not initialize the parameters of the Teacher network to those of the Student network at the beginning of every epoch.
from ssl_eln.
We have observed that training is more stable when the parameters of the Teacher network are initialized with the parameters of the Student network at the beginning of every epoch in the second stage. For every iteration within one epoch, the parameters of the Teacher network are updated with an exponential moving average, following standard mean-teacher framework
P.S. In the usual way using EMA, that is, the mean-teacher framework, it does not initialize the parameters of the Teacher network to those of the Student network at the beginning of every epoch.
Do i get it wrong? From the code, it means the parameters of the Student network are initialized with the parameters of the Teacher network ,contrary to what you said?
from ssl_eln.
You are right. The parameters of the Student network are initialized with the parameters of the Teacher network at the every beginning of the epoch.
from ssl_eln.
Related Issues (11)
- Environment installation HOT 2
- GPU Memory HOT 4
- Question about contrastiveloss HOT 1
- Question about performance comparison HOT 1
- Question about the requirements.txt HOT 2
- Question about the setting of iter_per_epoch HOT 1
- Question about contrastive loss HOT 1
- The training code for ELN HOT 1
- How to train the ELN network, the first training stage for ELN is not found. HOT 1
- 关于自建的数据集的问题
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ssl_eln.