a7b23 / distilling-the-knowledge-in-neural-network Goto Github PK

Excuse me @a7b23 Unfortunately, I can't reach the accuracy in the student network when running distill.py. In my attempt, the result is that test accuracy of the student model is 0.098. Would you please take a look at it?

The student model didnot arrive 96% at your code because the learning rate is too small.
If try learning rate =0.2 or loss=2*loss, studet model would be accurace>96.5% at this code .
The distill.py have a improvement at this code just because the studet loss=loss1+loss2, the loss2 could be replaced by loss1, do not need the teacher model.