lnsmith54 / super-convergence Goto Github PK

I'm trying to reproduce evidence of superconvergence in Figure 1A shown below:

I am using the following values for the solver.prototxt

net: "/home/jovarty/git/super-convergence/architectures/Resnet56Cifar.prototxt"
test_iter: 200
test_interval: 100
display: 100
lr_policy: "triangular"
base_lr: 0.1
max_lr:  3.0
stepsize: 5000
max_iter: 10000
solver_mode: GPU
weight_decay: 1e-4
momentum: 0.9

I have implemented triangular cyclical learning in Caffe as specified here.

I am using Resnet56Cifar.prototxt as the network.

I am achieving final training accuracy of ~90% but final test accuracy as low as 10-20%.

I note that the paper specifies that you achieve these results with large batch sizes of ~1,000 images. However, I wouldn't expect smaller batch sizes (125, as specified in Resnet56Cifar.prototxt) to completely destroy the results.

Are there any additional steps I must take to reproduce this work?

Dear author:
I can't use your prototxt immediately because "caffe.SolverParameter" has no field named "max_lr".
Could you provide the caffe code you use? thank you very much.