Acc lower than paper claims about prototree

[M-Nauta]

Thanks for your interest in our work and I'm sorry to hear you get a lower accuracy. It is difficult to pinpoint the main reason but it could be that the training hyperparameters, data preprocessing or other settings were not the same as used in the paper. Some default arguments, such as the milestones for lr decay, need to be changed in order to follow the setup of the paper. All hyperparameters are reported in the appendix of the paper. Maybe also check the PyTorch version. Since CUB is a small dataset it could also be that you had a very bad seed.

[huygens12]

The training hyperparameters are set strictly following the paper. I reran the experiments several times. So far, my acc distribution across these runs is [72.4, 78.6, 82.4, 82.4, 80.5]. Seed issue could be the reason. Did this happen to you? And will a larger batch size help?

[huygens12]

Follow-up: I slightly adjusted the hyperparameters to "epochs=150 milestones=60,80,100,120,140" and conducted 5 runs, and the test accs are now [0.828, 0.825, 0.820, 0.831, 0.815]. Hopefully this will help other readers to reproduce the results. Thank @M-Nauta again for this interesting work.

The training hyperparameters are set strictly following the paper. I reran the experiments several times. So far, my acc distribution across these runs is [72.4, 78.6, 82.4, 82.4, 80.5]. Seed issue could be the reason. Did this happen to you? And will a larger batch size help?

[M-Nauta]

I was already looking into potential causes of the issue, but I am glad to hear that your results are now in line with the paper and that you shared your solution here.