Nearly 3 points below the provided VQAv2 fine-tuning result about vilt

[dandelin]

I didn't test gradient accumulation for max_epoch based training (=fine-tuning), unlike max_step setting (=pre-training) I've set max_step manually in the max_epoch setting.

Going over the code again, I suspect this line https://github.com/dandelin/ViLT/blob/master/vilt/modules/vilt_utils.py#L247 could be the source of the degradation.

If I'm guessing it right, the gradient accumulation in current fine-tuning will follow wrong, condensed learning rate schedule.

Please delete the line and try again.
Plus, Can you report lr summary in the tensorboard log?

[4444xhc]

@dandelin Thank you for your advice and time. I happened to find that I made some mistakes during the dataset preparation. The code and paper provide sufficient information. Now, I can achieve the same VQAv2 test-dev result as the paper.
Thank you again for your patient help. I will close this issue.