selecting pretraining checkpoints / monitoring pretraining performance about albef

[LiJunnan1992]

Hi,

(1) I always use the last checkpoint
(2) I'm not sure what you refer to by runs, because I only pretrain once
(3) no I haven't monitored with downstream tasks during pretraining

[jayleicn]

Thanks! For (2) I thought you'd tune the pretraining a bit for best performance.

[LiJunnan1992]

To keep things simple, I haven't tuned much on the pretraining hyperparameters.

[jayleicn]

Thanks! Then the method becomes even more convincing!!