Question about reproduce result about tap

[zyang-ur]

Hi @JayZhu0104 ,

Could you please provide additional details? The first step in debugging could be to separate the pre-training and fine-tuning steps. E.g., 1. whether fine-tuning from the released pre-training work? 2. what is the training from scratch baseline on your side (the 44.50 in the table)?

The smaller batch size could influence the performance. You could check its influence by comparing the training from scratch baseline (44.50) in the table.

Feel free to send me an email if I wasn't checking the git repo that frequently :)

Best,
Zhengyuan

[HenryJunW]

@JayZhu0104 May I know how many GPUs you're using for training and the corresponding training time? Thanks!