RoBERTa-wwm-ext-large应用到全新领域不收敛 about chinese-bert-wwm

[ymcui]

1、输入文本不需要经过中文分词，直接使用bert原始的tokenizer处理。
2、学习率太大了，一般都是在1e-5的量级周围。
3、finetune用这个batch size没什么问题。

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale]

Closing the issue, since no updates observed. Feel free to re-open if you need any further assistance.

[yyxx1997]

我也遇到了类似的问题，将这里提到的roberta chinese进行分类任务微调，使用cls+pooler+mlp的方式不改动模型本身，发现损失基本不降，准确率不升，lr5e-5，bs128 按理说lr已经跟1e-5一个数量级了啊