Baselines in Code Defect Detection task about codexglue

[guoday]

1. The paper of RoBERTa is here and we use RoBERTa base model. The architecture of RoBERTa is same as CodeBERT but Roberta is pre-trained on text. You can easily use RoBERTa model that we used by changing tokenizer_name and model_name_or_path from microsoft/codebert-base to roberta-base.

2. Yes, the CodeBERT is directly fine-tuned on the Devign dataset.

[wasiahmad]

Instead of RoBERTa, shouldn't RoBERTa (code) be used in the defect detection task? I saw RoBERTa (code) is used in the translation task. I want to confirm that RoBERTa is used in defect detection, instead of RoBERTa (code).

[guoday]

RoBERTa is used in defect detection. You are right and RoBERTa (code) should be used in the defect detection task. Maybe I can do the experiment later.