About the paradigm of code-BERT training about codebert

[fengzhangyin]

Yes, we use masked language modeling objective and replaced token detection objective to pre-train CodeBERT.

As stated in the paper, in the fine-tuning step, we only use discriminator to produce general-purpose representations.

Each subdirectory corresponds to the fine-tuning source code of a downstream task. We release the pre-trained model and the source code of downstream tasks, not including the source code of the pre-training step.

[Sewens]

Will there be a further plan to release the pre-training sources?

[fengzhangyin]

Sorry, we don't have this plan.

[Sewens]

Okay, thank you for your patience :)