TPU training: No matching devices found for about electra

[stefan-it]

Well, I compared the TPU configuration with the code in the official BERT repository.

It seems that ELECTRA specifies the following argument:

tpu_job_name=("train_tpu_worker" if config.do_train else
                    "lm_eval_tpu_worker"),

See complete context here.

And this train_tpu_worker name is not the same as for my TPU (with is only worker).

After I removed that argument, training on a v3-8 TPU is working 🎉

[clarkkev]

Thanks for finding this issue! tpu_job_name is actually useful when we run ELECTRA internally, so I added it the configure_finetuning instead (with it defaulting to None so the issue should be fixed): 4e0f2a5

[stefan-it]

@clarkkev do you think we can savely remove the tpu_job_name argument here 🤔 I could prepare a PR for that!