Multi-GPU training about electra

[sadhikamalladi]

@008karan @Palang2014 How are you getting the model to run on a GPU? Even with a GPU available, I'm only able to run on CPU. Mostly interested in running the fine-tuning, not the pre-training.

[zxybazh]

I've tried the current starter's code for pretraining a small network. It seems like the model is trained on a single GPU.

[008karan]

Yes. It is working on a single GPU. Looking for multi-gpu support. @clarkkev

[hamidpalangi]

Gentle follow up Kevin, any thoughts?

Thanks,
-Hamid

[008karan]

Is there any plans for multi-gpu support @clarkkev

[MarkClemens301]

Hope to support multiple GPU and provide detailed configuration. @clarkkev

[kukrishna]

@008karan @Palang2014 How are you getting the model to run on a GPU? Even with a GPU available, I'm only able to run on CPU. Mostly interested in running the fine-tuning, not the pre-training.

I had the same issue and I realised it was because the program could not find the cuda libraries. Check if you get messages like
"Successfully opened dynamic library libcudnn.so.7"
If not, or if you see errors saying it couldn't find some cuda libraries, maybe you have the same problem?