Continue pretraining on custom dataset about electra

[DayuanJiang]

@ViktorAlm Did you solved this problem? I am encountering the same issue.

[ViktorAlm]

The necessary inputs from the training is not in the files, you would have to load the weights in from another checkpoint with everything in it or something. I'm not not sure exactly how, it was just an experiment on my part so it wasnt that much of an issue for me. I'm just speculating though.

[hyusterr]

@clarkkev I suffer similar problem here, do i have to specify checkpoint or something? electra-small can't continue as you mentioned in readme.
thanks in advanced, here is my command
python3 run_pretraining.py --data-dir ori-vocab-data-mda --model-name electra_small --hparams '{"model_size": "small", "num_train_steps": 4020000, "electra_objective": true, "vocab_file": "ori-vocab.txt", "learning_rate": 1e-1, "train_batch_size": 128, "generator_hidden_size": 1.0}' date