Clarification on files in config_gan_train.yaml about nmt_gan

[ZhenYangIACAS]

• There is no difference between the train.src_path and generator.src_path. Actually, they are not used at the same time. The former is used to pre-train the model and the latter is used to train the generator during the gan training.
• The dis_positive_data and the dis_negative_data is generated automatically. In our experiments, it will includes 5000 sentence for training the discriminator during gan training.
• the parameter λ is not defined in the config file, just set in the model.py
• We stop the training when the model achieves no improvement on the development sets. The process for the evaluating is the same with the testing. We didn't test the PPL score for evaluation.

[zoharai]

Hi @ZhenYangIACAS ,
Thanks for the answer!
You said that the dis_positive_data and the dis_negative_data are generated automatically, but we need to generate it for the first iteration of the first epoch right?
Another question about the evaluation - did you use evaulate.sh for evaluation?
When did you evaluate bleu score on the dev set? every epoch?
Did you stop training when the model achieves no improvement for the tenth evaluation on the dev set or first evaluation or another number?

Thanks!

[borgr]

After the preprocessing you start training with GANs, do you use early stopping there as well (with patience 10?) or do you have another way to decide when to stop training? (similar to @zoharai's last question)

[ZhenYangIACAS]

@zoharai @borgr During the GAN training, you do not need to generate the dis_positive_data and dis_negative_data even for the first epoch. You can find the logic behind in the gan_train.py. For the evaluation, we stop training when the model achieves no improvement for the tenth evaluation on the dev set, which has been clarified in our paper. And we evaluate the model at regular intervals.

[xieexiaotuzi]

@zoharai @ZhenYangIACAS Sorry to bother both of you. I am still confused about the evaluation. I can't find it where it is used. Could help me ?