The model would generated one repeated sentences about relgan

[weilinie]

Main issues:

1. Synthetic data: generating repeated sentences (extreme mode collapse) when gpre_lr=0.005, while your model behaves normal under gpre_lr=0.01.

Right after pre-training, the nll_gen is also very high (~0.65 as you showed). So I think it’s because by setting gpre_lr=0.005 and npre_epochs=150, the pre-training may not be sufficient. A quick suggestion is to increase the npre_epochs (and/or increase the inverse temperature) to see if you can get good results.

2. Image COCO data: generating repeated sentences when temperature=1.

It would be as expected since there is a tradeoff between sample quality and diversity, tuned by the maximum inverse temperature. In the extreme case where temperatue=1, which means no temperature control at all, the model will suffer from severe mode collapse.

3. Reasons of generating repeated sentences:
   Very sensitive to temperature?
   Type of adversarial training loss?
   Or others?

For the gumbel-softmax trick, the temperature control plays a crucial role in the overall performance. So yes, it is mainly because “very sensitive to temperature”.

4. Could you please explain why your model would arise the problem of generating repeated sentences? Or what modules do you think would lead to this problem?

The gumbel-softmax trick.

5. Do you have any suggestions for solving the mode collapse problem?

I would recommend to improve the gumbel-softmax trick. In this work, we just use the vanilla version of the gumbel-softmax with some temperature control. I believe there is still large room for improving this module. For example, REBAR would be the first thing to try.

6. Previous works like SeqGAN and LeakGAN don’t have extreme mode collapse (only generate one repeated sentence) even the temperature is set to 1. Do you think the temperature exceed 1 is essential to your RelGAN?

Yes, temperature>1 is essential for RelGAN from the temperature control perspective. SeqGAN and LeakGAN do not reply on temperature control as they apply REINFORCE, so they are less sensitive to the temperature.

[williamSYSU]

I am very grateful that you can take your time to answer my questions in detail and patiently. Your answer does help me have a better understanding of your RelGAN, while I am also confused about another thing.

According to my understanding of your code, the calculation process of g_pretrain_loss and nll_gen is exactly the same, except for the parameters of the “generator”. In fact, nll_gen is the g_pretrain_loss calculated by the "generator" whose parameters have been updated after pre-training. Therefore, the value of g_pretrain_loss and nll_gen should be close, and the value of g_pretrain_loss should be generally larger than the value of nll_gen from the training perspective. However, from the log file with gpre_lr=0.005 on Synthetic Data, the g_pretrain_loss is already small (~1.7) while the nll_gen is still large (~6.3). According the above analysis, this situation should not happen.

Is there mistake of my understanding or my analysis? Or is the way you calculate the g_pretrain_loss and the nll_gen different?

[weilinie]

I think the difference between g_pretrain_loss and nll_gen mainly lies in how each of them is calculated over mini-batches: For nll_gen, we fix the generator parameters and take average of g_loss over all mini-batches (please refer to nll_loss() in Nll.py). For g_pretrain_loss, however, we always first adapt the value of g_loss to each mini-batch and then take average of the adapted g_losss over all mini-batches (please refer to pre_train_epoch() in utils.py). It explains why g_pretrain_loss is lower than nll_gen.

[williamSYSU]

Thank you again for your answers and code :)