How to calculate and evaluate the ppl of D1 and D2. about bob

[haoyusoong]

have you ever taken a look at the generated results?

[haoyusoong]

The ppl calculation: https://huggingface.co/transformers/perplexity.html

[haoyusoong]

However, as long as the final output of the model, the d2 score, improves, we don't need to worry about the d1 score.
Please tell us why you decided that epoch 7 (Perplexity on test set is 21.037 and 7.813.) is optimal.

PPL is just one of the indicators, and there are many other metrics. Our goal is to generate good dialogue responses rather than getting the extremely low ppl. The Epochs > 15 are usually overfitted on the ppl metric and suffer a significant quality drop of the responses. In our test run, epoch 7 delivers good responses and has a competitive performance on all metrics, including the relatively good ppl (cf. baselines).

[iyo-0713]

I see. I understand now.
Thank you very much for answering my question.

[Wenze7]

Hi, bro, I would like to ask that have you reproduced the results mentioned in paper?

[iyo-0713]

Hi, bro. I could not reproduce the results mentioned in paper.

[Wenze7]

That's too bad, i tried to contact with the author, but I never receive reply. Do you know of any papers that use NLI and can reproduce the results？

[iyo-0713]

Uhm...
My friend also used this model, but couldn't reproduce the result. I think it is difficult to reproduce the result in this paper.
I changed the model using in my research. I don't know other models using NLI.

[Wenze7]

Fine, thanks!