Why some of the data is repeated while training? about chatlearner

[bshao001]

1. Because some data are more important than others. You may have heard about publications talking about how to maintain a consistent personality of a chatbot, this is a practical way to do it. You manual create a small data set, in which, you answer questions about the name, age, birthday, gender, place of living, basic personal interest of the chatbot. You can also try to get rid of the possible conflicting pairs in the base dataset, and repeating your important data, therefore, your model can remember what you want it do remember. You can think that, the base dataset is to train the model to learn grammar, but the dataset you are repeating is to train the model for knowledge. That's same way as a human to learn: you spend more time on more important/useful knowledge, while you also need to read a lot of books, such as novels.

2. It depends on your model and your data. Normally, I suggest the epoch number be in 50 to 100. Generally, a small model with more epochs may get a slightly better result, but may take you much longer time to train.

Hope these make sense to you.

[sainimohit23]

@bshao001 thank you for your response. I have doubts about the small model. Right now I'm using embedding_size = 1024 and num_units = 1024, as used in your code. I don't have a sense of a smaller model. So, do these values come in a range of small model values, large model values or moderate models?

[bshao001]

There is no absolute large model or small model. It is described based on your data set. You have to experience quite several rounds of training and testing (involve some manual review and check) in order to get a feeling what is a good size of the model for your data set. When you reach that step, you may fine tune your model, in which case, you will understand what I have originally said above.