How to deal with Out of Vocabulary tokens? about textgan-pytorch

[williamSYSU]

I am not sure if I fully understand your question. There are two vocabularies for training data and test data. The vocabulary of training data is initialized (or loaded) at the begin of the program (as in main.py) based on the given training data file. The two vocabularies share common words, while the vocabulary of test data may have additional words that are not in the training data. The training data and the test data are loaded by load_dict() and load_test_dict() in utils/text_process.py, respectively.
That is, before training the model, the vocabularies of training data and test data have included all tokens we need. Thus, there should not be an Out of Vocabulary Tokens error while running the program.

[lethaiq]

@williamSYSU thanks for the response it took me a while to figure it out that in your code we do not have constraints on the vocabulary size. We can add this option to the current code base, it might be helpful!