Killed while initialize feature about cw2vec

[dalinvip]

80G corpus ? maybe it's out of memory, be killed.

[Jun-jie-Huang]

It only takes 20% of the memory, and I can run 'fasttext' with the 80G corpus freely, so I guess it's not out of memory? It's killed while initializing stoke feature.

[dalinvip]

try modify max_vocab_size rebuild https://github.com/bamtercelboo/cw2vec/blob/master/word2vec/src/include/dictionary.h#L43.
I did not use such a large data set for training, it may be a problem of setting restrictions, you can try to modify.