Unable to Replicate Text Classification Results about compact-transformers

[stevenwalton]

I'm not quite sure what's going on without looking too closely, but you can see here that we basically only call torch's embedding which expects longs. This just looks like a casting issue to me. Are you double embedding by accident? Or is your input data float instead of long?

The code is pretty straight forward and honestly any embedder should work. The call graph is just embedder -> text tokenizer -> MaskedTransformerClassifier. Modifications should be fairly trivial as all our stuff is in the latter two.

[SethPoulsen]

I read in the paper that you used GloVe, so I ran the data set through GloVe on my own because I didn't see that happening anywhere in the codebase. The output of that was floats, which doesn't match the longs that are being expected by your embedding layer, as you say.

[stevenwalton]

https://medium.com/mlearning-ai/load-pre-trained-glove-embeddings-in-torch-nn-embedding-layer-in-under-2-minutes-f5af8f57416a