facebookresearch / dme Goto Github PK

Thanks for this great repo. Is the code for CDME going to be available?

I believe the checkpoint for DME you provide is incompatible with the model that is constructed using SNLI data for task=snli. Mainly, the vocabulary somehow mismatches and doesn't allow the checkpoint's outermost layers, for embeddings, to be loaded into the model so constructed. Of course, we could train our model using the script, but just thought I should mention this in case more people are facing the same issue.

I see that the readme links to the corresponding Arxiv paper. Please update the link to point to the HTML abstract, i.e. to https://arxiv.org/abs/1804.07983 , and not to the PDF. Some readers merely want to read the abstract only. If they want the PDF, they can easily click on the PDF link on the Arxiv HTML page.

"out = F.dropout(out, p=self.args.emb_dropout)"
It seems that this line will trigger dropout during inference.

@kahne

why filtering out the words based on the combined embedding vocabulary in the preprocessing function ? instead of replacing with <unk> ?

For exapmle :

sentence : hide new secretions from the parental units

if the word secretions is not in the emb_vocab ,
the sentence will be converted into hide new from the parental units ,
I think correct sentence should be hide new <unk> from the parental units

Hi, thanks for sharing the code! It really helped me understanding the paper.
I have a question on the way you calculate the attention score, especially on where to apply softmax on.

From this, it seems that the softmax function(g) takes as input "word vectors" FROM TIME STEP 1 TO S, meaning that alpha distribution over the SEQUENCE, not on NUMBER OF PRETRAINED VECTORS.

However, line 203 takes softmax over dim=2, which is NUMBER OF PRETRAINED VECTORS(self.n_emb).

I am little confused about the mismatch here. I think the formula on the paper should be revised. Am I missing something? Please help me!

The readme mentions Python 2 which is almost obsolete. Please upgrade to and formally support Python 3.7 which is the current version. Python 2 code cannot be readily integrated into modern codebases.