mismatch on the attention formula in the paper and implementation about dme

[kahne]

Hi @hwijeen ,

For a fixed time step j, we have unnormalized attention weight for each embedding i. They are computed using the same function (the part inside function phi). The input of the function is the full sequence of word vectors from embedding i (for DME, only the vector at time step j is actually used; for CDME, all vectors are used). And then function phi (softmax/sigmoid) is applied to dimension i to normalize these weights.

Hope this answers your question.

[hwijeen]

for DME, only the vector at time step j is actually used;

This answers my question. Thank you!