About the mutli-choice task in t-gif qa about hcrn-videoqa

[eustcPL]

I found the problem.

ans_candidates_agg = ans_candidates.view(-1, ans_candidates.size(2))  
ans_candidates_len_agg = ans_candidates_len.view(-1)
batch_agg = np.reshape(
                np.tile(np.expand_dims(np.arange(batch_size), axis=1), [1, 5]), [-1])
ans_candidates_embedding = self.linguistic_input_unit(ans_candidates_agg, ans_candidates_len_agg)
ans_candidates_emb_mul = ans_candidates_embedding.view(batch_size, 5, -1).sum(1)
out = self.output_unit(ans_candidates_embedding, ans_candidates_emb_mul[batch_agg])

Only concatenating the ans_candidates_embedding and ans_candidates_emb_mul[batch_agg] can make the results strange.