kevinlight831 / ctp Goto Github PK

Thanks for sharing great work and dataset!
I have two questions about paper.

First of all, I think that authors mainly follow the losses and architecture of ALBEF. But, CTP do not use the ITM loss as the loss function. Is there any reason? Moreover, do you have any experimental results when ITM loss is added?

Secondly, what is the main difference between "cross-modal retrieval" vs "Multi-modal retrieval" and what is the reason of the experiment about multi-modal retrieval? I guess that the latter use the [CLS] representation of multi-modal encoder and calculate the zero-shot clustering (same class). Similar to the first question, if the ITM loss is added, I wonder about the results of cross-modal retrieval and multi-modal retrieval.