Linear probing on vision tasks about multimodal

[ankitade]

Hi, for image net linear probing the encoded_image is used (not projected output)
https://github.com/facebookresearch/multimodal/blob/main/examples/flava/callbacks/multimodal_eval.py#L66

[KMnP]

Thanks for the quick response!

I wonder if you use any specific augmentations for the linear probing. Maybe I missed something, I couldn't find the details in the paper.

[KMnP]

Hi @ankitade, I wonder for the output dict of encoded_image, do you use pooler_output or the cls_token embedding from last_hidden_state.

Thank you!

[ankitade]

Hi @KMnP, it is the cls token embedding

[ankitade]

Hope this answered your questions. Please open another issue if you have more questions

[KMnP]

Hi @ankitade , yes this answered my questions. Thank you so much!