image reconstruction result comparision of the pretrained ConvNeXt-V1/V2 model about convnext-v2

[s9xie]

Yes we have visualized the reconstruction results several times and found the reconstructed results very similar to those obtained by ViT MAE (or Swin SimMIM), even before adding the GRN layer (ie. using the v1 model). In other words, it seems one cannot really judge the representation quality from the pixel space reconstruction quality: even we get “perfect” reconstructions, the finetuning results can still have a large gap.

[songkq]

@s9xie Does ConvNeXt-v1/v2 model equipped without sparse conv also work well in image reconstruction?

[songkq]

Yes we have visualized the reconstruction results several times and found the reconstructed results very similar to those obtained by ViT MAE (or Swin SimMIM), even before adding the GRN layer (ie. using the v1 model). In other words, it seems one cannot really judge the representation quality from the pixel space reconstruction quality: even we get “perfect” reconstructions, the finetuning results can still have a large gap.

Since the gap between pretraining and finetuning in the self-supervised paradigm, could we introcude the MAE-based ConvNeXt Encoder in a Semi-supvised framework such as FixMatch?

[songkq]

@s9xie @shwoo93 Did you make a comparison among the supervised ConvNeXt-v2 models across different model sizes?
As model size shrinks, does the superiority of MAE-pretrained vanish compared with the supervised ones?