Simple question on video classification of self-supervised learning and full-supervision methods about coclr

[TengdaHan]

90.6 means [self-supervised pretrain on K400 (no label)] -> [finetune on UCF101]
96.8 means [supervised pretrain on K400 (use label)] -> [finetune on UCF101]
The meaningful random init baseline is 77.0 in our Table 1, meaning [pretrain on nothing] -> [finetune on UCF101].

[June01]

Hi, thanks for the answer. I checked [1] again, and found out in fully-supervised learning, I3D pretrained on nothing and achieve 88.8% (Table 4) with rgb only. I am wondering what makes such a big gap between 77.0 and 88.8?

[1] Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset

[TengdaHan]

1. UCF101 top1 88.8% in "Quo Vadis, Action Recognition" Table 4 is two-stream result. Two-stream is known to be better than single stream.
2. Video resolution, they use 224x224, we use 128x128. Larger resolution is known to be better than low resolution on classification tasks.
3. Also backbone is different, they use I3D (inflated Inception network), we use S3D. But I think 1&2 is the main reason for this baseline accuracy gap.

[June01]

I see! Appreciate your help! I am wondering is there any ImageNet pretrained-weights provided by S3D? As many paper claimed using it.

[TengdaHan]

S3D is a 3D CNN, usually being used for the video input.
I don't think people train S3D on ImageNet. If there is paper claiming using S3D with ImageNet weights, I guess that means "ImageNet inflated" weights, where they take the I3D network, expand convolution kernels by coping multiple times along the time dimension.

[June01]

Agree! Thanks very much!:)