a lot of questions about how to reproduce and cite the experimental results about uno

[DonkeyShot21]

Hi! Thanks for the interest in our work.

An easy solution is to checkout the commit before the addition of UNOv2, run one of the commands in the readme and print the args and the transforms. Then you can reuse those hyperparam for UNOv2, if that is what you would like to do.

However, an even easier solution would be to just mention boh UNO v1 and v2. For UNOv1 you can use the result in the paper and compare to RS. For UNOv2 you can use the results on the readme and compare with RS+ (uses 400 epochs). The batch size does not impact UNO too much so I would not worry about that. The best thing in my opinion is laying down the results in your paper and let the reviewer decide.

Regarding augmentations, multicrop does not use that much more compute because crops are resized to a lower resolution. Moreover it is reasonable that different methods work better with different augmentations. For instance, I found UNO to be extremely robust and tolerant to different augmentations, while RS decreases performance with stronger augmentations. I think it is fair to compare with the best set of augmentations for each method. This is what people have been doing in the SSL literature: SimCLR, BYOL, SwAV and DINO all have different augmentations that fit the method.

I hope that helps.

[vicmax]

Hi @DonkeyShot21

Thank you for your reply! It helps!

And I did have some observations by running your code.

The setting of the four groups listed in the following are (from top to bottom):

group index of wandb "Name"	different settings	task agnostic/test/acc
1-CIFAR100-UNO-discover-resnet18-80_20_bs256_smallcrop	batch_size=256, num_small_crops=2	0.6661$\pm$0.03
2-CIFAR100-UNO-discover-resnet18-80_20_bs256_woSmallcrop	batch_size=256, num_small_crops=0	0.7321$\pm$0.02
3-CIFAR100-UNO-discover-resnet18-80_20_bs512_woSmallcrop	batch_size=512, num_small_crops=0	0.7648$\pm$0.009
4-CIFAR100-discover-resnet18-80_20	batch_size=512, num_small_crops=2	0.7803$\pm$0.006

Other arguments are kept as same (e.g., max_epoch=500, base_lr=0.4 (maybe should be set smaller for smaller batch size), num_large_crops=2)

We can observe that smaller batch size will lead to worse performance (and larger perturbations)...

[DonkeyShot21]

Of course when you modify the batch size you need to retune the hyperparams, at least the learning rate! It is not normal that your accuracy on the unlab set is going down.

[vicmax]

Hi @DonkeyShot21

Thank you. I will do more experiments.

And I just wonder why you adjusted the learning rate from 0.1 to 0.4 in UNO v2. Will it bring benefits? Or is that just because longer training epochs can bear a larger learning rate as we have longer steps under small learning rates during the process of annealing?

[DonkeyShot21]

I think I tuned it but I don't remember exactly. Also remember that for very small batch sizes you might need a queue (take a look at SwAV's code)

[vicmax]

Thank you very much for your detailed replies! I don't have further questions so far.

Have a nice day!