Request for configs and question about the MaxBoxAcc about wsolevaluation

[junsukchoe]

Thank you for your interest in our work!

Regarding the configurations, we will release all hyperparameter settings for Table 1 and 6 within a month. Sorry for the inconvenience.

Regarding the evaluation metric, we are aware that most existing works primarily evaluate their methods with the Top-1 Loc metric. However, we believe that Top-1 Loc is insufficient to make claims about improved localization. A technique can potentially see an improvement in Top-1 Loc just by having significantly better classification (even with worse localization!). We believe that any localization claims would have to be validated by more explicit localization-centric metrics. In this regard, PxAP with mask annotations is the ideal metric for evaluating WSOL. However, in many cases, only the box annotations are available. In this case, we suggest using MaxBoxAcc. Please see Sec. 3 and 4 for more detail.

However, as you mentioned, Top-1 Loc is a widely used metric in WSOL literature, so we reported them in Appendix Table 6. We will also include classification accuracies in the revised version.

[won-bae]

Appreciate for your reply.

Would it be possible to release the configs at least for MaxBoxAcc for CAM on CUB and Imagenet as well as Top-1 Loc for ACoL on CUB and Imagenet? It would be highly appreciated!

Also, for CUBV2, I realized 059. California_Gull only has 3 images. Does this mean the maximum value for num_val_sample_per_class is 3 for CUB?

Thank you

[junsukchoe]

Some classes include less than five images. Currently, our implementation does not support 4 or 5 for CUB dataset. Instead, you can use all images in train_fullsup by setting num_val_sample_per_class to 0.

Here are the configs:

CAM on CUB (epochs 50, lr_decay_frequency 15)
vgg: lr (0.00003562485), large_feature_map (False)
resnet: lr (0.00025028018), large_feature_map (True)
inceptionv3: lr (0.00051145010), large_feature_map (True)

CAM on ImageNet (epochs 10, lr_decay_frequency 3)
vgg: lr (0.00010902234), large_feature_map (False)
resnet: lr (0.00015072583), large_feature_map (True)
inceptionv3: lr (0.00012730641), large_feature_map (True)

ACoL on CUB (epochs 50, lr_decay_frequency 15)
resnet: lr (0.00015396656), large_feature_map (True), acol_threshold (0.64)
vgg: lr (0.00000349871), large_feature_map (True), acol_threshold (0.76)
inceptionv3: lr (0.00721313070), large_feature_map (True), acol_threshold (0.79)

ACoL on ImageNet (epochs 10, lr_decay_frequency 3)
resnet: lr (0.00035247105), large_feature_map (True), acol_threshold (0.81)
vgg: lr (0.00047688934), large_feature_map (True), acol_threshold (0.95)
inceptionv3: lr (0.00046360434), large_feature_map (True), acol_threshold (0.81)

[won-bae]

I see. Thank you for the clarification and configs!