sungonce / senet

Official PyTorch Implementation of Revisiting Self-Similarity: Structural Embedding for Image Retrieval, CVPR 2023

Revisiting Self-Similarity: Structural Embedding for Image Retrieval

Official Pytorch Implementation of the paper "Revisiting Self-Similarity: Structural Embedding for Image Retrieval"
accept to CVPR 2023
by Seongwon Lee, Suhyeon Lee, Hongje Seong, and Euntai Kim
Yonsei University

Introduction

➡️ Guide to Our Code

Data preparation

Download ROxford5k and RParis6k. Unzip the files and make the directory structures as follows.

revisitiop
 └ data
   └ datasets
     └ roxford5k
       └ gnd_roxford5k.pkl
       └ jpg
         └ ...
     └ rparis6k
       └ gnd_rparis6k.pkl
       └ jpg
         └ ...

Pretrained models

You can download our pretrained models from Google Drive.

Testing

For ResNet-50 model, run the command

python test.py SENET.RESNET_DEPTH 50 TEST.WEIGHTS <path-to-R50-pretrained-model> TEST.DATA_DIR <path_to_revisitop>/data/datasets

and for ResNet-101 model, run the command

python test.py SENET.RESNET_DEPTH 101 TEST.WEIGHTS <path-to-R101-pretrained-model> TEST.DATA_DIR <path_to_revisitop>/data/datasets

🙏 Acknowledgments

Our pytorch implementation is derived from Revisiting Oxford and Paris and DELG-pytorch. We thank for these great works and repos.

✏️ Citation

If you find our paper useful in your research, please cite us using the following entry:

@InProceedings{lee2023senet, 
    author    = {Lee, Seongwon and Lee, Suhyeon and Seong, Hongje and Kim, Euntai},
    title     = {Revisiting Self-Similarity: Structural Embedding for Image Retrieval},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2023},
    pages     = {23412-23421}
}

senet's Issues

About train

I feel your work is very good. Do you plan to upload training code in the future? Looking forward to your reply

AssertionError: The model is not fully loaded.

Hi! Nice work.
When I ran python test.py SENET.RESNET_DEPTH 101 TEST.WEIGHTS pretrained_weights/SENet_R101_con.pt TEST.DATA_DIR datadir_examples
There was a mismatching error between weight_dict and state_dict. That is why I wasn't able to load the checkpoint

Traceback (most recent call last):
  File "test.py", line 16, in <module>
    main()
  File "test.py", line 13, in main
    SENet_tester.__main__()
  File "/mount/sdc2_10TB/harry/tensorboard_embeddings/SENet/core/SENet_tester.py", line 30, in __main__
    checkpoint.load_checkpoint(cfg.TEST.WEIGHTS, model)
  File "/mount/sdc2_10TB/harry/tensorboard_embeddings/SENet/core/checkpoint.py", line 45, in load_checkpoint
    raise AssertionError("The model is not fully loaded.")
AssertionError: The model is not fully loaded.

@sungonce

small fix

SENet/model/resnet.py

Line 191 in 7261e52

trans_fun = "basic_transform"

class ResStage_basetransform(nn.Module):
"""Stage of ResNet."""

def __init__(self, w_in, w_out, stride, d, w_b=None, num_gs=1):
    super(ResStage_basetransform, self).__init__()      # <------------------------
    for i in range(d):
        b_stride = stride if i == 0 else 1
        b_w_in = w_in if i == 0 else w_out
        trans_fun = get_trans_fun("basic_transform")      # <------------------------
        res_block = ResBlock(b_w_in, w_out, b_stride, trans_fun, w_b, num_gs)
        self.add_module("b{}".format(i + 1), res_block)

def forward(self, x):
    for block in self.children():
        x = block(x)
    return x

Pytorch Verison

"What version of PyTorch are you using? My testing version may be different from the version of your pre-trained model, which could lead to code execution issues."

Custom dataset

What are the values of dataset and gnd_fn for a custom dataset?

        if dataset == 'roxford5k':
            gnd_fn = 'gnd_roxford5k.pkl'
        elif dataset == 'rparis6k':
            gnd_fn = 'gnd_rparis6k.pkl'
        else:
            assert dataset