xishen0220 / ssr

(NeurIPS 2021) Pytorch implementation of paper "Re-ranking for image retrieval and transductive few-shot classification"

SSR

(NeurIPS 2021) Pytorch implementation of paper "Re-ranking for image retrieval and transductivefew-shot classification"

[Paper] [Project webpage] [Video] [Slide]

The project is an extension work to SIB. If our project is helpful for your research, please consider citing :

@inproceedings{shen2021reranking,
  title={Re-ranking for image retrieval and transductive few-shot classification},
  author={Shen, Xi and Xiao, Yang and Hu, Shell Xu, and Sbai, Othman and Aubry, Mathieu},
  booktitle={Conference on Neural Information Processing Systems (NeurIPS)},
  year={2021}
}

Table of Content

• 1. Installation
• 2. Method
  • 2.1 Image retrieval
    • 2.1.1 SSR module
    • 2.1.2 Results
  • 2.2 Transductive few-shot classification
    • 2.2.1 SSR module
    • 2.2.2 Results
• 3. Acknowledgement
• 4. ChangeLog
• 5. License

1. Installation

Code is tested under Pytorch > 1.0 + Python 3.6 environment.

Please refer to image retrieval and transductive few-shot classification to download datasets.

2. Methods and Results

SSR learns updates for a similarity graph.

It decomposes the N * N similarity graph into N subgraphs where rows and columns of the matrix are ordered depending on similarities to the subgraph reference image.

The output of SSR is an improved similarity matrix.

2.1 Image retrieval

2.1.1 SSR module

Rows : the subgraph reference image (red) and the query image (green);

Columns : top retrieved images of the query image (green). These images are ordered according to the reference image (red).

2.1.2 Results

To reproduce the results on image retrieval datasets (rOxford5k, rParis6k), please refer to Image Retrieval

2.2 Transductive few-shot classification

2.2.1 SSR module

We illustrate our idea with an 1-shot-2way example:

Rows: the subgraph reference image (red) and the support set S;

Columns: the support set S and the query set Q. Both S and Q are ordered according to the reference image (red).

2.2.2 Results

To reproduce the results on few-shot datasets (CIFAR-FS, Mini-ImageNet, TieredImageNet), please refer to transductive few-shot classification

3. Acknowledgement

• The implementation of k-reciprocal is adapted from its public code

• The implementation of few-shot training, evaluation and synthetic gradient is adapted from SIB

4. ChangeLog

• 21/10/29, model, evaluation + training released

5. License

This code is distributed under an MIT LICENSE.

Note that our code depends on Pytorch, and uses datasets which each have their own respective licenses that must also be followed.

ssr's Issues

How to reproduce the work with CUB dataset?

How to reproduce the work with CUB dataset?

No improvements of mAP during training

I was training the model with my dataset. As the logs say, there were no improvements of mAP during training. Why was that? Or was it because I didn't have the test module?

Some questions about re_ranking_list function

What are q_g_dist, q_q_dist, g_g_dist? And can you please explain how the function re_ranking_list works?

Asking about the dataset

What do you mean by "database", "query", and "gt"?

    ### Oxford
    oxford_db = info_oxford['database'].T
    oxford_query = info_oxford['query'].T
    oxford_gt = info_oxford['gt']

eval_landmark.compute_map_and_print and related problems?

I am trying to reproduce your work by training with another dataset. However, my dataset doesn't have ground-truth file such as good, ok, bad, junk. How should I train the model without using the eval_landmark.compute_map_and_print function?

.

.

Given an embedding list, how to perform ssr on it?

Given an embedding list, how to perform SSR on it? An embedding list includes a list of images that are encoded into embeddings using a neural network.