hnanhtuan / selectiveconvfeature

Selective Deep Convolutional Features for Image Retrieval

This is the Matlab implements of our methods accepted in ACM MM 2017. [pdf]

This code implements

1. The framework of produce global image representation

• Pre-processing: PCA + l2-normalize
• Masking scheme: MAX-mask/SUM-mask
• Embedding: Triangular Embedding (Temb) [1] and Fast Function Approximation Embedding (F-FAemb) [2]
• Aggregating: Democractic pooling [1]
• Post-processing: rotation and whitening, Power-law normalization.

2. Extract conv. feature of images

[1] Hervé Jégou and Andrew Zisserman. 2014. Triangulation embedding and democratic aggregation for image search. In CVPR.

[2] Thanh-Toan Do and Ngai-Man Cheung. 2017. Embedding based on function approximation for large scale image search. TPAMI (2017).

BibTex

@INPROCEEDINGS{selectiveDeepConvFea,
 author    = {Tuan Hoang and Do, Thanh-Toan and Dang-Khoa Le Tan and Cheung, Ngai-Man},
 title     = {Selective Deep Convolutional Features for Image Retrieval},
 bookTitle = {ACM Multimedia},
 year      = {2017},
 month     = {Oct},
}

Prerequisites

The code include:

• The Yael library (obtained from the website https://gforge.inria.fr/frs/?group_id=2151) A copy of this library is included in 'tools' folder.

Usage

1. Modify the parameters in 'opt.m' file appropriately.
2. Run the following script

main

Files and subfolders

Filename	Description
README	This file
main.m	The main script for running whole process
opt.m	The script contains all parameter setting.
extract_feature_map/	Contains files for extracting conv. features. See the README file inside this folder for more information.
tools/
tools/make.m	Script to build the mex file for faemb and temb methods.
tools/democratic/	contains matlab script files for democratic pooling methods.
tools/faemb/	contains matlab script files for F-FAemb method.
tools/faemb_mex/	contains mex files for F-FAemb method.
tools/triemb/	contains matlab script/mex files for Triangular Embedding method.
tools/evaluation/	contains matlab script files for evaluation.
tools/yael/	contains the yael library
utils	---
utils/embedding.m	Process embedding and aggregating
utils/apply_mask.m	Compute and then apply mask on the conv. features.
utils/crop_qim.m	Crop query images.
utils/learn_rn.m	Learn projection (PCA/whitening) matrix
utils/vecpostproc.m	Apply post-processing
data/	contains ground truth and data. In case you may want to retrain, please download the provided data and put them in this folder
data/workdir/	contains output files (i.e., parameters, embedded features)

selectiveconvfeature's Issues

We need some files or data for this code

Excause me , I couldn't find the way just like
data/workdir/paris6k_31_1024_param_20_32_temb_max.mat for the param_file in the main.m . Could please give me some flies or data about this ?

strongly recommend to use Tensorflow or Pytorch as computing platform in the future　:) #3

Hi hnanhtuan, your code is written very elegant and with amazing performance. Since matconvnet is not friendly to quickly use and development community is not strong, and its training efficiency and deployment would be inferior to the above platforms. Python is more convenient to process network and data. Would you be so kind to provide codes of Tensorflow or Pytorch version?

Thanks for your code sharing again !

About train and test dataset

Could I ask you a simple question? Why you train on one dataset (paris6k dataset), then evaluate on the
another dataset (oxford5k dataset)? Is this just to avoid overfitting?