xinshaoamoswang / osm_caa_weightedcontrastiveloss

Deep Metric Learning by Online Soft Mining and Class-Aware Attention, AAAI 2019 Oral

Deep Metric Learning by Online Soft Mining and Class-Aware Attention, AAAI 2019 Oral, ML Technical Track

[Paper] [Slides] [Poster]

Code is under legal check...

Citation

If you find our code and paper make your research or work a little bit easier, it would be our great pleasure. If that is the case, please kindly cite our paper. Thanks.

@inproceedings{wang2019deep,
  title={Deep metric learning by online soft mining and class-aware attention},
  author={Wang, Xinshao and Hua, Yang and Kodirov, Elyor and Hu, Guosheng and Robertson, Neil M},
  booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},
  volume={33},
  pages={5361--5368},
  year={2019}
}

Dependencies & Setup

The core functions are implemented in the caffe framework. We use matlab interfaces matcaffe for data preparation.

• Clone our repository:

  git clone [email protected]:XinshaoAmosWang/OSM_CAA_WeightedContrastiveLoss.git

• Install dependencies on Ubuntu 16.04

  sudo apt-get install libprotobuf-dev libleveldb-dev libsnappy-dev libopencv-dev libhdf5-serial-dev protobuf-compiler
  sudo apt-get install --no-install-recommends libboost-all-dev
  sudo apt-get install libopenblas-dev
  sudo apt-get install python-dev
  sudo apt-get install libgflags-dev libgoogle-glog-dev liblmdb-dev

• Install MATLAB 2017b

  Download and Run the install binary file

  ./install

• Compile Caffe and matlab interface

  • Note you may need to change some paths in Makefile.config according your system environment and MATLAB path;
  • To exactly reproduce our results, please do not use cudnn. (It is commented in the Makefile.config file.)

  cd OSM_CAA_WeightedContrastiveLoss/CaffeMex_V28
  make -j8  && make matcaffe

Usage

• Download the corresponding data files of each dataset.

• Unzip and Copy to their corresponding training folders.

rsync -a -v Data_OSM_CAA_WeightedContrastiveLoss/*V01 .
cp Data_OSM_CAA_WeightedContrastiveLoss/googlenet_bn.caffemodel ./CARS196_V01/pretrain_model/
cp Data_OSM_CAA_WeightedContrastiveLoss/googlenet_bn.caffemodel ./CUB_V01/pretrain_model/
cp Data_OSM_CAA_WeightedContrastiveLoss/googlenet_bn.caffemodel ./MARS_V01/pretrain_model/
cp Data_OSM_CAA_WeightedContrastiveLoss/googlenet_bn.caffemodel ./LPW_V01/pretrain_model/

• LPW data preparation
• MARS data preparation
• googlenet_bn.caffemodel

Examples for reproducing our results on CARS196, CUB-200-2011, MARS, LPW are given.

• Data preparation for CARS196, CUB-200-2011

  please refer to Ranked List Loss, the pipeline is similar.

• Data preparation for MARS or LPW:

  please refer to Ranked List Loss, the pipeline is similar.

• Custom data preparation

  please refer to Ranked List Loss, the pipeline is similar.

• Train & Test

  Run the training and testing scripts in the training folder of a specific setting defined by its corresponding prototxt folder.

  Examples:

  • CARS196: cd CARS196_V01/train_M_WIDE_ASiamese_V62_e1
    • Train:

    matlab -nodisplay -nosplash -nodesktop -r "run('train.m');exit;" | tail -n +11

    • Test:

    matlab -nodisplay -nosplash -nodesktop -r "run('test_model.m');exit;" | tail -n +11

  • CUB-200-2011: cd CUB_V01/train_M_WIDE_ASiamese_V42_e1
    • Train:

    matlab -nodisplay -nosplash -nodesktop -r "run('train.m');exit;" | tail -n +11

    • Test:

    matlab -nodisplay -nosplash -nodesktop -r "run('test_model.m');exit;" | tail -n +11

Acknowledgements

Our work benefits from:

• Hyun Oh Song, Yu Xiang, Stefanie Jegelka and Silvio Savarese. Deep Metric Learning via Lifted Structured Feature Embedding. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016. http://cvgl.stanford.edu/projects/lifted_struct/

• CaffeMex_v2 library: https://github.com/sciencefans/CaffeMex_v2/tree/9bab8d2aaa2dbc448fd7123c98d225c680b066e4

• Caffe library: https://caffe.berkeleyvision.org/

Licence

BSD 3-Clause "New" or "Revised" License

Affiliations:

• Queen's University Belfast, UK
• Anyvision Research Team, UK

Contact

Xinshao Wang (You can call me Amos as well) xwang39 at qub.ac.uk

Relevant Work

ID-aware Quality for Set-based Person Re-identification: Without weighted contrastive loss.

@article{wang2019id,
  title={ID-aware Quality for Set-based Person Re-identification},
  author={Wang, Xinshao and Kodirov, Elyor and Hua, Yang and Robertson, Neil M},
  journal={arXiv preprint arXiv:1911.09143},
  year={2019}
}

osm_caa_weightedcontrastiveloss's Issues

code

方便开源一下代码嘛？想复现一下

Instance Cross Entropy for Deep Metric Learning and its application in SimCLR-A Simple Framework for Contrastive Learning of Visual Representations

https://xinshaoamoswang.github.io/paperlists/2020-02-16-arXiv/#foundation-of-deep-learning

Regarding its flexibility and data download link

1. The code is not very flexible now since some variables are inside the function instead of being input parameters.
   I will find some time to make it more flexible later. Or someone else who is interested in our work can help me on this. Cheers!

2. Currently the data is stored in google drive for downloading. https://drive.google.com/drive/folders/1PHBiqQ8-t4DxgMgERX_eDA8LP6gayW6T?usp=sharing
   If someone else finds a better way to share the data, please kindly let me know. Thanks.