The code for our paper EANet: Enhancing Alignment for Cross-Domain Person Re-identification. A Chinese version of introduction for this paper can be found here.
This repository provides
- Almost all experiments, with trained models
- Used data
- Easy-to-extend framework
- Python 2.7 (Recommend to use Anaconda for easy package management.)
- Pytorch 1.0.0
- Torchvision 0.2.1
The other packages and versions are listed in requirements.txt. You can install them by pip install -r requirements.txt.
Prepare datasets to have following structure:
${project_dir}/dataset
market1501
Market-1501-v15.09.15 # Extracted from Market-1501-v15.09.15.zip, http://www.liangzheng.org/Project/project_reid.html
Market-1501-v15.09.15_ps_label
bounding_box_train_duke_style
im_path_to_kpt.pkl
cuhk03_np_detected_jpg
cuhk03-np # Extracted from cuhk03-np.zip, https://pan.baidu.com/s/1RNvebTccjmmj1ig-LVjw7A
cuhk03-np-jpg_ps_label
im_path_to_kpt.pkl
duke
DukeMTMC-reID # Extracted from DukeMTMC-reID.zip, https://github.com/layumi/DukeMTMC-reID_evaluation
DukeMTMC-reID_ps_label
bounding_box_train_market1501_style
im_path_to_kpt.pkl
msmt17
MSMT17_V1 # Extracted from MSMT17_V1.tar.gz, https://www.pkuvmc.com/publications/msmt17.html
MSMT17_V1_ps_label
im_path_to_kpt.pkl
partial_reid
Partial-REID_Dataset # Extracted from Partial-REID_Dataset.rar, http://isee.sysu.edu.cn/files/resource/Partial-REID_Dataset.rar
partial_ilids
Partial_iLIDS # Provided by https://github.com/lingxiao-he/Partial-Person-ReID
coco
images
masks_7_parts
im_name_to_kpt.pkl
im_name_to_h_w.pkl
Keypoints and part segmentation labels: Baidu Cloud or Google Drive.
The following example shows the keypoint format.
from __future__ import print_function
import cPickle
res = cPickle.load(open('dataset/market1501/im_path_to_kpt.pkl'))
# 'Market-1501-v15.09.15/bounding_box_train/0742_c1s4_014906_01.jpg' ...
print(res.keys()[:5])
# ['kpt', 'im_h_w']
print(res.values()[0].keys())
# A numpy array with shape [17, 3], for 17 keypoints. Each row is (x, y, confidence); we treat those with confidence > 0.1 as visible.
print(res.values()[0]['kpt'])
# [h, w] of the image
print(res.values()[0]['im_h_w'])For COCO, there is a bit difference.
from __future__ import print_function
import cPickle
res = cPickle.load(open('dataset/coco/im_name_to_kpt.pkl'))
# Currently only contain train set.
# 'COCO_train2014_000000426663_185693.jpg' ...
print(res.keys()[:5])
# A numpy array with shape [17, 3], each row is (x, y, visibility), visibility is one of [0, 1, 2], refer to COCO dataset for detail
print(res.values()[0])
# image size
res = cPickle.load(open('dataset/coco/im_name_to_h_w.pkl'))
print(res.keys()[0])
print(res.values()[0])Part segmentation label for each image is a single-channel PNG file, with same resolution as the image. Label mapping is as follows
{
'background': 0,
'head': 1,
'torso': 2,
'upper_arm': 3,
'lower_arm': 4,
'upper_leg': 5,
'lower_leg': 6,
'foot': 7,
}
-
My trained models: Baidu Cloud or Google Drive.
-
Train model
GlobalPoolonmarket1501cd ${project_dir} CUDA_VISIBLE_DEVICES=0 python -m package.optim.eanet_trainer --exp_dir exp/eanet/GlobalPool/market1501 --cfg_file package/config/default.py --ow_file paper_configs/GlobalPool.txt --ow_str "cfg.dataset.train.name = 'market1501'"
-
Test the
GlobalPoolmodel that was trained onmarket1501. Make sure directoryexp_direxists and ackpt.pthis inside it.cd ${project_dir} CUDA_VISIBLE_DEVICES=0 python -m package.optim.eanet_trainer --exp_dir exp/eanet/GlobalPool/market1501 --cfg_file package/config/default.py --ow_file paper_configs/GlobalPool.txt --ow_str "cfg.dataset.train.name = 'market1501'; cfg.only_test = True"
-
(Almost) All experiments of the paper is in
script/exp/train_all.sh. Look at it for details. -
To test (almost) all models of the paper. Download and place the trained models in the following structure
${project_dir}/exp/test_paper_models ├── GlobalPool │ ├── cuhk03_np_detected_jpg │ │ └── ckpt.pth │ ├── duke │ │ └── ckpt.pth │ └── market1501 │ └── ckpt.pth ├── PAP │ ├── cuhk03_np_detected_jpg │ │ └── ckpt.pth │ ├── duke │ │ └── ckpt.pth │ └── market1501 │ └── ckpt.pth ├── PAP_6P │ ├── cuhk03_np_detected_jpg │ │ └── ckpt.pth │ ├── duke │ │ └── ckpt.pth │ └── market1501 │ └── ckpt.pth ├── PAP_ST_PS │ ├── cuhk03_np_detected_jpg_to_duke │ │ └── ckpt.pth │ ├── cuhk03_np_detected_jpg_to_market1501 │ │ └── ckpt.pth │ ├── duke_to_cuhk03_np_detected_jpg │ │ └── ckpt.pth │ ├── duke_to_market1501 │ │ └── ckpt.pth │ ├── market1501_to_cuhk03_np_detected_jpg │ │ └── ckpt.pth │ └── market1501_to_duke │ └── ckpt.pth ├── PAP_ST_PS_SPGAN │ ├── duke_to_market1501 │ │ └── ckpt.pth │ └── market1501_to_duke │ └── ckpt.pth ├── PAP_ST_PS_SPGAN_CFT │ ├── duke_to_market1501 │ │ └── ckpt.pth │ └── market1501_to_duke │ └── ckpt.pth ├── PAP_S_PS │ ├── cuhk03_np_detected_jpg │ │ └── ckpt.pth │ ├── duke │ │ └── ckpt.pth │ └── market1501 │ └── ckpt.pth ├── PAP_StC_PS │ ├── cuhk03_np_detected_jpg │ │ └── ckpt.pth │ ├── duke │ │ └── ckpt.pth │ └── market1501 │ └── ckpt.pth └── PCB ├── cuhk03_np_detected_jpg │ └── ckpt.pth ├── duke │ └── ckpt.pth └── market1501 └── ckpt.pthThen, run
cd ${project_dir} bash script/exp/test_all.sh
You should get the following testing scores (Refer to the paper for table format). Note: The original models for
PAP-ST-PSandPAP_ST_PS_SPGAN_CFTare missing, so we train the models again and their scores are not identical to the paper's.M->M C->C D->D M->C M->D C->M C->D D->M D->C GlobalPool 88.2 (71.3) 42.4 (39.6) 79.2 (61.9) 10.7 ( 9.3) 38.7 (21.5) 45.7 (21.8) 32.5 (15.7) 47.9 (21.6) 9.1 ( 7.7) PCB 93.2 (81.1) 65.2 (60.0) 86.3 (72.7) 8.9 ( 7.8) 42.9 (23.8) 52.1 (26.5) 29.2 (15.2) 56.5 (27.7) 8.4 ( 6.9) PAP-6P 94.4 (84.2) 68.1 (62.4) 85.6 (72.4) 11.6 ( 9.9) 47.6 (28.3) 54.6 (29.3) 33.9 (18.1) 59.7 (31.4) 9.2 ( 8.2) PAP 94.4 (84.5) 72.0 (66.2) 86.1 (73.3) 11.4 ( 9.9) 46.4 (27.9) 55.5 (30.0) 34.0 (17.9) 59.5 (30.6) 9.7 ( 8.0) PAP-S-PS 94.6 (85.6) 72.5 (66.7) 87.5 (74.6) 14.2 (12.8) 51.4 (31.7) 59.4 (33.3) 39.3 (22.0) 61.7 (32.8) 11.4 ( 9.6) PAP-StC-PS 94.7 (84.9) 70.1 (64.4) 87.0 (73.4) 19.1 (16.4) 56.3 (35.1) 65.5 (38.6) 45.2 (26.1) 65.2 (35.7) 12.2 (10.5) PAP-ST-PS - - - 22.4 (19.5) 55.3 (35.7) 67.8 (40.7) 43.9 (25.9) 66.5 (36.6) 15.1 (13.4) PAP-ST-PS-SPGAN - - - - 61.4 (39.4) - - 69.6 (39.3) - PAP-ST-PS-SPGAN-CFT - - - - 67.0 (47.0) - - 76.4 (50.3) -
The training logic is defined in package/optim/eanet_trainer.py and package/optim/cft_trainer.py, the latter for Clustering and Finetuning. A training command looks like
cd ${project_dir}
CUDA_VISIBLE_DEVICES=0 python -m package.optim.${trainer} --exp_dir ${exp_dir} --cfg_file ${cfg_file} [--ow_file ${ow_file}] [--ow_str ${ow_str}]${trainer}is one ofeanet_trainerorcft_trainer.${exp_dir}is the directory for experiment output.${cfg_file}is a file defining configurations. Look atpackage/config/default.pyfor an example.- [Optional]
${ow_file}is a text file defining configurations to overwrite your${cfg_file}. - [Optional]
${ow_str}is a string defining configurations to overwrite your${cfg_file}. ${ow_file}is mainly for storing the configurations used in the paper. Look atpaper_configs/*.txtfor details.${ow_str}is a handy way to modify some settings in command line without manually modify the config file. This is convenient, e.g. when running the same model on many datasets.- The code will copy
${cfg_file}into${exp_dir}, and then overwrite the items specified in${ow_file}(if provided) and then those in${ow_str}(if provided).
Test sets and testing interval can be set in config file, and the training script will test the model during training. If you want to test a trained model, create a exp_dir and place the ckpt.pth inside it, then set cfg.only_test = True in ${config_file} and run package/optim/eanet_trainer.py. In this case, the code only performs testing.
Current datasets
- Market1501
- CUHK03-NP detected Subset
- DukeMTMC-reID
- MSMT17
- Partial-REID
- Partial-iLIDs
You can create new dataset class in package/data/datasets/ and then register it in package/data/create_dataset.py.
Current backbone is ResNet. You can implement new backbones and then register them in package/model/backbone.py. Or you can re-define the whole model package/model/model.py, as long as you implement the methods declared in package/model/base_model.py.
Note: If you re-implement the whole model, you may have to modify the model.forward() logic in following files as well
package/optim/eanet_trainer.pypackage/optim/cft_trainer.pypackage/eval/extract_feat.py
This model calling can be abstracted away in the future.
package/optim/reid_trainer.py covers the common logic for ReID training, with some abstraction to be implemented by sub classes. package/optim/eanet_trainer.py and package/optim/cft_trainer.py are concrete implementations, which also demonstrate the usage of hybrid batches and multi losses, etc.
We heavily use dict for passing data / configuration arguments when calling functions. This simplifies function headers and reduces the amount of code to modify when changing experiment settings.
- More datasets
- More backbones
- Run TripletLoss
- Run warmup
- Test TensorBoard
- Implement
forward_type == 'ps_reid_serial' - Write commit num to log
- Debug why it prints
at the beginning of every epoch?
Loaded pickle file dataset/market1501/im_path_to_kpt.pkl Loaded pickle file dataset/market1501/im_path_to_kpt.pkl
- Tricky!
EasyDict.__setattr__will transform tuple into list! So don't rely on it to store tuples! You have to transform them into tuples wherever tuple is needed. - If you meet error
ImportError: /lib64/libstdc++.so.6: version CXXABI_1.3.9 not found (required by ${Anaconda}/lib/python2.7/site-packages/scipy/sparse/_sparsetools.so), tryconda install libgcc; export LD_LIBRARY_PATH=${YOUR_ANACONDA_HOME}/lib:${LD_LIBRARY_PATH} - The CUHK03 dataset provides image data in
.matformat. open-reid transforms it to JPG images, while CUHK03-NP provides PNG images. Throughout the paper, we use JPG version of CUHK03-NP, due to some historical reasons. After the paper, we find that CUHK03-NP PNG has better performance than JPG on the GlobalPool baseline; We did not perform further experiments on other model architectures, due to time limitation.
If you find our work useful, please kindly cite our paper:
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent