deepalchemist / deep-person-search

deep learning person search in PyTorch.

A PyTorch based strong baseline for person search

This repository develops a baseline model with high performance, and implements the widely used baseline OIM [1], and NAE [4].

• Separating detection and re-ID head on the top of model
• PK sampling for training re-ID head
• Warm-up training

About this repository

• It is pure PyTorch code, which requires the PyTorch version >= 1.1.0
• It supports multi-image batch training.
• End-to-end training and evaluation. Both PRW and CUHK-SYSU are supported.
• Standard protocol (including PRW-mini in [3]) used by most research papers
• Highly extensible (easy to add models, datasets, training methods, etc.)
• Visualization tools (proposals, losses in training)
• High performance baseline.

Preparation

• Clone repository and build the environment

git clone https://github.com/DeepAlchemist/deep-person-search.git && cd deep-person-search
conda env create -f env.yml

• Download PRW and CUHK-SYSU (also named SSM) to path_to_your_data.

• In the config file ./lib/cfg/config.py, change --data_root to path_to_your_data, --ckpt_dir to path_you_want_to_save_the_checkpoints.

• Download ImageNet pre-trained ResNet models from GoogleDrive to deep-person-search/cache/pretrained_model/

Training & Evaluation

• Training

CUDA_VISIBLE_DEVICES=0 python main.py \
    --benchmark ssm --batch_size 5 \
    --backbone bsl --in_level C5 --cls_type oim \
    --lr 0.003 --warmup_epochs 1 --max_epoch 7 \
    --suffix "" 

-dis enable display (visualization), then tensorboard --bind_all --logdir your_log_dir, which shows the loss curves and the input image with proposals.

• Evaluation

CUDA_VISIBLE_DEVICES=0 python main.py --is_test \
    --benchmark ssm --eval_batch_size 5 \
    --backbone bsl --in_level C5 --cls_type oim \
    --load_ckpt "absolute_path_to_your_checkpoint" \

• Comparison

		CUHK-SYSU		PRW
Method	mAP	rank1	mAP	rank1
OIM [1]	88.1	89.2	36.0	76.7
NAE [4]	89.8	90.7	37.9	77.3
baseline	90.0	91.0	40.5	81.3

The download link of the trained models are available in the table. Note that all the models are trained with image size of 600x1000, the larger image size, e.g., 900x1500, would yield better performance.

TODO

• DistributedDataParallel
• Training with larger image size, i.e., 900x1500
• Supporting more SOTA methods
• Visualizing ranking list in test
• A technological report for this repository

[1] Joint Detection and Identification Feature Learning for Person Search. In CVPR 2017.
[2] Person Re-Identification in the Wild. In CVPR 2017.
[3] Query-guided End-to-End Person Search. In CVPR 2019.
[4] Norm-Aware Embedding for Efficient Person Search. In CVPR 2020.

Citation

@article{yang2021bottom,
  title={Bottom-up foreground-aware feature fusion for practical person search},
  author={Yang, Wenjie and Huang, Houjing and Chen, Xiaotang and Huang, Kaiqi},
  journal={IEEE Transactions on Circuits and Systems for Video Technology},
  volume={32},
  number={1},
  pages={262--274},
  year={2021},
  publisher={IEEE}
}

deep-person-search's Issues

multi gpu run

FileNotFoundError: [Errno 2] No such file or directory: './run.sh'

Traceback (most recent call last):
File "main.py", line 282, in
cfg = Config().parse()
File "/home/deep-person-search-master/lib/cfg/config.py", line 108, in parse
self.print_options(cfg)
File "/home/deep-person-search-master/lib/cfg/config.py", line 92, in print_options
shutil.copy("./run.sh", cfg.expr_dir)
File "/home/qd/anaconda3/envs/dps/lib/python3.6/shutil.py", line 245, in copy
copyfile(src, dst, follow_symlinks=follow_symlinks)
File "/home/qd/anaconda3/envs/dps/lib/python3.6/shutil.py", line 120, in copyfile
with open(src, 'rb') as fsrc:
FileNotFoundError: [Errno 2] No such file or directory: './run.sh'

I cannot find "run.sh" in your project, how to solve the problem?

Training error

When I train on the original PRW dataset, the same will appear
‘File "main.py", line 43, in train_one_epoch
_, losses = engine.optimize_parameters(cfg)
TypeError:'NoneType' object is not iterable
’
Why is that?
Looking forward to your reply

Out of cpu memory

Hello! When I run the code, the CPU memory will keep increasing, and finally the memory will be exceeded, and the code will report an error. I tried to change the value of num_workers, but it didn't work.Do you know why this happens? What should I do?

Training problem

Excuse me, in this code, the unrecognizable id is set to -1, and then these pictures are deleted during training. I found that during training, all the remaining frames (training and testing frames) are trained instead of The remaining frames in the training set. Why is that?

请问您的代码支持前面提到的NAE、OIM吗？

因为数据集还在下载，还没测试

Why freeze the weights of conv1?

As the codes in lib/detector/getdet.py ln15, why do you set the conv1 froze?

# Fix blocks
for p in self.conv1.parameters(): p.requires_grad = False

Thanks for your explanation.

About the lower performance of NAE on PRW

Is there any idea on the lower performance of NAE on PRW compared with the performance reported in original paper?

Amazing result

Hi, I tested your pretrained model, and I got some amazing result.My test script was python main.py --is_test --benchmark prw --eval_batch_size 5 --backbone bsl --in_level C5 --cls_type oim --load_ckpt /workplace/.torch/checkpoints/prw-bsl.pth. And I got this.

Rank-1 is the top-1 in the picture? thanks.

Evaluation: 0.0% result

Hi,
during evaluation with default values I obtain the following result:

Experiment dir: /home/roberto/PersonReID/deep-person-search/20201004/test
Loaded dataset ssm_test for testing
6978 roidb entries
Backbone loading ImageNet pretrained weights from
./cache/pretrained_model/resnet50_caffe.pth
Engine [baseline] was created
Not loading checkpoint!
#ModelParams : 35.05 M
extracting probe features ...
im_exfeat: 2900/2900 0.008s
extracting gallery features ...
total time:1.297s, reid time:0.002s
evaluating detections
all detection:
recall = 0.00%
ap = 0.00%
/home/roberto/anaconda3/envs/dps/lib/python3.6/site-packages/sklearn/metrics/ranking.py:528: RuntimeWarning: invalid value encountered in true_divide
recall = tps / tps[-1]
labeled_only detection:
recall = 0.00%
ap = nan%
search ranking:
recall = 0.00%
mAP = 0.00%
top- 1 = 0.00%
top- 5 = 0.00%
top-10 = 0.00%
test time: 36.9min

The command used is:

CUDA_VISIBLE_DEVICES=0 python main.py --is_test --benchmark ssm --eval_batch_size 5 --backbone bsl --in_level C5 --cls_type oim

It is quite a strange result. What can be wrong?
Thanks

你好，请问模型的速度怎样？

The highest result of the benchmark network I trained was 90.83, how to solve?

GPU:1080ti
CPU:64 core
pytorch 1.4.0
dataset: CUHK-SYSU

• #DetParams: 16.73M #reIDParams: 18.32M
  loading checkpoint ./checkpoints/20210409/ssmbsloimb5dowu/latest.pth

evaluating detections
all detection:
recall = 88.83%
ap = 84.76%
labeled_only detection:
recall = 99.20%
ap = 37.35%
search ranking:
recall = 99.18%
mAP = 90.06%
top- 1 = 90.83%
top- 5 = 96.62%
top-10 = 97.66%
Finished. Total elapsed time (h:m:s): 4:47:02
Experiment name: ssmbsloimb5dowu

The highest result of the benchmark network I trained was 90.83. When I train Baseline many times, most results are 89.* how to solve?

trainning problem with display

Hello! when I train the model with the option '-dis' ,the code will report an error

I have found a lot of solutions online but they didn't work.Do you know why this happens?
thanks!

detections is None, when training by 'oim' backbone. OIM.py #152

OIM.py #152 detections, detector_losses = self.roi_heads(features, proposals, images.image_sizes, targets)

It seems that you forgot to return result when self.training=True
OIM.py #215

`
result, losses = [], {}
if self.training:
loss_detection, loss_box_reg = fastrcnn_loss(class_logits, box_regression,labels, regression_targets)

        loss_reid, acc = self.reid_loss(embeddings_, pids)

        losses = dict(loss_classifier=loss_detection,
                      loss_box_reg=loss_box_reg,
                      loss_ide=loss_reid)
    else:
        boxes, scores, embeddings, labels = \
            self._postprocess_detections(class_logits, box_regression, embeddings_,
                                         proposals, image_shapes)
        num_images = len(boxes)
        for i in range(num_images):
            result.append(
                dict(
                    boxes=boxes[i],
                    labels=labels[i],
                    scores=scores[i],
                    feats=embeddings[i],
                )
            )
    return result, losses

`

The difference between baseline and OIM

Hello, I would like to ask what is the main difference between the baseline you implemented and OIM and NAE?