Nums is the total number of pictures.What I find is that convert_lmdb_to_numpy.py only works for N*D arrary data.

I have not yet obtained the author's reply about the Shinpuhkan dataset. So I just deleted the code that handle the Shinpuhkan dataset. The modifications are as follows:

• in format_rawdatat.sh
  echo "Formatting Shinpuhkan ..."
  unzip -q -d $RAW/ $RAW/Shinpuhkan2014dataset.zip
  python2 data/format_shinpuhkan.py $RAW/Shinpuhkan2014dataset $EXP/datasets/shinpuhkan

• in make_dbs.sh
  for d in cuhk03 cuhk01 prid viper 3dpes ilids shinpuhkan; do
  echo "Making $d"
  python2 tools/make_lists_id_training.py $EXP/datasets/$d $EXP/db/$d
  make_db $EXP/datasets/$d $EXP/db/$d
  done

• in merge_dbs.sh
  python2 tools/merge_lists_single_task.py
  --dataset-dirs $DATASETS/3dpes $DATASETS/cuhk01 $DATASETS/cuhk03
  $DATASETS/ilids $DATASETS/prid $DATASETS/viper
  ~~$DATASETS/shinpuhkan ~~
  --db-dirs $DB/3dpes $DB/cuhk01 $DB/cuhk03
  $DB/ilids $DB/prid $DB/viper
  ~~$DB/shinpuhkan ~~
  -- $DB/jstl

Did I do this in a wrong way? I got bad results on every experiment. The accuracy on validation set are always 0, and the loss value almost didn't fall down.
I may get the Shinpuhkan dataset a few days later. But I just wonder if I can run the code without the Shinpuhkan dataset.

hi
I'm trying to run
scripts/exp_jstl.sh
but I got some errors in extracting features I use a system with one GPU and I couldn't use the mpirun
my errors is
E0112 14:23:14.002522 1848 extract_features.cpp:54] Using GPU
E0112 14:23:14.002907 1848 extract_features.cpp:60] Using Device_id=0
F0112 14:23:14.791924 1848 db_lmdb.hpp:13] Check failed: mdb_status == 0 (2 vs. 0) No such file or directory
*** Check failure stack trace: ***
@ 0x7f54f14b2daa (unknown)
@ 0x7f54f14b2ce4 (unknown)
@ 0x7f54f14b26e6 (unknown)
@ 0x7f54f14b5687 (unknown)
@ 0x7f54f194338e caffe::db::LMDB::Open()
@ 0x7f54f18a57c6 caffe::DataLayer<>::DataLayerSetUp()
@ 0x7f54f18713e9 caffe::BasePrefetchingDataLayer<>::LayerSetUp()
@ 0x7f54f1854a62 caffe::Net<>::Init()
@ 0x7f54f1856d91 caffe::Net<>::Net()
@ 0x40805c feature_extraction_pipeline<>()
@ 0x7f54f08cff45 (unknown)
@ 0x403bce (unknown)
@ (nil) (unknown)
scripts/routines.sh: line 101: 1848 Aborted (core dumped) ${CAFFE_DIR}/build/tools/extract_features ${trained_model} ${model} ${blob},label ${result_dir}/${subset}_features_lmdb,${result_dir}/${subset}_labels_lmdb ${num_iters} lmdb GPU 0

and also in a /dgd_person_reid/models/jstl package are some solver file for each database for example viper_solver.prototxt could I run especial command before running scripts/exp_jstl.sh ?
thank you

您好，因为我用的980显卡，显存不够大，因此把 train 的batchsize 改为15，test 15.跑完之后，单独的cuhk03 50000次accuary0.98,测试之后top1仅为63.5%。因此咨询一下，是不是batchsize改小了这个原因导致，训练还没到最优。另外您的显卡是什么型号，多大显存。

Hi, I'm trying to test and verify results on cuhk01 dateset with the given pre-trained model and euclidean distance. I need produce trainset and testset. But we know that the given pre-trained model is trained on cuhk01, so I don't know how to divide cuhk01. If you provided the file which save trained images' indexes, my problem would be solved. So, could you please provide the file external/exp/db/cuhk01/train.txt?

你好，想引用你们的文章，但是时间比较紧，无法完成跑你们的代码，能否提供一下数据集CUHK03 01 Viper 的rank1-30的CMC值呢，，谢谢

I compute the euclidean distance between two images, and the jstl_dgd_inference.caffemodel get the distance is 64.3684, then i change the caffemodel to jstl_dgd.caffemodel that getting the distance is 1.26359441957e-05, we extract the fc7 layer feature, what's problem in this process, thx.

file format_cuhk01.py missing function mkdir_if_missing(), please fix this. THX!

When I run scripts/exp_jstl.sh on my Tesla K80 （12G memory per GPU）,and cudnn v3 , errors happened like this:
///////////////////////////////////////////////////
I0516 02:05:37.825336 5661 solver.cpp:258] Learning Rate Policy: stepdecr
I0516 02:05:37.868683 5661 solver.cpp:316] Iteration 0, Testing net (#0)
I0516 02:05:38.794275 5662 cudnn_conv_layer.cpp:179] Optimized cudnn conv
I0516 02:05:39.005478 5661 cudnn_conv_layer.cpp:179] Optimized cudnn conv
I0516 02:08:07.847208 5662 solver.cpp:373] Test net output #0: accuracy = 0
I0516 02:08:07.847247 5662 solver.cpp:373] Test net output #1: loss = 7.87437 (* 1 = 7.87437 loss)
I0516 02:08:09.886698 5662 solver.cpp:221] Iteration 0, loss = 7.87508
I0516 02:08:09.886761 5662 solver.cpp:236] Train net output #0: loss = 7.87812 (* 1 = 7.87812 loss)
I0516 02:08:09.886782 5662 solver.cpp:542] Iteration 0, lr = 0.1
I0516 02:08:09.908810 5661 solver.cpp:373] Test net output #0: accuracy = 0
I0516 02:08:09.908860 5661 solver.cpp:373] Test net output #1: loss = 7.87291 (* 1 = 7.87291 loss)
F0516 02:08:10.520869 5661 syncedmem.cpp:51] Check failed: error == cudaSuccess (2 vs. 0) out of memory
*** Check failure stack trace: ***
@ 0x7f085e6939fd google::LogMessage::Fail()
@ 0x7f085e69589d google::LogMessage::SendToLog()
@ 0x7f085e6935ec google::LogMessage::Flush()
@ 0x7f085e6961be google::LogMessageFatal::~LogMessageFatal()
@ 0x7f085ed8b98b caffe::SyncedMemory::mutable_gpu_data()
@ 0x7f085edb08c1 caffe::Blob<>::mutable_gpu_data()
@ 0x7f085edddd1b caffe::BNLayer<>::Forward_gpu()
@ 0x7f085ec70a2c caffe::Net<>::ForwardFromTo()
@ 0x7f085ec70cbf caffe::Net<>::ForwardPrefilled()
@ 0x7f085eda6813 caffe::Solver<>::Step()
@ 0x7f085eda730b caffe::Solver<>::Solve()
@ 0x408b04 train()
@ 0x4067ad main
@ 0x7f085dbaaec5 (unknown)
@ 0x406bc8 (unknown)
F0516 02:08:10.585333 5662 syncedmem.cpp:51] Check failed: error == cudaSuccess (2 vs. 0) out of memory
*** Check failure stack trace: ***
@ 0x7f8cbd9429fd google::LogMessage::Fail()
@ 0x7f8cbd94489d google::LogMessage::SendToLog()
@ 0x7f8cbd9425ec google::LogMessage::Flush()
@ 0x7f8cbd9451be google::LogMessageFatal::~LogMessageFatal()
@ 0x7f8cbe03a98b caffe::SyncedMemory::mutable_gpu_data()
@ 0x7f8cbe05f8c1 caffe::Blob<>::mutable_gpu_data()
@ 0x7f8cbe04c4dd caffe::SGDSolver<>::ComputeUpdateValue()
@ 0x7f8cbe049b53 caffe::SGDSolver<>::ApplyUpdate()
@ 0x7f8cbe055ed6 caffe::Solver<>::Step()
@ 0x7f8cbe05630b caffe::Solver<>::Solve()
@ 0x408b04 train()
@ 0x4067ad main
@ 0x7f8cbce59ec5 (unknown)

@ 0x406bc8 (unknown)

mpirun noticed that process rank 0 with PID 5661 on node k-SYS-7048GR-TR exited on signal 6 (Aborted).

Extracting train set
E0516 02:08:12.211585 7957 extract_features.cpp:54] Using GPU
E0516 02:08:12.211988 7957 extract_features.cpp:60] Using Device_id=0
F0516 02:08:33.885123 7957 io.cpp:52] Check failed: fd != -1 (-1 vs. -1) File not found: external/exp/snapshots/jstl/jstl_iter_55000.caffemodel
*** Check failure stack trace: ***
@ 0x7f7df020d9fd google::LogMessage::Fail()
@ 0x7f7df020f89d google::LogMessage::SendToLog()
@ 0x7f7df020d5ec google::LogMessage::Flush()
@ 0x7f7df02101be google::LogMessageFatal::~LogMessageFatal()
@ 0x7f7df0688f84 caffe::ReadProtoFromBinaryFile()
@ 0x7f7df06a69cc caffe::ReadNetParamsFromBinaryFileOrDie()
@ 0x7f7df059e47a caffe::Net<>::CopyTrainedLayersFromBinaryProto()
@ 0x7f7df059e501 caffe::Net<>::CopyTrainedLayersFrom()
@ 0x408879 feature_extraction_pipeline<>()
@ 0x7f7def642ec5 (unknown)
@ 0x403e3e (unknown)
scripts/routines.sh: line 93: 7957 Aborted (core dumped) ${CAFFE_DIR}/build/tools/extract_features ${trained_model} ${model} ${blob},label ${result_dir}/${subset}_features_lmdb,${result_dir}/${subset}_labels_lmdb ${num_iters} lmdb GPU 0
Extracting val set
E0516 02:08:36.303149 8098 extract_features.cpp:54] Using GPU
E0516 02:08:36.303547 8098 extract_features.cpp:60] Using Device_id=0
F0516 02:08:55.747742 8098 io.cpp:52] Check failed: fd != -1 (-1 vs. -1) File not found: external/exp/snapshots/jstl/jstl_iter_55000.caffemodel
*** Check failure stack trace: ***
@ 0x7fb37d22a9fd google::LogMessage::Fail()
@ 0x7fb37d22c89d google::LogMessage::SendToLog()
@ 0x7fb37d22a5ec google::LogMessage::Flush()
@ 0x7fb37d22d1be google::LogMessageFatal::~LogMessageFatal()
@ 0x7fb37d6a5f84 caffe::ReadProtoFromBinaryFile()
@ 0x7fb37d6c39cc caffe::ReadNetParamsFromBinaryFileOrDie()
@ 0x7fb37d5bb47a caffe::Net<>::CopyTrainedLayersFromBinaryProto()
@ 0x7fb37d5bb501 caffe::Net<>::CopyTrainedLayersFrom()
@ 0x408879 feature_extraction_pipeline<>()
@ 0x7fb37c65fec5 (unknown)
@ 0x403e3e (unknown)
scripts/routines.sh: line 93: 8098 Aborted (core dumped) ${CAFFE_DIR}/build/tools/extract_features ${trained_model} ${model} ${blob},label ${result_dir}/${subset}_features_lmdb,${result_dir}/${subset}_labels_lmdb ${num_iters} lmdb GPU 0
Extracting test_probe set
E0516 02:08:58.312100 8225 extract_features.cpp:54] Using GPU
E0516 02:08:58.312475 8225 extract_features.cpp:60] Using Device_id=0
F0516 02:09:22.046344 8225 io.cpp:52] Check failed: fd != -1 (-1 vs. -1) File not found: external/exp/snapshots/jstl/jstl_iter_55000.caffemodel
*** Check failure stack trace: ***
@ 0x7f90784889fd google::LogMessage::Fail()
@ 0x7f907848a89d google::LogMessage::SendToLog()
@ 0x7f90784885ec google::LogMessage::Flush()
@ 0x7f907848b1be google::LogMessageFatal::~LogMessageFatal()
@ 0x7f9078903f84 caffe::ReadProtoFromBinaryFile()
@ 0x7f90789219cc caffe::ReadNetParamsFromBinaryFileOrDie()
@ 0x7f907881947a caffe::Net<>::CopyTrainedLayersFromBinaryProto()
@ 0x7f9078819501 caffe::Net<>::CopyTrainedLayersFrom()
@ 0x408879 feature_extraction_pipeline<>()
@ 0x7f90778bdec5 (unknown)
@ 0x403e3e (unknown)
scripts/routines.sh: line 93: 8225 Aborted (core dumped) ${CAFFE_DIR}/build/tools/extract_features ${trained_model} ${model} ${blob},label ${result_dir}/${subset}_features_lmdb,${result_dir}/${subset}_labels_lmdb ${num_iters} lmdb GPU 0
Extracting test_gallery set
E0516 02:09:23.331884 8362 extract_features.cpp:54] Using GPU
E0516 02:09:23.332203 8362 extract_features.cpp:60] Using Device_id=0
F0516 02:09:46.931263 8362 io.cpp:52] Check failed: fd != -1 (-1 vs. -1) File not found: external/exp/snapshots/jstl/jstl_iter_55000.caffemodel
*** Check failure stack trace: ***
@ 0x7fd12e2489fd google::LogMessage::Fail()
@ 0x7fd12e24a89d google::LogMessage::SendToLog()
@ 0x7fd12e2485ec google::LogMessage::Flush()
@ 0x7fd12e24b1be google::LogMessageFatal::~LogMessageFatal()
@ 0x7fd12e6c3f84 caffe::ReadProtoFromBinaryFile()
@ 0x7fd12e6e19cc caffe::ReadNetParamsFromBinaryFileOrDie()
@ 0x7fd12e5d947a caffe::Net<>::CopyTrainedLayersFromBinaryProto()
@ 0x7fd12e5d9501 caffe::Net<>::CopyTrainedLayersFrom()
@ 0x408879 feature_extraction_pipeline<>()
@ 0x7fd12d67dec5 (unknown)
@ 0x403e3e (unknown)
scripts/routines.sh: line 93: 8362 Aborted (core dumped) ${CAFFE_DIR}/build/tools/extract_features ${trained_model} ${model} ${blob},label ${result_dir}/${subset}_features_lmdb,${result_dir}/${subset}_labels_lmdb ${num_iters} lmdb GPU 0
Extracting train set
E0516 02:09:48.710749 8503 extract_features.cpp:54] Using GPU
E0516 02:09:48.717394 8503 extract_features.cpp:60] Using Device_id=0
F0516 02:10:12.159739 8503 io.cpp:52] Check failed: fd != -1 (-1 vs. -1) File not found: external/exp/snapshots/jstl/jstl_iter_55000.caffemodel
*** Check failure stack trace: ***
@ 0x7ffceedaa9fd google::LogMessage::Fail()
@ 0x7ffceedac89d google::LogMessage::SendToLog()
@ 0x7ffceedaa5ec google::LogMessage::Flush()
@ 0x7ffceedad1be google::LogMessageFatal::~LogMessageFatal()
@ 0x7ffcef225f84 caffe::ReadProtoFromBinaryFile()
@ 0x7ffcef2439cc caffe::ReadNetParamsFromBinaryFileOrDie()
@ 0x7ffcef13b47a caffe::Net<>::CopyTrainedLayersFromBinaryProto()
@ 0x7ffcef13b501 caffe::Net<>::CopyTrainedLayersFrom()
@ 0x408879 feature_extraction_pipeline<>()
@ 0x7ffcee1dfec5 (unknown)
@ 0x403e3e (unknown)
scripts/routines.sh: line 93: 8503 Aborted (core dumped) ${CAFFE_DIR}/build/tools/extract_features ${trained_model} ${model} ${blob},label ${result_dir}/${subset}_features_lmdb,${result_dir}/${subset}_labels_lmdb ${num_iters} lmdb GPU 0
Extracting val set
E0516 02:10:14.713034 8647 extract_features.cpp:54] Using GPU
E0516 02:10:14.713467 8647 extract_features.cpp:60] Using Device_id=0
F0516 02:10:37.292449 8647 io.cpp:52] Check failed: fd != -1 (-1 vs. -1) File not found: external/exp/snapshots/jstl/jstl_iter_55000.caffemodel
*** Check failure stack trace: ***
@ 0x7f8d79dc19fd google::LogMessage::Fail()
@ 0x7f8d79dc389d google::LogMessage::SendToLog()
@ 0x7f8d79dc15ec google::LogMessage::Flush()
@ 0x7f8d79dc41be google::LogMessageFatal::~LogMessageFatal()
@ 0x7f8d7a23cf84 caffe::ReadProtoFromBinaryFile()
@ 0x7f8d7a25a9cc caffe::ReadNetParamsFromBinaryFileOrDie()
@ 0x7f8d7a15247a caffe::Net<>::CopyTrainedLayersFromBinaryProto()
@ 0x7f8d7a152501 caffe::Net<>::CopyTrainedLayersFrom()
@ 0x408879 feature_extraction_pipeline<>()
@ 0x7f8d791f6ec5 (unknown)
@ 0x403e3e (unknown)
scripts/routines.sh: line 93: 8647 Aborted (core dumped) ${CAFFE_DIR}/build/tools/extract_features ${trained_model} ${model} ${blob},label ${result_dir}/${subset}_features_lmdb,${result_dir}/${subset}_labels_lmdb ${num_iters} lmdb GPU 0
Extracting test_probe set
E0516 02:10:39.804507 8794 extract_features.cpp:54] Using GPU
E0516 02:10:39.804853 8794 extract_features.cpp:60] Using Device_id=0
F0516 02:11:02.423830 8794 io.cpp:52] Check failed: fd != -1 (-1 vs. -1) File not found: external/exp/snapshots/jstl/jstl_iter_55000.caffemodel
*** Check failure stack trace: ***
@ 0x7fe71b0db9fd google::LogMessage::Fail()
@ 0x7fe71b0dd89d google::LogMessage::SendToLog()
@ 0x7fe71b0db5ec google::LogMessage::Flush()
@ 0x7fe71b0de1be google::LogMessageFatal::~LogMessageFatal()
@ 0x7fe71b556f84 caffe::ReadProtoFromBinaryFile()
@ 0x7fe71b5749cc caffe::ReadNetParamsFromBinaryFileOrDie()
@ 0x7fe71b46c47a caffe::Net<>::CopyTrainedLayersFromBinaryProto()
@ 0x7fe71b46c501 caffe::Net<>::CopyTrainedLayersFrom()
@ 0x408879 feature_extraction_pipeline<>()
@ 0x7fe71a510ec5 (unknown)
@ 0x403e3e (unknown)
scripts/routines.sh: line 93: 8794 Aborted (core dumped) ${CAFFE_DIR}/build/tools/extract_features ${trained_model} ${model} ${blob},label ${result_dir}/${subset}_features_lmdb,${result_dir}/${subset}_labels_lmdb ${num_iters} lmdb GPU 0
Extracting test_gallery set
E0516 02:11:04.005192 8961 extract_features.cpp:54] Using GPU
E0516 02:11:04.011663 8961 extract_features.cpp:60] Using Device_id=0
F0516 02:11:27.216060 8961 io.cpp:52] Check failed: fd != -1 (-1 vs. -1) File not found: external/exp/snapshots/jstl/jstl_iter_55000.caffemodel
*** Check failure stack trace: ***
@ 0x7fcbfa62d9fd google::LogMessage::Fail()
@ 0x7fcbfa62f89d google::LogMessage::SendToLog()
@ 0x7fcbfa62d5ec google::LogMessage::Flush()
@ 0x7fcbfa6301be google::LogMessageFatal::~LogMessageFatal()
@ 0x7fcbfaaa8f84 caffe::ReadProtoFromBinaryFile()
@ 0x7fcbfaac69cc caffe::ReadNetParamsFromBinaryFileOrDie()
@ 0x7fcbfa9be47a caffe::Net<>::CopyTrainedLayersFromBinaryProto()
@ 0x7fcbfa9be501 caffe::Net<>::CopyTrainedLayersFrom()
@ 0x408879 feature_extraction_pipeline<>()
@ 0x7fcbf9a62ec5 (unknown)
@ 0x403e3e (unknown)
scripts/routines.sh: line 93: 8961 Aborted (core dumped) ${CAFFE_DIR}/build/tools/extract_features ${trained_model} ${model} ${blob},label ${result_dir}/${subset}_features_lmdb,${result_dir}/${subset}_labels_lmdb ${num_iters} lmdb GPU 0
Extracting train set
E0516 02:11:29.147552 9129 extract_features.cpp:54] Using GPU
E0516 02:11:29.147971 9129 extract_features.cpp:60] Using Device_id=0
F0516 02:11:49.092274 9129 io.cpp:52] Check failed: fd != -1 (-1 vs. -1) File not found: external/exp/snapshots/jstl/jstl_iter_55000.caffemodel
*** Check failure stack trace: ***
@ 0x7f29d27f19fd google::LogMessage::Fail()
@ 0x7f29d27f389d google::LogMessage::SendToLog()
@ 0x7f29d27f15ec google::LogMessage::Flush()
@ 0x7f29d27f41be google::LogMessageFatal::~LogMessageFatal()
@ 0x7f29d2c6cf84 caffe::ReadProtoFromBinaryFile()
@ 0x7f29d2c8a9cc caffe::ReadNetParamsFromBinaryFileOrDie()
@ 0x7f29d2b8247a caffe::Net<>::CopyTrainedLayersFromBinaryProto()
@ 0x7f29d2b82501 caffe::Net<>::CopyTrainedLayersFrom()
@ 0x408879 feature_extraction_pipeline<>()
@ 0x7f29d1c26ec5 (unknown)
@ 0x403e3e (unknown)
scripts/routines.sh: line 93: 9129 Aborted (core dumped) ${CAFFE_DIR}/build/tools/extract_features ${trained_model} ${model} ${blob},label ${result_dir}/${subset}_features_lmdb,${result_dir}/${subset}_labels_lmdb ${num_iters} lmdb GPU 0
Extracting val set
E0516 02:11:50.690008 9278 extract_features.cpp:54] Using GPU
E0516 02:11:50.690439 9278 extract_features.cpp:60] Using Device_id=0
F0516 02:12:13.775943 9278 io.cpp:52] Check failed: fd != -1 (-1 vs. -1) File not found: external/exp/snapshots/jstl/jstl_iter_55000.caffemodel
*** Check failure stack trace: ***
@ 0x7f2b838689fd google::LogMessage::Fail()
@ 0x7f2b8386a89d google::LogMessage::SendToLog()
@ 0x7f2b838685ec google::LogMessage::Flush()
@ 0x7f2b8386b1be google::LogMessageFatal::~LogMessageFatal()
@ 0x7f2b83ce3f84 caffe::ReadProtoFromBinaryFile()
@ 0x7f2b83d019cc caffe::ReadNetParamsFromBinaryFileOrDie()
@ 0x7f2b83bf947a caffe::Net<>::CopyTrainedLayersFromBinaryProto()
@ 0x7f2b83bf9501 caffe::Net<>::CopyTrainedLayersFrom()
@ 0x408879 feature_extraction_pipeline<>()
@ 0x7f2b82c9dec5 (unknown)
@ 0x403e3e (unknown)
scripts/routines.sh: line 93: 9278 Aborted (core dumped) ${CAFFE_DIR}/build/tools/extract_features ${trained_model} ${model} ${blob},label ${result_dir}/${subset}_features_lmdb,${result_dir}/${subset}_labels_lmdb ${num_iters} lmdb GPU 0
Extracting test_probe set
E0516 02:12:16.299953 9420 extract_features.cpp:54] Using GPU
E0516 02:12:16.300331 9420 extract_features.cpp:60] Using Device_id=0
F0516 02:12:36.427316 9420 io.cpp:52] Check failed: fd != -1 (-1 vs. -1) File not found: external/exp/snapshots/jstl/jstl_iter_55000.caffemodel
*** Check failure stack trace: ***
@ 0x7ffa727e39fd google::LogMessage::Fail()
@ 0x7ffa727e589d google::LogMessage::SendToLog()
@ 0x7ffa727e35ec google::LogMessage::Flush()
@ 0x7ffa727e61be google::LogMessageFatal::~LogMessageFatal()
@ 0x7ffa72c5ef84 caffe::ReadProtoFromBinaryFile()
@ 0x7ffa72c7c9cc caffe::ReadNetParamsFromBinaryFileOrDie()
@ 0x7ffa72b7447a caffe::Net<>::CopyTrainedLayersFromBinaryProto()
@ 0x7ffa72b74501 caffe::Net<>::CopyTrainedLayersFrom()
@ 0x408879 feature_extraction_pipeline<>()
@ 0x7ffa71c18ec5 (unknown)
@ 0x403e3e (unknown)
scripts/routines.sh: line 93: 9420 Aborted (core dumped) ${CAFFE_DIR}/build/tools/extract_features ${trained_model} ${model} ${blob},label ${result_dir}/${subset}_features_lmdb,${result_dir}/${subset}_labels_lmdb ${num_iters} lmdb GPU 0
Extracting test_gallery set
E0516 02:12:38.635035 9580 extract_features.cpp:54] Using GPU
E0516 02:12:38.635535 9580 extract_features.cpp:60] Using Device_id=0
F0516 02:12:57.957043 9580 io.cpp:52] Check failed: fd != -1 (-1 vs. -1) File not found: external/exp/snapshots/jstl/jstl_iter_55000.caffemodel
*** Check failure stack trace: ***
@ 0x7f706e5339fd google::LogMessage::Fail()
@ 0x7f706e53589d google::LogMessage::SendToLog()
@ 0x7f706e5335ec google::LogMessage::Flush()
@ 0x7f706e5361be google::LogMessageFatal::~LogMessageFatal()
@ 0x7f706e9aef84 caffe::ReadProtoFromBinaryFile()
@ 0x7f706e9cc9cc caffe::ReadNetParamsFromBinaryFileOrDie()
@ 0x7f706e8c447a caffe::Net<>::CopyTrainedLayersFromBinaryProto()
@ 0x7f706e8c4501 caffe::Net<>::CopyTrainedLayersFrom()
@ 0x408879 feature_extraction_pipeline<>()
@ 0x7f706d968ec5 (unknown)
@ 0x403e3e (unknown)
scripts/routines.sh: line 93: 9580 Aborted (core dumped) ${CAFFE_DIR}/build/tools/extract_features ${trained_model} ${model} ${blob},label ${result_dir}/${subset}_features_lmdb,${result_dir}/${subset}_labels_lmdb ${num_iters} lmdb GPU 0
prid
Traceback (most recent call last):
File "eval/metric_learning.py", line 104, in
main(args)
File "eval/metric_learning.py", line 85, in main
M = _learn_metric(X, Y, args.method)
File "eval/metric_learning.py", line 43, in _learn_metric
M = np.eye(X.shape[1])
IndexError: tuple index out of range

viper
Traceback (most recent call last):
File "eval/metric_learning.py", line 104, in
main(args)
File "eval/metric_learning.py", line 85, in main
M = _learn_metric(X, Y, args.method)
File "eval/metric_learning.py", line 43, in _learn_metric
M = np.eye(X.shape[1])
IndexError: tuple index out of range

3dpes
Traceback (most recent call last):
File "eval/metric_learning.py", line 104, in
main(args)
File "eval/metric_learning.py", line 85, in main
M = _learn_metric(X, Y, args.method)
File "eval/metric_learning.py", line 43, in _learn_metric
M = np.eye(X.shape[1])
IndexError: tuple index out of range

cogito2012@k-SYS-7048GR-TR:~/dgd_person_reid-master$
////////////////////////////////////////////////////////////////

I wonder whether this problem results from small GPU memory (Actually 12G is enough), or caused by any other reason?

When I run the "scripts/format_rawdata.sh", it gives some errors

error: invalid compressed data to inflateexternal/raw/cuhk03_release/cuhk-03.mat

Hi all

I wrote a Python implementation covering the feature extraction and on VIPeR dataset.
(The CMC function is imported from the CMC.py of this repo)

Using the pretrained JSTL+DGD mode provided by @Cysu, I only got 19.33% of Top 1 accuracy. It's far from the 37.7% on the paper.

I guess there are bugs in my script, but I couldn't find it.

I add new dataset, but it has a error

montego7878@montego7878-All-Series:~$ ./下載/dgd_person_reid/scripts/exp_individually.sh Test
I0217 17:26:38.334089 12962 caffe.cpp:170] Use GPU with device ID 0
I0217 17:26:38.502809 12962 caffe.cpp:178] Starting Optimization
I0217 17:26:38.502868 12962 solver.cpp:41] Initializing solver from parameters:
test_iter: 3
test_interval: 20
base_lr: 0.1
display: 20
max_iter: 2700
lr_policy: "stepdecr"
gamma: 0.04
momentum: 0.9
weight_decay: 0.0005
stepsize: 100
snapshot_prefix: "external/exp/snapshots/individually/Test"
solver_mode: GPU
net: "models/individually/Test_trainval.prototxt"
test_initialization: true
average_loss: 20
iter_size: 2
min_lr: 0.0005
I0217 17:26:38.502887 12962 solver.cpp:79] Creating training net from net file: models/individually/Test_trainval.prototxt
F0217 17:26:38.502918 12962 io.cpp:34] Check failed: fd != -1 (-1 vs. -1) File not found: models/individually/Test_trainval.prototxt
*** Check failure stack trace: ***
@ 0x7fa22b389daa (unknown)
@ 0x7fa22b389ce4 (unknown)
@ 0x7fa22b3896e6 (unknown)
@ 0x7fa22b38c687 (unknown)
@ 0x7fa22ba6a6ed caffe::ReadProtoFromTextFile()
@ 0x7fa22ba7f934 caffe::ReadNetParamsFromTextFileOrDie()
@ 0x7fa22ba8b922 caffe::Solver<>::InitTrainNet()
@ 0x7fa22ba8c7de caffe::Solver<>::Init()
@ 0x7fa22ba8c976 caffe::Solver<>::Solver()
@ 0x40e4e3 caffe::GetSolver<>()
@ 0x407080 train()
@ 0x4053b6 main
@ 0x7fa22a89bf45 (unknown)
@ 0x4059b1 (unknown)
@ (nil) (unknown)
Extracting train set
E0217 17:26:38.640938 12986 extract_features.cpp:54] Using GPU
E0217 17:26:38.641165 12986 extract_features.cpp:60] Using Device_id=0
F0217 17:26:43.106864 12986 io.cpp:52] Check failed: fd != -1 (-1 vs. -1) File not found: external/exp/snapshots/individually/Test_iter_2700.caffemodel
*** Check failure stack trace: ***
@ 0x7f1726fdedaa (unknown)
@ 0x7f1726fdece4 (unknown)
@ 0x7f1726fde6e6 (unknown)
@ 0x7f1726fe1687 (unknown)
@ 0x7f17274618a3 caffe::ReadProtoFromBinaryFile()
@ 0x7f17274769b4 caffe::ReadNetParamsFromBinaryFileOrDie()
@ 0x7f172742e0b7 caffe::Net<>::CopyTrainedLayersFromBinaryProto()
@ 0x7f172742e126 caffe::Net<>::CopyTrainedLayersFrom()
@ 0x4080b8 feature_extraction_pipeline<>()
@ 0x7f172640ef45 (unknown)
@ 0x403bce (unknown)
@ (nil) (unknown)
scripts/routines.sh: line 96: 12986 已經終止 (core dumped) ${CAFFE_DIR}/build/tools/extract_features ${trained_model} ${model} ${blob},label ${result_dir}/${subset}_features_lmdb,${result_dir}/${subset}_labels_lmdb ${num_iters} lmdb GPU 0
Extracting val set
E0217 17:26:44.411387 13007 extract_features.cpp:54] Using GPU
E0217 17:26:44.411638 13007 extract_features.cpp:60] Using Device_id=0
F0217 17:26:48.852941 13007 io.cpp:52] Check failed: fd != -1 (-1 vs. -1) File not found: external/exp/snapshots/individually/Test_iter_2700.caffemodel
*** Check failure stack trace: ***
@ 0x7f2f07ea4daa (unknown)
@ 0x7f2f07ea4ce4 (unknown)
@ 0x7f2f07ea46e6 (unknown)
@ 0x7f2f07ea7687 (unknown)
@ 0x7f2f083278a3 caffe::ReadProtoFromBinaryFile()
@ 0x7f2f0833c9b4 caffe::ReadNetParamsFromBinaryFileOrDie()
@ 0x7f2f082f40b7 caffe::Net<>::CopyTrainedLayersFromBinaryProto()
@ 0x7f2f082f4126 caffe::Net<>::CopyTrainedLayersFrom()
@ 0x4080b8 feature_extraction_pipeline<>()
@ 0x7f2f072d4f45 (unknown)
@ 0x403bce (unknown)
@ (nil) (unknown)
scripts/routines.sh: line 96: 13007 已經終止 (core dumped) ${CAFFE_DIR}/build/tools/extract_features ${trained_model} ${model} ${blob},label ${result_dir}/${subset}_features_lmdb,${result_dir}/${subset}_labels_lmdb ${num_iters} lmdb GPU 0
Extracting test_probe set
E0217 17:26:50.129348 13028 extract_features.cpp:54] Using GPU
E0217 17:26:50.129575 13028 extract_features.cpp:60] Using Device_id=0
F0217 17:26:54.599745 13028 io.cpp:52] Check failed: fd != -1 (-1 vs. -1) File not found: external/exp/snapshots/individually/Test_iter_2700.caffemodel
*** Check failure stack trace: ***
@ 0x7fd5e7a20daa (unknown)
@ 0x7fd5e7a20ce4 (unknown)
@ 0x7fd5e7a206e6 (unknown)
@ 0x7fd5e7a23687 (unknown)
@ 0x7fd5e7ea38a3 caffe::ReadProtoFromBinaryFile()
@ 0x7fd5e7eb89b4 caffe::ReadNetParamsFromBinaryFileOrDie()
@ 0x7fd5e7e700b7 caffe::Net<>::CopyTrainedLayersFromBinaryProto()
@ 0x7fd5e7e70126 caffe::Net<>::CopyTrainedLayersFrom()
@ 0x4080b8 feature_extraction_pipeline<>()
@ 0x7fd5e6e50f45 (unknown)
@ 0x403bce (unknown)
@ (nil) (unknown)
scripts/routines.sh: line 96: 13028 已經終止 (core dumped) ${CAFFE_DIR}/build/tools/extract_features ${trained_model} ${model} ${blob},label ${result_dir}/${subset}_features_lmdb,${result_dir}/${subset}_labels_lmdb ${num_iters} lmdb GPU 0
Extracting test_gallery set
E0217 17:26:55.872711 13048 extract_features.cpp:54] Using GPU
E0217 17:26:55.872951 13048 extract_features.cpp:60] Using Device_id=0
F0217 17:27:00.327179 13048 io.cpp:52] Check failed: fd != -1 (-1 vs. -1) File not found: external/exp/snapshots/individually/Test_iter_2700.caffemodel
*** Check failure stack trace: ***
@ 0x7fcb9c08ddaa (unknown)
@ 0x7fcb9c08dce4 (unknown)
@ 0x7fcb9c08d6e6 (unknown)
@ 0x7fcb9c090687 (unknown)
@ 0x7fcb9c5108a3 caffe::ReadProtoFromBinaryFile()
@ 0x7fcb9c5259b4 caffe::ReadNetParamsFromBinaryFileOrDie()
@ 0x7fcb9c4dd0b7 caffe::Net<>::CopyTrainedLayersFromBinaryProto()
@ 0x7fcb9c4dd126 caffe::Net<>::CopyTrainedLayersFrom()
@ 0x4080b8 feature_extraction_pipeline<>()
@ 0x7fcb9b4bdf45 (unknown)
@ 0x403bce (unknown)
@ (nil) (unknown)
scripts/routines.sh: line 96: 13048 已經終止 (core dumped) ${CAFFE_DIR}/build/tools/extract_features ${trained_model} ${model} ${blob},label ${result_dir}/${subset}_features_lmdb,${result_dir}/${subset}_labels_lmdb ${num_iters} lmdb GPU 0
Traceback (most recent call last):
File "eval/metric_learning.py", line 158, in
main(args)
File "eval/metric_learning.py", line 139, in main
M = _learn_metric(X, Y, args.method)
File "eval/metric_learning.py", line 95, in _learn_metric
M = np.eye(X.shape[1])
IndexError: tuple index out of range

@Cysu
Thankyou very much for your code on person re_identification
I have install openmpi-1.6.5
But when I make -j8 in the external/caffe,I got a error:

CXX .build_release/src/caffe/proto/caffe.pb.cc
CXX src/caffe/layers/data_layer.cpp
CXX src/caffe/layers/multinomial_logistic_loss_layer.cpp
CXX src/caffe/layers/loss_layer.cpp
CXX src/caffe/layers/base_conv_layer.cpp
CXX src/caffe/layers/memory_data_layer.cpp
CXX src/caffe/layers/cudnn_relu_layer.cpp
CXX src/caffe/layers/reshape_layer.cpp
CXX src/caffe/layers/cudnn_softmax_layer.cpp
CXX src/caffe/layers/threshold_layer.cpp
CXX src/caffe/layers/sigmoid_layer.cpp
CXX src/caffe/layers/argmax_layer.cpp
CXX src/caffe/layers/hdf5_data_layer.cpp
In file included from src/caffe/layers/data_layer.cpp:16:0:
./include/caffe/util/mpi_templates.hpp: In function ‘int MPIAllgather(int, const void*, void*, MPI_Comm) [with Dtype = float; MPI_Comm = ompi_communicator_t*]’:
./include/caffe/util/mpi_templates.hpp:45:28: error: invalid conversion from ‘const void*’ to ‘void*’ [-fpermissive]
comm);
^
In file included from ./include/caffe/util/mpi_templates.hpp:5:0,
from src/caffe/layers/data_layer.cpp:16:
/usr/local/include/mpi.h:1033:20: note: initializing argument 1 of ‘int MPI_Allgather(void*, int, MPI_Datatype, void*, int, MPI_Datatype, MPI_Comm)’
OMPI_DECLSPEC int MPI_Allgather(void sendbuf, int sendcount, MPI_Datatype sendtype,
^
In file included from src/caffe/layers/data_layer.cpp:16:0:
./include/caffe/util/mpi_templates.hpp: In function ‘int MPIAllgather(int, const void, void*, MPI_Comm) [with Dtype = double; MPI_Comm = ompi_communicator_t*]’:
./include/caffe/util/mpi_templates.hpp:51:28: error: invalid conversion from ‘const void*’ to ‘void*’ [-fpermissive]
comm);
^
In file included from ./include/caffe/util/mpi_templates.hpp:5:0,
from src/caffe/layers/data_layer.cpp:16:
/usr/local/include/mpi.h:1033:20: note: initializing argument 1 of ‘int MPI_Allgather(void*, int, MPI_Datatype, void*, int, MPI_Datatype, MPI_Comm)’
OMPI_DECLSPEC int MPI_Allgather(void sendbuf, int sendcount, MPI_Datatype sendtype,
^
In file included from src/caffe/layers/data_layer.cpp:16:0:
./include/caffe/util/mpi_templates.hpp: In function ‘int MPIScatter(int, const void, void*, int, MPI_Comm) [with Dtype = float; MPI_Comm = ompi_communicator_t*]’:
./include/caffe/util/mpi_templates.hpp:61:17: error: invalid conversion from ‘const void*’ to ‘void*’ [-fpermissive]
root, comm);
^
In file included from ./include/caffe/util/mpi_templates.hpp:5:0,
from src/caffe/layers/data_layer.cpp:16:
/usr/local/include/mpi.h:1375:20: note: initializing argument 1 of ‘int MPI_Scatter(void*, int, MPI_Datatype, void*, int, MPI_Datatype, int, MPI_Comm)’
OMPI_DECLSPEC int MPI_Scatter(void sendbuf, int sendcount, MPI_Datatype sendtype,
^
In file included from src/caffe/layers/data_layer.cpp:16:0:
./include/caffe/util/mpi_templates.hpp: In function ‘int MPIScatter(int, const void, void*, int, MPI_Comm) [with Dtype = double; MPI_Comm = ompi_communicator_t*]’:
./include/caffe/util/mpi_templates.hpp:67:17: error: invalid conversion from ‘const void*’ to ‘void*’ [-fpermissive]
root, comm);
^
In file included from ./include/caffe/util/mpi_templates.hpp:5:0,
from src/caffe/layers/data_layer.cpp:16:
/usr/local/include/mpi.h:1375:20: note: initializing argument 1 of ‘int MPI_Scatter(void*, int, MPI_Datatype, void*, int, MPI_Datatype, int, MPI_Comm)’
OMPI_DECLSPEC int MPI_Scatter(void *sendbuf, int sendcount, MPI_Datatype sendtype,
^
make: *** [.build_release/src/caffe/layers/data_layer.o] Error 1
make: *** Waiting for unfinished jobs....

The following is my Makefile.config:

Refer to http://caffe.berkeleyvision.org/installation.html

Contributions simplifying and improving our build system are welcome!

cuDNN acceleration switch (uncomment to build with cuDNN).

USE_CUDNN := 1

MPI data parallelization switch (uncomment to build with MPI).

USE_MPI := 1
MPI_INCLUDE := /usr/local/include
MPI_LIB := /usr/local/lib

CPU-only switch (uncomment to build without GPU support).

CPU_ONLY := 1

To customize your choice of compiler, uncomment and set the following.

N.B. the default for Linux is g++ and the default for OSX is clang++

CUSTOM_CXX := g++

CUDA directory contains bin/ and lib/ directories that we need.

CUDA_DIR := /usr/local/cuda

On Ubuntu 14.04, if cuda tools are installed via

"sudo apt-get install nvidia-cuda-toolkit" then use this instead:

CUDA_DIR := /usr

CUDA architecture setting: going with all of them.

For CUDA < 6.0, comment the *_50 lines for compatibility.

CUDA_ARCH := -gencode arch=compute_20,code=sm_20
-gencode arch=compute_20,code=sm_21
-gencode arch=compute_30,code=sm_30
-gencode arch=compute_35,code=sm_35
-gencode arch=compute_50,code=sm_50
-gencode arch=compute_50,code=compute_50

BLAS choice:

atlas for ATLAS (default)

mkl for MKL

open for OpenBlas

BLAS := atlas

Custom (MKL/ATLAS/OpenBLAS) include and lib directories.

Leave commented to accept the defaults for your choice of BLAS

(which should work)!

BLAS_INCLUDE := /path/to/your/blas

BLAS_LIB := /path/to/your/blas

Homebrew puts openblas in a directory that is not on the standard search path

BLAS_INCLUDE := $(shell brew --prefix openblas)/include

BLAS_LIB := $(shell brew --prefix openblas)/lib

This is required only if you will compile the matlab interface.

MATLAB directory should contain the mex binary in /bin.

MATLAB_DIR := /usr/local/MATLAB/R2014a

MATLAB_DIR := /Applications/MATLAB_R2012b.app

NOTE: this is required only if you will compile the python interface.

We need to be able to find Python.h and numpy/arrayobject.h.

PYTHON_INCLUDE := /usr/include/python2.7
/usr/lib/python2.7/dist-packages/numpy/core/include

Anaconda Python distribution is quite popular. Include path:

Verify anaconda location, sometimes it's in root.

ANACONDA_HOME := $(HOME)/anaconda

PYTHON_INCLUDE := $(ANACONDA_HOME)/include \

	# $(ANACONDA_HOME)/include/python2.7 \
	# $(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include \

We need to be able to find libpythonX.X.so or .dylib.

PYTHON_LIB := /usr/local/lib

PYTHON_LIB := $(ANACONDA_HOME)/lib

Homebrew installs numpy in a non standard path (keg only)

PYTHON_INCLUDE += $(dir $(shell python -c 'import numpy.core; print(numpy.core.file)'))/include

PYTHON_LIB += $(shell brew --prefix numpy)/lib

Uncomment to support layers written in Python (will link against Python libs)

WITH_PYTHON_LAYER := 1

Whatever else you find you need goes here.

INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib

If Homebrew is installed at a non standard location (for example your home directory) and you use it for general dependencies

INCLUDE_DIRS += $(shell brew --prefix)/include

LIBRARY_DIRS += $(shell brew --prefix)/lib

Uncomment to use pkg-config to specify OpenCV library paths.

(Usually not necessary -- OpenCV libraries are normally installed in one of the above $LIBRARY_DIRS.)

USE_PKG_CONFIG := 1

BUILD_DIR := build
DISTRIBUTE_DIR := distribute

Uncomment for debugging. Does not work on OSX due to BVLC/caffe#171

DEBUG := 1

The ID of the GPU that 'make runtest' will use to run unit tests.

TEST_GPUID := 0

enable pretty build (comment to see full commands)

Q ?= @

Could you please tell me how to solve it?
Thank you very much!

Thanks for your time reading my questions!

I have downloaded the caffe (not the whole dgd project) provided by you and trained cuhk03 dataset individually on my computer(prototxt file is the same as yours).
However, my top1 accuracy is around 65%, which is 7% lower than the result in your paper, 72.6%.
For I used your prototxt file directly, so the problem can only be on either data preprocessing or the choice of test set.

Before training stage, I computed the mean file for both training set and validation set instead of using the fixed vector [102,102,101]. I am not sure whether this will reduce the final accuracy.

Besides, for the choice of test set, I randomly choose two pictures per person(one picture from probe set and the other from gallery set) and the size of test set is 200 pictures (100 person). Then I calculate the CMC curve by Euclidean distance. Do you choose your test set the same way as me? If it's a right way to randomly choose the test set, the top1 accuracy may fluctuate, and perhaps with more experiments I will get 72.6% as my top1 accuracy. If not, could you tell me how to choose a test set?

Hi,

when I use your trained model (jstl_dgd_deploy.prototxt) to extract features on my datasets, the command line shows "duplicate blobs produced by multiple sources" error. I guess probably there is something wrong with my code, but I have no idea what is the correct way to make it work. I have to ask for your help. Could you give me some cues to extract feature on other datsets? Thanks.

Best

how to use cmc.py to compute cmc
Have any example?
thanks you

Hello Cysu,

I am facing the "NameError: global name 'osp' is not defined" while running format_rawdata.sh. Do you have any suggestions?

Thanks,

As per the readme, Shinpuhkan is also included in the datasets used. But it is not used in the code that is available in the repository.
Is the pretrained JSTL+DGD model trained with Shinpuhkan dataset also?

Could you please provide pretrained weight? especially on Tensorflow model

你好，不好意思再一次打擾你，我是問題#16的提問者，我試著寫一個format_market1501.py將數據轉換成統一的格式並且修改對應的prototxt和bash scripts，但執行scripts/exp_individually.sh market1501後出現以下問題。想請問您該如何解決，感謝!
E0110 18:31:07.437958 27153 extract_features.cpp:54] Using GPU
E0110 18:31:07.453989 27153 extract_features.cpp:60] Using Device_id=0
F0110 18:31:18.749990 27153 io.cpp:52] Check failed: fd != -1 (-1 vs. -1) File not fou.caffemodell/exp/snapshots/individually/market1501_iter_5000
*** Check failure stack trace: ***
@ 0x7effb9480daa (unknown)
@ 0x7effb9480ce4 (unknown)
@ 0x7effb94806e6 (unknown)
@ 0x7effb9483687 (unknown)
@ 0x7effb992ec84 caffe::ReadProtoFromBinaryFile()
@ 0x7effb992753c caffe::ReadNetParamsFromBinaryFileOrDie()
@ 0x7effb9947cda caffe::Net<>::CopyTrainedLayersFromBinaryProto()
@ 0x7effb9947d61 caffe::Net<>::CopyTrainedLayersFrom()
@ 0x409689 feature_extraction_pipeline<>()
@ 0x7effb88b0f45 (unknown)
@ 0x404c4e (unknown)
@ (nil) (unknown)
scripts/routines.sh: line 93: 27153 Aborted (core dumped) ${CAFFE_DIR}/build/tools/extract_features ${trained_model} ${model} ${blob},label ${result_dir}/${subset}_features_lmdb,${result_dir}/${subset}_labels_lmdb ${num_iters} lmdb GPU 0
Extracting val set
E0110 18:31:21.511122 27173 extract_features.cpp:54] Using GPU
E0110 18:31:21.553755 27173 extract_features.cpp:60] Using Device_id=0
F0110 18:31:32.846850 27173 io.cpp:52] Check failed: fd != -1 (-1 vs. -1) File not fou.caffemodell/exp/snapshots/individually/market1501_iter_5000
*** Check failure stack trace: ***
@ 0x7fef4be5adaa (unknown)
@ 0x7fef4be5ace4 (unknown)
@ 0x7fef4be5a6e6 (unknown)
@ 0x7fef4be5d687 (unknown)
@ 0x7fef4c308c84 caffe::ReadProtoFromBinaryFile()
@ 0x7fef4c30153c caffe::ReadNetParamsFromBinaryFileOrDie()
@ 0x7fef4c321cda caffe::Net<>::CopyTrainedLayersFromBinaryProto()
@ 0x7fef4c321d61 caffe::Net<>::CopyTrainedLayersFrom()
@ 0x409689 feature_extraction_pipeline<>()
@ 0x7fef4b28af45 (unknown)
@ 0x404c4e (unknown)
@ (nil) (unknown)
scripts/routines.sh: line 93: 27173 Aborted (core dumped) ${CAFFE_DIR}/build/tools/extract_features ${trained_model} ${model} ${blob},label ${result_dir}/${subset}_features_lmdb,${result_dir}/${subset}_labels_lmdb ${num_iters} lmdb GPU 0
Extracting test_probe set
E0110 18:31:35.480466 27194 extract_features.cpp:54] Using GPU
E0110 18:31:35.487788 27194 extract_features.cpp:60] Using Device_id=0
F0110 18:31:35.970708 27194 data_transformer.cpp:552] Check failed: datum_channels > 0 (0 vs. 0)
*** Check failure stack trace: ***
@ 0x7f8c24682daa (unknown)
@ 0x7f8c24682ce4 (unknown)
@ 0x7f8c246826e6 (unknown)
@ 0x7f8c24685687 (unknown)
@ 0x7f8c24a16b51 caffe::DataTransformer<>::InferBlobShape()
@ 0x7f8c24aca1a0 caffe::DataLayer<>::DataLayerSetUp()
@ 0x7f8c24aa971b caffe::BaseDataLayer<>::LayerSetUp()
@ 0x7f8c24aa97f9 caffe::BasePrefetchingDataLayer<>::LayerSetUp()
@ 0x7f8c24b4aa09 caffe::Net<>::Init()
@ 0x7f8c24b4d96c caffe::Net<>::Net()
@ 0x40960a feature_extraction_pipeline<>()
@ 0x7f8c23ab2f45 (unknown)
@ 0x404c4e (unknown)
@ (nil) (unknown)
scripts/routines.sh: line 93: 27194 Aborted (core dumped) ${CAFFE_DIR}/build/tools/extract_features ${trained_model} ${model} ${blob},label ${result_dir}/${subset}_features_lmdb,${result_dir}/${subset}_labels_lmdb ${num_iters} lmdb GPU 0
Extracting test_gallery set
E0110 18:31:38.982168 27212 extract_features.cpp:54] Using GPU
E0110 18:31:38.999352 27212 extract_features.cpp:60] Using Device_id=0
F0110 18:31:39.510656 27212 data_transformer.cpp:552] Check failed: datum_channels > 0 (0 vs. 0)
*** Check failure stack trace: ***
@ 0x7f3e7a8ecdaa (unknown)
@ 0x7f3e7a8ecce4 (unknown)
@ 0x7f3e7a8ec6e6 (unknown)
@ 0x7f3e7a8ef687 (unknown)
@ 0x7f3e7ac80b51 caffe::DataTransformer<>::InferBlobShape()
@ 0x7f3e7ad341a0 caffe::DataLayer<>::DataLayerSetUp()
@ 0x7f3e7ad1371b caffe::BaseDataLayer<>::LayerSetUp()
@ 0x7f3e7ad137f9 caffe::BasePrefetchingDataLayer<>::LayerSetUp()
@ 0x7f3e7adb4a09 caffe::Net<>::Init()
@ 0x7f3e7adb796c caffe::Net<>::Net()
@ 0x40960a feature_extraction_pipeline<>()
@ 0x7f3e79d1cf45 (unknown)
@ 0x404c4e (unknown)
@ (nil) (unknown)
scripts/routines.sh: line 93: 27212 Aborted (core dumped) ${CAFFE_DIR}/build/tools/extract_features ${trained_model} ${model} ${blob},label ${result_dir}/${subset}_features_lmdb,${result_dir}/${subset}_labels_lmdb ${num_iters} lmdb GPU 0
Traceback (most recent call last):
File "eval/metric_learning.py", line 110, in
main(args)
File "eval/metric_learning.py", line 85, in main
M = _learn_metric(X, Y, args.method)
File "eval/metric_learning.py", line 43, in _learn_metric
M = np.eye(X.shape[1])
IndexError: tuple index out of range

Hi,
I have run the JSTL scripts/exp_jstl.sh for training as the Readme explained. I didnot miss a single step from the top. But the output accuracies are tremendously bad for all the databases. I have attached the results here.

(It had gone through 55,000 iterations)
I0208 15:25:00.855653 31178 solver.cpp:410] Snapshotting to binary proto file external_iter_55000.caffemodel
I0208 15:25:01.026494 31178 solver.cpp:705] Snapshotting solver state to binary proto ts/jstl/jstl_iter_55000.solverstate
I0208 15:25:01.125866 31178 solver.cpp:296] Iteration 55000, loss = 6.55498
I0208 15:25:01.138092 31178 solver.cpp:316] Iteration 55000, Testing net (#0)
I0208 15:25:09.126555 31178 solver.cpp:373] Test net output #0: accuracy = 0.01851
I0208 15:25:09.126617 31178 solver.cpp:373] Test net output #1: loss = 6.57055 (*
I0208 15:25:09.126626 31178 solver.cpp:301] Optimization Done.
I0208 15:25:09.126631 31178 caffe.cpp:191] Optimization Done.

After extraction over, the result is

cuhk03
top-1 7.2%
top-5 36.7%
top-10 73.5%
top-20 85.9%

cuhk01
top-1 2.1%
top-5 10.3%
top-10 20.6%
top-20 41.2%

prid
top-1 1.0%
top-5 5.0%
top-10 10.0%
top-20 20.0%

viper
top-1 1.3%
top-5 6.3%
top-10 12.7%
top-20 24.1%

3dpes
top-1 2.0%
top-5 10.2%
top-10 21.0%
top-20 42.4%

ilids
top-1 1.8%
top-5 9.4%
top-10 19.0%
top-20 37.6%

.
May I know, did I miss any thing or do I need to change anything to get back the JSTL training accuracy as per paper table-3 output by scratch training?

And the loss is not also decreasing as in the archived logs

Thanks in advance

### montego7878@montego7878-All-Series:~/下載/dgd_person_reid/scripts$ ./exp_individually.sh prid

[libprotobuf ERROR google/protobuf/text_format.cc:245] Error parsing text-format caffe.SolverParameter: 16:7: Message type "caffe.SolverParameter" has no field named "min_lr".
[libprotobuf ERROR google/protobuf/text_format.cc:245] Error parsing text-format caffe.SolverParameter: 16:7: Message type "caffe.SolverParameter" has no field named "min_lr".
F1208 19:46:38.284919 17113 upgrade_proto.cpp:1095] Check failed: ReadProtoFromTextFile(param_file, param) Failed to parse SolverParameter file: models/individually/prid_solver.prototxt
F1208 19:46:38.284919 17114 upgrade_proto.cpp:1095] Check failed: ReadProtoFromTextFile(param_file, param) Failed to parse SolverParameter file: models/individually/prid_solver.prototxt
*** Check failure stack trace: ***
*** Check failure stack trace: ***
@ 0x7fbc2053edaa (unknown)
@ 0x7fc90f42ddaa (unknown)
@ 0x7fbc2053ece4 (unknown)
@ 0x7fc90f42dce4 (unknown)
@ 0x7fbc2053e6e6 (unknown)
@ 0x7fc90f42d6e6 (unknown)
@ 0x7fbc20541687 (unknown)
@ 0x7fc90f430687 (unknown)
@ 0x7fbc20cbcf3e caffe::ReadSolverParamsFromTextFileOrDie()
@ 0x407f14 train()
@ 0x405b3c main
@ 0x7fc90fbabf3e caffe::ReadSolverParamsFromTextFileOrDie()
@ 0x407f14 train()
@ 0x405b3c main
@ 0x7fbc1f54af45 (unknown)
@ 0x4063ab (unknown)
@ 0x7fc90e439f45 (unknown)
@ 0x4063ab (unknown)
@ (nil) (unknown)
@ (nil) (unknown)

mpirun noticed that process rank 0 with PID 17113 on node montego7878-All-Series exited on signal 6 (Aborted).

2 total processes killed (some possibly by mpirun during cleanup)
Extracting train set
E1208 19:46:38.383947 17138 extract_features.cpp:52] Using GPU
E1208 19:46:38.384215 17138 extract_features.cpp:58] Using Device_id=0
[libprotobuf ERROR google/protobuf/text_format.cc:245] Error parsing text-format caffe.NetParameter: 7:46: Message type "caffe.TransformationParameter" has no field named "crop_height".
F1208 19:46:38.587419 17138 upgrade_proto.cpp:88] Check failed: ReadProtoFromTextFile(param_file, param) Failed to parse NetParameter file: /tmp/tmp.S4fBxURPtd
*** Check failure stack trace: ***
@ 0x7fa7e766cdaa (unknown)
@ 0x7fa7e766cce4 (unknown)
@ 0x7fa7e766c6e6 (unknown)
@ 0x7fa7e766f687 (unknown)
@ 0x7fa7e7b8698e caffe::ReadNetParamsFromTextFileOrDie()
@ 0x7fa7e7b6e61f caffe::Net<>::Net()
@ 0x408cf3 feature_extraction_pipeline<>()
@ 0x7fa7e6898f45 (unknown)
@ 0x4048be (unknown)
@ (nil) (unknown)
scripts/routines.sh: line 93: 17138 已經終止 (core dumped) ${CAFFE_DIR}/build/tools/extract_features ${trained_model} ${model} ${blob},label ${result_dir}/${subset}_features_lmdb,${result_dir}/${subset}_labels_lmdb ${num_iters} lmdb GPU 0
Traceback (most recent call last):
File "tools/convert_lmdb_to_numpy.py", line 2, in
import lmdb
ImportError: No module named lmdb
Traceback (most recent call last):
File "tools/convert_lmdb_to_numpy.py", line 2, in
import lmdb
ImportError: No module named lmdb
Extracting val set
E1208 19:46:38.741158 17152 extract_features.cpp:52] Using GPU
E1208 19:46:38.741426 17152 extract_features.cpp:58] Using Device_id=0
[libprotobuf ERROR google/protobuf/text_format.cc:245] Error parsing text-format caffe.NetParameter: 7:46: Message type "caffe.TransformationParameter" has no field named "crop_height".
F1208 19:46:38.930703 17152 upgrade_proto.cpp:88] Check failed: ReadProtoFromTextFile(param_file, param) Failed to parse NetParameter file: /tmp/tmp.KOZuJPVKch
*** Check failure stack trace: ***
@ 0x7fdb7dc05daa (unknown)
@ 0x7fdb7dc05ce4 (unknown)
@ 0x7fdb7dc056e6 (unknown)
@ 0x7fdb7dc08687 (unknown)
@ 0x7fdb7e11f98e caffe::ReadNetParamsFromTextFileOrDie()
@ 0x7fdb7e10761f caffe::Net<>::Net()
@ 0x408cf3 feature_extraction_pipeline<>()
@ 0x7fdb7ce31f45 (unknown)
@ 0x4048be (unknown)
@ (nil) (unknown)
scripts/routines.sh: line 93: 17152 已經終止 (core dumped) ${CAFFE_DIR}/build/tools/extract_features ${trained_model} ${model} ${blob},label ${result_dir}/${subset}_features_lmdb,${result_dir}/${subset}_labels_lmdb ${num_iters} lmdb GPU 0
Traceback (most recent call last):
File "tools/convert_lmdb_to_numpy.py", line 2, in
import lmdb
ImportError: No module named lmdb
Traceback (most recent call last):
File "tools/convert_lmdb_to_numpy.py", line 2, in
import lmdb
ImportError: No module named lmdb
Extracting test_probe set
E1208 19:46:39.085443 17166 extract_features.cpp:52] Using GPU
E1208 19:46:39.085688 17166 extract_features.cpp:58] Using Device_id=0
[libprotobuf ERROR google/protobuf/text_format.cc:245] Error parsing text-format caffe.NetParameter: 7:46: Message type "caffe.TransformationParameter" has no field named "crop_height".
F1208 19:46:39.277931 17166 upgrade_proto.cpp:88] Check failed: ReadProtoFromTextFile(param_file, param) Failed to parse NetParameter file: /tmp/tmp.MnmAclhImz
*** Check failure stack trace: ***
@ 0x7f5029da6daa (unknown)
@ 0x7f5029da6ce4 (unknown)
@ 0x7f5029da66e6 (unknown)
@ 0x7f5029da9687 (unknown)
@ 0x7f502a2c098e caffe::ReadNetParamsFromTextFileOrDie()
@ 0x7f502a2a861f caffe::Net<>::Net()
@ 0x408cf3 feature_extraction_pipeline<>()
@ 0x7f5028fd2f45 (unknown)
@ 0x4048be (unknown)
@ (nil) (unknown)
scripts/routines.sh: line 93: 17166 已經終止 (core dumped) ${CAFFE_DIR}/build/tools/extract_features ${trained_model} ${model} ${blob},label ${result_dir}/${subset}_features_lmdb,${result_dir}/${subset}_labels_lmdb ${num_iters} lmdb GPU 0
Traceback (most recent call last):
File "tools/convert_lmdb_to_numpy.py", line 2, in
import lmdb
ImportError: No module named lmdb
Traceback (most recent call last):
File "tools/convert_lmdb_to_numpy.py", line 2, in
import lmdb
ImportError: No module named lmdb
Extracting test_gallery set
E1208 19:46:39.435348 17181 extract_features.cpp:52] Using GPU
E1208 19:46:39.435585 17181 extract_features.cpp:58] Using Device_id=0
[libprotobuf ERROR google/protobuf/text_format.cc:245] Error parsing text-format caffe.NetParameter: 7:46: Message type "caffe.TransformationParameter" has no field named "crop_height".
F1208 19:46:39.623297 17181 upgrade_proto.cpp:88] Check failed: ReadProtoFromTextFile(param_file, param) Failed to parse NetParameter file: /tmp/tmp.oZi1WOYwfm
*** Check failure stack trace: ***
@ 0x7fac5ef6edaa (unknown)
@ 0x7fac5ef6ece4 (unknown)
@ 0x7fac5ef6e6e6 (unknown)
@ 0x7fac5ef71687 (unknown)
@ 0x7fac5f48898e caffe::ReadNetParamsFromTextFileOrDie()
@ 0x7fac5f47061f caffe::Net<>::Net()
@ 0x408cf3 feature_extraction_pipeline<>()
@ 0x7fac5e19af45 (unknown)
@ 0x4048be (unknown)
@ (nil) (unknown)
scripts/routines.sh: line 93: 17181 已經終止 (core dumped) ${CAFFE_DIR}/build/tools/extract_features ${trained_model} ${model} ${blob},label ${result_dir}/${subset}_features_lmdb,${result_dir}/${subset}_labels_lmdb ${num_iters} lmdb GPU 0
Traceback (most recent call last):
File "tools/convert_lmdb_to_numpy.py", line 2, in
import lmdb
ImportError: No module named lmdb
Traceback (most recent call last):
File "tools/convert_lmdb_to_numpy.py", line 2, in
import lmdb
ImportError: No module named lmdb
Traceback (most recent call last):
File "eval/metric_learning.py", line 104, in
main(args)
File "eval/metric_learning.py", line 73, in main
X, Y = _get_train_data(args.result_dir)
File "eval/metric_learning.py", line 11, in get_train_data
features = np.r[np.load(osp.join(result_dir, 'train_features.npy')),
File "/home/montego7878/anaconda2/lib/python2.7/site-packages/numpy/lib/npyio.py", line 362, in load
fid = open(file, "rb")
IOError: [Errno 2] No such file or directory: 'external/exp/results/individually/prid_prid_iter_11000_fc7_bn/train_features.npy'
Extracting train set
E1208 19:46:39.992875 17211 extract_features.cpp:52] Using GPU
E1208 19:46:39.993129 17211 extract_features.cpp:58] Using Device_id=0
[libprotobuf ERROR google/protobuf/text_format.cc:245] Error parsing text-format caffe.NetParameter: 7:46: Message type "caffe.TransformationParameter" has no field named "crop_height".
F1208 19:46:40.181699 17211 upgrade_proto.cpp:88] Check failed: ReadProtoFromTextFile(param_file, param) Failed to parse NetParameter file: /tmp/tmp.6rvyXNylk5
*** Check failure stack trace: ***
@ 0x7f2c5fdb0daa (unknown)
@ 0x7f2c5fdb0ce4 (unknown)
@ 0x7f2c5fdb06e6 (unknown)
@ 0x7f2c5fdb3687 (unknown)
@ 0x7f2c602ca98e caffe::ReadNetParamsFromTextFileOrDie()
@ 0x7f2c602b261f caffe::Net<>::Net()
@ 0x408cf3 feature_extraction_pipeline<>()
@ 0x7f2c5efdcf45 (unknown)
@ 0x4048be (unknown)
@ (nil) (unknown)
scripts/routines.sh: line 93: 17211 已經終止 (core dumped) ${CAFFE_DIR}/build/tools/extract_features ${trained_model} ${model} ${blob},label ${result_dir}/${subset}_features_lmdb,${result_dir}/${subset}_labels_lmdb ${num_iters} lmdb GPU 0
Traceback (most recent call last):
File "tools/convert_lmdb_to_numpy.py", line 2, in
import lmdb
ImportError: No module named lmdb
Traceback (most recent call last):
File "tools/convert_lmdb_to_numpy.py", line 2, in
import lmdb
ImportError: No module named lmdb
Extracting val set
E1208 19:46:40.338579 17225 extract_features.cpp:52] Using GPU
E1208 19:46:40.338822 17225 extract_features.cpp:58] Using Device_id=0
[libprotobuf ERROR google/protobuf/text_format.cc:245] Error parsing text-format caffe.NetParameter: 7:46: Message type "caffe.TransformationParameter" has no field named "crop_height".
F1208 19:46:40.527302 17225 upgrade_proto.cpp:88] Check failed: ReadProtoFromTextFile(param_file, param) Failed to parse NetParameter file: /tmp/tmp.2fKuobB8uG
*** Check failure stack trace: ***
@ 0x7f2b7aa3cdaa (unknown)
@ 0x7f2b7aa3cce4 (unknown)
@ 0x7f2b7aa3c6e6 (unknown)
@ 0x7f2b7aa3f687 (unknown)
@ 0x7f2b7af5698e caffe::ReadNetParamsFromTextFileOrDie()
@ 0x7f2b7af3e61f caffe::Net<>::Net()
@ 0x408cf3 feature_extraction_pipeline<>()
@ 0x7f2b79c68f45 (unknown)
@ 0x4048be (unknown)
@ (nil) (unknown)
scripts/routines.sh: line 93: 17225 已經終止 (core dumped) ${CAFFE_DIR}/build/tools/extract_features ${trained_model} ${model} ${blob},label ${result_dir}/${subset}_features_lmdb,${result_dir}/${subset}_labels_lmdb ${num_iters} lmdb GPU 0
Traceback (most recent call last):
File "tools/convert_lmdb_to_numpy.py", line 2, in
import lmdb
ImportError: No module named lmdb
Traceback (most recent call last):
File "tools/convert_lmdb_to_numpy.py", line 2, in
import lmdb
ImportError: No module named lmdb
Extracting test_probe set
E1208 19:46:40.683516 17239 extract_features.cpp:52] Using GPU
E1208 19:46:40.683779 17239 extract_features.cpp:58] Using Device_id=0
[libprotobuf ERROR google/protobuf/text_format.cc:245] Error parsing text-format caffe.NetParameter: 7:46: Message type "caffe.TransformationParameter" has no field named "crop_height".
F1208 19:46:40.879796 17239 upgrade_proto.cpp:88] Check failed: ReadProtoFromTextFile(param_file, param) Failed to parse NetParameter file: /tmp/tmp.7Yctcxn0J1
*** Check failure stack trace: ***
@ 0x7f19b94e7daa (unknown)
@ 0x7f19b94e7ce4 (unknown)
@ 0x7f19b94e76e6 (unknown)
@ 0x7f19b94ea687 (unknown)
@ 0x7f19b9a0198e caffe::ReadNetParamsFromTextFileOrDie()
@ 0x7f19b99e961f caffe::Net<>::Net()
@ 0x408cf3 feature_extraction_pipeline<>()
@ 0x7f19b8713f45 (unknown)
@ 0x4048be (unknown)
@ (nil) (unknown)
scripts/routines.sh: line 93: 17239 已經終止 (core dumped) ${CAFFE_DIR}/build/tools/extract_features ${trained_model} ${model} ${blob},label ${result_dir}/${subset}_features_lmdb,${result_dir}/${subset}_labels_lmdb ${num_iters} lmdb GPU 0
Traceback (most recent call last):
File "tools/convert_lmdb_to_numpy.py", line 2, in
import lmdb
ImportError: No module named lmdb
Traceback (most recent call last):
File "tools/convert_lmdb_to_numpy.py", line 2, in
import lmdb
ImportError: No module named lmdb
Extracting test_gallery set
E1208 19:46:41.036201 17253 extract_features.cpp:52] Using GPU
E1208 19:46:41.036490 17253 extract_features.cpp:58] Using Device_id=0
[libprotobuf ERROR google/protobuf/text_format.cc:245] Error parsing text-format caffe.NetParameter: 7:46: Message type "caffe.TransformationParameter" has no field named "crop_height".
F1208 19:46:41.226066 17253 upgrade_proto.cpp:88] Check failed: ReadProtoFromTextFile(param_file, param) Failed to parse NetParameter file: /tmp/tmp.yAlDCC7flD
*** Check failure stack trace: ***
@ 0x7f4ea9623daa (unknown)
@ 0x7f4ea9623ce4 (unknown)
@ 0x7f4ea96236e6 (unknown)
@ 0x7f4ea9626687 (unknown)
@ 0x7f4ea9b3d98e caffe::ReadNetParamsFromTextFileOrDie()
@ 0x7f4ea9b2561f caffe::Net<>::Net()
@ 0x408cf3 feature_extraction_pipeline<>()
@ 0x7f4ea884ff45 (unknown)
@ 0x4048be (unknown)
@ (nil) (unknown)
scripts/routines.sh: line 93: 17253 已經終止 (core dumped) ${CAFFE_DIR}/build/tools/extract_features ${trained_model} ${model} ${blob},label ${result_dir}/${subset}_features_lmdb,${result_dir}/${subset}_labels_lmdb ${num_iters} lmdb GPU 0
Traceback (most recent call last):
File "tools/convert_lmdb_to_numpy.py", line 2, in
import lmdb
ImportError: No module named lmdb
Traceback (most recent call last):
File "tools/convert_lmdb_to_numpy.py", line 2, in
import lmdb
ImportError: No module named lmdb
Traceback (most recent call last):
File "eval/metric_learning.py", line 104, in
main(args)
File "eval/metric_learning.py", line 73, in main
X, Y = _get_train_data(args.result_dir)
File "eval/metric_learning.py", line 11, in get_train_data
features = np.r[np.load(osp.join(result_dir, 'train_features.npy')),
File "/home/montego7878/anaconda2/lib/python2.7/site-packages/numpy/lib/npyio.py", line 362, in load
fid = open(file, "rb")
IOError: [Errno 2] No such file or directory: 'external/exp/results/individually/prid_prid_iter_11000_fc7_bn/train_features.npy'

Hi Cysu,
I'm trying to reproduce your paper with keras because I am not familiar with caffe. My question is: Why the first dimension of the output size of inception(4a) is 256? It seems the number should be (32 + 32 +32 + 32) = 128. I don't understand what the the word "double" in "double #3x3 reduce" means in the paper. Sorry for the naive question but...Can you help me?

I can train and test all training script on the dataset(except Shinpuhkan dataset), but when it comes to compute cmc(cumulative match characteristic), when running eval/metric_learning.py, it gives the following error:

sklearn.externals.joblib.externals.loky.process_executor.TerminatedWorkerError: A worker process managed by the executor was unexpectedly terminated. This could be caused by a segmentation fault while calling the function or by an excessive memory usage causing the Operating System to kill the worker. The exit codes of the workers are {SIGSEGV(-11)}

at line
C =_eval_cmc(PX,PY,GX,GY,M)
and
D = pairwise_distance(GX, PX, metric='mahalanobis', VI=M, n_jobs=-1)

in cuhk01, cuhk03 and prid dataset.

I get correct result (nearly the same with paper) on viper, 3dpes, and ilids.
Which means the _eval_cmc method and scipt.pairwise_distance are working just fine.

I do some search on stackoverflow and scipy doc, even loky, someone suggest this answer

And someone said it is a joblib backend bug using uwsgi, but I can produce the result on last 3 dataset so I guess it's not the problem.

Anyone can run this metric_learning.py without error can give me some idea?

Is it really memory problem.
The input GX and PX shape which induce error on pairwise_distances are

cuhk03
(978, 256) (956, 256)

cuhk01
(972, 256) (972, 256)

prid
(649, 256) (100, 256)

The following works just fine and gives correct top-1, top-5, top-10 and top-20 score.

viper
(308, 256) (316, 256)

3dpes
(308, 256) (228, 256)

ilids
(142, 256) (118, 256)

I'm running this test on GTX 2080, 8g DDR4 ram , i7 7700. Is this really relate to excessive memory usage?

Thank you for the great work.

I have a question regarding the MPI gathering and boardcasting process. It seems that the current code gather all the data whenever it does inner product. Yet, I would expect that the gathering to happen during batch norm or the loss layer only. Can you please point me to where I can modify the gathering code?

Thanks,

format-rawdata.sh时，运行format_数据集.py 的时候，提示很多函数未定义，并且script文件夹下的init.py文件为空。是不是少上传错了呢，还是我哪里弄错了

你好，我想請問用兩張GPU並行訓練的用意是可以增強訓練還是只是為了增加訓練速度？
因為我用單張GPU(GTX Titan X)且沒動任何參數，跑出的結果cuhk03 top-1 高於paper的數值，我覺得不太合理，想請問你有什麼看法，謝謝！

Hi, I'm trying to build caffe with openmpi flags on, getting error messages as follows.
How can I resolve this?

.build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue<double>::singleton_' .build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue::MPIJobQueue()'
.build_release/lib/libcaffe.so: undefined reference to caffe::Net<float>::SyncDiff()' .build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue::Push(caffe::MPIJobQueue::Job const&)'
.build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue<double>::~MPIJobQueue()' .build_release/lib/libcaffe.so: undefined reference to caffe::Net::SyncDiff()'
.build_release/lib/libcaffe.so: undefined reference to caffe::Net<double>::SyncData()' .build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue::WaitAll()'
.build_release/lib/libcaffe.so: undefined reference to caffe::Net<double>::SyncOutput()' .build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue::WaitAll()'
.build_release/lib/libcaffe.so: undefined reference to caffe::Net<float>::SyncData()' .build_release/lib/libcaffe.so: undefined reference to caffe::Net::SyncOutput()'
.build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue<float>::Push(caffe::MPIJobQueue<float>::Job const&)' .build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue::singleton_'
.build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue<double>::MPIJobQueue()' .build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue::~MPIJobQueue()'
.build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue<double>::singleton_' .build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue::MPIJobQueue()'
.build_release/lib/libcaffe.so: undefined reference to caffe::Net<float>::SyncDiff()' .build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue::Push(caffe::MPIJobQueue::Job const&)'
.build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue<double>::~MPIJobQueue()' .build_release/lib/libcaffe.so: undefined reference to caffe::Net::SyncDiff()'
.build_release/lib/libcaffe.so: undefined reference to caffe::Net<double>::SyncData()' .build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue::WaitAll()'
.build_release/lib/libcaffe.so: undefined reference to caffe::Net<double>::SyncOutput()' .build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue::WaitAll()'
.build_release/lib/libcaffe.so: undefined reference to caffe::Net<float>::SyncData()' .build_release/lib/libcaffe.so: undefined reference to caffe::Net::SyncOutput()'
.build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue<float>::Push(caffe::MPIJobQueue<float>::Job const&)' .build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue::singleton_'
.build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue<double>::MPIJobQueue()' .build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue::~MPIJobQueue()'
.build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue<double>::singleton_' .build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue::MPIJobQueue()'
.build_release/lib/libcaffe.so: undefined reference to caffe::Net<float>::SyncDiff()' .build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue::Push(caffe::MPIJobQueue::Job const&)'
.build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue<double>::~MPIJobQueue()' .build_release/lib/libcaffe.so: undefined reference to caffe::Net::SyncDiff()'
.build_release/lib/libcaffe.so: undefined reference to caffe::Net<double>::SyncData()' .build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue::WaitAll()'
.build_release/lib/libcaffe.so: undefined reference to caffe::Net<double>::SyncOutput()' .build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue::WaitAll()'
.build_release/lib/libcaffe.so: undefined reference to caffe::Net<float>::SyncData()' .build_release/lib/libcaffe.so: undefined reference to caffe::Net::SyncOutput()'
.build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue<float>::Push(caffe::MPIJobQueue<float>::Job const&)' .build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue::singleton_'
.build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue<double>::MPIJobQueue()' .build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue::~MPIJobQueue()'
.build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue<double>::singleton_' .build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue::MPIJobQueue()'
.build_release/lib/libcaffe.so: undefined reference to caffe::Net<float>::SyncDiff()' .build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue::Push(caffe::MPIJobQueue::Job const&)'
.build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue<double>::~MPIJobQueue()' .build_release/lib/libcaffe.so: undefined reference to caffe::Net::SyncDiff()'
.build_release/lib/libcaffe.so: undefined reference to caffe::Net<double>::SyncData()' .build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue::WaitAll()'
.build_release/lib/libcaffe.so: undefined reference to caffe::Net<double>::SyncOutput()' .build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue::WaitAll()'
.build_release/lib/libcaffe.so: undefined reference to caffe::Net<float>::SyncData()' .build_release/lib/libcaffe.so: undefined reference to caffe::Net::SyncOutput()'
.build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue<float>::Push(caffe::MPIJobQueue<float>::Job const&)' .build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue::singleton_'
.build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue<double>::MPIJobQueue()' .build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue::~MPIJobQueue()'
.build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue<double>::singleton_' .build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue::MPIJobQueue()'
.build_release/lib/libcaffe.so: undefined reference to caffe::Net<float>::SyncDiff()' .build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue::Push(caffe::MPIJobQueue::Job const&)'
.build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue<double>::~MPIJobQueue()' .build_release/lib/libcaffe.so: undefined reference to caffe::Net::SyncDiff()'
.build_release/lib/libcaffe.so: undefined reference to caffe::Net<double>::SyncData()' .build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue::WaitAll()'
.build_release/lib/libcaffe.so: undefined reference to caffe::Net<double>::SyncOutput()' .build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue::WaitAll()'
.build_release/lib/libcaffe.so: undefined reference to caffe::Net<float>::SyncData()' .build_release/lib/libcaffe.so: undefined reference to caffe::Net::SyncOutput()'
.build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue<float>::Push(caffe::MPIJobQueue<float>::Job const&)' .build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue::singleton_'
.build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue<double>::MPIJobQueue()' .build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue::~MPIJobQueue()'
.build_release/tools/caffe.o: In function test()': caffe.cpp:(.text+0xaf6): undefined reference to caffe::Net::SyncOutput()'
caffe.cpp:(.text+0xb33): undefined reference to caffe::MPIJobQueue<float>::Push(caffe::MPIJobQueue<float>::Job const&)' caffe.cpp:(.text+0xb40): undefined reference to caffe::MPIJobQueue::WaitAll()'
.build_release/tools/caffe.o: In function boost::detail::sp_counted_impl_p<caffe::MPIJobQueue<float> >::dispose()': caffe.cpp:(.text._ZN5boost6detail17sp_counted_impl_pIN5caffe11MPIJobQueueIfEEE7disposeEv[_ZN5boost6detail17sp_counted_impl_pIN5caffe11MPIJobQueueIfEEE7disposeEv]+0xe): undefined reference to caffe::MPIJobQueue::~MPIJobQueue()'
.build_release/tools/caffe.o: In function caffe::MPIJobQueue<float>::Get()': caffe.cpp:(.text._ZN5caffe11MPIJobQueueIfE3GetEv[_ZN5caffe11MPIJobQueueIfE3GetEv]+0x9): undefined reference to caffe::MPIJobQueue::singleton_'
caffe.cpp:(.text.ZN5caffe11MPIJobQueueIfE3GetEv[ZN5caffe11MPIJobQueueIfE3GetEv]+0x31): undefined reference to caffe::MPIJobQueue<float>::MPIJobQueue()' caffe.cpp:(.text._ZN5caffe11MPIJobQueueIfE3GetEv[_ZN5caffe11MPIJobQueueIfE3GetEv]+0xa0): undefined reference to caffe::MPIJobQueue::~MPIJobQueue()'
.build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue<double>::singleton_' .build_release/lib/libcaffe.so: undefined reference to caffe::Net::SyncDiff()'
.build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue<double>::Push(caffe::MPIJobQueue<double>::Job const&)' .build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue::~MPIJobQueue()'
.build_release/lib/libcaffe.so: undefined reference to caffe::Net<double>::SyncDiff()' .build_release/lib/libcaffe.so: undefined reference to caffe::Net::SyncData()'
.build_release/lib/libcaffe.so: undefined reference to caffe::Net<double>::SyncOutput()' .build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue::WaitAll()'
.build_release/lib/libcaffe.so: undefined reference to caffe::Net<float>::SyncData()' .build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue::MPIJobQueue()'
.build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue<double>::singleton_' .build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue::MPIJobQueue()'
.build_release/lib/libcaffe.so: undefined reference to caffe::Net<float>::SyncDiff()' .build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue::Push(caffe::MPIJobQueue::Job const&)'
.build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue<double>::~MPIJobQueue()' .build_release/lib/libcaffe.so: undefined reference to caffe::Net::SyncDiff()'
.build_release/lib/libcaffe.so: undefined reference to caffe::Net<double>::SyncData()' .build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue::WaitAll()'
.build_release/lib/libcaffe.so: undefined reference to caffe::Net<double>::SyncOutput()' .build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue::WaitAll()'
.build_release/lib/libcaffe.so: undefined reference to caffe::Net<float>::SyncData()' .build_release/lib/libcaffe.so: undefined reference to caffe::Net::SyncOutput()'
.build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue<float>::Push(caffe::MPIJobQueue<float>::Job const&)' .build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue::singleton'
.build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue<double>::MPIJobQueue()' .build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue::~MPIJobQueue()'
.build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue<double>::singleton_' .build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue::MPIJobQueue()'
.build_release/lib/libcaffe.so: undefined reference to caffe::Net<float>::SyncDiff()' .build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue::Push(caffe::MPIJobQueue::Job const&)'
.build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue<double>::~MPIJobQueue()' .build_release/lib/libcaffe.so: undefined reference to caffe::Net::SyncDiff()'
.build_release/lib/libcaffe.so: undefined reference to caffe::Net<double>::SyncData()' .build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue::WaitAll()'
.build_release/lib/libcaffe.so: undefined reference to caffe::Net<double>::SyncOutput()' .build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue::WaitAll()'
.build_release/lib/libcaffe.so: undefined reference to caffe::Net<float>::SyncData()' .build_release/lib/libcaffe.so: undefined reference to caffe::Net::SyncOutput()'
.build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue<float>::Push(caffe::MPIJobQueue<float>::Job const&)' .build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue::singleton'
.build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue<double>::MPIJobQueue()' .build_release/lib/libcaffe.so: undefined reference to caffe::MPIJobQueue::~MPIJobQueue()'
collect2: error: ld returned 1 exit status
collect2: error: ld returned 1 exit status
collect2: error: ld returned 1 exit status
make: *** [.build_release/tools/upgrade_net_proto_binary.bin] Error 1
make: *** Waiting for unfinished jobs....
make: *** [.build_release/tools/convert_labelset.bin] Error 1
make: *** [.build_release/tools/extract_features.bin] Error 1
collect2: error: ld returned 1 exit status
collect2: error: ld returned 1 exit status
make: *** [.build_release/tools/upgrade_net_proto_text.bin] Error 1
make: *** [.build_release/tools/convert_bboxset.bin] Error 1
collect2: error: ld returned 1 exit status
collect2: error: ld returned 1 exit status
collect2: error: ld returned 1 exit status
make: *** [.build_release/tools/compute_image_mean.bin] Error 1
make: *** [.build_release/tools/caffe.bin] Error 1
make: *** [.build_release/tools/convert_imageset.bin] Error 1

Traceback (most recent call last):
File "tools/merge_lists_single_task.py", line 48, in
main(args)
File "tools/merge_lists_single_task.py", line 18, in main
train_files, train_labels = read_kv(osp.join(db_dir, 'train.txt'))
File "/home/software_mount/wty/dgd_person_reid/tools/../utils/core.py", line 45, in read_kv
arr = read_list(file_path, coding)
File "/home/software_mount/wty/dgd_person_reid/tools/../utils/core.py", line 26, in read_list
with open(file_path, 'r') as f:

IOError: [Errno 2] No such file or directory: 'external/exp/db/train.txt'

All the previous experiments have passed.
The last one FT-(JSTL+DGD) encountered IndexError, I ran it twice, and in the first time the cuhk03 also show the same IndexError.

Here is the result:

cuhk03
top-1 9.0%
top-5 45.5%
top-10 88.5%
top-20 91.9%

cuhk01
top-1 2.1%
top-5 10.3%
top-10 20.6%
top-20 41.2%

prid
top-1 1.0%
top-5 5.0%
top-10 10.0%
top-20 20.0%

viper
Traceback (most recent call last):
File "eval/metric_learning.py", line 110, in
main(args)
File "eval/metric_learning.py", line 91, in main
M = _learn_metric(X, Y, args.method)
File "eval/metric_learning.py", line 49, in _learn_metric
M = np.eye(X.shape[1])
IndexError: tuple index out of range

3dpes
top-1 2.5%
top-5 12.0%
top-10 23.1%
top-20 47.5%

ilids
top-1 2.1%
top-5 11.5%
top-10 23.2%
top-20 46.2%

最近在學習DGD，從results\individually的輸出去追發現輸出在routines.sh內的extract_features()定義。
其中train, val, test_probe, test_gallery的意義與資料來源，無法理解。
原本猜測是在format_資料庫.py中定義的trainval, test_probe, test_gallery，但數目上對不起來。

I don't understand how to do person re-identification with the pretrained JSTL+DGD model found here: https://drive.google.com/open?id=0B67_d0rLRTQYZnB5ZUZpdTlxM0k

I have two problems, one related to input, one related to output :

1. In person re-identification, we input two different pictures, and we ask the model if they depict the same person or not.

But here, in the file 'jstl_dgd_deploy_inference.prototxt', the input data is (1,3,144,56) and not, for example, (2,3,144,56).

2. In the file 'jstl_dgd_deploy_inference.prototxt', I don't see the output layer, it should be a binary softmax, with output '1' if the two photos represent the same person, and '0' if the persons are different.

Moreover, when loading the caffemodel weights, I receive the warnings:

I1108 08:48:31.324759 1525 net.cpp:752] Ignoring source layer relu7
I1108 08:48:31.324795 1525 net.cpp:752] Ignoring source layer drop7
I1108 08:48:31.324802 1525 net.cpp:752] Ignoring source layer fc8_jstl

This suggests that something is missing in the prototxt file.

hi every body
I wanna run these instructions in caffe and as wrote in *_train.prototxt files ,the value of batch_size for training set to 50 and for test set to 20, I use a GeForce GTX 980 Ti GPU and when I trying to run with these values I ahead with this error
error == cudaSuccess (2 vs. 0) out of memory
and I have to decrease batch_size value and it causes my accuracy change?
how should I manage this problem?
thanks

At first time when I make caffe, everything is good. Today when I try to make it again ( I did make clean before make), it shows following error.

AR -o .build_release/lib/libcaffe.a
LD -o .build_release/lib/libcaffe.so
CXX/LD -o .build_release/tools/device_query.bin
CXX/LD -o .build_release/tools/net_speed_benchmark.bin
CXX/LD -o .build_release/tools/convert_imageset.bin
CXX/LD -o .build_release/tools/convert_bboxset.bin
CXX/LD -o .build_release/tools/train_net.bin
CXX/LD -o .build_release/tools/finetune_net.bin
CXX/LD -o .build_release/tools/compute_image_mean.bin
CXX/LD -o .build_release/tools/extract_features.bin
CXX/LD -o .build_release/tools/caffe.bin
CXX/LD -o .build_release/tools/upgrade_net_proto_binary.bin
CXX/LD -o .build_release/tools/convert_labelset.bin
CXX/LD -o .build_release/tools/upgrade_net_proto_text.bin
.build_release/tools/convert_bboxset.o: In function `void caffe::shuffle<__gnu_cxx::__normal_iterator<LineItem*, std::vector<LineItem, std::allocator<LineItem> > > >(__gnu_cxx::__normal_iterator<LineItem*, std::vector<LineItem, std::allocator<LineItem> > >, __gnu_cxx::__normal_iterator<LineItem*, std::vector<LineItem, std::allocator<LineItem> > >)':
convert_bboxset.cpp:(.text._ZN5caffe7shuffleIN9__gnu_cxx17__normal_iteratorIP8LineItemSt6vectorIS3_SaIS3_EEEEEEvT_S9_[_ZN5caffe7shuffleIN9__gnu_cxx17__normal_iteratorIP8LineItemSt6vectorIS3_SaIS3_EEEEEEvT_S9_]+0x14): undefined reference to `caffe::Caffe::singleton_'
.build_release/tools/convert_bboxset.o: In function `main':
convert_bboxset.cpp:(.text.startup+0xb14): undefined reference to `caffe::ReadImageToDatumResizeShortSide(std::string const&, int, int, int, int, int, int, bool, std::string const&, caffe::Datum*)'
convert_bboxset.cpp:(.text.startup+0xe2e): undefined reference to `caffe::ReadImageToDatum(std::string const&, int, int, int, int, int, int, int, int, int, bool, std::string const&, caffe::Datum*)'
collect2: error: ld returned 1 exit status
make: *** [.build_release/tools/convert_bboxset.bin] Error 1
make: *** Waiting for unfinished jobs....
.build_release/tools/convert_imageset.o: In function `void caffe::shuffle<__gnu_cxx::__normal_iterator<std::pair<std::string, int>*, std::vector<std::pair<std::string, int>, std::allocator<std::pair<std::string, int> > > > >(__gnu_cxx::__normal_iterator<std::pair<std::string, int>*, std::vector<std::pair<std::string, int>, std::allocator<std::pair<std::string, int> > > >, __gnu_cxx::__normal_iterator<std::pair<std::string, int>*, std::vector<std::pair<std::string, int>, std::allocator<std::pair<std::string, int> > > >)':
convert_imageset.cpp:(.text._ZN5caffe7shuffleIN9__gnu_cxx17__normal_iteratorIPSt4pairISsiESt6vectorIS4_SaIS4_EEEEEEvT_SA_[_ZN5caffe7shuffleIN9__gnu_cxx17__normal_iteratorIPSt4pairISsiESt6vectorIS4_SaIS4_EEEEEEvT_SA_]+0x14): undefined reference to `caffe::Caffe::singleton_'
.build_release/tools/convert_imageset.o: In function `main':
convert_imageset.cpp:(.text.startup+0x7ff): undefined reference to `caffe::ReadImageToDatumResizeShortSide(std::string const&, int, int, int, int, int, int, bool, std::string const&, caffe::Datum*)'
convert_imageset.cpp:(.text.startup+0xc98): undefined reference to `caffe::ReadImageToDatum(std::string const&, int, int, int, int, int, int, int, int, int, bool, std::string const&, caffe::Datum*)'
collect2: error: ld returned 1 exit status
make: *** [.build_release/tools/convert_imageset.bin] Error 1
.build_release/tools/extract_features.o: In function `caffe::Caffe::Get()':
extract_features.cpp:(.text._ZN5caffe5Caffe3GetEv[_ZN5caffe5Caffe3GetEv]+0x7): undefined reference to `caffe::Caffe::singleton_'
.build_release/tools/extract_features.o: In function `int feature_extraction_pipeline<float>(int, char**)':
extract_features.cpp:(.text._Z27feature_extraction_pipelineIfEiiPPc[_Z27feature_extraction_pipelineIfEiiPPc]+0xcd): undefined reference to `caffe::Caffe::singleton_'
extract_features.cpp:(.text._Z27feature_extraction_pipelineIfEiiPPc[_Z27feature_extraction_pipelineIfEiiPPc]+0x148): undefined reference to `caffe::Net<float>::Net(std::string const&, caffe::Phase)'
collect2: error: ld returned 1 exit status
make: *** [.build_release/tools/extract_features.bin] Error 1
.build_release/tools/convert_labelset.o: In function `void caffe::shuffle<__gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int> > > >(__gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int> > >, __gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int> > >)':
convert_labelset.cpp:(.text._ZN5caffe7shuffleIN9__gnu_cxx17__normal_iteratorIPiSt6vectorIiSaIiEEEEEEvT_S8_[_ZN5caffe7shuffleIN9__gnu_cxx17__normal_iteratorIPiSt6vectorIiSaIiEEEEEEvT_S8_]+0x17): undefined reference to `caffe::Caffe::singleton_'
collect2: error: ld returned 1 exit status
make: *** [.build_release/tools/convert_labelset.bin] Error 1
.build_release/tools/caffe.o: In function `test()':
caffe.cpp:(.text+0xa6d): undefined reference to `caffe::Net<float>::Net(std::string const&, caffe::Phase)'
.build_release/tools/caffe.o: In function `time()':
caffe.cpp:(.text+0x1b6b): undefined reference to `caffe::Net<float>::Net(std::string const&, caffe::Phase)'
caffe.cpp:(.text+0x1c9e): undefined reference to `caffe::Net<float>::Backward(int)'
.build_release/tools/caffe.o: In function `caffe::Caffe::Get()':
caffe.cpp:(.text._ZN5caffe5Caffe3GetEv[_ZN5caffe5Caffe3GetEv]+0x7): undefined reference to `caffe::Caffe::singleton_'
.build_release/tools/caffe.o: In function `main':
caffe.cpp:(.text.startup+0x80): undefined reference to `caffe::GlobalFinalize()'
caffe.cpp:(.text.startup+0xea): undefined reference to `caffe::GlobalFinalize()'
.build_release/tools/caffe.o: In function `caffe::Solver<float>* caffe::GetSolver<float>(caffe::SolverParameter const&)':
caffe.cpp:(.text._ZN5caffe9GetSolverIfEEPNS_6SolverIT_EERKNS_15SolverParameterE[_ZN5caffe9GetSolverIfEEPNS_6SolverIT_EERKNS_15SolverParameterE]+0x3f): undefined reference to `caffe::Solver<float>::Solver(caffe::SolverParameter const&)'
caffe.cpp:(.text._ZN5caffe9GetSolverIfEEPNS_6SolverIT_EERKNS_15SolverParameterE[_ZN5caffe9GetSolverIfEEPNS_6SolverIT_EERKNS_15SolverParameterE]+0x14f): undefined reference to `caffe::Solver<float>::Solver(caffe::SolverParameter const&)'
caffe.cpp:(.text._ZN5caffe9GetSolverIfEEPNS_6SolverIT_EERKNS_15SolverParameterE[_ZN5caffe9GetSolverIfEEPNS_6SolverIT_EERKNS_15SolverParameterE]+0x1ff): undefined reference to `caffe::Solver<float>::Solver(caffe::SolverParameter const&)'
caffe.cpp:(.text._ZN5caffe9GetSolverIfEEPNS_6SolverIT_EERKNS_15SolverParameterE[_ZN5caffe9GetSolverIfEEPNS_6SolverIT_EERKNS_15SolverParameterE]+0x297): undefined reference to `caffe::Solver<float>::Solver(caffe::SolverParameter const&)'
caffe.cpp:(.text._ZN5caffe9GetSolverIfEEPNS_6SolverIT_EERKNS_15SolverParameterE[_ZN5caffe9GetSolverIfEEPNS_6SolverIT_EERKNS_15SolverParameterE]+0x33f): undefined reference to `caffe::Solver<float>::Solver(caffe::SolverParameter const&)'
collect2: error: ld returned 1 exit status
make: *** [.build_release/tools/caffe.bin] Error 1

I cannot figure out the reason. Could you give me some suggestions? Thanks.

when I run the format-rawdata.py,it comes wrong with the information
Traceback (most recent call last):
File "data/format_cuhk03.py", line 5, in
from utils import *
ImportError: No module named utils
I have already add export PYTHONPATH=".:$PYTHONPATH" to ~/.bashrc,but it still comes wrong.

I am working on a project for human re-identification using Caffe. I have obtained the CUHK01 public dataset which contains 971 identities with 2 images per view. I want to spilt the training set into 971 labels for each identity but i am unsure of the python code for doing it. The images are labelled 0001001.png0001002.png,0001003.png,0001004.png for the first identity and then it repeats for the reamining 970 identities. Is there any suggestion or codes advice for me to allocate the images to the correct labels

cuhk03
top-1 93.8%

cuhk01
top-1 77.5%

prid
top-1 69.0%

viper
top-1 38.3%

3dpes
top-1 53.5%

ilids
top-1 62.4%

hi
I have a question about running this code on system has one gpu,my system has a GeForce GTX 980 Ti GPU and I comment MPI_LIB & MPI_INCLUDE in config file, and as a wrote in expriments part I have to change GPU device id in these files:
train_model() and extract_features() of scripts/rountines.sh
main() of tools/compute_impact_score.py**
how to change Gpu Id in these parts?
I change train_model() method like as this

Training

if [[ $# -eq 2 ]]; then
GLOG_logtostderr=1 ${CAFFE_DIR}/build/tools/caffe train
-solver ${solver} -gpu 0 2>&1 | tee ${log}
else
GLOG_logtostderr=1 ${CAFFE_DIR}/build/tools/caffe train
-solver ${solver} -weights ${pretrained_model} -gpu 0 2>&1 | tee ${log}
but I don't know how to change extract_feature() and compute_impact_score..py parts?
thank you

Hi @Cysu, I just read about your excellent DGD paper and found a problem which made me confused.

In Table 1 of the DGD paperhttps://arxiv.org/pdf/1604.07528.pdf, the strides of conv1 to conv3 are 2.
I am confused why the output sizes are retained.
So I checked your prototxt of the models and find that the conv kernels are 3x3 with stride 1 and pad 1. Is this a typo in your paper?

Hi, I'm doing some research on person re-identification, and I'm now trying to input only one image as test probe using dgd_person_reid.
Take PRID_2011 dataset as example, the code /data/format_prid.py will randomly choose 100 people as test probe, so if I change it to randomly choose 1 person, it will input all the images of that person's tracklet. So I'm guessing if I only want to input single image as test probe, I will probably need to do lots of changing on your code, but I don't really know where to start... Could you give me some suggestions?

And I'm think that if I want to know the top-5 or top-10 ids that are the most similar to the test probe, sorting the distance produce by D = pairwise_distances(GX, PX, metric='mahalanobis', VI=M, n_jobs=-2) is the only way to get the similarity, the shortest distance will be the most similar to the test probe, so I'll need to sort 5 or 10 smallest distances with its id to get my answer, am I correct?

Sorry for disturbing you, I'm looking forward to your reply! Thank you very much!

I only have one GPU ， so I do not want to use openmpi ,if without openmpi ,can I using this project sucessfully ?

你好，下面是运行ｎｖｉｄｉａ－smi　得到的结果，我怎么修改代码，才能使得这些gpu高效工作呢？因为每次进行训练时，总是只有一个gpu在运行，每小时才能迭代１４００次左右？
0 GeForce GTX TIT... Off 　　| 0000:02:00.0 On | N/A |
| 22% 38C P8 15W / 250W | 72MiB / 12286MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
| 1 GeForce GTX TIT... Off 　 | 0000:03:00.0 Off | 　N/A |
| 22% 39C P8 15W / 250W | 23MiB / 12287MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
| 2 GeForce GTX TIT... Off 　| 0000:82:00.0 Off | 　 N/A |
| 22% 37C P8 15W / 250W | 23MiB / 12287MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
| 3 GeForce GTX TIT... Off 　 | 0000:83:00.0 Off | 　 N/A |
| 22% 38C P8 14W / 250W | 23MiB / 12287MiB | 0% Default

Really grateful for your pretrained model for person re-identification. However the costumed BN layer somehow makes it hard to import the model into tensorflow. Is it possible to develop a tensorflow version? Thanks very much.

hi, One question is that where is the guideddropout layer??
and shouldn't it in caffe folder?

can we input the surveillance video directly? or we can only input the processed image?