mahyarnajibi / sniper Goto Github PK

Hi, I see that in openimages2 branch, you train for 14 epochs. What are the results?

Hi,I am training my model in openimages dataset. I see in your openimage config file that you used a pretrained model called CRCNN-1epoch.But I download your pretrain model with bash scripts/download_pretrained_model.sh ,but there is no file named CRCNN-0001.Can you tell me where is this file to download.

loading annotations into memory... Done (t=0.11s) creating index... index created! num_images 8100 COCO_train2014 gt roidb loaded from ./data/cache/COCO_train2014_gt_roidb.pkl append flipped images to roidb loading annotations into memory... Done (t=0.01s) creating index... index created! num_images 900 COCO_val2014 gt roidb loaded from ./data/cache/COCO_val2014_gt_roidb.pkl append flipped images to roidb filtered 0 roidb entries: 18000 -> 18000 add bounding box regression targets bbox target means: [[0. 0. 0. 0.] [0. 0. 0. 0.]] [0. 0. 0. 0.] bbox target stdevs: [[0.1 0.1 0.2 0.2] [0.1 0.1 0.2 0.2]] [0.1 0.1 0.2 0.2] Creating Iterator with 18000 Images Total number of extracted chips: 30025 Done! Traceback (most recent call last): File "main_train.py", line 78, in <module> threads=config.TRAIN.NUM_THREAD, pad_rois_to=400) File "lib/iterators/MNIteratorE2E.py", line 32, in __init__ super(MNIteratorE2E, self).__init__(roidb, config, batch_size, threads, nGPUs, pad_rois_to, False) File "lib/iterators/MNIteratorBase.py", line 33, in __init__ self.get_batch() File "lib/iterators/MNIteratorE2E.py", line 100, in get_batch self.batch = self._get_batch() File "lib/iterators/MNIteratorE2E.py", line 195, in _get_batch processed_list = processed_list.get() File "/usr/lib/python2.7/multiprocessing/pool.py", line 567, in get raise self._value TypeError: 'NoneType' object has no attribute '__getitem__'

could you help me to fix it?

Hi, I am trying to clone the openimage branch by using command:

git clone -b openimages2 --recursive https://github.com/mahyarnajibi/SNIPER.git

and I got a fatal error stating that I cannot read from SNIPER-mxnet remote module. The complete error message is as follow:

Cloning into 'SNIPER'...
remote: Counting objects: 1663, done.
remote: Compressing objects: 100% (89/89), done.
remote: Total 1663 (delta 45), reused 96 (delta 34), pack-reused 1540
Receiving objects: 100% (1663/1663), 22.00 MiB | 9.45 MiB/s, done.
Resolving deltas: 100% (1009/1009), done.
Checking connectivity... done.
Submodule 'SNIPER-mxnet' ([email protected]:mahyarnajibi/SNIPER-mxnet.git) registered for path 'SNIPER-mxnet'
Cloning into 'SNIPER-mxnet'...
The authenticity of host 'github.com (192.30.255.112)' can't be established.
RSA key fingerprint is SHA256:nThbg********************************6E5SY8.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'github.com,192.30.255.112' (RSA) to the list of known hosts.
Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
fatal: clone of '[email protected]:mahyarnajibi/SNIPER-mxnet.git' into submodule path 'SNIPER-mxnet' failed

The first line of the README contains one typo: "instance-level segmentataion" instead of "segmentation"

/usr/local/lib/python2.7/dist-packages/h5py/init.py:36: FutureWarning: Conversion of the second argument of issubdtype from float to np.floating is deprecated. In future, it will be treated as np.float64 == np.dtype(float).type.
from ._conv import register_converters as _register_converters
Traceback (most recent call last):
File "demo.py", line 123, in
main()
File "demo.py", line 87, in main
convert=True, process=True)
File "lib/train_utils/utils.py", line 90, in load_param
arg_params, aux_params = load_checkpoint(prefix, epoch)
File "lib/train_utils/utils.py", line 56, in load_checkpoint
save_dict = mx.nd.load('%s-%04d.params' % (prefix, epoch))
File "/home/czn/scc/SNIPER/SNIPER-mxnet/python/mxnet/ndarray/utils.py", line 175, in load
ctypes.byref(names)))
File "/home/czn/scc/SNIPER/SNIPER-mxnet/python/mxnet/base.py", line 149, in check_call
raise MXNetError(py_str(_LIB.MXGetLastError()))
mxnet.base.MXNetError: [04:49:05] /home/czn/scc/SNIPER/SNIPER-mxnet/src/ndarray/ndarray.cc:1770: Check failed: fi->Read(data) Invalid NDArray file format

Stack trace returned 10 entries:
[bt] (0) /home/czn/scc/SNIPER/SNIPER-mxnet/python/mxnet/../../build/libmxnet.so(dmlc::StackTraceabi:cxx11+0x56) [0x7ff817364586]
[bt] (1) /home/czn/scc/SNIPER/SNIPER-mxnet/python/mxnet/../../build/libmxnet.so(dmlc::LogMessageFatal::~LogMessageFatal()+0x28) [0x7ff817364d38]
[bt] (2) /home/czn/scc/SNIPER/SNIPER-mxnet/python/mxnet/../../build/libmxnet.so(mxnet::NDArray::Load(dmlc::Stream*, std::vector<mxnet::NDArray, std::allocatormxnet::NDArray >, std::vector<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::allocator<std::__cxx11::basic_string<char, std::char_traits, std::allocator > > >)+0x204) [0x7ff817586e44]
[bt] (3) /home/czn/scc/SNIPER/SNIPER-mxnet/python/mxnet/../../build/libmxnet.so(MXNDArrayLoad+0xc1) [0x7ff81754c471]
[bt] (4) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7ff82fa2ee40]
[bt] (5) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7ff82fa2e8ab]
[bt] (6) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7ff82fc3e3df]
[bt] (7) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(+0x11d82) [0x7ff82fc42d82]
[bt] (8) python(PyEval_EvalFrameEx+0x578f) [0x4c15bf]
[bt] (9) python(PyEval_EvalCodeEx+0x306) [0x4b9ab6]

when FLIP is set to true, running the repository against coco will produce some error like below:

wrote gt roidb to /data/cache/COCO_train2014_gt_roidb.pkl
append flipped images to roidb
Traceback (most recent call last):
  File "main_train.py", line 59, in <module>
    for image_set in image_sets]
  File "lib/data_utils/load_data.py", line 36, in load_proposal_roidb
    roidb = imdb.append_flipped_images(roidb)
  File "lib/dataset/imdb.py", line 249, in append_flipped_images
    'proposal_scores': roidb[i]['proposal_scores']}
KeyError: 'proposal_scores'

And changing FLIP to false will eliminate the error. I suspect there's something wrong with the flipping logic (didn't dig deep though).

Hi, I got an index out of bound error when training my own model.

loading annotations into memory...
Done (t=0.35s)
creating index...
index created!
num_images 11500
COCO_train2014 gt roidb loaded from ./data/cache/COCO_train2014_gt_roidb.pkl
appending ground truth annotations
Reading cached proposals after NMS* from data/proposals/COCO_train2014_rpn_after_nms.pkl
Done!
append flipped images to roidb
filtered 7928 roidb entries: 23000 -> 15072
add bounding box regression targets
bbox target means:
[[0. 0. 0. 0.]
[0. 0. 0. 0.]]
[0. 0. 0. 0.]
bbox target stdevs:
[[0.1 0.1 0.2 0.2]
[0.1 0.1 0.2 0.2]]
[0.1 0.1 0.2 0.2]
Creating Iterator with 15072 Images
Total number of extracted chips: 72201
Done!
The Iterator has 72201 samples!
Initializing the model...
Optimizer params: {'wd': 0.01, 'lr_scheduler': <train_utils.lr_scheduler.WarmupMultiBatchScheduler object at 0x7fee7fce4610>, 'multi_precision': True, 'learning_rate': 0.00015, 'rescale_grad': 1.0, 'clip_gradient': None, 'momentum': 0.9}
aaa
[00:32:22] src/operator/nn/./cudnn/./cudnn_algoreg-inl.h:107: Running performance tests to find the best convolution algorithm, this can take a while... (setting env variable MXNET_CUDNN_AUTOTUNE_DEFAULT to 0 to disable)
Exception in thread Thread-8:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/usr/lib/python2.7/threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "lib/iterators/PrefetchingIter.py", line 61, in prefetch_func
self.next_batch[i] = self.iters[i].next()
File "lib/iterators/MNIteratorBase.py", line 90, in next
if self.iter_next():
File "lib/iterators/MNIteratorBase.py", line 87, in iter_next
return self.get_batch()
File "lib/iterators/MNIteratorE2E.py", line 99, in get_batch
self.batch = self._get_batch()
File "lib/iterators/MNIteratorE2E.py", line 136, in _get_batch
nids = processed_roidb[i]['props_in_chips'][cropid]
IndexError: index 12 is out of bounds for axis 0 with size 12

How can I fix this?

1. there are three branch, i want to know theirs relations, is the model structure is the same, but only train on different data set?
2. as the master branch i try demo.py(output/sniper_res101_bn/sniper_res101_e2e/train2014_val2014/SNIPER-0007.params) which is coco 81 class, the result is very well especially for small object, so i want to try the openimage image v4's 600 class, i download the pretrained model by scripts, how can i modify the config file and demo.py?

Thanks for your code very much.
Could you report the human AP on the coco test-dev dataset ?

Hello,

First of all thank you for putting the code here on github so we can all use and experiment with it.
I am trying to train on my own dataset,but I am having a little bit of trouble.
More specifically, I have this error:
File "lib/iterators/MNIteratorE2E.py", line 194, in _get_batch
processed_list = processed_list.get()
File "/home/radud/anaconda3/envs/mxnetpy2/lib/python2.7/multiprocessing/pool.py", line 572, in get
raise self._value
TypeError: 'NoneType' object has no attribute 'getitem'

Any help would be appreciated.
Thank you.

Traceback (most recent call last):
File "main_train.py", line 64, in
for image_set in image_sets]
File "lib/data_utils/load_data.py", line 29, in load_proposal_roidb
roidb = imdb.gt_roidb()
File "lib/dataset/coco.py", line 135, in gt_roidb
roientry,flag = self._load_coco_annotation(index)
File "lib/dataset/coco.py", line 192, in _load_coco_annotation
if obj['area'] > 0 and x2 >= x1 and y2 >= y1:
KeyError: 'area'

Who have met the same question?Please help me, thanks!!

Hello, @mahyarnajibi, I am new to mxnet and when I run the cmake command, it turns out:

-- The C compiler identification is GNU 4.8.4
-- The CXX compiler identification is GNU 4.8.4
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- CMake version '3.11.4' using generator 'Unix Makefiles'
-- The CUDA compiler identification is NVIDIA 8.0.26
-- Check for working CUDA compiler: /usr/local/cuda-8.0/bin/nvcc
-- Check for working CUDA compiler: /usr/local/cuda-8.0/bin/nvcc -- works
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- Performing Test SUPPORT_CXX11
-- Performing Test SUPPORT_CXX11 - Success
-- Performing Test SUPPORT_CXX0X
-- Performing Test SUPPORT_CXX0X - Success
-- Performing Test SUPPORT_MSSE2
-- Performing Test SUPPORT_MSSE2 - Success
CMake Deprecation Warning at 3rdparty/mkldnn/CMakeLists.txt:21 (cmake_policy):
The OLD behavior for policy CMP0048 will be removed from a future version
of CMake.

The cmake-policies(7) manual explains that the OLD behaviors of all
policies are deprecated and that a policy should be set to OLD only under
specific short-term circumstances. Projects should be ported to the NEW
behavior and not rely on setting a policy to OLD.

CMake Deprecation Warning at 3rdparty/mkldnn/CMakeLists.txt:22 (cmake_policy):
The OLD behavior for policy CMP0054 will be removed from a future version
of CMake.

The cmake-policies(7) manual explains that the OLD behaviors of all
policies are deprecated and that a policy should be set to OLD only under
specific short-term circumstances. Projects should be ported to the NEW
behavior and not rely on setting a policy to OLD.

-- CMAKE_BUILD_TYPE is unset, defaulting to Release
-- Detecting Intel(R) MKL: trying mklml_intel
-- Detecting Intel(R) MKL: trying mklml
-- Detecting Intel(R) MKL: trying mkl_rt
CMake Warning at 3rdparty/mkldnn/cmake/MKL.cmake:178 (message):
Intel(R) MKL not found. Some performance features may not be available.
Please run scripts/prepare_mkl.sh to download a minimal set of libraries or
get a full version from https://software.intel.com/en-us/intel-mkl
Call Stack (most recent call first):
3rdparty/mkldnn/cmake/OpenMP.cmake:25 (include)
3rdparty/mkldnn/CMakeLists.txt:57 (include)

-- Found OpenMP_C: -fopenmp (found version "3.1")
-- Found OpenMP_CXX: -fopenmp (found version "3.1")
-- Found OpenMP: TRUE (found version "3.1")
-- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE)
-- VTune profiling environment is unset
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - found
-- Found Threads: TRUE
-- Could NOT find MKL (missing: MKL_INCLUDE_DIR MKLML_GNU_LIBRARY MKLDNN_LIBRARY)
-- MKL not found
-- Found CUDA: /usr/local/cuda-8.0 (found version "8.0")
-- Could NOT find MKL (missing: MKL_INCLUDE_DIR MKLML_GNU_LIBRARY MKLDNN_LIBRARY)
-- Found OpenBLAS libraries: /usr/lib/libopenblas.so
-- Found OpenBLAS include: /usr/include
-- Could NOT find Gperftools (missing: GPERFTOOLS_LIBRARIES GPERFTOOLS_INCLUDE_DIR)
-- Found PkgConfig: /usr/bin/pkg-config (found version "0.26")
-- Could NOT find Jemalloc (missing: JEMALLOC_LIBRARY JEMALLOC_INCLUDE_DIR)
-- OpenCV_LIBS=opencv_core;opencv_highgui;opencv_imgproc
-- OpenCV found (/usr/share/OpenCV)
-- Found OpenMP_C: -fopenmp
-- Found OpenMP_CXX: -fopenmp
-- Found OpenMP: TRUE
CMake Error at SNIPER/SNIPER-mxnet/build/3rdparty/openmp/runtime/CMakeFiles/CMakeTmp/CMakeLists.txt:3 (project):
Generator

Unix Makefiles

does not support toolset specification, but toolset

cuda=,host=x64

was specified.

CMake Error: CMAKE_C_COMPILER not set, after EnableLanguage
CMake Error at 3rdparty/openmp/runtime/cmake/LibompGetArchitecture.cmake:57 (try_run):
Failed to configure test project build system.
Call Stack (most recent call first):
3rdparty/openmp/runtime/CMakeLists.txt:40 (libomp_get_architecture)

-- Configuring incomplete, errors occurred!

Would you please be so kind as to help me out? Thanks.

Hi, I use SNIPER to train my own dataset. When training, I got nan loss as bellow,

Iter[0] Batch[100]   
Train-RPNAcc=nan  
Train-RPNLogLoss=nan  
Train-RPNL1Loss=nan  
Train-RCNNAcc=nan  
Train-RCNNLogLoss=nan  
Train-RCNNL1LossCRCNN=nan

It just occurs every several batches. My batch size is 4, and every 100 batches, it occurs. Other time everything just looks fine like bellow,

Iter[0] Batch[195]   
Train-RPNAcc=0.782257  
Train-RPNLogLoss=0.652749  
Train-RPNL1Loss=0.023370  
Train-RCNNAcc=0.979158  
Train-RCNNLogLoss=2.133237  
Train-RCNNL1LossCRCNN=0.046825

I trying to decrease lr to 0.0005, but not work.

Hello,
I have read the SNIPER paper and something is not clear to me.
Is this training/inference method compatible only with 2 stage detectors like Faster RCNN?
Or can it also be applied to SSD?

Thank you

I've noticed that you replace the last few rois with gt_boxes in MultiProposalTarget Operater, which means that there are accurate proposals for each RoI batch.

1. How much does this replacement benefit the overall accuracy?
2. Should we disable these lazy RoIs for END2END training?

Hi,In openimages.py ,i want to know the reason why you make gt_roidb's max_overlaps and max_classes only equal boxes' class but not include crowed' class.

Hi, I got a problem which really confused me for several days, and I cannot solve. If you may help, that will be great.
The error message shows bellow:

loading annotations into memory...
Done (t=4.05s)
creating index...
index created!
num_images 40504
COCO_val2014 gt roidb loaded from ./data/cache/COCO_val2014_gt_roidb.pkl
filtered 367 roidb entries: 40504 -> 40137
add bounding box regression targets
bbox target means:
[[0. 0. 0. 0.]
 [0. 0. 0. 0.]]
[0. 0. 0. 0.]
bbox target stdevs:
[[0.1 0.1 0.2 0.2]
 [0.1 0.1 0.2 0.2]]
[0.1 0.1 0.2 0.2]
Creating Iterator with 40137 Images
Total number of extracted chips: 155974
Done!
Traceback (most recent call last):
  File "/home/arias/Projects/Python/SNIPER/main_train.py", line 74, in <module>
    threads=config.TRAIN.NUM_THREAD, pad_rois_to=400)
  File "lib/iterators/MNIteratorE2E.py", line 31, in __init__
    super(MNIteratorE2E, self).__init__(roidb, config, batch_size, threads, nGPUs, pad_rois_to, False)
  File "lib/iterators/MNIteratorBase.py", line 33, in __init__
    self.get_batch()
  File "lib/iterators/MNIteratorE2E.py", line 99, in get_batch
    self.batch = self._get_batch()
  File "lib/iterators/MNIteratorE2E.py", line 171, in _get_batch
    all_labels = self.pool.map(self.anchor_worker.worker, worker_data)
  File "/home/arias/anaconda3/envs/python2/lib/python2.7/multiprocessing/pool.py", line 253, in map
    return self.map_async(func, iterable, chunksize).get()
  File "/home/arias/anaconda3/envs/python2/lib/python2.7/multiprocessing/pool.py", line 572, in get
    raise self._value
TypeError: Cannot cast ufunc subtract output from dtype('float64') to dtype('uint16') with casting rule 'same_kind'

Hi,

First of all, thanks for providing this nicely organized repo.
I have successfully run SNIPER demo without any problem.
However, for R-FCN 3K, I am not able to run the demo. There is strange error, which originates from asnumpy() function.
The error message I got is
SNIPER/CRCNN-mxnet/3rdparty/nnvm/include/nnvm/tuple.h:438: Check failed: dim == static_cast(ndim()) (2 vs. 1) dimension do not match target dimension 2 vs 1

I am using CUDA 9.1 with cuDNN 7.02 on a P3.x16 machine.

I have tried diffrent configures on the cmake, e.g., with/withou mkl, but I still cannot figure out what is the problem.

I can easily convert other NDArray to numpy array without any problem. I just cannot convert the output of the R-FCN network.

Please advise.

Installation and all compilations went smoothly. However, when run "python demo.py", encountered following error:

*** invalid %N$ use detected ***
Aborted (core dumped)

Further inspection shows a simple test command "import mxnet" gives same error.

Env: ubuntu 16.04.04 (kernel 4.4), CUDA9.0/cudnn 7.0, Nvidia 1080 ti.

Tried another machine with ubuntu 16.04.04 (kernel 4.13) with different CPUs and builds, has exact same error.

Wondering anyone sees same error?

Thanks.

ps: same env runs several mxnet projects without any issue, including Deformable-ConvNet project.

I'm using a custom dataset. But it's asking for RPN weights, is there any way to bypass that. If I load the weights, it failing with the following error:

Loading data/proposals/voc_2007_trainval_rpn.pkl....
Done!
Applying NMS...
Traceback (most recent call last):
  File "main_train.py", line 59, in <module>
    for image_set in image_sets]
  File "lib/data_utils/load_data.py", line 32, in load_proposal_roidb
    roidb = eval('imdb.' + proposal + '_roidb')(roidb, append_gt,proposal_path=proposal_path)
  File "lib/dataset/imdb.py", line 139, in rpn_roidb
    rpn_roidb = self.load_rpn_roidb(gt_roidb,proposal_path)
  File "lib/dataset/imdb.py", line 126, in load_rpn_roidb
    box_list, mapping_list = self.load_rpn_data(proposal_path)
  File "lib/dataset/imdb.py", line 109, in load_rpn_data
    keeps = p.map(nmsp, ttboxes)
  File "/usr/lib/python2.7/multiprocessing/pool.py", line 251, in map
    return self.map_async(func, iterable, chunksize).get()
  File "/usr/lib/python2.7/multiprocessing/pool.py", line 567, in get
    raise self._value
IndexError: too many indices for array

Hi, I try to train SNIPER on my own dataset, but the performance always keep low as bellow,

 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.004
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.015
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.000
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.002
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.005
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.067
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.125
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.134
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.011
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.176

Loss and accuracy seem just fine as bellow,

Train-RPNAcc=0.965601  
Train-RPNLogLoss=0.114530  
Train-RPNL1Loss=0.009350  
Train-RCNNAcc=0.963693  
Train-RCNNLogLoss=0.225078  
Train-RCNNL1LossC
RCNN=0.085674

Here is the yaml config,

# --------------------------------------------------------------
# SNIPER: Efficient Multi-Scale Training
# Licensed under The Apache-2.0 License [see LICENSE for details]
# by Mahyar Najibi, Bharat Singh
# --------------------------------------------------------------
---
MXNET_VERSION: "mxnet"
output_path: "./output/sniper_res101_bn"
symbol: resnet_mx_101_e2e
gpus: '0'
CLASS_AGNOSTIC: true
default:
  kvstore: device
network:
  pretrained: "./data/pretrained_model/resnet_mx_101_open"
  pretrained_epoch: 0
  PIXEL_MEANS:
  - 103.939
  - 116.779
  - 123.68
  RPN_FEAT_STRIDE: 16
  FIXED_PARAMS:
  - conv0
  - bn0
  - stage1

  ANCHOR_RATIOS:
  - 0.25
  - 0.5
  - 1
  - 2
  - 4
  ANCHOR_SCALES:
  - 2
  - 4
  - 7
  - 10
  - 13
  - 16
  - 24
  NUM_ANCHORS: 35
dataset:
  NUM_CLASSES: 5
  dataset: coco
  dataset_path: "./data/n_classes"
  image_set: train2014
  root_path: "./data"
  test_image_set: val2014
  proposal: rpn
TRAIN:
  ## CHIP GENERATION PARAMS
  # Whether to use C++ or python code for chip generation
  CPP_CHIPS: true
  # How many parts the dataset should be divided to for parallel chip generation
  # This is used to keep the memory limited
  CHIPS_DB_PARTS: 20


  # Multi-processing params
  # These parameters are used for parallel chip generation, NMS, etc.
  # Please consider adjusting them for your system
  NUM_PROCESS: 8
  NUM_THREAD: 8

  # Whether to train with segmentation mask
  WITH_MASK: false

  # Training scales
  # The last scale (or the only scale) should be the desired max resolution in pixels
  # Other scales should be scaling coefficients
  SCALES:
  - 0.5
  - 0.8
  - 2
  - 3.0
  - 1.667
  - 512.0

  # Valid ranges in each scale
  VALID_RANGES:
  - !!python/tuple [-1,-1]
  - !!python/tuple [-1,-1]
  - !!python/tuple [-1,-1]
  - !!python/tuple [-1,-1]
  - !!python/tuple [-1,-1]
  - !!python/tuple [-1,-1]


  lr: 0.0001 #0.002 #0.0005
  lr_step: '5.33'
  warmup: true
  fp16: true
  warmup_lr: 0.0005 #0.00005
  wd: 0.0001
  scale: 100.0
  warmup_step: 1000 #4000 #1000
  begin_epoch: 0
  end_epoch: 100

  # whether flip image
  FLIP: false
  # whether shuffle image
  SHUFFLE: true
  # whether use OHEM
  ENABLE_OHEM: true
  # size of images for each device, 2 for rcnn, 1 for rpn and e2e
  BATCH_IMAGES: 4
  # e2e changes behavior of anchor loader and metric
  END2END: true
  # R-CNN
  # rcnn rois batch size
  BATCH_ROIS: -1
  BATCH_ROIS_OHEM: 256
  # rcnn rois sampling params
  FG_FRACTION: 0.25
  FG_THRESH: 0.5
  BG_THRESH_HI: 0.5
  BG_THRESH_LO: 0.0
  # rcnn bounding box regression params
  BBOX_REGRESSION_THRESH: 0.5
  BBOX_WEIGHTS:
  - 1.0
  - 1.0
  - 1.0
  - 1.0

  # RPN anchor loader
  # rpn anchors batch size
  RPN_BATCH_SIZE: 256
  # rpn anchors sampling params
  RPN_FG_FRACTION: 0.5
  RPN_POSITIVE_OVERLAP: 0.5
  RPN_NEGATIVE_OVERLAP: 0.4
  RPN_CLOBBER_POSITIVES: false
  # rpn bounding box regression params
  RPN_BBOX_WEIGHTS:
  - 1.0
  - 1.0
  - 1.0
  - 1.0
  RPN_POSITIVE_WEIGHT: -1.0
  # used for end2end training
  # RPN proposal
  CXX_PROPOSAL: false
  RPN_NMS_THRESH: 0.7
  RPN_PRE_NMS_TOP_N: 6000
  RPN_POST_NMS_TOP_N: 300
  RPN_MIN_SIZE: 0
  # approximate bounding box regression
  BBOX_NORMALIZATION_PRECOMPUTED: true
  BBOX_MEANS:
  - 0.0
  - 0.0
  - 0.0
  - 0.0
  BBOX_STDS:
  - 0.1
  - 0.1
  - 0.2
  - 0.2
  USE_NEG_CHIPS: false
TEST:
  # Maximum number of detections per image
  # Set to -1 to disable
  MAX_PER_IMAGE: 50

  # Whether to do multi-scale inference
  SCALES:
#  - !!python/tuple [1400, 2000]
  - !!python/tuple [800, 1280]
  - !!python/tuple [480, 512]

  # Number of images per gpu for each scale
  BATCH_IMAGES:
  - 1
  - 1
  - 1

  # Number of concurrent jobs used for inference
  # if greater than 1, the roidb is distributed over
  # concurrent jobs to increase throughput
  CONCURRENT_JOBS: 2

  # Ranges to specify valid proposal length
  # in each of the test scale, square area
  # would be computed based on the lengths
  # to invalidate, -1 means unbounded, use
  # -1 everywhere if you want to have all proposals
  VALID_RANGES:
#  - !!python/tuple [-1,90]
#  - !!python/tuple [32,180]
#  - !!python/tuple [75,-1]
   - !!python/tuple [-1, -1]
   - !!python/tuple [-1, -1]

  # Use rpn to generate proposal
  HAS_RPN: true

  # RPN Parameters
  RPN_NMS_THRESH: 0.7
  RPN_PRE_NMS_TOP_N: 6000
  RPN_POST_NMS_TOP_N: 300
  RPN_MIN_SIZE: 0
  PROPOSAL_NMS_THRESH: 0.7
  PROPOSAL_PRE_NMS_TOP_N: 20000
  PROPOSAL_POST_NMS_TOP_N: 2000
  PROPOSAL_MIN_SIZE: 0

  # NMS Parameters
  # Whether to apply NMS based on threshold or sigma
  NMS: -1 #0.45
  NMS_SIGMA: 0.55

  # Which epoch of the training be used for testing
  TEST_EPOCH: 100

  # VISUALIZATION CONFIG
  VISUALIZATION_PATH: './debug/visualization'
  # Whether to visualize all intermediate scales
  # before aggregation (when doing multi-scale inference)
  # If False, only final detections are saved to 
  # VISUALIZATION_PATH
  VISUALIZE_INTERMEDIATE_SCALES: false

  # PROPOSAL EXTRACTION FLAGS
  # If true only would extract proposals
  EXTRACT_PROPOSALS: false

  # The folder path to be used for saving proposals
  PROPOSAL_SAVE_PATH: 'output/proposals'

  # Number of proposals extracted per scale
  # SCALES and BATCH_IMAGES above would be used to
  # Specify scales and number of images per batch for
  # each scale, no valid ranges would be applied for
  # aggregating proposals
  N_PROPOSAL_PER_SCALE: 300

I have struggled for several days but couldn't figure it out.
Hope you can give some advice.
Thanks :)

Hi,

Thank you for your repo and paper. This is an awesome work!

I want to ask that in your paper An Analysis of Scale Invariance in Object Detection – SNIP

You mentioned an ensemble operation. You said that:

Since proposals are shared across all networks, we
average the scores and box-predictions for each RoI. During
flipping we average the detection scores and bounding
box predictions.

How to average box predictions for each RoI?

How did you implemented in your code?

Thank you!

when i train use main_train.py, appear the following question.

Optimizer params: {'wd': 0.01, 'lr_scheduler': <train_utils.lr_scheduler.WarmupMultiBatchScheduler object at 0x7fe7702edd90>, 'multi_precision': True, 'learning_rate': 0.00015, 'rescale_grad': 1.0, 'clip_gradient': None, 'momentum': 0.9}
aaa
[00:33:49] /home/czn/scc/SNIPER/SNIPER-mxnet/src/operator/nn/./cudnn/./cudnn_algoreg-inl.h:107: Running performance tests to find the best convolution algorithm, this can take a while... (setting env variable MXNET_CUDNN_AUTOTUNE_DEFAULT to 0 to disable)
CUDA error: too many resources requested for launch

Hello, @mahyarnajibi ! It is possible to test your pretrained model to detect 600 classes? I check branch openimages2 and in config see pretrained: "./data/pretrained_model/CRCNN" row, but bash scripts/download_pretrained_model.sh doesn't download anything, but only resnet_mx_101_open-0000(and imagenet ofc). This is what we need, this model is pretraied for 600 classes? and we need only move(or replace config's row with path) model to CRCNN folder and run python demo.py with custom config?

Hi,

I am currently trying to improve my performance for a certain dataset, and I am not using negative chips to train my algorithms. From the visualization it looks fine, however there are some false positives that I do want to avoid. I am pretty sure that is because of the fact that I don't do negative chip training.

Can you specify more on how to get the negative chips? From the paper it says:

To this end, for negative chip mining, we first train RPN for a couple of epochs. No negative chips are used for training this network. The task of this network is to roughly guide us in selecting regions which are likely to contain false positives, so it is not necessary for it to be very accurate. This RPN is used to generate proposals over the entire training set. We assume that if no proposals are generated in a major portion of the image by RPN, then it is unlikely to contain an object instance.

And in the code, I found that there's a setting called TEST.EXTRACT_PROPOSALS - but reading thru the code, I am not sure if it is doing what's described in the paper.

Any pointers will be appreciated! Thanks!

Great work. I think it would be really great if there is an inference example script for running openimagev4 pretrained model just similar the current demo.py (COCO).

Thank You for sharing code.
fyi cannot perform git clone --recursive on SNIPER:

fatal: repository 'https://github.com/mahyarnajibi/SNIPER-mxnet/' not found
fatal: clone of 'https://github.com/mahyarnajibi/SNIPER-mxnet' into submodule path 'SNIPER-mxnet' failed

Hi team,

Another question I have is how to avoid resizing the output image (i.e. output the same image resolution as the input)? One particular example would be for the image below (some random image on Internet), the input resolution is 1023 * 968, however the output resolution is 640 * 480, plus additional white edges. BTW I tried to modify the TEST.SCALES option in the yaml file to a large number but with no luck.
SCALES:

• !!python/tuple [1400, 2000]
• !!python/tuple [1400, 2100]
• !!python/tuple [1400, 2200]

input:

output:

i test the coco pretrained model, the result is really very good, as openimage v4 has 600 classes, can this pretrained model can share?

If I change rpn_post_nms_top_n, the metric will get wrong at

pred_label = pred.asnumpy().reshape(-1, last_dim).argmax(axis=1).astype('int32')

Really awesome work!
Do you plan to release an inference script for the openimages2 branch, as well as a pretrained model?

Hi, what is the cuda version? python version

hi, I've tried to run the 'demo.py' file, then met the error like:
mxnet.base.MXNetError: [22:56:26] /home/vis/xuxiaqing/code/code_my/video_detection/SNIPER/SNIPER-mxnet/src/operator/nn/batch_norm.cu:527: Check failed: err == cudaSuccess (7 vs. 0) Name: BatchNormalizationUpdateOutput ErrStr:too many resources requested for launch

My cuda version is 8.0, the cudnn version is 5.0. Are there any tips for solving this problem? many thanks~

Hi,

Which specific model did you use for Resnet 50? Can you provide the link to download the pre-trained resnet50 on imagenet? Looks like it's not available in the mxnet model zoo...

Thanks!

Hi , I tried to use SNIPER on a new dataset but the perfomance is not satisfying. I think the problem is that my own neg-chips are not as good as yours in coco dataset. Could you please release the SNIP code so that I can train my dataset on SNIP and compare their performances?

Thanks a lot

Hi, @mahyarnajibi ,, I am new to mxnet and when I run the command,:python demo.py. it turns out:
mxnet.base.MXNetError: [15:56:34] /home/chase/SNIPER/SNIPER-mxnet/src/operator/nn/batch_norm.cu:527: Check failed: err == cudaSuccess (7 vs. 0) Name: BatchNormalizationUpdateOutput ErrStr:too many resources requested for launch

Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/libmxnet.so(dmlc::StackTraceabi:cxx11+0x1bc) [0x7f8cdb156b5c]
[bt] (1) /usr/local/lib/libmxnet.so(dmlc::LogMessageFatal::~LogMessageFatal()+0x28) [0x7f8cdb157f08]
[bt] (2) /usr/local/lib/libmxnet.so(void mxnet::op::BatchNormForwardImpl<mshadow::gpu, mshadow::half::half_t, float>(mshadow::Streammshadow::gpu, mxnet::OpContext const&, mxnet::op::BatchNormParam const&, std::vector<mxnet::TBlob, std::allocatormxnet::TBlob > const&, std::vector<mxnet::OpReqType, std::allocatormxnet::OpReqType > const&, std::vector<mxnet::TBlob, std::allocatormxnet::TBlob > const&, std::vector<mxnet::TBlob, std::allocatormxnet::TBlob > const&)+0x31e) [0x7f8cdcd2a05e]
[bt] (3) /usr/local/lib/libmxnet.so(void mxnet::op::BatchNormForward<mshadow::gpu, mshadow::half::half_t, float>(mxnet::OpContext const&, mxnet::op::BatchNormParam const&, std::vector<mxnet::TBlob, std::allocatormxnet::TBlob > const&, std::vector<mxnet::OpReqType, std::allocatormxnet::OpReqType > const&, std::vector<mxnet::TBlob, std::allocatormxnet::TBlob > const&, std::vector<mxnet::TBlob, std::allocatormxnet::TBlob > const&)+0x301) [0x7f8cdcd2a841]
[bt] (4) /usr/local/lib/libmxnet.so(void mxnet::op::BatchNormComputemshadow::gpu(nnvm::NodeAttrs const&, mxnet::OpContext const&, std::vector<mxnet::TBlob, std::allocatormxnet::TBlob > const&, std::vector<mxnet::OpReqType, std::allocatormxnet::OpReqType > const&, std::vector<mxnet::TBlob, std::allocatormxnet::TBlob > const&)+0x852) [0x7f8cdcd1fcc2]
[bt] (5) /usr/local/lib/libmxnet.so(mxnet::exec::FComputeExecutor::Run(mxnet::RunContext, bool)+0x50) [0x7f8cdb2a0660]
[bt] (6) /usr/local/lib/libmxnet.so(+0xc55ba8) [0x7f8cdb2a3ba8]
[bt] (7) /usr/local/lib/libmxnet.so(mxnet::engine::ThreadedEngine::ExecuteOprBlock(mxnet::RunContext, mxnet::engine::OprBlock)+0x429) [0x7f8cdcb149c9]
[bt] (8) /usr/local/lib/libmxnet.so(void mxnet::engine::ThreadedEnginePerDevice::GPUWorker<(dmlc::ConcurrentQueueType)0>(mxnet::Context, bool, mxnet::engine::ThreadedEnginePerDevice::ThreadWorkerBlock<(dmlc::ConcurrentQueueType)0>, std::shared_ptrdmlc::ManualEvent const&)+0xeb) [0x7f8cdcb18c4b]
[bt] (9) /usr/local/lib/libmxnet.so(std::_Function_handler<void (std::shared_ptrdmlc::ManualEvent), mxnet::engine::ThreadedEnginePerDevice::PushToExecute(mxnet::engine::OprBlock, bool)::{lambda()#3}::operator()() const::{lambda(std::shared_ptrdmlc::ManualEvent)#1}>::_M_invoke(std::_Any_data const&, std::shared_ptrdmlc::ManualEvent&&)+0x4e) [0x7f8cdcb18ece]

Would you please to help me?Thank you.

HI.
Thinks for provide this nice project.
When I compile this repo,I met some problems, and I solve it with compile OpenBLAS and lapcak,link lapack/liblapack.a to /usr/lib/liblapack.a. And then,i run the demo.py,But it not response after print two line 'aaa' and no any other message.Then i change the default config to fit my double 1080ti GPU.I just change the gpus from '0,1,2,3,4,5,6,7' to '0,1',and change the lr from '0.015' to '0.04'.Download pretrained_model and proposals,run the main_train.py script.But the process exhaust all my 32G memory and the computer not responsed.I don't know how to solve this problem.
Env: ubuntu 16.04.04 (kernel 4.4), CUDA9.0/cudnn 7.1.4, double Nvidia 1080 ti,python 2.7
Can you give me some abvise.

What would be the best way to setup a python script to get a detection from a single image from a trained R-FCN-3k model? Thanks!

Hey,
Sniper is an awesome work. However ,FPN is a standard component of object detection for scales with slightly computation increased. Have you tried to combine Sniper with FPN ?

Hi authors, Great work! And I know that you're migrating caffe SSH to this repo which will be re-write by MXnet. I have done such work these days but can not achieve the same accuracy as it was by Caffe. For example, single scale 79.1% mAP on widerface-hard, vs reported 81.4% from caffe SSH.
Do you have any tips on it?

Thanks.

i run the demo.py with mobilenetv2 model on the P40 ,about 150ms,the speed is slow,something wrong?
i never change the config.

Tester: 1/1, Detection: 0.1813s, Post Processing: 0.001738s
Tester: 1/1, Detection: 0.1461s, Post Processing: 0.001268s
Tester: 1/1, Detection: 0.1313s, Post Processing: 0.001222s

Thanks for open sourcing the code, it looks great!

One question I have is how to train new proposals? Since I want to use SNIPER in a very different dataset, I assume the best way for me is to re-calculate the proposals? In that case, how do I re-train the proposal network?

Thanks!

Hi;
I noticed ms < (self.chip_size - self.chip_stride - 1) / im_scale is served as a constraint to choose the valid gt_boxes id(please see below), which however is unmentioned in you paper. Could you please explain the idea behind this constraint? Thank you！

            elif i == 0:
                # The finest scale (but not the only scale)
                ids = np.where((area < self.valid_ranges[i][1]) &
                               (ms < (self.chip_size - self.chip_stride - 1) / im_scale) & (ws >= 2) & (hs >= 2))[0]
            else:
                # An intermediate scale
                ids = np.where((area >= self.valid_ranges[i][0]) & (area < self.valid_ranges[i][1])
                       & (ms < (self.chip_size - self.chip_stride - 1) / im_scale))[0]

When I try bash scripts/compile.sh
Here are bugs:

x86_64-linux-gnu-gcc: error: unrecognized command line option ‘-Wdate-time’
x86_64-linux-gnu-gcc: error: unrecognized command line option ‘-fstack-protector-strong’
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

My solution is as follows:
sudo rm -r /usr/bin/gcc
sudo ln -s /usr/bin/gcc-5 /usr/bin/gcc
sudo ln -s /usr/bin/gcc-5 /usr/bin/gcc
sudo rm /usr/bin/x86_64-linux-gnu-gcc
sudo ln -s /usr/bin/gcc-5 /usr/bin/x86_64-linux-gnu-gcc

links

What are the CRCNN-0007 weights trained on?

For context, it fails to detect people in images.

Hi, I have a quick question regarding model inference. Let's say I've trained a model using SNIPER, for a regular object detection task, can the model be served:

• using the official MXNet build?
• using CPU?

The reason is that for some reason, we need to inference the model on CPU. I guess it might be possible since the NN is still standard resnet + faster RCNN, but I am not sure about this (since I do see a few updates in the SNIPER-mxnet module which might be relevant to accomplish the goal above).

Thanks!