msracver / fcis Goto Github PK

I wonder what exactly do the SCALES numbers in the yaml configuration file mean. The settings for PASCAL VOC and COCO are:
`SCALES:

• 600
• 1000`

It seems that if you input an image of size say 1000x2000, your object detections will be restricted to the upperleft 600x1200 region (shorter side is 600, keeping the aspect ratio unchanged). Image size remains 1000x2000, it does not seem to be resized to 600x1200 first. Is this a bug, or if I missed something?

I would appreciate if someone could let me know what do these SCALES numbers in the yaml config file really mean, and the rule of thumb for setting their values.

can it use webcam?

how can I initialize the model param, if I don"t use any pretrained model (resnet101.param & fcis_coco_0000)?

If I don"t initialize param,will be the erro: conv1_weight not initialized

hi, i run python ./fcis/demo.py and get this error

Traceback (most recent call last):
File "./fcis/demo.py", line 147, in
main()
File "./fcis/demo.py", line 43, in main
sym = sym_instance.get_symbol(config, is_train=False)
File "/home/boston_lea/AutoMap/FCIS/fcis/symbols/resnet_v1_101_fcis.py", line 799, in get_symbol
psroipool_cls_seg = mx.contrib.sym.PSROIPooling(name='psroipool_cls_seg', data=fcis_cls_seg, rois=rois,
AttributeError: 'module' object has no attribute 'PSROIPooling'

any idea?

I compile the code with CUDA 7.5.

When I try to train the model, I run into segmenation fault error without any warning.

Everything is default apart from GPU 0,1,2,3

Best Wishes

Hi there, I was wondering if you still plan to release the code for your paper? Your results look fantastic and we'd love to learn from it and apply some of the techniques else where.

Hi,
I was able to successfully train and test on PASCAL VOC dataset from scratch and score accuracy of Mean [email protected] = 65.5%. Next, I want to conduct FCIS training on my own dataset, however the object boundaries ground truth information I have is in polygon points + class format like this: {class: "pen", "polygon": [(x1,y1), (x2,y2),(x3,y3),(x4,y4)]}. I'd appreciate if anyone is aware of an existing code or an easy way to convert this into SBD format. Note that closest thing I found was converting SpaceNet getJson labels to Pascal VOC labels (however, this requires converting what I have to getJson labels, which is still not as straightforward).

Any help/tips on a way to easily convert polygons/class to SBD would be appreciate.

An fatal error occurred in asynchronous engine operation. If you do not know what caused this error, you can try set environment variable MXNET_ENGINE_TYPE to NaiveEngine and run with debugger (i.e. gdb). This will force all operations to be synchronous and backtrace will give you the series of calls that lead to this error. Remember to set MXNET_ENGINE_TYPE back to empty after debugging.

Stack trace returned 6 entries:
[bt] (0) /home/shel/mxnet/python/mxnet/../../lib/libmxnet.so(_ZN4dmlc15LogMessageFatalD1Ev+0x3c) [0x7fcf6d268f7c]
[bt] (1) /home/shel/mxnet/python/mxnet/../../lib/libmxnet.so(_ZN5mxnet6engine14ThreadedEngine15ExecuteOprBlockENS_10RunContextEPNS0_8OprBlockE+0x4ac) [0x7fcf6e08cd4c]
[bt] (2) /home/shel/mxnet/python/mxnet/../../lib/libmxnet.so(_ZNSt17_Function_handlerIFvvEZZN5mxnet6engine23ThreadedEnginePerDevice13PushToExecuteEPNS2_8OprBlockEbENKUlvE1_clEvEUlvE_E9_M_invokeERKSt9_Any_data+0x60) [0x7fcf6e08fef0]
[bt] (3) /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb8c80) [0x7fcfe2828c80]
[bt] (4) /lib/x86_64-linux-gnu/libpthread.so.0(+0x76ba) [0x7fcfe90d16ba]
[bt] (5) /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fcfe8e0782d]

I am running mxnet in a gpu-enabled docker container (https://hub.docker.com/r/kaixhin/cuda-mxnet/) which uses cuda8. Apparently the FCIS expects cuda7.5? After copying the operator_cxx files to mxnet home and recompiling , I hit the libcudart error when trying to run the demo. Any hints on how to proceed?

The mxnet itself seems fine from the python check:

In [2]: import mxnet as mx
In [5]: a=mx.nd.ones((2,3),mx.gpu())
In [6]: b=a*2+1
In [7]: b.asnumpy()
Out[7]: array([[ 3.,  3.,  3.],
       [ 3.,  3.,  3.]], dtype=float32)

xbw@xbw-P65xRP:~/mxnet$ git checkout 62ecb60
warning: unable to rmdir dlpack: 目录非空
M cub
M dmlc-core
M mshadow
M nnvm
Note: checking out '62ecb60'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

git checkout -b

HEAD 目前位于 62ecb60... Add 'argnum' for autograd (#5787)

RUTI

is there any pdf of the paper?

xbw@xbw-P65xRP:~/mxnet$ git submodule update
fatal: 引用不是一个树：89de7ab20167909bc2c4f8acd397671c47cf3c0d
error: Your local changes to the following files would be overwritten by checkout:
CMakeLists.txt
cmake/lint.cmake
include/dmlc/any.h
include/dmlc/logging.h
include/dmlc/registry.h
include/dmlc/serializer.h
include/dmlc/thread_local.h
tracker/dmlc_tracker/local.py
tracker/dmlc_tracker/mesos.py
tracker/dmlc_tracker/mpi.py
tracker/dmlc_tracker/opts.py
tracker/dmlc_tracker/submit.py
Please, commit your changes or stash them before you can switch branches.
Aborting
error: Your local changes to the following files would be overwritten by checkout:
.gitignore
CMakeLists.txt
cmake/Cuda.cmake
cmake/mshadow.cmake
guide/defop.cpp
guide/exp-template/README.md
guide/exp-template/exp_template_op.cpp
guide/mshadow-ps/local_sum-inl.h
make/mshadow.mk
mshadow-ps/ps_local-inl.h
mshadow/cuda/reduce.cuh
mshadow/dot_engine-inl.h
mshadow/expression.h
mshadow/extension/broadcast_with_axis.h
mshadow/extension/channel_pool.h
mshadow/extension/implicit_gemm.h
mshadow/extension/transpose.h
mshadow/half.h
mshadow/packet-inl.h
mshadow/packet/plain-inl.h
mshadow/packet/sse-inl.h
mshadow/random.h
mshadow/stream_gpu-inl.h
mshadow/tensor_blob.h
Please, commit your changes or stash them before you can switch branches.
Aborting
error: Your local changes to the following files would be overwritten by checkout:
CMakeLists.txt
Makefile
include/nnvm/op_attr_types.h
include/nnvm/pass_functions.h
include/nnvm/symbolic.h
include/nnvm/tuple.h
src/core/node.cc
src/core/symbolic.cc
src/pass/gradient.cc
src/pass/plan_memory.cc
tests/cpp/tuple_test.cc
Please, commit your changes or stash them before you can switch branches.
Aborting
无法在子模组路径 'cub' 中检出 '89de7ab20167909bc2c4f8acd397671c47cf3c0d'
无法在子模组路径 'dmlc-core' 中检出 'b5bec5481df86e8e6728d8bd80a61d87ef3b2cd5'
无法在子模组路径 'mshadow' 中检出 '23210f3939428e42bc34553469ed9ce8c63001ed'
无法在子模组路径 'nnvm' 中检出 'ddf3c17e3455db9cd10f5b18bc9753a146971819'

From the following code, we can compute the mask loss, but I cannot understand what does the mask label look like, I've read through the code for loading data, I found that the mask label are just binary masks, which contains 0 and 1 only.... So, would you please help me make it clear?

class FCISMaskLossMetric(mx.metric.EvalMetric):
    def __init__(self, cfg):
        super(FCISMaskLossMetric, self).__init__('FCISMaskLoss')
        self.e2e = cfg.TRAIN.END2END
        self.pred, self.label = get_rcnn_names(cfg)
        self.cfg = cfg

    def update(self, labels, preds):
        mask_loss = preds[self.pred.index('fcis_mask_loss')]
        if self.e2e:
            label = preds[self.pred.index('fcis_mask_label')]
        else:
            raise NotImplementedError
        mask_size = mask_loss.shape[2]
        label = label.asnumpy().astype('int32').reshape((-1))
        mask_loss = mx.nd.transpose(mask_loss.reshape((mask_loss.shape[0], mask_loss.shape[1], mask_size * mask_size)), axes=(0, 2, 1))
        mask_loss = mask_loss.reshape((label.shape[0], 2))
        mask_loss = mask_loss.asnumpy()
        keep_inds = np.where(label != -1)[0]
        label = label[keep_inds]
        cls = mask_loss[keep_inds, label]
        cls += 1e-14
        cls_loss = -1 * np.log(cls)
        cls_loss = np.sum(cls_loss)

        self.sum_metric += cls_loss
        self.num_inst += len(keep_inds)

any one can help me? I'm just a fresher, thanks a lot!!!

I tried to implement end-to-end training but the network parameters soon became NaN. Do I have to use the same 5 step training as R-FCN?

When I train the model without OHTM, the loss will become Nan. Can you give more details on how to train the model without OHEM?

Why I want to train the model without OHEM?
Right now, I am developing a new algorithm based on your project. I wanna to train a baseline model with OHEM.

Hey, awesome project, and thank you for tagging the commit you built it with. Is there any way it could be built using a stable tag? MXNet doesn't guarantee stability between releases, so It just makes it a lot simpler to build :)

I used 0,1 gpus to train voc dataset, when I run the training program:

python experiments/fcis/fcis_end2end_train_test.py --cfg experiments/fcis/cfgs/resnet_v1_101_vocSDS_fcis_end2end.yaml

error occured
lr 0.0005 lr_epoch_diff [14.0] lr_iters [78722]
[14:03:16] src/operator/convolution.cu:87: This convolution is not supported by cudnn, MXNET convolution is applied.
[14:03:16] src/operator/convolution.cu:87: This convolution is not supported by cudnn, MXNET convolution is applied.
[14:03:16] src/operator/convolution.cu:87: This convolution is not supported by cudnn, MXNET convolution is applied.
[14:03:17] src/operator/convolution.cu:87: This convolution is not supported by cudnn, MXNET convolution is applied.
[14:03:17] src/operator/convolution.cu:87: This convolution is not supported by cudnn, MXNET convolution is applied.
[14:03:17] src/operator/convolution.cu:87: This convolution is not supported by cudnn, MXNET convolution is applied.
[14:03:18] src/operator/convolution.cu:87: This convolution is not supported by cudnn, MXNET convolution is applied.
[14:03:18] src/operator/convolution.cu:87: This convolution is not supported by cudnn, MXNET convolution is applied.
[14:03:18] src/operator/convolution.cu:87: This convolution is not supported by cudnn, MXNET convolution is applied.
[14:03:18] src/operator/convolution.cu:87: This convolution is not supported by cudnn, MXNET convolution is applied.
[14:03:18] src/operator/convolution.cu:87: This convolution is not supported by cudnn, MXNET convolution is applied.
[14:03:18] src/operator/convolution.cu:87: This convolution is not supported by cudnn, MXNET convolution is applied.
Traceback (most recent call last):
File "experiments/fcis/fcis_end2end_train_test.py", line 13, in
train_end2end.main()
File "experiments/fcis/../../fcis/train_end2end.py", line 181, in main
config.TRAIN.lr, config.TRAIN.lr_step)
File "experiments/fcis/../../fcis/train_end2end.py", line 173, in train_net
arg_params=arg_params, aux_params=aux_params, begin_epoch=begin_epoch, num_epoch=end_epoch)
File "experiments/fcis/../../fcis/core/module.py", line 955, in fit
self.update()
File "experiments/fcis/../../fcis/core/module.py", line 1037, in update
self._curr_module.update()
File "experiments/fcis/../../fcis/core/module.py", line 573, in update
self._kvstore)
TypeError: _update_params_on_kvstore() takes exactly 4 arguments (3 given)

FYI

Since OneDrive is blocked....

I have built mxnet with gpu support to run FCIS. Running python ./fcis/demo.py, the network run successfully on the gpu within a second, but the resulting image is just the original ones without masks.

By looking at the content and shape of various numpy arrays inside ./fcis/demo.py, I found that every thing looks sane (I mean there is some numbers like 9.96572733e-01 inside the arrays) until gpu_mask_voting, which is:

(Pdb) result_masks
[[], array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21
, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([
], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype
=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0,
 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), a
rray([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21),
 dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], sha
pe=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float
32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21
, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([
], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype
=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0, 1, 21, 21), dtype=float32), array([], shape=(0,
 1, 21, 21), dtype=float32)]

All the arrays are empty. Then I found there is a variant called cpu_mask_voting, so I just plug it in and then the masks are shown on images, and result_masks is not empty anymore.

             boxes = clip_boxes(boxes[0], (im_height, im_width))
-            result_masks, result_dets = gpu_mask_voting(masks, boxes, scores[0], num_classes,
+
+            result_masks, result_dets = cpu_mask_voting(masks, boxes, scores[0], num_classes,
                                                         100, im_width, im_height,
                                                         config.TEST.NMS, config.TEST.MASK_MERGE_THRESH,
-                                                        config.BINARY_THRESH, ctx_id[0])
+                                                        config.BINARY_THRESH)

             dets = [result_dets[j] for j in range(1, num_classes)]

Does it need any extra config to run gpu_mask_voting, and may I know how big is the performance gain by using it?

I am running FCIS master branch with the suggested commit of mxnet, on a amazon aws gpu instance, running ubuntu linux 16.04 and cudnn 5.

Thanks in advance.

Hello, thank you for your great code.
Right now, I am trying to evaluate the performance of your model. Here it the script :
After iterating through the minival5000, the code run into deadlock and never finish.

Looking forward to your advice.

python experiments/fcis/fcis_end2end_test.py --cfg experiments/fcis/cfgs/resnet_v1_101_coco_fcis_end2end_ohem.yaml
Namespace(cfg='experiments/fcis/cfgs/resnet_v1_101_coco_fcis_end2end_ohem.yaml', ignore_cache=False, shuffle=False, thresh=0.001, vis=False)
{'BINARY_THRESH': 0.4,
'CLASS_AGNOSTIC': True,
'MASK_SIZE': 21,
'MXNET_VERSION': 'mxnet',
'SCALES': [(600, 1000)],
'TEST': {'BATCH_IMAGES': 1,
'CXX_PROPOSAL': False,
'HAS_RPN': True,
'ITER': 2,
'MASK_MERGE_THRESH': 0.5,
'MIN_DROP_SIZE': 2,
'NMS': 0.3,
'PROPOSAL_MIN_SIZE': 2,
'PROPOSAL_NMS_THRESH': 0.7,
'PROPOSAL_POST_NMS_TOP_N': 2000,
'PROPOSAL_PRE_NMS_TOP_N': 20000,
'RPN_MIN_SIZE': 2,
'RPN_NMS_THRESH': 0.7,
'RPN_POST_NMS_TOP_N': 300,
'RPN_PRE_NMS_TOP_N': 6000,
'USE_GPU_MASK_MERGE': True,
'USE_MASK_MERGE': True,
'test_epoch': 8},
'TRAIN': {'ASPECT_GROUPING': True,
'BATCH_IMAGES': 1,
'BATCH_ROIS': -1,
'BATCH_ROIS_OHEM': 128,
'BBOX_MEANS': [0.0, 0.0, 0.0, 0.0],
'BBOX_NORMALIZATION_PRECOMPUTED': True,
'BBOX_REGRESSION_THRESH': 0.5,
'BBOX_STDS': [0.2, 0.2, 0.5, 0.5],
'BBOX_WEIGHTS': array([ 1., 1., 1., 1.]),
'BG_THRESH_HI': 0.5,
'BG_THRESH_LO': 0,
'BINARY_THRESH': 0.4,
'CONVNEW3': True,
'CXX_PROPOSAL': False,
'ENABLE_OHEM': True,
'END2END': True,
'FG_FRACTION': 0.25,
'FG_THRESH': 0.5,
'FLIP': True,
'GAP_SELECT_FROM_ALL': False,
'IGNORE_GAP': False,
'LOSS_WEIGHT': [1.0, 10.0, 1.0],
'RESUME': False,
'RPN_ALLOWED_BORDER': 0,
'RPN_BATCH_SIZE': 256,
'RPN_BBOX_WEIGHTS': [1.0, 1.0, 1.0, 1.0],
'RPN_CLOBBER_POSITIVES': False,
'RPN_FG_FRACTION': 0.5,
'RPN_MIN_SIZE': 2,
'RPN_NEGATIVE_OVERLAP': 0.3,
'RPN_NMS_THRESH': 0.7,
'RPN_POSITIVE_OVERLAP': 0.7,
'RPN_POSITIVE_WEIGHT': -1.0,
'RPN_POST_NMS_TOP_N': 300,
'RPN_PRE_NMS_TOP_N': 6000,
'SHUFFLE': True,
'begin_epoch': 0,
'end_epoch': 8,
'lr': 0.0005,
'lr_step': '5.33',
'model_prefix': 'e2e',
'momentum': 0.9,
'warmup': True,
'warmup_lr': 5e-05,
'warmup_step': 250,
'wd': 0.0005},
'dataset': {'NUM_CLASSES': 81,
'dataset': 'coco',
'dataset_path': './data/coco',
'image_set': 'train2014+valminusminival2014',
'proposal': 'rpn',
'root_path': './data',
'test_image_set': 'minival2014'},
'default': {'frequent': 100, 'kvstore': 'device'},
'gpus': '0',
'network': {'ANCHOR_RATIOS': [0.5, 1, 2],
'ANCHOR_SCALES': [4, 8, 16, 32],
'FIXED_PARAMS': ['conv1',
'bn_conv1',
'res2',
'bn2',
'gamma',
'beta'],
'FIXED_PARAMS_SHARED': ['conv1',
'bn_conv1',
'res2',
'bn2',
'res3',
'bn3',
'res4',
'bn4',
'gamma',
'beta'],
'IMAGE_STRIDE': 0,
'NUM_ANCHORS': 12,
'PIXEL_MEANS': array([ 103.06, 115.9 , 123.15]),
'RCNN_FEAT_STRIDE': 16,
'RPN_FEAT_STRIDE': 16,
'pretrained': './model/pretrained_model/resnet_v1_101',
'pretrained_epoch': 0},
'output_path': './output/fcis/coco',
'symbol': 'resnet_v1_101_fcis'}
loading annotations into memory...
Done (t=0.47s)
creating index...
index created!
num_images 5000
prepare gt_sdsdb using 1.46799492836 seconds
generate cache_seg_inst using 59.4011361599 seconds
testing 1/5000 data 0.0000s net 0.5483s post 0.1172s
testing 2/5000 data 0.0002s net 0.5420s post 0.1336s
testing 3/5000 data 0.0002s net 0.4863s post 0.1653s
testing 4/5000 data 0.0002s net 0.4686s post 0.1625s
testing 5/5000 data 0.0002s net 0.4714s post 0.1491s
testing 6/5000 data 0.0002s net 0.5424s post 0.1557s
testing 7/5000 data 0.0002s net 0.5172s post 0.1454s
testing 8/5000 data 0.0002s net 0.4875s post 0.1205s
testing 9/5000 data 0.0002s net 0.5642s post 0.0951s
testing 10/5000 data 0.0002s net 0.5434s post 0.0537s
testing 11/5000 data 0.0002s net 0.5247s post 0.1334s
testing 12/5000 data 0.0002s net 0.4702s post 0.1176s
testing 13/5000 data 0.0002s net 0.5445s post 0.0886s
testing 14/5000 data 0.0002s net 0.5181s post 0.1277s
testing 15/5000 data 0.0002s net 0.5344s post 0.1255s
testing 16/5000 data 0.0002s net 0.5619s post 0.1394s
testing 17/5000 data 0.0002s net 0.4823s post 0.0381s
testing 18/5000 data 0.0002s net 0.5213s post 0.1499s
testing 19/5000 data 0.0002s net 0.5446s post 0.1506s
testing 20/5000 data 0.0002s net 0.4844s post 0.1120s
testing 21/5000 data 0.0002s net 0.4683s post 0.1378s
testing 22/5000 data 0.0002s net 0.5321s post 0.1471s
testing 23/5000 data 0.0002s net 0.4832s post 0.1219s
testing 24/5000 data 0.0003s net 0.4679s post 0.1161s
testing 25/5000 data 0.0002s net 0.5483s post 0.1896s
testing 26/5000 data 0.0002s net 0.4867s post 0.1112s
..............................
testing 4987/5000 data 0.0002s net 0.4905s post 0.1353s
testing 4988/5000 data 0.0002s net 0.5457s post 0.1393s
testing 4989/5000 data 0.0002s net 0.5158s post 0.0691s
testing 4990/5000 data 0.0002s net 0.4863s post 0.1257s
testing 4991/5000 data 0.0002s net 0.4861s post 0.1124s
testing 4992/5000 data 0.0002s net 0.4891s post 0.1937s
testing 4993/5000 data 0.0002s net 0.5489s post 0.0714s
testing 4994/5000 data 0.0002s net 0.4860s post 0.1524s
testing 4995/5000 data 0.0002s net 0.4091s post 0.0633s
testing 4996/5000 data 0.0002s net 0.4887s post 0.1297s
testing 4997/5000 data 0.0002s net 0.5446s post 0.1039s
testing 4998/5000 data 0.0002s net 0.4870s post 0.1028s
testing 4999/5000 data 0.0002s net 0.4695s post 0.1214s
testing 5000/5000 data 0.0002s net 0.5398s post 0.1317s

Never finish.

It seems to me that for some images, gpu_mask_voting causes illegal memory access (the problem still exists after the latest fix on mask merge). This problem will go away if we use cpu_mask_voting instead.

Hi, thank you for sharing such great work. I'm confused about inside/outside scores discribed in your paper:
There are three cases: 1) high inside score and low outside score: detection+, segmentation+;
2) low inside score and high outside score: detection+, segmentation-; 3) both scores are low: detection-,segmentation-. The two scores answer the two questions jointly via softmax and max operations.
could you explain it a little bit? Thanks a lot.

When I use cv 3.2 to do test, here is the error tips I've got.
mxnet.base.MXNetError: [16:56:28] src/c_api/c_api_ndarray.cc:385: Operator _zeros cannot be run; requires at least one of FCompute, NDArrayFunction, FCreateOperator be registered

Hi, in my dataset, there's lots of small object(e.g.1010 object in a 512512 image), I use your code but there're little positive samples in rpn (less than 20), I change the config file,anchor scale and anchor ratio, It made some improvement but not obvious. Do you have some advice?Thanks in advance!

Hey,

I had already commented in another issue that there is no problem in renaming the repository to daijifeng001/FCIS. The old link (daijifeng001/TA-FCN) will then redirect to daijifeng001/FCIS. Cloning of either daijifeng001/TA-FCN or daijifeng001/FCIS results in the same repository.

I believe this will avoid some confusion due to the name change.

Best regards,
Hans

Hello,
I'm trying to checkout this specific version of mxnet you have mentioned, https://github.com/dmlc/mxnet/tree/62ecb60

I do the following steps:
git clone --recursive https://github.com/dmlc/mxnet
git checkout 62ecb60

and start compiling but nothing works, am I getting the sub modules somehow wrong ?

Can anyone share it on the other cloud-based driver? Like Baidu yun drive. Thanks!

AP for aeroplane = 77.89
AP for bicycle = 68.62
AP for bird = 68.83
AP for boat = 49.43
AP for bottle = 42.97
AP for bus = 79.79
AP for car = 67.78
AP for cat = 84.87
AP for chair = 41.89
AP for cow = 63.35
AP for diningtable = 37.02
AP for dog = 80.84
AP for horse = 74.19
AP for motorbike = 73.22
AP for person = 75.00
AP for pottedplant = 41.68
AP for sheep = 71.04
AP for sofa = 50.47
AP for train = 77.81
AP for tvmonitor = 63.24
Mean [email protected] = 64.50

AP for aeroplane = 64.60
AP for bicycle = 44.06
AP for bird = 55.21
AP for boat = 34.83
AP for bottle = 32.65
AP for bus = 75.09
AP for car = 55.12
AP for cat = 75.42
AP for chair = 23.91
AP for cow = 49.74
AP for diningtable = 21.99
AP for dog = 70.42
AP for horse = 51.64
AP for motorbike = 54.40
AP for person = 51.63
AP for pottedplant = 27.24
AP for sheep = 54.16
AP for sofa = 34.56
AP for train = 64.77
AP for tvmonitor = 53.27
Mean [email protected] = 49.74

I run demo.py with amazon EC2 instance, a remote computer, the terminal shows me that:

('use mxnet at', '/home/carnd/mxnet/python/mxnet/__init__.pyc')
{'BINARY_THRESH': 0.4,
 'CLASS_AGNOSTIC': True,
 'MASK_SIZE': 21,
 'MXNET_VERSION': 'mxnet',
 'SCALES': [(600, 1000)],
 'TEST': {'BATCH_IMAGES': 1,
          'CXX_PROPOSAL': False,
          'HAS_RPN': True,
          'ITER': 2,
          'MASK_MERGE_THRESH': 0.5,
          'MIN_DROP_SIZE': 2,
          'NMS': 0.3,
          'PROPOSAL_MIN_SIZE': 2,
          'PROPOSAL_NMS_THRESH': 0.7,
          'PROPOSAL_POST_NMS_TOP_N': 2000,
          'PROPOSAL_PRE_NMS_TOP_N': 20000,
          'RPN_MIN_SIZE': 2,
          'RPN_NMS_THRESH': 0.7,
          'RPN_POST_NMS_TOP_N': 300,
          'RPN_PRE_NMS_TOP_N': 6000,
          'USE_GPU_MASK_MERGE': True,
          'USE_MASK_MERGE': True,
          'test_epoch': 8},
 'TRAIN': {'ASPECT_GROUPING': True,
           'BATCH_IMAGES': 1,
           'BATCH_ROIS': -1,
           'BATCH_ROIS_OHEM': 128,
           'BBOX_MEANS': [0.0, 0.0, 0.0, 0.0],
           'BBOX_NORMALIZATION_PRECOMPUTED': True,
           'BBOX_REGRESSION_THRESH': 0.5,
           'BBOX_STDS': [0.2, 0.2, 0.5, 0.5],
           'BBOX_WEIGHTS': array([ 1.,  1.,  1.,  1.]),
           'BG_THRESH_HI': 0.5,
           'BG_THRESH_LO': 0,
           'BINARY_THRESH': 0.4,
           'CONVNEW3': True,
           'CXX_PROPOSAL': False,
           'ENABLE_OHEM': True,
           'END2END': True,
           'FG_FRACTION': 0.25,
           'FG_THRESH': 0.5,
           'FLIP': True,
           'GAP_SELECT_FROM_ALL': False,
           'IGNORE_GAP': False,
           'LOSS_WEIGHT': [1.0, 10.0, 1.0],
           'RESUME': False,
           'RPN_ALLOWED_BORDER': 0,
           'RPN_BATCH_SIZE': 256,
           'RPN_BBOX_WEIGHTS': [1.0, 1.0, 1.0, 1.0],
           'RPN_CLOBBER_POSITIVES': False,
           'RPN_FG_FRACTION': 0.5,
           'RPN_MIN_SIZE': 2,
           'RPN_NEGATIVE_OVERLAP': 0.3,
           'RPN_NMS_THRESH': 0.7,
           'RPN_POSITIVE_OVERLAP': 0.7,
           'RPN_POSITIVE_WEIGHT': -1.0,
           'RPN_POST_NMS_TOP_N': 300,
           'RPN_PRE_NMS_TOP_N': 6000,
           'SHUFFLE': True,
           'begin_epoch': 0,
           'end_epoch': 8,
           'lr': 0.0005,
           'lr_step': '5.33',
           'model_prefix': 'e2e',
           'momentum': 0.9,
           'warmup': True,
           'warmup_lr': 5e-05,
           'warmup_step': 250,
           'wd': 0.0005},
 'dataset': {'NUM_CLASSES': 81,
             'dataset': 'coco',
             'dataset_path': './data/coco',
             'image_set': 'train2014+valminusminival2014',
             'proposal': 'rpn',
             'root_path': './data',
             'test_image_set': 'test-dev2015'},
 'default': {'frequent': 20, 'kvstore': 'device'},
 'gpus': '0',
 'network': {'ANCHOR_RATIOS': [0.5, 1, 2],
             'ANCHOR_SCALES': [4, 8, 16, 32],
             'FIXED_PARAMS': ['conv1',
                              'bn_conv1',
                              'res2',
                              'bn2',
                              'gamma',
                              'beta'],
             'FIXED_PARAMS_SHARED': ['conv1',
                                     'bn_conv1',
                                     'res2',
                                     'bn2',
                                     'res3',
                                     'bn3',
                                     'res4',
                                     'bn4',
                                     'gamma',
                                     'beta'],
             'IMAGE_STRIDE': 0,
             'NUM_ANCHORS': 12,
             'PIXEL_MEANS': array([ 103.06,  115.9 ,  123.15]),
             'RCNN_FEAT_STRIDE': 16,
             'RPN_FEAT_STRIDE': 16,
             'pretrained': './model/pretrained_model/resnet_v1_101',
             'pretrained_epoch': 0},
 'output_path': '../output/fcis',
 'symbol': 'resnet_v1_101_fcis'}
[14:51:26] src/operator/convolution.cu:87: This convolution is not supported by cudnn, MXNET convolution is applied.
[14:51:26] src/operator/convolution.cu:87: This convolution is not supported by cudnn, MXNET convolution is applied.
[14:51:26] src/operator/convolution.cu:87: This convolution is not supported by cudnn, MXNET convolution is applied.
[14:51:26] src/operator/convolution.cu:87: This convolution is not supported by cudnn, MXNET convolution is applied.
[14:51:26] src/operator/convolution.cu:87: This convolution is not supported by cudnn, MXNET convolution is applied.
[14:51:26] src/operator/convolution.cu:87: This convolution is not supported by cudnn, MXNET convolution is applied.
(426, 640)
invalid device function
invalid device function
testing COCO_test2015_000000000275.jpg 0.5945s
[14:51:28] src/operator/convolution.cu:87: This convolution is not supported by cudnn, MXNET convolution is applied.
[14:51:28] src/operator/convolution.cu:87: This convolution is not supported by cudnn, MXNET convolution is applied.
[14:51:28] src/operator/convolution.cu:87: This convolution is not supported by cudnn, MXNET convolution is applied.
(427, 640)
invalid device function
invalid device function
testing COCO_test2015_000000001412.jpg 0.6275s
(427, 640)
invalid device function
invalid device function
testing COCO_test2015_000000073428.jpg 0.5950s
[14:51:29] src/operator/convolution.cu:87: This convolution is not supported by cudnn, MXNET convolution is applied.
[14:51:29] src/operator/convolution.cu:87: This convolution is not supported by cudnn, MXNET convolution is applied.
[14:51:29] src/operator/convolution.cu:87: This convolution is not supported by cudnn, MXNET convolution is applied.
(428, 640)
invalid device function
invalid device function
testing COCO_test2015_000000393281.jpg 0.6167s
done

I don't know what does invalid device function and [14:51:29] src/operator/convolution.cu:87: This convolution is not supported by cudnn, MXNET convolution is applied. mean, but it seems run correctly. But there are no files in folder FCIS/output.

So I want to see what does demo.py output, I add these codes into demo.py:
im = cv2.imread(cur_path + '/../demo/' + im_name) im = cv2.cvtColor(im, cv2.COLOR_BGR2RGB) pre_img = show_masks(im, dets, masks, classes, config, show = True) pre_img = pre_img.astype(np.uint8) pre_img = Image.fromarray(pre_img) pre_img.save("{}".format(im_name)) , in oder to save these output pictures, but there are no changes at all.

[sensetime@localhost mxnet]$ git submodule update

fatal: reference is not a tree: 89de7ab20167909bc2c4f8acd397671c47cf3c0d
Unable to checkout '89de7ab20167909bc2c4f8acd397671c47cf3c0d' in submodule path 'cub'

I don't know why it's like this

Hi first of all thanks for publishing your code!
Unfortunately I can not reproduce your reported running time (0.23s, single Maxwell Titan X GPU, mini-batch size 1).
I'm using a single GeForce GTX 1080 Ti (Pascal, mini-batch 1) and get 2.4329s. Measured with demo.py (but modified code such that batch-size is 1). I've expected an even better performance on this GPU...

Any ideas? How exactly have you measured the reported time?

Is this repository works with windows 7? windows 10?

In rpn.py there is an assertion about batch size:

assert len(roidb) == 1, 'Single batch only'

If I set BATCH_IMAGES > 1, the above assertion will fail. Is there any particular reason to restrict training to one image per device? Is training with 2 or more images per device not implemented in the current version of code?

I was wondering if there are any implications if I wanted to train like 2 or 3 images per device, as the memory consumption for training a single image per device is about 4GB so I assume a Titan X would be able to handle 2~3 images?

Thanks!

hello,
problem 1 :
for Preparation for Training & Testing step have three point .
can I only do the first one? ( the second and the third do not do it ).

problem 2 :
can I train my own dataset ?
if it can, how to do ?(I see the Preparation for Training & Testing step , but I don't know)

thanks

I try to download instances_valminusminival2014.json and instances_minival2014.json ,but the link that you gave isn't helpful,can you upload them into baiduyun? @leoxiaobin @orpine @ancientmooner

What is the meaning of pooled_size parameter (set to 21)?

If I understand well, the output of pooling is [B x R x C x G x G] where B is batch size, R is number of ROIs, C is number of channels and is equal to 2*num_classes, G is the group size and equal to 7.

Thanks!

And why CLASS_AGNOSTIC is set to True? I think we're not doing Object Segment Proposal, so it should be set to False..

i try to train the network on my own dataset .
and i get warning like this :

data "gt_boxes" has a shape (1L, 174L, 5L), which is larger than already allocated shape (1L, 100L, 5L). Need to re-allocate. Consider putting default_bucket_key to be the bucket taking the largest input for better memory sharing

someone know what is mean...

Hi all,

I've followed the README instruction to clone MXNet w/ the 62ecb60 commit along with following the advice provided by abduallahadel.

I also made sure to copy the contents of the ./fcis/operator_cxx/* to $(MY_MXNET_FOLDER)/src/operator/contrib and then compiled MXNet.

I was able to download the model to run demo.py and placed it in ./model/fcis_coco-0000.params

However, when I run the code:
python ./fcis/demo.py

I encounter a Segmentation fault (core dumped)

use mxnet at /usr/local/lib/python2.7/dist-packages/mxnet-0.9.5-py2.7.egg/mxnet/__init__.pyc
ctx_id [3]
{'BINARY_THRESH': 0.4,
 'CLASS_AGNOSTIC': True,
 'MASK_SIZE': 21,
 'MXNET_VERSION': 'mxnet',
 'SCALES': [(600, 1000)],
 'TEST': {'BATCH_IMAGES': 1,
          'CXX_PROPOSAL': False,
          'HAS_RPN': True,
          'ITER': 2,
          'MASK_MERGE_THRESH': 0.5,
          'MIN_DROP_SIZE': 2,
          'NMS': 0.3,
          'PROPOSAL_MIN_SIZE': 2,
          'PROPOSAL_NMS_THRESH': 0.7,
          'PROPOSAL_POST_NMS_TOP_N': 2000,
          'PROPOSAL_PRE_NMS_TOP_N': 20000,
          'RPN_MIN_SIZE': 2,
          'RPN_NMS_THRESH': 0.7,
          'RPN_POST_NMS_TOP_N': 300,
          'RPN_PRE_NMS_TOP_N': 6000,
          'USE_GPU_MASK_MERGE': True,
          'USE_MASK_MERGE': True,
          'test_epoch': 8},
 'TRAIN': {'ASPECT_GROUPING': True,
           'BATCH_IMAGES': 1,
           'BATCH_ROIS': -1,
           'BATCH_ROIS_OHEM': 128,
           'BBOX_MEANS': [0.0, 0.0, 0.0, 0.0],
           'BBOX_NORMALIZATION_PRECOMPUTED': True,
           'BBOX_REGRESSION_THRESH': 0.5,
           'BBOX_STDS': [0.2, 0.2, 0.5, 0.5],
           'BBOX_WEIGHTS': array([ 1.,  1.,  1.,  1.]),
           'BG_THRESH_HI': 0.5,
           'BG_THRESH_LO': 0,
           'BINARY_THRESH': 0.4,
           'CONVNEW3': True,
           'CXX_PROPOSAL': False,
           'ENABLE_OHEM': True,
           'END2END': True,
           'FG_FRACTION': 0.25,
           'FG_THRESH': 0.5,
           'FLIP': True,
           'GAP_SELECT_FROM_ALL': False,
           'IGNORE_GAP': False,
           'LOSS_WEIGHT': [1.0, 10.0, 1.0],
           'RESUME': False,
           'RPN_ALLOWED_BORDER': 0,
           'RPN_BATCH_SIZE': 256,
           'RPN_BBOX_WEIGHTS': [1.0, 1.0, 1.0, 1.0],
           'RPN_CLOBBER_POSITIVES': False,
           'RPN_FG_FRACTION': 0.5,
           'RPN_MIN_SIZE': 2,
           'RPN_NEGATIVE_OVERLAP': 0.3,
           'RPN_NMS_THRESH': 0.7,
           'RPN_POSITIVE_OVERLAP': 0.7,
           'RPN_POSITIVE_WEIGHT': -1.0,
           'RPN_POST_NMS_TOP_N': 300,
           'RPN_PRE_NMS_TOP_N': 6000,
           'SHUFFLE': True,
           'begin_epoch': 0,
           'end_epoch': 8,
           'lr': 0.0005,
           'lr_step': '5.33',
           'model_prefix': 'e2e',
           'momentum': 0.9,
           'warmup': True,
           'warmup_lr': 5e-05,
           'warmup_step': 250,
           'wd': 0.0005},
 'dataset': {'NUM_CLASSES': 81,
             'dataset': 'coco',
             'dataset_path': './data/coco',
             'image_set': 'train2014+valminusminival2014',
             'proposal': 'rpn',
             'root_path': './data',
             'test_image_set': 'test-dev2015'},
 'default': {'frequent': 20, 'kvstore': 'device'},
 'gpus': '3',
 'network': {'ANCHOR_RATIOS': [0.5, 1, 2],
             'ANCHOR_SCALES': [4, 8, 16, 32],
             'FIXED_PARAMS': ['conv1',
                              'bn_conv1',
                              'res2',
                              'bn2',
                              'gamma',
                              'beta'],
             'FIXED_PARAMS_SHARED': ['conv1',
                                     'bn_conv1',
                                     'res2',
                                     'bn2',
                                     'res3',
                                     'bn3',
                                     'res4',
                                     'bn4',
                                     'gamma',
                                     'beta'],
             'IMAGE_STRIDE': 0,
             'NUM_ANCHORS': 12,
             'PIXEL_MEANS': array([ 103.06,  115.9 ,  123.15]),
             'RCNN_FEAT_STRIDE': 16,
             'RPN_FEAT_STRIDE': 16,
             'pretrained': './model/pretrained_model/resnet_v1_101',
             'pretrained_epoch': 0},
 'output_path': '../output/fcis',
Segmentation fault (core dumped)

I'm adding prints to find out what the exact issue is and it seems to be in the function _bind_ith_exec in
fcis/core/DataParallelExecutorGroup.py where the code calls the following but I'm not sure how to resolve this issue:

executor = self.symbol.bind(ctx=context, args=arg_arrays,
                                    args_grad=grad_arrays, aux_states=aux_arrays,
                                    grad_req=self.grad_req, shared_exec=shared_exec)

Has anyone encounter this issue and knows how to fix it? Thanks for the help and suggestions in advance!

Hi,
1.i followed instruction to install mxnet and check the sample code. it was working fine.
2.i clone FCIS folder to my ubuntu system and run sh ./init.sh

~/Videos/FCIS$ sh ./init.sh

after running:

running build_ext
skipping 'bbox.c' Cython extension (up-to-date)
running build_ext
running build_ext
skipping 'cpu_nms.c' Cython extension (up-to-date)
skipping 'gpu_nms.cpp' Cython extension (up-to-date)
running build_ext
cythoning gpu_mv.pyx to gpu_mv.cpp

Error compiling Cython file:

...
cimport numpy as np

assert sizeof(int) == sizeof(np.int32_t)

cdef extern from "gpu_mv.hpp":
void _mv(np.float32_t* all_boxes, np.float32_t* all_masks, np.int32_t all_boxes_num, np.int32_t* candidate_inds, np.int32_t* candidate_start, np.float32_t* candidate_weights, np.int32_t candidate_num, np.float32_t binary_thresh, np.int32_t image_height, np.int32_t image_width, np.int32_t box_dim, np.int32_t mask_size, np.int32_t result_num, np.float32_t* result_mask, np.int32_t* result_box, np.int32_t device_id);
^

gpu_mv.pyx:14:419: Syntax error in C variable declaration
building 'gpu_mv' extension
/usr/local/cuda-8.0/bin/nvcc -I/usr/local/lib/python2.7/dist-packages/numpy/core/include -I/usr/local/cuda-8.0/include -I/usr/include/python2.7 -c mv_kernel.cu -o build/temp.linux-x86_64-2.7/mv_kernel.o -arch=sm_35 --ptxas-options=-v -c --compiler-options '-fPIC'
ptxas info : 0 bytes gmem
ptxas info : Compiling entry function '_Z15reduce_mask_rowiPKfiifPb' for 'sm_35'
ptxas info : Function properties for _Z15reduce_mask_rowiPKfiifPb
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Used 12 registers, 360 bytes cmem[0]
ptxas info : Compiling entry function '_Z17reduce_bounding_yiPKbPii' for 'sm_35'
ptxas info : Function properties for _Z17reduce_bounding_yiPKbPii
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Used 15 registers, 348 bytes cmem[0]
ptxas info : Compiling entry function '_Z14mask_aggregateiPKfPfPKiS3_S0_iif' for 'sm_35'
ptxas info : Function properties for _Z14mask_aggregateiPKfPfPKiS3_S0_iif
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Used 36 registers, 380 bytes cmem[0], 4 bytes cmem[2]
ptxas info : Compiling entry function '_Z17reduce_bounding_xiPKbPii' for 'sm_35'
ptxas info : Function properties for _Z17reduce_bounding_xiPKbPii
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Used 15 registers, 348 bytes cmem[0]
ptxas info : Compiling entry function '_Z11mask_renderiPKfS0_iiiiPf' for 'sm_35'
ptxas info : Function properties for _Z11mask_renderiPKfS0_iiiiPf
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Used 33 registers, 368 bytes cmem[0], 12 bytes cmem[2]
ptxas info : Compiling entry function '_Z11mask_resizeiPKfPKiS2_Pfiii' for 'sm_35'
ptxas info : Function properties for _Z11mask_resizeiPKfPKiS2_Pfiii
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Used 32 registers, 372 bytes cmem[0], 4 bytes cmem[2]
ptxas info : Compiling entry function '_Z15reduce_mask_coliPKfiifPb' for 'sm_35'
ptxas info : Function properties for _Z15reduce_mask_coliPKfiifPb
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Used 14 registers, 360 bytes cmem[0]
x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/local/lib/python2.7/dist-packages/numpy/core/include -I/usr/local/cuda-8.0/include -I/usr/include/python2.7 -c gpu_mv.cpp -o build/temp.linux-x86_64-2.7/gpu_mv.o -Wno-unused-function
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++ [enabled by default]
gpu_mv.cpp:1:2: error: #error Do not use this file, it is the result of a failed Cython compilation.
#error Do not use this file, it is the result of a failed Cython compilation.
^
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
(mxnet)hubino@hubino:~/Videos/FCIS$ sh ./init.sh
running build_ext
skipping 'bbox.c' Cython extension (up-to-date)
running build_ext
running build_ext
skipping 'cpu_nms.c' Cython extension (up-to-date)
skipping 'gpu_nms.cpp' Cython extension (up-to-date)
running build_ext
cythoning gpu_mv.pyx to gpu_mv.cpp

Error compiling Cython file:

...
cimport numpy as np

assert sizeof(int) == sizeof(np.int32_t)

cdef extern from "gpu_mv.hpp":
void _mv(np.float32_t* all_boxes, np.float32_t* all_masks, np.int32_t all_boxes_num, np.int32_t* candidate_inds, np.int32_t* candidate_start, np.float32_t* candidate_weights, np.int32_t candidate_num, np.float32_t binary_thresh, np.int32_t image_height, np.int32_t image_width, np.int32_t box_dim, np.int32_t mask_size, np.int32_t result_num, np.float32_t* result_mask, np.int32_t* result_box, np.int32_t device_id);
^

gpu_mv.pyx:14:419: Syntax error in C variable declaration
building 'gpu_mv' extension
/usr/local/cuda-8.0/bin/nvcc -I/usr/local/lib/python2.7/dist-packages/numpy/core/include -I/usr/local/cuda-8.0/include -I/usr/include/python2.7 -c mv_kernel.cu -o build/temp.linux-x86_64-2.7/mv_kernel.o -arch=sm_35 --ptxas-options=-v -c --compiler-options '-fPIC'
ptxas info : 0 bytes gmem
ptxas info : Compiling entry function '_Z15reduce_mask_rowiPKfiifPb' for 'sm_35'
ptxas info : Function properties for _Z15reduce_mask_rowiPKfiifPb
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Used 12 registers, 360 bytes cmem[0]
ptxas info : Compiling entry function '_Z17reduce_bounding_yiPKbPii' for 'sm_35'
ptxas info : Function properties for _Z17reduce_bounding_yiPKbPii
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Used 15 registers, 348 bytes cmem[0]
ptxas info : Compiling entry function '_Z14mask_aggregateiPKfPfPKiS3_S0_iif' for 'sm_35'
ptxas info : Function properties for _Z14mask_aggregateiPKfPfPKiS3_S0_iif
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Used 36 registers, 380 bytes cmem[0], 4 bytes cmem[2]
ptxas info : Compiling entry function '_Z17reduce_bounding_xiPKbPii' for 'sm_35'
ptxas info : Function properties for _Z17reduce_bounding_xiPKbPii
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Used 15 registers, 348 bytes cmem[0]
ptxas info : Compiling entry function '_Z11mask_renderiPKfS0_iiiiPf' for 'sm_35'
ptxas info : Function properties for _Z11mask_renderiPKfS0_iiiiPf
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Used 33 registers, 368 bytes cmem[0], 12 bytes cmem[2]
ptxas info : Compiling entry function '_Z11mask_resizeiPKfPKiS2_Pfiii' for 'sm_35'
ptxas info : Function properties for _Z11mask_resizeiPKfPKiS2_Pfiii
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Used 32 registers, 372 bytes cmem[0], 4 bytes cmem[2]
ptxas info : Compiling entry function '_Z15reduce_mask_coliPKfiifPb' for 'sm_35'
ptxas info : Function properties for _Z15reduce_mask_coliPKfiifPb
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Used 14 registers, 360 bytes cmem[0]
x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/local/lib/python2.7/dist-packages/numpy/core/include -I/usr/local/cuda-8.0/include -I/usr/include/python2.7 -c gpu_mv.cpp -o build/temp.linux-x86_64-2.7/gpu_mv.o -Wno-unused-function
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++ [enabled by default]
gpu_mv.cpp:1:2: error: #error Do not use this file, it is the result of a failed Cython compilation.
#error Do not use this file, it is the result of a failed Cython compilation.
^
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

how can i resolve the issue.

when I run the trainning of coco data, it reports "This convolution is not supported by cudnn, MXNET convolution is applied."
but, when I run the demo, it does not reports this.
it is very slow.
Can you help me?

Any plans to upgrade this repo to CUDA 8.0 ?

.data/coco/images/train2014/ ... ...
.data/coco/annotations/instances_valminusminival2014.json
.data/coco/annotations/instances_minival2014.json

I put the train set into the path like this style, but it show the error "the path doen not exist". Somebody who can help me, thx pretty much .

“AssertionError: Path does not exist: ./data/coco/images/val2014/COCO_val2014_000000262148.jpg“

when i use jupyter in server, it can't support the cv2.imshow function . sombody who can tell me how to rewrite the fuction "show_masks(im, dets, masks, classes, config) " to just save the processed image save to one path. Because i can't visualize the processed image on server.