daijifeng001 / r-fcn

R-FCN: Object Detection via Region-based Fully Convolutional Networks

R-FCN: Object Detection via Region-based Fully Convolutional Networks

By Jifeng Dai, Yi Li, Kaiming He, Jian Sun

It is highly recommended to use the deformable R-FCN implemented in MXNet, which significantly increases the accuracy at very low extra computational overhead.

A python version of R-FCN is available, which supports end-to-end training/inference of R-FCN for object detection.

Introduction

R-FCN is a region-based object detection framework leveraging deep fully-convolutional networks, which is accurate and efficient. In contrast to previous region-based detectors such as Fast/Faster R-CNN that apply a costly per-region sub-network hundreds of times, our region-based detector is fully convolutional with almost all computation shared on the entire image. R-FCN can natually adopt powerful fully convolutional image classifier backbones, such as ResNets, for object detection.

R-FCN was initially described in a NIPS 2016 paper.

This code has been tested on Windows 7/8 64 bit, Windows Server 2012 R2, and Ubuntu 14.04, with Matlab 2014a.

License

R-FCN is released under the MIT License (refer to the LICENSE file for details).

Citing R-FCN

If you find R-FCN useful in your research, please consider citing:

@article{dai16rfcn,
    Author = {Jifeng Dai, Yi Li, Kaiming He, Jian Sun},
    Title = {{R-FCN}: Object Detection via Region-based Fully Convolutional Networks},
    Journal = {arXiv preprint arXiv:1605.06409},
    Year = {2016}
}

Main Results

	training data	test data	mAP	time/img (K40)	time/img (Titian X)
R-FCN, ResNet-50	VOC 07+12 trainval	VOC 07 test	77.4%	0.12sec	0.09sec
R-FCN, ResNet-101	VOC 07+12 trainval	VOC 07 test	79.5%	0.17sec	0.12sec

Requirements: software

0. Caffe build for R-FCN (included in this repository, see external/caffe)
   • If you are using Windows, you may download a compiled mex file by running fetch_data/fetch_caffe_mex_windows_vs2013_cuda75.m
   • If you are using Linux or you want to compile for Windows, please recompile our Caffe branch.
1. MATLAB 2014a or later

Requirements: hardware

GPU: Titan, Titan X, K40, K80.

Demo

0. Run fetch_data/fetch_caffe_mex_windows_vs2013_cuda75.m to download a compiled Caffe mex (for Windows only).
1. Run fetch_data/fetch_demo_model_ResNet101.m to download a R-FCN model using ResNet-101 net (trained on VOC 07+12 trainval).
2. Run rfcn_build.m.
3. Run startup.m.
4. Run experiments/script_rfcn_demo.m to apply the R-FCN model on demo images.

Preparation for Training & Testing

0. Run fetch_data/fetch_caffe_mex_windows_vs2013_cuda75.m to download a compiled Caffe mex (for Windows only).
1. Run fetch_data/fetch_model_ResNet50.m to download an ImageNet-pre-trained ResNet-50 net.
2. Run fetch_data/fetch_model_ResNet101.m to download an ImageNet-pre-trained ResNet-101 net.
3. Run fetch_data/fetch_region_proposals.m to download the pre-computed region proposals.
4. Download VOC 2007 and 2012 data to ./datasets.
5. Run rfcn_build.m.
6. Run startup.m.

Training & Testing

0. Run experiments/script_rfcn_VOC0712_ResNet50_OHEM_ss.m to train a model using ResNet-50 net with online hard example mining (OHEM), leveraging selective search proposals. The accuracy should be ~75.4% in mAP.
   • Note: the training time is ~13 hours on Titian X.
1. Run experiments/script_rfcn_VOC0712_ResNet50_OHEM_rpn.m to train a model using ResNet-50 net with OHEM, leveraging RPN proposals (using ResNet-50 net). The accuracy should be ~77.4% in mAP.
   • Note: the training time is ~13 hours on Titian X.
2. Run experiments/script_rfcn_VOC0712_ResNet101_OHEM_rpn.m to train a model using ResNet-101 net with OHEM, leveraging RPN proposals (using ResNet-101 net). The accuracy should be ~79.5% in mAP.
   • Note: the training time is ~19 hours on Titian X.
3. Check other scripts in ./experiments for more settings.

Note:

• In all the experiments, training is performed on VOC 07+12 trainval, and testing is performed on VOC 07 test.
• Results are subject to some random variations. We have run 'experiments/script_rfcn_VOC0712_ResNet50_OHEM_rpn.m' for 5 times, the results are 77.1%, 77.3%, 77.7%, 77.9%, and 77.0%. The mean is 77.4%, and the std is 0.39%.
• Running time is not recorded in the test log (which is slower), but instead in an optimized implementation.

Resources

0. Experiment logs: OneDrive, BaiduYun

If the automatic "fetch_data" fails, you may manually download resouces from:

0. Pre-complied caffe mex (Windows):
   • OneDrive, BaiduYun
1. Demo R-FCN model:
   • OneDrive, BaiduYun
2. ImageNet-pretrained networks:
   • ResNet-50 net OneDrive, BaiduYun
   • ResNet-101 net OneDrive, BaiduYun
3. Pre-computed region proposals:
   • OneDrive, BaiduYun

r-fcn's Issues

How to test image

I had trained the model with ResNet50

How can I run the detection of one image end-to-end?

Thank you very much

Running demo code would be crash in Matlab2016?

Running demo code steps 1,2,3,4 is OK but steps 5 would be crash.

I have no idea that steps 5 crash because of notebook GPU?

Matlab would show "Matlab has encounterr an internal problem and need to close" and shut down.

I set interrupt point and found that it would be crash in steps "WARM UP" in "script_rfcn_demo" so the problem is insufficien RAM or not powerful GPU?

The compiler environment is win7(64bits) , VS2013(profession) , Matlab2016 .

Thanks

do I need add the OHEM layers to the first-stage rpn prototxt

Dear daijifeng:
when I write the first-stage rpn prototxt, do I need add the OHEM layers to the first-stage rpn prototxt?
(I want use OHEM to do hard examples mining).

By the way, Is it possible to share your rpn prototxt to me? ^_^

Why the highest training accuracy happened in Iteration 0 ?

Now I am training the 4th R-FCN stage (totally 4 stage, with OHEM, 229 category LOGOs), and
I met a situation that the highest training accuracy(0.949) happened in Iteration 0 (that means just
use the stage 2' model). And after training several iteration(max 110000 as the author set), the final accuray
is still lower than Iteration 0.

I feel confuse that Why this happens..., if this always happens, we even do not need the last training stage..., this problem bother me a lot......, help..

I try 3 kind of base learning rate: 0.01, 0.001, 0.0001. the situation I mentioned just now is almost same. just like this:

------------------------- Iteration 0 -------------------------
Training : # accuracy 0.949, loss (cls 0.124, reg 0.243)
Testing : accuracy 0.926, loss (cls 0.201, reg 0.223)

------------------------- Iteration 2000 -------------------------
Training : accuracy 0.938, loss (cls 0.148, reg 0.0596)
Testing : accuracy 0.927, loss (cls 0.2, reg 0.0525)

------------------------- Iteration 4000 -------------------------
Training : accuracy 0.938, loss (cls 0.147, reg 0.0418)
Testing : accuracy 0.927, loss (cls 0.199, reg 0.0504)

------------------------- Iteration 6000 -------------------------
Training : accuracy 0.94, loss (cls 0.144, reg 0.0399)
Testing : accuracy 0.927, loss (cls 0.198, reg 0.0497)

.....
.....

------------------------- Iteration 98000 -------------------------
Training : accuracy 0.943, loss (cls 0.135, reg 0.0315)
Testing : accuracy 0.929, loss (cls 0.193, reg 0.0431)

------------------------- Iteration 100000 -------------------------
Training : accuracy 0.943, loss (cls 0.137, reg 0.0317)
Testing : accuracy 0.929, loss (cls 0.193, reg 0.0431)

------------------------- Iteration 102000 -------------------------
Training : # accuracy 0.943, loss (cls 0.134, reg 0.0317)
Testing : accuracy 0.929, loss (cls 0.193, reg 0.0429)

....
....

how to generate proposal?

I was running the demo code:
https://github.com/daijifeng001/R-FCN/blob/master/experiments/script_rfcn_demo.m

the proposal are loaded from file:
proposals = load(fullfile(demo_dir, [im_names{j}, '_boxes.mat']));

How can I run the detection of one image end-to-end?

Thank you very much

Fine Tuning

How to fine tune your model? I don't have sufficient data to retrain your model from scratch.I want to fine tune your model on my data which has only two classes ?

when i train model this is some errors

Hi dai：
错误使用 parpool (line 111)
Failed to start a parallel pool. (For information in addition to the causing error, validate the
profile 'local' in the Cluster Profile Manager.)

出错 new_parpool (line 10)
p = parpool(number);

出错 rfcn_train (line 94)
p = new_parpool(1);

出错 script_rfcn_VOC0712_ResNet101_rpn (line 40)
opts.rfcn_model = rfcn_train(conf, dataset.imdb_train, dataset.roidb_train, ...

原因:
错误使用 parallel.internal.pool.InteractiveClient/start (line 330)
Failed to start pool.
错误使用 parallel.Job/submit (line 304)
所有维度参数必须大于零

python version

Hi, Dai

I don't have an access of Matlab so it is not possible for me to experiment your code.
Do you have a plan to convert matlab script to python?

Thanks

Does it support to be trained by multiple GPUs?

In your paper, you mention that you trained the model by 8 GPUs,
But I notice that this repository does not support to be trained in this way.

loading datasets error

Loading dataset...错误使用 textread (line 165)
未找到文件。

出错 imdb_from_voc (line 46)
imdb.image_ids = textread(sprintf(VOCopts.imgsetpath, image_set), '%s');

出错 Dataset.voc0712_trainval_sp (line 12)
dataset.imdb_train = { imdb_from_voc(devkit2007, 'trainval', '2007', use_flip), ...

出错 script_rfcn_VOC0712_ResNet101_rpn (line 31)
dataset = Dataset.voc0712_trainval_sp(dataset, 'train', conf.use_flipped, 'resnet101');

Can not download your proposal file

I try to download your proposal file in BaiduYun, proposal file in baiduyun , but it seems like the link has broken
Upload a new one? Thank you!

why the bbox_targets has 8 dim not 4?

why the bbox_targets have 8 dim, not 4?
and the begin 4 dim is all 0?

Is there any plan to realize these CPU parts?

SoftmaxWithLossLayer::Forward_cpu in src/caffe/layers/softmax_loss_layer.cpp is NOT_IMPLEMENTED, while
./src/caffe/layers/box_annotator_ohem_layer.cpp
./src/caffe/layers/psroi_pooling_layer.cpp
./src/caffe/layers/smooth_L1_loss_layer.cpp
are the same.

Is there any plan to realize these CPU part?

make runtest error

because i cannot train the model, so I tried to exam the caffe first.
Because you don't implement the cpu code, so I cancel the cpu test.
But each time I did the forward in BatchNormLayer, it will show Check failed: data_
detail are shown as follows:

[----------] 3 tests from BatchNormLayerTest/1, where TypeParam = caffe::GPUDevice<double>
[ RUN      ] BatchNormLayerTest/1.TestGradient
F0826 15:31:45.527961 20768 blob.cpp:121] Check failed: data_ 
*** Check failure stack trace: ***
    @     0x7f4d6d476b4d  google::LogMessage::Fail()
    @     0x7f4d6d47ab67  google::LogMessage::SendToLog()
    @     0x7f4d6d4789e9  google::LogMessage::Flush()
    @     0x7f4d6d478ced  google::LogMessageFatal::~LogMessageFatal()
    @     0x7f4d65d04ac9  caffe::Blob<>::mutable_gpu_data()
    @     0x7f4d65e6fb82  caffe::BatchNormLayer<>::Forward_gpu()
    @           0x55fdf6  caffe::Layer<>::Forward()
    @           0x561a2f  caffe::GradientChecker<>::CheckGradientSingle()
    @           0x55fb3e  caffe::GradientChecker<>::CheckGradientExhaustive()
    @           0x814052  caffe::BatchNormLayerTest_TestGradient_Test<>::TestBody()
    @           0x837557  testing::internal::HandleSehExceptionsInMethodIfSupported<>()
    @           0x8329bc  testing::internal::HandleExceptionsInMethodIfSupported<>()
    @           0x81ecbe  testing::Test::Run()
    @           0x81f4f8  testing::TestInfo::Run()
 create mode 100644 windows/scripts/PythonPreBuild.cmd
 create mode 100644 windows/test_all/packages.config
    @           0x81fb2c  testing::TestCase::Run()
    @           0x824eb9  testing::internal::UnitTestImpl::RunAllTests()
    @           0x838588  testing::internal::HandleSehExceptionsInMethodIfSupported<>()
    @           0x833665  testing::internal::HandleExceptionsInMethodIfSupported<>()
    @           0x823a41  testing::UnitTest::Run()

Hope you can help me to solve this issue

Error when running "script_rfcn_demo.m"?

Hi, when i run the Code "script_rfcn_demo.m" ,there is the error as follows, (my platform is win64 + matlab2014a)
Invalid MEX-file 'D:\FCN\R-FCN-master\external\caffe\matlab\caffe_rfcn+caffe\private\caffe_.mexw64': D:\FCN\R-FCN-master\external\caffe\matlab\caffe_rfcn+caffe\private\caffe_.mexw64 不是有效的 Win32 应用程序。

出错 caffe.set_device (line 9)
caffe_('set_device', device_id);

出错 active_caffe_mex (line 26)
caffe.set_device(gpu_id-1);

出错 script_rfcn_demo (line 21)
active_caffe_mex(opts.gpu_id, opts.caffe_version);

Training on MSCOCO dataset

Could you give an example on how to training on MSCOCO dataset? Thank you.

about the dim of bbox_pred

Dear Dai,
the dim of bbox_pred should be R_(21_4) rather than R*8, as your prototxt shows
because the later applies,

 boxes_cell = cell(length(classes), 1);
    thres = 0.6;
    for i = 1:length(boxes_cell)
        boxes_cell{i} = [boxes(:, (1+(i-1)*4):(i*4)), scores(:, i)];
        boxes_cell{i} = boxes_cell{i}(nms(boxes_cell{i}, 0.3), :);

        I = boxes_cell{i}(:, 5) >= thres;
        boxes_cell{i} = boxes_cell{i}(I, :);
    end

I wonder what's wrong with these code.

The Test Time

from the demo
proposals = load(fullfile(demo_dir, [im_names{j}, '_boxes.mat']));
seems the test time exclude the proposal generating time ? Can you give the testing time with rpn generating, Tks

About the caffe version

Is it this code support cuda 8.0 and GTX10*0 serial?

Are there pretrained ResNet-18 and ResNet-34 for Caffe?

Hi @daijifeng001, hi guys

I'm trying R-FCN. However, due to the limit of my GPU card, I need to use smaller net such as resnet-18 and resnet-34.

I want to ask are there pretrained caffe models of resnet-18 and resnet-34?

I see there are Torch pretrained models of these nets at https://github.com/facebook/fb.resnet.torch/tree/master/pretrained, but I am using Caffe, so please help.

prototxt files are also required.

Thanks,

There seems we do not benefit from OHEM module?

Dear jifei:
When I download your training experiment.zip from BaiduYun, I find that it seems we do NOT benefit from OHEM module. Actually it hurt the accuracy:
For example, in your experiment log:

Without OHEM: rfcn_VOC0712_ResNet101_rpn_resnet101
------------------------- Iteration 105000 -------------------------
Training : accuracy 0.922, loss (cls 0.195, reg 0.0867)
Testing : accuracy 0.892, loss (cls 0.284, reg 0.126)

With OHEM: rfcn_VOC0712_ResNet101_OHEM_rpn_resnet101
------------------------- Iteration 105000 -------------------------
Training : accuracy 0.898, loss (cls 0.25, reg 0.118)
Testing : accuracy 0.858, loss (cls 0.366, reg 0.162)

So, I feel confuse about such result, Would you please.... Why OHEM does not work...

learning rate schedule questions

First off, thanks a lot for the great work and code release. For Lack of a better place to ask, these are more of implementation questions than actual issues. My apologies!

1. The NIPS paper states (Pg.4) that R-FCN is finetuned with LR=1e-3 for 20k minibatches and then LR=1e-4 for 10k on VOC. So that's a 20k30k schedule. However, this doesn't seem to match the prototxts included here, which are 80k110k.
2. I wonder if you would have LR schedule recommendations for other training sets, eg. just VOC'07. I'm guessing the 30k40k schedule from fast(er)-rcnn can still be successful here, but I haven't tried. If you could comment on that, this information can be helpful for many people :)

Thanks in advance!

Can't run the demo and the Baiduyun link is not exist for Pre-computed region proposals

Hi
When I tried to run the demo ,I met the error : ./R-FCN-master/experiments/external/caffe/matlab/caffe_rfcn) is not exist.
I have done as the steps you mentioned.

运行script_rfcn_demo.m时出现错误，matlab崩溃

你好！感谢分享！运行script_rfcn_demo.m时出现错误，matlab崩溃。
我使用的是MATLAB 2015b，GTX770（2G）
请问可能的错误是什么？
请问运行script_rfcn_demo.m时需要更大的内存，显存吗？

Windows Matlab code CPU version

Hi
Is it possible for a computer without GPU using Windows to run the demo code using Matlab?

Under experiment\script_rfcn_demo.m, even if I change the opts.use_gpu = false, it still doesn't work. Can you tell me what should I modify in order to make this work please?

Best

RPN proposals

Hi Dai
This package assumes the proposals available beforehand. Isnt it??
So like the faster-rcnn one this package cannot finetune for both the rpn and detection network right?

About the input data in training

Hi all,

1. In train_val.prototxt, I notice that the input data such as label is 1 * 5 * 1 * 1, the dim 5 means [batch_ind, x1, y1, x2, y2]. I wonder what dose the batch_ind means. Dose it mean the rois' index or others?
2. Also in train_val.prototxt, the bbox_targets and bbox_loss_weights are both 1 * 8 * 1 * 1. Why not 1 * 4 * 1 * 1?
3. If I wonder input multi rois for training, how should I change the input data?

Hope someone can give me some advice, thanks!

F0824 09:52:00.273586 48097 cudnn_relu_layer.cu:24] Check failed: status == CUDNN_STATUS_SUCCESS (3 vs. 0) CUDNN_STATUS_BAD_PARAM

When I run the demo , It shows that:
F0824 09:52:00.273586 48097 cudnn_relu_layer.cu:24] Check failed: status == CUDNN_STATUS_SUCCESS (3 vs. 0) CUDNN_STATUS_BAD_PARAM

please help~

How to get voc2007 and voc 2012

is this datasets files include voc2007 and voc2012 ? can you help me?

DEPRECATED: ForwardPrefilled() will be removed in a future version. Use Forward().

When I run experiments/script_rfcn_demo.m
then the terminal shows:
DEPRECATED: ForwardPrefilled() will be removed in a future version. Use Forward().
then the Matlab crashed

Error when run script_rfcn_VOC0712_ResNet101_OHEM_rpn.m

[libprotobuf ERROR google/protobuf/descriptor_database.cc:57] File already exists in database: caffe.proto
[libprotobuf FATAL google/protobuf/descriptor.cc:954] CHECK failed: generated_database_->Add(encoded_file_descriptor, size):

Could anybody kindly tell me what's wrong with it? thanks a lot.

Besides, What should I do if I want to train R-FCN on my own dataset? I'm confused about how to get the pre-computed region proposals.

What is the purpose of spatial_scale ?

What is the meaning of spatial_scale parameter of the PSRoiPooli layer and how do you set its value ?

Can i change mydatesets

Hi Dai:

Can you tell me how to change my dataset ? Or give some other suggestions?

Thanks!

Best

how to run the test demo in c++

hi,now i want to run the demo,just test,in c++,can you give some advice
tks

how to use rpn for detection as your paper described?

I wanna use rpn for multi class detection, and your paper give a good result, so can you release it? Thanks

cant download file in fetch_data from dropbox or BaiduYun

I run all .m file in fetch_data folder and there is the same error "Error in downloading, please try links in README.md https://github.com/daijifeng001/R-FCN"

I open the matlab and found the dropbox account . The message in the dropbox account is "This account's links are generating too much traffic and have been temporarily disabled!"

I download from BaiduYun cloud but the zip file is broken .

Is there problem of dropbox account or just the compiler processing in matlab going wrong?

Thank for helping

crash when train R-FCN with Resnet50/101

1. The script_rfcn_demo.m is work well with Resnet101
2. But the matlab go crash when running with Resnet50/101, the code is down at line 117 "caffe_solver.step(1)" of rfcn_train.m, without any message. It seems to be the reason for bn layers.
3. I wrote a solver with VGG16, everything is ok

Can you help me ?Thank you!
@daijifeng001

Do I need 9 anchors during I implement RPN to generate proposals?

Dear daijifeng:

I want to implement RPN to generate proposals when do the 4-step training(just like faster-rcnn).

I confuse that Do I need the same anchor boxes as faster RCNN? (i.e. 9 anchors [0.5:1,1:1,1:2]x[128 256 512]) ?

about dilate convolution

Dear daijifeng:
I don't understand why only the res5a_branch2b, res5b_branch2b, res5c_branch2b convolution layer has dilation parameter? what is the point of do that? Thank you!

About the spatial_scale parameter's meaning in prototxt file

I have implement the forward_cpu function in your psroipooling_layer, but the results are strange, I wonder what is the meaning of the parameter named spatial_scale.

ask for sharing edge box data

Could you please share some codes and data used to enable edge box proposals in your experiments (the last table in Sect. 4.1 in Page 8 of the NIPS 2016 paper)?

I modified the codes in script_rfcn_VOC0712_ResNet50_OHEM_ss.m, voc0712_trainval_ss.m, and voc2007_test_ss.m by simply substituting "with_selective_search" with "with_edge_box". As there is no "edge_box_data" folder made publicly available, I employed the data provided at https://github.com/gidariss/LocNet. I only have ~11..4% training accuracy ~9% test accuracy in training stage, a pretty large gap with 77.8% reported in your paper.

ask for help again

hi dai ：
when i train completed. testing some errors .can you tell me where this questions. 3q
Starting parallel pool (parpool) using the 'local' profile ... connected to 12 workers.
{�错误使用 matlab:helpUtils.errorDocCallback('imdb_eval_voc', 'G:\Muhz\R-FCN-master\imdb\imdb_eval_voc.m', 63)
文件标识符无效。使用 fopen 生成有效的文件标识符。

出错 rfcn_test>(parfor body)
res(model_ind) = imdb.eval_func(cls, aboxes{model_ind}, imdb, opts.cache_name,
opts.suffix);

出错 <a href="matlab:helpUtils.errorDocCallback('rfcn_test', 'G:\Muhz\R-FCN-master\functions\rfcn\rfcn_test.m', 169)"
parfor model_ind = 1:num_classes

出错 )
rfcn_test(conf, dataset.imdb_test, dataset.roidb_test, ...
}�

How to generate .mat file in demo image?

I found that each demo image in demo folder has its own .mat but i dont know how to generate .mat for other image.

I want to testing other image using "script_rfcn_demo" .

How to generate .mat file for other image ? Dose .mat file generate from training processing ?

I use demo .mat file to testing other image (not demo image) and it can be tested .Is that accurancy for testing ?

Thanks.

Question about training

Hi, I've run the domo just fine, but when I intend to try training R-FCN, it seems that the program is always stuck in imdb_from_voc and couldn't process. So I wonder what kind of problem would that be and is there anyone else training the model well? Could you please be nice and tell me what are the possible mistakes I have made that causes this issue? Or is that an issue or not?

Thank you so much for your kind help.

Proposal generation error

Dear Dai,
I tried to train the network end-to-end (i.e. add region proposal part similarly to faster-rcnn) but the results are not nearly as good as the ones presented in your paper (although not junk so it isn't an obvious bug). I assume my implementation isn't correct. Can you please release the full training code you used (i.e. region proposal and detection)?

Thanks,
Ethan

关于运行script_rfcn_demo.m失败

你好！感谢分享！依照Readme的说明，运行script_rfcn_demo.m时出现错误，matlab提示：
**Error using CHECK_FILE_EXIST (line 4)
E:\R-FCN-master\output\rfcn_demo\rfcn_VOC0712_ResNet101_OHEM_rpn_resnet101\final does not exist

Error in caffe.Net/copy_from (line 166)
CHECK_FILE_EXIST(weights_file);

Error in script_rfcn_demo (line 39)
caffe_net.copy_from(rfcn_net);**
根据提示，说final这个文件不存在，我知道output文件夹是在运行rfcn-build.m时产生的，但final文件需要自己添加的？还是说是程序生成的。这个问题如何解决？谢谢。
如果可以，能否通过邮件回答我，[email protected]，不胜感激。

About position-sensitive score maps

How the position-sensitive score maps generate? I don not find anything in detail in paper and author just say that use a bank of specialized convolutional layers as the FCN output.

Thank you very much.

Windows version

HI @daijifeng001
when I use R-FCN for windows. when I Run rfcn_build.m (I have got the caffe mex for windwos first),something wrong happened . Error message is as follows
Error using mex
Microsoft (R) Manifest Tool version 6.3.9600.17298
Copyright (c) Microsoft Corporation 2012.
All rights reserved.
mt : general error c101008d: Failed to write the updated manifest to the resource of file
"H:\R-FCN-master\bin\nms_mex.mexw64". The operation failed.
Error in rfcn_build (line 14)
mex -O -outdir bin ...
My running environment : matlab2015a vs2013 cuda7.5 gpu:NVIDIA Quadro K2100M
Eager for your kind reply
Yamin

implement R-FCN with py-faster-rcnn

Hi, I'm trying to implement R-FCN in py-faster-rcnn, but encounter serval issues,
I make these changes with py-faster-rcnn:

1. As smoothL1Loss in R-FCN is different with py-faster-rcnn, so i set the py-faster-rcnn loss as a new type of loss.
2. move PSROIPooling and BoxAnnotatorOHEM to py-faster-rcnn
   And changes with R-FCN:
3. change rfcn_bbox output channel to 42149=4116 since fasterrcnn use a class specific bbox regresison

Now I am able to re-compie caffe and train rpn net, but when traing fast-rcnn, I got a bug:

I0721 12:24:04.370088 12785 layer_factory.hpp:77] Creating layer per_roi_loss_cls
I0721 12:24:04.370101 12785 net.cpp:106] Creating Layer per_roi_loss_cls
I0721 12:24:04.370105 12785 net.cpp:454] per_roi_loss_cls <- cls_score_ave_cls_score_rois_0_split_0
I0721 12:24:04.370110 12785 net.cpp:454] per_roi_loss_cls <- labels_data_2_split_0
I0721 12:24:04.370113 12785 net.cpp:411] per_roi_loss_cls -> temp_loss_cls
I0721 12:24:04.370120 12785 net.cpp:411] per_roi_loss_cls -> temp_prob_cls
I0721 12:24:04.370124 12785 net.cpp:411] per_roi_loss_cls -> per_roi_loss_cls
I0721 12:24:04.370139 12785 layer_factory.hpp:77] Creating layer per_roi_loss_cls
I0721 12:24:04.370525 12785 net.cpp:150] Setting up per_roi_loss_cls
I0721 12:24:04.370537 12785 net.cpp:157] Top shape: (1)
I0721 12:24:04.370553 12785 net.cpp:157] Top shape: 1 21 1 1 (21)
I0721 12:24:04.370556 12785 net.cpp:157] Top shape: 1 (1)
I0721 12:24:04.370559 12785 net.cpp:165] Memory required for data: 678497932
I0721 12:24:04.370563 12785 layer_factory.hpp:77] Creating layer per_roi_loss_bbox
I0721 12:24:04.370575 12785 net.cpp:106] Creating Layer per_roi_loss_bbox
I0721 12:24:04.370579 12785 net.cpp:454] per_roi_loss_bbox <- bbox_pred_ave_bbox_pred_rois_0_split_0
I0721 12:24:04.370584 12785 net.cpp:454] per_roi_loss_bbox <- bbox_targets_data_3_split_0
I0721 12:24:04.370589 12785 net.cpp:454] per_roi_loss_bbox <- bbox_inside_weights_data_4_split_0
I0721 12:24:04.370595 12785 net.cpp:411] per_roi_loss_bbox -> temp_loss_bbox
I0721 12:24:04.370601 12785 net.cpp:411] per_roi_loss_bbox -> per_roi_loss_bbox
I0721 12:24:04.370652 12785 net.cpp:150] Setting up per_roi_loss_bbox
I0721 12:24:04.370658 12785 net.cpp:157] Top shape: (1)
I0721 12:24:04.370662 12785 net.cpp:157] Top shape: 1 1 1 1 (1)
I0721 12:24:04.370664 12785 net.cpp:165] Memory required for data: 678497940
I0721 12:24:04.370667 12785 layer_factory.hpp:77] Creating layer per_roi_loss
I0721 12:24:04.370679 12785 net.cpp:106] Creating Layer per_roi_loss
I0721 12:24:04.370682 12785 net.cpp:454] per_roi_loss <- per_roi_loss_cls
I0721 12:24:04.370688 12785 net.cpp:454] per_roi_loss <- per_roi_loss_bbox
I0721 12:24:04.370692 12785 net.cpp:411] per_roi_loss -> per_roi_loss
F0721 12:24:04.370698 12785 eltwise_layer.cpp:34] Check failed: bottom[i]->shape() == bottom[0]->shape() 1 0 0 0
*** Check failure stack trace: ***

thanks for you help!

Always need proposal for test？

In 'data/demo' there are .mat files indicating the proposals(used in your demo script), why do we need pre-computed proposals for detecting objects in a image, can we generate them in the RPN automatically?