vlfeat / matconvnet-fcn

A MatConvNet-based implementation of the Fully-Convolutional Networks for image segmentation

MatConvNet implementation of the FCN models for semantic segmentation

This package contains an implementation of the FCN models (training and evaluation) using the MatConvNet library.

For training, look at the fcnTrain.m script, and for evaluation at fcnTest.m. The script fcnTestModelZoo.m is designed to test third party networks imported in MatConvNet (mainly from Caffe).

While we are still tuning parameters, on the PASCAL VOC 2011 validation data subset used in the FCN paper, this code has been used to train networks with this performance:

Model	Test data	Mean IOU	Mean pix. accuracy	Pixel accuracy
FCN-32s (ours)	RV-VOC11	60.80	89.61	75.49
FCN-16s (ours)	RV-VOC11	62.25	90.08	77.81
FCN-8s (ours)	RV-VOC11	in prog.	in prog.	in prog.
FNC-32s (orig.)	RV-VOC11	59.43	89.12	73.28
FNC-16s (orig.)	RV-VOC11	62.35	90.02	75.74
FNC-8s (orig.)	RV-VOC11	62.69	90.33	75.86

The original FCN models can be downloaded from the MatConvNet model repository.

About

This code was developed by

• Sebastien Ehrhardt
• Andrea Vedaldi

References

'Fully Convolutional Models for Semantic Segmentation', Jonathan Long, Evan Shelhamer and Trevor Darrell, CVPR, 2015 (paper).

Changes

• v0.9.1 -- Bugfixes.
• v0.9 -- Initial release. FCN32s and FCN16s work well.

matconvnet-fcn's Issues

Error training network on voc 2012

Hi there

I run the 'fcnTrain' on VOC 2012 dataset, but the matlab comes with an error saying this:

Error using  ' 
Transpose on ND array is not defined. Use PERMUTE instead.

Error in vl_argparse (line 76)
    values = horzcat(values, struct2cell(args{ai})') ;

Error in cnn_train_dag (line 31)
opts = vl_argparse(opts, varargin) ;

Error in fcnTrain (line 98)
info = cnn_train_dag(net, imdb, getBatchWrapper(bopts), ...

Anyone knows what happened with the code or anything missing in my computer setting?

Regards,
Vic

I want to use 4 gpus but I got "boost::filesystem::canonical" error

I have 4 gpus and I set opts.train.gpus = [1,2,3,4] .When I run the fcnTrain.m I got this error:
boost::filesystem::canonical
why?Have anyone else tried this before?

Error while training fcn32s

Hi,

I am trying to train the fcn32s using my own data for segmentation purposes. I load the "imagenet-vgg-verydeep-16.mat" model and run the function "fcnInitializeModel" to create the fcn32 model.
My training set is a 224X224X3XN matrix (rgb images) and labels set is 224X224X1XN matrix (pixel-wise labels). However, I encounter the following error:

Subscript indices must either be real positive integers or logicals.
Error in dagnn.DagNN/eval (line 74)
[obj.vars(v).value] = deal(inputs{2:2:end}) ;
Error in cnn_train_dag>process_epoch (line 179)
net.eval(inputs, opts.derOutputs) ;
Error in cnn_train_dag (line 84)
stats.train(epoch) = process_epoch(net, state, opts, 'train') ;

The variable "v" in line 74 gets a NaN. Does anybody know how to fix this error?

Thanks,
Best regards

how to fine-tune fcn to my own data

I have 2000 gray pictures. I need to segment the image to two parts,object and background.Now I want to use fcn to do the image segment.How can I fine tune the pretrained fcn models,like pascal-fcn8s-dag.mat to adapt my segment task?

NaN Problem With Multiple GPU

When I try the code with single GPU everything is fine.
But when I increase the number of GPUs to more than 1 the objective function returns NaN after second batch.
Would you please help me to figure out what is the problem?

Thanks

for the cnn_train(net, imdb, getBatch, varargin), the format of mnist 'imdb' and the fcn 'imdb' is different

for the imdb data of the mnist example and the fcn example , the 'imdb.images' contains totally different subvariables
mnist 'imdb.images': data , data_mean , labels , set
fcn 'imdb.images' : id , name , set, classification ,segmentation, size
Since the two examples are using the same cnntrain function, I really can not understand how could cnntrain function or cnn_train_dag function can process the two kind of variables in different formats. Can anyone explain on this issue? Thanks a lot !

Proposal: better training script.

Hello.

Thank you so much for such a great training script for FCN.

I wanted to offer a small improvement that you might find useful:

While using your script I was a little confused with the 3-stage training process for FCN-8.
I understood the idea that I have to first train FCN-32, then use those weights to fine-tune FCN-16, and finally use FCN-16 weights to fine-tune FCN-8 only after reading the original paper from Berkley.

I wrote a small add-on that makes it more clear and easier to do.

If you find it interesting, let me know, I can make a pull request.

net.accumulateParamDers method !

whatd does net.accumulateParamDers method do for the gradients for the parameters, what it is usefull for ?

Why the output is not in the same size as input image?

Hi There

I used the FCN-matconvnet to train a fcn8s network. I tested the trained network with images of various sizes ranged from 260x347 to 1040x1388, but the output of the network is fixed to 224x224. To my knowledge, the output of network should be in the same size to the input.

So is anything wrong with the code or my training setting?

Best,
Vic

fcnTrain Error using imread (line 349) File "dataoc11" does not exist.

I am trying to train the fcn model when I got this error, have you ever met this before?
Warning: Control Character '\S' is not valid. See 'doc sprintf' for control characters valid in the format string.

Error while using ResNet

Hey,

I would like to implement Resnet-50 pretrained model. I just downloaded it, and it works fine. But when I just want to remove layers from the network. There is a inbuilt function net.removeLayer('LayerName'). This works for removing layer. But in my case, I would like to remove some 10 layers. So I tried to use a for- loop.

for hj=1:173
net7.layers(hj)=net.layers(hj);
end
for hj=1:174
net7.vars(hj)=net.vars(hj);
end
net7.params=net.params;
net7.meta=net.meta;

net7 = dagnn.DagNN.loadobj(net7) ;

it worked for 1:174 layers. But when I just want to remove other layers using this technique. It's showing an error in converting the struct to DAG obj as
Subscript indices must either be real positive integers or logicals.

Error in dagnn.DagNN.loadobj (line 47)
obj.params(p).(f) = s.params(i).(f) ;

Need your help.
Thank YOu,

Semantic segmentation on aerial image

Hi i'm studying in semantic segmentation on aerial image

I try to implement caffe source code in github
https://github.com/mitmul/ssai

i just followed framework in paper and it doesn't work

The input image is 64x64x3 patch and ground truth is 16x16x3 center part of input image

I found another paper that used it in vgg model, so i used vgg framework and implement again.

but it also doesn't work

The classes are { building, roads, others }, and i used softmaxlog in vl_nnloss

Below is my initalize model part.

function net = init_building(varargin)
opts.networkType = 'dagnn' ;
opts = vl_argparse(opts, varargin) ;

lr = [.1 .2] ;
drop1 = struct('name', 'dropout1', 'type', 'dropout', 'rate' , 0.5) ;
drop2 = struct('name', 'dropout2', 'type', 'dropout', 'rate' , 0.5) ;
% Define network CIFAR10-quick
net.layers = {} ;

% Block 1
net.layers{end+1} = struct('type', 'conv', ...
                           'weights', {{0.01*randn(3,3,3,64, 'single'), zeros(1, 64, 'single')}}, ...
                           'learningRate', lr, ...
                           'stride', [1 1], ...
                           'pad', [1 1 1 1]) ;
net.layers{end+1} = struct('type', 'relu') ;

net.layers{end+1} = struct('type', 'conv', ...
                           'weights', {{0.01*randn(3,3,64,64, 'single'), zeros(1,64,'single')}}, ...
                           'learningRate', lr, ...
                           'stride', [1 1], ...
                           'pad', [1 1 1 1]) ;
net.layers{end+1} = struct('type', 'relu') ;

net.layers{end+1} = struct('type', 'pool', ...
                           'method', 'max', ...
                           'pool', [2 2], ...
                           'stride', [2 2], ...
                           'pad', [0 0 0 0]) ; % Emulate caffe

% Block 2
net.layers{end+1} = struct('type', 'conv', ...
                           'weights', {{0.01*randn(3,3,64,128, 'single'), zeros(1,128,'single')}}, ...
                           'learningRate', lr, ...
                           'stride', [1 1], ...
                           'pad', [1 1 1 1]) ;
net.layers{end+1} = struct('type', 'relu') ;

net.layers{end+1} = struct('type', 'conv', ...
                           'weights', {{0.01*randn(3,3,128,128, 'single'), zeros(1,128,'single')}}, ...
                           'learningRate', lr, ...
                           'stride', [1 1], ...
                           'pad', [1 1 1 1]) ;
net.layers{end+1} = struct('type', 'relu') ;

net.layers{end+1} = struct('type', 'pool', ...
                           'method', 'max', ...
                           'pool', [2 2], ...
                           'stride', [2 2], ...
                           'pad', [0 0 0 0]) ; % Emulate caffe

% Block 3
net.layers{end+1} = struct('type', 'conv', ...
                           'weights', {{0.01*randn(3,3,128,256, 'single'), zeros(1,256,'single')}}, ...
                           'learningRate', lr, ...
                           'stride', [1 1], ...
                           'pad', [1 1 1 1]) ;
net.layers{end+1} = struct('type', 'relu') ;

net.layers{end+1} = struct('type', 'conv', ...
                           'weights', {{0.01*randn(3,3,256,256, 'single'), zeros(1,256,'single')}}, ...
                           'learningRate', lr, ...
                           'stride', [1 1], ...
                           'pad', [1 1 1 1]) ;
net.layers{end+1} = struct('type', 'relu') ;

net.layers{end+1} = struct('type', 'conv', ...
                           'weights', {{0.01*randn(3,3,256,256, 'single'), zeros(1,256,'single')}}, ...
                           'learningRate', lr, ...
                           'stride', [1 1], ...
                           'pad', [1 1 1 1]) ;
net.layers{end+1} = struct('type', 'relu') ;

net.layers{end+1} = struct('type', 'pool', ...
                           'method', 'max', ...
                           'pool', [2 2], ...
                           'stride', [2 2], ...
                           'pad', [0 0 0 0]) ; % Emulate caffe
% Block 4

net.layers{end+1} = struct('type', 'conv', ...
                           'weights', {{0.01*randn(3,3,256,512, 'single'), zeros(1,512,'single')}}, ...
                           'learningRate', lr, ...
                           'stride', [1 1], ...
                           'pad', [1 1 1 1]) ;
net.layers{end+1} = struct('type', 'relu') ;

net.layers{end+1} = struct('type', 'conv', ...
                           'weights', {{0.01*randn(3,3,512,512, 'single'), zeros(1,512,'single')}}, ...
                           'learningRate', lr, ...
                           'stride', [1 1], ...
                           'pad', [1 1 1 1]) ;
net.layers{end+1} = struct('type', 'relu') ;

net.layers{end+1} = struct('type', 'conv', ...
                           'weights', {{0.01*randn(3,3,512,512, 'single'), zeros(1,512,'single')}}, ...
                           'learningRate', lr, ...
                           'stride', [1 1], ...
                           'pad', [1 1 1 1]) ;
net.layers{end+1} = struct('type', 'relu') ;

net.layers{end+1} = struct('type', 'pool', ...
                           'method', 'max', ...
                           'pool', [2 2], ...
                           'stride', [2 2], ...
                           'pad', [0 0 0 0]) ; % Emulate caffe
% Block 5
net.layers{end+1} = struct('type', 'conv', ...
                           'weights', {{0.01*randn(3,3,512,512, 'single'), zeros(1,512,'single')}}, ...
                           'learningRate', lr, ...
                           'stride', [1 1], ...
                           'pad', [1 1 1 1]) ;
net.layers{end+1} = struct('type', 'relu') ;

net.layers{end+1} = struct('type', 'conv', ...
                           'weights', {{0.01*randn(3,3,512,512, 'single'), zeros(1,512,'single')}}, ...
                           'learningRate', lr, ...
                           'stride', [1 1], ...
                           'pad', [1 1 1 1]) ;
net.layers{end+1} = struct('type', 'relu') ;

net.layers{end+1} = struct('type', 'conv', ...
                           'weights', {{0.01*randn(3,3,512,512, 'single'), zeros(1,512,'single')}}, ...
                           'learningRate', lr, ...
                           'stride', [1 1], ...
                           'pad', [1 1 1 1]) ;
net.layers{end+1} = struct('type', 'relu') ;

net.layers{end+1} = struct('type', 'pool', ...
                           'method', 'max', ...
                           'pool', [2 2], ...
                           'stride', [2 2], ...
                           'pad', [0 0 0 0]) ; % Emulate caffe
% Block 6
net.layers{end+1} = struct('type', 'conv', ...
                           'weights', {{0.01*randn(2,2,512,4096, 'single'), zeros(1,4096,'single')}}, ...
                           'learningRate', lr, ...
                           'stride', [1 1],...
                           'pad', [0 0 0 0]) ;
net.layers{end+1} = struct('type', 'relu') ;
net.layers{end+1} = drop1;
net.layers{end+1} = struct('type', 'conv', ...
                           'weights', {{0.01*randn(1,1,4096,4096, 'single'), zeros(1,4096,'single')}}, ...
                           'learningRate', lr, ...
                           'stride', [1 1],...
                           'pad', [0 0 0 0]) ;
net.layers{end+1} = struct('type', 'relu') ;
%net.layers{end+1} = drop2;
net.layers{end+1} = struct('name','fc8','type', 'conv', ...
                           'weights', {{0.01*randn(1,1,4096,512, 'single'), zeros(1,512,'single')}}, ...
                           'learningRate', lr, ...
                           'stride', [1 1],...
                           'pad', [0 0 0 0]) ;
% Loss layer
net.layers{end+1} = struct('name', 'prob','type', 'softmaxloss') ;

% Meta parameters
net.meta.inputSize = [16 16 3] ;
net.meta.trainOpts.learningRate = 0.0001 * ones(1,50) ;
net.meta.trainOpts.weightDecay = 0.0005 ;
net.meta.trainOpts.batchSize = 100 ;
net.meta.trainOpts.numEpochs = numel(net.meta.trainOpts.learningRate) ;


net = dagnn.DagNN.fromSimpleNN(net, 'canonicalNames', true) ;

net.removeLayer('prob') ;

net.addLayer('objective', ...
  SegmentationLoss_building('loss', 'softmaxlog'), ...
  {'prediction', 'label'}, 'objective') ;

% Add accuracy layer
net.addLayer('accuracy', ...
    SegmentationAccuracy2(), ...
    {'prediction', 'label'}, 'accuracy') ;

Is there any problem in initializing?
I really stuck in this problem

Plz help me

Using InstanceWeights in vl_nnloss

I'm trying to use weighted loss for the FCN training on only 1 category (+BG).
I want to use the InstanceWeights but it seems like for very different weighting the result is the same. I was wondering if someone used this parameter for training and it helped him
The size of the InstanceWeights Im passing is : HxWx1xN with value between 0-1 where each label (object, BG) get the same value i.e.
for j=1:length(classWeights)
instanceWeights (labels==j)=classWeights(j);
end

I will be thankful to know how you decided on the value of instanceWeights and what are the value range you used …

Thanks

confusion matrix in SegmentationAccuracy

Hi,
I believe there is an error in the confusion matrix due to the usage of the same index variable on matrices with different size . This results in a wrong accuracy.
Before using the 'ok' variable for indexing 'predictions', 'predictions' should be the same size as 'labels'.
A possible fix would be using 'max' instead of 'sort' for the predictions, e.g.
[~,predictions] = max(predictions, [], 3) ;

Thanks.

Reference to non-existent field 'gpus'.

I was successfully build matconvnet with GPU. Then I tried to run fcnTrain.m and got error as

Reference to non-existent field 'gpus'.

Error in fcnTrain (line 95)
bopts.useGpu = numel(opts.train.gpus) > 0 ;

If I comment the line bopts.useGpu = numel(opts.train.gpus) > 0 ;, I got a new error as

Error using  ' 
Transpose on ND array is not defined. Use PERMUTE instead.

Error in vl_argparse (line 76)
    values = horzcat(values, struct2cell(args{ai})') ;

Error in cnn_train_dag (line 32)
opts = vl_argparse(opts, varargin) ;

Error in fcnTrain (line 98)
info = cnn_train_dag(net, imdb, getBatchWrapper(bopts), ...

I am using MATLAB 2015b in Window OS, newest matconvnet and matconvnet-fcn versions. Someone got the same error? How can I solve it.

Thanks everybody

why the fcnTrain.m running so slow?

Is it because the DriverVersion and the ToolkitVersion not matched?
How to solve this problem? Please help me,thank you!
CUDADevice with properties:

                  Name: 'GeForce GTX TITAN X'
                 Index: 1
     ComputeCapability: '5.2'
        SupportsDouble: 1
         DriverVersion: 7.5000
        ToolkitVersion: 6.5000
    MaxThreadsPerBlock: 1024
      MaxShmemPerBlock: 49152
    MaxThreadBlockSize: [1024 1024 64]
           MaxGridSize: [2.1475e+09 65535 65535]
             SIMDWidth: 32
           TotalMemory: 1.2885e+10
       AvailableMemory: 1.2609e+10
   MultiprocessorCount: 24
          ClockRateKHz: 1076000
           ComputeMode: 'Default'
  GPUOverlapsTransfers: 1
KernelExecutionTimeout: 1
      CanMapHostMemory: 1
       DeviceSupported: 1
        DeviceSelected: 1

train: epoch 01: 1/565: 0.9 Hz accuracy: 0.677 0.048 0.032 objective: 3.044
train: epoch 01: 2/565: 1.0 Hz accuracy: 0.691 0.048 0.033 objective: 3.035
train: epoch 01: 3/565: 1.0 Hz accuracy: 0.698 0.048 0.033 objective: 2.994

Spend long time to traning data

Hello all, I am running the fcnTrain on my PC (core i7, 16GB Ram, GPU). However, it spend long time (0.5 day to 1 day) to running each epoch. If anyone meet the issue, could you give me the solution to solve that issue? Thank all

Is it true that we must use GPU?

I run the file fcnTrain.m only with cpu bopts.useGpu =0;, I got the following running error;
Error using '
Transpose on ND array is not defined. Use PERMUTE instead.

Error in vl_argparse (line 76)
values = horzcat(values, struct2cell(args{ai})') ;

Error in cnn_train_dag (line 32)
opts = vl_argparse(opts, varargin) ;

Error in fcnTrain (line 109)
info = cnn_train_dag(net, imdb, getBatchWrapper(bopts), ...

Why the speed of proceesing pictures in the GPU(gtx1080) became slow?

Hello
At first,the speed is about 6Hz .However,after about an hour, the speed is about 5Hz, by the time the speed became slower. If we restart the training process , the speed can recover.

Index exceeds matrix dimensions error from the vl_nnloss fucntion

Hi, I used my own data with only 2 classes. But I got the following errors when I started to train. I do not know why it is. Could any body help me with the problem?
Xmax
384 384 1 2

X
384 384 2 2

ci
384 384 1 2

Index exceeds matrix dimensions.

Error in vl_nnloss (line 225)
t = Xmax + log(sum(ex,3)) - X(ci) ;

Error in SegmentationLoss/forward (line 6)
outputs{1} = vl_nnloss(inputs{1}, inputs{2}, [], ...

Error in dagnn.Layer/forwardAdvanced (line 85)
outputs = obj.forward(inputs, {net.params(par).value}) ;

Error in dagnn.DagNN/eval (line 88)
obj.layers(l).block.forwardAdvanced(obj.layers(l)) ;

Error in cnn_train_dag>process_epoch (line 195)
net.eval(inputs, opts.derOutputs) ;

Error in cnn_train_dag (line 90)
stats.train(epoch) = process_epoch(net, state, opts, 'train') ;

Error in fcnTrain (line 138)
info = cnn_train_dag(net, imdb, getBatchWrapper(bopts), ...

Not able to read imagenet-vgg-verydeep-16.mat

Hi,

I get the following error when I try to run fcn_train. Any suggestions please. The file exists in the folder.

load('imagenet-vgg-verydeep-16.mat')
Error using load
Can't read file /home/mbaxkcr4/matconvnet-fcn/data/models/imagenet-vgg-verydeep-16.mat.

CUDA error when trying to use multiple GPUs: nnconv_forward_blas: CUDA error [cuda: forward: pointer does not correspond to a registered memory region]

I'm gettng a CUDA memory error when trying to run my FCN code on more than 1 GPUs. As soon as the training finishes in the first epoch and validation is about to start, I get the following

the error is trailed by a string of "cudaHostUnregister failied" warnings as shown below:

Has anyone faced a similar issue? If yes, any help would be much appreciated. Thanks!

Does any contributor accept payed contract for specific code ?

I would like to reproduce paper results using MatConvNet.
Contact me for further details.

How to modify the loss function?

Hello,
Thanks for this wonderful project, I can train a FCN on my own data.
I want to modify the loss function to a combination of losses on several labels.
But I find it hard to change the loss function after reading trough the scripts.
Can you give me some instruction on how to do it?
Thanks a lot !!!

fcntrain.m code does not run out of the box fix

Hi,

First off, thanks guys for making a great package for deep learning (MatConvNet).

So when I tried to run fcntrain.m after downloading matconvnet-fcn, I ran into several errors where lines using sprintf function where not working as below:

Warning: Control Character '\J' is not valid. See 'doc sprintf' for control characters valid in the format
string.

In getBatch at 50
In fcnTrain>@(imdb,batch)getBatch(imdb,batch,opts,'prefetch',nargout==0) at 104
In cnn_train_dag>process_epoch at 164
In cnn_train_dag at 84
In fcnTrain at 97
Warning: Control Character '\S' is not valid. See 'doc sprintf' for control characters valid in the format
string.
In getBatch at 51
In fcnTrain>@(imdb,batch)getBatch(imdb,batch,opts,'prefetch',nargout==0) at 104
In cnn_train_dag>process_epoch at 164
In cnn_train_dag at 84
In fcnTrain at 97
Warning: could not read image 'data�oc11'
Error using imread (line 349)
File "data�oc11" does not exist.

It turns out lines with this:
lb = imread(sprintf(imdb.paths.classSegmentation, imdb.images.name{train(i)})) ;

can be replaced with this:
lb = imread([imdb.paths.classSegmentation(1:end-6),imdb.images.name{train(i)},'.png']) ;

and then the code runs as intended. I'm operating on Matlab2014a on Windows 7 64 bit with a GPU NVIDIA Quadro K1200.

How to adapt 'fcnTrain' to my two problem?

Hi,

I built my own labeled dataset, which is composed of a 7000x(64x64x3) images and their corresponding label-2D-image. I am interested in distinguishing between two classes, the object of interest (pixels labeled as 1) and background(pixels labeled as 0).

As my aim is detecing the object of interest in any image, I think that a segmentation model such as fcntrain would be appropriate. So,

1. I converted my dataset into "imdb.mat" to feed fcntrain,
2. I used 'imagenet-vgg-verydeep-16.mat' as source 'net' and changed the number of classes in "fcnInitializeModel" from 21 to 2. The next step is to use 'net-epoch-50.mat'.

Do I need to provide the model with more information about my data?

When I run the adapted code it keeps running forever without producing any output.

Here is the 'fcntrain.m' I am trying to run :

opts.sourceModelPath = 'data\models\imagenet-vgg-verydeep-16.mat' ;
[opts, varargin] = vl_argparse(opts, varargin) ;

opts.train = struct([]) ;
[opts, varargin] = vl_argparse(opts, varargin) ;

trainOpts.batchSize = 20 ;
trainOpts.numSubBatches = 10 ;
trainOpts.continue = true ;
trainOpts.gpus = [] ;
trainOpts.prefetch = true ;
trainOpts.expDir = opts.expDir ;
trainOpts.learningRate = 0.0001 * ones(1,50) ;
trainOpts.numEpochs = numel(trainOpts.learningRate) ;
net = MyfcnInitializeModel('sourceModelPath', opts.sourceModelPath) ;
bopts.numThreads = 1 ;
bopts.labelStride = 1 ;
bopts.labelOffset = 1 ;
bopts.classWeights = ones(1,2,'single') ;
%-------------------------------------------------------------------------------------------------
load dataMatLabel.mat
data = indata;
cls =class(:,:,:);
for i=1:7000
imdb.images.data(:,:,1,i) = data(:,:,1,i);
imdb.images.data(:,:,2,i) = data(:,:,2,i);
imdb.images.data(:,:,3,i) = data(:,:,3,i);
imdb.images.labels(:,:,i) = cls(:,:,i);
imdb.images.set (1,i)= 1; %1 for train, 2 for test (val?)

         imdb.images.data(:,:,:,i+7000 ) = indata(:,end:-1:1,:,i);            
         imdb.images.labels(:,:,i+7000) = cls(:,:,i);
         imdb.images.set(1,i+7000) = 2; %1 for train, 2 for test (val?)

end

train=find(imdb.images.set==1) ;
val=find(imdb.images.set==2) ;
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Launch SGD
info = cnn_train_dag(net, imdb, getBatchWrapper(bopts), ...
trainOpts, ....
'train', train, ...
'val', val, ...
opts.train) ;

% -------------------------------------------------------------------------
function fn = getBatchWrapper(opts)
% -------------------------------------------------------------------------
fn = @(imdb,batch) MygetBatch(imdb,batch,opts,'prefetch',nargout==0) ;

Any suggestion is more than wellcome! Thank you!

error to execute fcntrain.m

Hello,
Does anyone run this code (matconvnet-fcn). I am trying to run this (fcntrain.m) code, but get error at line 99.

% Launch SGD
info = cnn_train_dag(net, imdb, getBatchWrapper(bopts), ...
trainOpts, ....
'train', train, ...
'val', val, ...
opts.train) ;
"opts.train" is always null, I could not understand should I change something??

[Error (first portion):
Error using '
Transpose on ND array is not defined. Use PERMUTE instead.

Error in vl_argparse (line 78)
values = struct2cell(args{ai})' ;]

Error in fcnInitializeModel.m

I was going through your code to adapt it to the Kitti Benchmark, and i found an error in fcnTrain. More particularly in the function fcnInitializeModel.m here :

LINE 49 -LINE 55
% Modify the last fully-connected layer to have 21 output classes
% Initialize the new filters to zero
for i = net.getParamIndex(net.layers(end-1).params) ;
sz = size(net.params(i).value) ;
sz(end) = 21 ;
net.params(i).value = zeros(sz, 'single') ;
end

This is not working as for i=2, BIASES size is [1000,1]. Thus modifying sz(end) is wrong because you want to change 1000 to 21.
This mistake leads to an error when try to train the network, but you can easily correct it by changing
sz(end) = 21 ;
into
sz(sz == 1000) = 21 ;

This is it. By the way I wanted some advice concerning the training of the model on Kitti dataset. As there are only 2 classes : road and background, is it good to keep the same type of loss function as with PASCAL dataset? I keep being stuck at around 54% accuracy which is pretty bad

image segmentation implementation

Hi
is there any tutorial step by step to train and test image segmentation architecture in matconvnet?

Index exceeds matrix dimensions problem

I am working with binary classification ( 2 labels). I had problem in the following line:
t = Xmax + log(sum(ex,3)) - X(ci) ==>X(ci) "Index exceeds matrix dimensions"
X ==> 384x384 x 3 x 2
ci ==> 384 x384 x 1 x2

Do you have an idea about this error

how could I test one single image for the model ?

I mean the demo code are too integrated, I want to test one single image and get familiar with this framework and codes ,like this link http://www.robots.ox.ac.uk/~szheng/crfasrnndemo

Image input that is not a multiple of 32

Hello,

Thank you for your repository!

I have a small question: when working with FCN-32s model, the input should be of multiple of 32.
When the input is not, the resizing to the closest multiple of 32 is performed and nearest neighbor
interpolation of predicted labels is performed to get the predictions of the same size as the input.
This is my understanding of what is performed in your code here:
https://github.com/vlfeat/matconvnet-fcn/blob/master/fcnTest.m#L143
Is it correct?

what are the pros and cons compared to the approach of implementation of FCN-32s in Caffe where they use padding to tackle this problem?

How to test this code in Matlab?

Hello,

I'm starting to learn CNN to use it in a project. For begin, I have installed MatConvNet, compiled it, and tried the examples into the web.

The problem is that I have dowloaded this code to test how semantic segmentation works and do not get it, I have differents errors in Matlab?

Is there any post where I can read how to execute this code or similar?

Thanks in advance.

Seeking help on using pretrained models for inferencing

Dear MatConvNet FCN community,

I'm new to semantic segmentation and was hoping to test the pretrained models posted on the MatConvNet website. I'm simply trying to view inferences made by the model "pascal-fcn32s-dag.mat" on peppers.png.

I'm trying two approaches and getting stuck in both.

1. In fcnTest.m, redirecting opts.modelPath from 'data/fcn32-voc11/net-epoch-50.mat' to the pretrained model. I get errors regarding the structure of the pretrained model not matching what is expected.
2. loading the pretrained model in a test script using the DagNN wrapper and running net.eval({'data', im_}). It runs and I get a 384x512x21 value corresponding to 'upscore,' but I'm not sure how to correctly interpret/print the segmentation.

Any help would be greatly appreciated!

Thank you,
Stephen

Finetuning Alex-Net for Semantic Segmentation

I'm trying to reproduce FCN paper results using Alex-Net (imagenet-matconvnet-alex.mat) which
doesn't contain Local response normalization layer as in Alex-Net (imagene-caffe-ref.mat).
This the results (shown in the pdf attached) i obtained using :
minibatch size = 1 ;
learning rate = 1e-5 ;
after 100 epoch i got meanIU = 25 on validation dataset (1111 images) and it seems that there is some sort of saturation in meanIU value for validation dataset , so what i should do next !

1-train for more epochs ?
2- decrease the learning rate and train from the start ?
3- add local response normalization layer like the paper ?

net-train.pdf

Test Image Segmentation

Hi,

I am trying to see the semantic segmentation results for a test image like peppers.png using the pretrained model, pascal-fcn16s-dag.mat. The quickstart guide at http://www.vlfeat.org/matconvnet/quick/ shows how to use DAG for classification, but not segmentation. How would I go about doing this?

Can someone help?

What to do with arbitrary input size for GPU efficiency

Hi,
Short Question : I have a data set with arbitrary input sizes. How should I edit my input samples to keep all of them in one matrix for GPU efficiency?

Long Questions :
Currently I am processing each sample individually, which of course, really inefficient. I should store them in 4D matrix (where 4th dimension is for images), but due to their varying size, I can't.

Do you think I should sample same sized inputs from the data? How should I do it?

What if I set the size of my 4D matrix to maximum size of among all input samples and fill the remaining part of other samples with zeros? What happens when we give an image of zeros to the network? Does it have any influence in learning?

Thank you.

how to train fcn on my own data on the matconvnet

I want to train FCN on the RGB image of 2569X1919X3,the size of ground truth is 2569X1919X3,how should to slove it?

Extracting results from the fc6/fc7 layers of the fcn network

Hi,

Does anyone have an idea about how to extract the 4096-dimensional feature vector from the fc6/fc7 layer of the fcn network? I see that the network stores only the upscore in the net.vars while all the other variables are empty. How to make the network store the results of fc6/fc7 layer?

Thanks

Error when executing fcnTrain.m

I'm trying to run the training of matconvnet on Pascal voc.
I just compiled and tested matconvnet then downloaded your code. Adapted the paths to matconvnet and data but the execution of fcnTrain.m keeps on failing with this error message:

Error using  ' 
Transpose on ND array is not defined. Use PERMUTE instead.

Error using  ' 
Transpose on ND array is not defined. Use PERMUTE instead.

Error in vl_argparse (line 77)
    values = struct2cell(args{ai})' ;

Error in cnn_train_dag (line 34)
opts = vl_argparse(opts, varargin) ;

Error in fcnTrain (line 99)
info = cnn_train_dag(net, imdb, getBatchWrapper(bopts), ...

It seems like in vl_argparse it tries to transpose an empty 0x0 matrix. I'm a little confused since there is quite a lot of argument parsing. Do you have any idea how to fix this?

Do I need to pass fcnTrain with some special argument? (eg Database-Name or Model Name?)
Thanks in advance

getBatch nearest neighbor interpolation

In get batch, the images seem to be re-sized using a nearest-neighbor interpolation,
look at line 77:
ims(oky,okx,:,si) = bsxfun(@minus, rgb(sy(oky),sx(okx),:), opts.rgbMean) ;

Is this on purpose ? Seems that it may hinder the results.

The GPU failed to allocate memory when started to run epoch 2 with fcnTrain.m

I used Pascal Titan X to run fcnTrain.m. The first epoch went smoothly with 5.5Hz of GPU. However, when it finished epoch 01 and started to do epoch 02, following error showed up. I inspected size of ims. This variable size is 798 kb, and my GPU has 12GB.
Matconvnet was compiled under :Windows 10+Pascal Titan X +CUDN 8.0RC+CuDNN 5.1

train: epoch 02: 1/ 56:Error using gpuArray
The GPU failed to allocate memory. To continue, reset the GPU by running 'gpuDevice(1)'. If this problem persists,
partition your computations into smaller pieces.

Error in getBatch (line 91)
ims = gpuArray(ims) ;

Error in fcnTrain>@(imdb,batch)getBatch(imdb,batch,opts,'prefetch',nargout==0) (line 108)
fn = @(imdb,batch) getBatch(imdb,batch,opts,'prefetch',nargout==0) ;

Error in cnn_train_dag>process_epoch (line 197)
inputs = state.getBatch(state.imdb, batch) ;

Error in cnn_train_dag (line 83)
[stats.train(epoch),prof] = process_epoch(net, state, opts, 'train') ;

Error in fcnTrain (line 99)
info = cnn_train_dag(net, imdb, getBatchWrapper(bopts), ...

Following is the epoch 01

adding absolute spatial info of each pixel to the network

Does anybody know, if I want to add the "absolute position of each pixel" while training, like what they have done in this article(http://hera.inf-cv.uni-jena.de:6680/pdf/Brust15:CPN.pdf), how can I implement it in Matconvnet-FCN. It should not be complicated but I am new to this framework. I will appreciate it if any body can help me.

Error in deconv layer

Hello,

I am trying to use "convt" to deconvolute image (input size 256X256, perform max pool 4 times, so image size becomes 16X16). If I define weight as "Wzj = 0.5*randn(256,256,512,500,'single');"........it requires 62GB which is impossible array size in Matlab.

My question is at "convt" layer how I can define weight to convert pooled image (16X16) to original image size (256X256)? Any kind of help will be appreciated.

Thanks
Hosna

DAG Batch Normalization Parameter Initialization

I'm a little new to MatConvNet, so this may not be an issue but I thought I'd submit it anyway and see what shakes out.

I'm using beta23, and I insert a batch normalization layer like this:
nn.addLayer('u4_bn',dagnn.BatchNorm(),{'u4_c2_x'},{'u4_bn_x'},{'filters' 'bias' 'moments'})

I run training using the dag network training included in the examples folder, and it gives an error that the multipliers don't have the same depth as the data. I looked at nn, and the parameters for filters, bias, and moments are empty. So, I changed the code to this:

nn.addLayer('u4_bn',dagnn.BatchNorm(),{'u4_c2_x'},{'u4_bn_x'},{'filters' 'bias' 'moments'})
f = nn.getParamIndex('filters');
b = nn.getParamIndex('bias');
m = nn.getParamIndex('moments');
nn.params(f).value = ones(64, 1, 'single') ;
nn.params(b).value = ones(64, 1, 'single') ;
nn.params(m).value = zeros(64, 2, 'single') ;
nn.params(f).weightDecay = 0 ;
nn.params(b).weightDecay = 0 ;
nn.params(m).weightDecay = 0 ;

I initialize the network and the params for filter, bias, and moments are still empty. I was able to narrow it down to 'nn.initParams()' seemed to clear the values I was setting. If I set the values of filters, bias, and moments after 'nn.initParams()', then things seem to run smooth. I haven't dug into the code for initParams, but there seems to be a bug here.

Edit: I posted this in the wrong spot. My apologies.

How to convert the SegmentationLoss to Euclidean Loss?

I am trying to do regression using FCN, is there any easy way to convert SegmentationLoss to Euclidean Loss? As those new FCN files lack of documentations, I am quite confused on how to edit the SegmentationLoss file. Any hints would be appreciated.

Below is the code of SegmentationLoss:

classdef SegmentationLoss < dagnn.Loss

  methods
    function outputs = forward(obj, inputs, params)
      mass = sum(sum(inputs{2} > 0,2),1) + 1 ;
      outputs{1} = vl_nnloss(inputs{1}, inputs{2}, [], ...
                             'loss', obj.loss, ...
                             'instanceWeights', 1./mass) ;
      n = obj.numAveraged ;
      m = n + size(inputs{1},4) ;
      obj.average = (n * obj.average + double(gather(outputs{1}))) / m ;
      obj.numAveraged = m ;
    end

    function [derInputs, derParams] = backward(obj, inputs, params, derOutputs)
      mass = sum(sum(inputs{2} > 0,2),1) + 1 ;
      derInputs{1} = vl_nnloss(inputs{1}, inputs{2}, derOutputs{1}, ...
                               'loss', obj.loss, ...
                               'instanceWeights', 1./mass) ;
      derInputs{2} = [] ;
      derParams = {} ;
    end

    function obj = SegmentationLoss(varargin)
      obj.load(varargin) ;
    end
  end
end

Can I just simply replace the vl_nnloss function with a Euclidean Loss function?

Error using vl_nnconv

Hello:
I meet some problem when I using "trainOpts.gpus = [1] ;",the error like these:

" Error using vl_nnconv
DATA and FILTERS do not have compatible formats.

Error in dagnn.Conv/forward (line 11)
  outputs{1} = vl_nnconv(...

....."
if I using trainOpts.gpus = [],it worked, very strange
Looking for your help ,thanks!!!

code explain

I am confused about the fcnTest.m function.Can any body explain the code for me:

net.eval({inputVar, im_}) ;
scores_ = gather(net.vars(predVar).value) ;
[~,pred_] = max(scores_,[],3) ;

I found that the scores_ is not a probability matrix ,why ?How can I make the scores_ to a probability matrix? what is scores_ matrix ?

How to converge a fcn?

Hi,
I'm trying fcn of matconvnet now. First, I tried PASCAL VOC as described in the paper, and I found it could converge sometimes: once the accuracy began from 74%, it converged within 50 epoches (91% accuracy on training set). However, sometimes the accuracy bagan from 40%, it could not converge after 150 epoches. I guess it is caused by random initial values we set.
When I take fcn to train on my own dataset, the accuracy always began from 30~40% and it didn't converge. Nearly 52% the accuracy became saturated, and training loss and validation loss separated (over-fitting?). I resized images to 512x512 and it didn't work. I also try removing the deconv layer and mapping labels (512x512) into an activation map (16x16), also failed.
I wonder if there anyone who meets the same problem or knows how to solve? Thanks a lot!
some parameters: I used 5000 images with pixel-level annotation and augmented them to 10000, and set learning rate to 0.0001.

Problem with vl_nnconv while training a FCN

I am running the fcnTrain function in order to train a FCN model.
I am running it while using the VGG-16 model as the initialization (just as the script proposes).
However, when I do so, I get an error that:

Error using vl_nnconv
The number of elements of BIASES is not the same as the number of filters.

Note that the function is able to operate.
The error occurs in a specific layer in which:
size(params{1})=
1 1 4096 21
size(params{2})
1000 21

I have a feeling that it's got something to do with a lacking transpose operation.

Thanks,
Omry