LinkNet implementation working? about pytorch-semseg

[meetps]

@peteflorence - Yes, you are right. All datasets are at the moment not compatible with all the models due to differences in input sizes.

My first task is to make all models working on Pascal VOC (512*512) and then add dataset specific cropping or padding arguments.

[peteflorence]

Thanks @ibadami and @meetshah1995

It's training now... think we have it working... will PR and share evaluation

@travelbureau thx for help too!

[peteflorence]

OK, thanks for letting me know. Respectfully I think this issue should remain open though, because it is unknown whether or not the implementation is working.

[ibadami]

@peteflorence So I just checked, It seems that the initial pspnet was copied from linknet while @meetshah1995 was working on the repo. I have changed the pspnet into fully functional linknet code.
It is not really pspnet yet as there are few more changes required. But what I can assure you is that current pspnet is the corrected linknet. Moreover there you can also find multiple resnet support which I added for pspnet. So I would suggest you to use pspnet from resnet_encoder branch. Let me know if that code is throwing you any error. I will fix it as soon as possible. I know it is bit confusing. But it is work in progress.
Cheers!

[peteflorence]

Hmm... maybe just need to crop/pad 360 since not divisible by 16?

[ibadami]

@peteflorence @meetshah1995 , I just updated the linknet with pretrained resnet option.
Could you guys check it out?

[meetps]

I haven't verified the linknet module myself. The changes you see were made by @ibadami. I don't plan to verify them in the coming weeks. The fix takes minimal effort, please feel free to send in a PR !

[ibadami]

@peteflorence remove the 'nn.' in front of it. Cuda bad params can occur if your input dimensions are not as expected. Most often this occurs at the last layer where the target shape and last conv layer has mismatched shape. Kindly print the shape of the tensor after every operation. You will be able to see the what the problem is.

[ibadami]

@peteflorence In linknet from lines 35 to 38 names of decoders are incorrect. Change them from self.decoder4 4 times to decoder4, decoder3 decoder2, decoder1. Something I could spot right away.
I will check what else is required to be corrected. Stay tuned!

[peteflorence]

Sure @ibadami. Have you tested the implementation? I'd be happy to help make some changes / PR, just want to get a sense of how well it's been tested.

[peteflorence]

OK @ibadami, thanks! :)

[ibadami]

@peteflorence checkout patch #17.

[peteflorence]

Thanks @ibadami, looking at it now!

[peteflorence]

Hi @ibadami, I'm getting an error on d4 = d4 + e3 in forward() in the model

When I add prints as below in linknet.py, I get these sizes for d4 and e3 respectively:

(1L, 256L, 24L, 30L)
(1L, 256L, 23L, 30L)

def forward(self, x):
        # Encoder
        x = self.convbnrelu1(x)
        x = self.maxpool(x)

        e1 = self.encoder1(x)
        e2 = self.encoder2(e1)
        e3 = self.encoder3(e2)
        e4 = self.encoder4(e3)

        # Decoder with Skip Connections
        d4 = self.decoder4(e4)
        print d4.size()
        print e3.size()
        d4 = d4 + e3
        d3 = self.decoder3(d4)
        d3 = d3 + e2
        d2 = self.decoder2(d3)
        d2 = d2 + e1
        d1 = self.decoder1(d2)

        # Final Classification
        f1 = self.finaldeconvbnrelu1(d1)
        f2 = self.finalconvbnrelu2(f1)
        f3 = self.finalconv3(f2)

        return f3

Looks like maybe an off by one error? I'm feeding it CamVid data, 480 x 360

[peteflorence]

To close the loop here -- we have the training scripts 'running', but have not been able to get test segmentations that look meaningful.

If interested you can review my changes here (just use python train.py, I changed the defaults in train.py)

[meetps]

Thanks for the update. You haven't used the ResNet pretrained weights as initialization for the LinkNet encoder weights. Training from scratch did not get me good results with FCN-8s on Pascal VOC and I'd expect the same to happen with CamVid and LinkNet.

What would be ideal would be to have pre-trained ResNet from torchvision and use them to construct the linkNet as done here. I have this in mind, but haven't quite found the time to implement them.

[stereomatchingkiss]

Implement LinkNet at here, without using any pretrained weights, I am able to get mean accuracy close to 90%, you can see more details in this post.