The question about awesome-semantic-segmentation-pytorch

[Tramac]

It's possible. Please give me some time to find these bugs.

[Tramac]

Please see folder /home/yourpcname/.torch/models, is there trained model? for example: fcn32s_vgg16_pascal_voc.pth.

[Tramac]

Maybe there are some bugs that are hard to find in my code. Please keep this question, I will also check the reason.
Thank you for your careful and valuable thinking:+1:.

[82018303kazuma]

Thank you for the meaningful code.

Caffe does AlexNet's weight by fine-tuning the final layer conv.
The conv of the last layer is changed to 1x1xclass_number.

I think it is strange that almost the same performance is obtained even if using the initial VGG 16 weights without training.
I feel that the weights of all layers are not updated correctly.
I do not know if the weight of each layer is frozen or if the hyperparameter was accidentally dropped to the local solution.

[Tramac]

Hi, My opinions on the above is as follows:

1. Although the change of loss is small during epoch=1, 10, 100, the difference of prediction results may be large, you can change parameter no_val in train.py, which can run validation() in every epochs. I think it's big enough when epoch=1000, and less improvement of results is normal.

2. There is no problem with python2.

3. The error maybe caused by parallel training, you can comment out the code self.model = DataParallelModel(self.model).cuda() and self.criterion = DataParallelCriterion(self.criterion).cuda(). And remember to put them to cuda() when defining.

   Change preds, target = tuple(inputs) must ensure that preds is a tensor list, such as [(batch, classes, H, W), (batch, classes, H, W)], you can print its type and length. I will test it later and reply you.

[82018303kazuma]

Hi thank you for reply.

Regarding 1:
If you evaluate with learning data (alternative estimation method), the performance should be high.
However, in practice the performance is not high.
I feel that training is not going well.
Much lower than when I implemented it with other frameworks (caffe and tensorflow).

This is the output When you evaluate 2007_000032.jpg by putting 2007_000032.jpg into the network where you trained 2007_000032.jpg.(validation pixAcc: 96.488%, mIoU: 7.564%)

[Tramac]

Hi, thank you for your detailed comparison!
The mIoU value in score.py refers to the average mIoU of 21 categories (for VOC). For this test image, only the plane and background appeared here, so the mIoU is low due to the calculation method.
If you want to evaluate single image, you can use function hist_info and compute_score in score.py.

Is the visual result worse than the result from caffe and tf?

[82018303kazuma]

According to the result of the alternative estimation method (a method of making the training and test data the same), the accuracy is high even at 10 epoch in the case of fcn32s to Caffe.

[Tramac]

It seems that the test result of caffe is better than pytorch's. I guess this result may be caused by:

1. Upsampling method. (interpolate in pytorch. vs deconvolution in caffe?)
2. convolution kernel initialization method and base_lr.
3. The parameters of pretrained base model (vgg) are not fixed in training process, how about caffe?

I also found that the results is not as good as paper. if you find the bug in this code, which leads to the worse result, please tell me.
Thank you again for your detailed comparative experiment. I hope to improve this project with you together.

[82018303kazuma]

Thank you for your reply.

1. It is considered to affect performance. (Caffe is a deconvolution.)
2. Even with full scratch, the performance is higher. lr did the same.
3. I think that 3 is the cause.
   The results of VGG 16 and epoch 60 in this code are exactly the same.
   I feel this code is fixed at VGG16. It may be caused by one's own environment ....

---

epoch0->base_model(vgg16)

---

epoch60

---

I think there are few people who want to use FCN's Pascal_voc to this point with pytorch. I think it's amazing.

[Tramac]

Hi, the result of epoch=0 refers to no training? Dose it use only pretrained mode vgg16 and initialization parameters in _FCNHeard?

[82018303kazuma]

epoch = 0 does not train, it used only pre trained model vgg16 and initialization parameters in _FCNHeard.

Some performance has been achieved without training.
I'm learning with augmentation data (I've increased Pascal data by 36 times), but Caffe has a good effect, but Pytorch has no significant effect.
It is hard to think that augmentation does not have much effect. It is thought that there is a problem in updating training.

Does eval.py correctly load the learned model weights?
Where is the code?
Is the result different even for the same image because the weights being loaded are different?

demo.py

---

eval.py

---

[Tramac]

1. I think that just using the pretrained model and initialization parameters in _FCNHead cannot achieve the performance. Has the trained model (such as epochs=60) been misused?

2. class SegmentationDataset() in data_loader/segbase.py includes data augmentation, please see function _sync_tanform and _val_sync_transform.

3. Trained model loading in eval.py is implemented by function get_segmentation_model() in models/model_zoo.py, it will run get_fcn32s() -> https://github.com/Tramac/Awesome-semantic-segmentation-pytorch/blob/ec4882a9e2025fb5c000cb21be8ebac07c09c923/models/fcn.py#L155
   Trained model loading in demo.py is implemented by functionget_model() in models/model_zoo.py, it will run get_fcn32s_vgg16_voc() -> get_fcn32s() -> https://github.com/Tramac/Awesome-semantic-segmentation-pytorch/blob/ec4882a9e2025fb5c000cb21be8ebac07c09c923/models/fcn.py#L155
   So, they should have the same result.

[82018303kazuma]

It understood.

1. Do git clone https://github.com/Tramac/Awesome-semantic-segmentation-pytorch.git.

2. Prepare the data.

3. python eval.py
   Why is it possible to make an evaluation?
   Fcn 32 can be loaded without training.
   The performance at that time is, for example, Sample 1450, validation pixAcc: 85.044%, mIoU: 46.411%.
   I do not know this.

[82018303kazuma]

I understand the reason.
I noticed that I was competing with the models I collected for pytorch.