Problem about the generated segmentation mask about blitznet

[dvornikita]

Hi @ganeshtata. Yes, it does improve the process. I pushed the modification.
You need to generate the dataset in tf_record format again and run the training.

[dvornikita]

Hi @tjulyz, I guess these scripts were not used to train the model in the mixed setting (with gt segmentation masks and without). If I remember correctly, in the file voc_loader.py, in the line add 255 to the zero-valued segmentation.
In the training.py file, where the segmentation loss is defined, the pixels that have a value of gt segmentation higher than the number of categories in the dataset don't contribute to the loss, they are filtered out.
Don't hesitate to come back with the results.
Good luck.

[ShihuaHuang95]

Thanks for the rely. @dvornikita
However, if I am not downloading a wrong repository, you did not add 255 to the zero-valued segmentation in the original source code. Does it mean we should add by ourselves?
Best,

[dvornikita]

Hi @hshihua, that's correct. Sorry for the inconvenience. Can't modify the code or run the experiment right now. I'll come back soon and check this myself.

[ShihuaHuang95]

Thanks for your reply. And I can understand.
Maybe the code should be like: segmentation = np.ones([height, width], dtype=np.uint8) * 255

[dvornikita]

Exactly.
If you inspect the existing gt segmentation masks in the dataset, you may see that object borders are marked with this value - 255, not with its class value, to reduce the confusion during training. By design, our loss function doesn't take into account those pixels.
In the current situation, you are doing the same thing but with the whole segmentation mask.

[ShihuaHuang95]

Thanks~
If I am doing the right thing, and you just need to modify the code by replacing "segmentation = np.zeros([height, width], dtype=np.uint8)" with "segmentation = np.ones([height, width], dtype=np.uint8) * 255" in the voc_loader.py.
Best,

[dvornikita]

Yes, that's right.

[tjulyz]

Thanks a lot! @dvornikita @hshihua

[tataganesh]

Hi @tjulyz . I am facing the same issue as yours. For me, The detection performance is 80% map, but only 49% mIoU for segmentation. Does the above change improve the training process?

[tataganesh]

Great @dvornikita . Thank you for the prompt response! Just a few things that I wanted to add -

• Could you please add the reasoning behind this change in the comment above the line? The current comment reads that an image of all zeros is generated. But now, we aren't doing that, but instead generating an image with all 255 values. You have mentioned the reasoning behind it as well in this thread. It would be awesome if you could write the same ( In short ) in the comment above the line.

• Also, instead of writing
  segmentation = np.zeros([height, width], dtype=np.uint8) + 255
  Could we write -
  segmentation = np.full([height, width], value=255, dtype=np.uint8)

These are just a few suggestions. Thank you for this project!

[dvornikita]

@ganeshtata you can write either of the lines, they do the same thing.
0 corresponds to the background class and the value of 255 stands for ignoring the pixel in the loss. If we don’t know the true image semantics it’s better to not make updates on it rather than to treat all its content as background and to learn from a wrong signal.