Found 2 (potential) bugs when training on semantic-only datasets. about deeplab2

[sebastian-woessner]

Thank you for your response,

To be clear, the issue that I originally posted is currently not hindering me since I applied the fixes that I proposed above. I opened this issue to highlight 2 problems/bugs which might come up when a semantic-only dataset is used.
The code requires the semantic label to be encoded as a one-channel image and that panoptic_label_divisor is set to 1, unless the label is multiplied by panoptic_label_divisor during encoding. Users who encode the labels as RGB images will run into a “silent” bug.
I am closing the issue since this is not a bug in the code but rather a matter of usage.

[markweberdev]

Thanks Sebastian for your comment! We do not regularly use semantic segmentation, so we're always looking for feedback!

Regarding your 2nd issue, I'd recommend to use a panoptic_label_divisor of 1, if you don't want to rescale your labels. This way, we make sure that there are no other side-effects in the code.

Regarding your 1st issue: Could you provide some more information on how you encoded the labels into tfrecords? If my understanding is correct, the issue is that you encode the image as a 3 channel image, but it's read out as 1 channel which leads to issues?

Best,
Mark

[sebastian-woessner]

Regarding the 1st issue:
I encoded the images as 3 channel images. The problem is that setting channels=1 as arg for tf.io.decode_image() controls the number of channels after decoding rather than the expected number of channels of the input. Thus, a 3 channel image is converted to a 1 channel grayscale image. This leads to an error if users follow the standard practice of encoding images as 3 channel images as in data/build_cityscapes_data.py.
Regarding the 2nd issue:
I agree that setting panoptic_label_divisor to 1 is the best solution. However, the code block before requires panoptic_label_divisor to be greater than 1 if ignore_label is greater than 1.

[aquariusjay]

Hi @sebastian-woessner,

Thank you for reporting the issue.
Most of our codes assume panoptic label format (panoptic = semantic * label_divisor + instance).
As a quick workaround for training pure semantic segmentation datasets, you could (1) set panptic_label_divisor to be a large value (say 1000), which should be compatible with ignore_label 255 (assume you are using PASCAL or Cityscapes), and (2) encode instance identity 0 when converting the dataset.

Cheers,