No instance of 'pedestrian' in ins_seg/polygon/ins_seg_val.json and ins_seg_train.json about bdd100k

[pcicales]

Can confirm @deepakksingh , to_coco breaks person annotations. Here are the annotation class counts for the 8 classes after to_coco in the training set (7000 images):

{1: 0, 2: 273, 3: 67034, 4: 3178, 5: 1253, 6: 52, 7: 262, 8: 462}

Aside from the missing class, there are also severe class imbalances, and many noisy annotations (e.g. class objects not segmented, noisy segmentation mask edges, random annotations of bushes/trees labeled as trains/cars) in the training and val folds, which makes training and evaluating a model for instance segmentation very difficult. I believe they will need to correct these annotations for this dataset to be viable for instance segmentation, or they will need to publish a list of the problematic images/annotations. In their paper, they also report performance well below the COCO benchmark (scores that are less than half of COCO SOTA), implying that the combination of class imbalance and noisy annotations is causing real problems (this task should certainly not be harder than COCO, even if it is scarce). I also experienced similar issues with the seg_track_20 dataset.

Hopefully these issues can be resolved quickly, this dataset has a lot of untapped potential.

[deepaksinghcv]

Yes, there are no instances with category_id as 1 after the conversion using to_coco.

[XiaLiPKU]

Could the authors(@fyu) kindly verify this?
I downloaded the latest annotations of instance segmentation along with the latest image set and wanted to train mask-rcnn.
I notice that there are no instances of 'pedestrian' which is category id: 1.

Could someone please cross-check this.

There is really no pedestrian here, but are persons here.
That's a legacy problem, and our scripts have considered this issue.

To mapping it, you may call bdd100k.label.to_scalabel.bdd100k_to_scalabel, hope this may help you.

[fyu]

It is actually not a legacy issue. We tried to adopt cityscapes labels for segmentation. In cityscapes, the label is called person instead of pedestrian.

[deepaksinghcv]

Thank you for the quick replies.
I used the label.to_coco to convert

python3 -m bdd100k.label.to_coco -m ins_seg -i ./bdd/bdd100k/labels/ins_seg/polygons/ins_seg_train.json -o ./in_coco/inseg_train_coco.json -mb ./bdd/bdd100k/labels/ins_seg/bitmasks/train/
python3 -m bdd100k.label.to_coco -m ins_seg -i ./bdd/bdd100k/labels/ins_seg/polygons/ins_seg_val.json -o ./in_coco/inseg_val_coco.json -mb ./bdd/bdd100k/labels/ins_seg/bitmasks/val/

So I think that changing the "categories" json object in the generated coco files should resolve this issue.
Am I correct?

PS: Is the above command to convert correct?

I'm using Detectron2 to train Mask-RCNN on BDD's Instance Segmentation.

[deepaksinghcv]

Could the authors(@fyu) kindly verify this?
I downloaded the latest annotations of instance segmentation along with the latest image set and wanted to train mask-rcnn.
I notice that there are no instances of 'pedestrian' which is category id: 1.
Could someone please cross-check this.

There is really no pedestrian here, but are persons here.
That's a legacy problem, and our scripts have considered this issue.

To mapping it, you may call bdd100k.label.to_scalabel.bdd100k_to_scalabel, hope this may help you.

Hello @XiaLiPKU,
Where should i run bdd100k.label.to_scalabel.bdd100k_to_scalabel?
I also noticed that the original ins_seg_train.json and ins_seg_val.json have person as the category label, but after converting it to coco using label.to_coco it generates pedestrian category in categories, but there are no instances of it in annotations

[XiaLiPKU]

Thank you for the quick replies.
I used the label.to_coco to convert

python3 -m bdd100k.label.to_coco -m ins_seg -i ./bdd/bdd100k/labels/ins_seg/polygons/ins_seg_train.json -o ./in_coco/inseg_train_coco.json -mb ./bdd/bdd100k/labels/ins_seg/bitmasks/train/
python3 -m bdd100k.label.to_coco -m ins_seg -i ./bdd/bdd100k/labels/ins_seg/polygons/ins_seg_val.json -o ./in_coco/inseg_val_coco.json -mb ./bdd/bdd100k/labels/ins_seg/bitmasks/val/

So I think that changing the "categories" json object in the generated coco files should resolve this issue.
Am I correct?

PS: Is the above command to convert correct?

I'm using Detectron2 to train Mask-RCNN on BDD's Instance Segmentation.

Yes, these command are correct

[XiaLiPKU]

Could the authors(@fyu) kindly verify this?
I downloaded the latest annotations of instance segmentation along with the latest image set and wanted to train mask-rcnn.
I notice that there are no instances of 'pedestrian' which is category id: 1.
Could someone please cross-check this.

There is really no pedestrian here, but are persons here.
That's a legacy problem, and our scripts have considered this issue.
To mapping it, you may call bdd100k.label.to_scalabel.bdd100k_to_scalabel, hope this may help you.

Hello @XiaLiPKU,
Where should i run bdd100k.label.to_scalabel.bdd100k_to_scalabel?
I also noticed that the original ins_seg_train.json and ins_seg_val.json have person as the category label, but after converting it to coco using label.to_coco it generates pedestrian category in categories, but there are no instances of it in annotations

to_coco functions has included this conversion, so you don't have to explicitly call it.

So, what do you mean by " but there are no instances of it in annotations"?
You mean, there's no instance with category_id as 1?

[XiaLiPKU]

Thanks for your findings, there are really bugs in to_scalabel.py.
It has been fixed in #142.

Now you may need to pull the lasted version to your local repo

[deepaksinghcv]

Thank you @XiaLiPKU.

I pulled the latest commit and converted it. Now the converted json has instances for all the 8 categories needed for instance segmentation.

Image: from detectron2's log while training Mask-RCNN.
I notice that 109 images are being removed out of 7000 images. I will look into it.
Thank you once again :)