[DetectionDataset] - extend `from_coco` and `as_coco` with support for masks in RLE format about supervision

[David-rn]

Hi @SkalskiP, sure! It does sound okay!

[David-rn]

Hi @SkalskiP 👋 . Don't worry about the confusion! I have checked the backlog and if I'm not wrong seems that all tasks are assigned or someone is already working on it. So if any other new issue arises I'll be glad to help 👍

[emSko]

@SkalskiP In the future, I will definitely do so. Lesson learned.

Once again @David-rn I'm really sorry about that situation.

[David-rn]

Hi, great job with supervision! I would like to try contributing, but first I have some doubts regarding this issue. According to coco data format the RLE format is used only when iscrowd=1 but I don't see this attribute in Detections class so, does this mean that the idea is to select one type of segmentation storage when loading or saving annotations? Thanks in advance!

[SkalskiP]

Hi @David-rn 👋🏻 I'm really glad you want to contribute to supervision. It's true that detections don't have an iscrowd field, and we don't need it but don't worry about that for now. Let's start with implementing two functions:

• mask_to_rle, which would convert a 2D boolean np.array into a valid RLE (Run-Length Encoding) representation.
• rle_to_mask, which converts an RLE representation back into a 2D boolean np.array.

In the second phase, we'll discuss how to integrate them into methods as_coco and from_coco. Does that sound okay?

[SkalskiP]

@David-rn I'm excited. Let me know if you need any help. I'm assigning this issue to you. Good luck!

[emSko]

Hi, I have not noticed that someone is interested in this issue and already have done some work (encode/decode function+modification to coco_annotations_to_detections+tests).

If you have not started implementation @David-rn I would be happy to complete this task.

[David-rn]

Hi @emSko , I was about to start but I'm not sure if there is any procedure in this situation. @SkalskiP how is this normally handled?

[emSko]

Hi @David-rn, thanks, and sorry for the mix-up.

SAVING DETECTIONS:
@SkalskiP I have opened the PR for this issue. It is still a work in progress since I am not sure how to handle saving detections to coco format. For sure the additional flag is needed and I have added a comment in the place where I think it is appropriate. But where and how to pass it?

I have 2 ideas for that however they are quite limiting.

1. Add optional bool parameter to as_coco method that determines if ALL detections should be saved in RLE format or not. The same format will be applied to all images in the dataset.
2. Add optional list parameter to as_coco method that allows to specify which images (by filenames) will have masks saved in RLE format. All annotations for a single image will have the same format (either RLE or polygon).

I am not sure how to specify the mask format on a single annotation/detection level.
Do you have any other ideas for this problem?

COLAB NOTEBOOK:
Here is the colab notebook where the encoding/decoding scheme is tested.
Is that the kind of notebook that is sufficient to test a new feature (requirement mentioned in the first comment)? Could you provide an example of what would you expect?

[SkalskiP]

Hi @David-rn and @emSko 👋🏻

I'm very sorry to see such confusion. I've thought about how to resolve this situation, and especially considering that @David-rn hasn't started working on the task yet and @emSko already has a draft PR open, we will allow @emSko to continue working on this task.

@David-rn, I'm very sorry, and I hope you won't hold it against me. 🙏🏻 If you had already started writing your code, I would have given you priority since you were already assigned to the task. Is there another task in our backlog that you might be interested in?

@emSko, standard practice in larger open-source projects is to inform under the GH issue that you want to work on the task, which helps avoid such misunderstandings. As for your PR, it's a good start! 🔥 I have already left comments under it.