squares cut out of predicted region about root_painter

[Abe404]

We found RootPainter to be effective for extracting the plants. We will provide more details of the approach we used soon. The image above shows some of the results and demonstrate how ensembling can further improve accuracy. The image on the left uses a single final model segmentation, whilst the right image was generated using a segmentation generated from an ensemble of the last 5 models. This is functionality that is built into RootPainter (just select multiple models instead of 1 when performing segmentation).

[Abe404]

Thanks for reporting this. To reproduce I believe I'd need access to the dataset.

For now I can give my best guess.

Although the errors look unusual, I suspect that they are due to the model not yet being able to handle the outer regions/edges of the images. I believe the model/dataset requires more foreground/leaf (positive) labels to detect the leaves properly.

I suspect that the problem will go away with increased interactive training time. If you label the false negative regions (the missed leaves) in red, then the model should adapt and learn to detect the leaves properly.

If the issue isn't resolved with extra interactive training time, then I'd be happy to have a zoom meeting and see if I can provide better guidance.

BTW in the paper we describe a procedure called 'create training dataset' that I recommend to speed up the training process when images are large.

Please let me know how it goes.

[Abe404]

To provide a little more context on why the errors might look like this:
The network processes images after they have been split up into patches. This is primarily due to limitations in GPU memory that mean we can't always naively process full images.

In the case of RootPainter an input patch/tile has a size 572x572 and an output size of 500x500. The network uses information from within the entire patch to decide on the class of each individual pixel. That means that something small (or alternatively a global property) within a patch may cause all predictions within that patch to be incorrect.

It is also possible that a bug has been introduced/discovered, but for now I suspect the model just needs more data and positive/foreground annotation in particular. Then it should learn to handle the patches/tiles we can see detected as background incorrectly in your image.

I hope that helps.

[geezacoleman]

That makes a lot of sense - thanks for clarifying the process. I'll keep adding more data, as you say it looks like that is probably the issue here.

[Abe404]

We figured out a way to segment this dataset. There weren't any bugs with the software, but for this particular dataset a two stage course-to-fine approach (first localisation on smaller images and then segmentation of the fine details) seemed to be much more efficient for training an accurate model.

[Abe404]

Turns out there was actually a bug with RootPainter, or more specifically pytorch when using half precision for inference on recent versions of google colab.

See https://mobile.twitter.com/Abe_404/status/1501620198109024265

And then: https://mobile.twitter.com/Abe_404/status/1501622887102529542