Output submissions seem to change out of nowhere??? about pneumothorax-seg-cnn

[jchen42703]

First big lead: Able to recreate my original 0.8500 cascade solution by taking the same ensembled_classification.csv and replacing all of the pneumothorax (+) patients' (=1) encoded pixels with those from the 0.8510 (segmentation only) submission. In doing so, I was able to perfectly recreate the 0.8500 submission.

Note that I was not able to recreate the 0.8500 solution by doing SegmentationOnlyInference on the positive ids and replacing those ids in the classification df. It just recreated the 0.76 submission-like results.
So what now?_

This implies that there is some sort of problem with how the predictions are ensembled/averaged. Why? Well, since SegmentationOnlyInference is confirmed to be good (inputs/output pipeline is good, because it was able to generate a 0.8510 solution), then there has to be something wrong with:

• how I extract the positive ids (highly unlikely because I hard-coded it)
• how predictions are averaged.
  • 0.8510 solution must have taken surrounding pixels outside of each group of 9 for each id and averaged those in to create another prediction.
  • I'm thinking this because the only major difference between the 0.8510 sub and the 0.85 sub is how the 0.8510 ensembles with the entire dataset prediction while the 0.85 sub ensembles with the positive subset predictions
  • I'm also thinking this because when visualizing the results from run_seg_prediction with the 9 swa models on only the classification-predicted positive set, I got the same garbage results as those from the 0.76 subs.

[jchen42703]

Another lead is how I don't sort the fpaths for x_test in the inference pipeline so the file ids associated with seg_ids don't match those at each idx for x_test; i.e. seg_ids[0] doesn't correspond to x_test[0]

[jchen42703]

The right answer seems to have been the second lead.