How are the masks calculated for length of stay, decomposition and in-hospital mortality? about mimic3-benchmarks

[hrayrhar]

When we are doing multitasking, for each input we have multiple outputs (mortality, remaining length of stay at each hour, decompensation at each hour and phenotypes). There are stays for which some of the outputs are not present. For example, when the stay is shorter than 2 days, the mortality label will not be provided. The masks are introduced to help us to keep everything in the same format. Moreover, later in the neural networks we will multiply the loss function of each output with the mask of that output. Whenever the mask is 0, that loss function will not be added into our total loss function.

[activatedgeek]

Oh I see that makes sense. Thank you so much for all the work on this dataset!

[activatedgeek]

Hey @Harhro94, I'm sorry for reopening the issue, but I observed in the Multi Task dataset, there are sequences which contain information of only < 48 hours. I tried looking for an explanation in the paper but could not. Sorry if I missed it. Do you mind telling how that case is handled to predict the In-Hospital Mortality? (which the paper states is only done at t = 48).

[hrayrhar]

That is possible, in that case the multitask network will try to predict length of stay, decompensation and phenotyping, but not in-hospital mortality. In general, for each sample we predict all labels that are present for that sample.

[activatedgeek]

Hi @Harhro94,

On further inspection and crashing codes, I found this error as well:

ValueError: Only one class present in y_true. ROC AUC score is not defined in that case.

It appears for some files, phenotype labels are all zeros. Could this be possible bug or this is a known possibility?

I discovered this while I was computing my ROC AUC score.

[hrayrhar]

@salmanazarr, that is not a bug. Probably you are running the scripts on a small subset of the dataset, where there is no positive cases for some phenotype. If you run the script on the whole dataset, you will not get that error.
Another option is to catch that ValueError and return ROC AUC=nan.