What is the compute infrastructure used for the experiment? about mimic3-benchmarks

[hrayrhar]

I am afraid we cannot give a definite answer to your question, since we trained models on different machines. What is common that all models were trained on single GPUs (such as GTX 980, GTX 980, Tesla K20, Tesla K40, Tesla P100 PCI-E). Below is very general summary on run times for each task:

1. In-hospital mortality - training in-hospital mortality prediction task is the easiest and takes at most 2 hours.
2. LOS and Decompensation - if deep supervision is disabled then decompensation and length of stay models will require a lot of time to get the best performance. Usually it takes about 1-2 days. If deep supervision is enabled, then it will take at most one day.
3. Phenotyping - getting the best performance on the phenotyping task requires 4 days, but you can get very close score (0.773 AUC score) in just 6 hours.
4. Multitasking - training multitask models takes from 4 hours to 3 days depending which task is the most important for you.

A common rule for all models of all tasks is that when you have high dropout rate (>= 0.3) the model will overfit much later (2-4x more epochs) compared to the same model with zero dropout, but eventually will give better score on the test set. This is why getting the best performance requires a lot of training, while you can get a bit worse performance comparatively very quickly.

[turambar]

Can we close this issue?

[yulonglin]

Thanks for the reply! Will close this issue.