aauss / epi-quark Goto Github PK

current version:

"Scoring the detection, labeling, and prediction performance of disease outbreak detection algorithms across different algorithm families."

proposal:

"Based on labeled data, multi-class and multi-label scoring of outbreak detection, annotation or prediction algorithms."

This should be coherent with documentation (https://github.com/aauss/epi-quark/blob/master/README.md, https://github.com/aauss/epi-quark/blob/master/docs/source/index.rst).

This could be implemented by adding an optional column "weight" to the input case data:

If the column is absent, and weights are not computed via epi-quark, then all weights are 1.

Accordingly, one more value for parameter weighting of calc_score() should be added to the values allowed. (This seems safer to me than implicitly taking the user-defined weights when they are provided... this also ensures that the user chooses which type of weight they want, e.g., not user defined and time-space at the same time.)

They should be stored in a new input table of the following form:

The resulting data frame is used as argument of score functions computing the probabilities.

For outbreak labels not present: flat beside endemic and non-case.

Relevant for demographics or administrative units, for example.

In particular output, for chosen score types (e.g., accuracy) or for all available score types, the respective groups with minimum score. This allows for Rawlsian maximin strategy, see e.g. https://arxiv.org/abs/2006.13114

Relevant for scoring is the expected case number for each outbreak class and data cell.

Since each case should be allowed different probabilities of belonging to different outbreak classes, the expected case number need not be integer.

N.B. Since a case can belong to many classes, these probabilities need not sum up to 1 and there is no constraint on the expected case number beside being >= 0.

If I'm not mistaken no code change is necessary, only the description of value in the API and scorer documentations should be modified and sentence This value must be a positive integer. removed across https://github.com/aauss/epi-quark/blob/master/epiquark/api.py and https://github.com/aauss/epi-quark/blob/master/epiquark/scorer.py