Functional Confusion Matrix with Multi-Label about torchmetrics

[github-actions]

Hi! thanks for your contribution!, great first issue!

[SpontaneousDuck]

So I guess the more usefull expected behavior would be the Multi-Label Confusion Matrix output like sklearn.metrics.multilabel_confusion_matrix where the output is (n_outputs, 2, 2). It appears like torchmetrics is kind of doing this but extending the 2x2 matrix to the size of num_classes where the multi-label confusion matrix occupies the first two rows and columns. pytorchmetrics is also then summing along the first axis and only returning the overall matrix instead of class-wise.

[FlorianMF]

I'm in the same case as you.
The docstring of confusion_matrix says that it works for multi-label targets.
It calculates the confusion matrix in a binary way. Using num_classes > 2 or num_classes=2 gives the same result except for the additional rows and colums (all zeros) for the additional classes.

I don't expect the same output as you. In my opinion the output should be a list/Tensor of binary confusion matrices of length num_classes like in this sklearn example

Personally I iterate over the the predictions and targets as _auroc_compute does:

[confusion_matrix(class_preds[:, idx], y_true[:, idx], num_classes=2)
                  for idx in range(class_preds.shape[-1])]

[SpontaneousDuck]

Thanks for the response! Yeah I look in to sklearn and understand what I should be getting now. I am doing what you suggested to get the response now. confusion_matrix should match sklearn's if it was performing correctly right?

[FlorianMF]

IMO yes if the philosophy is to create a pytorch optimised sklearn.