Disagreement for macro f1 with torchmetrics and sklearn about torcheval

[gounley]

Ah ok, I found the error - it was in our test environment. I'll close the issue.

[JKSenthil]

Hi @gounley thanks for bringing this up. I'm not able to replicate this locally on my end. Can you try and see if using torcheval-nightly also results in the discrepancy?

[gounley]

Nightly (2023.8.20) gives me the same incorrect result.

[JKSenthil]

if you create a new environment and install torcheval-nightly, does it still produce the same incorrect result?

[gounley]

I made a new environment on a different machine and the result is still the same.

Versions:
Collecting environment information...
PyTorch version: 2.0.1
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 13.5.1 (arm64)
GCC version: Could not collect
Clang version: Could not collect
CMake version: Could not collect
Libc version: N/A

Python version: 3.11.4 (main, Jul 5 2023, 08:40:20) [Clang 14.0.6 ] (64-bit runtime)
Python platform: macOS-13.5.1-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Apple M2

Versions of relevant libraries:
[pip3] numpy==1.25.2
[pip3] torch==2.0.1
[pip3] torchaudio==2.0.2
[pip3] torcheval-nightly==2023.8.20
[pip3] torchmetrics==1.1.0
[pip3] torchvision==0.15.2
[conda] numpy 1.25.2 pypi_0 pypi
[conda] torch 2.0.1 pypi_0 pypi
[conda] torchaudio 2.0.2 pypi_0 pypi
[conda] torcheval-nightly 2023.8.20 pypi_0 pypi
[conda] torchmetrics 1.1.0 pypi_0 pypi
[conda] torchvision 0.15.2 pypi_0 pypi