Val-Test metrics need to work accurately across batches and across data points about ganslate

[ibro45]

this is without DDP? do you think that torchmetrics would take care of that?

[surajpaib]

Not dependent on DDP really, its about how we aggregate metrics. Not sure if torch metrics would take care of it, the issue lies within how we do metrics over multiple data points. I've fixed it for now, I'll push the update in a bit.

[ibro45]

the issue lies within how we do metrics over multiple data points

you mean how we deal with data points in the batch, no? if that's the case, we can actually get rid of how we do it and just use torchmetrics since it handles batches anyways

[surajpaib]

Nope, I mean doing it over, for example, multiple patients - then we average over each patient, if we already average over batch, then the final average will be incorrect.