Multi GPU training about gigagan-pytorch

[dprze]

@lucidrains thank you, now I'm able to run multi gpu training - although for some reason one iteration takes longer than on single gpu ;)

sorry for the line mix-up, it was 2479 indeed, but I was confused as I couldn't find the troublesome inplace operation myself

[lucidrains]

@dprze thanks Dominika!

took care of the first bullet point, but wasn't sure if you gave me the right line for the second, as i don't see any inplace updates

did you mean this line? https://github.com/lucidrains/gigagan-pytorch/blob/main/gigagan_pytorch/gigagan_pytorch.py#L2479

[lucidrains]

@dprze spotted a few issues and put in a few fixes

i don't have access to my deep learning rig (w/ multi-gpus), so let me know if it works or not

[lucidrains]

no problem! happy training!

I spent a year a long time ago training GANs and would not wish it on my worst enemy 🤣