A strange time cost in denoising loop about diffusers

[bghira]

you might also want to try generating a flame graph on a Linux system.

[tolgacangoz]

In addition to what Sayak said, to my knowledge, PyTorch makes calculations asynchronously in GPU. So, could you put torch.cuda.synchronise() before each time.time()? Also, time.perf_counter() is recommended most of the time instead of time.time().

[FEIFEIEIAr]

To my knowledge, PyTorch makes calculations asynchronously in GPU. So, could you put torch.cuda.synchronise() before each time.time()? Also, time.perf_counter() is recommended most of the time instead of time.time().

Thanks!
I add torch.cuda.synchronize() after UNet, and the time seems right!

Also, I'm curious about what happened during torch.cuda.synchronize().
And it goes strange when I remove if self.do_classifier_free_guidance:, will that cause any problem?
Thanks again!

[sayakpaul]

Thanks for the detailed thread. Might be a good candidate for discussions.

Have you tried warming up the GPU before reporting the numbers? If not, I would suggest running a few warm-up steps first before reporting the numbers.

[FEIFEIEIAr]

Thanks for the detailed thread. Might be a good candidate for discussions.

Have you tried warming up the GPU before reporting the numbers? If not, I would suggest running a few warm-up steps first before reporting the numbers.

I also tried num_inference_steps=50, even in the last loop, the time did not change alot (even a little slower). So, I didn't try warming up.