Friction log from new user about diffusers

[osanseviero]

As a second step I trained a model with a new dataset based on the example script. It was very straightforward 🔥 and everything worked together very nicely, except for my GPU not allowing larger models, so I had to decrease resolution. I would have loved a push_to_hub method to upload it

[patil-suraj]

Thanks a lot for the feedback !

I1. (first code example):

• Yes, diffusers tag would be awesome.
• Yes, we could definitely add code snippet once the API is finalised.

I2 (first code example):
The example is failing because of renaming. GaussianDDPMScheduler is now DDPMScheduler. These will be fixed once we settle on names. Fixed this example for now :)

This also makes me wonder what's the difference between just using the diffusion pipeline directly as in the model card vs using DDPMScheduler + UNetModel approach as in the README. Is the pipeline approach just a wrapper of both?

We are aiming for two APIs here:

• press a button API with pipeline, where user could just load the pipeline and use it as it is for inference. This is more abstract and black-box API to play with pre-trained pipelines.
• barebone/follow-the-equation: Which offers more control , allows to mix and match different schedulers and models, write your own denoising loop. This is close to how these models are presented in research papers.

I3 (second code example)

Should num_inference_steps be len(noise_scheduler) as in first example?

some schedulers like DDIM allows you to use different num_inference_steps than what was used during the training to allow fast inference. So this is intended, num_inference_steps and len(noise_scheduler) can be different.

I4 and I6:
The models were added in a fast hacky way to get the initial version rolling. All of these will improved before formal release.

I would have loved a push_to_hub method to upload it

@anton-l is already working on it :)

[osanseviero]

I1

• Happy to help with setting up the code snippet + tag once the API is finalised.

I2

• Thanks!

I3

• Cool, then this is analogous to *Model classes and pipeline in transformers. Sounds exciting!

For push to Hub, feel free me to tag me in the PR once opened, it would be cool to have metrics and other metadata out-of-the-box

[patrickvonplaten]

Thanks for the feedback here @osanseviero - think we applied almost all of it now :-)