Support Lumina T2I 5B flow matching T2I DiT model about diffusers

[bghira]

i was working on this support but after seeing the results of the model i'm not sure it's ready to be added yet:

there's a lot of residual noise - like, a lot.. it reminds me of Pixart Sigma's similar issues

[JincanDeng]

i was working on this support but after seeing the results of the model i'm not sure it's ready to be added yet:

there's a lot of residual noise - like, a lot.. it reminds me of Pixart Sigma's similar issues

I meet a similar problem when using EDM training method. Do you have any idea about how the noise arise arises?

[bghira]

@JincanDeng how are you doing caption dropout? zeroes or "" prompt encoded by both TEs?

[JincanDeng]

@bghira I use "" prompt with 0.1 probability for dropout.

[bghira]

SDXL relies on zeroing uncond space, try using torch.zeros_like()