Comments (8)
from diffusers.
Thanks for bringing this up. Possible for you to show a comparison between what happens when you zero out like the way you mentioned compared to the existing approach?
I've checked the values of the embeds, and classifier-free guidance at inference time definitely makes use of the zero embed and not just "", which end up producing very different results.
The SD IP2P pipeline uses ""
, though when negative prompt is not provided:
However, it makes use to zeros_like
for the unconditional image embeddings:
from diffusers.
the base model was trained using it, so i figured aligning with the base model's training and inference has better results.
from my own tests, i can now reduce the step count required when running the default config on the SDXL pipelines, eg. force zeroes is set to True
I also have much better learning. this model started from ptx0/terminus-xl-velocity-v1
and it was unable to spell.
1000 steps of tuning later:
the base model was trained using ""
and it never really ends up with better CFG performance... but now it does!
from diffusers.
see the base SDXL pipeline:
# get unconditional embeddings for classifier free guidance
zero_out_negative_prompt = negative_prompt is None and self.config.force_zeros_for_empty_prompt
if do_classifier_free_guidance and negative_prompt_embeds is None and zero_out_negative_prompt:
negative_prompt_embeds = torch.zeros_like(prompt_embeds)
negative_pooled_prompt_embeds = torch.zeros_like(pooled_prompt_embeds)
and the config:
{
"_class_name": "StableDiffusionXLPipeline",
"_diffusers_version": "0.19.0.dev0",
"force_zeros_for_empty_prompt": true
}
from diffusers.
Ah makes a ton of sense. Do you want to take a step at opening a PR to fix this?
from diffusers.
can i also open the pull request for all of the other training examples, to add general dropout capabilities to them?
from diffusers.
We can open that up for the community. This way everyone gets to participate.
from diffusers.
like the ticket for updating the fp16 error? #6231
from diffusers.
Related Issues (20)
- Add Conditional Diffusion Distillation
- [`Docs`] JAX/Flax Installation HOT 1
- `AsymmetricAutoencoderKL`: missing `generator` argument in `decode()` called from `forward()` HOT 2
- [0.27.2]: from_single_file: 'AutoencoderKL' object has no attribute '__name__'. Did you mean: '__ne__'? HOT 5
- scheduler文件无法识别 HOT 2
- EDMEulerScheduler fails when fineturning CosXL_edit model. HOT 2
- Bug in FlaxDiffusionPipeline's from_pt option in Google Collab's TPUv2 HOT 1
- [0.28.0]: from_single_file doesn't work with custom checkpoints HOT 6
- $100 Free Google Play Gift Card Codes Instantly 2024
- Free Google Gift Card (Redeem Code) Generator 2024
- Free Google Play Gift Card Codes - Free Redeem Codes 2024!
- fReE!!✔Google Play Gift Cards Codes Redeem 2024
- Free $750 Google Play Gift Card Codes - Free Redeem Codes 2024!
- fReE!! Dice Dreams Free Rolls 2024- Daily Links
- Daily Links Dice Dreams Free Rolls (June 2024)
- LaTeSt!!**Dice Dreams Free Rolls (June 2024) - Daily Links
- ~Real$} Dice dreams free rolls {2024} Updated Daily #WIN24
- [[GeT!]] Dice Dreams Free Rolls (June 2024) - Daily Links
- prs-eth/marigold-depth-lcm-v1-0 is not available on huggingface HOT 5
- "Input is in incorrect format." incorrect error message HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from diffusers.