Comments (5)
Okay thank you @patrickvonplaten ! That explanation makes a lot of sense~
from diffusers.
from diffusers.
Hey @vvvm23,
It's set to False because we don't want to train those parameters. I followed the implementaton of the original model here: https://github.com/yang-song/score_sde_pytorch/blob/1618ddea340f3e4a2ed7852a0694a809775cf8d0/models/layerspp.py#L37
Does this make sense?
from diffusers.
I somewhat misphrased my original question, I'm aware setting requires_grad
to False
prevents that particular parameter from accumulating gradients, essentially stopping the training of those parameters.
But why would we not want to train the noise level embeddings? Or is this just a simple, fixed (albeit randomly initialised) projection from a per-batch noise value to a different space, which would later have some learned transformation applied to it?
Thanks!
from diffusers.
Hey @vvvm23,
sinusoidal position features like GaussianFourierProjection
don't need training because every embedding already has a distinctly different vector that the model can use a "cue" to know what time position has been passed to it.
If one wants to train position embedding vectors (or time embedding vectors here), one can just randomly initialize such a vector and let the model learn it. If however we use sinusoidal embeddings, there is no need to learn it
from diffusers.
Related Issues (20)
- Mismatching size in matmul when using StableDiffusionInstructPix2Pix pipeline with IP-Adapter HOT 3
- A strange time cost in denoising loop HOT 5
- set mixed_precision="fp16",but the model is not fp16 HOT 5
- Support OLSS Scheduler
- Something went wrong HOT 1
- Make the sampling loop of pipelines modular, with designated inputs that can be handled iteratively with functions and a default sampler step function HOT 2
- Feature Request: new release HOT 3
- Inquiry About Using Non-Square Images for ControlNet Training HOT 1
- I feel confused about this TODO issue. how to pass timesteps as tensors? HOT 1
- Multi-controlnet formatting issue HOT 1
- CLIP Training Example Bug - Overfitting HOT 1
- with XL ,When the image is enlarged for viewing,The drawn image will appear as red dots in most cases HOT 8
- deepfloyd stage 2 crashes with tensor size mismatch when input image size is not divisible by 8 HOT 2
- examples/community/lpw_stable_diffusion_xl.py Not correctly decoded HOT 1
- Severe difference with A1111 HOT 5
- Index 11 is out of bounds for dimension 0 with size 11 on concurrent pipeline calls HOT 1
- RuntimeError: Input type (c10::Half) and bias type (float) should be the same HOT 2
- Multi-vector Token already in tokenizer vocabulary. Please choose a different token name HOT 2
- FlashFace implementation in diffusers
- Support Residual Classifier-Free Guidance (RCFG) ?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from diffusers.