carzit / sd-webui-samplers-scheduler

a samplers scheduler for stable diffusion webui

sd-webui-samplers-scheduler Seniorious

Introduction

A samplers scheduler which can apply different sampler in diffrent generation steps.

I hope it will be helpful to achieve a balance between generation speed and image quality.

My paper on arXiv: Sampler Scheduler for Diffusion Models

WebUI Version

Now compatible with all.

The script scripts/load_Seniorious.py will load corresponding script according to your SD WebUI version automatically.
(Seniorious.py for version <= 1.5.2, Seniorious_16.py for version between 1.6.0 and 1.8.0, Seniorious_19.py for version >= 1.9.0)

How to use

This repository is a extension for sd webui. Just place it in the extension folder!😉

Choose the Sampler Seniorious, Seniorious Karras or Seniorious Exponential to enable the samplers scheduler.

Seniorious uses nomal noise schedule, Seniorious Karras uses the noise schedule recommended in Elucidating the Design Space of Diffusion-Based Generative Models (Karras et al., 2022) and Seniorious Exponential uses exponential noise schedule recommended in Improved Order Analysis and Design of Exponential Integrator for Diffusion Models Sampling

Note that SD WebUI has implemented the separation of samplers and noise schedules since version 1.9.0. Therefore, Seniorious Karras and Seniorious Exponential will be deprecated since 1.9.0. Just like other samplers in SD WebUI, you can specify the noise schedule used by Seniorious in the Schedule Type.

This samplers scheduler provides 8 sampler units (Sampler 1-8).

You can choose what kind of sampler used in each unit(choose None to unable), and the inference steps for each unit.

The image generation process will follow the configurations of these 8 units in sequence.

Attention: The total steps should be equal to the sum of the steps in every unit!
Open the Check accordion and press the Check button to check total steps.
Total steps in Seniorious shows the sum of steps in your Sampler Scheduler Settings and Total steps required shows the steps you set in webui. They must be equal.

Available Samplers

14 kinds of mainstream samplers in k-diffusion are available:

• Euler
• Euler a
• Heun
• Heun++
• LMS
• DPM2
• DPM2 a
• DPM++ 2S a
• DPM++ SDE
• DPM++ 2M
• DPM++ 2M SDE
• DPM++ 3M SDE
• Restart
• LCM

You can also choose Skip to skip certain steps.

Heun++ is my test version.
Restart is a new sampler in SD WebUI since version 1.6, using the recommended hyperparameters recommended in Implements restart sampling in Restart Sampling for Improving Generative Processes.
LCM is a new sampler used in latent consistency model(LCM).

FID Result

I calculate the FID Score based on EDM.

config:

• seeds=0-49999
• network=https://nvlabs-fi-cdn.nvidia.com/edm/pretrained/edm-cifar10-32x32-cond-vp.pkl
• NFE=30

Sampler[steps]	Type	FID
Euler[30]	Global ODE	4.20923
Heun[16]	Global ODE	8.8082
DPM++2M[30]	Global ODE	2.06226
DPM2[16]	Global ODE	1.94304
Euler a[30]	Global SDE	14.5688
DPM2 a[16]	Global SDE	3.58586
DPM++ 2S a[16]	Global SDE	9.74262
DPM++ SDE[16]	Global SDE	17.807
Heun[10] - DPM2[5]	Sampler Scheduler ODE	1.86846
DPM2 a[10] - DPM2[5]	Sampler Scheduler SDE+ODE	1.84535

Discretely scheduling different samplers during the sampling process has proven to be effective at a practical level.

Examples

ODE Samplers

BRISQUE Score:

Sampler	BRISQUE
Euler	23.3771
DPM++ 2M	27.5705
DPM++ 2M Karras	24.8244
Heun	21.4943
DPM 2	22.1520
Seniorious	21.3698
Seniorious Karras	21.6955

Shared Parameters:

• Prompt: best quality, masterpiece, 1girl, solo, standing, sky, portrait, looking down, floating light particles, sunshine, cloud, depth of field, field, wide shot
• Negative prompt: badhandv4, By bad artist -neg, EasyNegative, NegfeetV2, ng_deepnegative_v1_75t, verybadimagenegative_v1.3
• Steps: 30
• CFG scale: 7
• Seed: 2307198650
• Size: 512x768,
• Model: anything-v5-PrtRE

Sampler Scheduler Parameters:

Unit	Sampler	Steps
Sampler1	Heun	10 steps
Sampler2	DPM++2M	10 steps
Sampler3	Euler	10 steps

*Seniorious and Seniorious Karras use the same parameters in this example.

Put ODE and SDE Samplers Together

Sampler Scheduler Parameters:

Unit	Sampler	Steps
Sampler1	DPM2 a	20 steps
Sampler2	DPM2	10 steps

I recommend to use SDE in the early sampling steps and ODE in the later sampling steps to solve the inherent problems previously caused by using either singly.

More

The idea of this extension was inspired by Seniorious, a Carillon composed of different talismans.

Different talismans correspond in sequence to make Seniorious a powerful weapon, and so do the samplers in the samplers scheduler.

In the end, many thanks to Chtholly Nota Seniorious, the happiest girl in the world.

sd-webui-samplers-scheduler's Issues

diffuser上实现混合采样

请问您的混合采样可以在diffuser库上实现吗

The Seniorious can no longer be found in the new version

I don't know if this project is still under maintenance, Just found that the webUI was in the process of being updated, Decouple the sampler and scheduler, It is no longer possible to find it in the sampler Seniorious, I'm looking forward to seeing it in comfyUI, I'm also trying to get a little more tricky to try custom samplers

Can we support 1.6.1?

Today i read paper and find this page,but i cant use it in 1.6.1...
Even i switch to 1.5.2,it still happended errors.

Traceback (most recent call last):
File "D:\sd-webui-aki-v4.1\modules\call_queue.py", line 58, in f
res = list(func(*args, **kwargs))
File "D:\sd-webui-aki-v4.1\modules\call_queue.py", line 37, in f
res = func(*args, **kwargs)
File "D:\sd-webui-aki-v4.1\modules\txt2img.py", line 62, in txt2img
processed = processing.process_images(p)
File "D:\sd-webui-aki-v4.1\modules\processing.py", line 677, in process_images
res = process_images_inner(p)
File "D:\sd-webui-aki-v4.1\extensions\sd-webui-controlnet\scripts\batch_hijack.py", line 42, in processing_process_images_hijack
return getattr(processing, 'controlnet_original_process_images_inner')(p, *args, **kwargs)
File "D:\sd-webui-aki-v4.1\modules\processing.py", line 794, in process_images_inner
samples_ddim = p.sample(conditioning=p.c, unconditional_conditioning=p.uc, seeds=p.seeds, subseeds=p.subseeds, subseed_strength=p.subseed_strength, prompts=p.prompts)
File "D:\sd-webui-aki-v4.1\modules\processing.py", line 1054, in sample
samples = self.sampler.sample(self, x, conditioning, unconditional_conditioning, image_conditioning=self.txt2img_image_conditioning(x))
File "D:\sd-webui-aki-v4.1\modules\sd_samplers_kdiffusion.py", line 464, in sample
samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args={
File "D:\sd-webui-aki-v4.1\modules\sd_samplers_kdiffusion.py", line 303, in launch_sampling
return func()
File "D:\sd-webui-aki-v4.1\modules\sd_samplers_kdiffusion.py", line 464, in
samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args={
File "D:\sd-webui-aki-v4.1\python\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "D:\sd-webui-aki-v4.1\extensions\sd-webui-samplers-scheduler\scripts\Seniorious.py", line 811, in seniorious
x = samplers[i](model=model, x=x, sigmas=s, extra_args=extra_args, callback=callback_)
TypeError: 'NoneType' object is not callable

Have you considered mixing LCM?

I've been heavily involved in using LCM which I can use to produce 512x512 4 step images in 45ms on my 4090. I'm a retired performance specialist from Microsoft.

A couple of weeks ago I started exploring mixing not only schedulers but models and seem to have stumbled across a way to get even better quality than standard LDM but with some of the speed of LCM. I call the tool I wrote 'SDHybrid'. In the image below the left image is standard LDM 20 steps. The right image is 6 step LCM(extremely fast). The middle image is where I take the partially denoised latent and pass it to LCM for the remainder of the generation. NOTE: The tool is based on my RTSD tool where you can move the sliders and get realtime feedback to explore ideas.

Note that the statement:

only a single sampler algorithm can be specified on all sampling steps in the generative process.

isn't strictly true. Some samplers combination do work as is as I've discovered through rapid realtime experimentation. The mixing of not just a scheduler but also LDM and LCM was the breakthrough for me. I've posted about this on reddit and a few discord groups.

Any interest? I have a great setup. Ubuntu 22.04 i9-13900K and a 4090. I can apply TensorRT or torch.compile for super speed. Right now I'm writing a diffusers pipeline to do this Hybrid scheduler approach I've been using.

SDNEXT / LOADING ERROR

15:23:31-459502 ERROR Module load: extensions\sd-webui-samplers-scheduler\scripts\load_Seniorious.py: ValueError
Traceback (most recent call last)
│ C:\AI\SDNEXT\automatic\modules\script_loading.py:21 in load_module │
│ │
│ 20 │ │ │ with contextlib.redirect_stdout(io.StringIO()) as stdout: │
│ ❱ 21 │ │ │ │ module_spec.loader.exec_module(module) │
│ 22 │ │ │ setup_logging() # reset since scripts can hijaack logging │
│ in exec_module:883 │
│ in _call_with_frames_removed:241 │
│ │
│ C:\AI\SDNEXT\automatic\extensions\sd-webui-samplers-scheduler\scripts\load_Seniorious.py:18 in │
│ │
│ 17 │
│ ❱ 18 if int(version) >= 6: │
│ 19 │ from Seniorious_16 import * │
ValueError: invalid literal for int() with base 10: '#'