fizzledorf / ait Goto Github PK
View Code? Open in Web Editor NEWThis was orginally written by: https://github.com/hlky
License: Apache License 2.0
This was orginally written by: https://github.com/hlky
License: Apache License 2.0
Is there any information on how to reproduce the windows-native modules you have provided at https://huggingface.co/Fizzledorf/AITemplate_V1_V2/tree/main/modules/windows ? From what I've seen, the AITemplate codebase seems very linux-specific, and I can't find any information anywhere on cross-compiling from Linux to Windows either (not even with WSL). How were those modules made?
This is great, gives about a 40% speed boost for me when it works, but oddly going above above some upper limit of characters in the positive prompt throws a 'Error in function: AITemplateModelContainerRun' when executing. Syntax nor number of tokens doesn't seem to affect it. Nor does the same happen for negative prompt. And the limit is different depending on the nodes that are loaded, model used, etc but I don't notice a pattern there necessarily. With the default workflow with AITemplateLoader and AITemplateVAEDecode subbed/chained in, basev1.5pruned model, the character limit appears to be ~590 for instance.
Traceback attached. Any clue what's going on?
stack.txt
After changes in fb84060 compilation fails, works fine before.
python unet.py --hf-hub-or-path "runwayml/stable-diffusion-v1-5" --width 512 512 --height 512 512 --batch-size 1 1 --clip-chunks 10 --model-name "v15_unet_512_512" --work-dir "/app/tmp/"
Traceback (most recent call last):
File "AITemplate/unet.py", line 141, in <module>
compile_diffusers()
File "venv/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "venv/lib/python3.11/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "venv/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "venv/lib/python3.11/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "AITemplate/unet.py", line 104, in compile_diffusers
compile_unet(
File "AITemplate/ait/compile/unet.py", line 271, in compile_unet
Y = ait_mod(
^^^^^^^^
File "venv/lib/python3.11/site-packages/aitemplate/frontend/nn/module.py", line 389, in _call_impl
return forward_call(*input, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "AITemplate/ait/modeling/unet_2d_condition.py", line 394, in forward
sample = upsample_block(
^^^^^^^^^^^^^^^
File "venv/lib/python3.11/site-packages/aitemplate/frontend/nn/module.py", line 389, in _call_impl
return forward_call(*input, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "AITemplate/ait/modeling/unet_blocks.py", line 671, in forward
hidden_states = upsampler(hidden_states, upsample_size)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "venv/lib/python3.11/site-packages/aitemplate/frontend/nn/module.py", line 389, in _call_impl
return forward_call(*input, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "AITemplate/ait/modeling/resnet.py", line 71, in forward
x = nn.Upsampling2d(scale_factor=2.0, mode="nearest")(x, out)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "venv/lib/python3.11/site-packages/aitemplate/frontend/nn/module.py", line 389, in _call_impl
return forward_call(*input, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "venv/lib/python3.11/site-packages/aitemplate/frontend/nn/upsample.py", line 47, in forward
assert len(args) == 1
^^^^^^^^^^^^^^
AssertionError
Can AITemplate support the recent FreeU node added to ComfyUI, or at least its functionality? It uses a transformer patch which gets ignored by AIT inference. This is unfortunate since the FreeU algorithm can have very interesting effects on generation quality.
I tried poking around the code but I couldn't find a place to plug it in. It might be that it needs a specifically compiled FreeU module, but that goes over my head currently.
Is there a way for AIT to support FreeU? I'm not opposed to writing a patch myself, but I need a pointer in the right direction.
Thank you for your hard work on the AITemplate port for comfy!
I have found a bug, whenever you change a lora weight, change a lora or connect a new one, the generated image becomes a noisy mess, this is when using the keep loaded option enabled, and if disabled it can sometimes trigger the same type of generation when using a loRA even when not changing its weights nor adding/changing the lora itself.
The bug corrects when changing the whole model or by restarting comfy, seems related to the autoload AIT module feature. And is more likely to generate without lora change when using 3 or more loras.
The following image has only the AIT nodes and a custom CLIP encode which I tested using the normal CLIP encode and the bug persists
when will there be modules for custom batch sizes? I compiled my own, but I can't get the node to see them.
I played around with AMD's AITemplate fork on my 7900XTX a while ago, and it seems to be the fastest Stable Diffusion implementation on AMD hardware right now. That said, RDNA3 support hasn't been upstreamed yet, but it is available here: navi3_rel_ver_1.0
as far as I'm aware SDXL should be supported when using the node on SDXL, any idea?
Modules have to be compiled for a range of resolutions, batch sizes, windows and linux, and multiple cuda architectures. This process takes a long time, so the goal is to minimize the number of modules that have to be recompiled in case of an update like has just happened with 8px increment support, while maximizing the overall support.
The profiling that occurs when compiling AITemplate modules selects the underlying kernels based on resolution, batch size and cuda architecture.
Most importantly is that a module profiled on sm80 is not likely to work on sm75 because sm75 supports different kernels.
The other factor is performance, one kernel may perform better at higher resolution and another at smaller resolution.
The amount of memory allocated by the module is pre-calculated based on the profiling.
At the moment, there are modules for batch size 1, windows and linux, sm80. Check your cuda architecture here.
sm80 covers everything 80+. sm70 and sm75 however are unique. sm6X and older are unsupported. there needs to be some testing done on whether sm89 benefits from modules profiled for it
Note: v1 control
is for use with v1 controlnet modules, controlnet requires an accompanying unet module that accepts the additional inputs
vae modules are compatible across architectures, however there could be support for fp32 modules, these would work with the vae models that don't work in fp16
v1 is supported, v2 could be supported
React to the comments below to vote on what you will use
VRAM usage as estimated by AITemplate is in modules.json
, in mb, excluding the size of the weights, this might help narrow the choices of resolution and batch size.
It would also be great to verify these figures against what people are seeing.
Example:
https://i.imgur.com/8QTd3cH.png
Simple Setup to reproduce:
Example.txt
Hello! I try to use this AiTemplate plugin on linux, but I receive this error when using it on linux ubuntu 22.04:
Found 3 modules for linux xl sm80 1 1024 unet
Using b5caabe98aeb69bada9d1566c897aed66a84d4fb21f31482160d7ef9987f04fd
!!! Exception during processing !!!
Traceback (most recent call last):
File "/home/cybertimon/Repositories/ComfyUI/execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "/home/cybertimon/Repositories/ComfyUI/execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "/home/cybertimon/Repositories/ComfyUI/execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "/home/cybertimon/Repositories/ComfyUI/nodes.py", line 1206, in sample
return common_ksampler(model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise=denoise)
File "/home/cybertimon/Repositories/ComfyUI/custom_nodes/AIT/AITemplate/AITemplate.py", line 171, in common_ksampler
samples = comfy.sample.sample(model, noise, steps, cfg, sampler_name, scheduler, positive, negative, latent_image,
File "/home/cybertimon/Repositories/ComfyUI/custom_nodes/AIT/AITemplate/AITemplate.py", line 259, in sample
AITemplate.unet[module['sha256']] = AITemplate.loader.load_module(module['sha256'], module['url'])
File "/home/cybertimon/Repositories/ComfyUI/custom_nodes/AIT/AITemplate/ait/load.py", line 69, in load_module
self.download_module(sha256, url)
File "/home/cybertimon/Repositories/ComfyUI/custom_nodes/AIT/AITemplate/ait/load.py", line 56, in download_module
g.write(f.read())
File "/usr/lib/python3.10/lzma.py", line 200, in read
return self._buffer.read(size)
File "/usr/lib/python3.10/_compression.py", line 118, in readall
while data := self.read(sys.maxsize):
File "/usr/lib/python3.10/_compression.py", line 103, in read
data = self._decompressor.decompress(rawblock, size)
_lzma.LZMAError: Input format not supported by decoder
I'm on the sdxl branch.
Thanks for help!
This issue can serve as a discussion point for any issues related to compiling modules on Linux
My setup is as follows:
I use WSL, with Ubuntu 20.04
wsl --install Ubuntu-20.04
ubuntu2004.exe
CUDA installation is specific to WSL, for other distros see cuda-downloads.
Earliest known working CUDA toolkit version is 11.6, I'd recommend using the latest
wget https://developer.download.nvidia.com/compute/cuda/repos/wsl-ubuntu/x86_64/cuda-wsl-ubuntu.pin
sudo mv cuda-wsl-ubuntu.pin /etc/apt/preferences.d/cuda-repository-pin-600
wget https://developer.download.nvidia.com/compute/cuda/12.2.1/local_installers/cuda-repo-wsl-ubuntu-12-2-local_12.2.1-1_amd64.deb
sudo dpkg -i cuda-repo-wsl-ubuntu-12-2-local_12.2.1-1_amd64.deb
sudo cp /var/cuda-repo-wsl-ubuntu-12-2-local/cuda-*-keyring.gpg /usr/share/keyrings/
sudo apt-get update
sudo apt-get -y install cuda
Ubuntu 20.04 includes Python 3.8.10, minimum Python version is 3.8, and I use PyTorch nightly CUDA 12.1, PyTorch version working at time of AITemplate release was 1.12.1, so anything newer than that should work
The following instructions use the build of AITemplate mentioned here, it includes the pending PRs, it is set up for CMake so that is disabled here, CUDA path is set so nvcc is found, a venv is used
sudo apt install python3-pip python3-venv unzip
wget https://github.com/FizzleDorf/AIT/files/12283714/aitemplate.zip
unzip aitemplate.zip
python3 -m venv venv
mv aitemplate/ venv/lib/python3.8/site-packages/
venv/bin/python -m pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu121
venv/bin/python -m pip install click transformers diffusers accelerate
export CUDA_HOME=/usr/local/cuda
export PATH=${CUDA_HOME}/bin:${PATH}
export LD_LIBRARY_PATH=${CUDA_HOME}/lib64:$LD_LIBRARY_PATH
export AIT_USE_CMAKE_COMPILATION=0
To note, using that build is not necessary, and it will eventually become outdated, Installation from source is like
git clone --recursive https://github.com/facebookincubator/AITemplate
cd python
python setup.py bdist_wheel
pip install dist/*.whl
Until the pending PRs mentioned are merged they will need to be merged to a local branch before installation, or the changes applied to the installed package.
Here's some SDXL UNet modules I compiled earlier, covering 1024, 1536, 2048 max resolution, batch size 1, sm80
unet_xl_linux.zip
This is actually a 7z
archive rather than .zip
due to GitHub's file extension constraints.
cc @FizzleDorf These UNet modules are ready uploaded to Huggingface and added to modules.json, existing Linux VAE modules will work for SDXL with the fp16 fixed version
Sorry this isn't technically related to this project but to your hugginface AITemplate models.
I was just wondering, if it is possible to have AIT models for >1 batchsize for VAE Decode/Encode?
Thanks for your hard work!
I'm contemplating how to support AIT in the Detailer node, and it seems quite challenging to apply in a different way.
So, when the model passed to the Detailer is an AIT model, I intend to approach it by restricting the upscale factor to the resolution supported by AIT.
To achieve this, I need to determine the supported resolutions from the model passed to the Detailer. Is it possible for the model to provide such information?
Reproduction:
I have been using AIT for a while now and while it is REALLY FAST there are some kinks I've found out.
I'm using the MultiAreaConditioning node from this repo: https://github.com/Davemane42/ComfyUI_Dave_CustomNode
And when using the LoadAITemplate node connected to the normal KSampler it pukes out the following error.
It does with any combination in the inputs of MultiAreaCond, besides the default of two conditions only
And this is the related workspace to the node, all other nodes to the right of the pic are the usual hi res fix and face detail combo
Is there any chance of adding compatibility or a patch for it?
got most recent version of this repo, it errors out
File "G:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1527, in call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "G:\ComfyUI_windows_portable\ComfyUI\comfy\k_diffusion\external.py", line 125, in forward
eps = self.get_eps(input * c_in, self.sigma_to_t(sigma), **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "G:\ComfyUI_windows_portable\ComfyUI\comfy\k_diffusion\external.py", line 151, in get_eps
return self.inner_model.apply_model(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "G:\ComfyUI_windows_portable\ComfyUI\comfy\samplers.py", line 311, in apply_model
out = sampling_function(self.inner_model.apply_model, x, timestep, uncond, cond, cond_scale, cond_concat, model_options=model_options, seed=seed)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "G:\ComfyUI_windows_portable\ComfyUI\comfy\samplers.py", line 289, in sampling_function
cond, uncond = calc_cond_uncond_batch(model_function, cond, uncond, x, timestep, max_total_area, cond_concat, model_options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "G:\ComfyUI_windows_portable\ComfyUI\comfy\samplers.py", line 241, in calc_cond_uncond_batch
c['control'] = control.get_control(input_x, timestep, c, len(cond_or_uncond))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "G:\ComfyUI_windows_portable\ComfyUI\custom_nodes\AIT\AITemplate\AITemplate.py", line 416, in get_control
control = self.aitemplate_controlnet(x_noisy, t, cond, self.cond_hint)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "G:\ComfyUI_windows_portable\ComfyUI\custom_nodes\AIT\AITemplate\AITemplate.py", line 378, in aitemplate_controlnet
return controlnet_inference(
^^^^^^^^^^^^^^^^^^^^^
File "G:\ComfyUI_windows_portable\ComfyUI\custom_nodes\AIT\AITemplate\ait\inference.py", line 146, in controlnet_inference
exe_module.run_with_tensors(inputs, ys, graph_mode=False)
File "G:\ComfyUI_windows_portable\ComfyUI\custom_nodes\AIT\AITemplate\ait\module\model.py", line 566, in run_with_tensors
outputs_ait = self.run(
^^^^^^^^^
File "G:\ComfyUI_windows_portable\ComfyUI\custom_nodes\AIT\AITemplate\ait\module\model.py", line 469, in run
return self._run_impl(
^^^^^^^^^^^^^^^
File "G:\ComfyUI_windows_portable\ComfyUI\custom_nodes\AIT\AITemplate\ait\module\model.py", line 408, in _run_impl
self.DLL.AITemplateModelContainerRun(
File "G:\ComfyUI_windows_portable\ComfyUI\custom_nodes\AIT\AITemplate\ait\module\model.py", line 212, in _wrapped_func
raise RuntimeError(f"Error in function: {method.name}")
RuntimeError: Error in function: AITemplateModelContainerRun
Can you present a clear explanation of the benefit of using this? IE performance numbers or some such.
Got this error when using the core Conditioning Combine node:
Error: [SetValue] Dimension got value out of bounds; expected value to be in [1, 2], but got 3.
The extension seems to do something that breaks controlnet usage even when no AITemplate loaders are in use.
The attached image contains a workflow (modified straight from a ComfyUI controlnet example, so it should be pretty minimal) that works when AIT is not in the custom_nodes folder, and breaks when it is.
The error it fails with is:
Error occurred when executing KSampler:
Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument mat1 in method wrapper_CUDA_addmm)
File "/home/sd/git/ComfyUI/execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "/home/sd/git/ComfyUI/execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "/home/sd/git/ComfyUI/execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "/home/sd/git/ComfyUI/nodes.py", line 1207, in sample
return common_ksampler(model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise=denoise)
File "/home/sd/git/ComfyUI/custom_nodes/AIT/AITemplate/AITemplate.py", line 175, in common_ksampler
samples = comfy.sample.sample(model, noise, steps, cfg, sampler_name, scheduler, positive, negative, latent_image,
File "/home/sd/git/ComfyUI/custom_nodes/AIT/AITemplate/AITemplate.py", line 308, in sample
samples = sampler.sample(noise, positive_copy, negative_copy, cfg=cfg, latent_image=latent_image, start_step=start_step, last_step=last_step, force_full_denoise=force_full_denoise, denoise_mask=noise_mask, sigmas=sigmas, callback=callback, disable_pbar=disable_pbar, seed=seed)
File "/home/sd/git/ComfyUI/comfy/samplers.py", line 727, in sample
samples = getattr(k_diffusion_sampling, "sample_{}".format(self.sampler))(self.model_k, noise, sigmas, extra_args=extra_args, callback=k_callback, disable=disable_pbar)
File "/home/sd/git/stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/home/sd/git/ComfyUI/comfy/k_diffusion/sampling.py", line 539, in sample_dpmpp_sde
denoised = model(x, sigmas[i] * s_in, **extra_args)
File "/home/sd/git/stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/home/sd/git/ComfyUI/comfy/samplers.py", line 317, in forward
out = self.inner_model(x, sigma, cond=cond, uncond=uncond, cond_scale=cond_scale, cond_concat=cond_concat, model_options=model_options, seed=seed)
File "/home/sd/git/stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/home/sd/git/ComfyUI/comfy/k_diffusion/external.py", line 125, in forward
eps = self.get_eps(input * c_in, self.sigma_to_t(sigma), **kwargs)
File "/home/sd/git/ComfyUI/comfy/k_diffusion/external.py", line 151, in get_eps
return self.inner_model.apply_model(*args, **kwargs)
File "/home/sd/git/ComfyUI/comfy/samplers.py", line 305, in apply_model
out = sampling_function(self.inner_model.apply_model, x, timestep, uncond, cond, cond_scale, cond_concat, model_options=model_options, seed=seed)
File "/home/sd/git/ComfyUI/comfy/samplers.py", line 283, in sampling_function
cond, uncond = calc_cond_uncond_batch(model_function, cond, uncond, x, timestep, max_total_area, cond_concat, model_options)
File "/home/sd/git/ComfyUI/comfy/samplers.py", line 235, in calc_cond_uncond_batch
c['control'] = control.get_control(input_x, timestep_, c, len(cond_or_uncond))
File "/home/sd/git/ComfyUI/comfy/controlnet.py", line 161, in get_control
control = self.control_model(x=x_noisy.to(self.control_model.dtype), hint=self.cond_hint, timesteps=t, context=context.to(self.control_model.dtype), y=y)
File "/home/sd/git/stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/home/sd/git/ComfyUI/comfy/cldm/cldm.py", line 283, in forward
emb = self.time_embed(t_emb)
File "/home/sd/git/stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/home/sd/git/stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/nn/modules/container.py", line 217, in forward
input = module(input)
File "/home/sd/git/stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/home/sd/git/ComfyUI/comfy/ops.py", line 18, in forward
return torch.nn.functional.linear(input, self.weight, self.bias)
Queue size: 0
Extra options
AITemplate must move something to the wrong device at some point, but I haven't been able to figure out where it happens.
"Apply ControlNet (Advanced)" node is broken when this extension is loaded - without using any custom nodes. "Apply ControlNet" node seems to work fine
Error below:
!!! Exception during processing !!!
Traceback (most recent call last):
File "C:\Users\josef\projects\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "C:\Users\josef\projects\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "C:\Users\josef\projects\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "C:\Users\josef\projects\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\nodes.py", line 657, in apply_controlnet
c_net = control_net.copy().set_cond_hint(control_hint, strength, (1.0 - start_percent, 1.0 - end_percent))
TypeError: ControlNet.set_cond_hint() takes from 2 to 3 positional arguments but 4 were given
Minimal workflow example for reproducing
advanced_minimal_example.zip
Works fine without AIT in custom_nodes
error :
!!! Exception during processing !!!
Traceback (most recent call last):
File "/home/gnuwu/ComfyUI/execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "/home/gnuwu/ComfyUI/execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "/home/gnuwu/ComfyUI/execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "/home/gnuwu/ComfyUI/nodes.py", line 1211, in sample
return common_ksampler(model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise=denoise)
File "/home/gnuwu/ComfyUI/custom_nodes/AIT/AITemplate/AITemplate.py", line 176, in common_ksampler
samples = comfy.sample.sample(model, noise, steps, cfg, sampler_name, scheduler, positive, negative, latent_image,
File "/home/gnuwu/ComfyUI/custom_nodes/AIT/AITemplate/AITemplate.py", line 294, in sample
sampler = comfy.samplers.KSampler(real_model, steps=steps, device=device, sampler=sampler_name, scheduler=scheduler, denoise=denoise, model_options=model.model_options)
File "/home/gnuwu/ComfyUI/comfy/samplers.py", line 548, in __init__
self.model_wrap = CompVisVDenoiser(self.model_denoise, quantize=True)
File "/home/gnuwu/ComfyUI/comfy/samplers.py", line 299, in __init__
super().__init__(model, model.alphas_cumprod, quantize=quantize)
File "/home/gnuwu/ComfyUI/comfy/k_diffusion/external.py", line 158, in __init__
super().__init__(((1 - alphas_cumprod) / alphas_cumprod) ** 0.5, quantize)
File "/home/gnuwu/ComfyUI/comfy/k_diffusion/external.py", line 48, in __init__
self.register_buffer('log_sigmas', sigmas.log())
RuntimeError: "log_vml_cpu" not implemented for 'Half'
I've found a case where AITemplate fails a generation that succeeds without it.
It's got something to do with how ComfyUI batches conds; in some cases, it'll do chunks with size > 2, which breaks AITemplate.
I can work around the problem by removing the batching logic in comfy/samplers.py
after the line to_batch = to_batch_temp[:1]
and limiting it to two at most. See this patch
I've attached a workflow to reproduce the issue (on my system, at least).
To run the workflow you'll need my prompt control nodes from https://github.com/asagi4/comfyui-prompt-control since I haven't hit this bug without timestep ranges yet.
It doesn't seem to happen with all seeds, either. I used RevAnimated for the reproduction, but any model should do.
import argparse
import json
import os
import hashlib
import lzma
import tempfile
from tqdm import tqdm
"""
#{
"cuda":"sm80",
"batch_size":1,
"resolution":64,
"sd":"v1",
"model":"vae",
"os":"windows",
"url":"",
"compressed_sha256":"",
"compressed_file_size":0,
"sha256":"",
"file_size":0
}
Directory structure
{os}/
{cuda}/
bs{batch_size}/
{resolution}/
`{sd}_{module_type}_{min}_{resolution}.so.xz`
"""
def sha256sum(filename):
h = hashlib.sha256()
b = bytearray(128*1024)
mv = memoryview(b)
with open(filename, 'rb', buffering=0) as f:
for n in iter(lambda : f.readinto(mv), 0):
h.update(mv[:n])
return h.hexdigest()
# extract .xz to temp file, sha256sum, and return, also return file size
def sha256sum_xz(filename):
with tempfile.TemporaryDirectory() as tmpdirname:
with lzma.open(filename) as f:
with open(os.path.join(tmpdirname, "module.so"), "wb") as f2:
f2.write(f.read())
return sha256sum(os.path.join(tmpdirname, "module.so")), os.path.getsize(os.path.join(tmpdirname, "module.so"))
def modules_json_to_sha256sums(modules_json):
with open(modules_json, "r") as f:
modules = json.load(f)
sha256sums = []
pbar = tqdm(total=0)
for module in modules:
sha256sums.append(module["compressed_sha256"])
pbar.update(1)
pbar.close()
return sha256sums
def folder_to_json(modules_folder, modules_json_out, url_prefix, modules_json_in=None):
existing_modules = []
if modules_json_in is not None:
existing_modules = modules_json_to_sha256sums(modules_json_in)
pbar = tqdm(total=0)
modules = []
for os_name in os.listdir(modules_folder):
os_path = os.path.join(modules_folder, os_name)
if not os.path.isdir(os_path):
continue
for cuda in os.listdir(os_path):
cuda_path = os.path.join(os_path, cuda)
if not os.path.isdir(cuda_path):
continue
for batch_size in os.listdir(cuda_path):
batch_size_path = os.path.join(cuda_path, batch_size)
if not os.path.isdir(batch_size_path):
continue
for resolution in os.listdir(batch_size_path):
resolution_path = os.path.join(batch_size_path, resolution)
if not os.path.isdir(resolution_path):
continue
for module in os.listdir(resolution_path):
if "clip" in module:
continue
module_path = os.path.join(resolution_path, module)
if not os.path.isfile(module_path):
continue
compressed_sha256 = sha256sum(module_path)
if compressed_sha256 in existing_modules:
print(f"Module: {module_path} already exists in {modules_json_in}, skipping")
pbar.update(1)
continue
module_info = module.split("_")
print(module_info)
sd = module_info[0]
module_type = module_info[1]
if module_type == "vae" and sd != "v1":
print(f"Module: {module_path} can be removed, v1 VAE modules support all sd values")
if module_type == "vae" and module_info[2] == "encode":
module_type = "vae_encode"
min_resolution = module_info[3]
resolution = module_info[4].split(".")[0]
else:
min_resolution = module_info[2]
resolution = module_info[3].split(".")[0]
decompressed_sha256, decompressed_file_size = sha256sum_xz(module_path)
module_json = {
"cuda": cuda,
"batch_size": int(batch_size[2:]),
"resolution": int(resolution),
"sd": sd,
"model": module_type,
"os": os_name,
"url": f"{url_prefix}{os_name}/{cuda}/{batch_size}/{resolution}/{module}",
"compressed_sha256": compressed_sha256,
"compressed_file_size": os.path.getsize(module_path),
"sha256": decompressed_sha256,
"file_size": decompressed_file_size
}
modules.append(module_json)
pbar.update(1)
pbar.close()
with open(modules_json_out, "w") as f:
json.dump(modules, f)
if __name__ == "__main__":
parser = argparse.ArgumentParser()
parser.add_argument(
"--modules-folder",
)
parser.add_argument(
"--modules-json-out"
)
parser.add_argument(
"--modules-json-in",
default=None
)
parser.add_argument(
"--url-prefix",
help="https://huggingface.co/{hf-path}/resolve/main/"
)
args = parser.parse_args()
folder_to_json(args.modules_folder, args.modules_json_out, args.url_prefix, args.modules_json_in)
This script can assist with managing modules.json until a more permanent solution is developed.
Usage:
--modules-json-out
specified it will process every file in the folder, hashing both the decompressed and compressed versions, and writing out the json with details of all modules.--modules-json-in
is specified, only the new modules are written to --modules-json-out
, check is based on compressed_sha256
value--url-prefix
is the base of the huggingface download url, e.g. https://huggingface.co/Fizzledorf/AITemplate_V1_V2/resolve/main/modules/
Therefore the procedure for adding new modules would be to compress them, I use 7-zip on Windows and xz
on Linux,
move to the appropriate folder, then run the script and provide the existing modules.json, and finally update existing modules.json with the new data
cc @FizzleDorf
On a resolution of 840x1776 840x1176 (windows xl sm80 1 1176 unet) images are not getting diffused correctly. It goes away if you lower the resolution to ex. 848x1184 (windows xl sm80 1 1184 unet works) and AIT switches to a different module.
EDIT: resolution typo corrected to 840x1176
It's not seed, prompt or sampler dependent.
@Shaistrong kindly helped confirming the issue.
Tested on Windows 11, 4090 (536.67), latest version of ComfyUI and the AIT custom node
If you get Error: Got cutlass error: Error Internal
the module needs to be recompiled, report it here.
The following modules return this error on my machine:
If these modules are working on your machine, report it here.
In my local version and the SDXL support PR to facebookincubator/AITemplate I renamed inputs from inputN
.
All modules will be recompiled soon, this will include the new naming scheme, and enable 8px increment. Suggestions on resolutions, batch sizes etc that should be supported are welcome, and could be shared here or in a separate issue.
Name changes are described below and need to be applied in AITemplate/ait/inference.py
.
To use the recent SDXL modules, I would recommend applying only the UNet changes and using the existing VAE modules, as the changes are less involved. Note that the changes will then be incompatible with existing UNet modules for v1 and v2.
Depending on how long compiling a new set takes, and how long adoption of new modules would take, it may be advantageous to implement some sort of compatibility check, the module has capability of returning the input and output name to index maps, I'll leave this open to comment, SM70/SM75 may take longer to replace although I'm not sure of their popularity.
input0
-> latent_model_input
input1
-> timesteps
input2
-> encoder_hidden_states
input3
-> class_labels
(unused by this plugin)vae_input
-> pixels
when encode, latent
when decodevae_sample
-> random_sample
, only used by encodeNote: module selection for VAE uses v1
, meaning the recent VAE modules labelled with sd
: xl
will not be used, the main difference between the recent and existing is logging, they will still work.
Unused by this plugin, but it's input0
-> input_ids
and input1
to position_ids
Attached are fresh modules with new naming and 8px increment, batch 1, Windows, sm80
Note plugin does not support selection of refiner modules at this time.
VAE Decode and VAE Encode aren't versioned because it is compatible between v1, v2 and xl
The filename format for the is also different, {type}{sd}{max_resolution}, and the extension is .dll, minimum resolution is always 8px
There should be some group input on the filename format, and which resolutions and batch sizes will be supported, whether to get rid of the folders and put cuda version/os etc in the filename
clip.zip
unet_control_v1.zip
unet_v1.zip
unet_v2.zip
unet_xl.zip
unet_xlr.zip
vae_decode.zip
vae_encode.zip
These modules are also being hosted for my upcoming UI that uses AIT, over at modules.diffuse.rs, that file windows_sm80.json
contains info for Windows SM80 modules, there will be others following the same format when available (linux_sm80.json
, windows_sm75.json
etc), the modules themselves are hosted on the same domain i.e. sm80_unet_xlr_2048.dll.xz
, linux modules will have .so
extension.
This plugin can be changed to use this source, or modules can be rehosted from that source if desired.
After updating build commands that previously worked are failing now.
python unet.py --hf-hub-or-path "runwayml/stable-diffusion-v1-5" --width 512 512 --height 512 512 --batch-size 2 2 --clip-chunks 1 --model-name "v1_unet_512_512_2_2_c1_test" --work-dir "/home/primary/Documents/code/ML/AIT/modules"
2023-09-09 14:08:36,292 INFO <aitemplate.compiler.compiler> compiled the final .so file elapsed time: 0:03:22.475189
[14:08:36] model_container.cu:69: Device Runtime Version: 11080; Driver Version: 12010
[14:08:36] model_container.cu:83: Hardware accelerator device properties:
Device:
ASCII string identifying device: NVIDIA GeForce RTX 4090
Major compute capability: 8
Minor compute capability: 9
UUID: GPU-3caeb811-53d4-7669-01dc-ca502ea01048
Unique identifier for a group of devices on the same multi-GPU board: 0
PCI bus ID of the device: 12
PCI device ID of the device: 0
PCI domain ID of the device: 0
Memory limits:
Constant memory available on device in bytes: 65536
Global memory available on device in bytes: 25385107456
Size of L2 cache in bytes: 75497472
Shared memory available per block in bytes: 49152
Shared memory available per multiprocessor in bytes: 102400
[14:08:36] model_container.cu:87: Init AITemplate Runtime with 1 concurrency
Traceback (most recent call last):
File "/home/primary/Documents/code/ML/AIT/AITemplate/unet.py", line 141, in <module>
compile_diffusers()
File "/home/primary/mambaforge/envs/AITemplate/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/primary/mambaforge/envs/AITemplate/lib/python3.11/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "/home/primary/mambaforge/envs/AITemplate/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/primary/mambaforge/envs/AITemplate/lib/python3.11/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/primary/Documents/code/ML/AIT/AITemplate/unet.py", line 104, in compile_diffusers
compile_unet(
File "/home/primary/Documents/code/ML/AIT/AITemplate/ait/compile/unet.py", line 305, in compile_unet
vram = round(total_usage / 1024 / 1024)
~~~~~~~~~~~~^~~~~~
TypeError: unsupported operand type(s) for /: 'Model' and 'int'
Line 348 in d544b7b
ModelPatcher
is moved to separated file(comfy/model_patcher.py
) from comfy/sd.py
hey, would it be possible to add another AIT node for upscaling using upscale model? so far that's the only part of the AIT workflow I made that's holding it back from going under 10 seconds per image.
File ComfyUI/custom_nodes/AIT/AITemplate/AITemplate.py", line 414
else:
^^^^
IndentationError: expected an indented block after 'if' statement on line 413
If you hook up a LoraLoader node before the AITemplateLoader node, every change to the LoRA will further corrupt gens. Putting the LoraLoader node after the AITemplateLoader node seems to be okay.
EDIT: Applying LoRAs in general and alternating their values corrupts gens.
Reproduction:
There seems to be no issue if I let the images generate without canceling. I think the issue has something to do with cancelling gens.
Hi! Thanks for the great node. I could use it in the example and it works very well: I had to change:
self.control_model_wrapped = comfy.sd.ModelPatcher(self.control_model, load_device=comfy.model_management.get_torch_device(), offload_device=comfy.model_management.unet_offload_device())
for
self.control_model_wrapped = comfy.model_patcher.ModelPatcher(self.control_model, load_device=comfy.model_management.get_torch_device(), offload_device=comfy.model_management.unet_offload_device())
However, I am trying to use it in a workflow that I have that has Inpainting and ControlNet. I have issues with the inpainting model and with controlnet.
Error occurred when executing KSampler:
can't multiply sequence by non-int of type 'float'
File "/home/ubuntu/ComfyUI/execution.py", line 152, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "/home/ubuntu/ComfyUI/execution.py", line 82, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "/home/ubuntu/ComfyUI/execution.py", line 75, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "/home/ubuntu/ComfyUI/nodes.py", line 1265, in sample
return common_ksampler(model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise=denoise)
File "/home/ubuntu/ComfyUI/custom_nodes/AIT/AITemplate/AITemplate.py", line 176, in common_ksampler
samples = comfy.sample.sample(model, noise, steps, cfg, sampler_name, scheduler, positive, negative, latent_image,
File "/home/ubuntu/ComfyUI/custom_nodes/AIT/AITemplate/AITemplate.py", line 310, in sample
samples = sampler.sample(noise, positive_copy, negative_copy, cfg=cfg, latent_image=latent_image, start_step=start_step, last_step=last_step, force_full_denoise=force_full_denoise, denoise_mask=noise_mask, sigmas=sigmas, callback=callback, disable_pbar=disable_pbar, seed=seed)
File "/home/ubuntu/ComfyUI/comfy/samplers.py", line 742, in sample
samples = getattr(k_diffusion_sampling, "sample_{}".format(self.sampler))(self.model_k, noise, sigmas, extra_args=extra_args, callback=k_callback, disable=disable_pbar)
File "/home/ubuntu/ComfyUI/env/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/home/ubuntu/ComfyUI/comfy/k_diffusion/sampling.py", line 707, in sample_dpmpp_sde_gpu
return sample_dpmpp_sde(model, x, sigmas, extra_args=extra_args, callback=callback, disable=disable, eta=eta, s_noise=s_noise, noise_sampler=noise_sampler, r=r)
File "/home/ubuntu/ComfyUI/env/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/home/ubuntu/ComfyUI/comfy/k_diffusion/sampling.py", line 539, in sample_dpmpp_sde
denoised = model(x, sigmas[i] * s_in, **extra_args)
File "/home/ubuntu/ComfyUI/env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/home/ubuntu/ComfyUI/comfy/samplers.py", line 323, in forward
out = self.inner_model(x, sigma, cond=cond, uncond=uncond, cond_scale=cond_scale, cond_concat=cond_concat, model_options=model_options, seed=seed)
File "/home/ubuntu/ComfyUI/env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/home/ubuntu/ComfyUI/comfy/k_diffusion/external.py", line 125, in forward
eps = self.get_eps(input * c_in, self.sigma_to_t(sigma), **kwargs)
File "/home/ubuntu/ComfyUI/comfy/k_diffusion/external.py", line 151, in get_eps
return self.inner_model.apply_model(*args, **kwargs)
File "/home/ubuntu/ComfyUI/comfy/samplers.py", line 311, in apply_model
out = sampling_function(self.inner_model.apply_model, x, timestep, uncond, cond, cond_scale, cond_concat, model_options=model_options, seed=seed)
File "/home/ubuntu/ComfyUI/comfy/samplers.py", line 289, in sampling_function
cond, uncond = calc_cond_uncond_batch(model_function, cond, uncond, x, timestep, max_total_area, cond_concat, model_options)
File "/home/ubuntu/ComfyUI/comfy/samplers.py", line 242, in calc_cond_uncond_batch
c['control'] = control.get_control(input_x, timestep_, c, len(cond_or_uncond))
File "/home/ubuntu/ComfyUI/custom_nodes/AIT/AITemplate/AITemplate.py", line 435, in get_control
x *= self.strength
Error occurred when executing KSampler:
Error in function: AITemplateModelContainerSetManyConstants
File "/home/ubuntu/ComfyUI/execution.py", line 152, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "/home/ubuntu/ComfyUI/execution.py", line 82, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "/home/ubuntu/ComfyUI/execution.py", line 75, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "/home/ubuntu/ComfyUI/nodes.py", line 1265, in sample
return common_ksampler(model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise=denoise)
File "/home/ubuntu/ComfyUI/custom_nodes/AIT/AITemplate/AITemplate.py", line 176, in common_ksampler
samples = comfy.sample.sample(model, noise, steps, cfg, sampler_name, scheduler, positive, negative, latent_image,
File "/home/ubuntu/ComfyUI/custom_nodes/AIT/AITemplate/AITemplate.py", line 274, in sample
AITemplate.unet[module['sha256']] = AITemplate.loader.apply_unet(
File "/home/ubuntu/ComfyUI/custom_nodes/AIT/AITemplate/ait/load.py", line 222, in apply_unet
return self.apply(aitemplate_module, ait_params)
File "/home/ubuntu/ComfyUI/custom_nodes/AIT/AITemplate/ait/load.py", line 201, in apply
aitemplate_module.set_many_constants_with_tensors(ait_params)
File "/home/ubuntu/ComfyUI/custom_nodes/AIT/AITemplate/ait/module/model.py", line 840, in set_many_constants_with_tensors
self.set_many_constants(ait_tensors)
File "/home/ubuntu/ComfyUI/custom_nodes/AIT/AITemplate/ait/module/model.py", line 791, in set_many_constants
self.DLL.AITemplateModelContainerSetManyConstants(
File "/home/ubuntu/ComfyUI/custom_nodes/AIT/AITemplate/ait/module/model.py", line 212, in _wrapped_func
raise RuntimeError(f"Error in function: {method.__name__}")
It works fine if I run it without ControlNet and with a non-inpainting model.
Thanks!
since fooocus is using comfy as inference engine, it is possible to integrate ait to fooocus. are there any plans for supporting ait inference in fooocus?
Line 171 in 155bd9d
Need to move to device other than cpu.
x0 = x0.to(comfy.model_management.get_torch_device())
I'm not sure what is causing it but the ksampler is throwing a 'tuple' object has no attribute 'model'
error. Maybe caused by changes to the sampler here?:
comfyanonymous/ComfyUI@4eab00e
I'm not sure if this is caused by the smart memory management changes Comfy introduced or a strange interaction with previews (turning previews off doesn't fix it) but can't run any workflows.
Error occurred when executing KSamplerAdvanced:
module 'comfy.model_management' has no attribute 'load_if_low_vram'
File "C:\Users\hoffl\Stable Diffusion\ComfyUI_windows_portable\ComfyUI\execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "C:\Users\hoffl\Stable Diffusion\ComfyUI_windows_portable\ComfyUI\execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "C:\Users\hoffl\Stable Diffusion\ComfyUI_windows_portable\ComfyUI\execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "C:\Users\hoffl\Stable Diffusion\ComfyUI_windows_portable\ComfyUI\nodes.py", line 1240, in sample
return common_ksampler(model, noise_seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise=denoise, disable_noise=disable_noise, start_step=start_at_step, last_step=end_at_step, force_full_denoise=force_full_denoise)
File "C:\Users\hoffl\Stable Diffusion\ComfyUI_windows_portable\ComfyUI\custom_nodes\AIT\AITemplate\AITemplate.py", line 175, in common_ksampler
samples = comfy.sample.sample(model, noise, steps, cfg, sampler_name, scheduler, positive, negative, latent_image,
File "C:\Users\hoffl\Stable Diffusion\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Impact-Pack\modules\impact\hacky.py", line 9, in informative_sample
return original_sample(*args, **kwargs)
File "C:\Users\hoffl\Stable Diffusion\ComfyUI_windows_portable\ComfyUI\custom_nodes\AIT\AITemplate\AITemplate.py", line 308, in sample
samples = sampler.sample(noise, positive_copy, negative_copy, cfg=cfg, latent_image=latent_image, start_step=start_step, last_step=last_step, force_full_denoise=force_full_denoise, denoise_mask=noise_mask, sigmas=sigmas, callback=callback, disable_pbar=disable_pbar, seed=seed)
File "C:\Users\hoffl\Stable Diffusion\ComfyUI_windows_portable\ComfyUI\comfy\samplers.py", line 733, in sample
samples = getattr(k_diffusion_sampling, "sample_{}".format(self.sampler))(self.model_k, noise, sigmas, extra_args=extra_args, callback=k_callback, disable=disable_pbar)
File "C:\Users\hoffl\Stable Diffusion\ComfyUI_windows_portable\python_embeded\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "C:\Users\hoffl\Stable Diffusion\ComfyUI_windows_portable\ComfyUI\comfy\k_diffusion\sampling.py", line 701, in sample_dpmpp_2m_sde_gpu
return sample_dpmpp_2m_sde(model, x, sigmas, extra_args=extra_args, callback=callback, disable=disable, eta=eta, s_noise=s_noise, noise_sampler=noise_sampler, solver_type=solver_type)
File "C:\Users\hoffl\Stable Diffusion\ComfyUI_windows_portable\python_embeded\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "C:\Users\hoffl\Stable Diffusion\ComfyUI_windows_portable\ComfyUI\comfy\k_diffusion\sampling.py", line 613, in sample_dpmpp_2m_sde
denoised = model(x, sigmas[i] * s_in, **extra_args)
File "C:\Users\hoffl\Stable Diffusion\ComfyUI_windows_portable\python_embeded\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "C:\Users\hoffl\Stable Diffusion\ComfyUI_windows_portable\ComfyUI\comfy\samplers.py", line 323, in forward
out = self.inner_model(x, sigma, cond=cond, uncond=uncond, cond_scale=cond_scale, cond_concat=cond_concat, model_options=model_options, seed=seed)
File "C:\Users\hoffl\Stable Diffusion\ComfyUI_windows_portable\python_embeded\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "C:\Users\hoffl\Stable Diffusion\ComfyUI_windows_portable\ComfyUI\comfy\k_diffusion\external.py", line 125, in forward
eps = self.get_eps(input * c_in, self.sigma_to_t(sigma), **kwargs)
File "C:\Users\hoffl\Stable Diffusion\ComfyUI_windows_portable\ComfyUI\comfy\k_diffusion\external.py", line 151, in get_eps
return self.inner_model.apply_model(*args, **kwargs)
File "C:\Users\hoffl\Stable Diffusion\ComfyUI_windows_portable\ComfyUI\comfy\samplers.py", line 311, in apply_model
out = sampling_function(self.inner_model.apply_model, x, timestep, uncond, cond, cond_scale, cond_concat, model_options=model_options, seed=seed)
File "C:\Users\hoffl\Stable Diffusion\ComfyUI_windows_portable\ComfyUI\comfy\samplers.py", line 289, in sampling_function
cond, uncond = calc_cond_uncond_batch(model_function, cond, uncond, x, timestep, max_total_area, cond_concat, model_options)
File "C:\Users\hoffl\Stable Diffusion\ComfyUI_windows_portable\ComfyUI\comfy\samplers.py", line 241, in calc_cond_uncond_batch
c['control'] = control.get_control(input_x, timestep_, c, len(cond_or_uncond))
File "C:\Users\hoffl\Stable Diffusion\ComfyUI_windows_portable\ComfyUI\custom_nodes\AIT\AITemplate\AITemplate.py", line 409, in get_control
self.control_model = comfy.model_management.load_if_low_vram(self.control_model)
Traceback (most recent call last):
File "/home/me/git/ComfyUI/nodes.py", line 1694, in load_custom_node
module_spec.loader.exec_module(module)
File "<frozen importlib._bootstrap_external>", line 940, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/home/me/git/ComfyUI/custom_nodes/AIT/__init__.py", line 1, in <module>
from .AITemplate.AITemplate import AITemplateLoader, AITemplateControlNetLoader, AITemplateVAEDecode, AITemplateVAEEncode, VAEEncodeForInpaint, AITemplateEmptyLatentImage, AITemplateLatentUpscale
File "/home/me/git/ComfyUI/custom_nodes/AIT/AITemplate/AITemplate.py", line 335, in <module>
from comfy.sd import ControlBase
ImportError: cannot import name 'ControlBase' from 'comfy.sd' (/home/me/git/ComfyUI/comfy/sd.py)
ControlBase
has been moved from comfy.sd
to comfy.controlnet
due to recent changes.
Was reported here comfyanonymous/ComfyUI#1426
!!! Exception during processing !!!
Traceback (most recent call last):
File "K:\AI ART UNIVERSE\COMFY-UI\ComfyUI_windows_portable\ComfyUI\execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "K:\AI ART UNIVERSE\COMFY-UI\ComfyUI_windows_portable\ComfyUI\execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "K:\AI ART UNIVERSE\COMFY-UI\ComfyUI_windows_portable\ComfyUI\execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "K:\AI ART UNIVERSE\COMFY-UI\ComfyUI_windows_portable\ComfyUI\nodes.py", line 577, in load_controlnet
controlnet = comfy.controlnet.load_controlnet(controlnet_path)
File "K:\AI ART UNIVERSE\COMFY-UI\ComfyUI_windows_portable\ComfyUI\comfy\controlnet.py", line 394, in load_controlnet
control = ControlNet(control_model, global_average_pooling=global_average_pooling)
File "K:\AI ART UNIVERSE\COMFY-UI\ComfyUI_windows_portable\ComfyUI\custom_nodes\AIT\AITemplate\AITemplate.py", line 347, in init
self.control_model_wrapped = comfy.sd.ModelPatcher(self.control_model, load_device=comfy.model_management.get_torch_device(), offload_device=comfy.model_management.unet_offload_device())
AttributeError: module 'comfy.sd' has no attribute 'ModelPatcher'
enable
or disable
, this error occurs when using ddim
, uni_pc
, or uni_pc_bh2
:
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!
Of date none of the workflows runs through?
Prompt executed in 8.11 seconds
got prompt
INFO:comfyui-prompt-control:Resolving wildcards...
Prompt executor has been patched by Job Iterator!
3
Found 5 modules for windows v1 sm80 1 512 unet
Using 8763aa0cc5f5008bd0734664ae47e874c2641d9e8293e8bd84c8c8ca7f6084d0
!!! Exception during processing !!!
Traceback (most recent call last):
File "A:\AI_Files\ComfyUI\execution.py", line 152, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "A:\AI_Files\ComfyUI\execution.py", line 82, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "A:\AI_Files\ComfyUI\execution.py", line 75, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "A:\AI_Files\ComfyUI\nodes.py", line 1236, in sample
return common_ksampler(model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise=denoise)
File "A:\AI_Files\ComfyUI\custom_nodes\AIT\AITemplate\AITemplate.py", line 176, in common_ksampler
samples = comfy.sample.sample(model, noise, steps, cfg, sampler_name, scheduler, positive, negative, latent_image,
File "A:\AI_Files\ComfyUI\custom_nodes\ComfyUI-Impact-Pack\modules\impact\hacky.py", line 9, in informative_sample
return original_sample(*args, **kwargs)
File "A:\AI_Files\ComfyUI\custom_nodes\AIT\AITemplate\AITemplate.py", line 300, in sample
sampler.model_denoise = comfy.samplers.CFGNoisePredictor(model_wrapper)
File "A:\AI_Files\ComfyUI\custom_nodes\ComfyUI_smZNodes\smZNodes.py", line 814, in init
self.inner_model2.num_timesteps = model.num_timesteps
File "A:\AI_Files\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1614, in getattr
raise AttributeError("'{}' object has no attribute '{}'".format(
AttributeError: 'AITemplateModelWrapper' object has no attribute 'num_timesteps'
Hello
When I use AIT and ComfyUi (both on the latest branch), it gives me this error.
I start ComfyUi with this command: python3 main.py --listen --port 21128 --gpu-only
and I'm on Ubuntu 22.04.
Please help! :)
Error occurred when executing KSampler:
Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! (when checking argument for argument mat1 in method wrapper_CUDA_addmm)
File "/home/cybertimon/Repositories/ComfyUI/execution.py", line 152, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "/home/cybertimon/Repositories/ComfyUI/execution.py", line 82, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "/home/cybertimon/Repositories/ComfyUI/execution.py", line 75, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "/home/cybertimon/Repositories/ComfyUI/nodes.py", line 1236, in sample
return common_ksampler(model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise=denoise)
File "/home/cybertimon/Repositories/ComfyUI/custom_nodes/AIT/AITemplate/AITemplate.py", line 176, in common_ksampler
samples = comfy.sample.sample(model, noise, steps, cfg, sampler_name, scheduler, positive, negative, latent_image,
File "/home/cybertimon/Repositories/ComfyUI/custom_nodes/AIT/AITemplate/AITemplate.py", line 310, in sample
samples = sampler.sample(noise, positive_copy, negative_copy, cfg=cfg, latent_image=latent_image, start_step=start_step, last_step=last_step, force_full_denoise=force_full_denoise, denoise_mask=noise_mask, sigmas=sigmas, callback=callback, disable_pbar=disable_pbar, seed=seed)
File "/home/cybertimon/Repositories/ComfyUI/comfy/samplers.py", line 785, in sample
return sample(self.model, noise, positive, negative, cfg, self.device, sampler(), sigmas, self.model_options, latent_image=latent_image, denoise_mask=denoise_mask, callback=callback, disable_pbar=disable_pbar, seed=seed)
File "/home/cybertimon/Repositories/ComfyUI/comfy/samplers.py", line 690, in sample
samples = sampler.sample(model_wrap, sigmas, extra_args, callback, noise, latent_image, denoise_mask, disable_pbar)
File "/home/cybertimon/Repositories/ComfyUI/comfy/samplers.py", line 630, in sample
samples = getattr(k_diffusion_sampling, "sample_{}".format(sampler_name))(model_k, noise, sigmas, extra_args=extra_args, callback=k_callback, disable=disable_pbar, **extra_options)
File "/home/cybertimon/.local/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/home/cybertimon/Repositories/ComfyUI/comfy/k_diffusion/sampling.py", line 137, in sample_euler
denoised = model(x, sigma_hat * s_in, **extra_args)
File "/home/cybertimon/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/home/cybertimon/Repositories/ComfyUI/comfy/samplers.py", line 323, in forward
out = self.inner_model(x, sigma, cond=cond, uncond=uncond, cond_scale=cond_scale, cond_concat=cond_concat, model_options=model_options, seed=seed)
File "/home/cybertimon/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/home/cybertimon/Repositories/ComfyUI/comfy/k_diffusion/external.py", line 125, in forward
eps = self.get_eps(input * c_in, self.sigma_to_t(sigma), **kwargs)
File "/home/cybertimon/Repositories/ComfyUI/comfy/k_diffusion/external.py", line 151, in get_eps
return self.inner_model.apply_model(*args, **kwargs)
File "/home/cybertimon/Repositories/ComfyUI/comfy/samplers.py", line 311, in apply_model
out = sampling_function(self.inner_model.apply_model, x, timestep, uncond, cond, cond_scale, cond_concat, model_options=model_options, seed=seed)
File "/home/cybertimon/Repositories/ComfyUI/comfy/samplers.py", line 289, in sampling_function
cond, uncond = calc_cond_uncond_batch(model_function, cond, uncond, x, timestep, max_total_area, cond_concat, model_options)
File "/home/cybertimon/Repositories/ComfyUI/comfy/samplers.py", line 265, in calc_cond_uncond_batch
output = model_function(input_x, timestep_, **c).chunk(batch_chunks)
File "/home/cybertimon/Repositories/ComfyUI/comfy/model_base.py", line 63, in apply_model
return self.diffusion_model(xc, t, context=context, y=c_adm, control=control, transformer_options=transformer_options).float()
File "/home/cybertimon/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/home/cybertimon/Repositories/ComfyUI/comfy/ldm/modules/diffusionmodules/openaimodel.py", line 622, in forward
emb = emb + self.label_emb(y)
File "/home/cybertimon/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/home/cybertimon/.local/lib/python3.10/site-packages/torch/nn/modules/container.py", line 217, in forward
input = module(input)
File "/home/cybertimon/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/home/cybertimon/.local/lib/python3.10/site-packages/torch/nn/modules/container.py", line 217, in forward
input = module(input)
File "/home/cybertimon/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/home/cybertimon/Repositories/ComfyUI/comfy/ops.py", line 18, in forward
return torch.nn.functional.linear(input, self.weight, self.bias)
It will be viable to A1111 SD? I dont really like to use Comfy but tested AIT and render time is very good.
Getting this error Got cutlass error: Error InternalError: Got cutlass error: Error Internal
:
got prompt
2
3
3
Found 4 modules for windows v1 sm80 1 512 unet_control
Using 1b7bca45b73c3a9dd17a11070217f44d3ae6102d3b0d33400b3ed34e9a516712
0%| | 0/30 [00:00<?, ?it/s]Found 4 modules for windows v1 sm80 1.0 512 controlnet
Using ef60d80894710e8e5a6fffd2d55a5da2c6a6c5adff75946b99a1e18d75f02449
Got cutlass error: Error InternalError: Got cutlass error: Error Internal
0%| | 0/30 [00:00<?, ?it/s]
!!! Exception during processing !!!
Traceback (most recent call last):
File "C:\Users\Anon\Downloads\Programs\ComfyUI_windows_portable\ComfyUI\execution.py", line 152, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Anon\Downloads\Programs\ComfyUI_windows_portable\ComfyUI\execution.py", line 82, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Anon\Downloads\Programs\ComfyUI_windows_portable\ComfyUI\execution.py", line 75, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Anon\Downloads\Programs\ComfyUI_windows_portable\ComfyUI\nodes.py", line 1236, in sample
return common_ksampler(model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise=denoise)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Anon\Downloads\Programs\ComfyUI_windows_portable\ComfyUI\custom_nodes\AIT\AITemplate\AITemplate.py", line 176, in common_ksampler
samples = comfy.sample.sample(model, noise, steps, cfg, sampler_name, scheduler, positive, negative, latent_image,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Anon\Downloads\Programs\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Impact-Pack\modules\impact\hacky.py", line 22, in informative_sample
raise e
File "C:\Users\Anon\Downloads\Programs\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Impact-Pack\modules\impact\hacky.py", line 9, in informative_sample
return original_sample(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Anon\Downloads\Programs\ComfyUI_windows_portable\ComfyUI\custom_nodes\AIT\AITemplate\AITemplate.py", line 310, in sample
samples = sampler.sample(noise, positive_copy, negative_copy, cfg=cfg, latent_image=latent_image, start_step=start_step, last_step=last_step, force_full_denoise=force_full_denoise, denoise_mask=noise_mask, sigmas=sigmas, callback=callback, disable_pbar=disable_pbar, seed=seed)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Anon\Downloads\Programs\ComfyUI_windows_portable\ComfyUI\comfy\samplers.py", line 741, in sample
samples = getattr(k_diffusion_sampling, "sample_{}".format(self.sampler))(self.model_k, noise, sigmas, extra_args=extra_args, callback=k_callback, disable=disable_pbar)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Anon\Downloads\Programs\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Anon\Downloads\Programs\ComfyUI_windows_portable\ComfyUI\comfy\k_diffusion\sampling.py", line 137, in sample_euler
denoised = model(x, sigma_hat * s_in, **extra_args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Anon\Downloads\Programs\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1522, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Anon\Downloads\Programs\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1531, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Anon\Downloads\Programs\ComfyUI_windows_portable\ComfyUI\comfy\samplers.py", line 322, in forward
out = self.inner_model(x, sigma, cond=cond, uncond=uncond, cond_scale=cond_scale, cond_concat=cond_concat, model_options=model_options, seed=seed)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Anon\Downloads\Programs\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1522, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Anon\Downloads\Programs\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1531, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Anon\Downloads\Programs\ComfyUI_windows_portable\ComfyUI\comfy\k_diffusion\external.py", line 125, in forward
eps = self.get_eps(input * c_in, self.sigma_to_t(sigma), **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Anon\Downloads\Programs\ComfyUI_windows_portable\ComfyUI\comfy\k_diffusion\external.py", line 151, in get_eps
return self.inner_model.apply_model(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Anon\Downloads\Programs\ComfyUI_windows_portable\ComfyUI\comfy\samplers.py", line 310, in apply_model
out = sampling_function(self.inner_model.apply_model, x, timestep, uncond, cond, cond_scale, cond_concat, model_options=model_options, seed=seed)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Anon\Downloads\Programs\ComfyUI_windows_portable\ComfyUI\comfy\samplers.py", line 288, in sampling_function
cond, uncond = calc_cond_uncond_batch(model_function, cond, uncond, x, timestep, max_total_area, cond_concat, model_options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Anon\Downloads\Programs\ComfyUI_windows_portable\ComfyUI\comfy\samplers.py", line 241, in calc_cond_uncond_batch
c['control'] = control.get_control(input_x, timestep_, c, len(cond_or_uncond))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Anon\Downloads\Programs\ComfyUI_windows_portable\ComfyUI\custom_nodes\AIT\AITemplate\AITemplate.py", line 419, in get_control
control = self.aitemplate_controlnet(x_noisy, t, cond, self.cond_hint)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Anon\Downloads\Programs\ComfyUI_windows_portable\ComfyUI\custom_nodes\AIT\AITemplate\AITemplate.py", line 381, in aitemplate_controlnet
return controlnet_inference(
^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Anon\Downloads\Programs\ComfyUI_windows_portable\ComfyUI\custom_nodes\AIT\AITemplate\ait\inference.py", line 146, in controlnet_inference
exe_module.run_with_tensors(inputs, ys, graph_mode=False)
File "C:\Users\Anon\Downloads\Programs\ComfyUI_windows_portable\ComfyUI\custom_nodes\AIT\AITemplate\ait\module\model.py", line 566, in run_with_tensors
outputs_ait = self.run(
^^^^^^^^^
File "C:\Users\Anon\Downloads\Programs\ComfyUI_windows_portable\ComfyUI\custom_nodes\AIT\AITemplate\ait\module\model.py", line 469, in run
return self._run_impl(
^^^^^^^^^^^^^^^
File "C:\Users\Anon\Downloads\Programs\ComfyUI_windows_portable\ComfyUI\custom_nodes\AIT\AITemplate\ait\module\model.py", line 408, in _run_impl
self.DLL.AITemplateModelContainerRun(
File "C:\Users\Anon\Downloads\Programs\ComfyUI_windows_portable\ComfyUI\custom_nodes\AIT\AITemplate\ait\module\model.py", line 212, in _wrapped_func
raise RuntimeError(f"Error in function: {method.__name__}")
RuntimeError: Error in function: AITemplateModelContainerRun
With the AIT Loader and AIT ControlNet nodes muted, I get another error NameError: name 'c_crossattn' is not defined
:
!!! Exception during processing !!!
Traceback (most recent call last):
File "C:\Users\Anon\Downloads\Programs\ComfyUI_windows_portable\ComfyUI\execution.py", line 152, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Anon\Downloads\Programs\ComfyUI_windows_portable\ComfyUI\execution.py", line 82, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Anon\Downloads\Programs\ComfyUI_windows_portable\ComfyUI\execution.py", line 75, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Anon\Downloads\Programs\ComfyUI_windows_portable\ComfyUI\nodes.py", line 1236, in sample
return common_ksampler(model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise=denoise)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Anon\Downloads\Programs\ComfyUI_windows_portable\ComfyUI\custom_nodes\AIT\AITemplate\AITemplate.py", line 176, in common_ksampler
samples = comfy.sample.sample(model, noise, steps, cfg, sampler_name, scheduler, positive, negative, latent_image,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Anon\Downloads\Programs\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Impact-Pack\modules\impact\hacky.py", line 9, in informative_sample
return original_sample(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Anon\Downloads\Programs\ComfyUI_windows_portable\ComfyUI\custom_nodes\AIT\AITemplate\AITemplate.py", line 310, in sample
samples = sampler.sample(noise, positive_copy, negative_copy, cfg=cfg, latent_image=latent_image, start_step=start_step, last_step=last_step, force_full_denoise=force_full_denoise, denoise_mask=noise_mask, sigmas=sigmas, callback=callback, disable_pbar=disable_pbar, seed=seed)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Anon\Downloads\Programs\ComfyUI_windows_portable\ComfyUI\comfy\samplers.py", line 741, in sample
samples = getattr(k_diffusion_sampling, "sample_{}".format(self.sampler))(self.model_k, noise, sigmas, extra_args=extra_args, callback=k_callback, disable=disable_pbar)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Anon\Downloads\Programs\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Anon\Downloads\Programs\ComfyUI_windows_portable\ComfyUI\comfy\k_diffusion\sampling.py", line 137, in sample_euler
denoised = model(x, sigma_hat * s_in, **extra_args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Anon\Downloads\Programs\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1522, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Anon\Downloads\Programs\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1531, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Anon\Downloads\Programs\ComfyUI_windows_portable\ComfyUI\comfy\samplers.py", line 322, in forward
out = self.inner_model(x, sigma, cond=cond, uncond=uncond, cond_scale=cond_scale, cond_concat=cond_concat, model_options=model_options, seed=seed)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Anon\Downloads\Programs\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1522, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Anon\Downloads\Programs\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1531, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Anon\Downloads\Programs\ComfyUI_windows_portable\ComfyUI\comfy\k_diffusion\external.py", line 125, in forward
eps = self.get_eps(input * c_in, self.sigma_to_t(sigma), **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Anon\Downloads\Programs\ComfyUI_windows_portable\ComfyUI\comfy\k_diffusion\external.py", line 151, in get_eps
return self.inner_model.apply_model(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Anon\Downloads\Programs\ComfyUI_windows_portable\ComfyUI\comfy\samplers.py", line 310, in apply_model
out = sampling_function(self.inner_model.apply_model, x, timestep, uncond, cond, cond_scale, cond_concat, model_options=model_options, seed=seed)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Anon\Downloads\Programs\ComfyUI_windows_portable\ComfyUI\comfy\samplers.py", line 288, in sampling_function
cond, uncond = calc_cond_uncond_batch(model_function, cond, uncond, x, timestep, max_total_area, cond_concat, model_options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Anon\Downloads\Programs\ComfyUI_windows_portable\ComfyUI\comfy\samplers.py", line 241, in calc_cond_uncond_batch
c['control'] = control.get_control(input_x, timestep_, c, len(cond_or_uncond))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Anon\Downloads\Programs\ComfyUI_windows_portable\ComfyUI\custom_nodes\AIT\AITemplate\AITemplate.py", line 413, in get_control
context = c_crossattn
^^^^^^^^^^^
NameError: name 'c_crossattn' is not defined
`Found 3 modules for windows xl sm80 1 1024 unet
Using f64294f9bae73c4f742a4f61e8567460d1abefb461c792f69e8922c74331cd27
100%|██████████████████████████████████████████████████████████████████████████████████| 67/67 [00:08<00:00, 9.82it/s]
Found 3 modules for windows v1 sm80 1 1024 unet
Using 25fdbed983314f8b23c361180e861da73f8c24f4a194ecd6bac01fc8bb2d483a
[08:15:48]: NVIDIA GeForce RTX 4070 Ti
[08:15:48]: AITemplate modules compiled by hlky
[08:15:48]: Error: SetConstant did not receive correct number of bytes for unbound constant time_embedding_linear_1_weight: expected 819200 but got 1179648. Check that the provided tensor's shape is correct.
!!! Exception during processing !!!
Traceback (most recent call last):
File "C:\Users\joker\OneDrive\Desktop\A.I\ComfyUI\execution.py", line 144, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "C:\Users\joker\OneDrive\Desktop\A.I\ComfyUI\execution.py", line 74, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "C:\Users\joker\OneDrive\Desktop\A.I\ComfyUI\execution.py", line 67, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "C:\Users\joker\OneDrive\Desktop\A.I\ComfyUI\nodes.py", line 1165, in sample
return common_ksampler(model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise=denoise)
File "C:\Users\joker\OneDrive\Desktop\A.I\ComfyUI\custom_nodes\AIT\AITemplate\AITemplate.py", line 170, in common_ksampler
samples = comfy.sample.sample(model, noise, steps, cfg, sampler_name, scheduler, positive, negative, latent_image,
File "C:\Users\joker\OneDrive\Desktop\A.I\ComfyUI\custom_nodes\AIT\AITemplate\AITemplate.py", line 269, in sample
AITemplate.unet[module['sha256']] = AITemplate.loader.apply_unet(
File "C:\Users\joker\OneDrive\Desktop\A.I\ComfyUI\custom_nodes\AIT\AITemplate\ait\load.py", line 215, in apply_unet
return self.apply(aitemplate_module, ait_params)
File "C:\Users\joker\OneDrive\Desktop\A.I\ComfyUI\custom_nodes\AIT\AITemplate\ait\load.py", line 194, in apply
aitemplate_module.set_many_constants_with_tensors(ait_params)
File "C:\Users\joker\OneDrive\Desktop\A.I\ComfyUI\custom_nodes\AIT\AITemplate\ait\module\model.py", line 839, in set_many_constants_with_tensors
self.set_many_constants(ait_tensors)
File "C:\Users\joker\OneDrive\Desktop\A.I\ComfyUI\custom_nodes\AIT\AITemplate\ait\module\model.py", line 790, in set_many_constants
self.DLL.AITemplateModelContainerSetManyConstants(
File "C:\Users\joker\OneDrive\Desktop\A.I\ComfyUI\custom_nodes\AIT\AITemplate\ait\module\model.py", line 211, in _wrapped_func
raise RuntimeError(f"Error in function: {method.name}")
RuntimeError: Error in function: AITemplateModelContainerSetManyConstants
Prompt executed in 25.88 seconds`
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.