kijai / comfyui-supir Goto Github PK

Getting this error how can I correct it?

Error occurred when executing SUPIR_Upscale:

"upsample_nearest2d_out_frame" not implemented for 'BFloat16'

Is there a way to make it work on 8gb vram system with 16 gb ram. I have rtx 2060 super 8gb vram and 16 gb system memory.

When creating a new node, the node may not display correctly and needs to be dragged to change its size to display correctly

Can't find it :(

Hi!

Good job with this node! On the description you mentioned video upscaling and posted and example. Could you share the workflow to do it properly? I'm trying some image batch nodes but it doesn't work very well.

Thanks!

Feel free to close the issue if needed, it would be nice if you put the workflow on the readme too, for others to try too.

Isn't there two text encoders in the SDXL model? Why do we need to download them separately?

custom vae choice, the default sdxl vae has invisible-watermark

I found that ckpt_pth.py can fill in the address, but I don't know where to download.

Error occurred when executing SUPIR_Upscale:

Input type (struct c10::Half) and bias type (float) should be the same

I met this problem, how to solve it?

Hello,

I am not a specialized lawyer but you made project license GPL (coming from original MIT which is ok) but have a "non-commercial restriction" which came from a commit three days ago. Non-commercial cannot be combined with GPL
see https://opensource.stackexchange.com/questions/7397/restrict-gplv3-to-non-commercial-use

I also made a ticket in main repo because in my opinion it is similar problem with MIT:
See: Fanghua-Yu/SUPIR#51

For your project I suggest to fork version three days ago which was MIT only from main project and do not merge other changes from main project anymore here making this as an independent project. So it should be fine.

Hi!

Was hoping to give this a test, been running into this error and can't really figure out what's going on, perhaps you could make sense of it?

Error occurred when executing SUPIR_Upscale:

'MemoryEfficientAttnBlock' object has no attribute 'group_norm'

File "D:\ComfyUI02\ComfyUI\execution.py", line 152, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI02\ComfyUI\execution.py", line 82, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI02\ComfyUI\execution.py", line 75, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI02\ComfyUI\custom_nodes\ComfyUI-SUPIR\nodes.py", line 168, in process
samples = self.model.batchify_sample(resized_image[i].unsqueeze(0), captions_list, num_steps=steps, restoration_scale= restoration_scale, s_churn=s_churn,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI02\python_embeded\Lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI02\ComfyUI\custom_nodes\ComfyUI-SUPIR\SUPIR\models\SUPIR_model.py", line 118, in batchify_sample
_z = self.encode_first_stage_with_denoise(x, use_sample=False)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI02\python_embeded\Lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI02\ComfyUI\custom_nodes\ComfyUI-SUPIR\SUPIR\models\SUPIR_model.py", line 56, in encode_first_stage_with_denoise
h = self.first_stage_model.denoise_encoder(x)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI02\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI02\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI02\ComfyUI\custom_nodes\ComfyUI-SUPIR\SUPIR\utils\tilevae.py", line 703, in call
return self.vae_tile_forward(x)
^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI02\ComfyUI\custom_nodes\ComfyUI-SUPIR\SUPIR\utils\tilevae.py", line 586, in wrapper
ret = fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI02\python_embeded\Lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI02\ComfyUI\custom_nodes\ComfyUI-SUPIR\SUPIR\utils\tilevae.py", line 936, in vae_tile_forward
tile = task1
^^^^^^^^^^^^^
File "D:\ComfyUI02\ComfyUI\custom_nodes\ComfyUI-SUPIR\SUPIR\utils\tilevae.py", line 374, in
task_queue.append(('attn', lambda x, net=net: attn_forward_new_pt2_0(net, x)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI02\ComfyUI\custom_nodes\ComfyUI-SUPIR\SUPIR\utils\tilevae.py", line 197, in attn_forward_new_pt2_0
if self.group_norm is not None:
^^^^^^^^^^^^^^^
File "D:\ComfyUI02\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1695, in getattr
raise AttributeError(f"'{type(self).name}' object has no attribute '{name}'")

File "K:\software\ComfyU\ComfyUI_windows_portable\ComfyUI\execution.py", line 152, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "K:\software\ComfyU\ComfyUI_windows_portable\ComfyUI\execution.py", line 82, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "K:\software\ComfyU\ComfyUI_windows_portable\ComfyUI\execution.py", line 75, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "K:\software\ComfyU\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-SUPIR\nodes.py", line 102, in process
samples = self.model.batchify_sample(resized_image, captions_list, num_steps=steps, restoration_scale= restoration_scale, s_churn=s_churn,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "K:\software\ComfyU\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "K:\software\ComfyU\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-SUPIR\SUPIR\models\SUPIR_model.py", line 118, in batchify_sample
_z = self.encode_first_stage_with_denoise(x, use_sample=False)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "K:\software\ComfyU\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "K:\software\ComfyU\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-SUPIR\SUPIR\models\SUPIR_model.py", line 52, in encode_first_stage_with_denoise
with torch.autocast("cuda", dtype=self.ae_dtype):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "K:\software\ComfyU\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\amp\autocast_mode.py", line 305, in init
raise RuntimeError(
RuntimeError: Current CUDA Device does not support bfloat16. Please switch dtype to float16.

This is the progress:
tia@Tia ~ % pip install /Users/tia/ComfyUI/custom_nodes/ComfyUI-SUPIR/requirements.txt
/Library/Frameworks/Python.framework/Versions/3.10/bin/python3.10: can't open file '/Users/tia/install': [Errno 2] No such file or directory

system：macos

I wander know if any workflow nodes or images can test？Thank you

Fresh ComfyUI portable installation. Followed ComfyUI-SUPIR installation steps, copied the files in custom_nodes, and downloaded all files in models/checkpoints. Here is the log when executing SUPIR.

Error occurred when executing SUPIR_Upscale:

No module named 'ComfyUI-SUPIR'

File "C:\Users\X\Desktop\ComfyUI_windows_portable\ComfyUI\execution.py", line 152, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\X\Desktop\ComfyUI_windows_portable\ComfyUI\execution.py", line 82, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\X\Desktop\ComfyUI_windows_portable\ComfyUI\execution.py", line 75, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\X\Desktop\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-SUPIR-main\nodes.py", line 180, in process
self.model = instantiate_from_config(config.model).cpu()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\X\Desktop\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-SUPIR-main\sgm\util.py", line 175, in instantiate_from_config
return get_obj_from_str(config["target"])(**config.get("params", dict()))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\X\Desktop\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-SUPIR-main\sgm\util.py", line 185, in get_obj_from_str
return getattr(importlib.import_module(module, package='ComfyUI-SUPIR'), cls)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "importlib_init_.py", line 126, in import_module
File "", line 1204, in _gcd_import
File "", line 1176, in _find_and_load
File "", line 1126, in _find_and_load_unlocked
File "", line 241, in _call_with_frames_removed
File "", line 1204, in _gcd_import
File "", line 1176, in _find_and_load
File "", line 1126, in _find_and_load_unlocked
File "", line 241, in _call_with_frames_removed
File "", line 1204, in _gcd_import
File "", line 1176, in _find_and_load
File "", line 1126, in _find_and_load_unlocked
File "", line 241, in _call_with_frames_removed
File "", line 1204, in _gcd_import
File "", line 1176, in _find_and_load
File "", line 1140, in _find_and_load_unlocked

Traceback (most recent call last):
File "E:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\nodes.py", line 1887, in load_custom_node
module_spec.loader.exec_module(module)
File "", line 940, in exec_module
File "", line 241, in call_with_frames_removed
File "E:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-SUPIR_init.py", line 1, in
from .nodes import NODE_CLASS_MAPPINGS, NODE_DISPLAY_NAME_MAPPINGS
File "E:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-SUPIR\nodes.py", line 13, in
from .SUPIR.util import convert_dtype, load_state_dict
File "E:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-SUPIR\SUPIR\util.py", line 4, in
import cv2
ModuleNotFoundError: No module named 'cv2'

Cannot import E:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-SUPIR module for custom nodes: No module named 'cv2'

I already tried most fixes that I found on the web

I had this issue before where I would dreambooth train over juggernaut v9 and do a model merge, with another db trained checkpoint. (only an issue when jug v9 is involved with their baked vae)

White orbs would appear. To fix it I use the SDXL fp16 fixed vae. it would also fix the orange haze.

The problem is I don't see any way to change the VAE being used in the supir node. I am not sure if it's just using the baked vae in the model. Is there any way to specify a vae in the node? I would prefer not to retrain a bunch of the models, as they came out before jugg v9, but I merge a little juggernaut at a very low 0.05 and it improves the db quality a ton.

I am unsure if fp16 supported for supir though, it seems like it's using bf16 when I checked the nodes.py?

Here's what happens when using any dreambooth trained checkpoint that has any juggernaut v9 merges and upscaling with supir (this is not the subject):

Low res:

Upscaled:

Hey!
On a fresh ComfyUI install, whenever I run the script, it seems to load the FrozenCLIPembedder, and then stop the script. What could be causing this?

Total VRAM 24564 MB, total RAM 32693 MB
xformers version: 0.0.24
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA GeForce RTX 4090 : cudaMallocAsync
VAE dtype: torch.bfloat16
Using xformers cross attention

Import times for custom nodes:
3.3 seconds: E:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-SUPIR

Starting server

To see the GUI go to: http://127.0.0.1:8188
got prompt
Diffusion using bf16
Encoder using bf16
Building a Downsample layer with 2 dims.
--> settings are:
in-chn: 320, out-chn: 320, kernel-size: 3, stride: 2, padding: 1
constructing SpatialTransformer of depth 2 w/ 640 channels and 10 heads
constructing SpatialTransformer of depth 2 w/ 640 channels and 10 heads
Building a Downsample layer with 2 dims.
--> settings are:
in-chn: 640, out-chn: 640, kernel-size: 3, stride: 2, padding: 1
constructing SpatialTransformer of depth 10 w/ 1280 channels and 20 heads
constructing SpatialTransformer of depth 10 w/ 1280 channels and 20 heads
constructing SpatialTransformer of depth 10 w/ 1280 channels and 20 heads
constructing SpatialTransformer of depth 10 w/ 1280 channels and 20 heads
constructing SpatialTransformer of depth 10 w/ 1280 channels and 20 heads
constructing SpatialTransformer of depth 10 w/ 1280 channels and 20 heads
constructing SpatialTransformer of depth 2 w/ 640 channels and 10 heads
constructing SpatialTransformer of depth 2 w/ 640 channels and 10 heads
constructing SpatialTransformer of depth 2 w/ 640 channels and 10 heads
Initialized embedder #0: FrozenCLIPEmbedder with 123060480 params. Trainable: False

E:\AI\ComfyUI_windows_portable>pause
Press any key to continue . . .

Thanks for your help already ^^

It was said in the original repo, and you also thought it was the case, that it is possible to get running within 12GB VRAM, but I just can't get it to work with this wrapper.

I downloaded all the models, resolved issues with xformers and pytorch+cuda, used an integrated GPU for my display, fiddled with the settings (use_tiled_vae, diffusion_dtype and encoder_dtype) and input a test 512x512 image. Am I missing something?

** Platform: Windows
** Python version: 3.10.13

Total VRAM 12288 MB, total RAM 65309 MB
xformers version: 0.0.23.post1
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA GeForce RTX 3060 : cudaMallocAsync
VAE dtype: torch.bfloat16
Using xformers cross attention

...
...
...

Diffusion using using bf16
Encoder using using bf16
Building a Downsample layer with 2 dims.
  --> settings are:
 in-chn: 320, out-chn: 320, kernel-size: 3, stride: 2, padding: 1
constructing SpatialTransformer of depth 2 w/ 640 channels and 10 heads
constructing SpatialTransformer of depth 2 w/ 640 channels and 10 heads
Building a Downsample layer with 2 dims.
  --> settings are:
 in-chn: 640, out-chn: 640, kernel-size: 3, stride: 2, padding: 1
constructing SpatialTransformer of depth 10 w/ 1280 channels and 20 heads
constructing SpatialTransformer of depth 10 w/ 1280 channels and 20 heads
constructing SpatialTransformer of depth 10 w/ 1280 channels and 20 heads
constructing SpatialTransformer of depth 10 w/ 1280 channels and 20 heads
constructing SpatialTransformer of depth 10 w/ 1280 channels and 20 heads
constructing SpatialTransformer of depth 10 w/ 1280 channels and 20 heads
constructing SpatialTransformer of depth 2 w/ 640 channels and 10 heads
constructing SpatialTransformer of depth 2 w/ 640 channels and 10 heads
constructing SpatialTransformer of depth 2 w/ 640 channels and 10 heads
Initialized embedder #0: FrozenCLIPEmbedder with 123060480 params. Trainable: False
Initialized embedder #1: FrozenOpenCLIPEmbedder2 with 694659841 params. Trainable: False
Initialized embedder #2: ConcatTimestepEmbedderND with 0 params. Trainable: False
Initialized embedder #3: ConcatTimestepEmbedderND with 0 params. Trainable: False
Initialized embedder #4: ConcatTimestepEmbedderND with 0 params. Trainable: False
making attention of type 'vanilla-xformers' with 512 in_channels
building MemoryEfficientAttnBlock with 512 in_channels...
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
making attention of type 'vanilla-xformers' with 512 in_channels
building MemoryEfficientAttnBlock with 512 in_channels...
Building a Downsample layer with 2 dims.
  --> settings are:
 in-chn: 320, out-chn: 320, kernel-size: 3, stride: 2, padding: 1
constructing SpatialTransformer of depth 2 w/ 640 channels and 10 heads
constructing SpatialTransformer of depth 2 w/ 640 channels and 10 heads
Building a Downsample layer with 2 dims.
  --> settings are:
 in-chn: 640, out-chn: 640, kernel-size: 3, stride: 2, padding: 1
constructing SpatialTransformer of depth 10 w/ 1280 channels and 20 heads
constructing SpatialTransformer of depth 10 w/ 1280 channels and 20 heads
constructing SpatialTransformer of depth 10 w/ 1280 channels and 20 heads
Loaded state_dict from [C:\Users\User\ComfyUI\models\checkpoints\SUPIR-v0Q-001.ckpt]
Loaded state_dict from [C:\Users\User\stable-diffusion-webui\models\Stable-diffusion\xl\realvisXLv3.0_v30Bakedvae.safetensors]
ERROR:root:!!! Exception during processing !!!
ERROR:root:Traceback (most recent call last):
  File "C:\Users\User\ComfyUI\execution.py", line 152, in recursive_execute
    output_data, output_ui = get_output_data(obj, input_data_all)
  File "C:\Users\User\ComfyUI\execution.py", line 82, in get_output_data
    return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
  File "C:\Users\User\ComfyUI\execution.py", line 75, in map_node_over_list
    results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
  File "C:\Users\User\ComfyUI\custom_nodes\ComfyUI-SUPIR\nodes.py", line 142, in process
    self.model.to(device).to(dtype)
  File "C:\Users\User\anaconda3\envs\comfyui\lib\site-packages\lightning_fabric\utilities\device_dtype_mixin.py", line 54, in to
    return super().to(*args, **kwargs)
  File "C:\Users\User\anaconda3\envs\comfyui\lib\site-packages\torch\nn\modules\module.py", line 1160, in to
    return self._apply(convert)
  File "C:\Users\User\anaconda3\envs\comfyui\lib\site-packages\torch\nn\modules\module.py", line 810, in _apply
    module._apply(fn)
  File "C:\Users\User\anaconda3\envs\comfyui\lib\site-packages\torch\nn\modules\module.py", line 810, in _apply
    module._apply(fn)
  File "C:\Users\User\anaconda3\envs\comfyui\lib\site-packages\torch\nn\modules\module.py", line 810, in _apply
    module._apply(fn)
  [Previous line repeated 6 more times]
  File "C:\Users\User\anaconda3\envs\comfyui\lib\site-packages\torch\nn\modules\module.py", line 833, in _apply
    param_applied = fn(param)
  File "C:\Users\User\anaconda3\envs\comfyui\lib\site-packages\torch\nn\modules\module.py", line 1158, in convert
    return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)
torch.cuda.OutOfMemoryError: Allocation on device 0 would exceed allowed memory. (out of memory)
Currently allocated     : 11.31 GiB
Requested               : 6.25 MiB
Device limit            : 12.00 GiB
Free (according to CUDA): 0 bytes
PyTorch limit (set by user-supplied memory fraction)
                        : 17179869184.00 GiB

Error occurred when executing SUPIR_Upscale:

maximum recursion depth exceeded while calling a Python object

After the update, it cannot run anymore. Before this, everything could run smoothly

I apologize for this question, I have done a lot of testing and follow your daily updates closely. I keep updating and I want to have quality like the replicate version or the one-click install on Patreon. I really like your versions because they are flexible on comfy UI, easy to mix with other features, but I have done many tests and the results are far from the input image.

I have cloned the repository to my comfui/custom_nodes folder but when loading a workflow file containing the SUPIR_Upscaler node (https://comfyworkflows.com/workflows/b703fa8b-5fe0-4678-8692-021766a891c4) I am greeted with the following error:
When loading the graph, the following node types were not found:
SUPIR_Upscale
Nodes that have failed to load will show as red on the graph.

I get this error on every run now, some commit in the last few days or so must have changed something. I tried changing the bicubic on line 213 of nodes.py to bilinear, that runs - however output is then just pure noise.

Any advice?

Error occurred when executing SUPIR_Upscale:

The operator 'aten::upsample_bicubic2d.out' is not currently implemented for the MPS device. If you want this op to be added in priority during the prototype phase of this feature, please comment on https://github.com/pytorch/pytorch/issues/77764. As a temporary fix, you can set the environment variable `PYTORCH_ENABLE_MPS_FALLBACK=1` to use the CPU as a fallback for this op. WARNING: this will be slower than running natively on MPS.

File "/Users/s/ComfyUI/execution.py", line 152, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/s/ComfyUI/execution.py", line 82, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/s/ComfyUI/execution.py", line 75, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/s/ComfyUI/custom_nodes/ComfyUI-SUPIR/nodes.py", line 213, in process
resized_image = F.interpolate(image, size=(new_height, new_width), mode='bicubic', align_corners=False)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/s/miniconda3/lib/python3.11/site-packages/torch/nn/functional.py", line 4035, in interpolate
return torch._C._nn.upsample_bicubic2d(input, output_size, align_corners, scale_factors)

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

upscaling pictures with use_tiile_resampling is not working. Normal upscaling is working fine and I can do 3 times upscale on a 1024 by 1024 picture, but if I select tiile resampling and go 4 or 5 times upscaling, SUPIR works for some time and then comfyUI stops working. If I select tile resampling and select 2x on a 1024px picture then it works but takes longer than if I just upscale 2x on a 1024px picture with tile resampling. I'll attach my settings and terminal. I'm using runpod with a 20gb gpu.

WARNING[XFORMERS]: xFormers can't load C++/CUDA extensions. xFormers was built for:
PyTorch 2.2.0+cu121 with CUDA 1201 (you have 2.1.2+cu121)
Python 3.11.7 (you have 3.11.6)

I tried upgrading Torch for Comfy and it crashed comfy completely

Im getting this error. am I setting up anything wrong?

--> settings are:
in-chn: 640, out-chn: 640, kernel-size: 3, stride: 2, padding: 1
constructing SpatialTransformer of depth 10 w/ 1280 channels and 20 heads
constructing SpatialTransformer of depth 10 w/ 1280 channels and 20 heads
constructing SpatialTransformer of depth 10 w/ 1280 channels and 20 heads
constructing SpatialTransformer of depth 10 w/ 1280 channels and 20 heads
constructing SpatialTransformer of depth 10 w/ 1280 channels and 20 heads
constructing SpatialTransformer of depth 10 w/ 1280 channels and 20 heads
constructing SpatialTransformer of depth 2 w/ 640 channels and 10 heads
constructing SpatialTransformer of depth 2 w/ 640 channels and 10 heads
constructing SpatialTransformer of depth 2 w/ 640 channels and 10 heads
Initialized embedder #0: FrozenCLIPEmbedder with 123060480 params. Trainable: False
Initialized embedder #1: FrozenOpenCLIPEmbedder2 with 694659841 params. Trainable: False
Initialized embedder #2: ConcatTimestepEmbedderND with 0 params. Trainable: False
Initialized embedder #3: ConcatTimestepEmbedderND with 0 params. Trainable: False
Initialized embedder #4: ConcatTimestepEmbedderND with 0 params. Trainable: False
making attention of type 'vanilla-xformers' with 512 in_channels
building MemoryEfficientAttnBlock with 512 in_channels...
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
making attention of type 'vanilla-xformers' with 512 in_channels
building MemoryEfficientAttnBlock with 512 in_channels...
Building a Downsample layer with 2 dims.
--> settings are:
in-chn: 320, out-chn: 320, kernel-size: 3, stride: 2, padding: 1
constructing SpatialTransformer of depth 2 w/ 640 channels and 10 heads
constructing SpatialTransformer of depth 2 w/ 640 channels and 10 heads
Building a Downsample layer with 2 dims.
--> settings are:
in-chn: 640, out-chn: 640, kernel-size: 3, stride: 2, padding: 1
constructing SpatialTransformer of depth 10 w/ 1280 channels and 20 heads
constructing SpatialTransformer of depth 10 w/ 1280 channels and 20 heads
constructing SpatialTransformer of depth 10 w/ 1280 channels and 20 heads
Attempting to load SUPIR model: [Y:\Automatic1111_130\stable-diffusion-webui\models/Stable-diffusion\Supir\SUPIR-v0Q.ckpt]
Loaded state_dict from [Y:\Automatic1111_130\stable-diffusion-webui\models/Stable-diffusion\Supir\SUPIR-v0Q.ckpt]
Attempting to load SDXL model: [Y:\Automatic1111_130\stable-diffusion-webui\models/Stable-diffusion\SDXL\dreamshaperXL10_alpha2Xl10.safetensors]
Loaded state_dict from [Y:\Automatic1111_130\stable-diffusion-webui\models/Stable-diffusion\SDXL\dreamshaperXL10_alpha2Xl10.safetensors]
['']
Y:\AI\ComfyUI_windows_portable03\python_embeded\Lib\site-packages\lightning_fabric\utilities\seed.py:51: 725389782861152 is not in bounds, numpy accepts from 0 to 4294967295
Seed set to 3919478599
[Tiled VAE]: input_size: torch.Size([1, 3, 1024, 1024]), tile_size: 512, padding: 32
[Tiled VAE]: split to 2x2 = 4 tiles. Optimal tile size 480x480, original tile size 512x512
[Tiled VAE]: Executing Encoder Task Queue: 0%| | 0/364 [00:00<?, ?it/s]ERROR:root:!!! Exception during processing !!!
ERROR:root:Traceback (most recent call last):
File "Y:\AI\ComfyUI_windows_portable03\ComfyUI\execution.py", line 152, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "Y:\AI\ComfyUI_windows_portable03\ComfyUI\execution.py", line 82, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "Y:\AI\ComfyUI_windows_portable03\ComfyUI\execution.py", line 75, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "Y:\AI\ComfyUI_windows_portable03\ComfyUI\custom_nodes\ComfyUI-SUPIR\nodes.py", line 180, in process
samples = self.model.batchify_sample(resized_image[i].unsqueeze(0), captions_list, num_steps=steps, restoration_scale=restoration_scale, s_churn=s_churn,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "Y:\AI\ComfyUI_windows_portable03\python_embeded\Lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "Y:\AI\ComfyUI_windows_portable03\ComfyUI\custom_nodes\ComfyUI-SUPIR\SUPIR\models\SUPIR_model.py", line 125, in batchify_sample
_z = self.encode_first_stage_with_denoise(x, use_sample=False)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "Y:\AI\ComfyUI_windows_portable03\python_embeded\Lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "Y:\AI\ComfyUI_windows_portable03\ComfyUI\custom_nodes\ComfyUI-SUPIR\SUPIR\models\SUPIR_model.py", line 61, in encode_first_stage_with_denoise
h = self.first_stage_model.denoise_encoder(x)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "Y:\AI\ComfyUI_windows_portable03\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "Y:\AI\ComfyUI_windows_portable03\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "Y:\AI\ComfyUI_windows_portable03\ComfyUI\custom_nodes\ComfyUI-SUPIR\SUPIR\utils\tilevae.py", line 703, in call
return self.vae_tile_forward(x)
^^^^^^^^^^^^^^^^^^^^^^^^
File "Y:\AI\ComfyUI_windows_portable03\ComfyUI\custom_nodes\ComfyUI-SUPIR\SUPIR\utils\tilevae.py", line 586, in wrapper
ret = fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "Y:\AI\ComfyUI_windows_portable03\python_embeded\Lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "Y:\AI\ComfyUI_windows_portable03\ComfyUI\custom_nodes\ComfyUI-SUPIR\SUPIR\utils\tilevae.py", line 936, in vae_tile_forward
tile = task1
^^^^^^^^^^^^^
File "Y:\AI\ComfyUI_windows_portable03\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "Y:\AI\ComfyUI_windows_portable03\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "Y:\AI\ComfyUI_windows_portable03\python_embeded\Lib\site-packages\torch\nn\modules\conv.py", line 460, in forward
return self._conv_forward(input, self.weight, self.bias)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "Y:\AI\ComfyUI_windows_portable03\python_embeded\Lib\site-packages\torch\nn\modules\conv.py", line 456, in _conv_forward
return F.conv2d(input, weight, bias, self.stride,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Input type (float) and bias type (struct c10::Half) should be the same

Hey guys, when trying to run this in a more or less simple workflow, the processing stops while loading the models with an OOM error:

ERROR:root:!!! Exception during processing !!!
ERROR:root:Traceback (most recent call last):
  File "/config/05-comfy-ui/ComfyUI/execution.py", line 152, in recursive_execute
    output_data, output_ui = get_output_data(obj, input_data_all)
  File "/config/05-comfy-ui/ComfyUI/execution.py", line 82, in get_output_data
    return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
  File "/config/05-comfy-ui/ComfyUI/custom_nodes/ComfyUI-0246/utils.py", line 381, in new_func
    res_value = old_func(*final_args, **kwargs)
  File "/config/05-comfy-ui/ComfyUI/execution.py", line 75, in map_node_over_list
    results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
  File "/config/05-comfy-ui/ComfyUI/custom_nodes/ComfyUI-SUPIR/nodes.py", line 212, in process
    self.model.init_tile_vae(encoder_tile_size=encoder_tile_size_pixels, decoder_tile_size=decoder_tile_size_latent)
AttributeError: 'SUPIR_Upscale' object has no attribute 'model'

Failed to move model to device
Allocation on device 0 would exceed allowed memory. (out of memory)
Currently allocated     : 15.44 GiB
Requested               : 4.50 MiB
Device limit            : 15.70 GiB
Free (according to CUDA): 20.94 MiB
PyTorch limit (set by user-supplied memory fraction)
                        : 17179869184.00 GiB
Prompt executed in 38.67 seconds

This is using a 4060ti with 16gb of vram.

Here is also the workflow if someone is interested.
Simple-Supir_wFaceDetailer.json

I tried it with different resolutions for the image and even went down to 265x265 without a difference

Is it possible to have model loading as a separate input node, so that it can share the model loading node with previous workflows such as t2i ？

Also, I don't know what happens between the following two steps, it takes so long (about 20+ seconds on my 4090)。

thx

Error occurred when executing SUPIR_Upscale:

module 'comfy.model_management' has no attribute 'should_use_bf16'
Note: I use the portable package comfyui,cuda11.8
Please tell me how to solve this problem

Error occurred when executing SUPIR_Upscale:

No operator found for memory_efficient_attention_forward with inputs:
query : shape=(1, 16384, 1, 512) (torch.bfloat16)
key : shape=(1, 16384, 1, 512) (torch.bfloat16)
value : shape=(1, 16384, 1, 512) (torch.bfloat16)
attn_bias :
p : 0.0
decoderF is not supported because:
max(query.shape[-1] != value.shape[-1]) > 128
xFormers wasn't build with CUDA support
attn_bias type is
operator wasn't built - see python -m xformers.info for more info
[email protected] is not supported because:
max(query.shape[-1] != value.shape[-1]) > 256
xFormers wasn't build with CUDA support
operator wasn't built - see python -m xformers.info for more info
tritonflashattF is not supported because:
max(query.shape[-1] != value.shape[-1]) > 128
xFormers wasn't build with CUDA support
operator wasn't built - see python -m xformers.info for more info
triton is not available
Only work on pre-MLIR triton for now
cutlassF is not supported because:
xFormers wasn't build with CUDA support
operator wasn't built - see python -m xformers.info for more info
smallkF is not supported because:
max(query.shape[-1] != value.shape[-1]) > 32
xFormers wasn't build with CUDA support
dtype=torch.bfloat16 (supported: {torch.float32})
operator wasn't built - see python -m xformers.info for more info
unsupported embed per head: 512

Which one do I use to describe the image?

Is there any hope of running this in 8G vram?

I've tried fp8, which if it works in fp16 in 10G should fit on 8G, but it ends up throwing a out of memory error too.

installed all in comfy portable - all nodes show, when I run, I get error in GUI pop:

Error occurred when executing SUPIR_Upscale:
No module named 'ComfyUI-SUPIR'
File "w:\ComfyUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 152, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "w:\ComfyUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 82, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "w:\ComfyUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 75, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "W:\ComfyUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-SUPIR-main\nodes.py", line 148, in process
self.model = instantiate_from_config(config.model).cpu()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "W:\ComfyUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-SUPIR-main\sgm\util.py", line 175, in instantiate_from_config
return get_obj_from_str(config["target"])(**config.get("params", dict()))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "W:\ComfyUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-SUPIR-main\sgm\util.py", line 185, in get_obj_from_str
return getattr(importlib.import_module(module, package='ComfyUI-SUPIR'), cls)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "importlib_init_.py", line 126, in import_module
File "", line 1204, in _gcd_import
File "", line 1176, in _find_and_load
File "", line 1126, in _find_and_load_unlocked
File "", line 241, in _call_with_frames_removed
File "", line 1204, in _gcd_import
File "", line 1176, in _find_and_load
File "", line 1126, in _find_and_load_unlocked
File "", line 241, in _call_with_frames_removed
File "", line 1204, in _gcd_import
File "", line 1176, in _find_and_load
File "", line 1126, in _find_and_load_unlocked
File "", line 241, in _call_with_frames_removed
File "", line 1204, in _gcd_import
File "", line 1176, in _find_and_load
File "", line 1140, in _find_and_load_unlocked

error in prompt is sim:
moondream: loading model vikhyatk/moondream1, please stand by....
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Diffusion using bf16
Encoder using bf16
ERROR:root:!!! Exception during processing !!!
ERROR:root:Traceback (most recent call last):
File "w:\ComfyUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 152, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "w:\ComfyUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 82, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "w:\ComfyUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 75, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "W:\ComfyUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-SUPIR-main\nodes.py", line 148, in process
self.model = instantiate_from_config(config.model).cpu()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "W:\ComfyUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-SUPIR-main\sgm\util.py", line 175, in instantiate_from_config
return get_obj_from_str(config["target"])(**config.get("params", dict()))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "W:\ComfyUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-SUPIR-main\sgm\util.py", line 185, in get_obj_from_str
return getattr(importlib.import_module(module, package='ComfyUI-SUPIR'), cls)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "importlib_init_.py", line 126, in import_module
File "", line 1204, in _gcd_import
File "", line 1176, in _find_and_load
File "", line 1126, in _find_and_load_unlocked
File "", line 241, in _call_with_frames_removed
File "", line 1204, in _gcd_import
File "", line 1176, in _find_and_load
File "", line 1126, in _find_and_load_unlocked
File "", line 241, in _call_with_frames_removed
File "", line 1204, in _gcd_import
File "", line 1176, in _find_and_load
File "", line 1126, in _find_and_load_unlocked
File "", line 241, in _call_with_frames_removed
File "", line 1204, in _gcd_import
File "", line 1176, in _find_and_load
File "", line 1140, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'ComfyUI-SUPIR'

Prompt executed in 10.25 seconds
ERROR:root:!!! Exception during processing !!!
ERROR:root:Traceback (most recent call last):
File "w:\ComfyUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 152, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "w:\ComfyUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 82, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "w:\ComfyUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 75, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "W:\ComfyUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-SUPIR-main\nodes.py", line 148, in process
self.model = instantiate_from_config(config.model).cpu()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "W:\ComfyUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-SUPIR-main\sgm\util.py", line 175, in instantiate_from_config
return get_obj_from_str(config["target"])(**config.get("params", dict()))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "W:\ComfyUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-SUPIR-main\sgm\util.py", line 185, in get_obj_from_str
return getattr(importlib.import_module(module, package='ComfyUI-SUPIR'), cls)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "importlib_init_.py", line 126, in import_module
File "", line 1204, in _gcd_import
File "", line 1176, in _find_and_load
File "", line 1126, in _find_and_load_unlocked
File "", line 241, in _call_with_frames_removed
File "", line 1204, in _gcd_import
File "", line 1176, in _find_and_load
File "", line 1126, in _find_and_load_unlocked
File "", line 241, in _call_with_frames_removed
File "", line 1204, in _gcd_import
File "", line 1176, in _find_and_load
File "", line 1126, in _find_and_load_unlocked
File "", line 241, in _call_with_frames_removed
File "", line 1204, in _gcd_import
File "", line 1176, in _find_and_load
File "", line 1140, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'ComfyUI-SUPIR'

crashes comfy completely, console shows crash and pauses for close.
open_clip_pytorch_model.bin loaded into model\clip with chkt_pth.py edited to show:
SDXL_CLIP2_CKPT_PTH = "W:\ComfyUI\ComfyUI_windows_portable\ComfyUI\models\clip\open_clip_pytorch_model.bin"
(the location of the clip chkpt).

I saw the other bug report, but it indicated chkpt clip wasnt found, should be according to yet another report as I read that before install . . .

Encoder using bf16
Building a Downsample layer with 2 dims.
--> settings are:
in-chn: 320, out-chn: 320, kernel-size: 3, stride: 2, padding: 1
constructing SpatialTransformer of depth 2 w/ 640 channels and 10 heads
constructing SpatialTransformer of depth 2 w/ 640 channels and 10 heads
Building a Downsample layer with 2 dims.
--> settings are:
in-chn: 640, out-chn: 640, kernel-size: 3, stride: 2, padding: 1
constructing SpatialTransformer of depth 10 w/ 1280 channels and 20 heads
constructing SpatialTransformer of depth 10 w/ 1280 channels and 20 heads
constructing SpatialTransformer of depth 10 w/ 1280 channels and 20 heads
constructing SpatialTransformer of depth 10 w/ 1280 channels and 20 heads
constructing SpatialTransformer of depth 10 w/ 1280 channels and 20 heads
constructing SpatialTransformer of depth 10 w/ 1280 channels and 20 heads
constructing SpatialTransformer of depth 2 w/ 640 channels and 10 heads
constructing SpatialTransformer of depth 2 w/ 640 channels and 10 heads
constructing SpatialTransformer of depth 2 w/ 640 channels and 10 heads
Initialized embedder #0: FrozenCLIPEmbedder with 123060480 params. Trainable: False

I've noticed that on certain images the upscaled image has a different ratio than the original one so that the resulting image looks very slightly stretched.

It has happened when upscaling an image of size 696x768 by 2x. The resulting image size is 1344x1536, which has less width than 2x696 so that the image looks vertically stretched. I can tell it is not cropping the resulting image but just making it narrower. So far it has only happened with the width of the image; it has never happened with the height.

I get this error when executing node looks like it eats a lot of memory.

Error occurred when executing SUPIR_Upscale:

Allocation on device 0 would exceed allowed memory. (out of memory)
Currently allocated : 11.99 GiB
Requested : 63.00 MiB
Device limit : 6.00 GiB
Free (according to CUDA): 0 bytes
PyTorch limit (set by user-supplied memory fraction)
: 17179869184.00 GiB

File "K:\ComfyUI\ComfyUI_Ex\ComfyUI\execution.py", line 152, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "K:\ComfyUI\ComfyUI_Ex\ComfyUI\execution.py", line 82, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "K:\ComfyUI\ComfyUI_Ex\ComfyUI\execution.py", line 75, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "K:\ComfyUI\ComfyUI_Ex\ComfyUI\custom_nodes\ComfyUI-SUPIR\nodes.py", line 148, in process
image, = ImageScaleBy.upscale(self, image, resize_method, scale_by)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "K:\ComfyUI\ComfyUI_Ex\ComfyUI\nodes.py", line 1620, in upscale
s = comfy.utils.common_upscale(samples, width, height, upscale_method, "disabled")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "K:\ComfyUI\ComfyUI_Ex\ComfyUI\comfy\utils.py", line 416, in common_upscale
return lanczos(s, width, height)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "K:\ComfyUI\ComfyUI_Ex\ComfyUI\comfy\utils.py", line 395, in lanczos
return result.to(samples.device, samples.dtype)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

[process crashed with code 3221225477 (0xC0000005)]
The following is an analysis of the exit codes. This may not be accurate and it is for your reference only.
System exit code name: ACCESS_VIOLATION
System exit code description: The instruction at 0x%p referenced memory at 0x%p. The memory could not be %s.

Crash thread traceback:
Windows fatal exception: access violation

Stack (most recent call first):
File "E:\ComfyUI-aki-v1\python\lib\site-packages\torch\storage.py", line 314 in getitem
File "E:\ComfyUI-aki-v1\python\lib\site-packages\safetensors\torch.py", line 310 in load_file
File "E:\ComfyUI-aki-v1\python\lib\site-packages\transformers\modeling_utils.py", line 512 in load_state_dict
File "E:\ComfyUI-aki-v1\python\lib\site-packages\transformers\modeling_utils.py", line 3306 in from_pretrained
File "E:\ComfyUI-aki-v1\custom_nodes\ComfyUI-SUPIR\sgm\modules\encoders\modules.py", line 465 in init
File "E:\ComfyUI-aki-v1\custom_nodes\ComfyUI-SUPIR\sgm\util.py", line 175 in instantiate_from_config
File "E:\ComfyUI-aki-v1\custom_nodes\ComfyUI-SUPIR\sgm\modules\encoders\modules.py", line 91 in init
File "E:\ComfyUI-aki-v1\custom_nodes\ComfyUI-SUPIR\sgm\util.py", line 175 in instantiate_from_config
File "E:\ComfyUI-aki-v1\custom_nodes\ComfyUI-SUPIR\sgm\models\diffusion.py", line 63 in init
File "E:\ComfyUI-aki-v1\custom_nodes\ComfyUI-SUPIR\SUPIR\models\SUPIR_model.py", line 17 in init
File "E:\ComfyUI-aki-v1\custom_nodes\ComfyUI-SUPIR\sgm\util.py", line 175 in instantiate_from_config
File "E:\ComfyUI-aki-v1\custom_nodes\ComfyUI-SUPIR\nodes.py", line 181 in process
File "E:\ComfyUI-aki-v1\execution.py", line 75 in map_node_over_list
File "E:\ComfyUI-aki-v1\execution.py", line 82 in get_output_data
File "E:\ComfyUI-aki-v1\execution.py", line 152 in recursive_execute
File "E:\ComfyUI-aki-v1\execution.py", line 135 in recursive_execute
File "E:\ComfyUI-aki-v1\execution.py", line 392 in execute
File "E:\ComfyUI-aki-v1\custom_nodes\rgthree-comfy_init_.py", line 209 in rgthree_execute
File "E:\ComfyUI-aki-v1\main.py", line 111 in prompt_worker
File "E:\ComfyUI-aki-v1\python\lib\threading.py", line 953 in run
File "<enhanced_experience vendors.sentry_sdk.integrations.threading>", line 70 in run
File "E:\ComfyUI-aki-v1\python\lib\threading.py", line 1016 in _bootstrap_inner
File "E:\ComfyUI-aki-v1\python\lib\threading.py", line 973 in _bootstrap

Note: Exit codes and their descriptions are not sufficient to

diagnose the problem! Please upload the complete diagnostic log with codes.

Computer: 32 memory 8GB graphics card

He Kijai, I'm getting this error:

Error occurred when executing SUPIR_Upscale:

module 'comfy.model_management' has no attribute 'should_use_bf16'

File "F:\COMFY\ComfyUI\execution.py", line 152, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "F:\COMFY\ComfyUI\execution.py", line 82, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "F:\COMFY\ComfyUI\execution.py", line 75, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "F:\COMFY\ComfyUI\custom_nodes\ComfyUI-SUPIR\nodes.py", line 96, in process
if comfy.model_management.should_use_bf16():
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Can you please advise?

I get the following error.

It says I need Pytorch >= 2.0 but it seems I'm already running 2.1.2

Attention mode 'softmax-xformers' is not available. Falling back to native attention. This is not a problem in Pytorch >= 2.0. FYI, you are running with PyTorch version 2.1.2+cu121

Isn’t it normal? I still don’t need it.

A couple of days ago I was testing Supir online running in "replicate". At the end of the trial period and after the good results I decided to install it on my PC. I found this version and installed it very easily. It works very well with many images, but with others the quality is much worse than the replicate version or the example images. It especially fails with people's faces (like this moment of this video) or with details that are difficult to see (like this other moment), while the replicate version was almost perfect with faces and small details. I have tried to copy the values ​​of replicate in SUPIR-COMFYUI but the result varies a lot. I see that in replicate it has 2 stages and in COMFYUI it has only 1 stage. As an SDXL model I am using the base SDXL model, but I have also tried Juggernaut XL and RunDiffusion XL, obtaining worse results.

I want to try the gradio version with LLava, but I'm having problems getting it to work.

I have a 4080 16GB VRAM and 32GB RAM. I can render images without problems up to 2500x2500.

What could be the problem?

Loading 1 new model
Diffusion using bf16
Encoder using bf16
Building a Downsample layer with 2 dims.
--> settings are:
in-chn: 320, out-chn: 320, kernel-size: 3, stride: 2, padding: 1
constructing SpatialTransformer of depth 2 w/ 640 channels and 10 heads
constructing SpatialTransformer of depth 2 w/ 640 channels and 10 heads
Building a Downsample layer with 2 dims.
--> settings are:
in-chn: 640, out-chn: 640, kernel-size: 3, stride: 2, padding: 1
constructing SpatialTransformer of depth 10 w/ 1280 channels and 20 heads
constructing SpatialTransformer of depth 10 w/ 1280 channels and 20 heads
constructing SpatialTransformer of depth 10 w/ 1280 channels and 20 heads
constructing SpatialTransformer of depth 10 w/ 1280 channels and 20 heads
constructing SpatialTransformer of depth 10 w/ 1280 channels and 20 heads
constructing SpatialTransformer of depth 10 w/ 1280 channels and 20 heads
constructing SpatialTransformer of depth 2 w/ 640 channels and 10 heads
constructing SpatialTransformer of depth 2 w/ 640 channels and 10 heads
constructing SpatialTransformer of depth 2 w/ 640 channels and 10 heads
Initialized embedder #0: FrozenCLIPEmbedder with 123060480 params. Trainable: False
Initialized embedder #1: FrozenOpenCLIPEmbedder2 with 694659841 params. Trainable: False
Initialized embedder #2: ConcatTimestepEmbedderND with 0 params. Trainable: False
Initialized embedder #3: ConcatTimestepEmbedderND with 0 params. Trainable: False
Initialized embedder #4: ConcatTimestepEmbedderND with 0 params. Trainable: False
making attention of type 'vanilla-xformers' with 512 in_channels
building MemoryEfficientAttnBlock with 512 in_channels...
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
making attention of type 'vanilla-xformers' with 512 in_channels
building MemoryEfficientAttnBlock with 512 in_channels...
Building a Downsample layer with 2 dims.
--> settings are:
in-chn: 320, out-chn: 320, kernel-size: 3, stride: 2, padding: 1
constructing SpatialTransformer of depth 2 w/ 640 channels and 10 heads
constructing SpatialTransformer of depth 2 w/ 640 channels and 10 heads
Building a Downsample layer with 2 dims.
--> settings are:
in-chn: 640, out-chn: 640, kernel-size: 3, stride: 2, padding: 1
constructing SpatialTransformer of depth 10 w/ 1280 channels and 20 heads
constructing SpatialTransformer of depth 10 w/ 1280 channels and 20 heads
constructing SpatialTransformer of depth 10 w/ 1280 channels and 20 heads
Loaded state_dict from [D:\Program Files\Visions of Chaos\Machine Learning Files\Text To Image\ComfyUI\ComfyUI\models\checkpoints\SUPIR\v0F.ckpt]

I was able to reproduce in a fresh install of portable as well as my current install through Visions of Chaos.
I've gotten it to work on smaller 512p images, but even 1.1x on a 1024 image didn't work.
0Q and 0F models both.
Freshly updated and reinstalled node from most recent commit.
3090, 24GB VRAM

Any suggestions on the error below?

[Tiled VAE]: Executing Encoder Task Queue: 84%|███████████████████████████▋ | 1526/1820 [00:07<00:00, 1131.84it/s]ERROR:root:!!! Exception during processing !!!
ERROR:root:Traceback (most recent call last):
File "D:\SD\ComfyUI\ComfyUI\execution.py", line 152, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\SD\ComfyUI\ComfyUI\execution.py", line 82, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\SD\ComfyUI\ComfyUI\execution.py", line 75, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\SD\ComfyUI\ComfyUI\custom_nodes\ComfyUI-SUPIR\nodes.py", line 126, in process
samples = self.model.batchify_sample(resized_image[i].unsqueeze(0), captions_list, num_steps=steps, restoration_scale= restoration_scale, s_churn=s_churn,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\SD\ComfyUI\python_embeded\Lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "D:\SD\ComfyUI\ComfyUI\custom_nodes\ComfyUI-SUPIR\SUPIR\models\SUPIR_model.py", line 118, in batchify_sample
_z = self.encode_first_stage_with_denoise(x, use_sample=False)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\SD\ComfyUI\python_embeded\Lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "D:\SD\ComfyUI\ComfyUI\custom_nodes\ComfyUI-SUPIR\SUPIR\models\SUPIR_model.py", line 56, in encode_first_stage_with_denoise
h = self.first_stage_model.denoise_encoder(x)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\SD\ComfyUI\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\SD\ComfyUI\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\SD\ComfyUI\ComfyUI\custom_nodes\ComfyUI-SUPIR\SUPIR\utils\tilevae.py", line 703, in call
return self.vae_tile_forward(x)
^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\SD\ComfyUI\ComfyUI\custom_nodes\ComfyUI-SUPIR\SUPIR\utils\tilevae.py", line 586, in wrapper
ret = fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "D:\SD\ComfyUI\python_embeded\Lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "D:\SD\ComfyUI\ComfyUI\custom_nodes\ComfyUI-SUPIR\SUPIR\utils\tilevae.py", line 938, in vae_tile_forward
tile = task1
^^^^^^^^^^^^^
File "D:\SD\ComfyUI\ComfyUI\custom_nodes\ComfyUI-SUPIR\SUPIR\utils\tilevae.py", line 374, in
task_queue.append(('attn', lambda x, net=net: attn_forward_new_pt2_0(net, x)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\SD\ComfyUI\ComfyUI\custom_nodes\ComfyUI-SUPIR\SUPIR\utils\tilevae.py", line 197, in attn_forward_new_pt2_0
if self.group_norm is not None:
^^^^^^^^^^^^^^^
File "D:\SD\ComfyUI\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1688, in getattr
raise AttributeError(f"'{type(self).name}' object has no attribute '{name}'")
AttributeError: 'MemoryEfficientAttnBlock' object has no attribute 'group_norm'

Prompt executed in 8.43 seconds
[Tiled VAE]: Executing Encoder Task Queue: 85%|████████████████████████████▊ | 1541/1820 [00:07<00:01, 202.36it/s]