jordach / comfy-consistency-vae Goto Github PK

Probably needs some sort of optimization for 8 GB cards, if even possible, don't know if tiled VAE is even possible.
Kept throwing this:

ERROR:root:!!! Exception during processing !!!
ERROR:root:Traceback (most recent call last):
File "D:\WORK\conda_envs\ConfyUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 153, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "D:\WORK\conda_envs\ConfyUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 83, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "D:\WORK\conda_envs\ConfyUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 76, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "D:\WORK\conda_envs\ConfyUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfy-consistency-vae\nodes.py", line 29, in decode
consistent_latent = decoder_consistency(latent["samples"].to("cuda:0"))
File "D:\WORK\conda_envs\ConfyUI\ComfyUI_windows_portable\python_embeded\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "D:\WORK\conda_envs\ConfyUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfy-consistency-vae\consistencydecoder_init_.py", line 159, in call
model_output = self.ckpt(c_in * x_start, t_, features=features)
File "D:\WORK\conda_envs\ConfyUI\ComfyUI_windows_portable\python_embeded\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
RuntimeError: The following operation failed in the TorchScript interpreter.
Traceback of TorchScript, serialized code (most recent call last):
File "code/torch/dalle_runner_api/model_infra/modules/public_diff_vae.py", line 122, in forward
_28 = (up_1_conv_2).forward(_27, _6, _0, )
_29 = (up_1_conv_3).forward(_28, _5, _0, )
_30 = (up_0_conv_0).forward((up_1_upsamp).forward(_29, _0, ), _4, _0, )
~~~~~~~~~~~~~~~~~~~~ <--- HERE
_31 = (up_0_conv_1).forward(_30, _3, _0, )
_32 = (up_0_conv_2).forward(_31, _2, _0, )
File "code/torch/dalle_runner_api/model_infra/modules/public_diff_vae/___torch_mangle_210.py", line 24, in forward
m = torch.cat([argument_1, argument_2], 1)
x = torch.to(m, 5)
_0, _1, = (gn_1).forward(x, )
~~~~~~~~~~~~~ <--- HERE
_2 = (f_1).forward(_0, )
x0 = torch.silu(argument_3)
File "code/torch/dalle_runner_api/model_infra/modules/public_diff_vae/___torch_mangle_204.py", line 14, in forward
x0 = torch.contiguous(x)
input = torch.to(x0, 6)
input0 = torch.to(torch.group_norm(input, 32, g, b), 5)
~~~~~~~~~~~~~~~~ <--- HERE
x1 = torch.silu(input0)
return (x1, x0)

Traceback of TorchScript, original code (most recent call last):
/root/.pyenv/versions/3.11.5/lib/python3.11/site-packages/torch/nn/functional.py(2558): group_norm
/root/code/dalle-runner-api/dalle_runner_api/model_infra/modules/public_diff_vae.py(201): forward
/root/.pyenv/versions/3.11.5/lib/python3.11/site-packages/torch/nn/modules/module.py(1508): _slow_forward
/root/.pyenv/versions/3.11.5/lib/python3.11/site-packages/torch/nn/modules/module.py(1527): _call_impl
/root/.pyenv/versions/3.11.5/lib/python3.11/site-packages/torch/nn/modules/module.py(1518): _wrapped_call_impl
/root/code/dalle-runner-api/dalle_runner_api/model_infra/modules/public_diff_vae.py(728): forward
/root/.pyenv/versions/3.11.5/lib/python3.11/site-packages/torch/nn/modules/module.py(1508): _slow_forward
/root/.pyenv/versions/3.11.5/lib/python3.11/site-packages/torch/nn/modules/module.py(1527): _call_impl
/root/.pyenv/versions/3.11.5/lib/python3.11/site-packages/torch/nn/modules/module.py(1518): _wrapped_call_impl
/root/code/dalle-runner-api/dalle_runner_api/model_infra/modules/public_diff_vae.py(1037): forward
/root/.pyenv/versions/3.11.5/lib/python3.11/site-packages/torch/nn/modules/module.py(1508): _slow_forward
/root/.pyenv/versions/3.11.5/lib/python3.11/site-packages/torch/nn/modules/module.py(1527): _call_impl
/root/.pyenv/versions/3.11.5/lib/python3.11/site-packages/torch/nn/modules/module.py(1518): _wrapped_call_impl
/root/.pyenv/versions/3.11.5/lib/python3.11/site-packages/torch/jit/_trace.py(1065): trace_module
/root/.pyenv/versions/3.11.5/lib/python3.11/site-packages/torch/jit/_trace.py(798): trace
(442): test_diff_vae
/root/.pyenv/versions/3.11.5/lib/python3.11/site-packages/fire/core.py(681): _CallAndUpdateTrace
/root/.pyenv/versions/3.11.5/lib/python3.11/site-packages/fire/core.py(466): _Fire
/root/.pyenv/versions/3.11.5/lib/python3.11/site-packages/fire/core.py(141): Fire
(470):
/root/.pyenv/versions/3.11.5/lib/python3.11/site-packages/IPython/core/interactiveshell.py(3526): run_code
/root/.pyenv/versions/3.11.5/lib/python3.11/site-packages/IPython/core/interactiveshell.py(3466): run_ast_nodes
/root/.pyenv/versions/3.11.5/lib/python3.11/site-packages/IPython/core/interactiveshell.py(3284): run_cell_async
/root/.pyenv/versions/3.11.5/lib/python3.11/site-packages/IPython/core/async_helpers.py(129): _pseudo_sync_runner
/root/.pyenv/versions/3.11.5/lib/python3.11/site-packages/IPython/core/interactiveshell.py(3079): _run_cell
/root/.pyenv/versions/3.11.5/lib/python3.11/site-packages/IPython/core/interactiveshell.py(3024): run_cell
/root/.pyenv/versions/3.11.5/lib/python3.11/site-packages/IPython/terminal/interactiveshell.py(881): interact
/root/.pyenv/versions/3.11.5/lib/python3.11/site-packages/IPython/terminal/interactiveshell.py(888): mainloop
/root/.pyenv/versions/3.11.5/lib/python3.11/site-packages/IPython/terminal/ipapp.py(317): start
/root/.pyenv/versions/3.11.5/lib/python3.11/site-packages/traitlets/config/application.py(1053): launch_instance
/root/.pyenv/versions/3.11.5/lib/python3.11/site-packages/IPython/init.py(129): start_ipython
/root/.pyenv/versions/3.11.5/bin/ipython(8):
RuntimeError: Allocation on device 0 would exceed allowed memory. (out of memory)
Currently allocated : 5.86 GiB
Requested : 658.36 MiB
Device limit : 8.00 GiB
Free (according to CUDA): 0 bytes
PyTorch limit (set by user-supplied memory fraction)
: 17179869184.00 GiB

When downloading using wget, there are no issues, but when using the built-in downloader of the node, it fails as follows.

ERROR:root:Traceback (most recent call last):
  File "/usr/lib/python3.11/urllib/request.py", line 1348, in do_open
    h.request(req.get_method(), req.selector, req.data, headers,
  File "/usr/lib/python3.11/http/client.py", line 1286, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.11/http/client.py", line 1332, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.11/http/client.py", line 1281, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.11/http/client.py", line 1041, in _send_output
    self.send(msg)
  File "/usr/lib/python3.11/http/client.py", line 979, in send
    self.connect()
  File "/usr/lib/python3.11/http/client.py", line 1458, in connect
    self.sock = self._context.wrap_socket(self.sock,
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/ssl.py", line 517, in wrap_socket
    return self.sslsocket_class._create(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/ssl.py", line 1075, in _create
    self.do_handshake()
  File "/usr/lib/python3.11/ssl.py", line 1346, in do_handshake
    self._sslobj.do_handshake()
ConnectionResetError: [Errno 104] Connection reset by peer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/me/git/ComfyUI/execution.py", line 153, in recursive_execute
    output_data, output_ui = get_output_data(obj, input_data_all)
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/me/git/ComfyUI/execution.py", line 83, in get_output_data
    return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/me/git/ComfyUI/execution.py", line 76, in map_node_over_list
    results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/me/git/ComfyUI/custom_nodes/comfy-consistency-vae/nodes.py", line 28, in decode
    decoder_consistency = ConsistencyDecoder(device="cuda:0", download_root=pwd)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/me/git/ComfyUI/custom_nodes/comfy-consistency-vae/consistencydecoder/__init__.py", line 71, in __init__
    download_target = _download("https://openaipublic.azureedge.net/diff-vae/c9cebd3132dd9c42936d803e33424145a748843c8f716c0814838bdc8a2fe7cb/decoder.pt", download_root)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/me/git/ComfyUI/custom_nodes/comfy-consistency-vae/consistencydecoder/__init__.py", line 41, in _download
    with urllib.request.urlopen(url) as source, open(download_target, "wb") as output:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/urllib/request.py", line 216, in urlopen
    return opener.open(url, data, timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/urllib/request.py", line 519, in open
    response = self._open(req, data)
               ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/urllib/request.py", line 536, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/urllib/request.py", line 496, in _call_chain
    result = func(*args)
             ^^^^^^^^^^^
  File "/usr/lib/python3.11/urllib/request.py", line 1391, in https_open
    return self.do_open(http.client.HTTPSConnection, req,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/urllib/request.py", line 1351, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 104] Connection reset by peer>

There seems to be no difference on the outputs.