kahsolt / stable-diffusion-webui-vid2vid

Translate a video to some AI generated stuff, extension script for AUTOMATIC1111/stable-diffusion-webui.

• 👋 Hi, I’m @Kahsolt
• 👀 I’m interested in neural networks, computation theory, philosophy, music writing
• 🌱 I’m currently learning Stable-Diffusion ;)
• 💞️ I’m looking to collaborate on Generative Models and Adversarial Learning
• 📫 Find my ugly personal site at https://kahsolt.pythonanywhere.com/

stable-diffusion-webui-vid2vid's Issues

incompatibility

hello,

it seems this is not working after the release 2.7 of https://github.com/Kahsolt/stable-diffusion-webui-prompt-travel ( it seems that the reason is the disappearance of the function process_images_before at the release 3.0

[gif @ 000001d44592cc40] GIF muxer supports only a single video GIF stream.

[gif @ 000001d44592cc40] GIF muxer supports only a single video GIF stream.
Could not write header for output file #0 (incorrect codec parameters ?): Invalid argument
Error initializing output stream 0:0 --

I get that error when I try to export to gif. It works fine for mp4.

ValueError: operands could not be broadcast together with shapes (472,848,3) (476,848,3)

This error happened with vid2vid and:
flat2DAnimerge
revAnimated_v122
and v1-5-pruned-emaonly.safetensors

The first frame is generated successfully, but the error occurs. This is the part in the console:

Loading weights [6ce0161689] from D:\Stable Diffusion\webui\models\Stable-diffusion\v1-5-pruned-emaonly.safetensors
Applying optimization: xformers... done.
Weights loaded in 4.3s (load weights from disk: 0.4s, apply weights to model: 3.9s).
2023-07-09 14:15:49,133 - ControlNet - INFO - Loading model: control_v11p_sd15_softedge [a8575a2a]
2023-07-09 14:15:49,588 - ControlNet - INFO - Loaded state_dict from [D:\Stable Diffusion\webui\extensions\sd-webui-controlnet\models\control_v11p_sd15_softedge.pth]
2023-07-09 14:15:49,589 - ControlNet - INFO - Loading config: D:\Stable Diffusion\webui\extensions\sd-webui-controlnet\models\control_v11p_sd15_softedge.yaml
2023-07-09 14:15:52,202 - ControlNet - INFO - ControlNet model control_v11p_sd15_softedge [a8575a2a] loaded.
2023-07-09 14:15:52,295 - ControlNet - INFO - Loading preprocessor: pidinet
2023-07-09 14:15:52,296 - ControlNet - INFO - preprocessor resolution = 512
2023-07-09 14:15:52,322 - ControlNet - INFO - Loading model from cache: control_v11p_sd15_softedge [a8575a2a]
2023-07-09 14:15:52,323 - ControlNet - INFO - Loading preprocessor: pidinet
2023-07-09 14:15:52,323 - ControlNet - INFO - preprocessor resolution = 512
2023-07-09 14:15:52,354 - ControlNet - INFO - Loading preprocessor: reference_only
2023-07-09 14:15:52,354 - ControlNet - INFO - preprocessor resolution = 512
0%| | 0/12 [00:00<?, ?it/s]2023-07-09 14:15:53,821 - ControlNet - INFO - ControlNet used torch.float16 VAE to encode torch.Size([1, 4, 59, 106]).
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 12/12 [00:08<00:00, 1.39it/s]
Total progress: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 12/12 [00:06<00:00, 1.72it/s]
2023-07-09 14:16:07,362 - ControlNet - INFO - Loading model from cache: control_v11p_sd15_softedge [a8575a2a]███████████████████████████████████████████████████████████████████████████████████████████████| 12/12 [00:06<00:00, 1.80it/s]
2023-07-09 14:16:07,364 - ControlNet - INFO - Loading preprocessor: pidinet
2023-07-09 14:16:07,364 - ControlNet - INFO - preprocessor resolution = 512
2023-07-09 14:16:07,389 - ControlNet - INFO - Loading model from cache: control_v11p_sd15_softedge [a8575a2a]
2023-07-09 14:16:07,391 - ControlNet - INFO - Loading preprocessor: pidinet
2023-07-09 14:16:07,391 - ControlNet - INFO - preprocessor resolution = 512
2023-07-09 14:16:07,417 - ControlNet - INFO - Loading preprocessor: reference_only
2023-07-09 14:16:07,417 - ControlNet - INFO - preprocessor resolution = 512
0%| | 0/5 [00:00<?, ?it/s]2023-07-09 14:16:08,916 - ControlNet - INFO - ControlNet used torch.float16 VAE to encode torch.Size([1, 4, 59, 106]).
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5/5 [00:04<00:00, 1.16it/s]
Total progress: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5/5 [00:03<00:00, 1.63it/s]
An exception occurred while trying to process the frame: operands could not be broadcast together with shapes (472,848,3) (476,848,3) ████████████████████████████████████████████████████████████████████████| 5/5 [00:03<00:00, 2.02it/s]
Traceback (most recent call last):
File "D:\Stable Diffusion\webui\extensions\SD-CN-Animation\scripts\base_ui.py", line 144, in process
yield from vid2vid.start_process(*args)
File "D:\Stable Diffusion\webui\extensions\SD-CN-Animation\scripts\core\vid2vid.py", line 222, in start_process
processed_frame = processed_frame.astype(float) * alpha_mask + warped_styled_frame.astype(float) * (1 - alpha_mask)
ValueError: operands could not be broadcast together with shapes (472,848,3) (476,848,3)