关于单独使用Video_blend.py的问题 about rerender_a_video

[Adam34332]

补充一下, 还有关于关键帧的设置问题. 在代码中set KEY_FRAME_INTERVAL=被设定为一个固定的值, 如果我加入随机值(关键帧计算) 后续是否会出现比较大的问题? 因为我尝试提取的关键帧有可能是随机的, 彼此间的数值并不固定. 这种情况我在EbSynth studio中使用并成功了, 不过因为你们的脚本中加入了光流等其他合成方式在一起. 不太确定后续是否会出现其他问题. 因为修改的时候发生太多报错. 囧. 哈哈. 谢谢你们的努力.

[williamyang1991]

抱歉，仅根据你的描述无法确定问题的在哪里。我们并不知道你修改了什么，你修改的哪些地方导致了报错。
我们自己测试的时候是没有报错的。

如果你关键帧是随机的，最简单的方法是每两个相邻的关键帧使用一次Video_blend.py
比如关键帧是25和38，
那

python video_blend.py videos/xxx \
  --beg 25 \
  --end 38 \
  --itv 13 \
  --key keys \
  --output videos/xxxx/blend.mp4 \
  --fps xxxx \
  -ps

多调用几次。

[Adam34332]

关键帧的问题比较好解决, 现在问题发生在了RuntimeError: The size of tensor a (8) must match the size of tensor b (64) at non-singleton dimension 0这个上面, 这个问题在comfyui中有时候也会遇到. 对于这个有没有什么比较好的建议. 非常感谢

[williamyang1991]

我们自己测试的时候是没有报错的。
仅根据你的描述也无法确定问题的在哪里，所以没法提供有用的建议。

[Adam34332]

这是启动时报的错误代码,
Value of b: 2
Process Process-2:
Traceback (most recent call last):
File "C:\Users\AdamLiu\AppData\Local\Programs\Python\Python310\lib\multiprocessing\process.py", line 314, in _bootstrap
self.run()
File "C:\Users\AdamLiu\AppData\Local\Programs\Python\Python310\lib\multiprocessing\process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "J:\Rerender_A_Video\video_blend.py", line 109, in process_sequences
process_one_sequence(i, video_sequence)
File "J:\Rerender_A_Video\video_blend.py", line 76, in process_one_sequence
flow_calc.get_flow(i1, i2, flow_seq[j])
File "C:\Users\AdamLiu\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "J:\Rerender_A_Video\flow\flow_utils.py", line 172, in get_flow
results_dict = self.model(image1,
File "C:\Users\AdamLiu\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "J:\Rerender_A_Video\deps/gmflow\gmflow\gmflow.py", line 136, in forward
feature0, feature1 = self.transformer(feature0, feature1, attn_num_splits=attn_splits)
File "C:\Users\AdamLiu\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "J:\Rerender_A_Video\deps/gmflow\gmflow\transformer.py", line 338, in forward
concat0 = layer(concat0, concat1,
File "C:\Users\AdamLiu\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "J:\Rerender_A_Video\deps/gmflow\gmflow\transformer.py", line 256, in forward
source = self.self_attn(source, source,
File "C:\Users\AdamLiu\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "J:\Rerender_A_Video\deps/gmflow\gmflow\transformer.py", line 198, in forward
message = single_head_split_window_attention(query, key, value,
File "J:\Rerender_A_Video\deps/gmflow\gmflow\transformer.py", line 106, in single_head_split_window_attention
scores += attn_mask.repeat(b_new, 1, 1)
RuntimeError: The size of tensor a (8) must match the size of tensor b (64) at non-singleton dimension 0
Process Process-4:
Traceback (most recent call last):
File "C:\Users\AdamLiu\AppData\Local\Programs\Python\Python310\lib\multiprocessing\process.py", line 314, in _bootstrap
self.run()
File "C:\Users\AdamLiu\AppData\Local\Programs\Python\Python310\lib\multiprocessing\process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "J:\Rerender_A_Video\video_blend.py", line 109, in process_sequences
process_one_sequence(i, video_sequence)
File "J:\Rerender_A_Video\video_blend.py", line 76, in process_one_sequence
flow_calc.get_flow(i1, i2, flow_seq[j])
File "C:\Users\AdamLiu\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "J:\Rerender_A_Video\flow\flow_utils.py", line 172, in get_flow
results_dict = self.model(image1,
File "C:\Users\AdamLiu\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "J:\Rerender_A_Video\deps/gmflow\gmflow\gmflow.py", line 136, in forward
feature0, feature1 = self.transformer(feature0, feature1, attn_num_splits=attn_splits)
File "C:\Users\AdamLiu\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "J:\Rerender_A_Video\deps/gmflow\gmflow\transformer.py", line 338, in forward
concat0 = layer(concat0, concat1,
File "C:\Users\AdamLiu\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "J:\Rerender_A_Video\deps/gmflow\gmflow\transformer.py", line 256, in forward
source = self.self_attn(source, source,
File "C:\Users\AdamLiu\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "J:\Rerender_A_Video\deps/gmflow\gmflow\transformer.py", line 198, in forward
message = single_head_split_window_attention(query, key, value,
File "J:\Rerender_A_Video\deps/gmflow\gmflow\transformer.py", line 106, in single_head_split_window_attention
scores += attn_mask.repeat(b_new, 1, 1)
RuntimeError: The size of tensor a (8) must match the size of tensor b (64) at non-singleton dimension 0"""

[williamyang1991]

这个问题光看log我也解决不了，输入是啥都不知道，谁都帮不了你。

你自己debug一下，print一下中间的tensor的shape都是啥，看运行到哪里tensor的维度不匹配了。

flow_calc.get_flow(i1, i2, flow_seq[j])

而部分代码是gmflow的代码，我们只是调用也没有改过。实在不行你去gmflow的github页面问问。

以及gmflow好像要求视频的边长都是16的倍数，你可以试试把你的视频边长修改一下。

[Adam34332]

对, 行. 哈哈. 谢谢你的帮助. 确实被改的地方有些多. 我再查查看. 你们做的这个很棒. 加油.