Comments (33)
Godatplay
commented on July 20, 2024
1
I was able to get this working just fine on Linux myself. I'm on Peppermint (Ubuntu-based), using PyTorch 0.4.1 with this conda build: py37_cuda9.2.148_cudnn7.1.4_1
I had to uninstall ffmpeg from conda because the included package doesn't include libx264, so I used apt to install it plus the libavcodec-extra metapackage.
My test video is here and the results can be found here. They seem to line up with the poor results examples, but you can see it working well on the mans' pants, background environment, and smaller figures walking in the background. My assumption is that the model needs more data from overlapping biped movement and limb rotation perpendicular to the camera (potentially more non-Caucasian data as well?), but is otherwise working as expected.
from super-slomo.
avinashpaliwal
commented on July 20, 2024
That is weird. The channel wise mean is subtracted from the frames before passing them as input and added back to the output. Maybe something went wrong with that. Can you post your output after running the script. Also mention the pytorch version you have installed.
from super-slomo.
ybabs
commented on July 20, 2024
pytorch version is 1.0.0
This is my output after running the script:
ffmpeg version 4.0.3-1~18.04.york0 Copyright (c) 2000-2018 the FFmpeg developers
built with gcc 7 (Ubuntu 7.3.0-27ubuntu1~18.04)
configuration: --prefix=/usr --extra-version='1~18.04.york0' --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared
WARNING: library configuration mismatch
avcodec configuration: --prefix=/usr --extra-version='1~18.04.york0' --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared --enable-version3 --disable-doc --disable-programs --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libtesseract --enable-libvo_amrwbenc
libavutil 56. 14.100 / 56. 14.100
libavcodec 58. 18.100 / 58. 18.100
libavformat 58. 12.100 / 58. 12.100
libavdevice 58. 3.100 / 58. 3.100
libavfilter 7. 16.100 / 7. 16.100
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 1.100 / 5. 1.100
libswresample 3. 1.100 / 3. 1.100
libpostproc 55. 1.100 / 55. 1.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/home/homepc/Super-SloMo/input.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf58.12.100
Duration: 00:00:01.24, start: 0.000000, bitrate: 948 kb/s
Stream #0:0(und): Video: h264 (High 4:4:4 Predictive) (avc1 / 0x31637661), yuv444p, 480x270 [SAR 1:1 DAR 16:9], 940 kb/s, 25 fps, 25 tbr, 12800 tbn, 50 tbc (default)
Metadata:
handler_name : VideoHandler
Stream mapping:
Stream #0:0 -> #0:0 (h264 (native) -> mjpeg (native))
Press [q] to stop, [?] for help
[swscaler @ 0x55f8c904c7c0] deprecated pixel format used, make sure you did set range correctly
Output #0, image2, to 'tmpSuperSloMo/input/%06d.jpg':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf58.12.100
Stream #0:0(und): Video: mjpeg, yuvj444p(pc), 480x270 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbc (default)
Metadata:
handler_name : VideoHandler
encoder : Lavc58.18.100 mjpeg
Side data:
cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: -1
frame= 31 fps=0.0 q=2.0 Lsize=N/A time=00:00:01.24 bitrate=N/A speed=25.7x
video:1452kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
/home/homepc/anaconda3/lib/python3.6/site-packages/torch/nn/functional.py:2423: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
"See the documentation of nn.Upsample for details.".format(mode))
/home/homepc/anaconda3/lib/python3.6/site-packages/torch/nn/functional.py:1332: UserWarning: nn.functional.sigmoid is deprecated. Use torch.sigmoid instead.
warnings.warn("nn.functional.sigmoid is deprecated. Use torch.sigmoid instead.")
/usr/bin/ffmpeg -r 240.0 -i tmpSuperSloMo/output/%d.jpg -qscale:v 2 output.mp4
ffmpeg version 4.0.3-1~18.04.york0 Copyright (c) 2000-2018 the FFmpeg developers
built with gcc 7 (Ubuntu 7.3.0-27ubuntu1~18.04)
configuration: --prefix=/usr --extra-version='1~18.04.york0' --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared
WARNING: library configuration mismatch
avcodec configuration: --prefix=/usr --extra-version='1~18.04.york0' --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared --enable-version3 --disable-doc --disable-programs --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libtesseract --enable-libvo_amrwbenc
libavutil 56. 14.100 / 56. 14.100
libavcodec 58. 18.100 / 58. 18.100
libavformat 58. 12.100 / 58. 12.100
libavdevice 58. 3.100 / 58. 3.100
libavfilter 7. 16.100 / 7. 16.100
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 1.100 / 5. 1.100
libswresample 3. 1.100 / 3. 1.100
libpostproc 55. 1.100 / 55. 1.100
Input #0, image2, from 'tmpSuperSloMo/output/%d.jpg':
Duration: 00:00:04.80, start: 0.000000, bitrate: N/A
Stream #0:0: Video: mjpeg, yuvj420p(pc, bt470bg/unknown/unknown), 480x270 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 25 tbn, 25 tbc
Stream mapping:
Stream #0:0 -> #0:0 (mjpeg (native) -> h264 (libx264))
Press [q] to stop, [?] for help
[libx264 @ 0x556ca73d8000] -qscale is ignored, -crf is recommended.
[libx264 @ 0x556ca73d8000] using SAR=1/1
[libx264 @ 0x556ca73d8000] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0x556ca73d8000] profile High, level 3.2
[libx264 @ 0x556ca73d8000] 264 - core 155 r2917 0a84d98 - H.264/MPEG-4 AVC codec - Copyleft 2003-2018 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=8 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'output.mp4':
Metadata:
encoder : Lavf58.12.100
Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuvj420p(pc), 480x270 [SAR 1:1 DAR 16:9], q=-1--1, 240 fps, 15360 tbn, 240 tbc
Metadata:
encoder : Lavc58.18.100 libx264
Side data:
cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
[swscaler @ 0x556ca7936280] deprecated pixel format used, make sure you did set range correctly
Past duration 8.999992 too large
Last message repeated 3 times
[swscaler @ 0x556ca7936280] deprecated pixel format used, make sure you did set range correctly
Past duration 8.999992 too large
Last message repeated 3 times
[swscaler @ 0x556ca7936280] deprecated pixel format used, make sure you did set range correctly
Past duration 8.999992 too large
Last message repeated 3 times
[swscaler @ 0x556ca7936280] deprecated pixel format used, make sure you did set range correctly
Past duration 8.999992 too large
Last message repeated 3 times
[swscaler @ 0x556ca7936280] deprecated pixel format used, make sure you did set range correctly
Past duration 8.999992 too large
Last message repeated 3 times
[swscaler @ 0x556ca7936280] deprecated pixel format used, make sure you did set range correctly
Past duration 8.999992 too large
Last message repeated 3 times
[swscaler @ 0x556ca7936280] deprecated pixel format used, make sure you did set range correctly
Past duration 8.999992 too large
Last message repeated 3 times
[swscaler @ 0x556ca7936280] deprecated pixel format used, make sure you did set range correctly
Past duration 8.999992 too large
Last message repeated 3 times
[swscaler @ 0x556ca7936280] deprecated pixel format used, make sure you did set range correctly
Past duration 8.999992 too large
Last message repeated 3 times
[swscaler @ 0x556ca7936280] deprecated pixel format used, make sure you did set range correctly
Past duration 8.999992 too large
Last message repeated 3 times
[swscaler @ 0x556ca7936280] deprecated pixel format used, make sure you did set range correctly
Past duration 8.999992 too large
Last message repeated 3 times
[swscaler @ 0x556ca7936280] deprecated pixel format used, make sure you did set range correctly
Past duration 8.999992 too large
Last message repeated 3 times
[swscaler @ 0x556ca7936280] deprecated pixel format used, make sure you did set range correctly
Past duration 8.999992 too large
Last message repeated 3 times
[swscaler @ 0x556ca7936280] deprecated pixel format used, make sure you did set range correctly
Past duration 8.999992 too large
Last message repeated 3 times
[swscaler @ 0x556ca7936280] deprecated pixel format used, make sure you did set range correctly
Past duration 8.999992 too large
Last message repeated 3 times
[swscaler @ 0x556ca7936280] deprecated pixel format used, make sure you did set range correctly
Past duration 8.999992 too large
Last message repeated 3 times
[swscaler @ 0x556ca7936280] deprecated pixel format used, make sure you did set range correctly
Past duration 8.999992 too large
Last message repeated 3 times
[swscaler @ 0x556ca7936280] deprecated pixel format used, make sure you did set range correctly
Past duration 8.999992 too large
Last message repeated 3 times
[swscaler @ 0x556ca7936280] deprecated pixel format used, make sure you did set range correctly
Past duration 8.999992 too large
Last message repeated 3 times
[swscaler @ 0x556ca7936280] deprecated pixel format used, make sure you did set range correctly
Past duration 8.999992 too large
Last message repeated 3 times
[swscaler @ 0x556ca7936280] deprecated pixel format used, make sure you did set range correctly
Past duration 8.999992 too large
Last message repeated 3 times
[swscaler @ 0x556ca7936280] deprecated pixel format used, make sure you did set range correctly
Past duration 8.999992 too large
Last message repeated 3 times
[swscaler @ 0x556ca7936280] deprecated pixel format used, make sure you did set range correctly
Past duration 8.999992 too large
Last message repeated 3 times
[swscaler @ 0x556ca7936280] deprecated pixel format used, make sure you did set range correctly
Past duration 8.999992 too large
Last message repeated 3 times
[swscaler @ 0x556ca7936280] deprecated pixel format used, make sure you did set range correctly
Past duration 8.999992 too large
Last message repeated 3 times
[swscaler @ 0x556ca7936280] deprecated pixel format used, make sure you did set range correctly
Past duration 8.999992 too large
Last message repeated 3 times
[swscaler @ 0x556ca7936280] deprecated pixel format used, make sure you did set range correctly
Past duration 8.999992 too large
Last message repeated 3 times
[swscaler @ 0x556ca7936280] deprecated pixel format used, make sure you did set range correctly
Past duration 8.999992 too large
Last message repeated 3 times
[swscaler @ 0x556ca7936280] deprecated pixel format used, make sure you did set range correctly
Past duration 8.999992 too large
Last message repeated 3 times
[swscaler @ 0x556ca7936280] deprecated pixel format used, make sure you did set range correctly
Past duration 8.999992 too large
Last message repeated 2 times
frame= 129 fps=0.0 q=-1.0 Lsize= 408kB time=00:00:00.52 bitrate=6368.6kbits/s dup=9 drop=0 speed=0.931x
video:406kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.577492%
[libx264 @ 0x556ca73d8000] frame I:1 Avg QP:27.57 size: 19262
[libx264 @ 0x556ca73d8000] frame P:38 Avg QP:31.32 size: 3687
[libx264 @ 0x556ca73d8000] frame B:90 Avg QP:34.64 size: 2839
[libx264 @ 0x556ca73d8000] consecutive B-frames: 0.8% 18.6% 0.0% 80.6%
[libx264 @ 0x556ca73d8000] mb I I16..4: 0.4% 94.3% 5.3%
[libx264 @ 0x556ca73d8000] mb P I16..4: 0.4% 1.9% 1.4% P16..4: 49.2% 20.6% 11.3% 0.0% 0.0% skip:15.2%
[libx264 @ 0x556ca73d8000] mb B I16..4: 0.8% 3.3% 2.3% B16..8: 18.8% 5.5% 2.1% direct:34.0% skip:33.3% L0:80.0% L1:12.5% BI: 7.5%
[libx264 @ 0x556ca73d8000] 8x8 transform intra:56.9% inter:69.6%
[libx264 @ 0x556ca73d8000] coded y,uvDC,uvAC intra: 80.4% 49.8% 21.2% inter: 46.6% 11.3% 3.2%
[libx264 @ 0x556ca73d8000] i16 v,h,dc,p: 29% 45% 6% 21%
[libx264 @ 0x556ca73d8000] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 20% 19% 20% 6% 5% 7% 6% 7% 9%
[libx264 @ 0x556ca73d8000] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 16% 35% 14% 6% 5% 6% 6% 5% 7%
[libx264 @ 0x556ca73d8000] i8c dc,h,v,p: 75% 15% 9% 2%
[libx264 @ 0x556ca73d8000] Weighted P-Frames: Y:55.3% UV:34.2%
[libx264 @ 0x556ca73d8000] ref P L0: 53.4% 16.6% 3.5% 24.8% 1.6%
[libx264 @ 0x556ca73d8000] ref B L0: 40.2% 59.7% 0.0%
[libx264 @ 0x556ca73d8000] ref B L1: 55.3% 44.7%
[libx264 @ 0x556ca73d8000] kb/s:6175.27
from super-slomo.
avinashpaliwal
commented on July 20, 2024
I am getting the same issue when running on CPU. I am working on it.
from super-slomo.
avinashpaliwal
commented on July 20, 2024
I have added a temporary fix to the script. You can try running it now.
from super-slomo.
ybabs
commented on July 20, 2024
That fixed it thanks.
from super-slomo.
YaoooLiang
commented on July 20, 2024
Hi, @avinashpaliwal @InterestingWalrus
When I run the command below, the interpolation results are all copies of the previous frame.This is also found in other tests of video. Any ideas why this is?
python video_to_slomo.py --video misc/original.gif --sf 2 --checkpoint ./SuperSloMo.ckpt --fps 30 --output videos/out_gif.mp4
from super-slomo.
avinashpaliwal
commented on July 20, 2024
Can you post more details. Like cmd/terminal output of script. Post some example input and output video as well. Just to clarify, you are getting output frames where every interpolated frame is the copy of the original frame before it?
from super-slomo.
YaoooLiang
commented on July 20, 2024
This is my output after running the script:
[root@7a382a96-08ca-454d-b42c-d1a554518421 Super-SloMo]# python video_to_slomo.py --video misc/original.gif --sf 2 --checkpoint ./SuperSloMo.ckpt --fps 30 --output videos/out_gif.mp4
ffmpeg -i misc/original.gif -vsync 0 -qscale:v 2 tmpSuperSloMo/input/%06d.jpg
ffmpeg version 2.6.8 Copyright (c) 2000-2016 the FFmpeg developers
built with gcc 4.8.5 (GCC) 20150623 (Red Hat 4.8.5-4)
configuration: --prefix=/usr --bindir=/usr/bin --datadir=/usr/share/ffmpeg --incdir=/usr/include/ffmpeg --libdir=/usr/lib64 --mandir=/usr/share/man --arch=x86_64 --optflags='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic' --enable-bzlib --disable-crystalhd --enable-gnutls --enable-ladspa --enable-libass --enable-libcdio --enable-libdc1394 --enable-libfaac --enable-nonfree --enable-libfdk-aac --enable-nonfree --disable-indev=jack --enable-libfreetype --enable-libgsm --enable-libmp3lame --enable-openal --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libvorbis --enable-libv4l2 --enable-libx264 --enable-libx265 --enable-libxvid --enable-x11grab --enable-avfilter --enable-avresample --enable-postproc --enable-pthreads --disable-static --enable-shared --enable-gpl --disable-debug --disable-stripping --shlibdir=/usr/lib64 --enable-runtime-cpudetect
libavutil 54. 20.100 / 54. 20.100
libavcodec 56. 26.100 / 56. 26.100
libavformat 56. 25.101 / 56. 25.101
libavdevice 56. 4.100 / 56. 4.100
libavfilter 5. 11.102 / 5. 11.102
libavresample 2. 1. 0 / 2. 1. 0
libswscale 3. 1.101 / 3. 1.101
libswresample 1. 1.100 / 1. 1.100
libpostproc 53. 3.100 / 53. 3.100
Input #0, gif, from 'misc/original.gif':
Duration: N/A, bitrate: N/A
Stream #0:0: Video: gif, bgra, 480x270, 25 fps, 25 tbr, 100 tbn, 100 tbc
[swscaler @ 0x13b97e0] deprecated pixel format used, make sure you did set range correctly
Output #0, image2, to 'tmpSuperSloMo/input/%06d.jpg':
Metadata:
encoder : Lavf56.25.101
Stream #0:0: Video: mjpeg, yuvj444p(pc), 480x270, q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbc
Metadata:
encoder : Lavc56.26.100 mjpeg
Stream mapping:
Stream #0:0 -> #0:0 (gif (native) -> mjpeg (native))
Press [q] to stop, [?] for help
frame= 31 fps=0.0 q=0.0 Lsize=N/A time=00:00:01.24 bitrate=N/A
video:1906kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
/usr/local/lib/python2.7/site-packages/torch/nn/functional.py:1961: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
"See the documentation of nn.Upsample for details.".format(mode))
/usr/local/lib/python2.7/site-packages/torch/nn/functional.py:1006: UserWarning: nn.functional.sigmoid is deprecated. Use torch.sigmoid instead.
warnings.warn("nn.functional.sigmoid is deprecated. Use torch.sigmoid instead.")
ffmpeg -r 30.0 -i tmpSuperSloMo/output/%d.jpg -qscale:v 2 videos/out_gif.mp4
ffmpeg version 2.6.8 Copyright (c) 2000-2016 the FFmpeg developers
built with gcc 4.8.5 (GCC) 20150623 (Red Hat 4.8.5-4)
configuration: --prefix=/usr --bindir=/usr/bin --datadir=/usr/share/ffmpeg --incdir=/usr/include/ffmpeg --libdir=/usr/lib64 --mandir=/usr/share/man --arch=x86_64 --optflags='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic' --enable-bzlib --disable-crystalhd --enable-gnutls --enable-ladspa --enable-libass --enable-libcdio --enable-libdc1394 --enable-libfaac --enable-nonfree --enable-libfdk-aac --enable-nonfree --disable-indev=jack --enable-libfreetype --enable-libgsm --enable-libmp3lame --enable-openal --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libvorbis --enable-libv4l2 --enable-libx264 --enable-libx265 --enable-libxvid --enable-x11grab --enable-avfilter --enable-avresample --enable-postproc --enable-pthreads --disable-static --enable-shared --enable-gpl --disable-debug --disable-stripping --shlibdir=/usr/lib64 --enable-runtime-cpudetect
libavutil 54. 20.100 / 54. 20.100
libavcodec 56. 26.100 / 56. 26.100
libavformat 56. 25.101 / 56. 25.101
libavdevice 56. 4.100 / 56. 4.100
libavfilter 5. 11.102 / 5. 11.102
libavresample 2. 1. 0 / 2. 1. 0
libswscale 3. 1.101 / 3. 1.101
libswresample 1. 1.100 / 1. 1.100
libpostproc 53. 3.100 / 53. 3.100
[mjpeg @ 0x1fe2e20] Changeing bps to 8
Input #0, image2, from 'tmpSuperSloMo/output/%d.jpg':
Duration: 00:00:02.40, start: 0.000000, bitrate: N/A
Stream #0:0: Video: mjpeg, yuvj420p(pc, bt470bg/unknown/unknown), 480x270 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 25 tbn, 25 tbc
File 'videos/out_gif.mp4' already exists. Overwrite ? [y/N] y
No pixel format specified, yuvj420p for H.264 encoding chosen.
Use -pix_fmt yuv420p for compatibility with outdated media players.
[libx264 @ 0x2011300] -qscale is ignored, -crf is recommended.
[libx264 @ 0x2011300] using SAR=1/1
[libx264 @ 0x2011300] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX AVX2 FMA3 LZCNT BMI2
[libx264 @ 0x2011300] profile High, level 2.1
[libx264 @ 0x2011300] 264 - core 142 r2495 6a301b6 - H.264/MPEG-4 AVC codec - Copyleft 2003-2014 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=72 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'videos/out_gif.mp4':
Metadata:
encoder : Lavf56.25.101
Stream #0:0: Video: h264 (libx264) ([33][0][0][0] / 0x0021), yuvj420p(pc), 480x270 [SAR 1:1 DAR 16:9], q=-1--1, 30 fps, 15360 tbn, 30 tbc
Metadata:
encoder : Lavc56.26.100 libx264
Stream mapping:
Stream #0:0 -> #0:0 (mjpeg (native) -> h264 (libx264))
Press [q] to stop, [?] for help
frame= 60 fps=0.0 q=-1.0 Lsize= 322kB time=00:00:01.93 bitrate=1362.6kbits/s
video:320kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.474768%
[libx264 @ 0x2011300] frame I:1 Avg QP:26.12 size: 21812
[libx264 @ 0x2011300] frame P:30 Avg QP:26.46 size: 9479
[libx264 @ 0x2011300] frame B:29 Avg QP:30.68 size: 719
[libx264 @ 0x2011300] consecutive B-frames: 3.3% 96.7% 0.0% 0.0%
[libx264 @ 0x2011300] mb I I16..4: 0.0% 98.0% 2.0%
[libx264 @ 0x2011300] mb P I16..4: 0.0% 3.2% 0.3% P16..4: 31.3% 35.2% 26.2% 0.0% 0.0% skip: 3.7%
[libx264 @ 0x2011300] mb B I16..4: 0.0% 0.0% 0.0% B16..8: 45.5% 5.4% 1.3% direct: 1.3% skip:46.4% L0:59.2% L1: 1.5% BI:39.3%
[libx264 @ 0x2011300] 8x8 transform intra:94.5% inter:78.5%
[libx264 @ 0x2011300] coded y,uvDC,uvAC intra: 97.4% 80.9% 24.7% inter: 38.8% 22.3% 2.5%
[libx264 @ 0x2011300] i16 v,h,dc,p: 0% 100% 0% 0%
[libx264 @ 0x2011300] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 13% 20% 25% 6% 5% 6% 5% 8% 11%
[libx264 @ 0x2011300] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 13% 39% 13% 7% 6% 5% 4% 7% 6%
[libx264 @ 0x2011300] i8c dc,h,v,p: 61% 20% 16% 3%
[libx264 @ 0x2011300] Weighted P-Frames: Y:3.3% UV:0.0%
[libx264 @ 0x2011300] ref P L0: 68.7% 16.1% 12.0% 3.0% 0.1%
[libx264 @ 0x2011300] ref B L0: 99.8% 0.2%
[libx264 @ 0x2011300] kb/s:1308.20
from super-slomo.
YaoooLiang
commented on July 20, 2024
Input video is your original gif ; this is my output of video :out_gif.zip
from super-slomo.
avinashpaliwal
commented on July 20, 2024
I am getting correct results on my end. The video you have posted has no interpolated frames, they look like exact copies of previous frames. I haven't tested the whole thing on linux (I am using windows), so I will try that. In the mean time try running it again with configuration pytorch 0.4.1, python 3.6. Comment out this line in the script and check the hidden folder tmpSuperSloMo for the generated output frames. There will be a resolution mismatch which I am fixing, so ignore that.
from super-slomo.
ybabs
commented on July 20, 2024
@YaoooLiang @avinashpaliwal, if I don't specify the ffmpeg directory, the script doesn't run. Not sure why it's running for you. Also you seem to be using an old version of ffmpeg. Try updating ffmpeg maybe?
from super-slomo.
YaoooLiang
commented on July 20, 2024
@InterestingWalrus @avinashpaliwal
the hidden folder tmpSuperSloMo was existed, so ffmpeg worked. I had checked tmpSuperSloMo, the interpolated frames in output had a very little change from previous frame. the content in tmpSuperSloMo is :
input.zip
output.zip
from super-slomo.
YaoooLiang
commented on July 20, 2024
@YaoooLiang @avinashpaliwal, if I don't specify the ffmpeg directory, the script doesn't run. Not sure why it's running for you. Also you seem to be using an old version of ffmpeg. Try updating ffmpeg maybe?
Because I had changed extract_frames and create_video code , so it was worked for me .
from super-slomo.
avinashpaliwal
commented on July 20, 2024
@YaoooLiang @avinashpaliwal, if I don't specify the ffmpeg directory, the script doesn't run. Not sure why it's running for you. Also you seem to be using an old version of ffmpeg. Try updating ffmpeg maybe?
Like I said, I am using windows. The ffmpeg comes in a zip which you have to extract. That's why you have to provide the path for ffmpeg.exe if it's not in your project folder. I have not tested the project on linux, so I don't know the problems yet. I will work on that, but it will take some time.
from super-slomo.
avinashpaliwal
commented on July 20, 2024
@YaoooLiang @avinashpaliwal, if I don't specify the ffmpeg directory, the script doesn't run. Not sure why it's running for you. Also you seem to be using an old version of ffmpeg. Try updating ffmpeg maybe?
Because I had changed extract_frames and create_video code , so it was worked for me .
I don't know why you are getting such results. I will have to test the project on linux. Also, you said you modified the code, so did you also modify the code in section where frame is interpolated. Post your modified script so I can run it on my end.
from super-slomo.
YaoooLiang
commented on July 20, 2024
@avinashpaliwal
My test results on Windows are exactly the same as yours, but I don't know why the results on Linux are wrong
from super-slomo.
deng-wei
commented on July 20, 2024
Hi, @avinashpaliwal
I have got the same results as @YaoooLiang in Ubuntu16, the interpolation results are copies of the previous frame. Unfortunately, I don't have a Windows machine. Are there any updates about this?
Thank you very much.
from super-slomo.
avinashpaliwal
commented on July 20, 2024
Hi, @avinashpaliwal
I have got the same results as @YaoooLiang in Ubuntu16, the interpolation results are copies of the previous frame. Unfortunately, I don't have a Windows machine. Are there any updates about this?
Thank you very much.
I am currently busy with other tasks and am unable to take out time to test the script on Linux. Try following Godatplay's approach and see if it works. I will post an update about Linux after sometime.
from super-slomo.
Godatplay
commented on July 20, 2024
Specifically I'm on Peppermint 9, which is based on Ubuntu 18.04 LTS, but my guess is that this is an ffmpeg issue. I wanted to maintain the same 25 FPS as the input video, but 4x longer, so I used --sf 4 and --fps 25
from super-slomo.
jwrona
commented on July 20, 2024
Specifically I'm on Peppermint 9, which is based on Ubuntu 18.04 LTS, but my guess is that this is an ffmpeg issue. I wanted to maintain the same 25 FPS as the input video, but 4x longer, so I used
--sf 4and--fps 25
I don't think this is an ffmpeg issue. The input frames in .tmpSuperSlomo/input are fine, but the interpolated output frames in .tmpSuperSlomo/output are already defective.
from super-slomo.
avinashpaliwal
commented on July 20, 2024
Specifically I'm on Peppermint 9, which is based on Ubuntu 18.04 LTS, but my guess is that this is an ffmpeg issue. I wanted to maintain the same 25 FPS as the input video, but 4x longer, so I used
--sf 4and--fps 25I don't think this is an ffmpeg issue. The input frames in
.tmpSuperSlomo/inputare fine, but the interpolated output frames in.tmpSuperSlomo/outputare already defective.
Right. It is mostly likely something to do with the pytorch library on linux. I also used conda, so maybe installing pytorch from some other source is the issue.
from super-slomo.
jwrona
commented on July 20, 2024
I used conda too. I've tried pytorch 0.4.1 and the current version, both CPU, but the results were the same.
from super-slomo.
Godatplay
commented on July 20, 2024
Ah ok. I guess I was using GPU on Linux, so likely a pytorch CPU-specific issue.
from super-slomo.
jwrona
commented on July 20, 2024
Can confirm that this is a pytorch CPU-specific issue. Works fine using current pytorch and CUDA 9.0.
CPU:
CUDA:
from super-slomo.
avinashpaliwal
commented on July 20, 2024
Can confirm that this is a pytorch CPU-specific issue. Works fine using current pytorch and CUDA 9.0.
CPU:
CUDA:
That CPU result sholdn't be like this after the temporary fix. Are you using the latest commit?
from super-slomo.
jwrona
commented on July 20, 2024
That CPU result sholdn't be like this after the temporary fix. Are you using the latest commit?
Yes. I was using commit e8508bd
from super-slomo.
lxlee
commented on July 20, 2024
Hi, @avinashpaliwal
I have got the same results as @YaoooLiang in Ubuntu16, the interpolation results are copies of the previous frame. Unfortunately, I don't have a Windows machine. Are there any updates about this?
Thank you very much.
Were you running with python2 in linux?
from super-slomo.
deng-wei
commented on July 20, 2024
Hi, @avinashpaliwal
I have got the same results as @YaoooLiang in Ubuntu16, the interpolation results are copies of the previous frame. Unfortunately, I don't have a Windows machine. Are there any updates about this?
Thank you very much.Were you running with python2 in linux?
Yes
from super-slomo.
lxlee
commented on July 20, 2024
Hi, @avinashpaliwal
I have got the same results as @YaoooLiang in Ubuntu16, the interpolation results are copies of the previous frame. Unfortunately, I don't have a Windows machine. Are there any updates about this?
Thank you very much.Were you running with python2 in linux?
Yes
I had met the same problem.
The script video_to_slomo.py is not compatibility with python2.
It seems that Using python3 instead as suggested or modifying the script both make sense.
from super-slomo.
arskama
commented on July 20, 2024
Hi,
@avinashpaliwal
The fix for cpu color issues is the following:
diff --git a/video_to_slomo.py b/video_to_slomo.py
index ae021db..5586a12 100755
--- a/video_to_slomo.py
+++ b/video_to_slomo.py
@@ -125,7 +125,7 @@ def main():
# Temporary fix for issue #7 https://github.com/avinashpaliwal/Super-SloMo/issues/7 -
# - Removed per channel mean subtraction for CPU.
- if (device == "cpu"):
- if (device.type == "cpu"):
transform = transforms.Compose([transforms.ToTensor()])
TP = transforms.Compose([transforms.ToPILImage()])
else:
If you debug, you ll see that it falls always in the gpu path, because the if statement is not correct.
Sorry for not making a push request on that one.
from super-slomo.
e-sha
commented on July 20, 2024
Hi, @avinashpaliwal
I have got the same results as @YaoooLiang in Ubuntu16, the interpolation results are copies of the previous frame. Unfortunately, I don't have a Windows machine. Are there any updates about this?
Thank you very much.Were you running with python2 in linux?
Yes
I had met the same problem.
The script video_to_slomo.py is not compatibility with python2.
It seems that Using python3 instead as suggested or modifying the script both make sense.
The problem with python2 was in integer division. It was fixed in one of the pull requests.
from super-slomo.
rsjjdesj
commented on July 20, 2024
Hi,
@avinashpaliwalThe fix for cpu color issues is the following:
diff --git a/video_to_slomo.py b/video_to_slomo.py
index ae021db..5586a12 100755
--- a/video_to_slomo.py
+++ b/video_to_slomo.py
@@ -125,7 +125,7 @@ def main():# Temporary fix for issue #7 https://github.com/avinashpaliwal/Super-SloMo/issues/7 - # - Removed per channel mean subtraction for CPU.
if (device == "cpu"):
if (device.type == "cpu"):
transform = transforms.Compose([transforms.ToTensor()])
TP = transforms.Compose([transforms.ToPILImage()])
else:If you debug, you ll see that it falls always in the gpu path, because the if statement is not correct.
Sorry for not making a push request on that one.
Can this be pushed in the repository. This should fix the bug.
from super-slomo.
Related Issues (20)
- ffmpeg error while running evaluation code. HOT 1
- Output interpolated image frames are of different size then input frames?
- What is the function of transforms.Normalize(mean=negmean, std=std)? Why use a negative mean๏ผ
- Error converting file:D:\Coding\JAVASCRIPT\recless\out\4fpsRecording.mkv. Exiting.
- PackagesNotFoundError: torchvision-cpu==0.2.0 HOT 3
- Runtime Error (Traceback Attached) HOT 23
- Frames are not extracted for any input video
- Differences with butterflow
- Slowdown a point of the video HOT 1
- It does not seem to work for scale value below 4 HOT 1
- "index 1 is out of bounds for dimension 0 with size 1" HOT 2
- Reference frames are different from the original ones. HOT 1
- sh: /Users/*/Downloads/ffmpeg/ffmpeg: No such file or directory HOT 2
- Add local file to Google Colab?? Or change length??
- How can I find the dataset of 240-fps video from YouTube HOT 1
- Training
- Objective metrics
- Could be faster by not saving PNG frames
- Image edge flicker
- low rate of utilization in test video
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from super-slomo.