Cross-platform command-line AV1 / VP9 / HEVC / H264 encoding framework with per scene quality encoding
License: GNU General Public License v3.0
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
please package ffmpeg/aomenc executables for Windows that work with the --vmaf switch. or at least link to them (and where to put the files in relation to av1an.exe/av1an.py)
Hello,
I'm on Ubuntu 18.04 -- I've compiled the SVT-AV1 library, and added the correct path to the encoder (line 309). The encoding hangs at 0%, have you encountered this before?
If there are spaces in the source name the script seems to set the workers to 0, this obviously causes it to error out.
av1an.py -i "file with spaces.mp4" --encoder svt_av1 -e '-w 1920 -h 1080 -fps 29' -fmt yuv420p10le
Clips:0 Workers: 0
Params: -w 1920 -h 1080 -fps 29
|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| 0% 0/0 Traceback (most recent call last):
File "/home/nathanielcwm/Av1an//av1an.py", line 450, in <module>
av1an.main()
File "/home/nathanielcwm/Av1an//av1an.py", line 431, in main
pool = Pool(self.workers)
File "/usr/lib/python3.8/multiprocessing/context.py", line 119, in Pool
return Pool(processes, initializer, initargs, maxtasksperchild,
File "/usr/lib/python3.8/multiprocessing/pool.py", line 205, in __init__
raise ValueError("Number of processes must be at least 1")
ValueError: Number of processes must be at least 1Currently, it appears like Av1an doesn't display FPS values unlike FFmpeg while encoding I believe that it would be quite useful to display it alongside the progress bar.
Maybe it would be great to get some more examples in e.g. a wiki page.
Additionally a comparison speed wise between encoding only with ffmpeg and av1an would be great to have. I would also provide benchmarks if you want.
It would be nice to have ffmpeg video filter options (-vf) supported
This way video can for example be cropped remove black bars.
Thank you for consideration
It would be great if you could add an option to a fast first pass for rav1e and SVT-AV1
Either bump the speed by +n levels, or use a fixed very high speed level
Thank you for consideration
Running into a strange problem. I'm trying to encode big_buck_bunny_1080p_h264.mov, but only about a minute of the file gets encoded.
Command: av1an -i big_buck_bunny_1080p_h264.mov -enc aom -w 12 --passes 1 --logging bbb.av1an.log -v " --cpu-used=6 --end-usage=q --cq-level=40 " -o bbb.av1an.RF40.webm
The scene detection seems to run just fine, it processes 14315/14315 frames. Encoding starts fine as well, Av1an reports "Queue: 80 Workers: 12 Passes: 1". The encode finishes in 649.7s after processing 1534/14315 frames, which is 11% according to the progress bar.
Log file: bbb.av1an.log.log
Sorry, hit [ENTER] key accidentally...
When special characters (like Japanese) are present in input path, this error will be instantaneously reported.
Error in PySceneDetectOpenCV VideoCapture object failed to return True when calling isOpened().
OS: Windows 10 19H2
Release: 1.0.175 from AppVeyor Artifact.
When I'm encoding video sometime one of the av1an process start using more and more memory(16GB+) resulting in a system hang. I didn't find a way to replicate the issue consistently, but it happens fairly often(I think about 50% chance for an encode with 300 scenes). When it happens the av1an process start consuming ram very fast, so I think the reason is that it's getting stuck in a loop that create object or something like that. Last time that it happened I used the following command:
av1an -i in.mkv -enc aom -w 6 -v "--bit-depth=10 --cpu-used=5 --end-usage=q --cq-level=30 --threads=4 " --boost --boost_range 20 --boost_limit 10 -a "-c:a libopus -b:a 96k " --log log -o out.mkv
I ran av1an -i /path/to/myvideo.mp4. It ran through its first progress bar, and then crashed with this message:
Encoding error: invalid literal for int() with base 10: '/…/Av1an/.temp/encode/0.ivf: No such file or directory'
Looking in the log shows some
/bin/sh: 1: aomenc: not found
av_interleaved_write_frame(): Broken pipe
Error writing trailer of pipe:: Broken pipe
Could break this down as several issues:
call_cmd could check the exit code of the subprocess, so we can tell when the subprocess failed instead of passing back its stderr as if it was successful.Support for SVT-VP9 would be great, because libvpx is really slow. Already having support for SVT-AV1, I'd think that's probably relatively easy-ish.
Traceback (most recent call last):
File "./av1an.py", line 512, in <module>
av1an.main()
File "./av1an.py", line 476, in main
self.split_video(self.args.file_path)
File "./av1an.py", line 266, in split_video
stamps = self.read_csv(file_path)
File "./av1an.py", line 237, in read_csv
with open(file_path) as csv_file:
FileNotFoundError: [Errno 2] No such file or directory: '/[...]/av1tests/Av1an/.temp/../test-Scenes.csv'
This is the first run so I don't have any test-Scenes.csv...
I noticed this issue on Manjaro linux when av1an crashes. I had many python processes open from old av1an commands. This definitely happens when av1an crashes and might happen when it exits normally.
To reproduce, try to start an encoding that will crash (like the following command). After the crash, there will still be python processes in a task manager.
python ./av1an.py -i source.mkv -enc aom -ff ' --invalid option ' --split_method aom_keyframes
Can other people reproduce this?
This seems to be fixed by replacing the os.kill in the terminate function with sys.exit(1). Is there a reason that os.kill is used over sys.exit(1)?
My CPU has 6 cores and 12 threads. By default, Av1an defaults to using 6 workers (not sure if this is a bug or intended behavior). EDIT: Probably related to this line. os.cpu_count() returns the number of physical cores, not the number of usable threads.
I use --workers 12 to override the default. Scene detection and first pass use 12 processes, but the second pass only uses 6 workers. Av1an reports this in its CLI output (Queue: 80 Workers: 6 Passes: 2), and I verified this with System Monitor, there are only 6 encoding processes running and about 50% of the CPU is being used.
I'm using the latest git and invoking Av1an with the following command:
python3 ~/Av1an/av1an.py -i big_buck_bunny_1080p_h264.mov --passes 2 --workers 12 -v '--end-usage=q --kf-max-dist=600 --cpu-used=6 --cq-level=42' --output_file bbbtest.mkv
I'm on Linux Mint 19.
On Linux Mint 19 (based on Ubuntu 18.04), av1an can't find the up-to-date version of Python it needs. When trying to execute, it just outputs Python 3.7+ required.
The default version of the python3 binary on Ubuntu 18.04 is Python 3.6.9. According to advice on various StackExchange sites, this default cannot be changed without potentially breaking the system. However, newer versions are available in the repositories can be installed alongside it. Both Python 3.7 and 3.8 are available and can be accessed from the python3.7 and python3.8 binaries. I have Python 3.8 installed, but av1an still exits with the "Python 3.7+ required" error message.
Maybe it's my mistake, but I don't know what I did wrong in the command.
The source is encoded in x265 with 3 streams (video & 2 Audio streams) if that matters.
That's my command:
av1an -i '.\test.mkv' -enc aom -v " --cq-level=35 " -ff " -map 0:0 -map 0:1 -map 0:2 " -a " -c:a copy " -log test.log -o test2.mkv
And the log:
18:55:27 Starting scene detection Threshold: 50
19:12:18 Found scenes: 345
19:12:25 Audio processing
Params: -c:a copy
19:12:47 Encoding Queue Composed
Encoder: AOM Queue Size: 345 Passes: 2
Params: --cq-level=35
19:15:29 Started encoding queue with 8 workers
19:15:50 Enc: 00324.mkv, 4680 fr
19:15:50 Enc: 00154.mkv, 5064 fr
19:15:51 Enc: 00036.mkv, 5016 fr
19:15:53 Enc: 00149.mkv, 5760 fr
19:15:53 Enc: 00180.mkv, 5856 fr
19:16:13 Enc: 00319.mkv, 13584 fr
19:16:13 Enc: 00083.mkv, 10128 fr
19:16:17 Enc: 00344.mkv, 11664 fr
19:16:21 Enc: 00009.mkv, 4368 fr
19:16:29 Enc: 00186.mkv, 4248 fr
19:16:37 Enc: 00003.mkv, 3984 fr
19:16:46 Enc: 00276.mkv, 4248 fr
19:16:49 Enc: 00155.mkv, 4368 fr
19:16:57 Enc: 00325.mkv, 3216 fr
19:17:02 Enc: 00343.mkv, 3312 fr
19:17:04 Enc: 00027.mkv, 3648 fr
19:17:13 Enc: 00273.mkv, 3384 fr
19:17:14 Enc: 00113.mkv, 3096 fr
19:17:14 Enc: 00218.mkv, 3504 fr
19:17:16 Enc: 00139.mkv, 2952 fr
19:17:21 Enc: 00114.mkv, 2352 fr
19:17:24 Enc: 00268.mkv, 2088 fr
19:17:25 Enc: 00322.mkv, 3144 fr
19:17:33 Enc: 00320.mkv, 2112 fr
19:17:35 Enc: 00318.mkv, 2568 fr
19:17:37 Enc: 00338.mkv, 1848 fr
19:17:38 Enc: 00321.mkv, 2040 fr
19:17:40 Enc: 00257.mkv, 1536 fr
19:17:42 Enc: 00292.mkv, 1752 fr
19:17:42 Enc: 00096.mkv, 2040 fr
19:17:50 Enc: 00342.mkv, 1704 fr
19:17:50 Enc: 00143.mkv, 1776 fr
19:17:50 Enc: 00267.mkv, 1392 fr
19:17:51 Enc: 00202.mkv, 1416 fr
19:17:53 Enc: 00250.mkv, 1584 fr
19:17:53 Enc: 00236.mkv, 1248 fr
19:17:54 Enc: 00013.mkv, 1368 fr
19:18:00 Enc: 00253.mkv, 1392 fr
19:18:01 Enc: 00142.mkv, 1368 fr
19:18:01 Enc: 00296.mkv, 888 fr
19:18:01 Enc: 00140.mkv, 912 fr
19:18:02 Enc: 00293.mkv, 840 fr
19:18:03 Enc: 00134.mkv, 1008 fr
19:18:06 Enc: 00311.mkv, 1080 fr
19:18:07 Enc: 00275.mkv, 768 fr
19:18:09 Enc: 00126.mkv, 936 fr
19:18:10 Enc: 00308.mkv, 984 fr
19:18:10 Enc: 00018.mkv, 1032 fr
19:18:10 Enc: 00298.mkv, 864 fr
19:18:12 Enc: 00252.mkv, 960 fr
19:18:13 Enc: 00088.mkv, 792 fr
19:18:15 Enc: 00111.mkv, 864 fr
19:18:16 Enc: 00112.mkv, 840 fr
19:18:16 Enc: 00095.mkv, 744 fr
19:18:17 Enc: 00090.mkv, 744 fr
19:18:19 Enc: 00178.mkv, 768 fr
19:18:19 Enc: 00040.mkv, 768 fr
19:18:20 Enc: 00135.mkv, 648 fr
19:18:21 Enc: 00332.mkv, 816 fr
19:18:23 Enc: 00081.mkv, 552 fr
19:18:23 Enc: 00328.mkv, 480 fr
19:18:23 Enc: 00101.mkv, 576 fr
19:18:24 Enc: 00310.mkv, 696 fr
19:18:24 Enc: 00185.mkv, 480 fr
19:18:25 Enc: 00336.mkv, 456 fr
19:18:25 Enc: 00032.mkv, 624 fr
19:18:26 Enc: 00039.mkv, 504 fr
19:18:28 Enc: 00089.mkv, 504 fr
19:18:28 Enc: 00159.mkv, 384 fr
19:18:28 Enc: 00285.mkv, 432 fr
19:18:29 Enc: 00280.mkv, 480 fr
19:18:29 Enc: 00119.mkv, 576 fr
19:18:31 Enc: 00151.mkv, 384 fr
19:18:31 Enc: 00305.mkv, 360 fr
19:18:31 Enc: 00144.mkv, 432 fr
19:18:32 Enc: 00150.mkv, 360 fr
19:18:32 Enc: 00271.mkv, 336 fr
19:18:34 Enc: 00205.mkv, 360 fr
19:18:34 Enc: 00272.mkv, 384 fr
19:18:34 Enc: 00153.mkv, 552 fr
19:18:35 Enc: 00131.mkv, 336 fr
19:18:35 Enc: 00033.mkv, 360 fr
19:18:36 Enc: 00339.mkv, 432 fr
19:18:37 Enc: 00158.mkv, 336 fr
19:18:37 Enc: 00064.mkv, 456 fr
19:18:38 Enc: 00300.mkv, 360 fr
19:18:38 Enc: 00340.mkv, 408 fr
19:18:38 Enc: 00299.mkv, 456 fr
19:18:39 Enc: 00231.mkv, 360 fr
19:18:40 Enc: 00046.mkv, 288 fr
19:18:40 Enc: 00070.mkv, 360 fr
19:18:40 Enc: 00220.mkv, 408 fr
19:18:41 Enc: 00221.mkv, 360 fr
19:18:42 Enc: 00115.mkv, 288 fr
19:18:42 Enc: 00030.mkv, 264 fr
19:18:42 Enc: 00066.mkv, 288 fr
19:18:43 Enc: 00206.mkv, 288 fr
19:18:43 Enc: 00334.mkv, 240 fr
19:18:44 Enc: 00229.mkv, 312 fr
19:18:44 Enc: 00100.mkv, 312 fr
19:18:44 Enc: 00303.mkv, 240 fr
19:18:45 Enc: 00260.mkv, 264 fr
19:18:45 Enc: 00330.mkv, 288 fr
19:18:45 Enc: 00323.mkv, 240 fr
19:18:46 Enc: 00048.mkv, 288 fr
19:18:46 Enc: 00099.mkv, 264 fr
19:18:47 Enc: 00080.mkv, 216 fr
19:18:47 Enc: 00312.mkv, 336 fr
19:18:47 Enc: 00277.mkv, 240 fr
19:18:48 Enc: 00198.mkv, 312 fr
19:18:49 Enc: 00074.mkv, 264 fr
19:18:49 Enc: 00146.mkv, 288 fr
19:18:49 Enc: 00132.mkv, 216 fr
19:18:50 Enc: 00270.mkv, 216 fr
19:18:50 Enc: 00230.mkv, 264 fr
19:18:50 Enc: 00087.mkv, 312 fr
19:18:51 Enc: 00207.mkv, 336 fr
19:18:52 Enc: 00179.mkv, 192 fr
19:18:52 Enc: 00042.mkv, 240 fr
19:18:52 Enc: 00122.mkv, 264 fr
19:18:52 Enc: 00118.mkv, 216 fr
19:18:53 Enc: 00052.mkv, 264 fr
19:18:53 Enc: 00098.mkv, 216 fr
19:18:53 Enc: 00093.mkv, 192 fr
19:18:54 Enc: 00269.mkv, 168 fr
19:18:55 Enc: 00014.mkv, 264 fr
19:18:55 Enc: 00247.mkv, 192 fr
19:18:55 Enc: 00238.mkv, 216 fr
19:18:55 Enc: 00152.mkv, 240 fr
19:18:56 Enc: 00249.mkv, 192 fr
19:18:56 Enc: 00284.mkv, 240 fr
19:18:56 Enc: 00317.mkv, 216 fr
19:18:56 Enc: 00006.mkv, 192 fr
19:18:57 Enc: 00050.mkv, 192 fr
19:18:57 Enc: 00258.mkv, 240 fr
19:18:57 Enc: 00082.mkv, 168 fr
19:18:57 Enc: 00002.mkv, 216 fr
19:18:58 Enc: 00091.mkv, 144 fr
19:18:58 Enc: 00123.mkv, 240 fr
19:18:59 Enc: 00290.mkv, 192 fr
19:18:59 Enc: 00103.mkv, 144 fr
19:19:00 Enc: 00069.mkv, 144 fr
19:19:00 Enc: 00335.mkv, 144 fr
19:19:00 Enc: 00329.mkv, 168 fr
19:19:00 Enc: 00147.mkv, 216 fr
19:19:01 Enc: 00037.mkv, 144 fr
19:19:01 Enc: 00297.mkv, 168 fr
19:19:01 Enc: 00309.mkv, 144 fr
19:19:01 Enc: 00184.mkv, 144 fr
19:19:02 Enc: 00254.mkv, 168 fr
19:19:02 Enc: 00000.mkv, 168 fr
19:19:02 Enc: 00021.mkv, 168 fr
19:19:02 Enc: 00278.mkv, 168 fr
19:19:03 Enc: 00049.mkv, 168 fr
19:19:03 Enc: 00076.mkv, 192 fr
19:19:04 Enc: 00234.mkv, 144 fr
19:19:04 Enc: 00054.mkv, 144 fr
19:19:05 Enc: 00281.mkv, 144 fr
19:19:05 Enc: 00287.mkv, 168 fr
19:19:05 Enc: 00199.mkv, 168 fr
19:19:05 Enc: 00043.mkv, 168 fr
19:19:06 Enc: 00015.mkv, 216 fr
19:19:06 Enc: 00059.mkv, 168 fr
19:19:06 Enc: 00182.mkv, 120 fr
19:19:06 Enc: 00189.mkv, 144 fr
19:19:07 Enc: 00120.mkv, 168 fr
19:19:07 Enc: 00071.mkv, 120 fr
19:19:07 Enc: 00227.mkv, 192 fr
19:19:07 Enc: 00294.mkv, 120 fr
19:19:08 Enc: 00196.mkv, 144 fr
19:19:08 Enc: 00341.mkv, 120 fr
19:19:08 Enc: 00242.mkv, 144 fr
19:19:08 Enc: 00194.mkv, 120 fr
19:19:09 Enc: 00279.mkv, 120 fr
19:19:09 Enc: 00141.mkv, 144 fr
19:19:09 Enc: 00075.mkv, 168 fr
19:19:09 Enc: 00105.mkv, 96 fr
19:19:10 Enc: 00020.mkv, 120 fr
19:19:10 Enc: 00130.mkv, 96 fr
19:19:10 Enc: 00164.mkv, 96 fr
19:19:10 Enc: 00008.mkv, 120 fr
19:19:11 Enc: 00274.mkv, 144 fr
19:19:11 Enc: 00232.mkv, 120 fr
19:19:11 Enc: 00102.mkv, 96 fr
19:19:11 Enc: 00289.mkv, 120 fr
19:19:12 Enc: 00078.mkv, 144 fr
19:19:12 Enc: 00304.mkv, 192 fr
19:19:12 Enc: 00010.mkv, 120 fr
19:19:12 Enc: 00176.mkv, 96 fr
19:19:13 Enc: 00301.mkv, 96 fr
19:19:13 Enc: 00016.mkv, 144 fr
19:19:13 Enc: 00117.mkv, 96 fr
19:19:13 Enc: 00256.mkv, 120 fr
19:19:14 Enc: 00222.mkv, 144 fr
19:19:14 Enc: 00291.mkv, 96 fr
19:19:14 Enc: 00333.mkv, 96 fr
19:19:14 Enc: 00138.mkv, 120 fr
19:19:15 Enc: 00248.mkv, 96 fr
19:19:15 Enc: 00165.mkv, 96 fr
19:19:15 Enc: 00181.mkv, 96 fr
19:19:15 Enc: 00188.mkv, 96 fr
19:19:16 Enc: 00067.mkv, 120 fr
19:19:16 Enc: 00177.mkv, 96 fr
19:19:16 Enc: 00047.mkv, 96 fr
19:19:16 Enc: 00121.mkv, 120 fr
19:19:17 Enc: 00190.mkv, 96 fr
19:19:17 Enc: 00116.mkv, 120 fr
19:19:17 Enc: 00063.mkv, 72 fr
19:19:17 Enc: 00209.mkv, 144 fr
19:19:18 Enc: 00007.mkv, 72 fr
19:19:18 Enc: 00226.mkv, 120 fr
19:19:18 Enc: 00062.mkv, 120 fr
19:19:18 Enc: 00239.mkv, 96 fr
19:19:19 Enc: 00085.mkv, 96 fr
19:19:19 Enc: 00109.mkv, 72 fr
19:19:19 Enc: 00244.mkv, 72 fr
19:19:19 Enc: 00166.mkv, 72 fr
19:19:20 Enc: 00061.mkv, 72 fr
19:19:20 Enc: 00228.mkv, 96 fr
19:19:20 Enc: 00017.mkv, 120 fr
19:19:20 Enc: 00210.mkv, 168 fr
19:19:20 Enc: 00068.mkv, 72 fr
19:19:20 Enc: 00204.mkv, 72 fr
19:19:21 Enc: 00092.mkv, 72 fr
19:19:21 Enc: 00035.mkv, 72 fr
19:19:22 Enc: 00028.mkv, 96 fr
19:19:22 Enc: 00129.mkv, 72 fr
19:19:22 Enc: 00316.mkv, 96 fr
19:19:22 Enc: 00034.mkv, 120 fr
19:19:22 Enc: 00183.mkv, 72 fr
19:19:22 Enc: 00108.mkv, 72 fr
19:19:23 Enc: 00286.mkv, 72 fr
19:19:23 Enc: 00214.mkv, 120 fr
19:19:23 Enc: 00259.mkv, 72 fr
19:19:23 Enc: 00148.mkv, 72 fr
19:19:23 Enc: 00161.mkv, 72 fr
19:19:24 Enc: 00072.mkv, 72 fr
19:19:24 Enc: 00097.mkv, 72 fr
19:19:24 Enc: 00240.mkv, 72 fr
19:19:24 Enc: 00261.mkv, 72 fr
19:19:24 Enc: 00055.mkv, 72 fr
19:19:25 Enc: 00045.mkv, 72 fr
19:19:25 Enc: 00193.mkv, 96 fr
19:19:25 Enc: 00235.mkv, 72 fr
19:19:25 Enc: 00201.mkv, 72 fr
19:19:26 Enc: 00169.mkv, 48 fr
19:19:26 Enc: 00237.mkv, 72 fr
19:19:26 Enc: 00124.mkv, 72 fr
19:19:26 Enc: 00145.mkv, 72 fr
19:19:27 Enc: 00065.mkv, 48 fr
19:19:27 Enc: 00191.mkv, 72 fr
19:19:27 Enc: 00314.mkv, 72 fr
19:19:27 Enc: 00295.mkv, 48 fr
19:19:27 Enc: 00327.mkv, 48 fr
19:19:27 Enc: 00331.mkv, 72 fr
19:19:28 Enc: 00110.mkv, 48 fr
19:19:28 Enc: 00208.mkv, 120 fr
19:19:28 Enc: 00127.mkv, 48 fr
19:19:28 Enc: 00172.mkv, 48 fr
19:19:28 Enc: 00217.mkv, 96 fr
19:19:29 Enc: 00086.mkv, 72 fr
19:19:29 Enc: 00163.mkv, 72 fr
19:19:29 Enc: 00162.mkv, 48 fr
19:19:29 Enc: 00106.mkv, 48 fr
19:19:29 Enc: 00133.mkv, 48 fr
19:19:30 Enc: 00025.mkv, 48 fr
19:19:30 Enc: 00053.mkv, 48 fr
19:19:30 Enc: 00302.mkv, 48 fr
19:19:30 Enc: 00104.mkv, 48 fr
19:19:30 Enc: 00266.mkv, 72 fr
19:19:31 Enc: 00038.mkv, 48 fr
19:19:31 Enc: 00073.mkv, 72 fr
19:19:31 Enc: 00157.mkv, 48 fr
19:19:31 Enc: 00243.mkv, 48 fr
19:19:31 Enc: 00233.mkv, 48 fr
19:19:32 Enc: 00004.mkv, 72 fr
19:19:32 Enc: 00160.mkv, 48 fr
19:19:32 Enc: 00024.mkv, 48 fr
19:19:32 Enc: 00213.mkv, 96 fr
19:19:32 Enc: 00245.mkv, 48 fr
19:19:32 Enc: 00200.mkv, 72 fr
19:19:33 Enc: 00022.mkv, 48 fr
19:19:33 Enc: 00019.mkv, 48 fr
19:19:33 Enc: 00051.mkv, 48 fr
19:19:33 Enc: 00031.mkv, 48 fr
19:19:33 Enc: 00203.mkv, 48 fr
19:19:34 Enc: 00216.mkv, 72 fr
19:19:34 Enc: 00246.mkv, 48 fr
19:19:34 Enc: 00265.mkv, 48 fr
19:19:34 Enc: 00041.mkv, 48 fr
19:19:34 Enc: 00306.mkv, 48 fr
19:19:34 Enc: 00084.mkv, 48 fr
19:19:35 Enc: 00137.mkv, 48 fr
19:19:35 Enc: 00094.mkv, 48 fr
19:19:35 Enc: 00212.mkv, 72 fr
19:19:35 Enc: 00077.mkv, 48 fr
19:19:35 Enc: 00263.mkv, 48 fr
19:19:36 Enc: 00283.mkv, 48 fr
19:19:36 Enc: 00192.mkv, 48 fr
19:19:36 Enc: 00251.mkv, 48 fr
19:19:36 Enc: 00282.mkv, 48 fr
19:19:37 Enc: 00012.mkv, 48 fr
19:19:37 Enc: 00044.mkv, 48 fr
19:19:37 Enc: 00313.mkv, 48 fr
19:19:37 Enc: 00107.mkv, 24 fr
19:19:37 Enc: 00173.mkv, 24 fr
19:19:37 Enc: 00307.mkv, 24 fr
19:19:37 Enc: 00223.mkv, 48 fr
19:19:38 Enc: 00225.mkv, 48 fr
19:19:38 Enc: 00211.mkv, 72 fr
19:19:38 Enc: 00156.mkv, 24 fr
19:19:38 Enc: 00011.mkv, 48 fr
19:19:38 Enc: 00174.mkv, 24 fr
19:19:39 Enc: 00128.mkv, 24 fr
19:19:39 Enc: 00060.mkv, 24 fr
19:19:39 Enc: 00326.mkv, 24 fr
19:19:39 Enc: 00125.mkv, 48 fr
19:19:39 Enc: 00079.mkv, 24 fr
19:19:39 Enc: 00170.mkv, 24 fr
19:19:40 Enc: 00057.mkv, 48 fr
19:19:40 Enc: 00175.mkv, 24 fr
19:19:40 Enc: 00171.mkv, 24 fr
19:19:40 Enc: 00029.mkv, 24 fr
19:19:40 Enc: 00215.mkv, 48 fr
19:19:40 Enc: 00337.mkv, 24 fr
19:19:41 Enc: 00167.mkv, 24 fr
19:19:41 Enc: 00264.mkv, 24 fr
19:19:41 Enc: 00195.mkv, 24 fr
19:19:41 Enc: 00197.mkv, 24 fr
19:19:41 Enc: 00058.mkv, 24 fr
19:19:41 Enc: 00262.mkv, 24 fr
19:19:42 Enc: 00001.mkv, 24 fr
19:19:42 Enc: 00187.mkv, 24 fr
19:19:42 Enc: 00241.mkv, 24 fr
19:19:42 Enc: 00026.mkv, 24 fr
19:19:42 Enc: 00288.mkv, 24 fr
19:19:42 Enc: 00136.mkv, 24 fr
19:19:43 Enc: 00315.mkv, 24 fr
19:19:43 Enc: 00168.mkv, 24 fr
19:19:43 Enc: 00219.mkv, 24 fr
19:19:43 Enc: 00255.mkv, 24 fr
19:19:43 Enc: 00005.mkv, 48 fr
19:19:43 Enc: 00023.mkv, 24 fr
19:19:43 Enc: 00056.mkv, 24 fr
19:19:44 Enc: 00224.mkv, 24 fr
Well, the spirit of this application is to provide parallel encoding utilising scene detection. But sometimes when dealing a long video with limited CPU cores, equally splitting it by an interval is more efficient than cutting by scenes. Please consider add an option to do this, it also makes this application more versatile. Thank you.
I just tried to install & run Av1an in a fresh Ubuntu 20.04 VM. I installed Av1an via pip, but when trying to run Av1an, I got the following error:
Traceback (most recent call last):
File "/home/ubuntu/.local/bin/av1an", line 5, in <module>
from av1an import main
File "/home/ubuntu/.local/lib/python3.8/site-packages/av1an.py", line 19, in <module>
import cv2
File "/home/ubuntu/.local/lib/python3.8/site-packages/cv2/__init__.py", line 5, in <module>
from .cv2 import *
ImportError: libSM.so.6: cannot open shared object file: No such file or directory
I installed the distro package libsm6 to fix the issue.
Didn't run into the same problem as on Ubuntu 18.04 mentioned in #61, and managed fine without installing the distro-provided python3-opencv.
Hi.
Trying to encode a file using Av1an on Windows, it fails to start.
All prerequisites installed & binaries are in the same folder.
Tried troubleshooting with different ffmpeg versions, placing files & folders on root drive - didn't help, same traceback observed.
C:\av1an>python av1an.py -i C:\col.mkv -o C:\col_av1.mkv -v " --cpu-used=2 --cq-level=18 --end-usage=q "
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████
Traceback (most recent call last):
File "av1an.py", line 22, in <module>
main()
File "av1an.py", line 18, in main
Av1an().main_thread()
File "av1an.py", line 14, in main_thread
main_queue(self.args)
File "C:\av1an\Av1an\encode.py", line 46, in main_queue
encode_file(args)
File "C:\av1an\Av1an\encode.py", line 79, in encode_file
extract_audio(args.input, args.temp, args.audio_params)
File "C:\av1an\Av1an\ffmpeg.py", line 57, in extract_audio
is_audio_here = len(subprocess.run(check, stdout=PIPE, stderr=STDOUT).stdout) == 0
File "C:\Program Files\Python\lib\subprocess.py", line 466, in run
with Popen(*popenargs, **kwargs) as process:
File "C:\Program Files\Python\lib\subprocess.py", line 769, in __init__
restore_signals, start_new_session)
File "C:\Program Files\Python\lib\subprocess.py", line 1113, in _execute_child
args = list2cmdline(args)
File "C:\Program Files\Python\lib\subprocess.py", line 524, in list2cmdline
needquote = (" " in arg) or ("\t" in arg) or not arg
TypeError: argument of type 'WindowsPath' is not iterable
Hi,
I have noticed that some frames vanish in the encoding process.
The source file and the split files had the exact same frame count, as tested with ffmpeg
After the encode a little over 50 frames had vanished, also causing a 2 second audio desync towards the end
In which step do these frames go missing, because aomenc never dropped frames for me, and obviously the splitting did neither.
Could you please examine the situation?
Thank you
Hi
Is it possible to transcode the interlaced source with the frame-doubling deinterlacing? I have the 25i files and I want to transcode them to 50p with -vf bwdif. The filter itself is working good for me when ffmpeg is running directly from the command line but I wasn't able to do the splitting properly with av1an
The files themselves are transcoded but the splitting is wrong: the last piece is too long, so the transcoding is not optimal
I tried --ff " -vf bwdif" -xs 500 but it didnt work as expected
1584fr [08:51, 2.81fr/s]Frame Count Differ for Source 00002.mkv: 438/219
1659fr [09:22, 2.64fr/s]Frame Count Differ for Source 00001.mkv: 438/219
1849fr [11:05, 2.14fr/s]Frame Count Differ for Source 00000.mkv: 466/234
05:28:02 Done: 00002.mkv Fr: 438
Fps: 0.7805 Time: 561.21 sec.
05:28:32 Done: 00001.mkv Fr: 438
Fps: 0.7403 Time: 591.62 sec.
05:30:15 Done: 00000.mkv Fr: 466
Fps: 0.6711 Time: 694.39 sec.
05:45:18 Done: 00003.mkv Fr: 1348
Fps: 0.8441 Time: 1596.93 sec.
Then I added --probe_framerate 50 but did not see the difference
P.S. the above file's duration is ~54 seconds and the keyframes are roughly twice per each second so the keyframes are not the issue, one can split almost anywhere
I just tried to install & run Av1an in a fresh Ubuntu 18.04 VM. I installed Av1an via pip, but when trying to run Av1an, it hard crashed with the following error:
/usr/bin/python3: Relink `/lib/x86_64-linux-gnu/libsystemd.so.0' with `/lib/x86_64-linux-gnu/librt.so.1' for IFUNC symbol `clock_gettime'
/usr/bin/python3: Relink `/lib/x86_64-linux-gnu/libudev.so.1' with `/lib/x86_64-linux-gnu/librt.so.1' for IFUNC symbol `clock_gettime'
Segmentation fault (core dumped)
Installing the distro-provided package python3-opencv fixed the issue.
While by default it copies the audio and subtitles to the output file it doesn't copy the attachments. So subtitles that use a variety of fonts, that aren't installed onto the system, can look terrible due to the file and system lacking the fonts needed to properly display the subtitles.
My pc suddenly turned off because of power cut and when i try to resume it on windows it starts all over again but the process was already 48% done and it took 13hrs to reach that ;-;.
.temp folder along with csv and log file are available in the root directory of av1an with aom and ffmpeg binaries.
Currently, if you do not pass any options while attempting to encode with SVT-AV1 it errors out because the video width and height are not specified.
I think that if it isn't specified it should by default set it to the original height and width of the video.
How would you do distributed encoding when you need to have fixed GOP, and fixed segment-time for HLS/DASH output ?
Splitting the video by I-frame isn't possible. Splitting at exact frame will return wrong results (missing I-frame).
Any idea ?
I'm trying to run the following command:
python av1an.py --input_file input.mkv -enc aom -w 16 -v " --cq-level=30 --end-usage=q --aq-mode=2 " -ff " -colorspace bt2020nc -color_trc smpte2086 -color_primaries bt2020 " -log log.txt --output_file output.mkv
But immediately after the script starts running I get the following error message (after frame 3 of 15992):
19:52:23 Starting scene detection Threshold: 50 19:52:24 Error in PySceneDetect: OpenCV(4.2.0) C:\projects\opencv-python\opencv\modules\core\src\alloc.cpp:73: error: (-4:Insufficient memory) Failed to allocate 24883200 bytes in function 'cv::OutOfMemoryError'
Sometimes it happens after a single frame, sometimes after seven. It seems that the error occurs right as the first worker wants to allocate memory (somewhere in scene_detect it throws the exception).
I'm certain that I'm not out of memory as I still had 25/32GB of RAM left and 24883200 is only around 24MB.
I've installed the requirements according to the requirements.txt, all the needed executables are on the PATH and also in the folder that the python script is in. I'm running Windows 10 in case that makes a difference.
Did I not configure something properly?
Running on Ubuntu 18.04, python 3.8.2. After running the main script, bash commandline does not display input and doesn't create newlines on enter.
Would it be possible to pass in a target file size in megabytes, and have the script calculate the desired bitrate to pass on to the encoder(s)? Thanks!
ISOBMFF (e.g. MP4) output should be supported since the spec for AV1 support is more mature and widely adopted than even for MKV.
https://en.wikipedia.org/wiki/AV1#Supported_container_formats
https://en.wikipedia.org/wiki/ISO/IEC_base_media_file_format
Currently, if we set wrong encoding paramters we need to wait for the finished video splitting before we see an error. If we restart Av1an it starts again with splitting.
Maybe an option to use the already slit files would be a good idea?
I get for (almost?) every segment
Error at encode 'tqdm' object has no attribute 'reset'
At line 153
And in the end it fails to concatenate.
Ref file: https://download.blender.org/demo/movies/BBB/bbb_sunflower_1080p_60fps_normal.mp4
Commandline: av1an -i "bbb_sunflower_1080p_60fps_normal.mp4" -enc vpx
OS: Ubuntu 20.04
Python: Python 3.8.2
Installed: Via pip
vp9: WebM Project VP9 Encoder v1.8.2
Looks like this actually requires cython but that's not mentioned in the docs anywhere :)
c:>python ./av1an.py -i c:/Film/1.mkv -o c:/Film/2.mkv
Traceback (most recent call last):
File "./av1an.py", line 286, in
main()
File "./av1an.py", line 279, in main
Av1an().main_thread()
File "./av1an.py", line 271, in main_thread
startup_check()
NameError: name 'startup_check' is not defined
It should be clarified that PIP means PIP3, as in distributions using both python2 and python3, PIP will install stuff for python2 (and av1an won't be able to find what it needs).
I'm pretty new to this, but I've checked all the install requirements etc and I'm still getting this error when trying to encode.
C:\Av1an>av1an.py -i Sintel_1080p_YUV_RAW.avi -enc svt_av1 -e "-w 1920 -h 818 -fps 24" 'ffprobe' is not recognized as an internal or external command, operable program or batch file. 'scenedetect' is not recognized as an internal or external command, operable program or batch file. Video Sintel_1080p_YUV_RAW.avi splitted Splited videos: 0 Starting encoding with 0 workers. Parameters: -w 1920 -h 818 -fps 24 Encoding.. |---------------------------------------------------------------------------------------------------------| 0% 0/0 Traceback (most recent call last): File "C:\Av1an\av1an.py", line 406, in <module> av1an.main() File "C:\Av1an\av1an.py", line 393, in main pool = Pool(self.workers) File "C:\Users\jackc\AppData\Local\Programs\Python\Python38-32\lib\multiprocessing\context.py", line 119, in Pool return Pool(processes, initializer, initargs, maxtasksperchild, File "C:\Users\jackc\AppData\Local\Programs\Python\Python38-32\lib\multiprocessing\pool.py", line 205, in __init__ raise ValueError("Number of processes must be at least 1") ValueError: Number of processes must be at least 1
I have installed scenedetect and ffmpeg successfully so not sure what is causing this.
A bug or my error?
It seems that having a space in the path breaks the script even when path is wrapped with " ". See error:
D:\Downloads\x!downloaders\Av1an>av1an -i "D:\Downloads\x!downloaders\Av1an\test - Copy\test2\F0144M-F2.wmv"
100%|████████████████████████████████████████████████████████████████████████▊| 5649/5661 [00:08<00:00, 642.79frames/s]
Argument '-' provided as input filename, but 'D:\Downloads\x!downloaders\Av1an\test' was already specified.
Clips: 0 Workers: 0
Params: --cpu-used=6 --end-usage=q --cq-level=40
|---------------------------------------------------------------------------------------------------------| 0% 0/0 Traceback (most recent call last):
File "av1an.py", line 512, in
av1an.main()
File "av1an.py", line 490, in main
File "multiprocessing\context.py", line 119, in Pool
File "multiprocessing\pool.py", line 205, in init
ValueError: Number of processes must be at least 1
[46688] Failed to execute script av1an
See log below. Aomenc build from this guy: https://www.reddit.com/r/AV1/comments/f17649/is_there_a_download_for_aomenc_built_with_vmaf/fh2cz0x/
av1an.exe -i sample.mkv -v " --cpu-used=5 --end-usage=q --cq-level=40 --tune=vmaf"
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1416/1416 [00:05<00:00, 250.08frames/s]
Queue: 8 Workers: 6 Passes: 2
Params: --cpu-used=5 --end-usage=q --cq-level=40 --tune=vmaf
0%| | 0/1416 [00:00<?, ?fr/s]Frame Count Differ for Source 0004.mkv: 0/88
6%|███████▉ | 88/1416 [00:09<02:25, 9.14fr/s]Frame Count Differ for Source 0006.mkv: 0/142
16%|████████████████████▍ | 230/1416 [00:13<01:41, 11.68fr/s]Frame Count Differ for Source 0007.mkv: 0/132
26%|████████████████████████████████▏ | 362/1416 [00:16<01:09, 15.07fr/s]Frame Count Differ for Source 0001.mkv: 0/146
Frame Count Differ for Source 0003.mkv: 0/108
Frame Count Differ for Source 0005.mkv: 0/46
47%|██████████████████████████████████████████████████████████▉ | 662/1416 [00:20<00:21, 35.79fr/s]Frame Count Differ for Source 0000.mkv: 0/295
68%|█████████████████████████████████████████████████████████████████████████████████████▏ | 957/1416 [00:30<00:13, 32.93fr/s]Frame Count Differ for Source 0002.mkv: 0/459
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1416/1416 [00:43<00:00, 33.70fr/s]Concatenation failed
Looks like freetype is required, but this is not mentioned in the readme :)
I'm trying a very simple execution to get familiar with this tool, but it fails to produce a valid output. This is on Windows, running in PowerShell, using Python 3.8.2.
python C:\av1an\av1an.py -i .\ToS-4k-1920.mov -s scenes.csv -o defaults.mkv
100%|███████████████████████████████████████████████████████████████████████| 17620/17620 [01:43<00:00, 170.12frames/s]
Queue: 119 Workers: 8 Passes: 2
Params: --threads=4 --cpu-used=6 --end-usage=q --cq-level=40
0%| | 0/17620 [00:00<?, ?fr/s]Encoding failed, check validity of your encoding settings/commands and start again
Encoding failed, check validity of your encoding settings/commands and start again
Encoding failed, check validity of your encoding settings/commands and start again
Encoding failed, check validity of your encoding settings/commands and start again
Encoding failed, check validity of your encoding settings/commands and start again
3%|██ | 466/17620 [02:41<7:22:17, 1.55s/fr]
After blowing up like this, it does continue to encode (I have 8 aomenc processes running).
What info can I provide to help investigate these errors?
Note that -t = 10 and Workers = 2
CPU: 6 core 12 thread 3600x
av1an.exe -i sample-cut.mkv -t 10
100%|██████████████████████████████████████████████████████████████████████████▊| 308/309 [00:01<00:00, 253.62frames/s]
Clips: 2 Workers: 2
Params: --cpu-used=6 --end-usage=q --cq-level=40
|█████████████████████████████████████████████████████████████████████████████████████████████████████████| 100.0% 2/2
Completed in 320.7 seconds
2 pass encoding always fails after a certain amount of percentage or time when using --rt flag in aomenc video config but it works when i set -p 1 flag. i guess its a piping issue or --rt doesn't support 2 pass encoding yet?
i fetch issues on all encoder
After trying to execute ./av1an.py -i [filename] it spits out a syntax error.
File "./av1an.py", line 22 self.bar_iteration: int = 0 ^ SyntaxError: invalid syntax
Hello! --
I regularly see your posts on r/AV1 about this tool, and when I noticed that it supported AVIF, I was excited (as I am the author of libavif), but based on the implementation, it appears you are actually supporting IVF, not AVIF.
I'm not pushing for AVIF support or anything, I just want to note the distinction between IVF and AVIF, and that you accurately cite which one you're actually outputting (or not). Cheers!
If the directory .temp/split exists but is empty it errors out.
Maybe it should offer the user the option to delete the existing .temp folder?
As it is currently, the encoding speed fps and ETA can update very rapidly and vary wildly. At least for me, when using 12 workers, the ETA estimate isn't very useful because of this, as it fluctuates quickly between various values between 7 and 24 hours. It also updates at an inconsistent rate, somtimes multiple times a second, sometimes less than once a second.
I really have two suggestions:
The scene detection is relatively fast, but can still take quite a bit of time when you're encoding a movie. Looking at the CPU usage, the scene detection seems to be using only a single thread. Would it be possible to use multiple threads for this?
It also occured to me that having the option to downscale the source for the purpose of scene detection might speed things up.
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.