animethemes / animethemes-webm-verifier

Verify WebM(s) Against AnimeThemes Encoding Standards

Description

Verify WebM(s) Against AnimeThemes Encoding Standards

Executes a test suite on the input WebM(s) to verify compliance.

Test success/failure does NOT guarantee acceptance/rejection of submissions. In some tests, we are determining the correctness of our file properties. In other tests, we are flagging uncommon property values for inspection.

Install

Requirements:

• FFmpeg
• Python >= 3.6

Install:

pip install animethemes-webm-verifier

Usage

test_webm [-h] [--loglevel [{debug,info,error}]] [--groups [{format,video,audio} ...]] [file ...]

File

The WebM(s) to verify. If not provided, we will test all WebMs in the current directory.

Groups

The groups of tests that should be run.

The format group pertains to testing of the file format and context of streams.

The video group pertains to testing of the video stream of the file.

The audio group pertains to testing of the audio stream of the file.

By default, all test groups will be included.

Logging

Determines the level of the logging for the program.

--loglevel error will only output error messages.

--loglevel info will output error messages and script progression info messages.

--loglevel debug will output all messages, including variable dumps.

animethemes-webm-verifier's Issues

Distinguish warnings and errors in verification

Currently the verifier runs a suite of unit tests against the set of WebMs and reports success and failures based on assertion exceptions being raised.

However, there is an intended variance of severity level that is missed with this design. For some tests, we want to fail the test. For example, if the video codec is not VP9, this is an outright violation of the encoding standards. However, there are some tests that are intended as warnings, flagging a file property or format meant to be scrutinized further.

I believe we should adopt a new design that does not make use of unit tests and instead collects a list of warnings and errors using the same test criteria. This list is then outputted to the user upon completion.

Test position of first video frame

If a webm is trimmed or stream copied, it is possible that a video frame will not be defined until the first keyframe is reached and the resulting playback displays the first keyframe until its position is reached in playback.

Example: https://files.catbox.moe/j6q0g1.webm

ffprobe command used in initial scan:

ffprobe -show_frames -select_streams v -read_intervals %+#1 -print_format json input.webm > frames.json

False positives are a possibility here for sequences with black frames at the start of the cut. We should pick a best estimate reference position and warn the user if the first position is past this post.

Suggestion: 0.250s.

Verify position of first video frame

It is possible for a WebM to pass verification but contain a trimming defect that causes video frame data to be dropped. The effect is a "freezing" of the first frame from the start position to the first valid video frame while the audio plays back successfully.

Example: https://i.fiery.me/H6Nlc.webm

We should video read frame data, or at least retrieve the first video frame, in order to detect these issues. It is possible that we will have false positives depending on the position that we choose to flag this, so this test should be documented as such.

Include Video Assessment Scores

It would be nice to include VMAF / PSNR / SSIM scores to give some standard assessment of video quality.