popkartyeah / ffmpeg-patch

This repository contains a collection of FFmpeg* patches and samples to enable CNN model based video analytics capabilities (such as object detection, classification, recognition) in FFmpeg* framework.

FFmpeg Video Analytics (FFVA) Plugin

Overview

This repository contains FFmpeg patches that enable CNN model-based video analytics capabilities in the FFmpeg framework. These patches contain filter based implementation include such things as object detection, classification, and recognition. Following is an example that runs detection & emotion classification, using specific models on a video file:

ffmpeg -i test.mp4 -vf detect=model=face-detection-adas-0001.xml:model_proc=face-detection-adas-0001.json,classify=model=emotions-recognition-retail-0003.xml:model_proc=emotions-recognition-retail-0003.json -an -f null -

The solution leverages:

• Open-source FFmpeg framework for pipeline management
• FFmpeg components for input and output, such as media files and real-time streaming from a camera or network
• Video decode and encode plugins, including either CPU-optimized plugins or GPU-accelerated plugins, based on VAAPI

In addition, the solution installs the following Deep Learning-specific elements, also available in this repository:

• Inference plugins leveraging OpenVINO^™ Toolkit for high-performance inference using CNN models
• Visualization of computer vision results (such as bounding boxes and labels of detected objects) on top of video stream

License

The FFmpeg Video Analytics Patches are licensed under the GNU Lesser General Public License (LGPL) version 2.1 or later.

Prerequisites

Hardware

• OpenVINO^™ Toolkit has information about the hardware requirements for inference elements
• On platforms with Intel Gen graphics, see the ffmpeg vaapi wiki for Hardware Support

Software

• OpenVINO^™ Toolkit 2019 R3 (Inference Engine 2.1.0) or above
• Linux* system with kernel 4.15 or above
• FFmpeg framework version n4.2

Getting Started

• Start here: Getting Started Guide
• API reference
• OpenVisualCloud/Dockerfiles wiki

Samples

See the command-line examples

Reporting Bugs and Feature Requests

Report bugs and requests on the issues page

How to contribute

Pull requests aren't monitored, so if you have bug fix or an idea to improve this project, post a description on the issues page

---

* Other names and brands may be claimed as the property of others.