hxl1990 / scanner

Efficient video analysis at scale

Scanner: Efficient Video Analysis at Scale

To try out Scanner, check out our Quick start and browse our Wiki.

Scanner is like Spark for videos. It runs stateful functions across video frames using clusters of machines with CPUs and GPUs. For example, you could use Scanner to:

• Locate and recognize faces in a video
• Detect shots in a film
• Search videos by image

Click here to learn more about the design and usage of Scanner.

Scanner provides a Python API to organize your videos and run high-performance functions written in C++. For example, this program computes a histogram of colors for each frame in a set of videos on the GPU:

from scannerpy import Database, DeviceType, Job
from scannerpy.stdlib import parsers

with Database() as db:
    videos = db.ingest_video_collection('my_videos', ['vid0.mp4', 'vid1.mkv'])
    frame, frame_info = videos.as_op().all()
    histograms = db.ops.Histogram(frame = frame, frame_info = frame_info, device=DeviceType.GPU)
    job = Job(columns = [histograms], name = 'my_videos_hist')
    output = db.run(job)
    vid0_hists = output.tables(0).load(['histogram'], parsers.histograms)

Click here to see more code examples of using Scanner.

Scanner makes it easy to use existing computer vision and pixel processing tools. For example, Scanner supports deep neural networks with Caffe, image processing with OpenCV and Halide, and object tracking with Struck.

Scanner is an active research project, part of a collaboration between Carnegie Mellon and Stanford. Please contact Alex Poms and Will Crichton with questions.

Quick start

To quickly dive into Scanner, you can use one of our prebuilt Docker images. To run a GPU image, you must install and use nvidia-docker.

nvidia-docker run -d --name scanner -ti scannerresearch/scanner:gpu /bin/bash
nvidia-docker attach scanner

Note: if you don't have a GPU, then run docker instead of nvidia-docker and use scanner:cpu instead of scanner:gpu in the Docker image name.

Then inside your Docker container, run:

python examples/face_detection/face_detect.py

This runs a Scanner demo which detects faces in every frame of a short video from YouTube, creating a file example_faces.mp4. Type Ctrl-P + Ctrl-Q to detach from the container and then run:

nvidia-docker cp scanner:/opt/scanner/example_faces.mp4 .

Then you can view the generated video on your own machine. That's it!

Learning Scanner

To get started building your own applications with Scanner, check out:

• Build instructions
• Tutorials
• Documentation