libvips / vips-bench

a simple image processing benchmark implemented in a range of image processing packages

libvips: an image processing library

Introduction

libvips is a demand-driven, horizontally threaded image processing library. Compared to similar libraries, libvips runs quickly and uses little memory. libvips is licensed under the LGPL 2.1+.

It has around 300 operations covering arithmetic, histograms, convolution, morphological operations, frequency filtering, colour, resampling, statistics and others. It supports a large range of numeric types, from 8-bit int to 128-bit complex. Images can have any number of bands. It supports a good range of image formats, including JPEG, JPEG2000, JPEG-XL, TIFF, PNG, WebP, HEIC, AVIF, FITS, Matlab, OpenEXR, PDF, SVG, HDR, PPM / PGM / PFM, CSV, GIF, Analyze, NIfTI, DeepZoom, and OpenSlide. It can also load images via ImageMagick or GraphicsMagick, letting it work with formats like DICOM.

It comes with bindings for C, C++, and the command-line. Full bindings are available for :

Language	Binding
Ruby	ruby-vips
Python	pyvips
PHP	php-vips
C# / .NET	NetVips
Go	govips
Lua	lua-vips
Crystal	crystal-vips
Elixir	vix

libvips is used as an image processing engine by:

sharp (on node.js)
imgproxy
bimg
sharp for Go
Ruby on Rails
carrierwave-vips
mediawiki
PhotoFlow

and others. The official libvips GUI is nip2, a strange combination of a spreadsheet and a photo editor.

Install

There are packages for most Unix-like operating systems, including macOS. Check your package manager.

There are binaries for Windows in releases.

The libvips website has detailed install notes.

Building from source

libvips uses the Meson build system, version 0.56 or later. Meson can use ninja, Visual Studio or XCode as a backend, so you'll also need one of them.

libvips must have build-essential, pkg-config, libglib2.0-dev, libexpat1-dev. See the Dependencies section below for a full list of the libvips optional dependencies.

There are basic bash completions in completions/, see the README in there.

Cheatsheet

cd libvips-x.y.x
meson setup build --prefix /my/install/prefix
cd build
meson compile
meson test
meson install

Check the output of meson setup carefully and make sure it found everything you wanted it to find. Add arguments to meson setup to change the build configuration.

• Add flags like -Dnsgif=false to turn libvips options on and off, see meson_options.txt for a list of all the build options libvips supports.

• Add flags like -Dmagick=disabled to turn libvips dependencies on and off, see meson_options.txt and the list below for a summary of all the libvips dependencies.

• You might need to add --libdir lib on Debian if you don't want the arch name in the library path.

• Add --default-library static for a static build.

• Use eg. CC=clang CXX=clang++ meson setup ... to change compiler.

• You can have many build-dir, pick whatever names you like, for example one for release and one for debug.

There's a more comprehensive test suite you can run once libvips has been installed. Use pytest in the libvips base directory.

Optional dependencies

If suitable versions are found, libvips will add support for the following libraries automatically. Packages are generally found with pkg-config, so make sure that is working.

libjpeg

Anything that is compatible with the IJG JPEG library. Use mozjpeg if you can. Another option is libjpeg-turbo.

libexif

If available, libvips adds support for EXIF metadata in JPEG files.

librsvg

The usual SVG loader. If this is not present, vips will try to load SVGs via imagemagick instead.

PDFium

If present, libvips will attempt to load PDFs with PDFium. Download the prebuilt pdfium binary from:

https://github.com/bblanchon/pdfium-binaries

Untar to the libvips install prefix, for example:

cd ~/vips
tar xf ~/pdfium-linux.tgz

Create a pdfium.pc like this (update the version number):

VIPSHOME=/home/john/vips
cat > $VIPSHOME/lib/pkgconfig/pdfium.pc << EOF
     prefix=$VIPSHOME
     exec_prefix=\${prefix}
     libdir=\${exec_prefix}/lib
     includedir=\${prefix}/include
     Name: pdfium
     Description: pdfium
     Version: 4290
     Requires:
     Libs: -L\${libdir} -lpdfium
     Cflags: -I\${includedir}
EOF

If PDFium is not detected, libvips will look for poppler-glib instead.

poppler-glib

The Poppler PDF renderer, with a glib API. If this is not present, vips will try to load PDFs via imagemagick.

cgif

If available, libvips will save GIFs with cgif. If this is not present, vips will try to save gifs via imagemagick instead.

libarchive

If available, libvips adds support for creating image pyramids with dzsave.

libtiff

The TIFF library. It needs to be built with support for JPEG and ZIP compression. 3.4b037 and later are known to be OK.

fftw3

If libvips finds this library, it uses it for fourier transforms.

lcms2

If present, vips_icc_import(), vips_icc_export() and vips_icc_transform() can be used to manipulate images with ICC profiles.

libspng

If present, libvips will load and save PNG files using libspng. If not, it will look for the standard libpng package.

libimagequant, quantizr

If one of these quantisation packages is present, libvips can write 8-bit palette-ised PNGs and GIFs.

ImageMagick, or optionally GraphicsMagick

If available, libvips adds support for loading and saving all libMagick-supported image file types. You can enable and disable load and save separately.

Imagemagick 6.9+ needs to have been built with --with-modules. Most packaged IMs are, I think.

If you are going to be using libvips with untrusted images, perhaps in a web server, for example, you should consider the security implications of enabling a package with such a large attack surface.

pangocairo

If available, libvips adds support for text rendering. You need the package pangocairo in pkg-config --list-all.

highway

If present, libvips will accelerate some operations with SIMD. If not, it will look for the orc-0.4 package.

matio

If available, vips can load images from Matlab save files.

cfitsio

If available, vips can load FITS images.

libwebp

If available, vips can load and save WebP images.

libniftiio

If available, vips can load and save NIfTI images.

OpenEXR

If available, libvips will directly read (but not write, sadly) OpenEXR images.

OpenJPEG

If available, libvips will read and write JPEG2000 images.

libjxl

If available, libvips will read and write JPEG-XL images.

OpenSlide

If available, libvips can load OpenSlide-supported virtual slide files: Aperio, Hamamatsu, Leica, MIRAX, Sakura, Trestle, and Ventana.

libheif

If available, libvips can load and save HEIC and AVIF images. Your libheif (in turn) needs to be built with the correct decoders and encoders. You can check with eg.:

$ heif-convert --list-decoders
HEIC decoders:
- libde265 = libde265 HEVC decoder, version 1.0.9
AVIF decoders:
- dav1d = dav1d v6.6.0
- aom = AOMedia Project AV1 Decoder v3.5.0
$ heif-enc --list-encoders
HEIC encoders:
- x265 = x265 HEVC encoder (3.5+1-f0c1022b6) [default]
AVIF encoders:
- aom = AOMedia Project AV1 Encoder v3.5.0 [default]
- svt = SVT-AV1 encoder v1.1.0
- rav1e = Rav1e encoder

Contributors

Code Contributors

This project exists thanks to all the people who contribute.

Organizations

We've had generous financial support from our sponsors. Thank you very much!

vips-bench's Issues

Convolution resampling and other libraries

There is a comment in pillow.py:

starting with 2.7, Pillow uses a high-quality convolution-based resize for
BILINEAR ... the other systems in this benchmark are using affine + bilinear,
so this is rather unfair. Use NEAREST instead, it gets closest to what
everyone else is doing

I'm pretty sure that there are other libraries which do convolution resize. For example, ImageMagick and its successor GraphicsMagick. Maybe others.

Also, sips and libgd do supersampling resampling which quality is much closer to Bicubic convolution and complexity close to Bilinear convolution.

Using nearest neighbor only in Pillow unfair to other libraries. While resizing with 4-point bilinear interpolation (like in VIPS) is not suitable for scale factors < 0.5, for 0.9 (used in the test), it much closer (both quality and complexity) to Bilinear convolution rather than to nearest neighbor.

Rather than using nearest neighbor for all of this libraries, I believe it's better to use Bilinear for Pillow and add some note for VIPS, OpenCV and possibly others.

How to check

It's simple. Try to resize with scale factor 0.1. If a zoomed result will look like this, it's convolution or supersampling.

If it looks like this, it's 4-point bilinear.

Also compare against ExactImage

Any chance to add ExactImage to the comparison?

Convolution of a large mask

Hi,

I am trying to convolve a binary mask with a tiff WSI to blackout certain regions for an experiment. However, I am not able to solve it with pyvips.conv operator.

Masks are gray-scale in png format (1BPP) and 100 times smaller than the original WSI. I am trying to load them as vips object and resizing them to the same size of WSI.

Here is the code:
wsi = vips.Image.tiffload(os.path.join(path_to_dataset, name))
mask = vips.Image.pngload(mask_path).resize(100, vscale=100)
new_wsi = wsi.conv(mask)

Here is the error:
pyvips.error.Error: unable to call conv
conv: matrix image too large

Any suggestions on this?

try pike.image

http://pike.lysator.liu.se/generated/manual/modref/ex/predef_3A_3A/Image/Image/apply_matrix.html

Update results for Pillow-SIMD 4.3

Hi!

Pillow-SIMD 4.3.0 with significant performance improvements is out. Some of improvements are releted to this benchamark.

I modified the test script a bit just to make clear what is happening.

Full test script

#!/usr/bin/env python
from __future__ import print_function

import sys
from PIL import Image, ImageFilter, PILLOW_VERSION
import time

class Timer:
    def __init__(self, name):
        self.name = name

    def __enter__(self):
        self.start = time.time()
        self.cpu = time.clock()
        return self

    def __exit__(self, *args):
        print('>>>', self.name, time.time() - self.start)


# just to confirm we are getting the right version
print("pillow.py: PILLOW_VERSION =", PILLOW_VERSION)

def run():
    with Timer('open'):
        im = Image.open(sys.argv[1])
        im.load()

    with Timer('crop'):
        # Crop 100 pixels off all edges.
        im = im.crop((100, 100, im.size[0] - 100, im.size[1] - 100))

    with Timer('resize'):
        # Shrink by 10%
        im = im.resize((int(im.width * 0.9), int(im.height * 0.9)), Image.NEAREST)

    with Timer('apply filter'):
        # sharpen
        filter = ImageFilter.Kernel((3, 3),
                  (-1, -1, -1,
                   -1, 16, -1,
                   -1, -1, -1))
        im = im.filter(filter)

    with Timer('save'):
        # write back again
        # im.save(sys.argv[2], format='tiff')
        # I don't have pc with SSD for tests, write to memory
        im.tobytes()


with Timer('sum'):
    run()

The difference on my pc (only for example, of course):

$ pip install pillow-simd==4.2.1.post0
Successfully installed pillow-simd-4.2.1.post0

$ time ./pillow.py ./vips.tiff
pillow.py: PILLOW_VERSION = 4.2.1.post0
>>> open 0.106125116348
>>> crop 0.0340759754181
>>> resize 0.027755022049
>>> apply filter 0.59959602356
>>> save 0.0295100212097
>>> sum 0.803848028183

real	0m0.819s
user	0m0.748s
sys	0m0.068s

$ pip install pillow-simd==4.3.0.post0
Successfully installed pillow-simd-4.3.0.post0

$ time ./pillow.py ./vips.tiff 
pillow.py: PILLOW_VERSION = 4.3.0.post0
>>> open 0.0663890838623
>>> crop 0.0178101062775
>>> resize 0.0208790302277
>>> apply filter 0.0534710884094
>>> save 0.0313007831573
>>> sum 0.190583229065

real	0m0.223s
user	0m0.176s
sys	0m0.044s

Aggressive optimizations

Just for example of maximum performance. There are three tweaks and each of them can be used separately:

1. Avoid cropping and resize with box
2. Compile with AVX2
3. Use Pillow's memory pool and custom allocator

Difference in pillow_box_crop.py is:

    with Timer('crop'):
        # Crop 100 pixels off all edges.
        box = (100, 100, im.width - 100, im.height - 100)
        width, height = (im.width - 200, im.height - 200)

    with Timer('resize'):
        # Shrink by 10%
        im = im.resize((int(width * 0.9), int(height * 0.9)), Image.NEAREST, box=box)

# install and recompile with avx2
$ CFLAGS="-mavx2" pip install pillow-simd==4.3.0.post0 --no-cache-dir

# use custom memory allocator and tune memory settings
$ time LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1 PILLOW_BLOCK_SIZE=4m PILLOW_BLOCKS_MAX=64 ./pillow_box_crop.py ./vips.tiff
pillow.py: PILLOW_VERSION = 4.3.0.post0
>>> open 0.0655601024628
>>> crop 7.15255737305e-06
>>> resize 0.0184090137482
>>> apply filter 0.035365819931
>>> save 0.0275781154633
>>> sum 0.147360086441

real	0m0.182s
user	0m0.124s
sys	0m0.052s

Possible! Benchmark! Addition! Of! Yahoo! Ymagine!

File this under VIPS "evangelism"...

As well as a closed-source GPU pipeline, Flickr also use their own BitmapFactory-inspired Ymagine open source library.

When alerted to VIPS, specifically the Speed and Memory Use page, Flickr's Head of Engineering @tedd4u said he "would be curious where [Ymagine] fits into that benchmark list ;)".

Add CPU time to the results table

On the benchmark results test page there is a "Run time (secs real)" column. The time utility shows not only the real time but also user and sys time. Sum of user and sys time is total CPU time spent to the task. You can add this value to the table. I believe this could be useful for many users.