This is a (Cython-based) Python wrapper for Philipp Krähenbühl's Fully-Connected CRFs (version 2).

If you use this code for your reasearch, please cite:

Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
Philipp Krähenbühl and Vladlen Koltun
NIPS 2011

and provide a link to this repository as a footnote or a citation.

You can install this using pip by executing:

pip install git+https://github.com/lucasb-eyer/pydensecrf.git

and ignoring all the warnings coming from Eigen.

Note that you need a relatively recent version of Cython (at least version 0.22) for this wrapper, the one shipped with Ubuntu 14.04 is too old. (Thanks to Scott Wehrwein for pointing this out.) I suggest you use a virtual environment and install the newest version of Cython there (pip install cython), but you may update the system version by

sudo apt-get remove cython
sudo pip install -U cython

For images, the easiest way to use this library is using the DenseCRF2D class:

import numpy as np
import densecrf as dcrf

d = dcrf.DenseCRF2D(640, 480, 5)  # width, height, nlabels

You can then set a fixed unary potential in the following way:

U = np.array(...)     # Get the unary in some way.
print(U.shape)        # -> (5, 640, 480)
print(U.dtype)        # -> dtype('float32')
U = U.reshape((5,-1)) # Needs to be flat.
d.setUnaryEnergy(U)

# Or alternatively: d.setUnary(ConstUnary(U))

Remember that U should be negative log-probabilities, so if you're using probabilities py, don't forget to U = -np.log(py) them.

Requiring the reshape on the unary is an API wart that I'd like to fix, but don't know how to without introducing an explicit dependency on numpy.

The two-dimensional case has two utility methods for adding the most-common pairwise potentials:

# This adds the color-independent term, features are the locations only.
d.addPairwiseGaussian(sxy=(3,3), compat=3, kernel=dcrf.DIAG_KERNEL, normalization=dcrf.NORMALIZE_SYMMETRIC)

# This adds the color-dependent term, i.e. features are (x,y,r,g,b).
# im is an image-array, e.g. im.dtype == np.uint8 and im.shape == (640,480,3)
d.addPairwiseBilateral(sxy=(80,80), srgb=(13,13,13), rgbim=im, compat=10, kernel=dcrf.DIAG_KERNEL, normalization=dcrf.NORMALIZE_SYMMETRIC)

Both of these methods have shortcuts and default-arguments such that the most common use-case can be simplified to:

d.addPairwiseGaussian(sxy=3, compat=3)
d.addPairwiseBilateral(sxy=80, srgb=13, rgbim=im, compat=10)

The compat argument can be any of the following:

• A number, then a PottsCompatibility is being used.
• A 1D array, then a DiagonalCompatibility is being used.
• A 2D array, then a MatrixCompatibility is being used.

Possible values for the kernel argument are:

• CONST_KERNEL
• DIAG_KERNEL (the default)
• FULL_KERNEL

Possible values for the normalization argument are:

• NO_NORMALIZATION
• NORMALIZE_BEFORE
• NORMALIZE_AFTER
• NORMALIZE_SYMMETRIC (the default)

The easiest way to do inference is to simply call:

Q = d.inference(n_iterations=5)

And the MAP prediction is then:

map = np.argmax(Q, axis=0).reshape((640,480))

If for some reason you want to run the inference loop manually, you can do so:

Q, tmp1, tmp2 = d.startInference()
for i in range(5):
    print("KL-divergence at {}: {}".format(i, d.klDivergence(Q)))
    d.stepInference(Q, tmp1, tmp2)

The DenseCRF class can be used for generic (non-2D) dense CRFs. Its usage is exactly the same as above, except that the 2D-specific pairwise potentials addPairwiseGaussian and addPairwiseBilateral are missing.

Instead, you need to use the generic addPairwiseEnergy method like this:

d = dcrf.DenseCRF(100, 5)  # npoints, nlabels

feats = np.array(...)  # Get the pairwise features from somewhere.
print(feats.shape)     # -> (7, 100) = (feature dimensionality, npoints)
print(feats.dtype)     # -> dtype('float32')

dcrf.addPairwiseEnergy(feats)

In addition, you can pass compatibility, kernel and normalization arguments just like in the 2D gaussian and bilateral cases.

The potential will be computed as w*exp(-0.5 * |f_i - f_j|^2).

The learning has not been fully wrapped. If you need it, get in touch or better yet, wrap it and submit a pull-request!