kryndex / clif

Wrapper generator foundation to wrap C++ for Python and other languages using LLVM.

C++ Language Interface Foundation (CLIF)

CLIF provides a common foundation for creating C++ wrapper generators for various languages.

Overview

It consists of four parts:

1. Parser
2. Matcher
3. Generator
4. Runtime

Parser

The parser converts a language-friendly C++ API description to the language-agnostic internal format and passes it to the Matcher.

Matcher

The matcher parses selected C++ headers with Clang (LLVM's C++ compiler) and collects type information. That info is passed to the Generator.

Generator

The generator emits C++ source code for the wrapper.

The generated wrapper needs to be built according with language extension rules. Usually that wrapper will call into the Runtime.

Runtime

The runtime C++ library holds type conversion routines that are specific to each target language but are the same for every generated wrapper.

Python CLIF

See complete implementation of a Python wrapper generator in the /python/ subdirectory. Both Python 2 and 3 are supported.

Installation

Prerequisites

1. We use CMake, so make sure CMake version 3.5 or later is available.

2. We use Google protobuf for inter-process communication between the CLIF frontend and backend. Version 3.2.0 or later is required. Please install protobuf for both C++ and Python from source, as we will need some protobuf source code later on.

3. You must have virtualenv installed.

Building

The steps below are in INSTALL.sh but outlined here for clarification. The install script sets up a Python virtual environment where it installs CLIF.

1. Checkout LLVM and Clang source trees (the exact SVN version as specified here is required¹)

   # We keep it separate of the CLIF tree to avoid pip unwanted copying.
   mkdir $LLVMSRC_DIR
   cd $LLVMSRC_DIR
   svn co https://llvm.org/svn/llvm-project/llvm/trunk@299535 llvm
   cd llvm/tools
   svn co https://llvm.org/svn/llvm-project/cfe/trunk@299535 clang
   ln -sf "$CLIFSRC_DIR/clif" clif

2. Build and install the CLIF backend. If you use Ninja instead of make your build will go significantly faster. It is used by Chromium, LLVM et al. Look at INSTALL.sh for the directory setup and proper ...flags... to supply the cmake command here:

   # Builds must be done outside of the LLVM tree.
   mkdir ../../build_matcher
   cd ../../build_matcher
   cmake ...flags... $LLVMSRC_DIR/llvm
   make clif-matcher
   make install

   Replace the cmake and make commands with these to use Ninja:

   cmake -G Ninja ...flags... $LLVMSRC_DIR/llvm
   ninja clif-matcher
   ninja install

   If the clif-matcher build target is not found, check that you created the correct llvm/tools/clif symlink in the previous step. The CLIF backend builds as part of an LLVM build.

3. Get back to your CLIF python checkout and install it using pip.

   cd "$CLIFSRC_DIR"
   cp "$BUILD_DIR/tools/clif/protos/ast_pb2.py" clif/protos/
   pip install .

INSTALL.sh will build and install CLIF for Python (and LLVM Clang as an internal tool) to your system by default in $HOME/opt/clif and $HOME/opt/clif/clang.

To run Python CLIF use $HOME/opt/clif/bin/pyclif.

Using your newly built pyclif

First, try some examples:

cd ~/opt/clif/examples
less README.md

Next, read the CLIF Python Primer.

Disclaimer

This is not an official Google product.

Footnotes

1. That version is guaranteed to work. Older versions likely do not work (they lack some APIs); later versions might work, at your own risk. ↩