andrei-pokrovsky / jitify

A single-header C++ library for simplifying the use of CUDA Runtime Compilation (NVRTC).

Jitify

A single-header C++ library for simplifying the use of CUDA Runtime Compilation (NVRTC).

Rationale

Integrating NVRTC into existing and/or templated CUDA code can be tricky. Jitify aims to simplify this process by hiding the complexities behind a simple, high-level interface.

Quick example

const char* program_source = "my_program\n"
    "template<int N, typename T>\n"
    "__global__\n"
    "void my_kernel(T* data) {\n"
    "    T data0 = data[0];\n"
    "    for( int i=0; i<N-1; ++i ) {\n"
    "        data[0] *= data0;\n"
    "    }\n"
    "}\n";
static jitify::JitCache kernel_cache;
jitify::Program program = kernel_cache.program(program_source);
// ...set up data etc.
dim3 grid(1);
dim3 block(1);
using jitify::reflection::type_of;
program.kernel("my_kernel")
       .instantiate(3, type_of(*data))
       .configure(grid, block)
       .launch(data);

Features

Jitify provides/takes care of the following things:

• All NVRTC and CUDA Driver API calls
• Simple kernel instantiation and launch syntax
• Caching compiled kernels
• Loading source code from strings, files, or embedded in an executable
• Ignoring host code in runtime-compiled sources
• Skipping unneeded headers
• Support for JIT-safe standard library headers (e.g., float.h, stdint.h etc.)
• Dealing with kernel name mangling
• Reflecting kernel template parameters into strings
• Compiling specifically for the current device's compute capability
• Support for CUDA versions 7.0, 7.5, 8.0, 9.0
• Convenient parallel_for function and lambda support

Things you can do with Jitify and NVRTC:

• Rapidly port existing code to use CUDA Runtime Compilation
• Dramatically reduce code volume and offline-compilation times
• Increase kernel performance by baking in runtime constants and autotuning

How to build

Jitify is just a single header file:

include <jitify.hpp>

Link with: -ldl -lcuda -lnvrtc

A small utility called stringify is included for converting text files into C string literals, which provides a convenient way to integrate JIT-compiled sources into a build.

Documentation

Examples

See jitify_example.cpp for some examples of how to use the library. The Makefile also demonstrates how to use the provided stringify utility.

API documentation

Doxygen documentation can be generated by running:

$ make doc

The HTML and LaTeX results are placed into the doc/ subdirectory.

License

BSD-3-Clause

Authors

Ben Barsdell (NVIDIA)

Kate Clark (NVIDIA)