langston-barrett / nixpkgs-blight

Instrument the builds of tens of thousands of open-source packages

nixpkgs-blight

tl;dr

nixpkgs-blight can instrument the builds of tens of thousands of open-source packages. This can be used to:

• Record and analyze compiler/linker flags used by open-source projects
• Inject dynamic instrumentation like ASan into build processes and run the resulting program's test suite
• Build LLVM bitcode instead of normal executables using gllvm
• Stress test your static analysis tool, compiler, or LLVM pass against thousands of real-world programs

Table of Contents

Table of Contents

• nixpkgs-blight
  • tl;dr
  • Table of Contents
  • What
  • Usage
    • Recording a Build Log
    • Building LLVM Bitcode
  • How
  • Development
    • Debugging
    • Testing

What

blight is a wrapper for C/C++ build tools (CC, CXX, LD, etc.). Out of the box, it can perform a variety of useful actions:

• Recording flags passed to each tool
• Injecting flags
• Benchmarking builds
• Finding build outputs

nixpkgs is a massive, active package repository with extensible, reproducible builds.

This repo combines the two to allow for reproducible instrumentation of thousands of open-source packages. Specifically, it provides:

• A Nix expression for blight
• A framework for using blight inside nixpkgs builds that use common build tools:
  • Make
  • CMake
  • Autotools

Usage

First, install Make and Nix.

In all the following examples, package-name can be replaced with any package in nixpkgs, e.g. entr or hello. For a full list, see

nix eval '(builtins.attrNames (import <nixpkgs> { }))'

or

nix search "some package"

or https://search.nixos.org/packages.

Recording a Build Log

make out/instrument/package-name
less out/instrument/package-name/record.jsonl

Building LLVM Bitcode

make out/bitcode/package-name

Injecting -Werror

make out/inject/package-name

How

For each relevant build system, there is a Nix function that overrides a derivation to check if that derivation is using that build system, writing the result to a file in the overriden derivation's output. This can be used in further Nix expressions to instrument the build system appropriately (for example, by setting CC for Make-based builds, setting cmakeFlags for CMake-based builds, etc.).

After deducing a build system, Nix expressions can override the derivation again, this time configuring blight to do its job.

Development

PRs welcome!

Debugging

make NIXFLAGS="--show-trace --arg debug true"

Testing

• For a package with a Makefile, try entr.
• For a package with a CMakeLists.txt, try yarp.