trailofbits / ebpfault

A BPF-based syscall fault injector

ebpfault

This tool is a syscall fault injector built on top of eBPF that has no requirements on the target machine other than a kernel version good enough to support the required features.

Usage

Sample configuration

The configuration supports both integers and errno value names.

{
  "fault_injectors": [
    {
      "syscall_name": "fchmodat",

      "error_list": [
        {
          "exit_code": "-ENOENT",
          "probability": 50
        },

        {
          "exit_code": -100,
          "probability": 30
        }
      ]
    },

    {
      "syscall_name": "openat",

      "error_list": [
        {
          "exit_code": "-ENOENT",
          "probability": 50
        }
      ]
    }
  ]
}

Against a new process

ebpfault --config /path/to/config.json --exec /path/to/program arg1 arg2

Against one or more running processes

ebpfault --config /path/to/config.json --pid_list pid1,pid2,pid3,...

System wide, excluding one or more running processes

ebpfault --config /path/to/config.json --except-pid-list --pid_list pid1,pid2,pid3,...

Building

Prerequisites

• A recent Clang/LLVM installation (9.0 or better), compiled with BPF support
• A recent libc++ or stdc++ library, supporting C++17
• CMake >= 3.21.4. A pre-built binary can be downloaded from the CMake's download page.
• ⚠️ Linux kernel >= 5.x (tested on Ubuntu 19.10) with the CONFIG_BPF_KPROBE_OVERRIDE option enabled

Building

1. Download the osquery-toolchain from the following page: https://github.com/osquery/osquery-toolchain
2. Extract the osquery-toolchain and set the TOOLCHAIN_PATH environment variable to its location
3. Obtain the source code: git clone --recursive https://github.com/trailofbits/ebpfault
4. In case the --recursive flag was not provided, run git submodule update --init --recursive
5. Configure the project: cmake -S ebpfault -B build-ebpfault -DCMAKE_TOOLCHAIN_FILE=cmake/toolchain.cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo -DEBPFAULT_ENABLE_INSTALL=true
6. Build the project: cmake --build build-ebpfault

ebpfault's Issues

Add support for dynamically adding forked processes to the active pid list

Ebpfault does not inject child process of a program. If an application creates a fork and this fork open a file, ebpfault will not catch the event. Example:

bash -c "echo 'create file to read from: /mnt/data/disk-read-file' && dd if=/dev/zero of=/mnt/data/disk-read-file bs=10k count=1; while true; do time dd if=/mnt/data/disk-read-file of=/dev/null iflag=direct; sleep 1; done"

The previous command will not be impacted by ebpfault because the dd command will be executed in a child process:

strace  -e trace=openat -p xxx
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=16039, si_uid=0, si_status=0, si_utime=0, si_stime=1} ---
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=16042, si_uid=0, si_status=0, si_utime=1, si_stime=0} ---

Do you think we can implement this feature (We could add a flag like -f --follow)?

Outdated version of ebpf-common

In particular trailofbits/ebpf-common@b20da60 from Jun 23, 2021 is not present - the ebpf-common submodule is set to https://github.com/trailofbits/ebpf-common/tree/7d43a4f3ef205e8b6647cd3741d8a4de3619da4b from Jan 6, 2020.

Cherry picking the vDSO patch works fine, but there are also breaking API changes to address.

ebpfault fails to create syscall Kprobes/Kretprobes on ARM processors

Does the ebpfault can support ARM64?

Dumped register values do not correctly support ARM

PR #17 added support for ARM, but the registers that are dumped on screen when a fault is injected are not correct (they are still using x64 code).