aliyun / coolbpf

coolbpf

coolbpf's target is to build a platform for libbpf compile collection,
which is for creating efficient kernel tracing and manipulation
programs, is to wrapper main functions of libbpf for user development.

coolbpf's main function are:
1) Service for local compile, and some wraps for BPF function call.
2) Service for remote compile, it receives xx.bpf.c, and return bpf.so
to your local APP to loading. The user can focus on their main functions
and don't
care compile environment.
3) High kernel version backport to low version with kernel module, such
as 3.10 BPF support, and perf buffer replace with new feature of ring
buffer.
4) BTF auto generate.
5）Variety kernel version testing for new BPF functions and tools.
6）Support for many languags, python/go/c/rust.

Compiler Environment

Compiling libcoolbpf requires installing the following dependent libraries/tools:

• elfutils-devel
• gcc

Compiling the eBPF tool requires additional installation of the following dependent libraries/tools:

• clang
• llvm

Install/uninstall libcoolbpf

Installation: Run ./install.sh in the coolbpf root directory. Uninstall: Run ./uninstall.sh in the coolbpf root directory.

Usage example

In the tools/examples/syscall directory, we provide examples of using libcoolbpf to develop eBPF programs. The process of compiling the syscall eBPF tool is as follows:

• Install libcoolbpf: Run ./install.sh in the coolbpf root directory to install libcoolbpf
• Compile syscall: run mkdir -p build && cd build && cmake -DBUILD_EXAMPLE=on .. && make in the coolbpf root directory

The location of the final generated syscall executable program is: build/tools/examples/syscall/syscall.

coolbpf's Issues

do some common ssa optimization

we need to do below common optimization:

1. constant folding
2. DCE（Deadcode Elimination）
3. algebraic simplifications

【Anolis OS 8.6 ANCK 64位 ARM版】 运行hello.py 编译错误

1.在龙蜥实验室申请了一台【Anolis OS 8.6 ANCK 64位 ARM版】机器，并按照coolbpf_handbook.md安装好了coolbpf运行环境：

2.在运行hello.py时，出现如下编译错误：

执行或编译异常

python pip安装完之后运行hello报错
surftrace.surfException.DbException: db set return setup obj report: [Errno 2] No such file or directory: '.output/lbc.bpf.o'.

golcc中hello在make时也会报这个错，是什么依赖没安装吗

Support eBPF map related operations

New map-related operations are added, corresponding to the update and deletion of eBPF map. The main syntax is:

skbmap[skb] = nsecs;  // update
ns = skbmap[skb];        // lookup
delete(skbmap, skb);   // delete

coolbpf vs btfhub+btfgen

I think coolbpf is a cool combination of btfhub and btfgen.

Could u list some other advantages over btfhub and btfgen?

Convert VBPFInst to BPFInst

After the register allocation is completed, we know the preg corresponding to vreg, so we can convert VBPFInst to BPFInst. The corresponding conversion function is: regalloc_emit_inst, currently only realizes the conversion from VBPFInst::Alu32 to BPFInst::Alu32, and needs to complete the remaining instructions convert.

Can cranelift officially support the eBPF backend?

If cranelift officially supports the eBPF backend, we can save a lot of work. And it can ensure the simplicity of lwcb. I am also now filing issues[1] with the wasmtime community, but currently it may be just our needs, but we may need to consider the functional positioning of Cranelift itself.

[1]. bytecodealliance/wasmtime#7060

support bpf_probe_read

When accessing kernel memory, safe access needs to be performed through bpf_probe_read.

移除libfirm依赖，libfirm学习成本过高，不适合小型的脚本语言

需要设计一个bpf中间代码，主要流程：

1. 可以根据imple and Efficient Construction of Static Single Assignment Form来实现一个ssa ir，在生成ir的过程中即可做优化，优化主要包括：constant folding、algebraic simplification、cse、copy propogation等
2. 指令选择，需要将bpf ir替换成bpf指令，比如load指令，一般load的中间代码只有2个操作数，而bpf的load指令会有3个操作数，包括目的寄存器，源寄存器，偏移，也就是dst_reg = *(uint *) (src_reg + off16)这种形式。在指令选择期间可以做一些简单的优化。
3. 寄存器分配，可以使用rust wasmtime的regalloc2库，该库支持ssa ir形式的代码生成，符合我们的需求。
4. 最后就是代码生成。

Use cranelift-islec to generate instruction selection code

There are many conditions to consider when selecting instructions, and the logic of manually written code is complicated. Therefore, cranelift-islec is used to automatically generate this part of the code according to the rules. In the future, only these rules need to be maintained.

Add tests for struct access and verify functionality

Probably similar to the following structure access.

a = sk.sk_priority

golcc make异常

在lcc/golcc/rexample/hello下面执行make，出现了以下异常：

是下载的libbpfgo的版本问题么，我应当怎样解决？

Moving towards lwcb 2.0

In lwcb1.0, we used libfirm ir to generate eBPF bytecode. But Libfirm itself is more complicated and has a high learning cost. Therefore, in the future lwcb2.0, we will introduce the intermediate code of lwcb. lwcb 2.0 mainly includes the following features:

1. Introduce bpfir, the ir adopts the ssa form of basic block parameters, do some simple optimization on ssa, and then use regalloc2 to allocate registers. The overall process is probably: ir->optimize->instruction selection->register allocation->code generation.
2. Introduce blang to parse lwcb script language and generate bpfir.
3. lwcb as a whole acts as a runtime to realize the loading of eBPF bytecodes and the creation of maps, etc.

XDP是只能在内核程序里面修改数据包吗？

XDP是限定了只能在内核程序里面修改数据包吗？有没有办法在用户态修改数据包呢？

Add tests for if statement and verify functionality

Improve bpfir code generation part

At present, only the bpf_endian instruction is processed in code generation, and some instructions are still unprocessed. You can refer to the codegen_endian_inst function to implement.

fn codegen_endian_inst(dst: u8, bits: i32) -> bpf_insn {
    let mut insn = bpf_insn::default();
    insn.code = (BPF_ALU | BPF_END | BPF_X) as u8;
    insn.set_dst_reg(dst);
    insn.imm = bits;
    insn
}

code location at: lwcb/bpfir/src/be/codegen.rs

eliminate basic block arguments

We build ssa through basic block arguments, so after building ssa, we need to eliminate some non-essential basic block arguments.

Support print function

Improve the test examples of isel

coolbpf 是如何支持在低版本内核上运行 ebpf 程序的？

README 中有提到：

High kernel version backport to low version with kernel module, such
as 3.10 BPF support, and perf buffer replace with new feature of ring
buffer.

对这一块的具体实现，我挺感兴趣的，请问能否提供进一步的说明？

Generate elf file instead of raw bpf instructions

We can directly call libbpf-rs to load the eBPF program carried by the elf file into the kernel, saving a lot of work.

Add tests for relation expression and verify functionality

Add eBPF map information to the generated elf file

The following steps may be involved:

1. Add eBPF map information to the map segment
2. Add the relocate information of eBPF map
3. Write test cases, involving a simple eBPF program, using an eBPF map, and using libbpf-rs to load and pass.

For details, please refer to the logic of Libbpf processing. The functions that may be involved in libbpf include bpf_object__elf_collect (collecting elf information, such as eBPF program instructions, map information, relocation information, etc.) and bpf_object__relocate_data (mainly relocating map information, that is, the map's fd )

Register allocation using wasmtime's regalloc2

Cannot execute python hello.py

python hello.py
http://www.foxbeaver.cn:80/lbc

<title>500: Internal Server Error</title>500: Internal Server Error Traceback (most recent call last): File "/data/local/tmp/coolbpf-1.0.1/coolbpf-1.0.1/lcc/pylcc/guide/hello.py", line 42, in hello = Chello() ^^^^^^^^ File "/data/local/tmp/coolbpf-1.0.1/coolbpf-1.0.1/lcc/pylcc/guide/hello.py", line 37, in __init__ super(Chello, self).__init__("hello", bpf_str=bpfPog) File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/pylcc/lbcBase.py", line 313, in __init__ super(ClbcBase, self).__init__(bpf, bpf_str, server, arch, ver, File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/pylcc/lbcBase.py", line 76, in __init__ self._checkBtf(ver, arch) File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/pylcc/lbcBase.py", line 106, in _checkBtf dRecv = cli.getBtf() ^^^^^^^^^^^^ File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/surftrace/lbcClient.py", line 140, in getBtf return self._post(dSend) ^^^^^^^^^^^^^^^^^ File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/surftrace/lbcClient.py", line 82, in _post rd = json.loads(res.text.encode()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/data/com.termux/files/usr/lib/python3.11/json/__init__.py", line 346, in loads return _default_decoder.decode(s) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/data/com.termux/files/usr/lib/python3.11/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/data/com.termux/files/usr/lib/python3.11/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Introduce CLIF (cranelift ir) for optimization

clif is very lightweight and simple, so we can use clif to perform some common optimizations. #35 is deprecated.

请教一下bpf关于拼接长字符串的问题，

您好，麻烦请教一下就是我在内核态程序需要循环通过struct dentry结构获取某一个文件的路径，我的方法目前是首先有一个结构体，结构体内部有一个char filename[512];的数组，然后循环将每一级目录加上'/'拼成一个完整的路径，
问题是循环过程中调用bpf_probe_read_kernel_str接口向filename中拷贝时如果第二个参数是一个变量，那么运行时Bpf校验器就会说可能越界什么的，但是在前后我已经做了很完善的边界检查了，之所以bpf_probe_read_kernel_str第二个参数需要是变量是因为每一级目录名长度是不同的，当我改成一个定值就没问题了。
其实总的来说就是bpf如何拼接一个长字符串，我基本尝试过所有相关的接口也无法解决了
下面是部分代码

其中用到的部分定义如下

char filename[512];
#define MAX_DENTRY_NAME_LEN         20

    size_t pos = 0;
    const u32 MAX_BACKTRACE_DEPTH = 20;
    for (u32 cnt = MAX_BACKTRACE_DEPTH; cnt != 0; --cnt) {
        if (err || curr_dentry == NULL) {
            break;
        }
        int name_len = BPF_CORE_READ(curr_dentry, d_name.len);
        const u8 *name = BPF_CORE_READ(curr_dentry, d_name.name);
        if (name_len <= 1) {
            break;
        }
        name_len = bpf_probe_read_kernel_str(filename + pos, MAX_DENTRY_NAME_LEN, name);
        if (name_len <= 1) {
            BPFLOGD(BPF_TRUE, "failed to read dentry name from kernel stack buffer");
            break;
        }
        pos += name_len;
        filename[pos - 1] = '/';
        struct dentry *temp_dentry = BPF_CORE_READ(curr_dentry, d_parent);
        if (temp_dentry == curr_dentry || temp_dentry == NULL) {
            break;
        }
        curr_dentry = temp_dentry;
    }

目前的效果是只能去循环20层，每层目录拿20个字符，我也将filename改为了4096长度，但是bpf_probe_read_kernel_str接口还是不允许我使用变量（上面代码中的name_len）作为第二个参数，我最终想要的效果其实是想通过循环可以完整的拿到每一级目录名，以及最终拼接成完整的路径，请教大佬是否有好的解决方案，感谢！！！

编译make ebpfdrv KERNEL_VERSION=3.10.0-514.26.2.el7.x86_64报错

ome/coolbpf/build/lcc/glcc/lib/ebpf/bpf_trace.c:682:11: error: ‘bpf_get_stackid_proto_tp’ undeclared (first use in this function)
return &bpf_get_stackid_proto_tp;

请问这个什么原因怎么解决

golcc/user_slow的demo无法在kernel：3.10.0-1160.53.1.el7.x86_64运行，同时insmod也存在报错

[root@centos7test data]# ./user_slow
libbpf: failed to determine tracepoint 'tcp/tcp_probe' perf event ID: No such file or directory
libbpf: prog 'tcp_probe_hook': failed to create tracepoint 'tcp/tcp_probe' perf event: No such file or directory
failed to attach program: no such file or directory

通过strace观察应该是/sys/kernel/debug/tracing/events/tcp目录不存在导致
所以使用了glcc编译ebpfdrv.ko准备试试，发现make时报错

#编译3.10.0-1160.53.1.el7.x86_64的ko
make ebpfdrv KERNEL_VERSION=3.10.0-1160.53.1.el7.x86_64

/opt/github/coolbpf/build/lcc/glcc/lib/ebpf/linux/rbtree_latch.h:49:20: error: redefinition of 'rb_link_node_rcu'
 static inline void rb_link_node_rcu(struct rb_node *node, struct rb_node *parent,
                    ^
In file included from include/linux/sched.h:20:0,
                 from include/linux/ptrace.h:5,
                 from ./include/uapi/asm-generic/bpf_perf_event.h:4,
                 from arch/x86/include/generated/uapi/asm/bpf_perf_event.h:1,
                 from include/uapi/linux/bpf_perf_event.h:10,
                 from include/linux/perf_event.h:18,
                 from /opt/github/coolbpf/build/lcc/glcc/lib/ebpf/allsyms.h:4,
                 from /opt/github/coolbpf/build/lcc/glcc/lib/ebpf/linux/bpf.h:11,
                 from /opt/github/coolbpf/build/lcc/glcc/lib/ebpf/ebpf.c:13:
include/linux/rbtree.h:89:20: note: previous definition of 'rb_link_node_rcu' was here
 static inline void rb_link_node_rcu(struct rb_node *node, struct rb_node *parent,
                    ^
In file included from /opt/github/coolbpf/build/lcc/glcc/lib/ebpf/linux/bpf.h:18:0,
                 from /opt/github/coolbpf/build/lcc/glcc/lib/ebpf/ebpf.c:13:
/opt/github/coolbpf/build/lcc/glcc/lib/ebpf/linux/rbtree_latch.h:58:19: error: redefinition of 'raw_read_seqcount_latch'
 static inline int raw_read_seqcount_latch(seqcount_t *s)
                   ^
In file included from include/linux/time.h:5:0,
                 from include/uapi/linux/timex.h:56,

使用3.10.0-514.26.2.el7.x86_64可以make通过，但是发现insmod时报错：

[root@centos7test build]# insmod ./lcc/glcc/lib/ebpf/ebpfdrv.ko
insmod: ERROR: could not insert module ./lcc/glcc/lib/ebpf/ebpfdrv.ko: Cannot allocate memory
[root@centos7test build]#

内核版本

[root@centos7test build]# uname -r
3.10.0-1160.53.1.el7.x86_64
[root@centos7test build]#