Can't load CO-RE eBPF code that accesses enum values about ebpf

[dylandreimerink]

I was able to repo the issue and did some digging, found the following:

When I dump the program before CO-RE relocation logic the program looks like:

kprobe_tcp_close:
         ; void BPF_KPROBE(kprobe_tcp_close)
        0: LdImmDW dst: r1 imm: 1
         ; if (bpf_core_enum_value_exists(enum cgroup_subsys_id, cpuset_cgrp_id_lublub))
        2: JEqImm dst: r1 off: 3 imm: 0
        3: LdImmDW dst: r1 imm: 1
         ; __attribute__((unused)) const volatile int val =
        5: StXMemW dst: rfp src: r1 off: -4 imm: 0
         ; void BPF_KPROBE(kprobe_tcp_close)
        6: Exit

But afterwards, it looks like this:

kprobe_tcp_close:
         ; void BPF_KPROBE(kprobe_tcp_close)
        0: LdImmDW dst: r1 imm: 0
         ; if (bpf_core_enum_value_exists(enum cgroup_subsys_id, cpuset_cgrp_id_lublub))
        2: JEqImm dst: r1 off: 3 imm: 0
        3: Call BuiltinFunc(195896080)
         ; __attribute__((unused)) const volatile int val =
        4: StXMemW dst: rfp src: r1 off: -4 imm: 0
         ; void BPF_KPROBE(kprobe_tcp_close)
        5: Exit

Which in my case lead to a jump out of range from insn 2 to 6 error. This is because the LDIMM64 instruction which takes 2 instructions is replaced by a Call BuiltinFunc(195896080) which takes 1 instruction, this throws off offsets of the jumps.

The fixups are:

([]btf.COREFixup) (len=2 cap=2) {
 (btf.COREFixup) enumval_exists=1->0,
 (btf.COREFixup) enumval_value=poison
}

Tracing this, its this piece of code that is responsible

ebpf/btf/core.go

Lines 43 to 48 in ff37506

	if f.poison {
	const badRelo = 0xbad2310
	*ins = asm.BuiltinFunc(badRelo).Call()
	return nil
	}

[orishuss]

Cool! thanks
this also explains why when I used strace I saw BPF_PROG_LOAD with longer insn_cnt when using cilium than when using libbpf.