Use ebpf CO-RE to build EBPF prog

[daym]

It's probably easier to use the official eBPF CO-RE build system to build EBPF modules.

That's what this PR does.

[gustavo-iniguez-goya]

hi @daym , thanks for this proposal.

Unless the PR adds new interesting functionality or solves an issue, I'm usually reluctant to change things just because. Maybe the Makefile is not beautiful, but it works for compiling the modules on all the architectures Debian support.

So the first question is in what architectures have you tested your changes. We need to support the existing array or archs.

On the other hand, the Makefile seems to have a problem with the double backslash \\:

~ $ make KERNEL_DIR=../linux-6.0/ KERNEL_HEADERS=../linux-6.0/ ARCH=x86
Makefile:22: *** recipe commences before first target.  Stop.

Once fixed, it fails to compile:

~ $ make V=1 KERNEL_DIR=../linux-6.0/ KERNEL_HEADERS=../linux-6.0/ ARCH=x86
make -C ../linux-6.0/ M=/opensnitch/ebpf_prog CC=clang
make[1]: Entering directory '/opensnitch/linux-6.0'
echo >&2;							\
echo >&2 "  ERROR: Kernel configuration is invalid.";		\
echo >&2 "         include/generated/autoconf.h or include/config/auto.conf are missing.";\
echo >&2 "         Run 'make oldconfig && make prepare' on kernel src to fix it.";	\
echo >&2 ;							\
/bin/false)
warning: the compiler differs from the one used to build the kernel
  The kernel was built by: gcc (Debian 10.2.1-6) 10.2.1 20210110
  You are using:           Debian clang version 11.0.1-2
WARNING: Symbol version dump "Module.symvers" is missing.
         Modules may not have dependencies or modversions.
         You may get many unresolved symbol warnings.
  MODPOST /opensnitch/ebpf_prog/Module.symvers
make[1]: Leaving directory '/opensnitch/linux-6.0'

This is a pbuilder chroot, i686 arch. Other compilation scenarios are lxc containers for example.
We also generally use kernels downloaded from kernel.org, not only the ones of the distributions.

[daym]

Hi, the background is we are trying to package opensnitch for GNU Guix, see https://codeberg.org/guix/guix/pulls/2368

The reason why we even touched it is because it's including kernel headers using manual -I. This way, you wouldn't notice if the build flags changed because the kernel config did. Do you run without kernel config in the master version? How? Do the kernel headers even work without config?

[blshkv]

#712
I tried and failed to convince the author. FYI

[daym]

I have updated this PR after learning a lot more about eBPF in the mean time :)

BPF is an architecture, so one can just use a compiler targeting that architecture.

BPF with CO-RE: BTF relocation resolves struct field offsets at load time, so compiled objects handle field offset changes without recompilation. That means the .o files will work with newer Linux kernels than the one we built for, as long as those new versions only change field offsets (which is likely).

opensnitch uses some Linux kernel structs (obviously), so I have added the required struct definitions (with only the required fields) into vmlinux.h (from bpftool btf dump file /sys/kernel/btf/vmlinux format c). The output of bpftool btf dump file /sys/kernel/btf/vmlinux format c has 150000 lines, so might not want to just use all :)

I have removed generated files in favor of using libbpf.

Advantages:

• More independent of Linux kernel version. Does not need Linux kernel sources (much less compiled ones!) or Linux kernel headers at all at build time.

Disadvantages:

• If the Linux headers would change in a massive way (new field that we dont know but we do need, entire struct definition replaced etc) we wouldn't notice.

Flags removed:

• -emit-llvm - was for two-step build (clang -> LLVM IR -> ll -> BPF), replaced by -target bpf direct compilation

[gustavo-iniguez-goya]

On x86_64 works fine. On aarch64/debian 11/kernels 5.10/6.1 fails compiling with a minor issue, and opensnitchd-procs.o fails loading, due to btf issues.
On i386 also fails loading. I haven't tested it on armhf.

Did you compile and test the modules on these archs? I'll test on more modern versions

[gustavo-iniguez-goya]

opensnitch uses some Linux kernel structs (obviously), so I have added the required struct definitions (with only the required fields) into vmlinux.h (from bpftool btf dump file /sys/kernel/btf/vmlinux format c). The output of bpftool btf dump file /sys/kernel/btf/vmlinux format c has 150000 lines, so might not want to just use all :)

In the future we'll need to access nsproxy, cgroup and other structs and fields, so it'd be better to use the full vmlinux.h to compile the modules. Or document explicitely how to add the structs, but personally I'd prefer to avoid it.

[gustavo-iniguez-goya]

Some tests:

x86_64 (Debian 14, clang version 21.1.8):

Ok:

• 5.8.x ubuntu 20.10
• 5.15.x linux mint 21.2
• 5.15.x ubuntu 20
• 6.4.x OpenSuse 15.6
• 6.8.0 ubuntu 22
• 6.8.12 proxmox,
• 6.12.48 debian 12,
• 6.14.x ubuntu 25,
• 6.17.10 deban 14
• 6.17.x fedora 43

No ok:

• 5.4.x linux mint 20.3 (btf errors. Here the legacy module works.)
• 5.4.0 ubuntu 20 (network interceptor module loads, but processes/dns mods no (expected))

aarch64 (clang 14.0.6, debian 13.3):

• Ok: kernel 6.12.63 debian 13
• No ok: debian 11 with kernels 5.10.x and 6.1.x, debian 12 kernels 6.1.0

i686: will be not supported anymore I think. MX kernel 6.1.0 is not shipped with the item CONFIG_DEBUG_INFO_BTF configured.
armhf: not tested, but the last time I tested CO:RE on this arch suffered of the same problem than i686.

regardless of how we compile the procs module for aarch64, the cmdline is usually incorrect, so until we find a proper fix, it'll be better to read it from /proc.

So basically, we lose support for older kernels, but in return we gain better compatibility with newer ones.