eBPF program fails with "libbpf: failed to find valid kernel BTF" after upgrading to kernel 6.8.0-48-generic

I'm learning how to use eBPF in Linux environments via the libbpf library. I have a simple eBPF program that compiles and runs successfully on kernel version 5.15.0-125-generic, but after upgrading my kernel to 6.8.0-48-generic, the program fails to run with the following error messages:

libbpf: failed to find valid kernel BTF
libbpf: Error loading vmlinux BTF: -3
libbpf: failed to load object 'execve.bpf.o'
Failed to load eBPF object

Environment:

• OS: Ubuntu 22.04.5 LTS x86_64
• Kernel: 6.8.0-48-generic (upgraded from 5.15.0-125-generic)
• CPU: Intel i9-14900K
• neofetch command result is like below:

Project Details:

I'm working on a small eBPF project that tracks execve() system calls. The code is available on my GitHub repository:

https://github.com/KnightChaser/hello-eBPF/tree/main/application/00_execve_tracking

The project consists of:

• Makefile: Builds and runs the code.
• execve.bpf.c: Kernel-side code that captures execve() executions.
• execve_user.c: User-space code that interacts with the eBPF program via a ring buffer and displays the data.

Steps Taken:

1. Generated vmlinux.h:

   sudo apt update
   sudo apt install linux-headers-$(uname -r) clang llvm libbpf-dev gcc-multilib make
   bpftool btf dump file /sys/kernel/btf/vmlinux format c > vmlinux.h
   

2. Built the program:

   make
   

3. Ran the program:

   sudo ./execve_user
   

   This results in the error messages mentioned above.

Troubleshooting Attempts:

• Verified Kernel BTF Support: Ensured that the kernel is compiled with BTF support.

• Checked pahole Version: Confirmed that pahole is version 1.25.

• Set LIBBPF_LOG_LEVEL=debug: Attempted to get more detailed logs, but no additional output was produced.

  LIBBPF_LOG_LEVEL=debug sudo ./execve_user
  

• Used strace: Traced system calls and noticed that the program tries to access non-existent files like /boot/vmlinux-6.8.0-48-generic:

  access("/boot/vmlinux-6.8.0-48-generic", R_OK) = -1 ENOENT (No such file or directory)
  access("/lib/modules/6.8.0-48-generic/vmlinux-6.8.0-48-generic", R_OK) = -1 ENOENT (No such file or directory)
  ...
  

• In contrast, on kernel 5.15.0-125-generic, the program does not attempt to access these files and runs successfully.

Additional Information:

• My system is dual-booting Windows 11 and Ubuntu via grub2win.
• The vmlinux.h file exists and was generated without errors.
• The issue seems specific to the upgraded kernel version.

Question:

Why is my eBPF program failing with libbpf: failed to find valid kernel BTF after upgrading to kernel 6.8.0-48-generic, and how can I resolve this issue?

Any insights into why libbpf is unable to find valid kernel BTF on the new kernel and what steps I can take to fix this problem would be greatly appreciated.

---

What I've Tried So Far:

• Reinstalling Kernel Headers: Reinstalled the kernel headers for the new kernel version.
• Re-generating vmlinux.h: Ensured that vmlinux.h is up-to-date with the new kernel.
• Checking BTF Files: Verified that /sys/kernel/btf/vmlinux exists and is accessible.
• Exploring Alternative BTF Locations: Noticed that libbpf searches for BTF files in several locations, but they don't exist for the new kernel.

Any help or guidance on how to resolve this issue would be greatly appreciated!

(Note: Currently, this issue is ongoing on the GitHub issue in https://github.com/libbpf/libbpf/issues/863, which I wrote yesterday. Since the libbpf repository GitHub is not so active, I upload my question after refining sentences again to StackOverflow, where related developers might be reachable.)

The problem was occurred due to an old version of libbpf, which some features go incomaptible with upgraded kernel(kernel 6.x.x.). Since upgrading the libbpf package to 1.5.0 and reconfigure library configurations for my Linux machine, I could see that such problems don't arise again.

You can find more details in https://github.com/libbpf/libbpf/issues/863.