addr_space_cast insn can only be used in a program that has an associated arena

I'm trying to explore using BPF arena in XDP programs. I learned from this post that XDP programs aren't sleepable and, therefore, they cannot use the kfunc to allocate BPF arenas' pages. So, my option was to have eBPF syscall programs that simply add and remove elements from a linked list, like in this selftest. And I have an XDP program (in the same file where the arena and syscall programs are declared) that tries to access the elements in that list. However, when I try accessing the value of the first element of the list via list_entry_safe (defined in this other selftest file)

struct elem __arena *n;
struct arena_list_node *first = (&(global_head))->first;

n = list_entry_safe(first, typeof(*(n)), node);

I receive the following error message when I try to attach the XDP program with bpf_program__attach_xdp:

; n = list_entry_safe(first, typeof(*(n)), node); @ xdp_prog_kern.c:140
7: (bf) r1 = addr_space_cast(r1, 0, 1)
addr_space_cast insn can only be used in a program that has an associated arena

So, I'm confused as to why the XDP program wouldn't be associated with the arena, since it's included in the same file as the arena declaration and syscall programs, and is included in the skeleton generated for the arena.

Solution

As you noticed XDP can not allocate memory (at least for now). But, you could allocate pages in the user-space program and use it in XDP to overcome the challenge. But, unfortunately, since you are not referencing the Arena map, the verifier will not recognize the program as one with Arena functionality.

To address the issue you are facing, define a new helper (using kfunc). Then use this helper function in your XDP program so that the Arena map is referenced and associated with the program when the verifier is doing its pass.

Detail

You can add new helpers to eBPF environment by loading new kernel modules using the kfunc API (Look here for some info). You can define an empty helper that does nothing useful. For example:

__bpf_kfunc long my_kfunc_reg_arena(void *p__map)
{
    return 0;
}

Then on your XDP program you can use this helper passing the Arena maps reference to it. As a result the verifier will recognize that your program is using the Arena and you can avoid the error message.

Here I is an example of an XDP program using this techinque. The idea is that the user-space program (loader program) will actually allocate the memory, and pass the reference through the mem global variable. XDP will just use mem to read/write to Arena backed memory address.

long my_kfunc_reg_arena(void *p__map) __ksym;

struct {
    __uint(type, BPF_MAP_TYPE_ARENA);
    __uint(map_flags, BPF_F_MMAPABLE);
    __uint(max_entries, 2); /* number of pages */
} arena_map SEC(".maps");

__arena void *mem = NULL;

SEC("xdp")
int macchiato_main(struct xdp_md *xdp)
{
    bpf_printk("macchiato: hello world\n");

    if (mem == NULL) {
        my_kfunc_reg_arena(&arena_map);
        bpf_printk("not seein the memory!\n");
        return XDP_PASS;
    }

    void *data = (void *)(__u64)(xdp->data);
    void *data_end = (void *)(__u64)(xdp->data_end);
    struct ethhdr *eth = data;
    struct iphdr *ip = (void *)(eth+1);
    struct udphdr *udp = (void *)(ip + 1);
    if ((void *)(udp + 1) > data_end)
        return XDP_PASS;
    if (udp->dest != bpf_ntohs(8080))
        return XDP_PASS;

    __arena entry_t *e = mem;
    bpf_printk("macchiato: counter=%lld\n", e->counter);
    return XDP_DROP;
}

char _license[] SEC("license") = "GPL";

End note

I have written a blog post (and there is an accompanying Github repository with some examples) on this topic, it might be helpful to you.