AF_XDP packets not sending in zero copy mode until program terminates

[nahla-nee]

I hope this is the right place to ask this question, but I'm having some issues with getting AF_XDP to send packets when bind is called with zero copy mode. I'm using an X540 based Intel NIC using the ixgbe driver version 6.18.5-200.fc43.x86_64 and the kernel version is 6.18.5-200.fc43.x86_64. The only asterisk is that I'm not using libxdp mainly because I want to understand what it's doing behind the scenes and also because I'm using Rust and there are no good up to date bindings for it as it stands.

I've placed a read from stdio at the end of the program to test "pausing" the execution and not letting the program terminate (or the xdp stuff get deallocated/dropped). When binding in copy mode I can see that data is sent right away with no problem without needing to continue past the read. When binding in zero copy mode this is not the case, the data is seemingly held in place until "after" the read. I can see that the data is being sent roughly 5-10 seconds after the program has terminated, which I think is the time it takes for the driver to restart (and maybe do dhcp?). Is this a known problem? what is causing the data to be held in place like this?

The link below is for a gist of a minimal stripped down version of what I have with some hard coded values (which would need to be changed for anyone testing).

https://gist.github.com/nahla-nee/bf36f4c7593d421e486f28979f620348

An XDP program had to be created as sending data in zero copy mode seems to fail as calling sendto results in EINVAL being returned in errno. The XDP program referenced in the code above is as follows:

#include <linux/bpf.h>
#include <bpf/bpf_helpers.h>

SEC("xdp")
int xdp_pass(struct xdp_md* ctx) {
    void* data = (void*)(long)ctx->data;
    void* data_end = (void*)(long)ctx->data_end;
    int pkt_sz = data_end - data;

    bpf_printk("packet size is %d", pkt_sz);
    return XDP_PASS;
}

char __license[] SEC("license") = "GPL";