I've been spending some time learning about eBPF and XDP lately, and honestly, the Linux networking stack is a lot more interesting than I expected.
The basic idea behind eBPF is pretty cool. You can run small, verified programs inside the kernel at specific hooks without having to modify the kernel itself. Depending on where you attach them, these programs can help with tracing, monitoring, security, and networking.
Naturally, I ended up going down the networking rabbit hole.
I learnt about netfilter in linux and it was kind of interesting, I used an old book from 2002 for which iptables was the newest thing but hey, it was an interesting read.
Anyways, as my curiousity grew I ended up going into the actual network path that a packet must take, with the earliest section of it all being the XDP hook.
XDP (eXpress Data Path) lets us run eBPF programs very early in the packet reception path. So instead of letting every incoming packet travel through the entire networking stack before deciding what to do with it, we can make certain decisions much earlier.
Now some of you may ask, whats eBPF, so eBPF stands for extended berkeley packet filter which is pretty much irrelevant to the tasks it performs lol, eBPF is basically a tiny VM that runs inside the kernel at specific "hooks" and carries your programmable code, basically it's a similar kind of concept to an event based hook in React, you have an event that fires, and the eBPF code that triggers in response to that event.
For example, we can drop unwanted packets, pass them to the normal networking stack, or redirect them somewhere else. The interesting part is how much unnecessary work we might be able to avoid by making the right decision early, we use it daily at the company I intern at, for rule based filtering etc, it's much much much more faster than your typical socket based or netfilter based approaches since it's the earliest intercept point you can have, thus saving a lot of context switch cycles etc at the cost of added complexity of coding it, it's quite restrictive.
Then there's AF_XDP, which takes things a step further by allowing userspace applications to exchange packets with the kernel using shared memory and ring buffers. It also supports zero-copy operation on compatible setups, although that isn't guaranteed on every machine, since your NIC basically needs to support the fast path and userspace dump mechanics, otherwise it defaults to the skbuff approach which is still better than kernel based packet processing but, may not give that much of an advantage against the complexity it takes to code it up
One thing I'm beginning to appreciate is that kernel bypass doesn't magically make networking faster. Moving packet processing into userspace also means taking responsibility for things the kernel would otherwise handle. And if we're talking about TCP, there's a whole lot more to deal with than simply sending and receiving packets.
I still have plenty to learn about the internals, especially how XDP interacts with the driver and how AF_XDP manages packet buffers. But it's a fascinating area to explore, and I'm looking forward to getting my hands dirty with more experiments.
One packet at a time, I guess.
If you guys appreciate this rant, there's a also a repo I made which uses standard linux kernel mechanisms like sockets and epoll to have an asynchronous fast packet processing messaging engine with a custom binary fsm, I'm using AI to port it to support AF_XDP... however it will always remain as an external branch until I've custom coded it all, if ur working in this field.. let's connect and I'm always up for learning new things.. https://github.com/Vishisht-Dwivedi/Velora
Top comments (0)