How many web servers can one virtual machine hold? Not how many requests it can answer per second, and not how many containers you can squeeze onto it, but something more literal: how many servers can each open a socket, listen on their own address, and answer a request, all at the same time?
A writeup published this week by kapteko asked exactly that question under a hard budget: one VM, ten gigabytes of RAM, one CPU core, and a target of fifteen million web servers. It reads like a joke whose punchline is fixing the economics of hacking by making servers too abundant to attack, but underneath the humor sits one of the clearest public walkthroughs of what the Linux kernel actually charges you for networking, and what it costs to stop paying. Every wall the experiment hits is a wall that real systems hit later and rename as an outage.
Wall one: the kernel runs out of ports
The experiment starts with the smallest honest web server you can write in C: create a socket, listen, accept, read, reply, close.
int main(void)
{
int server = socket(AF_INET, SOCK_STREAM, 0);
if (listen(server, 128) < 0)
return 1;
const char reply[] = "HTTP/1.0 200 OK\r\nContent-Length: 2\r\n\r\nOK";
for (int client; (client = accept(server, NULL, NULL)) >= 0;) {
if (recv(client, NULL, 4096, MSG_TRUNC) > 0)
send(client, reply, sizeof reply - 1, MSG_NOSIGNAL);
close(client);
}
}
On one core this loop answers around 65 thousand requests per second, which sounds like a fine start. Then the experiment tries to launch fifteen million copies and dies at copy number 28,232. The failure is not CPU or memory. It is listen returning an error, because each server asked for port zero, and the kernel assigns those from net.ipv4.ip_local_port_range, which defaults to 32768 through 60999. That is roughly 28 thousand ports, and the experiment just exhausted every one of them.
Widening the range to the full 1 through 65535 space buys only about 64 thousand more, because a port is a sixteen-bit field baked into the TCP header itself. No sysctl stretches a protocol constant. The real escape hatch is a different observation: a listening socket is identified by the tuple of protocol, IP address, and port, so fresh addresses multiply the port space for free. Linux supports exactly this through a feature called AnyIP, which lets one machine listen on an entire routed subnet:
sudo ip -6 route add local 2001:db8:1234:5678::/64 dev lo
A single /64 prefix is about 18 quintillion addresses, which is generous headroom. The IPv4 equivalent is theoretically identical and practically awkward, since a routable block with fifteen million addresses trades at roughly a quarter of a billion US dollars these days. Address space, not silicon, is the scarce resource.
Wall two: every socket costs 3,384 bytes, even idle
With addressing solved, the next question is memory, and here the experiment catches a widely repeated claim being wrong. Ask a language model how much RAM an idle TCP connection takes and it will answer confidently: zero for idle, about five kilobytes for active. The experiment believed this, allocated fourteen million idle listening sockets, and watched the machine run out of memory almost immediately.
The truth is visible in the kernel source. A single TCP socket on Linux carries roughly 3,384 bytes of bookkeeping before one byte of user data moves. struct tcp_sock alone is 2,240 bytes of state for queues, timers, sequence numbers, congestion control, and retransmit accounting. socket_alloc adds 768 bytes for the socket object and its sockfs inode. The file structure costs 184 bytes, and the dentry for the pseudo-filesystem name costs 192. None of this is a buffer you can tune away with sysctl; it is the price of the kernel remembering that your socket exists.
Multiply by the target and the math turns brutal: fifteen million kernel sockets need about 48 gigabytes of RAM, five times the whole budget, before any server answers a single request. In the configuration the experiment actually measured, the stock networking path ran the machine dry at roughly 2.5 million servers, with throughput decaying well before that point.
Wall three: a process per server is worse
Even with free sockets, the classic Unix design would still lose. A minimal compiled server process costs about 1.5 megabytes of RAM, so fifteen million of them would need on the order of twenty terabytes. The only way forward is the move language runtimes made years ago: put every guest inside one process and schedule them yourself.
The writeup builds this on LFI, a sandboxing scheme from Stanford research that gives each guest memory isolation and, crucially, replaces every syscall with a call into the host runtime. That replacement is the whole trick: once the kernel is no longer involved in guest I/O, the runtime can implement networking however it wants. The machine becomes its own tiny operating system, with a scheduler that round-robins between guests.
How small can per-guest state get? Look at the server loop again. Between any two scheduling points, the only live values are the program counter and two file descriptors:
struct Context {
pc: u64, // where to resume
fd_server: i32, // listening socket
fd_client: i32, // current client
};
Sixteen bytes per guest, and less if you pack aggressively. Fifteen million guests of scheduling state fit in about 220 megabytes. Combine that with the corrected memory model — fourteen million idle connections held inside a userspace stack, one million active connections at five kilobytes each — and the total lands near five and a half gigabytes: just over half the budget. The plan survives contact with arithmetic. Now it has to survive contact with the network card.
Wall four: TCP has to leave the kernel
TCP is the stateful part of the stack; everything below it is stateless framing, so nothing forces the TCP implementation to live in the kernel. Embedded engineers have carried their own stacks for decades, and Adam Dunkels famously shipped Miniweb, a TCP/IP stack plus HTTP server that runs in twenty-eight bytes of RAM. Mature options exist at every weight class: gVisor reimplements the entire operating system interface including networking, Seastar ships a full runtime with its own stack, lwIP is the classic C implementation, and F-Stack ports the FreeBSD stack wholesale.
The experiment picks smoltcp, a standalone event-driven TCP/IP stack written in Rust for bare-metal systems. The choice is well supported: the crate needs no heap allocation, carries a 0BSD license, sits at roughly 4.7 thousand GitHub stars, and shipped release v0.14.0 in August 2026 with commits still landing this month. But smoltcp is deliberately only the protocol half. It parses and emits packets; it cannot move one byte to the wire by itself. You feed it through a device abstraction:
loop {
let now = Instant::now();
iface.poll(now, &mut device, &mut sockets);
wait(device.as_raw_fd(), iface.poll_delay(now, &sockets))?;
}
Everything interesting hides inside what device is.
Feeding the stack: TUN, TAP, DPDK, AF_XDP
There are four realistic ways to connect a userspace stack to the wire, and they differ in how much of the kernel you keep paying for.
TUN is a virtual kernel interface at the IP layer; TAP does the same job at the Ethernet layer. Both are easy — smoltcp supports them out of the box — but packets still cross the kernel boundary on every trip, so the per-syscall and per-copy costs remain.
DPDK removes the kernel from the data path entirely. The application talks to the network card through memory-mapped ring buffers: sending is writing to memory, receiving is reading memory the card already filled. Context explains why anyone bothers. A syscall costs on the order of 500 nanoseconds, and a kernel-to-userspace copy perhaps another 1,000, while a trading firm saturating a 400-gigabit link must process up to 600 million packets per second — 1.7 nanoseconds per packet. At that budget you cannot afford to even ask the kernel a question. The catch is operational: the poll loop pins an entire core at one hundred percent, and the DPDK driver takes exclusive ownership of the card. The interface vanishes from ifconfig, and if that card carried your SSH session, the session is gone.
AF_XDP is the modern middle path. The kernel keeps owning the device, but a small eBPF program attached at the XDP hook decides per packet whether to redirect the frame straight into a userspace memory region — bypassing the whole network stack — or hand it to normal processing. The region is managed as receive and transmit descriptor rings per socket, plus fill and completion rings over a shared UMEM buffer, which yields zero-copy I/O while SSH, monitoring, and every other kernel consumer keep working. A SO_PREFER_BUSY_POLL mode trades a core for interrupt-free polling when you want behavior even closer to DPDK. For smoltcp specifically, the rust-dpdk-net and xsk-rs crates already wrap the two fast paths, so the device trait is the only glue left to write.
What the final benchmark says
The writeup closes with the comparison the whole piece builds toward. The kernel path dies around 2.5 million servers when RAM runs out. The userspace path slows memory growth to about 620 bytes per idle server and roughly one kilobyte per active one, which is how the experiment finally passes its target: sixteen million web servers, alive, on ten gigabytes. On throughput, DPDK wins the benchmark outright with AF_XDP close behind — the expected result, since AF_XDP keeps a thin slice of kernel involvement that DPDK does without. The full proof of concept, including the scheduler, the LFI plumbing, and all four network backends, is public in the author's AI10M repository.
What this buys you at normal scale
Nobody needs fifteen million web servers. But the per-socket tax applies at every scale: a million idle WebSocket or SSE connections — ordinary for a chat product or a mobile backend — cost about 3.4 gigabytes of kernel memory before your application does anything at all. Knowing that number changes how you argue about architecture. Connection-heavy services can budget for the tax explicitly, move idle state out of the kernel with a userspace stack on the hot path, or adopt AF_XDP where operations teams cannot hand a network card to a single process. The joke experiment turns out to be a measurement of a price most of us pay without ever reading the receipt.
Originally published on Dispatch.
Top comments (0)