If you've spent any time learning networking or backend systems, you've probably run into questions like these:
- If Node.js is single-threaded, how can it handle thousands of connections?
- Why is Redis still single-threaded when modern CPUs have dozens of cores?
- What is Tokio actually doing when I write async Rust?
- Is io_uring really replacing epoll, or is that just internet hype?
- Should I build a thread pool, an event loop, or something else entirely?
At first glance these all seem like different topics, but they're really different answers to the same question:
How should a server handle many connections at the same time?
The problem is that most tutorials only explain one architecture in isolation. You'll find an epoll server, an io_uring server, or a thread-pool server, but they're usually different projects with different parsers, different benchmark setups, and different application code. When one benchmark is faster than another, it's hard to know whether you're measuring the concurrency model or everything else that changed.
I wanted to compare those designs as fairly as I could, so I built eleven TCP server concurrency models behind the same Sans-IO HTTP implementation. Every server shares the same HTTP parser, router, connection state machine, benchmark harness, and workload. The only thing that changes is the concurrency strategy, making it possible to compare classic UNIX designs, modern event loops, and io_uring on equal footing.
Going into the project, I expected io_uring to come out ahead. It did reduce syscall overhead almost exactly as advertised. What surprised me was that this wasn't the thing limiting throughput.
One protocol core, eleven concurrency models
The biggest challenge wasn't implementing eleven servers, it was making the comparison fair.
If every implementation had its own parser, router, and request handling code, any performance difference could come from the application rather than the concurrency model. I wanted to compare the architecture, not eleven slightly different web servers.
To avoid that, every server shares exactly the same HTTP parser, router, response encoder, and connection state machine. The only thing each implementation owns is how it performs I/O and schedules work.
In practice, every concurrency model implements the same Server trait, while the protocol logic remains unchanged.
/// Every concurrency model implements this ONE trait.
pub trait Server {
fn name(&self) -> &'static str;
/// Runs until the process is signalled to stop.
fn serve(&self, cfg: &ServerConfig, app: std::sync::Arc<App>) -> std::io::Result<()>;
}
Every implementation shares the same application logic. The only variable is the concurrency model, making the benchmark a comparison of I/O strategies rather than different web servers.
I included both classic and modern designs so the comparison covers the progression most developers encounter while learning network programming. The implementations start with iterative and thread-based servers, move through readiness-based designs using poll and epoll, and finish with Linux-specific approaches like a pinned SO_REUSEPORT multireactor and a purpose-built io_uring server.
- Iterative
- Forking
- Preforked
- Thread-per-connection
- Thread pool
- poll
- epoll (LT)
- epoll (ET)
- Single reactor
- Multireactor (SO_REUSEPORT)
- io_uring
Together, they cover almost every concurrency architecture you'll encounter while learning UNIX network programming.
Every implementation is benchmarked with the same open-loop, coordinated-omission-correct load generator, ensuring each model receives exactly the same workload.
C10K changes what "efficient" means
One of the goals of this project was to compare how different concurrency models behave under high connection counts, not just high request rates.
To do that, I ran all eleven servers with 10,000 concurrent keep-alive connections while offering the same workload to every implementation.
What surprised me was that throughput wasn't the biggest difference.
The poll, epoll, reactor, multireactor, and io_uring implementations all sustained the offered load.
The real difference was how much memory and operating-system resources each architecture needed to keep those 10,000 connections alive.
The event-loop models hold 10,000 connections flat at about **10.7 MiB. Thread-per-connection needs **261 MiB* for the same job. Source: c10k_summary.csv.*
The difference comes from how each architecture represents a connection.
A thread-per-connection server needs an operating-system thread for every active client, along with its stack and scheduler state. Event-driven servers don't create one thread per client. A connection is mostly just a file descriptor and a small amount of application state, while a handful of worker threads multiplex thousands of sockets using poll, epoll, or io_uring.
That's why the event-driven implementations all stay around 10.7 MiB, while the thread-per-connection server grows to roughly 261 MiB under the same workload.
Not every model reached the same point. The single-threaded iterative server eventually saturated because it can only process one connection at a time. The preforked and thread-pool servers behaved differently: once every worker was occupied, new connections could no longer be serviced, so the benchmark reported a high error rate instead of sustained throughput.
That doesn't mean those architectures are "bad." They're simply designed around bounded worker pools rather than extremely large numbers of mostly idle connections.
At this point, the event-driven models looked like the obvious winners. But there was still another question I wanted to answer. Among modern Linux I/O APIs, does io_uring actually outperform epoll, or is the difference smaller than people often claim?
Fewer syscalls didn't mean higher throughput
After looking at memory usage, I wanted to compare two architectures that are often discussed together: epoll and io_uring.
One of the biggest advantages of io_uring is that it can eliminate many of the system calls required by traditional readiness-based I/O. Features like multishot accept and provided buffer rings let the kernel do more work without repeatedly transitioning between user space and kernel space.
Going into the benchmark, I expected that reduction in syscall overhead to translate into noticeably higher throughput.
io_uring almost halved the number of syscalls per request, but that reduction didn't translate into proportionally higher throughput. Source: profiles/summary.csv.
The syscall reduction was real. Compared to the epoll implementation, the io_uring averaged 2.02 syscalls/request versus 4.03 for epoll., almost exactly a 2× reduction.
The surprising part was that throughput barely changed.
If fewer syscalls automatically meant a faster server, this shouldn't have happened. So the obvious question became: where was the processor actually spending its time?
To answer that, I profiled both implementations on AMD EPYC hardware using the native Zen 4 pipeline utilization counters.
The counters showed that syscall overhead was no longer the dominant bottleneck. Instead, the processor spent much of its time stalled in the frontend, waiting on frontend instruction delivery rather than kernel transitions. Once that became the dominant bottleneck, reducing the number of syscalls simply wasn't enough to produce a proportional increase in throughput.
The lesson wasn't that io_uring failed. It did exactly what it promised by reducing kernel transitions. The benchmark showed something different: removing one source of overhead doesn't necessarily improve end-to-end performance if something else has already become the limiting factor.
While comparing epoll and io_uring answered one question, another result stood out even more. Among all eleven implementations, the architecture that consistently produced the best latency wasn't the newest API, it was the pinned multireactor design built around SO_REUSEPORT.
The architecture I'd build on
Out of all eleven implementations, the one I'd choose as a starting point for a real server isn't necessarily the newest API, it's the pinned multireactor built around SO_REUSEPORT.
It combines the advantages of event-driven I/O with a shared-nothing design. Because each reactor stays on its own core, connections rarely migrate between CPUs, reducing cache disruption and synchronization between workers.
One listening address, one reactor per core. Each reactor owns its own event loop while the kernel distributes new connections using SO_REUSEPORT.
In my benchmarks, this architecture consistently produced the lowest median latency (≈70 μs) while continuing to scale cleanly as concurrency increased.
More importantly, it's a design that extends naturally to larger systems because each reactor is largely independent of the others.
Why it's more than a benchmark
I started this project because I wanted to understand how different TCP server architectures actually compare when they're solving the same problem under the same workload.
The benchmarks answered some questions I expected, like why event-driven servers use dramatically less memory than thread-per-connection designs. More interestingly, they also challenged a few assumptions. Reducing syscall overhead with io_uring didn't automatically produce higher throughput because syscall overhead wasn't the dominant bottleneck in this workload.
The value of benchmarking isn't proving one technology is universally better. It's understanding why it behaves the way it does under a particular workload.
This project is also laying the groundwork for a larger system I'm building around microVM-based agent execution.
A control plane for that kind of system still has to solve the same networking problems: accepting thousands of connections, distributing work across cores, and applying backpressure under load. The multireactor architecture from this project is the design I'll be carrying forward into that work.
Everything in this article is reproducible. Every benchmark, CSV, profiling result, and plotting script is committed in the repository.
If you'd like to reproduce the results, every benchmark, CSV, profiling result, and plotting script is available in the repository.
Repository: https://github.com/umangPokhriyall/Rust-Tcp-Server
Top comments (0)