DEV Community: Umang Pokhriyal

I benchmarked 11 TCP server concurrency models in Rust. Here's what actually mattered.

Umang Pokhriyal — Wed, 29 Jul 2026 13:39:39 +0000

If you've spent any time learning networking or backend systems, you've probably run into questions like these:

If Node.js is single-threaded, how can it handle thousands of connections?
Why is Redis still single-threaded when modern CPUs have dozens of cores?
What is Tokio actually doing when I write async Rust?
Is io_uring really replacing epoll, or is that just internet hype?
Should I build a thread pool, an event loop, or something else entirely?

At first glance these all seem like different topics, but they're really different answers to the same question:

How should a server handle many connections at the same time?

The problem is that most tutorials only explain one architecture in isolation. You'll find an epoll server, an io_uring server, or a thread-pool server, but they're usually different projects with different parsers, different benchmark setups, and different application code. When one benchmark is faster than another, it's hard to know whether you're measuring the concurrency model or everything else that changed.

I wanted to compare those designs as fairly as I could, so I built eleven TCP server concurrency models behind the same Sans-IO HTTP implementation. Every server shares the same HTTP parser, router, connection state machine, benchmark harness, and workload. The only thing that changes is the concurrency strategy, making it possible to compare classic UNIX designs, modern event loops, and io_uring on equal footing.

Going into the project, I expected io_uring to come out ahead. It did reduce syscall overhead almost exactly as advertised. What surprised me was that this wasn't the thing limiting throughput.

One protocol core, eleven concurrency models

The biggest challenge wasn't implementing eleven servers, it was making the comparison fair.

If every implementation had its own parser, router, and request handling code, any performance difference could come from the application rather than the concurrency model. I wanted to compare the architecture, not eleven slightly different web servers.

To avoid that, every server shares exactly the same HTTP parser, router, response encoder, and connection state machine. The only thing each implementation owns is how it performs I/O and schedules work.

In practice, every concurrency model implements the same Server trait, while the protocol logic remains unchanged.

/// Every concurrency model implements this ONE trait.
pub trait Server {
    fn name(&self) -> &'static str;
    /// Runs until the process is signalled to stop.
    fn serve(&self, cfg: &ServerConfig, app: std::sync::Arc<App>) -> std::io::Result<()>;
}

Every implementation shares the same application logic. The only variable is the concurrency model, making the benchmark a comparison of I/O strategies rather than different web servers.

I included both classic and modern designs so the comparison covers the progression most developers encounter while learning network programming. The implementations start with iterative and thread-based servers, move through readiness-based designs using poll and epoll, and finish with Linux-specific approaches like a pinned SO_REUSEPORT multireactor and a purpose-built io_uring server.

Iterative
Forking
Preforked
Thread-per-connection
Thread pool
poll
epoll (LT)
epoll (ET)
Single reactor
Multireactor (SO_REUSEPORT)
io_uring

Together, they cover almost every concurrency architecture you'll encounter while learning UNIX network programming.

Every implementation is benchmarked with the same open-loop, coordinated-omission-correct load generator, ensuring each model receives exactly the same workload.

C10K changes what "efficient" means

One of the goals of this project was to compare how different concurrency models behave under high connection counts, not just high request rates.

To do that, I ran all eleven servers with 10,000 concurrent keep-alive connections while offering the same workload to every implementation.

What surprised me was that throughput wasn't the biggest difference.

The poll, epoll, reactor, multireactor, and io_uring implementations all sustained the offered load.

The real difference was how much memory and operating-system resources each architecture needed to keep those 10,000 connections alive.

The event-loop models hold 10,000 connections flat at about **10.7 MiB. Thread-per-connection needs **261 MiB* for the same job. Source: c10k_summary.csv.*

The difference comes from how each architecture represents a connection.

A thread-per-connection server needs an operating-system thread for every active client, along with its stack and scheduler state. Event-driven servers don't create one thread per client. A connection is mostly just a file descriptor and a small amount of application state, while a handful of worker threads multiplex thousands of sockets using poll, epoll, or io_uring.

That's why the event-driven implementations all stay around 10.7 MiB, while the thread-per-connection server grows to roughly 261 MiB under the same workload.

Not every model reached the same point. The single-threaded iterative server eventually saturated because it can only process one connection at a time. The preforked and thread-pool servers behaved differently: once every worker was occupied, new connections could no longer be serviced, so the benchmark reported a high error rate instead of sustained throughput.

That doesn't mean those architectures are "bad." They're simply designed around bounded worker pools rather than extremely large numbers of mostly idle connections.

At this point, the event-driven models looked like the obvious winners. But there was still another question I wanted to answer. Among modern Linux I/O APIs, does io_uring actually outperform epoll, or is the difference smaller than people often claim?

Fewer syscalls didn't mean higher throughput

After looking at memory usage, I wanted to compare two architectures that are often discussed together: epoll and io_uring.

One of the biggest advantages of io_uring is that it can eliminate many of the system calls required by traditional readiness-based I/O. Features like multishot accept and provided buffer rings let the kernel do more work without repeatedly transitioning between user space and kernel space.

Going into the benchmark, I expected that reduction in syscall overhead to translate into noticeably higher throughput.

io_uring almost halved the number of syscalls per request, but that reduction didn't translate into proportionally higher throughput. Source: profiles/summary.csv.

The syscall reduction was real. Compared to the epoll implementation, the io_uring averaged 2.02 syscalls/request versus 4.03 for epoll., almost exactly a 2× reduction.

The surprising part was that throughput barely changed.

If fewer syscalls automatically meant a faster server, this shouldn't have happened. So the obvious question became: where was the processor actually spending its time?

To answer that, I profiled both implementations on AMD EPYC hardware using the native Zen 4 pipeline utilization counters.

The counters showed that syscall overhead was no longer the dominant bottleneck. Instead, the processor spent much of its time stalled in the frontend, waiting on frontend instruction delivery rather than kernel transitions. Once that became the dominant bottleneck, reducing the number of syscalls simply wasn't enough to produce a proportional increase in throughput.

The lesson wasn't that io_uring failed. It did exactly what it promised by reducing kernel transitions. The benchmark showed something different: removing one source of overhead doesn't necessarily improve end-to-end performance if something else has already become the limiting factor.

While comparing epoll and io_uring answered one question, another result stood out even more. Among all eleven implementations, the architecture that consistently produced the best latency wasn't the newest API, it was the pinned multireactor design built around SO_REUSEPORT.

The architecture I'd build on

Out of all eleven implementations, the one I'd choose as a starting point for a real server isn't necessarily the newest API, it's the pinned multireactor built around SO_REUSEPORT.

It combines the advantages of event-driven I/O with a shared-nothing design. Because each reactor stays on its own core, connections rarely migrate between CPUs, reducing cache disruption and synchronization between workers.

One listening address, one reactor per core. Each reactor owns its own event loop while the kernel distributes new connections using SO_REUSEPORT.

In my benchmarks, this architecture consistently produced the lowest median latency (≈70 μs) while continuing to scale cleanly as concurrency increased.

More importantly, it's a design that extends naturally to larger systems because each reactor is largely independent of the others.

Why it's more than a benchmark

I started this project because I wanted to understand how different TCP server architectures actually compare when they're solving the same problem under the same workload.

The benchmarks answered some questions I expected, like why event-driven servers use dramatically less memory than thread-per-connection designs. More interestingly, they also challenged a few assumptions. Reducing syscall overhead with io_uring didn't automatically produce higher throughput because syscall overhead wasn't the dominant bottleneck in this workload.

The value of benchmarking isn't proving one technology is universally better. It's understanding why it behaves the way it does under a particular workload.

This project is also laying the groundwork for a larger system I'm building around microVM-based agent execution.

A control plane for that kind of system still has to solve the same networking problems: accepting thousands of connections, distributing work across cores, and applying backpressure under load. The multireactor architecture from this project is the design I'll be carrying forward into that work.

Everything in this article is reproducible. Every benchmark, CSV, profiling result, and plotting script is committed in the repository.

If you'd like to reproduce the results, every benchmark, CSV, profiling result, and plotting script is available in the repository.

Repository: https://github.com/umangPokhriyall/Rust-Tcp-Server

The "Optimal" Order Book Data Structure Lost by 288 on Real Market Data

Umang Pokhriyal — Mon, 27 Jul 2026 13:24:50 +0000

While learning about limit order books, I kept coming across two ideas. The first was the textbook answer: a flat array indexed by price is the optimal solution for a dense book because updates are O(1). The second came from low-latency systems and HFT discussions, where people repeatedly stress that contiguous memory and cache locality often matter more than algorithmic complexity.

That got me wondering how these ideas compare in practice, so I implemented four different order book data structures and benchmarked each implementation using both synthetic workloads and a real BTCUSDT replay.

The setup

A limit order book is the core data structure of an exchange: for one symbol, it tracks resting buy and sell orders at each price level, and the hot operation is applying a stream of updates (add, cancel, modify) as fast as possible.

In Rust, this is implemented using a common OrderBook trait. I also wrote a differential test harness that replays the same event stream through every implementation and verifies that they all produce identical results before benchmarking them.

I chose four implementations that represent the trade-offs you'll commonly see discussed in systems programming: pointer-based trees, contiguous vectors, linear scans that exploit locality, and direct indexing with arrays.

BTreeMap the straightforward implementation. Memory usage grows only with occupied price levels, but updates involve tree traversal and pointer chasing.
Sorted Vec of (price, level). stores price levels in contiguous memory and locates them with binary search. Better cache locality than a tree, but insertions require shifting elements.
Reverse-sorted Vec with a linear scan. keeps the best prices at the front and relies on a simple linear scan. This sounds inefficient, but if most updates happen near the inside market, it can outperform binary search.
FlatBook (flat array indexed by price tick) indexes directly by price tick, giving constant-time access without searching.

Going into the project, I expected the flat array to come out ahead. It has constant-time lookups, avoids tree traversal entirely, and is the implementation that's usually recommended for dense order books. The benchmarks were mainly meant to measure how much faster it was than the alternatives.

The order book isn't the whole system, though. In a market data engine, updates also need to be distributed to multiple consumers, so each implementation plugs into the same lock-free pipeline shown below.

One pinned producer, two lock-free primitives, K independent consumers. The book is the hot path; the seqlock and ring are how its output fans out.

On synthetic data, the textbook answer wins. On real data, it comes in last by 288x

To avoid drawing conclusions from a single workload, I benchmarked each implementation under two very different conditions. The first was a synthetic workload with a relatively narrow, uniformly distributed price range. The second replayed a real BTCUSDT market session, where updates arrive with the same characteristics as they did on the exchange.

" width="799" height="458">

The synthetic benchmark matches the textbook expectation. The real market replay tells a completely different story. Every bar is re-derived from throughput.csv.

The surprising part isn't that FlatBook slows down—it's that the entire ranking reverses once the workload becomes realistic.

The result wasn't just that the ranking changed—it completely flipped. The FlatBook implementation, which was consistently the fastest on the synthetic benchmark, became the slowest on the real replay, while the BTreeMap implementation ended up on top by a wide margin.

If I had only benchmarked the synthetic workload, I would've concluded that the flat array was the obvious choice. Replaying real market data completely changed that conclusion.

At this point, the obvious question was why. The algorithms hadn't changed, only the workload had. The answer turned out to have very little to do with Big-O complexity and everything to do with how much memory the data structure actually touches.

Why the FlatBook slowed down

The biggest difference between the synthetic benchmark and the real replay wasn't the number of updates—it was the range of prices those updates covered.

A FlatBook allocates memory for every possible price tick in that range, whether there's an order there or not. That means its memory footprint grows with the price span, not with the number of price levels that actually contain orders.

The flat array's span on the real book is ~88 MiB, about 2.74x the 32 MiB L3. Every "O(1)" index is now a main-memory round trip.

This was the point where it became clear that memory footprint mattered more than lookup complexity.

Once the array grows beyond what the processor can comfortably keep in cache, the cost of an "O(1)" lookup changes dramatically. The indexing itself is still constant time, but the data you're indexing often has to be fetched from lower levels of the memory hierarchy.

The BTreeMap has the opposite trade-off. Tree traversal involves pointer chasing, but memory usage stays proportional to the number of occupied price levels instead of the total price span. On the real replay, that smaller working set outweighed the extra traversal cost.

In this workload, the deciding factor wasn't algorithmic complexity—it was the working set size. Once the FlatBook grew to roughly 88 MiB, cache locality became the dominant cost.

The same idea explains another result from the benchmarks. I originally expected the binary-search implementation to consistently outperform the linear scan, but that wasn't always true either.

If most updates happen near the best bid and ask, a short linear scan often finishes before a binary search has a chance to pay off. If updates are spread uniformly across the book, the scan eventually becomes too expensive and binary search wins instead.

Under uniform touches the linear scan degrades ~18x by depth 2048; under concentrated touches it never loses. Source: service_sweep.csv.

The choice between linear scan and binary search depends more on access patterns than on theoretical complexity.

At this point the benchmark results made sense, but I still wanted evidence that the processor was actually spending time where I thought it was. That's where the hardware performance counters came in.

One of the reasons low-latency systems often favor contiguous data structures is cache locality—but this benchmark also shows the other half of that advice. Contiguous memory only helps when the working set still fits in cache. Once it doesn't, the advantage can disappear surprisingly quickly.

What the hardware counters showed

The benchmarks told me which implementation was faster, but they couldn't explain why. For that, I needed hardware performance counters (PMUs), which expose where the processor is actually spending its time.

Before I had access to server hardware, I tried to explain each benchmark result just from its behavior. Looking at how throughput changed under different workloads, I formed a hypothesis about what each implementation was bottlenecked on.

Later, I reran the benchmarks on a rented AMD EPYC system and collected the native Zen 4 PMU counters. They matched those hypotheses surprisingly well. Here's a summary of what the counters showed.

The table below isn't meant to compare performance again—it summarizes where each implementation spends its time.

impl	IPC	bad-spec	backend-mem	verdict
`SortedVec`	2.50	0.1%	50.5%	memory-bound (branchless locate)
`BTreeBook`	1.33	28.3%	9.1%	pointer chase (frontend + memory)
`RevVec`	6.10	1.3%	2.3%	core-bound (scan length)
`FlatBook`	2.43	25.5%	13.4%	mispredict-bound at wide depth

The interesting part isn't which implementation is "best." It's that each one spends its time differently, even though they're solving exactly the same problem.

SortedVec spends most of its time waiting on memory. Its binary search uses partition_point, which is already branchless, so branch prediction barely shows up.
BTreeBook pays the expected cost of pointer chasing through the tree.
RevVec is mostly limited by the work done in the scan itself.
FlatBook, surprisingly, spends much more time recovering from branch mispredictions than waiting on memory. This isn't something I'd be comfortable concluding from the benchmark alone. The PMU data is what made the explanation convincing.

A quick note: these measurements come from AMD Zen 4's native pipeline-utilization counters. They're conceptually similar to Intel's Top-Down methodology, but they're not the same thing, so I've intentionally kept the terminology specific to AMD.

Earlier I mentioned that low-latency engineers often emphasize cache locality and real measurements over theoretical complexity. This is exactly why. The benchmark tells you what happened. The PMU counters are what let you build a credible explanation for why it happened.

The two primitives, and the seqlock that never blocks its writer

The order book wasn't the only component I built. Rather than benchmark the order book in isolation, I embedded it in a small market-data pipeline. That's where the other two components in the architecture diagram come in: a seqlock for publishing the latest top-of-book snapshot and an SPMC ring buffer for broadcasting every update. Both implementations are entirely safe Rust (#![forbid(unsafe_code)]) and were verified with Loom before benchmarking.

A seqlock uses a simple version counter. Even values mean the data is stable, while odd values indicate that a write is in progress. Readers optimistically copy the snapshot and then check whether the version changed while they were reading. If it did, they simply retry. Because readers never acquire a lock, the writer is never blocked.

pub fn load_counted(&self) -> (TopOfBook, u32) {
    loop {
        let s1 = self.seq.load(Acquire);      // (R1) pairs with the writer's Release
        if s1 & 1 == 0 {                      // even => no write in progress; snapshot it
            let t = TopOfBook { /* payload: Relaxed loads */ };
            fence(Acquire);                   // (R2) order payload reads before the re-check
            let s2 = self.seq.load(Relaxed);
            if s1 == s2 { return (t, retries); } // unchanged => no write straddled the read
        }
        // else: odd, or straddled => discard and retry
    }
}

In practice, a load() stays around 10 ns (p50) and remains essentially flat even as the number of readers increases. Across six million timed reads, no torn snapshots were observed.

The ring buffer did expose one limitation. As more consumers are added, producer throughput drops from 12.17 to 8.46 Mev/s. My first suspicion was false sharing, but perf c2c showed something different. The payload slots remain isolated on separate cache lines; the contention comes from the write cursor itself, which every consumer must observe. In other words, this is true sharing, not false sharing. It's an inherent cost of broadcasting to multiple readers rather than a bug in the implementation.

I left that behavior as-is because eliminating it would require changing the design rather than fixing an implementation issue. The producer remains wait-free; only the throughput changes.

These primitives weren't the focus of the benchmark, but they complete the data path and are reusable outside the order book itself.

What I learned

I started this project wanting to compare different order book implementations. What I ended up with was a much better appreciation for how easily synthetic benchmarks can hide real bottlenecks.

The FlatBook wasn't "wrong"—it was simply optimized for a workload that didn't resemble the one I eventually tested. Once the working set grew beyond cache, a data structure with theoretically better complexity became the slowest implementation in the benchmark.

This project is also serving as a stepping stone toward a larger goal: building low-latency infrastructure for microVM-based systems. The same approach—benchmark on realistic workloads, measure with hardware counters, and avoid relying on intuition alone—is the one I'll carry into that work.

The repository contains all benchmark inputs, raw CSVs, plotting scripts, and source code, so every figure in this article can be reproduced.

Repository: https://github.com/umangPokhriyall/low-latency-lob

I Built a Threshold Signer. Then I Forged It.

Umang Pokhriyal — Wed, 15 Jul 2026 08:55:42 +0000

I built a threshold signature library, audited my own code, and found it was forgeable. So I forged it. Then I rebuilt it properly.

What I set out to build

A t-of-n threshold Ed25519 signer. The idea is that a private key is split across several parties, and you need some threshold t of them to cooperate to produce a signature. No single party ever holds the whole key. It's the primitive behind custody systems, MPC wallets, and any place where "one machine holds the key" is an unacceptable single point of failure.

The existing reference for this on Solana is ZenGo's solana-tss, but it's MuSig2 (n-of-n, no DKG) and only signs native SOL transfers, not SPL token transactions. A real custody setup needs t-of-n, distributed key generation, and token transfers, so I started implementing one. That's where I reached for naive threshold Schnorr, and that's the scheme that turned out to be forgeable.

I got a version working end to end on devnet. Then I sat down to actually audit it and it fell apart in two places.

The first defect: the coordinator held the whole key

In my design, each node returned its final secret share to the coordinator, which collected all of them. That means the coordinator saw at least t shares and could reconstruct the private key and sign alone.

A threshold system where one party can reconstruct the key is not a threshold system. It's a key in one place with extra steps.

The fix, in the rebuild, is a Pedersen DKG where no party ever holds the group secret: each participant ends with only its own share sᵢ, and the group public key is assembled in the exponent, never the scalar. Here's the actual assembly, straight from dkg.rs` — notice it only ever adds public commitment points, never shares:

rust // group_public = Σ_j φ_{j,0} over all participants (peers + self). let mut group_public = secret.own_commitment.0[0]; for package in &round1_packages { group_public = group_public + package.commitments.0[0]; }

The second defect: the signing scheme was forgeable

I'd implemented naive interactive threshold Schnorr: one nonce per signer, sum the commitments, hash for the challenge, sum the partial signatures. There's no binding factor anywhere. That construction is broken by the ROS attack: given enough concurrent signing sessions, an attacker can treat the per-session challenges as a linear system and solve it for a forgery. This isn't folklore. It's the Benhamouda–Lefranc–Loss–Orsini–Raykova result from 2020, and it runs in polynomial time.

Forging it

Saying "this is theoretically forgeable" proves nothing. So I kept the old scheme's math under legacy/, reduced to its core (the threshold aggregate reduces to single-key concurrent Schnorr, so the oracle models that single key directly), and ran the actual ROS solver against it.

The attack opens 256 concurrent sessions, uses the binary-decomposition trick to pick each session's message after seeing every commitment, and solves for a forgery.

It works. A valid signature on an unsigned message, in about 50 milliseconds. That number, and the proof that the message is outside the signed set, are committed in legacy/results/ros_forgery.txt.

The rebuild, and why it resists the attack

The rebuild follows RFC 9591 FROST (Ed25519, SHA-512), hand-rolled on curve25519-dalek. The reason FROST resists what the old scheme didn't is one line: each signer's nonce is bound by a factor that hashes in everyone's commitments before anyone reveals a partial signature.

plaintext ρ_j = H1(group_public ‖ msg ‖ commitment_list ‖ id_j) // the binding factor R = Σ_j (D_j + ρ_j · E_j) // bound group commitment z_i = d_i + (ρ_i · e_i) + (λ_i · c · s_i) // partial signature

The challenge is no longer linear in quantities the attacker controls before committing, so the ROS solver has no linear system to solve.

Running the same solver against the FROST implementation returns NoSolution frost-core/tests/ros_resistance.rs. The attack fails because the binding factors remove the linear structure that the ROS solver depends on. That's the whole point of the audit-then-rebuild: the forgery is committed, reproducible evidence that the original was broken, and the new one is checked against the standard rather than asserted to be correct.

The implementation is hand-rolled on curve25519-dalek rather than wrapping an existing FROST library. That also means it needs stronger verification than "the tests pass", so the implementation is checked against independent references at every stage instead of assuming the implementation is correct.

RFC 9591 known-answer vectors are checked byte-for-byte, including intermediate values (binding factors, aggregate commitment, partial signatures, and final signature), so failures identify the first divergent step instead of only the final output.
A differential test compares the implementation against an independent FROST library across 10,000 randomized (t, n) configurations. The reference implementation is used only during testing and is not part of the shipped dependency graph.
Invalid partial signatures identify the offending participant (Err(Culprit(id))), allowing the coordinator to distinguish protocol failures from malicious or faulty signers.

The bug the fuzzer found

After I'd frozen the code, I ran a coverage-guided fuzzer, one target per deserializer, 104 million-plus executions. It found a bug. My group.rs accepted non-canonical point encodings: two distinct byte-strings decoding to the same group element. That's a malleability vector, and curve25519-dalek's decompress() silently canonicalizes exactly those inputs, so I'd never have caught it by hand.

The fix is RFC 8032 strict decoding: re-encode the point and reject on any byte mismatch.

rust pub fn from_compressed(b: [u8; 32]) -> Result<Self, Error> { let point = CompressedEdwardsY(b) .decompress() .ok_or(Error::InvalidPointEncoding)?; // RFC 8032 strict decoding: reject NON-CANONICAL encodings (a y-coordinate // >= the field prime, or a set sign bit on the x = 0 points). dalek's //decompress()accepts and silently canonicalizes these, which would let // two distinct byte strings denote the same point (a malleability vector). // (Found by the Phase 4 coverage-guided fuzz run.) if point.compress().to_bytes() != b { return Err(Error::InvalidPointEncoding); } if point.is_torsion_free() { Ok(GElement(point)) } else { Err(Error::NonPrimeOrderPoint) } }

This is a deserialization malleability issue rather than a key-recovery vulnerability, so that's how it's documented. The important part wasn't the bug itself—it was that coverage-guided fuzzing found it after the implementation had already been considered complete, with the failing inputs preserved as regression tests.

What came out of it

The repository now contains both versions:

the original threshold Schnorr implementation under legacy/, together with a reproducible ROS forgery;
an RFC 9591 FROST implementation validated against published test vectors and differential tests;
fuzz tests that uncovered and fixed a non-canonical point decoding issue before release.

The point wasn't simply to produce another signing library. It was to build one, break it under audit, fix it, and leave the evidence of each step in the repository.

shell cargo run --example in_process_2of3

Repository:
https://github.com/umangPokhriyall/frost-ed25519-kit

Threat model:
https://github.com/umangPokhriyall/frost-ed25519-kit/blob/main/docs/THREAT-MODEL.md