Nagle vs. Delayed ACK: The 40ms Stall Hiding in Your TCP Stack

A service I debugged a while back had a strange latency histogram: a tight cluster at 2ms, dead air, then a second cluster sitting almost exactly at 40ms. Not a long tail — a second peak, like the request either finished fast or got teleported 40 milliseconds into the future. No GC pause showed up in the logs. No lock contention. No downstream timeout. The thread was just sitting in a recv() call, waiting for bytes that the other side had, by every appearance, already sent.

The problem

The client was doing something extremely common: writing a small header in one write() call and the body in a second write() call, on the same socket, in the same request. Maybe 20 bytes, then 200 bytes. Nothing about that looks wrong — plenty of serialization code is written exactly this way, two writes instead of one buffer. And most of the time it was fine. Then, under certain request shapes, every single request on a connection would eat a flat 40ms penalty, like a tax that only applied when the planets aligned.

Why it happens

Two optimizations, each sensible on its own, ruin each other when combined.

The sender's side has Nagle's algorithm, which exists to stop chatty applications from sending a flood of tiny TCP segments. The rule is simple: if you have unacknowledged data already in flight, don't send a new small segment — buffer it and wait for the pending ACK first. Reasonable. It keeps a dumb line-by-line telnet client from generating one packet per keystroke.

The receiver's side has delayed ACK, which exists to avoid acknowledging every single packet separately. Instead of ACKing immediately, the receiver waits a short window — on Linux, the delayed-ACK timer defaults to 40ms — hoping either more data arrives to ACK in bulk, or the receiving application sends a response it can piggyback the ACK onto.

Put those two together on the same connection and you get a deadlock, not a race:

1. Client sends the small header. Nagle lets this one through (nothing in flight yet).
2. Server receives it, has nothing to say back yet, and delays the ACK — waiting to see if more data or a response shows up to batch it with.
3. Client's second write() (the body) now has unacknowledged data outstanding (the header), so Nagle holds it, waiting for the ACK of the header.
4. Nobody sends anything. The server is waiting to batch an ACK. The client is waiting for that ACK before it'll send more.

Nothing is actually broken — both sides are doing exactly what their algorithm says to do. The deadlock only breaks when the delayed-ACK timer finally fires and flushes the ACK on its own, which is precisely why the stall is a suspiciously flat, constant number: it's not network latency, not scheduling jitter, not a retry backoff curve. It's a timer value. On Linux that's commonly 40ms; other stacks and configurations use different defaults (historically up to 200-500ms on some systems), which is why "weirdly consistent latency spike that's suspiciously round" is the tell to look for.

This is why it's so easy to miss in isolation: a single write() call per request never triggers it. A request/response protocol that always sends one full message per flush never triggers it. It specifically needs split writes on the sending side and "I'll respond in a moment" behavior on the receiving side — which describes an uncomfortable amount of hand-rolled RPC and proxy code.

What to do about it

The fix that actually addresses the cause: stop doing split writes for things that belong together. Buffer the header and body into one write (or use writev/scatter-gather I/O) so there's only ever one segment per logical message. This removes the setup for the deadlock entirely, on both sides, without touching socket options.

The fix that's more commonly reached for: disable Nagle on the sender.

int flag = 1;
setsockopt(sock, IPPROTO_TCP, TCP_NODELAY, &flag, sizeof(flag));

sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)

This is safe for request/response protocols where you control the write pattern, but it's a blunt instrument — it means every small write becomes its own packet, which can matter if you're chatty at high connection counts. The receiver-side equivalent, TCP_QUICKACK on Linux, disables delayed ACK but has to be re-armed after every recv() call since the kernel resets it — which is why most people fix this from the sender side instead.

Most modern HTTP clients, gRPC implementations, and database drivers already set TCP_NODELAY by default, which is why you mostly don't see this anymore in off-the-shelf stacks. It resurfaces in exactly three places: hand-rolled binary protocols, language runtimes or socket libraries where someone changed the default, and anything that proxies traffic through a layer that creates a fresh socket and doesn't carry the option forward.

Key takeaways

• A flat, suspiciously round latency spike (40ms is the classic Linux signature) is a strong hint you're looking at a timer firing, not real network variance — go check delayed ACK before you chase anything else.
• Nagle (sender) and delayed ACK (receiver) are each reasonable in isolation and can deadlock each other when a request is split across multiple small writes with a "respond in a moment" receiver.
• The root-cause fix is coalescing small writes into one segment; TCP_NODELAY is the common workaround and is safe for most request/response code, but understand what you're disabling.
• Check this before blaming the network, the GC, or "flaky infra" for latency spikes that are oddly consistent in magnitude.