gRPC vs UDP for Low Latency: What You Actually Give Up

When your multi-agent system needs low latency, the default reflex is gRPC. Fast serialization, HTTP/2 multiplexing, bidirectional streams — it checks every box on paper. But between two agents on different machines, gRPC runs over TCP. And TCP has non-negotiables: in-order delivery, congestion control, retransmission backoff, head-of-line blocking inside a single HTTP/2 connection.

For agent-to-agent communication, those properties aren't free. The question isn't whether gRPC is fast — it's whether TCP's guarantees cost more latency than your use case can afford.

The TCP tax gRPC can't escape

gRPC inherits every TCP tradeoff even though most developers never configure a single TCP knob. Here's what you're paying for by default:

Head-of-line blocking inside HTTP/2. HTTP/2 multiplexes multiple streams over one TCP connection, but TCP delivers bytes in order. A lost packet on stream A stalls every other stream on that connection until retransmission arrives. HTTP/2's own flow control adds another per-stream queue on top. Two agents exchanging many small messages concurrently will hit this.

Congestion control ramps up slowly. TCP slow-start on a fresh connection means the first few round-trips carry far fewer packets than the path can handle. For an agent that sends a burst of 20 messages then goes silent, every burst pays the slow-start tax. BBR and other modern CCs help, but they still react to loss—and for agents on lossy wireless or home NAT links, that reaction is a latency spike.

Retransmission timeout (RTO) floor. Linux defaults to 200ms RTO minimum. A single lost TCP segment on a gRPC stream stalls that entire connection for at least 200ms. Over a cross-continent or tail-latency-sensitive agent link, that's an eternity.

Connection setup. gRPC adds TLS handshake on top of the TCP three-way handshake. For long-lived agent connections this is a one-time cost. But for agents that discover each other ephemerally—think a task-specific worker fleet that starts up, communicates, and shuts down—the setup latency before a single message is exchanged can be 2-3 round trips.

What UDP gives back

Moving to UDP removes all four tax lines at once. No head-of-line blocking (each datagram is independent). No slow-start (send at line rate immediately). No 200ms RTO (loss detection is yours to control). No handshake overhead (connectionless by design).

But UDP also removes TCP's guarantees. No ordering, no retransmission, no congestion control, no flow control. You get datagrams: best-effort, unordered, up to ~65KB (though real-world MTU limits make 1400 bytes the practical max).

This is the core trade. gRPC over TCP says: "I'll guarantee ordering and delivery, and you take the latency cost." Raw UDP says: "I'll get out of your way, and you handle order and loss yourself."

Userspace reliability: the middle path

Most agent transports that choose UDP don't stay at raw datagrams. They build userspace reliability on top: selective retransmission (not the whole accumulated TCP window), pluggable loss detection (not a fixed 200ms RTO), and optional ordering (not every message must be in-order; an agent status update can arrive after a later one without breaking anything).

This is a fundamentally different reliability model from TCP's. TCP treats all traffic as one ordered byte stream. A userspace transport over UDP can treat each message as an independent unit and decide per-message whether ordering matters, whether loss is tolerable, and whether the sender even needs an ack.

For agent workloads this maps well:

• Heartbeats and presence updates: loss is fine, order doesn't matter, no reliability needed.
• Task dispatch: ordering matters within one task, not across tasks. Selective ack per message group, not a TCP window.
• Large payloads (state sync, model config): fragment, sequence, retransmit only the missing fragments.

Where agents pay the gRPC-over-TCP tax hardest

Three patterns common in multi-agent systems expose the TCP gRPC tax more than a traditional client-server microservice setup:

1. One-to-many fan-out. A coordinator sends the same task to 10 workers. With gRPC, that's 10 TCP connections (or one HTTP/2 connection per worker). Each has its own congestion state. A lossy link to one worker stalls that stream — not the others, but you're still paying 10× the TCP setup cost. Over UDP the coordinator sends 10 datagrams in a single batch with one syscall batch.

2. Ephemeral agent groups. Agents that form teams per task and dissolve afterward pay TCP connection setup + TLS every time. A 3-RTT setup cost on a 50ms link adds 150ms before the first message arrives. For a 2-second task, that's 7% overhead just to start talking.

3. Lossy or asymmetric links. Agents on residential NAT, WiFi, cellular uplinks, or tail-latency cross-region links see more packet loss than datacenter TCP was designed for. TCP interprets loss as congestion and backs off. A user-space transport can distinguish congestion from corruption and retransmit aggressively on a known-clean link.

Pilot Protocol's approach: userspace reliability over UDP

Pilot Protocol implements exactly this middle path. It builds a userspace reliability layer over encrypted UDP tunnels (X25519 + AES-GCM), with pluggable loss detection and selective retransmission per message — not per byte. The transport is connectionless in the TCP sense but connection-oriented at the application layer: agents have permanent virtual addresses that survive restarts and IP changes, and message delivery semantics are negotiated per exchange rather than dictated by the transport.

The key difference from gRPC over TCP: Pilot's transport doesn't guarantee ordering unless you ask for it. A status update can arrive before or after a task result without stalling either. It doesn't do slow-start — it sends at line rate from message one. And it doesn't have a 200ms RTO floor — retransmission timing is tuned by the application, not the kernel.

This isn't a replacement for gRPC everywhere. gRPC's ecosystem — protobufs, service definitions, generated clients, streaming RPCs — is mature and valuable for request-response APIs between stable services. But for agent-to-agent messaging where latency variance matters more than strict ordering, the transport tradeoffs flip.

What you actually give up

If you move from gRPC over TCP to a userspace-UDP transport, you give up:

• Stream abstraction — you don't get a buffered bidirectional pipe for free. You get messages, and you compose ordering/reliability yourself per use case.
• Ecosystem — no protobuf codegen (though you can still use protobuf for serialization over UDP; serialization and transport are orthogonal).
• Middlebox familiarity — TCP goes through every NAT, firewall, and load balancer predictably. UDP hole-punching is well-understood but not universal; some enterprise networks drop all non-DNS UDP.

What you gain is control over your latency profile. For agent systems where a 200ms stall on a task-critical link is a failure, not a nuisance, that control is worth the trade.

---

Interested in how userspace reliability over UDP works in practice? Pilot Protocol's documentation covers the transport layer, trust model, and how to connect agents across NAT without a VPN.