DEV Community

Cover image for Don't put getaddrinfo on your proxy's hot path
Efrain Garay
Efrain Garay

Posted on Originally published at efraingaray.com

Don't put getaddrinfo on your proxy's hot path

I set out to benchmark Pingora 0.9.0 — the Rust proxy library Cloudflare runs at its edge — against nginx, in the environment I actually care about: a small pod with a CPU limit, two vCPUs, everything in containers so no number leans on the host.

I wrote the simplest possible reverse proxy with Pingora, hit it, and got 21k requests/sec against nginx's 126k in the same setup. Six times slower.

That number was a lie, and the fault was mine.

Six times is too much — Cloudflare would not replace nginx with something 6x slower. So I isolated it. Not CPU throttling (nr_throttled was zero). Not thread oversubscription (exactly two workers). Not connection reuse (both held ~130 upstream connections). The clue was latency: 4.78 ms in the container, 0.45 ms on the host with the backend at 127.0.0.1. The only thing that changed was how I named the backend.

My upstream_peer built the destination on every request:

// (&str, u16) → to_socket_addrs() → getaddrinfo, blocking, on the tokio worker
let peer = HttpPeer::new(("backend", 80), false, String::new());
Enter fullscreen mode Exit fullscreen mode

HttpPeer::new over a (&str, u16) calls getaddrinfo synchronously, inside the tokio worker thread. In a container, every request fired a DNS query to Docker's resolver and blocked the thread. On the host, an IP literal is just a parse — no syscall — so it barely showed.

The fix is one line: resolve once at startup, keep the SocketAddr, pass that on the hot path.

let addr = ("backend", 80).to_socket_addrs()?.next().unwrap();
// ...on every request, no lookup:
Ok(Box::new(HttpPeer::new(self.addr, false, String::new())))
Enter fullscreen mode Exit fullscreen mode

Pingora went from 21k to 89k req/s in the same pod. The 6x was a getaddrinfo per request, not the framework.

The lesson isn't about Pingora. A microbenchmark punishes any clumsiness of whoever writes it, and a blocking name resolution hidden on the hot path disguises itself perfectly as "the framework is slow."

With the proxy written properly, the real gap in a 2 vCPU pod is 1.49x — and perf pins it down to the cycle: Pingora runs 1.7x more instructions per request. I also found Pingora's thread default is 1 (a whole idle core if you don't set it), and that in a CPU-limited pod, more threads than cores wrecks the p99 through CFS throttling.

I documented the whole thing — the container setup, the full concurrency matrix with repetitions, the perf/strace profile, the flamegraph, and the thread-throttling numbers — with live diagrams and a reel, here:

👉 The full write-up on efraingaray.com

If you benchmark a Rust proxy in a container: resolve upstreams by IP or once, and set threads equal to your CPU quota. Those two lines matter more than the framework you pick.

Top comments (0)