I made my Go reverse proxy 3.8x faster by fixing one line

Aderemi Ademola — Wed, 29 Jul 2026 21:39:13 +0000

I've been building a reverse proxy in Go. Not a hello world one, something meant to sit in production, in the actual path of every request, doing real work on each one before forwarding it along.

For a while I didn't think much about its raw throughput. It worked, requests came in, got processed, got forwarded, came back. Then I actually sat down to benchmark it properly, and the number I got back was rough. Nothing was broken. Nothing errored. It was just slow in a way I couldn't immediately explain.

The setup

The proxy uses Go's standard library httputil.ReverseProxy under the hood, which is the normal, idiomatic way to build something like this in Go. It's a well-built piece of the standard library. The problem wasn't the tool. It was how I was using it.

Here's roughly what my handler looked like:

func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
    upstream := h.resolveUpstream(r)

    proxy := &httputil.ReverseProxy{
        Director: func(req *http.Request) {
            req.URL.Scheme = upstream.Scheme
            req.URL.Host = upstream.Host
        },
    }

    proxy.ServeHTTP(w, r)
}

Looks reasonable at a glance. Every request resolves its upstream, builds a proxy for it, and serves the request. It's readable. It's also wrong in a way that doesn't show up until you actually put load on it.

What was actually happening

Every single request was constructing a brand new ReverseProxy, and by extension a brand new http.Transport, from scratch.

That matters because http.Transport is the thing responsible for connection pooling and reuse in Go's HTTP client. When you build a fresh one on every request, you lose that pooling entirely. Every request pays the full cost of establishing a new connection to the upstream, instead of reusing one that's already open and warm.

I didn't catch this by reading the code and spotting it immediately. I caught it by running a load test and looking at where time was actually going. The latency numbers had a strange shape, most requests were fine, but there was a persistent, ugly tail of slow ones that didn't line up with anything else I was doing.

The fix

Stop building a new ReverseProxy per request. Build one per upstream, once, and reuse it.

var proxyCache sync.Map // map[string]*httputil.ReverseProxy

func (h *Handler) getProxy(upstream Upstream) *httputil.ReverseProxy {
    if p, ok := proxyCache.Load(upstream.Host); ok {
        return p.(*httputil.ReverseProxy)
    }

    proxy := &httputil.ReverseProxy{
        Director: func(req *http.Request) {
            req.URL.Scheme = upstream.Scheme
            req.URL.Host = upstream.Host
        },
    }

    actual, _ := proxyCache.LoadOrStore(upstream.Host, proxy)
    return actual.(*httputil.ReverseProxy)
}

func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
    upstream := h.resolveUpstream(r)
    proxy := h.getProxy(upstream)
    proxy.ServeHTTP(w, r)
}

sync.Map instead of a regular map plus a mutex, because this cache is read constantly and written to rarely, once per distinct upstream, ever. That access pattern is exactly what sync.Map is designed for.

Now the underlying http.Transport, and its connection pool, gets built exactly once per upstream and reused across every request that targets it. Connections stay warm. Nothing gets rebuilt from scratch on the hot path.

What changed

I want to be upfront about what this benchmark actually measures, because it's not a bare hello world server. Go can do 50 to 100 thousand requests a second doing nothing. This proxy does real work on every request, policy evaluation, field transformation, PII detection, audit logging, so the absolute numbers below aren't meant to compete with a raw throughput benchmark. They're meant to show the effect of one specific fix, holding everything else constant.

Metric	Before	After
Requests/sec	1,710	6,541
Average latency	~58ms	7.6ms
p50	~49ms	5.9ms
p90	~110ms	13.7ms
p95	~123ms	18.1ms
Error rate	0%	0%

Tested with hey -n 400000 -c 50 against a real upstream, full policy enforcement enabled on every request, sustained over a 61 second run rather than a short burst.

400,000 requests, 50 concurrent, and every single one came back 200. No degradation over the run, no memory creep, the throughput held steady from the first second to the last. 97.5% of requests resolved in under 25ms while actively running field level policy evaluation, PII scanning, audit logging, and JSON transformation on each one.

Throughput went up roughly 3.8x. Average latency dropped by roughly 8x. Error rate didn't move, it was already at zero before and after, this wasn't a correctness fix, purely a performance one.

The p95 number is the one I actually care about most. Averages hide tail behavior, and tail behavior is what users actually feel. Going from a p95 of roughly 123ms to 18.1ms, sustained across 400,000 requests rather than a quick burst, means the worst 5% of requests got dramatically less bad, not just the typical case, and it held up under real sustained load instead of just looking good in a short test.

The lesson

This is a genuinely easy mistake to make in Go, because the code that causes it doesn't look wrong. There's no compiler warning for it. Nothing crashes. It just quietly throws away one of the more important performance characteristics of the standard library's HTTP client, and the only way to notice is to actually measure.

If you're building anything on top of httputil.ReverseProxy, or honestly anything on top of http.Client or http.Transport, construct it once per destination and hold onto it. Don't rebuild it inside your request handler. It's a small change, and in my case it was worth more than 3x throughput for free.

DEV Community: Aderemi Ademola

I made my Go reverse proxy 3.8x faster by fixing one line

The setup

What was actually happening

The fix

What changed

The lesson