ReverseProxy panics on client disconnect and your cleanup code never runs

I maintain a proxy that tracks how much each caller spends on LLM API calls and cuts them off at a limit. Before adding anything else to it, I wanted to know whether the numbers it reports are actually right — not "does it compile" right, but right after a restart and right when two hundred requests arrive at once.

I went through the issue tracker of a much larger project doing a similar job, looking for cases where someone's spend numbers had come out wrong in production, and wrote a test for each failure mode against my own code.

Three of them failed the first time I ran them.

The cancelled stream

The proxy uses httputil.ReverseProxy. My bookkeeping ran right after:

go
func (p *Proxy) ServeHTTP(w http.ResponseWriter, r *http.Request) {
    p.rp.ServeHTTP(w, r)
    // record usage
}

Which is fine until a client hangs up mid-response. ReverseProxy doesn't return normally in that case. It panics with http.ErrAbortHandler, a value net/http recognizes and treats as a closed connection rather than a crash. The panic unwinds straight past everything after ServeHTTP.

So a request could be served, cost real money, and leave no record at all. Nothing logged, nothing crashed, nothing counted.

The fix is a defer, because deferred calls still run while a panic unwinds, before it reaches the stdlib's own per-request recovery:

go
defer p.finish(r, state, start)
p.rp.ServeHTTP(w, r)

I didn't recover the panic. net/http already handles ErrAbortHandler correctly and there was no reason to reimplement that.

The connection pool

Separate test, worse number. I fired two hundred concurrent requests at a fresh SQLite database and counted the rows. About 29% weren't there.

SQLite allows one writer. Go's database/sql opens a pool by default, which is the right choice for nearly every database and the wrong one here — a pool of writers is just more connections fighting over a single lock. Past busy_timeout they fail with SQLITE_BUSY, and the calling code could only log it.

go
db.SetMaxOpenConns(1)

Writes queue instead of colliding. It also made those inserts about 30x faster, since a pool blocked on one lock does a lot of expensive waiting that a queue skips.

This isn't specific to what I'm building. Any Go service writing to SQLite under concurrency has this by default, and it fails quietly.

The third one

Smaller. A stored row showing $0 could mean the request was free, or the response didn't parse, or the model wasn't in the pricing table. All identical on disk. A dashboard reading that couldn't tell "nothing happened" from "something happened and I don't know what it cost," which is worse than being wrong because it doesn't look like a problem.

Added a column recording which case applied.

One I left alone

Budget checks happen before a request, costs get recorded after, and a round trip to the provider sits in between. Enough concurrent requests arriving at the boundary all see the old total and all get through. Worst case I measured: $2.99 recorded against a $1.00 limit at a hundred concurrent.

Fixing it properly means reserving an estimate on the way in, truing it up on the way out, and releasing the reservation on every path that doesn't finish — disconnects, upstream errors, the process dying. Bigger and riskier than anything else here, so I wrote the bound down instead and opened an issue.