Short answer: if you see a trickle of 502s (or read ECONNRESET / socket hang up in clients) that has no pattern in your app logs, check whether your app server closes idle connections sooner than the proxy in front of it. The rule: the component behind must keep idle connections open longer than the component in front. Node's default keepAliveTimeout is 5 seconds; many load balancers default to 60. That gap is a race condition, and it only fires under moderate traffic.
What does the failure look like?
The symptoms I've seen, and that are commonly reported for this problem:
- A low, steady rate of 502 Bad Gateway (or 504) responses from the proxy, often well under 1% of requests.
- The app's own access log shows nothing for those requests. They never arrived, or arrived on a socket that was already dead.
- Errors cluster in "quiet-ish" periods, not at peak. At peak, connections are reused so fast they never idle out. At zero traffic, nothing is sent. The middle is where it hurts.
- Client-side, you may see
Error: read ECONNRESETorError: socket hang upfrom Node,ConnectionResetError: [Errno 104] Connection reset by peerfrom Python, orupstream prematurely closed connection while reading response header from upstreamin the nginx error log.
Dead ends I (and most people) try first: raising app memory, adding retries in the app, blaming the database, increasing proxy read timeouts. None of those touch the cause.
Why does it happen?
HTTP keep-alive lets the proxy reuse one TCP connection to your app for many requests. Both sides run an idle timer. If your app's timer is 5 seconds and the proxy's is 60, the proxy happily believes a connection is still good at second 5.0 while your app is closing it. If the proxy sends a request in that exact window, the request lands on a socket whose FIN is already in flight. The app never processes it; the proxy sees a reset and returns 502.
The window is tiny, which is why the error rate is low and why it looks random. It also scales with traffic volume in the "moderate" band, because you need enough requests for some to land in the window, but enough gaps for connections to idle at all.
Takeaway: an idle-timeout mismatch is a race, so it produces rare, patternless errors that no amount of app-level debugging will explain.
Which defaults conflict? (as of October 2026)
Defaults change between versions, so verify against your own versions. These are the ones I'd check first:
| Component | Setting | Typical default | Notes |
|---|---|---|---|
Node.js http server |
server.keepAliveTimeout |
5 seconds | Too short for most proxies |
Node.js http server |
server.headersTimeout |
60 seconds (in recent versions) | Must exceed keepAliveTimeout
|
| AWS Application Load Balancer | idle timeout | 60 seconds | Configurable per load balancer |
| nginx (as a proxy to upstream) |
keepalive_timeout inside upstream {}
|
60 seconds | Only applies if you enable upstream keepalive connections |
| Gunicorn | --keep-alive |
2 seconds | Very short; matters when a proxy reuses connections |
Go net/http server |
IdleTimeout |
falls back to ReadTimeout, else no limit |
Set it explicitly |
The correct ordering is always: client idle < proxy idle < app idle, measured at each hop. So for an ALB at 60 seconds, the app should be comfortably above that.
How do you fix it in Node.js?
Set the app's keep-alive timeout above the proxy's idle timeout, and keep headersTimeout above that:
const http = require("http");
const server = http.createServer((req, res) => {
res.end("ok");
});
// Proxy (e.g. an ALB) idles out at 60s, so stay above it.
server.keepAliveTimeout = 65_000;
// headersTimeout must be greater than keepAliveTimeout, or Node can
// still drop connections early.
server.headersTimeout = 66_000;
server.listen(3000);
Frameworks like Express and Fastify sit on top of this, so you set it on the underlying http.Server (the value returned by app.listen() in Express). Fastify exposes keepAliveTimeout as a server option.
For Go, set it explicitly instead of inheriting:
srv := &http.Server{
Addr: ":3000",
Handler: mux,
IdleTimeout: 65 * time.Second,
}
For Gunicorn, --keep-alive 65 (and note that the default sync worker doesn't really support keep-alive; a threaded or async worker class does).
Takeaway: pick one number for the proxy's idle timeout, then set every server behind it to at least 5 seconds more.
How do you prove it's the cause before changing anything?
Don't just apply the fix and hope. A quick reproduction: set the app timeout very low and hammer it at intervals near that value.
// repro.js: expects the server above with keepAliveTimeout = 1000
const http = require("http");
const agent = new http.Agent({ keepAlive: true, maxSockets: 1 });
function hit() {
return new Promise((resolve) => {
http
.get({ host: "localhost", port: 3000, agent }, (res) => {
res.resume();
res.on("end", () => resolve("ok"));
})
.on("error", (e) => resolve(e.code));
});
}
(async () => {
const results = {};
for (let i = 0; i < 200; i++) {
const r = await hit();
results[r] = (results[r] || 0) + 1;
// Wait close to the server's idle timeout to land in the race window
await new Promise((r) => setTimeout(r, 1000));
}
console.log(results); // ECONNRESET shows up in the tally
})();
With the server timeout at 1 second and the client waiting about 1 second, you should see some ECONNRESET results. Raise the server timeout to 5 seconds and rerun, and they should disappear. You are reproducing the race at small scale. In production, correlate by checking the proxy's logs: ALB access logs record the backend status as - for these cases, and nginx logs the prematurely closed line above.
One more detail: since Node 19, the global HTTP agent uses keep-alive by default, so outbound calls from your Node services to other services can hit the same race in the opposite direction. If a downstream closes idle sockets quickly, you get ECONNRESET on your side. Retrying idempotent requests once on ECONNRESET is a reasonable second line of defense, but fix the timeout first.
Is a managed load balancer worth it for this?
The mismatch exists with any proxy, managed or self-hosted. The difference is visibility. If you want the idle timeout as a single documented setting you can read in one place, AWS Application Load Balancer is the one that exposes it per load balancer with a clear default. Its limitation is that it only solves your side of the bargain; it can't know what your app does, and the fix still lives in your code. Self-hosted nginx gives more control, including upstream keepalive tuning, but you own every timeout and every default across versions.
FAQ
Why do I get intermittent 502 errors from my load balancer when my app logs show nothing?
Most often the app closed an idle keep-alive connection just as the load balancer reused it, so the request never reached your code. Raise the app's keep-alive timeout above the load balancer's idle timeout.
What should Node.js keepAliveTimeout be set to behind an AWS ALB?
Higher than the ALB idle timeout, which defaults to 60 seconds, so something like 65 seconds, with headersTimeout set slightly above that.
Is read ECONNRESET always a keep-alive problem?
No. It also appears when a process crashes, a container is OOM-killed, or a firewall drops idle connections. The keep-alive race is the likely cause when it is rare, patternless, and your app logs and metrics look healthy.
Bottom line
If you run Node, Gunicorn, or any app server behind a load balancer or reverse proxy, set the app's keep-alive timeout explicitly rather than inheriting defaults, and make it longer than the proxy's idle timeout. Reproduce with a low timeout if you need to convince yourself or your team. If errors persist after the fix, move on to crashes, memory limits, and network devices that drop idle flows.
Top comments (1)
The moderate-traffic pattern is a strong diagnostic clue because it requires both reuse and idle gaps. I would add a metric that records upstream connection age and reuse count alongside 502s; that makes the timeout-race hypothesis testable before changing defaults, and provides a regression signal after the timeout ordering is fixed.