A 502 Bad Gateway or 504 Gateway Timeout tells you that something failed around a gateway boundary.
It does not necessarily tell you which gateway failed.
That distinction matters when a request crosses multiple layers:
Client
|
v
Edge proxy
|
v
API gateway
|
v
Application
If the client receives:
504 Gateway Timeout
several different things may have happened:
- the gateway could not establish a connection to its upstream
- the gateway connected successfully but waited too long for a response
- an outer proxy could have timed out while an inner gateway was still waiting
- the application could have continued processing after an upstream proxy had already abandoned the request
I built a small Docker lab with two NGINX proxies and a Python application to make these failure modes deterministic.
The most interesting result was one request for which the different layers recorded:
External client: 504
Edge NGINX: 504
Gateway NGINX: 499
Application: eventually attempts 200
Those results are not contradictory.
They describe the same request from different points in the request path.
This lab shows how to reconstruct that path.
What we're building
The topology is deliberately simple:
localhost:8080
Client ───────────► Edge NGINX
|
v
Gateway NGINX
|
v
Python API
For debugging, each layer is also exposed directly:
Application localhost:8000
Gateway localhost:18081
Edge localhost:8080
I ran the lab on macOS using Colima with Docker Engine 29.2.1 on ARM64.
The containers used:
NGINX 1.31.4
Python 3.13.15
Container IP addresses shown later are specific to my run and should not be treated as fixed values.
Project structure
Create:
proxy-debug-lab/
├── app/
│ ├── app.py
│ └── Dockerfile
├── compose.yaml
├── edge/
│ └── nginx.conf
└── gateway/
└── nginx.conf
Build a small backend that can be intentionally slow
The backend uses only Python's standard library.
app/app.py:
import json
import time
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from urllib.parse import parse_qs, urlparse
class Handler(BaseHTTPRequestHandler):
def do_GET(self):
parsed = urlparse(self.path)
trace_id = self.headers.get("X-Request-ID", "-")
if parsed.path == "/health":
self.respond(
200,
{
"status": "healthy",
"trace_id": trace_id,
},
)
return
if parsed.path == "/fast":
self.respond(
200,
{
"status": "ok",
"endpoint": "fast",
"trace_id": trace_id,
},
)
return
if parsed.path == "/slow":
params = parse_qs(parsed.query)
try:
seconds = float(params.get("seconds", ["5"])[0])
except ValueError:
seconds = 5
seconds = max(0, min(seconds, 30))
print(
f"layer=app trace={trace_id} "
f"path={parsed.path} sleeping={seconds}s",
flush=True,
)
time.sleep(seconds)
self.respond(
200,
{
"status": "ok",
"endpoint": "slow",
"slept_seconds": seconds,
"trace_id": trace_id,
},
)
return
self.respond(
404,
{
"status": "not_found",
"path": parsed.path,
"trace_id": trace_id,
},
)
def respond(self, status, payload):
body = (json.dumps(payload) + "\n").encode()
self.send_response(status)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(body)))
self.end_headers()
try:
self.wfile.write(body)
except BrokenPipeError:
print("layer=app client_connection_closed=true", flush=True)
def log_message(self, fmt, *args):
trace_id = self.headers.get("X-Request-ID", "-")
print(
f'layer=app trace={trace_id} message="{fmt % args}"',
flush=True,
)
if __name__ == "__main__":
server = ThreadingHTTPServer(("0.0.0.0", 8000), Handler)
print("layer=app listening=0.0.0.0:8000", flush=True)
server.serve_forever()
The endpoint we care about most is:
/slow?seconds=5
It accepts the request, sleeps for five seconds without sending response headers, and then attempts to return 200.
That lets us deliberately place shorter proxy timeouts in front of it.
The Dockerfile is minimal:
FROM python:3.13.15-alpine3.24
WORKDIR /app
COPY app.py .
EXPOSE 8000
CMD ["python", "-u", "app.py"]
Instrument the gateway
Before introducing failures, make the proxies observable.
gateway/nginx.conf:
events {}
http {
map $http_x_request_id $trace_id {
default $http_x_request_id;
"" $request_id;
}
log_format trace
'$time_iso8601 '
'layer=gateway '
'trace=$trace_id '
'status=$status '
'upstream_status=$upstream_status '
'request_time=$request_time '
'upstream_response_time=$upstream_response_time '
'upstream_addr=$upstream_addr '
'request="$request"';
access_log /var/log/nginx/access.log trace;
error_log /var/log/nginx/error.log notice;
server {
listen 80;
add_header X-Gateway-Observed "true" always;
location = /health {
proxy_pass http://app:8000/health;
proxy_set_header X-Request-ID $trace_id;
proxy_read_timeout 2s;
}
location = /fast {
proxy_pass http://app:8000/fast;
proxy_set_header X-Request-ID $trace_id;
proxy_connect_timeout 2s;
proxy_read_timeout 10s;
}
# Reachable host, but nothing listens on port 9999.
location = /bad-gateway {
proxy_pass http://app:9999;
proxy_set_header X-Request-ID $trace_id;
proxy_connect_timeout 2s;
proxy_read_timeout 2s;
}
# App sleeps for 5s, but Gateway waits only 2s.
location = /gateway-timeout {
proxy_pass http://app:8000/slow?seconds=5;
proxy_set_header X-Request-ID $trace_id;
proxy_connect_timeout 2s;
proxy_read_timeout 2s;
}
# App sleeps for 5s and Gateway is willing to wait 10s.
location = /edge-timeout {
proxy_pass http://app:8000/slow?seconds=5;
proxy_set_header X-Request-ID $trace_id;
proxy_connect_timeout 2s;
proxy_read_timeout 10s;
}
error_page 502 = @gateway_502;
error_page 504 = @gateway_504;
location @gateway_502 {
default_type text/plain;
add_header X-Gateway-Observed "true" always;
add_header X-Debug-Generated-By "gateway" always;
return 502 "gateway generated 502\n";
}
location @gateway_504 {
default_type text/plain;
add_header X-Gateway-Observed "true" always;
add_header X-Debug-Generated-By "gateway" always;
return 504 "gateway generated 504\n";
}
}
}
The important log fields are:
$status
$upstream_status
$request_time
$upstream_response_time
$upstream_addr
NGINX also exposes fields such as $upstream_connect_time and $upstream_header_time, which are useful additions in a production logging format.
Configure the outer edge proxy
edge/nginx.conf:
events {}
http {
map $http_x_request_id $trace_id {
default $http_x_request_id;
"" $request_id;
}
log_format trace
'$time_iso8601 '
'layer=edge '
'trace=$trace_id '
'status=$status '
'upstream_status=$upstream_status '
'request_time=$request_time '
'upstream_response_time=$upstream_response_time '
'upstream_addr=$upstream_addr '
'request="$request"';
access_log /var/log/nginx/access.log trace;
error_log /var/log/nginx/error.log notice;
server {
listen 80;
add_header X-Edge-Observed "true" always;
add_header X-Request-ID $trace_id always;
# If Gateway itself returns an error, pass that response through.
proxy_intercept_errors off;
location = /health {
proxy_pass http://gateway/health;
proxy_set_header X-Request-ID $trace_id;
proxy_read_timeout 3s;
}
location = /fast {
proxy_pass http://gateway/fast;
proxy_set_header X-Request-ID $trace_id;
proxy_connect_timeout 2s;
proxy_read_timeout 10s;
}
location = /bad-gateway {
proxy_pass http://gateway/bad-gateway;
proxy_set_header X-Request-ID $trace_id;
proxy_connect_timeout 2s;
proxy_read_timeout 5s;
}
# Give Gateway enough time to generate its own 504.
location = /gateway-timeout {
proxy_pass http://gateway/gateway-timeout;
proxy_set_header X-Request-ID $trace_id;
proxy_connect_timeout 2s;
proxy_read_timeout 10s;
}
# Edge gives up first.
location = /edge-timeout {
proxy_pass http://gateway/edge-timeout;
proxy_set_header X-Request-ID $trace_id;
proxy_connect_timeout 2s;
proxy_read_timeout 2s;
}
error_page 502 = @edge_502;
error_page 504 = @edge_504;
location @edge_502 {
default_type text/plain;
add_header X-Edge-Observed "true" always;
add_header X-Request-ID $trace_id always;
add_header X-Debug-Generated-By "edge" always;
return 502 "edge generated 502\n";
}
location @edge_504 {
default_type text/plain;
add_header X-Edge-Observed "true" always;
add_header X-Request-ID $trace_id always;
add_header X-Debug-Generated-By "edge" always;
return 504 "edge generated 504\n";
}
}
}
The X-Debug-Generated-By headers are lab instrumentation, not standard NGINX headers.
I added them so that the experiments make the response origin visually obvious.
In a real environment, correlation IDs and logs are much more important.
Run everything with Docker Compose
compose.yaml:
services:
app:
build:
context: ./app
ports:
- "8000:8000"
gateway:
image: nginx:1.31.4-alpine3.24
volumes:
- ./gateway/nginx.conf:/etc/nginx/nginx.conf:ro
ports:
- "18081:80"
depends_on:
- app
edge:
image: nginx:1.31.4-alpine3.24
volumes:
- ./edge/nginx.conf:/etc/nginx/nginx.conf:ro
ports:
- "8080:80"
depends_on:
- gateway
Start the environment:
docker compose up -d --build
Then check:
docker compose ps
Validate the loaded NGINX configurations:
docker compose exec gateway nginx -t
docker compose exec edge nginx -t
Both should end with:
syntax is ok
test is successful
Establish a healthy baseline first
Always prove the healthy path before deliberately breaking it.
Test the application:
curl -i http://localhost:8000/health
Then Gateway → Application:
curl -i http://localhost:18081/health
Then the complete path:
curl -i http://localhost:8080/health
The complete request returned:
HTTP/1.1 200 OK
X-Gateway-Observed: true
X-Edge-Observed: true
X-Request-ID: d66701d1ec8700e4bf111ab99b49372d
I then measured a normal /fast request:
curl -sS -D - -o /dev/null \
-w '\nstatus=%{http_code}\nconnect=%{time_connect}s\nttfb=%{time_starttransfer}s\ntotal=%{time_total}s\n' \
http://localhost:8080/fast
One saved run produced:
status=200
connect=0.000248s
ttfb=0.010474s
total=0.010642s
Both proxies recorded successful upstream responses.
Gateway:
layer=gateway
status=200
upstream_status=200
Edge:
layer=edge
status=200
upstream_status=200
Now we have a control case.
Failure 1: Immediate upstream connection refusal
The /bad-gateway route points Gateway at:
app:9999
The application container exists and is reachable, but nothing is listening on that port.
Request it through the full chain:
curl -sS -D - http://localhost:8080/bad-gateway
The response was:
HTTP/1.1 502 Bad Gateway
X-Gateway-Observed: true
X-Debug-Generated-By: gateway
X-Edge-Observed: true
gateway generated 502
The measured request:
status=502
connect=0.000342s
ttfb=0.001889s
total=0.002027s
Gateway's error log explained why:
connect() failed (111: Connection refused) while connecting to upstream
Its access log showed:
layer=gateway
status=502
upstream_status=502
upstream_addr=172.20.0.2:9999
Edge recorded:
layer=edge
status=502
upstream_status=502
The sequence is therefore:
Client
|
v
Edge reached successfully
|
v
Gateway reached successfully
|
X
app:9999 connection refused
Gateway generates 502
Edge forwards 502
The client status alone does not show that sequence.
The adjacent logs do.
A backend being "down" did not always produce a 502
I also tried a less controlled experiment:
docker compose stop app
I initially expected another 502.
That did not happen in this environment.
The gateway attempted to connect to the application's previous container address, but the connection attempt did not immediately fail.
Instead, it waited until:
proxy_connect_timeout 2s;
expired.
The client received:
HTTP/1.1 504 Gateway Time-out
X-Debug-Generated-By: gateway
with:
status=504
connect=0.000293s
ttfb=2.003997s
total=2.004799s
Gateway logged:
upstream timed out ... while connecting to upstream
Compare that with the closed-port experiment:
Immediate connection refusal
-> 502 in this lab
Connection attempt times out
-> 504 in this lab
I would not generalize this into:
stopped container = 504
The network behavior of a stopped workload can vary by environment.
The useful lesson is:
"The backend is down" is not enough information to predict the status code. You need to determine how the upstream connection failed.
After this experiment I restored the application:
docker compose start app
Failure 2: Gateway connects successfully but times out reading the response
Now we create a very different failure.
The application waits:
5 seconds
Gateway waits:
2 seconds
Edge waits:
10 seconds
So Gateway must be the first layer to give up.
The route is:
location = /gateway-timeout {
proxy_pass http://app:8000/slow?seconds=5;
proxy_set_header X-Request-ID $trace_id;
proxy_connect_timeout 2s;
proxy_read_timeout 2s;
}
Request:
curl -sS -D - \
-o /dev/null \
-w '\nstatus=%{http_code}\nttfb=%{time_starttransfer}s\ntotal=%{time_total}s\n' \
http://localhost:8080/gateway-timeout
The result:
HTTP/1.1 504 Gateway Time-out
X-Gateway-Observed: true
X-Debug-Generated-By: gateway
X-Edge-Observed: true
status=504
ttfb=2.005843s
total=2.005997s
A saved run measured:
status=504
connect=0.000261s
ttfb=2.004355s
total=2.005004s
The application proves that the request successfully reached it:
layer=app
path=/slow
sleeping=5.0s
Gateway logged:
upstream timed out (110: Operation timed out)
while reading response header from upstream
This is different from the previous connect-timeout error:
while connecting to upstream
So we have now produced two client-visible 504 responses during different phases:
Connection phase
-> timeout while connecting to upstream
Response phase
-> timeout while reading response header
According to the NGINX proxy module documentation, proxy_connect_timeout controls the timeout for establishing the upstream connection, while proxy_read_timeout controls the interval between successive read operations from the upstream.
Those are different failure boundaries.
Why the application later logged 200
There was another interesting result.
The application continued sleeping after Gateway had already returned the 504.
Eventually it reached:
"GET /slow?seconds=5 HTTP/1.1" 200
and then:
layer=app client_connection_closed=true
That application log entry does not mean the original client received 200 OK.
Our Python handler logs the intended response status as it starts generating the response.
By that point, Gateway had already timed out and closed the upstream connection.
When Python later tried writing the body, the socket was gone and the handler caught BrokenPipeError.
The client had already received:
504 Gateway Time-out
This is a useful reminder that an application-side status log and a client-visible status are observations at different boundaries.
Failure 3: Same client-visible 504, different proxy
Now reverse the timeout relationship:
Application delay 5s
Gateway read timeout 10s
Edge read timeout 2s
Gateway is willing to wait long enough for the application.
Edge is not.
Gateway:
location = /edge-timeout {
proxy_pass http://app:8000/slow?seconds=5;
proxy_set_header X-Request-ID $trace_id;
proxy_connect_timeout 2s;
proxy_read_timeout 10s;
}
Edge:
location = /edge-timeout {
proxy_pass http://gateway/edge-timeout;
proxy_set_header X-Request-ID $trace_id;
proxy_connect_timeout 2s;
proxy_read_timeout 2s;
}
Run:
curl -sS -D - \
-o /dev/null \
-w '\nstatus=%{http_code}\nttfb=%{time_starttransfer}s\ntotal=%{time_total}s\n' \
http://localhost:8080/edge-timeout
The client again saw:
HTTP/1.1 504 Gateway Time-out
But now the diagnostic header changed:
X-Debug-Generated-By: edge
The measured request:
status=504
connect=0.000274s
ttfb=2.005233s
total=2.005678s
From the client's point of view, this looked almost identical to the previous gateway timeout.
Internally, it was completely different.
The 499 that reveals the real sequence
This was the most useful part of the experiment.
The request ID was:
d9921f9d15f2df8ea396c7baf6c81d5a
I correlated it across the three services:
docker compose logs --no-color app gateway edge \
| grep 'd9921f9d15f2df8ea396c7baf6c81d5a'
The application recorded:
layer=app
trace=d9921f9d15f2df8ea396c7baf6c81d5a
path=/slow
sleeping=5.0s
Edge recorded:
layer=edge
trace=d9921f9d15f2df8ea396c7baf6c81d5a
status=504
request_time=2.003
upstream_response_time=2.003
request="GET /edge-timeout HTTP/1.1"
Gateway recorded:
layer=gateway
trace=d9921f9d15f2df8ea396c7baf6c81d5a
status=499
upstream_status=-
request_time=2.003
upstream_response_time=2.003
request="GET /edge-timeout HTTP/1.1"
Edge's error log also said:
upstream timed out ... while reading response header from upstream
The sequence was:
app still processing
|
|
Client -> Edge -> Gateway -> Application
|
| 2 second timeout expires
|
+--> Edge returns 504 to Client
|
+--> Edge closes its request to Gateway
From Gateway's point of view, Edge is the client.
Gateway was willing to keep waiting for the application, but its client disconnected first.
NGINX therefore recorded 499.
NGINX defines 499 internally as NGX_HTTP_CLIENT_CLOSED_REQUEST. It is not a standard HTTP status sent by the external client; it is used by NGINX to represent the client closing the request before a response could be completed.
The application continued its five-second operation and later attempted to send its 200.
By then, the original request path was already gone.
So one request resulted in:
External client 504
Edge 504
Gateway 499
Application eventually attempts 200
All four observations are valid at their respective boundaries.
Be careful interpreting $upstream_status alone
There was one more subtle result.
For the edge-generated timeout, Edge logged:
status=504
upstream_status=504
It would be tempting to conclude:
Gateway returned HTTP 504 to Edge.
But Gateway's own log for the same trace ID showed:
status=499
upstream_status=-
and Edge's error log explicitly said that Edge itself timed out while reading from its upstream.
So in this experiment, using Edge's $upstream_status alone to identify which process emitted the client-visible 504 would have produced the wrong conclusion.
Correlation across adjacent layers gave us the actual sequence.
This is one reason I prefer to combine:
local status
upstream status
request timing
upstream timing
error log
correlation ID
instead of treating any single field as the whole story.
The NGINX upstream module documentation describes the available upstream variables, including $upstream_status, $upstream_connect_time, $upstream_header_time, and $upstream_response_time.
A practical debugging sequence
When a request crosses several gateways or proxies, I would debug it in this order:
Propagate one correlation ID through every hop.
At every proxy, log the local response status and upstream information.
Record timing for different phases of the request. For NGINX, useful fields include:
$request_time
$upstream_connect_time
$upstream_header_time
$upstream_response_time
- Read the error message carefully.
These are materially different:
while connecting to upstream
and:
while reading response header from upstream
Find the same request ID at the adjacent layer before changing timeout values.
Determine which component actually generated the client-visible error.
Only then investigate why the upstream was unavailable or slow.
A timeout setting is often where a problem becomes visible.
It is not necessarily where the underlying problem originated.
Results
The deterministic experiments produced:
| Scenario | Client result | Response generated by | Observed time |
|---|---|---|---|
| Healthy request | 200 |
Application response propagated | ~10 ms |
| Reachable host, closed upstream port | 502 |
Gateway | ~2 ms |
| Slow application, Gateway waits 2s | 504 |
Gateway | ~2.005 s |
| Slow application, Gateway waits 10s, Edge waits 2s | 504 |
Edge | ~2.006 s |
The additional stopped-container experiment produced another Gateway 504, but because the connection attempt timed out, not because Gateway timed out waiting for application response data.
So the model I would avoid is:
502 = backend down
504 = backend slow
A more useful set of questions is:
Which hop failed?
During which phase did it fail?
Which component generated the response?
What did the adjacent hop record for the same request?
Once those questions are answered, a multi-proxy 502 or 504 becomes much easier to localize.
References
- NGINX HTTP proxy module
- NGINX HTTP upstream module and upstream variables
- NGINX development guide, including internal HTTP status definitions
Disclosure
AI assistance was used for technical research and editorial review while developing this article. I manually built and ran the lab, and the configurations, commands, response codes, timings, request traces, and log behavior described above were validated against the running environment before publication.
Top comments (0)