DEV Community

Sanghun Yun
Sanghun Yun

Posted on Originally published at gearflowlab.com

How to Fix Nginx 502 Bad Gateway with Docker (Resolving Upstream Connection Refused)

#nginx #docker #devops #webdev

An HTTP 502 Bad Gateway error in an Nginx + Docker architecture is one of the most common—and frustrating—production roadblocks. Unlike a 504 Gateway Timeout where an upstream server is simply slow, a 502 Bad Gateway means Nginx is actively unable to complete the TCP handshake with your upstream container or received an invalid response header.

In this guide, we’ll break down the socket mechanics behind connect() failed (111: Connection refused), examine the three most common architectural misconfigurations, and implement bulletproof Docker DNS resolvers.

The System Architecture & Failure Flow

Here is how traffic flows through Nginx and Docker, and where the connection refused error is triggered:

flowchart TD
    subgraph Client["1. External Traffic Layer"]
        Browser["Web Client / API Request<br/>(HTTPS GET / POST)"]
    end

    subgraph NginxProxy["2. Nginx Reverse Proxy Container"]
        NginxListener["Nginx Listener (Port 80 / 443)"]
        NginxDNS["Nginx Dynamic Resolver<br/>(resolver 127.0.0.11 valid=10s)"]
        ProxyPass["proxy_pass $upstream_app<br/>(Prevents startup IP caching trap)"]
    end

    subgraph DockerNetwork["3. Shared Docker Bridge Network (app-tier)"]
        DockerDNS["Docker Embedded DNS<br/>(127.0.0.11:53)"]

        subgraph BackendApp["Backend Application Container"]
            PortListen["Bound to 0.0.0.0:3000<br/>(Container Interface)"]
            AppLogic["Node.js / Python / Go Server"]
        end

        subgraph StaleTrap["Failure Scenario: 502 Bad Gateway"]
            DeadSocket["connect() failed (111: Connection refused)<br/>- App bound to 127.0.0.1 only<br/>- Unshared Docker network<br/>- Stale IP after container recreate"]
        end
    end

    Browser -->|1. Incoming Request| NginxListener
    NginxListener --> ProxyPass
    ProxyPass -->|2. Resolve container hostname| NginxDNS
    NginxDNS -->|3. Query 127.0.0.11| DockerDNS
    DockerDNS -->|4. Return active container IP| NginxDNS
    ProxyPass -->|5. Clean TCP Stream| PortListen
    PortListen --> AppLogic
    ProxyPass -.->|Misconfiguration| DeadSocket

1. The Anatomy of POSIX Error 111 (Connection Refused)

When you inspect Nginx container logs:

docker logs <nginx_container> --tail 50
Enter fullscreen mode Exit fullscreen mode

You will almost always find this exact signature:

2026/10/10 05:14:22 [error] 28#28: *104 connect() failed (111: Connection refused) 
while connecting to upstream, client: 198.51.100.45, server: api.gearflowlab.com, 
request: "POST /v1/auth/login HTTP/1.1", upstream: "http://172.22.0.4:8080/v1/auth/login", 
host: "api.gearflowlab.com"
Enter fullscreen mode Exit fullscreen mode

In POSIX socket networking, 111 means the target host responded with an immediate TCP RST (Reset) flag. Nginx sent a SYN packet to establish a TCP session, but the operating system at 172.22.0.4 replied: "No process is listening on port 8080."

2. Root Cause 1: Binding to 127.0.0.1 Inside Containers

The #1 reason for a 502 Bad Gateway in Docker: your application server binds to localhost or 127.0.0.1.

In Docker, every container has an isolated network namespace and its own loopback interface (lo).

  • When your app listens on 127.0.0.1:3000, it is only accessible from inside that same container.
  • Nginx connects via the Docker bridge network interface (eth0, e.g., 172.22.0.4). Because your app is not listening on eth0, the kernel drops the packet with Connection refused.

The Solution: Bind to 0.0.0.0

In Node.js / Express:

// Correct: Listen on all network interfaces
const PORT = process.env.PORT || 3000;
app.listen(PORT, '0.0.0.0', () => {
  console.log(`Server listening on 0.0.0.0:${PORT}`);
});
Enter fullscreen mode Exit fullscreen mode

In Python FastAPI / Uvicorn:

uvicorn main:app --host 0.0.0.0 --port 8000
Enter fullscreen mode Exit fullscreen mode

3. Root Cause 2: The Stale IP Dynamic DNS Caching Trap

This is the silent killer of container deployments.

When you define a static proxy_pass in Nginx:

location / {
    proxy_pass http://backend-api:3000;
}
Enter fullscreen mode Exit fullscreen mode

Nginx queries DNS for backend-api once during boot or reload.

When you subsequently update your backend service (docker compose up -d --no-deps backend-api), Docker destroys the old container and creates a new container with a new IP address (e.g., 172.22.0.4 becomes 172.22.0.5).

Nginx continues sending packets to 172.22.0.4 until you reload Nginx. Every request fails with 502 Bad Gateway (Connection refused).

The Fix: Variable proxy_pass with Docker DNS

By using an Nginx variable inside proxy_pass and configuring Nginx's resolver to point to Docker's internal DNS daemon (127.0.0.11), Nginx dynamically re-resolves the IP every few seconds:

server {
    listen 80;
    server_name api.gearflowlab.com;

    # 127.0.0.11 is Docker's embedded DNS server
    resolver 127.0.0.11 valid=5s ipv6=off;
    resolver_timeout 3s;

    # Storing in a variable forces runtime DNS resolution
    set $backend_service "http://backend-api:3000";

    location / {
        proxy_pass $backend_service;
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;

        # Retry on transient rollout hiccups
        proxy_next_upstream error timeout http_502;
        proxy_next_upstream_tries 3;
    }
}
Enter fullscreen mode Exit fullscreen mode

4. Root Cause 3: Default Bridge vs. User-Defined Bridge Networks

Containers on Docker's default bridge network do not support automatic DNS resolution by container name.

Always ensure both services share a named, user-defined bridge network in docker-compose.yml:

version: '3.8'

services:
  reverse-proxy:
    image: nginx:alpine
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./nginx.conf:/etc/nginx/conf.d/default.conf:ro
    networks:
      - internal-net
    depends_on:
      backend-api:
        condition: service_healthy

  backend-api:
    image: my-node-api:latest
    environment:
      - HOST=0.0.0.0
      - PORT=3000
    networks:
      - internal-net
    healthcheck:
      test: ["CMD-SHELL", "wget -qO- http://127.0.0.1:3000/health || exit 1"]
      interval: 5s
      timeout: 3s
      retries: 3
      start_period: 10s

networks:
  internal-net:
    driver: bridge
Enter fullscreen mode Exit fullscreen mode

Notice the condition: service_healthy directive: Nginx will not accept traffic until the backend passes its internal healthcheck, eliminating startup race condition 502s.

Diagnostic Cheat Sheet

Run these commands inside your environment to pinpoint the root cause in 30 seconds:

# 1. Test DNS resolution from Nginx container
docker exec -it reverse-proxy getent hosts backend-api

# 2. Test HTTP response directly from Nginx container
docker exec -it reverse-proxy wget -qO- http://backend-api:3000/health

# 3. Check what port the app is listening on inside its container
docker exec -it backend-api netstat -tulpn | grep LISTEN
Enter fullscreen mode Exit fullscreen mode

Originally published on GearFlow Lab.

Top comments (0)