Killing the Shared Proxy: Direct Routing to Per-Customer Containers on Docker Swarm

We already ran one container per customer — but every request was relayed through a single shared one. Here's how we cut it out, and everything that turned out to be attached to it.

Our platform runs one container per customer on Docker Swarm. Each customer gets an isolated process with their own config, their own secrets, their own AI agent. (In the jargon this is a tenant — but they're customers, so I'll call them that.)

Clean isolation. And then every single request went through one shared container first.

This is the story of getting that container out of the path.

The shape of the problem

The architecture was a shared "control plane" container plus N per-customer containers — and everything entered through the shared one:

The shared container did the sensible thing: authenticate the JWT, check the subscription, then reverse-proxy the request to the right customer container over the overlay network.

It worked. It also meant every customer's every request — every chat load, every analytics query, every dashboard poll — was buffered and relayed through a single Node process. That process did a JWT verify, two database reads, and a full request/response relay before the customer's own container even saw the request.

One container. All customers. Both a bottleneck and a single point of failure, for traffic that had a perfectly good home elsewhere.

The customer containers were already doing the actual work — the AI, the message handling, the data. They just couldn't be reached without going through the middleman.

Why you can't just point nginx at the container

The obvious fix: have the edge route straight to customer-{id} and skip the relay.

Except the host nginx cannot resolve customer-{id}.

Customer containers live on a Docker overlay network. Their DNS names are served by Docker's embedded resolver at 127.0.0.11, which only exists inside containers attached to that network. The host nginx is a system process — it isn't on the overlay, so those names may as well not exist.

That leaves two options:

Publish a host port per customer. customer-a → :4001, customer-b → :4002, and nginx maps them. This breaks the moment you have more than one node — Swarm can schedule a container anywhere, and it moves on redeploy — so nginx would need to know which node holds which customer, and update on every reschedule. It also needs config regeneration on every signup and offboard, and it exhausts ports. Dead end, and worse, a dead end you'd have to rip out later.

Put an nginx inside the overlay network. It can resolve Docker DNS natively. The host nginx terminates TLS and forwards to it; it does the lookup and the routing.

We went with the second one. It works identically on one node and on many, which matters — the whole point was to avoid building something we'd have to redo.

Deterministic routing: the subdomain is the address

Give each customer a subdomain whose first label is their container's short ID:

{shortId}.api.example.com  →  customer-{shortId} container

That single decision removes an entire category of work. There's no service discovery, no per-customer config file, and no reload when someone signs up or leaves — the subdomain label is the container name. The edge doesn't look anything up. It does string interpolation.

The result splits the world in two — a control plane that genuinely needs to be shared, and everything else going straight to the container that owns it:

The host nginx can't resolve the container names, so it terminates TLS and hands off to the edge — which lives on the overlay network and can. That's the whole trick.

The whole edge config is about forty lines:

# Docker's embedded DNS. Not optional — see below.
resolver 127.0.0.11 valid=10s ipv6=off;

server {
    listen 4100;
    server_name "~^(?<shortid>[0-9a-f]{8})\..+$";

    location /api/ {
        set $upstream "customer-${shortid}";
        proxy_pass http://$upstream:4000;
        # ... standard proxy headers
    }
}

Two things in there are load-bearing and easy to get wrong:

resolver is required. Without it, nginx resolves upstream names once at config load and caches the IP forever. Containers restart and get new IPs — you'd 502 until someone reloaded nginx.

The variable in proxy_pass is what forces per-request resolution. Write proxy_pass http://customer-abc:4000 with a literal hostname and nginx resolves it at startup regardless of your resolver. The set $upstream indirection is what makes it re-resolve. This is a long-standing nginx quirk, and it's the difference between "works" and "works until a container restarts."

And it's multi-node ready for free: overlay DNS resolves cluster-wide, so when containers eventually spread across worker nodes, this config doesn't change at all.

The traffic moved. So did everything attached to it.

This is the part that's easy to underestimate. The shared proxy wasn't only forwarding requests — it was the place a bunch of other things happened to live. Take it out of the path and you find out what was quietly depending on it.

Auth and gating

The shared container authenticated the JWT and checked the subscription before proxying. Remove the proxy and that gate is simply gone.

So each customer container now does its own:

• JWT verification — it already had the signing keys. Same image, same secret bootstrap; the code path was there, just skipped because "the proxy already did it."
• Subscription check — cached, so it's a map lookup on the hot path, not a database read per request.
• Active-account check — replacing the proxy's "is this account live?" gate. A paused customer's container might still be running for a moment during scale-down; it now refuses to serve.

The guard that only exists because containers are public now

Here's the one that matters. Once containers are individually addressable from the internet, a request can arrive at the wrong one.

A container must refuse to serve any request that isn't for its own customer. The URL carries an ID; the container knows its own. If they don't match — 404, and log it.

Without that check, a perfectly valid token for customer A, pointed at customer B's subdomain, would be processed by B's container — using B's config, B's credentials, B's connections. The same check goes on the WebSocket handshake.

That single guard is the entire reason public per-customer addressing is safe rather than reckless. If you do this, do that.

Every gate the shared proxy used to run now runs on the container itself, in front of the route:

The identity bound step is the new one. The rest existed already — they were just running somewhere else.

Inbound webhooks

Third-party webhooks (in our case, inbound messages) were also landing on the shared container, which validated the signature and forwarded to the customer's container.

Now the webhook URL each customer registers points at their own subdomain. The message arrives directly, and their container validates the signature with their own secret. The shared container isn't involved in a single inbound message anymore.

The one we almost missed: WebSockets

We moved the HTTP traffic. Dashboards loaded straight from customer containers. The shared container's logs went quiet.

Then we looked at what was actually left in them:

socket_connected
socket_connected
POST /api/internal/emit-socket   200 5ms
POST /api/internal/emit-socket   200 3ms

The socket hub was still there. Every open browser tab, for every customer, held a live WebSocket to the shared container. And every message a customer's AI produced was making an internal HTTP call back to the shared container so it could fan out to that browser:

This is worse than the HTTP bottleneck we'd just removed, not better:

• An HTTP request completes and frees its resources.
• A WebSocket sits there. Memory and file descriptors grow with concurrent users, not with request rate.

We had removed the bottleneck that scales with traffic and left in place the one that scales with people logged in.

The fix was the same trick — the edge already routed /socket.io/, so the browser just connects to {shortId}.api.example.com instead. Each container now holds only its own customer's sockets and emits to them locally. No cross-container hop per message.

One nice wrinkle: the internal forward channel didn't disappear. It flipped direction. Billing and quota events are raised on the shared container (scheduled jobs, subscription lifecycle) but need to reach a customer's browsers, which now live on the customer's container. So shared forwards to the container instead of the container forwarding to shared. Same channel, opposite arrow:

The per-message path no longer touches the shared container at all. The only thing crossing containers now is the rare billing event — which is exactly the traffic that should originate centrally.

What stays on the shared container — and why that's correct

Not everything should move, and it's worth being explicit about why:

• Login — there is no customer container to route to yet. You don't know who you are.
• Billing — it's per-account, not per-customer-workspace. An account can own several.
• Onboarding — it creates the container. It can't run on it.
• Admin — cross-customer by definition.

That's the honest test for whether something belongs on a shared control plane: does it have customer context yet, and does it need exactly one customer? If either answer is no, it stays shared. Everything else goes to the customer's own container.

Where it landed

Browser
  ├─ auth, billing, onboarding, admin   → shared container
  │                                       (no customer context exists at login)
  └─ everything customer-scoped         → {shortId}.api.example.com
                                           → edge → customer container
                                              ├─ JWT verified here
                                              ├─ identity bound here
                                              ├─ subscription checked here
                                              └─ sockets held here

Inbound webhooks → {shortId}.api.example.com → customer container (directly)

The shared container is now out of the per-message path and out of the per-session path entirely. It handles login, billing, onboarding, admin — and nothing that happens while a customer is actually using the product.

Small traps, for anyone doing this

A few things cost us a deploy cycle each. In case they save you one:

nginx reads {8} in your regex as a config block. The parser treats { and } as block delimiters, so a repetition count in an unquoted regex produces the baffling error directive "server_name" is not terminated by ";". Quote the whole regex and it's fine.

Docker Swarm silently ignores the bind address in 127.0.0.1:4100:4100. It publishes on 0.0.0.0 and mentions this in a WARN you will scroll straight past. Our firewall happened to block the port — but "we were fine because of a rule we didn't write for this purpose" isn't a security posture. Check the firewall; don't trust the YAML.

Swarm configs are immutable. Editing the file and re-running stack deploy changes nothing — the config object already exists under that name. Remove the service and the config object, or version the config name so a rolling update actually rolls.

Wildcard certs can't use HTTP-01. *.api.example.com requires a DNS-01 challenge. If your DNS provider has no certbot plugin, you fall back to --manual, which does not auto-renew — and when it lapses, every customer subdomain goes TLS-invalid at the same moment. If your registrar's DNS can't do API-driven ACME, delegate just that subdomain to one that can. (Also: *.example.com does not match example.com. Separate names, separate certs.)

The takeaway

The interesting part of this wasn't the nginx config. It was noticing how much had accumulated in the shared path because it was there — auth, gating, webhook validation, the socket hub — none of which needed to be, and each of which was quietly making one container responsible for every customer.

If you already isolate per customer, the routing layer is worth the effort to match. And when you go looking for what's still shared, check the long-lived connections. HTTP load scales with requests. WebSocket load scales with humans.