Originally published on kuryzhev.cloud
Every container reports healthy, docker ps shows no restarts, and the browser still throws a certificate warning. This is the recurring shape of nginx-proxy acme-companion troubleshooting: nothing has crashed, and no certificate has issued.
Symptoms
The browser shows "Not Secure" or ERR_CERT_AUTHORITY_INVALID, while the acme-companion container itself sits there running with no exit code. A running container is not evidence of a functioning ACME client — it only proves the process hasn't died.
docker logs <acme-companion> usually shows one of the real acme.sh/acme-companion failure strings: "Verify error", "Invalid status", or a line like "Creating/renewal ... certificate for '<domain>' failed". The domain resolves fine in a browser and plain HTTP works, but the ACME validation path specifically doesn't.
Port 443 is often still serving nginx-proxy's default self-signed certificate. That default is expected right after first startup — it's only a symptom once it's still there hours later. Worth checking separately: if /etc/nginx/certs/default.crt is missing entirely (verify with docker exec <nginx-proxy> ls -la /etc/nginx/certs/default.crt), port 443 refuses the connection outright instead of serving an untrusted cert. That's a different failure signature pointing at a broken nginx-proxy startup, not a stalled acme-companion.
Together — running containers, real ACME error strings in the logs, and a stuck self-signed cert — these point at a broken link in the challenge or volume chain, not a Let's Encrypt outage.
Root cause
nginx-proxy and acme-companion aren't one service; they're three moving parts coordinating over shared state — nginx-proxy (docker-gen plus nginx), acme-companion (the ACME client), and the Docker socket that ties them together via container labels like VIRTUAL_HOST and LETSENCRYPT_HOST.
The HTTP-01 flow itself is easy to get backwards: acme-companion writes the challenge token into the shared html volume, and nginx-proxy — not acme-companion — serves it on port 80. acme-companion has no listener of its own; it never receives an inbound HTTP request. If port 80 is blocked, redirected, or intercepted upstream, nginx-proxy never gets asked for the file, and the failure happens on nginx-proxy's side of the handshake even though the error surfaces in acme-companion's logs.
Three failure modes show up repeatedly in nginx-proxy's and acme-companion's own issue trackers: the challenge path never reaching nginx-proxy at all, containers that mount volumes under the same name but not actually shared, and env vars like LETSENCRYPT_EMAIL or LETSENCRYPT_HOST missing or mismatched against VIRTUAL_HOST.
A fourth cause is easy to miss: acme-companion tries to autodetect which container is nginx-proxy over the Docker socket, and that autodetection isn't guaranteed, especially with custom container names or more than one nginx-proxy on the host. Set the com.github.nginx-proxy.nginx-proxy=true label on the nginx-proxy container, or the NGINX_PROXY_CONTAINER env var on acme-companion pointing at its container name, and stop relying on the guess.
Watch out: env vars cannot be changed on a running container — you have to recreate it, and that recreation is itself a Docker event docker-gen already watches for. If config still doesn't re-render after a recreate, the real problem is usually a dead docker-gen watcher or a stale socket mount, not a timing issue with variables. Restarting nginx-proxy resets the watcher.
Fix #1 — Verify the ACME HTTP-01 path is actually reachable
Before touching any container config, rule out network and DNS. Confirm the domain's A/AAAA record points at the host's public IP — Let's Encrypt validates from the internet, not from inside a VPC or local network.
Check whether anything upstream blocks port 80: a Cloudflare orange-cloud proxy, a cloud load balancer, or a security group rule. HTTP-01 needs plain HTTP on port 80, even on a site that forces HTTPS everywhere else. Gotcha: an aggressive "redirect HTTP to HTTPS" rule at the edge — not inside nginx-proxy — silently swallows every challenge request before it reaches the container.
curl -I http://yourdomain/.well-known/acme-challenge/anything
# expect: HTTP/1.1 404, with a "Server: nginx" header
# a 404 alone can come from the proxied app instead of nginx-proxy;
# confirm by dropping a real file into the shared html volume and curling it directly
docker exec <nginx-proxy> sh -c 'mkdir -p /usr/share/nginx/html/.well-known/acme-challenge && echo ok > /usr/share/nginx/html/.well-known/acme-challenge/probe'
curl http://yourdomain/.well-known/acme-challenge/probe
# expect: "ok"
If the probe file comes back as "ok", nginx-proxy is serving the challenge path correctly and the html volume is genuinely shared. A timeout, or a response from something that isn't nginx, means the network layer is the problem — fix that before touching any compose file.
Fix #2 — Fix shared volumes and required env vars
Containers that look configured but don't actually share state come up repeatedly in nginx-proxy acme-companion troubleshooting threads. Confirm all three containers mount the same named volumes: certs, html, and vhost (mounted at /etc/nginx/vhost.d — the volume name and the mount path aren't the same string, which trips up anyone grepping compose files for "vhost.d"). acme-companion additionally mounts acme plus the Docker socket, read-only.
Set VIRTUAL_HOST and LETSENCRYPT_HOST identically on the app container. For multiple domains, use comma-separated values, but every value in LETSENCRYPT_HOST must also appear in VIRTUAL_HOST — a domain listed only under LETSENCRYPT_HOST is a recurring copy-paste mistake. Also set LETSENCRYPT_EMAIL (or DEFAULT_EMAIL globally on acme-companion). A missing email doesn't fail issuance outright, but it's the only way to recover the ACME account or hear from Let's Encrypt about a required action — Let's Encrypt discontinued expiration notification emails in June 2025, so don't rely on this field for renewal alerting; use an independent expiry check instead.
services:
nginx-proxy:
image: nginxproxy/nginx-proxy:1.7
restart: unless-stopped
ports:
- "80:80" # required for ACME HTTP-01 challenge
- "443:443"
labels:
- "com.github.nginx-proxy.nginx-proxy=true"
volumes:
- certs:/etc/nginx/certs:ro # shared: read-only here, RW on acme-companion
- vhost:/etc/nginx/vhost.d
- html:/usr/share/nginx/html
- /var/run/docker.sock:/tmp/docker.sock:ro
acme-companion:
image: nginxproxy/acme-companion:2.4
restart: unless-stopped
depends_on:
- nginx-proxy
volumes:
- certs:/etc/nginx/certs:rw # must be RW here
- vhost:/etc/nginx/vhost.d
- html:/usr/share/nginx/html
- acme:/etc/acme.sh
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
DEFAULT_EMAIL: ops@example.com
NGINX_PROXY_CONTAINER: nginx-proxy # skip autodetection instead of the label above
app:
image: your-app:latest
restart: unless-stopped
environment:
VIRTUAL_HOST: example.com,www.example.com
LETSENCRYPT_HOST: example.com,www.example.com
LETSENCRYPT_EMAIL: ops@example.com
volumes:
certs:
vhost:
html:
acme:
Gotcha: older tutorial snippets sometimes define certs as visually identical but separately declared volumes per service. It breaks the handshake with no obvious error — container names differ, volume names look right, but the data never actually crosses between services.
Fix #3 — Diagnose CA/rate-limit issues without burning production quota
Sometimes the challenge passes and no certificate appears anyway. That's usually a CA-side rejection, not a container bug.
# Diagnostic sequence — run in this order
# 1. Is acme-companion seeing your container's env vars?
docker logs <acme-companion-container> --tail 100 | grep -i example.com
# 2. Confirm volumes are actually shared, not per-container copies
docker inspect <nginx-proxy-container> --format '{{range .Mounts}}{{.Name}} -> {{.Destination}}{{"\n"}}{{end}}'
docker inspect <acme-companion-container> --format '{{range .Mounts}}{{.Name}} -> {{.Destination}}{{"\n"}}{{end}}'
# both should list the SAME volume names for certs/vhost/html
# 3. Check for a rate-limit rejection specifically
docker logs <acme-companion-container> 2>&1 | grep -iE "rate limit|too many certificates|ratelimited"
# 4. Confirm the served cert matches the domain and isn't the default
openssl s_client -connect example.com:443 -servername example.com </dev/null 2>/dev/null \
| openssl x509 -noout -issuer -dates
Historical figures on the Let's Encrypt rate-limit page put failed validations at roughly 5 per account/hostname per hour and 50 issued certificates per registered domain per week — but Let's Encrypt revised its rate-limit model in 2025 around rolling New Order windows, so treat any specific number here as dated and check the live page before planning around it. That weekly limit is a rolling window, not a flat lockout: hitting it mid-incident doesn't force a fixed week-long wait, it blocks new issuance until older certificates in the window age out, which can still be slow but isn't absolute.
Switch acme-companion to Let's Encrypt's staging directory before debugging repeatedly against production: set ACME_CA_URI to https://acme-staging-v02.api.letsencrypt.org/directory. Removing that override alone doesn't finish the job — acme-companion caches the staging cert/key under the certs volume and the staging ACME account under the acme volume. Delete the staging cert files for that domain from certs and the account directory under acme, then restart acme-companion so it registers a fresh production account and reissues from scratch. Skip that cleanup and the untrusted staging cert just keeps getting served, with no reissue triggered.
If logs show a successful challenge but nginx never reloads with the new cert, check docker-gen's own logs next. A syntax error in a custom nginx snippet under vhost.d/<domain> can block the template render entirely, even after a valid certificate has already landed in the certs volume.
Prevention
Add a monitoring check for certificate expiry instead of trusting acme-companion's internal renewal loop as the only signal. A Prometheus blackbox exporter probe, or a cron job running openssl x509 -checkend, catches a silent renewal failure weeks before it becomes an outage — especially now that Let's Encrypt no longer sends expiry emails.
Pin nginxproxy/nginx-proxy and nginxproxy/acme-companion to explicit tags, as in the compose example above, instead of latest. An untested major-version bump landing exactly when a cert is due for renewal is a bad time to discover a breaking change.
The older jrcs/letsencrypt-nginx-proxy-companion repository, still referenced in some tutorials, has been archived by its maintainers — check the repository's own deprecation notice on GitHub before using it for anything new.
Document the volume and env-var contract directly in the compose file with comments, so the next edit doesn't silently break the certs/html/vhost chain. Restrict the Docker socket mount on acme-companion to read-only, and never expose that socket to less-trusted app containers; consult the Docker security documentation for the tradeoffs of socket-mounting in multi-container setups.
For new deployments, weigh whether Traefik or Caddy — both with built-in ACME support and no companion-container choreography — sidesteps this exact class of failure. nginx-proxy's split-container design is legacy-compatible and well understood, but it has more moving parts and shared state to misconfigure than a single binary managing its own certificate lifecycle. If the current stack already works and the volume contract is documented, there's no urgent reason to migrate. If nginx-proxy acme-companion troubleshooting keeps recurring across routine compose edits, that recurrence is the signal worth acting on. For related Docker networking and TLS patterns, see the DevOps_DayS archive.
Top comments (0)