A deploy that silently does nothing is worse than a deploy that fails, and we shipped two flavours of it before either one announced itself.
The first one: a deploy with an expiry date
The fast dev loop is to tar a built dist/ into /usr/share/nginx/html of a running container. It works. The page updates, the change is live, and the loop is seconds instead of minutes.
It survives docker restart. It does not survive a docker compose up -d that recreates the container, because the new container comes up from the image, and everything deployed that way reverts to whatever the image was built with.
Nothing errors when this happens. No log line, no failed request; the app simply becomes an older app, and every measurement taken afterwards is of software nobody built.
It cost one session a reskin, a glyph set and two bug fixes on 2026-08-27. It was noticed only because the served bundle hash changed — not because anything complained.
The second one: a recreate is not a rebuild
The subtler version has no hot-patching in it at all.
Measured the same day: the running image had been built the previous evening and carried 19 supervisord programs against the tree's 20. The missing one was cert-watch, the certificate-expiry watcher. So on a deployment serving portal custom domains, the alarm for a certificate about to expire was not running — and the image's own healthcheck was one probe short of being able to notice that it wasn't.
The instinct here is --force-recreate, and it does not help. It recreates the container from the same image. Only docker compose build produces a new one. The distance between those two commands is one word, and the feedback that tells them apart is nothing at all: docker ps, docker inspect and the health status all look identical either way.
Why "it looks fine" is not evidence
Both failures share a shape worth naming, because it generalises past Docker.
A deployment has two states that present identically to every casual check: running what you built, and running something older that still starts cleanly. Nothing in the second state is broken enough to complain. The process is up. The healthcheck passes — it is the old healthcheck, and it passes honestly. Requests return 200. The only difference is that the thing answering them is not the thing in your tree.
The measurement is what gives the game away, in both directions. A bundle budget that improved without anybody optimising anything. A test that started passing without anybody fixing it. Both of those are the same report: you are measuring a build you did not make.
What we do about it
Three things, none of them clever.
Name the fast path as a dev loop, in the script itself. scripts/deploy-to-container.sh opens with the line NOT A DEPLOY. See docs/deployment.md — a release is an image. This is the dev loop. A comment is not a gate, but the cost of this failure is entirely in the belief that a deploy happened, so the belief is what to attack.
Have a command that answers "what is actually running". node scripts/env-preflight.mjs diffs the container's [program:] set against docker/supervisord.conf and names the programs that are missing. That is the check that catches the 19-against-20 case, and it exists because nothing in Docker's own output does.
Give the database question its own tool, with three outcomes. npm run ops:drift compares the init files against the database you are serving from, and it reports IN SYNC (0), DRIFT (1) or NOT MEASURED (2) when the database could not be read. The third one matters more than it looks: an unmeasured state must never be readable as a good one. Most tooling folds "I couldn't check" into "fine", which is how you end up trusting a green light that was never on.
The rule this leaves us with
If you recreate the container, redeploy. And if a number moves in a direction nobody worked for, check which build is serving before you believe it.
That is not a Docker lesson. It is what happens any time the artefact you deploy and the artefact you edit are two different objects, and only one of them is on screen.
Top comments (0)