DEV Community

octave Nkurunziza
octave Nkurunziza

Posted on Originally published at blog.hubfly.space

Docker health checks that catch real problems, explained simply

A plain guide to Docker health checks for your app, Postgres (pg_isready) and Redis, and three mistakes that make a check pass when it shouldn't.

Your app starts, tries to reach the database, and crashes because Postgres is still waking up. Or worse: everything looks "healthy", but Redis is rejecting every request because of a password typo. A health check is a small command that answers one question, over and over: is this service actually ready to do its job? Get it right and you can see when a service is truly ready, broken ones are easy to spot, and your users stop seeing errors during deploys.

In this guide you'll add working health checks to a web app, Postgres and Redis, deploy them as one stack, and learn the three mistakes that make a health check say "all good" when it isn't.

Try it on Hubfly space: the whole stack runs on the free shared tier (360 hours a month, no card needed).

TL;DR

  • What you get: a Compose file with checks for Postgres (pg_isready), Redis (redis-cli ping | grep -q PONG) and a web app (wget, shown with nginx), deployed in dependency order.
  • Who it's for: anyone running more than one container, from side projects to small production apps.
  • Time: about 15 minutes. The stack deploys in about a minute.
  • The three traps: redis-cli exits 0 even when the password is wrong; pg_isready can pass while Postgres is still running its first-boot scripts; and the image may not contain the tool your check calls.

What a health check is (and what it isn't)

A health check is a command that runs inside the container on a timer. Exit code 0 means healthy. Anything else counts as a failure. After a set number of failures in a row, the container is marked unhealthy.

Every check has four timing settings:

Setting What it means Docker default
interval How often the check runs 30s
timeout How long one run may take before it counts as failed 30s
retries Failures in a row before "unhealthy" 3
start_period Grace time after start when failures don't count 0s

Two things a health check does not do on its own:

  • It doesn't restart anything. A container that fails its check keeps running. Restart policies react to a process exiting, not to a failed check.
  • It doesn't fix your app's startup. It tells other services, and you, when something is ready. Your app should still retry its database connection for a few seconds on startup, because networks blink.

Why it matters: without checks, "the container is running" is the only signal you have, and a running container can be completely useless.

Step 1: Write a check for Postgres

db:
  image: postgres:16-alpine
  environment:
    POSTGRES_USER: shop
    POSTGRES_PASSWORD: change-me
    POSTGRES_DB: shop
  volumes:
    - pgdata:/var/lib/postgresql/data
  healthcheck:
    test: ["CMD-SHELL", "pg_isready -h 127.0.0.1 -U shop -d shop"]
    interval: 5s
    timeout: 3s
    retries: 5
    start_period: 20s
Enter fullscreen mode Exit fullscreen mode

Why each piece is there:

  • pg_isready ships with the Postgres image, so there's nothing to install. It asks the server "are you accepting connections?" and exits 0 when the answer is yes.
  • -h 127.0.0.1 is the important bit. On first boot, the official image starts a temporary Postgres that listens only on a local socket while it runs your init scripts. A plain pg_isready checks that socket and can pass too early. Forcing TCP with -h 127.0.0.1 waits for the real server.
  • start_period: 20s gives a fresh database time to initialize before failures count.
  • The volume keeps your data when the container restarts or redeploys. Without it, every update would start from an empty database.

Step 2: Write a check for Redis (and avoid the exit-0 trap)

cache:
  image: redis:7-alpine
  environment:
    REDIS_PASSWORD: change-me
  command: ["sh", "-c", "exec redis-server --requirepass \"$REDIS_PASSWORD\""]
  healthcheck:
    test: ["CMD-SHELL", "REDISCLI_AUTH=\"$REDIS_PASSWORD\" redis-cli ping | grep -q PONG"]
    interval: 5s
    timeout: 3s
    retries: 5
Enter fullscreen mode Exit fullscreen mode

This is the syntax for the Hubfly space Stack Composer. In a local docker compose file, write $$REDIS_PASSWORD instead (see "What can go wrong" #2).

Most examples online use redis-cli ping on its own. That check passes even when Redis is refusing you. Run it against a password-protected Redis and look at the exit code:

Console output: redis-cli ping without a password prints NOAUTH and still exits 0; with the wrong password it prints WRONGPASS and still exits 0

redis-cli prints the error and still exits 0. Your check would say "healthy" while every real request fails. The fix is to check for the actual answer: | grep -q PONG exits 1 unless Redis really replied PONG.

Why REDISCLI_AUTH instead of -a password: redis-cli reads the password from this variable, so it doesn't show up in the process list or the shell history.

Why exec in the command: it makes redis-server the main process, so it receives stop signals directly and shuts down cleanly.

Step 3: Write a check for your web app

Add a small /health route to your app that returns 200 when it can reach what it needs. Then check it from inside the container:

web:
  image: your-app:latest
  healthcheck:
    test: ["CMD-SHELL", "wget -q -O /dev/null http://127.0.0.1:3000/health || exit 1"]
    interval: 10s
    timeout: 3s
    retries: 3
    start_period: 15s
  depends_on:
    db:
      condition: service_healthy
    cache:
      condition: service_healthy
Enter fullscreen mode Exit fullscreen mode

Two things to get right:

  • Use a tool the image actually has. Alpine-based images include wget (from BusyBox); many, like redis:7-alpine, have no curl, while nginx:alpine has both. Run which curl wget in the container to be sure. Slim Debian images (python:3.13-slim, node:22-slim) have neither. For those, call your runtime instead, for example python -c "import urllib.request,sys; urllib.request.urlopen('http://127.0.0.1:3000/health', timeout=2)". Distroless and scratch images have no shell at all, so build a tiny check into your app binary.
  • Check 127.0.0.1, not your public URL. The check runs inside the container. Testing the public URL measures your proxy and DNS, not your app.

depends_on with condition: service_healthy sets the order: db and cache first, web last. In local Docker Compose it also holds web back until both checks pass. On Hubfly space, the order is the same, but don't count on the gate to wait for a passing check: a dependency can be marked ready within seconds of deploying. Your app's connection retry is what actually protects you, so keep it.

Step 4: Deploy the stack on Hubfly space

  1. In the dashboard, click Create Project. Name it (for example shop-api), pick a region, and optionally open Advanced Options to set a monthly budget cap such as $10.
  2. In the project, click Create → Compose Stack, open the YAML tab, paste the full file below, and click Import YAML.
  3. Click Deploy Stack.

The file uses nginx:alpine as a stand-in web server so you can deploy it exactly as written. To use your own app, change image and point the check at your /health route and port, for example http://127.0.0.1:3000/health.

services:
  db:
    image: postgres:16-alpine
    environment:
      POSTGRES_USER: shop
      POSTGRES_PASSWORD: change-me
      POSTGRES_DB: shop
    volumes:
      - pgdata:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -h 127.0.0.1 -U shop -d shop"]
      interval: 5s
      timeout: 3s
      retries: 5
      start_period: 20s
  cache:
    image: redis:7-alpine
    environment:
      REDIS_PASSWORD: change-me
    command: ["sh", "-c", "exec redis-server --requirepass \"$REDIS_PASSWORD\""]
    healthcheck:
      test: ["CMD-SHELL", "REDISCLI_AUTH=\"$REDIS_PASSWORD\" redis-cli ping | grep -q PONG"]
      interval: 5s
      timeout: 3s
      retries: 5
  web:
    image: nginx:alpine
    healthcheck:
      test: ["CMD-SHELL", "wget -q -O /dev/null http://127.0.0.1/ || exit 1"]
      interval: 10s
      timeout: 3s
      retries: 3
      start_period: 15s
    depends_on:
      db:
        condition: service_healthy
      cache:
        condition: service_healthy
volumes:
  pgdata:
Enter fullscreen mode Exit fullscreen mode

Replace change-me with real passwords before you deploy anything that matters. Note that --requirepass on the command line is visible in the container's process list; for production, put the password in a Redis config file on a volume instead.

The deploy log shows the order: the volume first, then db, then cache, and web last, once both dependencies have deployed. The whole stack takes about a minute.

The Compose deployment log: volume ready, db deployed in stage 1, cache in stage 2, and web waiting for db before it deploys

The project canvas with db, cache and web running on the internal network

Step 5: Make sure each check can fail

Don't trust a check you've never seen fail. Open each container, go to the Console tab, and run its test command by hand:

# In the db container
pg_isready -h 127.0.0.1 -U shop -d shop; echo "check exit=$?"
pg_isready -h 127.0.0.1 -p 5433; echo "wrong port exit=$?"

# In the cache container
sh -c 'REDISCLI_AUTH="$REDIS_PASSWORD" redis-cli ping | grep -q PONG'; echo "check exit=$?"
redis-cli ping | grep -q PONG; echo "without password exit=$?"

# In the web container
wget -q -O /dev/null http://127.0.0.1/; echo "check exit=$?"
wget -q -O /dev/null http://127.0.0.1/missing; echo "missing page exit=$?"
Enter fullscreen mode Exit fullscreen mode

You want 0 for each real check and a non-zero code for each deliberately broken one. Now you know the check can tell the difference.

Console output in the db container: pg_isready accepts connections with exit 0, and gets no response with exit 2 on a wrong port

Console output in the cache container: the real check exits 0, the check without a password exits 1, and the image has wget but no curl

Console output in the web container: the check exits 0, and a missing page returns 404 and exits 1

The container's status badge doesn't always show the check result, so the Console is the reliable place to look.

You can change a check later on a single container: open it, go to Configuration → Health Check → Edit, update the test command, interval, timeout, start period or retries, and click Save Changes. The container restarts with the new settings.

The Health Check editor on a container's Configuration tab

What can go wrong

1. The check passes, but the service is broken. Classic cause: redis-cli ping without | grep -q PONG, or an HTTP check that treats any response as success. Fix: check for the exact good answer, then prove it fails by running a broken version by hand (Step 5).

2. Your password variable turns into nonsense. In a local docker compose file you must write $$REDIS_PASSWORD, because Compose replaces single $ variables with values from your laptop's environment. The Hubfly space Stack Composer passes your command to the shell exactly as written, so there you write a single $REDIS_PASSWORD. If you paste $$REDIS_PASSWORD into Hubfly, the shell reads $$ as its process ID and Redis ends up with a password like 1REDIS_PASSWORD. The symptom: your app gets WRONGPASS with the password you set. Worse, a failing check doesn't stop the stack from deploying, and the container keeps running, so nothing looks wrong at first. Fix: use $ on Hubfly, $$ locally, and check with REDISCLI_AUTH="$REDIS_PASSWORD" redis-cli ping in the Console.

3. "executable file not found" or the check always fails. The image doesn't contain curl, wget or even a shell. Fix: run which curl wget in the Console, then pick a tool the image has (Step 3). For scratch or distroless images, add a /health subcommand to your binary and use the exec form: ["CMD", "/app", "health"].

Takeaway

A health check is only useful if it can fail. Check for the real answer (PONG, a 200 from /health, pg_isready over TCP), give slow starters a start_period, use a tool the image actually contains, and run every check by hand once. Pair that with a connection retry in your app, and deploys stop being the moment your users see errors.

Related guides:

Deploy this

Paste the Compose file from Step 4 into a new project on Hubfly space. On the free shared tier, the dashboard projects about $0.02 a month for this stack. That's the 1 GB database volume; shared runtime is free within 360 hours a month, and sleeping containers use no compute.

Go further on Hubfly space

  • Send traffic only to healthy copies of your app: put a load balancer in front of two or more web containers. It uses health probes to route around a target that stops answering.
  • Ship a release slowly: send 10% of traffic to a new version first with a weighted load balancer.
  • Reach your database from your laptop without a public port: open a short-lived tunnel with hubfly tunnel create.

Top comments (0)