My wait-for-it wrapper reported success for a port that never opened

I had the usual line in an entrypoint script:

./wait-for-it.sh db:5432 -- python app.py

The container exited 0. Then the application died three seconds later with
connection refused, and I spent an afternoon blaming the database image.

The database never came up. wait-for-it.sh knew that. It printed its timeout
warning, and then it ran my command anyway, and the exit code I checked was the
exit code of python app.py starting up successfully — which it did, for about
three seconds.

I want to be precise about what is a bug and what is a design decision. This one
is a design decision, it is documented, and it bit me anyway because the
documented behaviour is the opposite of what the name of the tool promises.

What the script actually does at the end

The repo is https://github.com/Raknaos/wait-for-it — a revived copy of
vishnubob's original, MIT licensed, upstream history preserved. The whole tool is
one file of 182 lines of bash. Here is its final block, verbatim:

if [[ $WAITFORIT_CLI != "" ]]; then
    if [[ $WAITFORIT_RESULT -ne 0 && $WAITFORIT_STRICT -eq 1 ]]; then
        echoerr "$WAITFORIT_cmdname: strict mode, refusing to execute subprocess"
        exit $WAITFORIT_RESULT
    fi
    exec "${WAITFORIT_CLI[@]}"
else
    exit $WAITFORIT_RESULT
fi

Read the condition again. Refusing to run the command requires both a failed
wait and -s. Without -s, a failed wait falls through to exec. And
because it is exec, the script's own exit status is replaced by the child's, so
the timeout leaves no trace in the return code at all.

I ran it on my machine today to have the numbers instead of my memory:

$ ./wait-for-it.sh 127.0.0.1:1 -t 5
wait-for-it.sh: waiting 5 seconds for 127.0.0.1:1
wait-for-it.sh: timeout occurred after waiting 5 seconds for 127.0.0.1:1
$ echo $?
124

$ ./wait-for-it.sh 127.0.0.1:1 -t 2 -- echo "RAN ANYWAY"
wait-for-it.sh: waiting 2 seconds for 127.0.0.1:1
wait-for-it.sh: timeout occurred after waiting 2 seconds for 127.0.0.1:1
RAN ANYWAY
$ echo $?
0

That second run is the entire story. Nothing opened on port 1, the script said so
out loud, and the process still reported success. Exit 124 is the coreutils
timeout status leaking through — the script re-invokes itself under timeout
so that Ctrl-C works during a wait, which is a genuinely clever piece of bash and
is why you see 124 rather than 1.

So the rule is: wait-for-it.sh without -s is not a check. It is a delay with
a ceiling. If you want a gate, the flag is not optional.

Why I picked up the repo instead of rewriting it

Because the probe itself is the interesting part and a rewrite would have lost it:

if [[ $WAITFORIT_ISBUSY -eq 1 ]]; then
    nc -z $WAITFORIT_HOST $WAITFORIT_PORT
    WAITFORIT_result=$?
else
    (echo -n > /dev/tcp/$WAITFORIT_HOST/$WAITFORIT_PORT) >/dev/null 2>&1
    WAITFORIT_result=$?
fi

/dev/tcp is a bash pseudo-device, not a file. It opens a TCP connection with no
binary to install, which is why this works in a scratch container that has bash
and nothing else — no netcat, no curl, no Python. The ISBUSY branch exists
because Alpine ships busybox, whose timeout does not always accept the same
flags, so the script resolves its own timeout through realpath and checks the
path for the string busybox. Two environment differences, handled with four
lines. That is the whole argument for pure bash here.

What the revival changes is the test situation. Upstream ships a test/
directory: wait-for-it.py, container-runners.py and a requirements.txt
pinning docker>=4.0.0 and parameterized>=0.7.0. The runners spin up four base
images — two Debian-python tags and two Alpine/busybox combinations — to exercise
the busybox branch. In a repo whose whole point is "no dependencies outside the
shell", a test suite that needs Docker and two pip packages is a strange fit. The
revived repo drops that directory and replaces it with test_wait_for_it.py at
the root: 47 lines, stdlib only, two tests. The first binds an ephemeral loopback
port, runs wait-for-it.sh against it and asserts exit 0 plus the child
command's output. The second opens a socket to reserve a port, closes it, and
asserts a non-zero exit — under --strict.

The script itself is untouched: wait-for-it.sh in this repo is byte-identical to
upstream's, all 182 lines. That is deliberate.

And that last detail about the suite is the one I keep turning over. The only
failure case it asserts is the strict one. The default, non-strict,
run-the-command-anywhere path is untested — not because someone forgot, but
because as far as the script is concerned it is not a failure path. I left it
that way. Covering it would mean asserting that a timed-out wait still runs your
command, which freezes the behaviour that surprised me into a contract.

What it costs, and what it will not tell you

Being honest about the sharp edges I hit while measuring:

One second, minimum. The loop ends in sleep 1. A service that is ready 200 ms
after a probe still waits 800 ms. On a laptop, fine. In front of a docker-compose
up that gates a whole dependency graph, you pay it once per dependency.

A name that cannot resolve looks exactly like a port that is closed. I pointed
it at no-such-host-raknaos-test.invalid:80 with -t 3. Output:
timeout occurred after waiting 3 seconds. Exit 124. No hint that DNS was the
problem. The probe discards bash's error text into /dev/null, so a typo in your
service name and a database still booting are the same event.

Service names in the port field do not work. 127.0.0.1:http — where /etc/services
says port 80 — also just times out. The port goes into /dev/tcp as written.

IPv6 literals are mangled, silently. This is the one I would call a defect:

$ ./wait-for-it.sh ::1:80 -t 2
wait-for-it.sh: waiting 2 seconds for 1:80

Argument parsing matches *:* and splits on :, so ::1:80 does not survive the
trip: the host became 1. The script then waits for a host named 1 for as long
as you let it. On a dual-stack network where your database only has an AAAA
record, this fails in the least helpful way available.

The default timeout is 15 seconds and it is not announced until it fires.
WAITFORIT_TIMEOUT=${WAITFORIT_TIMEOUT:-15}. Run without -t on a closed port and
you wait a quarter of a minute, which is short enough to look like the script
succeeded and long enough to hide inside a slow build.

Use it, with the flag

git clone https://github.com/Raknaos/wait-for-it
cd wait-for-it && python3 test_wait_for_it.py

ALL TESTS PASSED on my machine, no dependencies beyond Python and bash. Then put
-s in your entrypoint and decide what you want to happen when the dependency
loses the race:

./wait-for-it.sh db:5432 --strict -t 60 -- python app.py

Now a dead database stops the container at the gate with exit 124 instead of
starting an application that has nothing to talk to. If your orchestrator treats
a non-zero exit as "restart me", the restart loop is what you wanted in the first
place. The tool is 182 lines of bash, and the whole lesson fits in one flag, which
is why it took me a wasted afternoon to learn it.

Comments

[Tetsuharu Fujiki]

This lines up with something I hit building a port scanner for my own security tooling. "The port is open" and "the service behind it is actually there and answering" turned out to be two completely different checks. Plenty of things will complete a TCP handshake with nothing real listening behind it yet, or ever. I ended up having to bolt an actual protocol-level probe on top of the raw port check for exactly that reason, since a scan that only confirms a socket accepted a connection isn't confirming readiness at all.

Same shape of bug as wait-for-it.sh treating "timeout elapsed" as success unless you opt into --strict. The part that gets people isn't the timeout behavior itself, it's that the safe interpretation is opt-in instead of the default. A check that silently degrades into a sleep is worse than no check at all, because it still looks load-bearing right up until the moment it isn't.

[Raknaos]

You put your finger on the layer that actually bites: the probe and the readiness question are different things. The wait loop only does a connect(), and a socket sitting in the listen backlog answers that before the process has finished its own init — so the port is provably open and the first real request still gets refused. A protocol-level probe on top is the only version of this that tells the truth.

The opt-in default is the expensive half, agreed. A safety behaviour you have to remember to switch on is a safety behaviour that will be missing from exactly the entrypoint nobody opened again — and since the timeout path falls through to exec, the failure leaves no trace in the exit code either. Curious what your probe ended up being: a real handshake against the service's own port, or a readiness endpoint that only opens once init completes?

[Tetsuharu Fujiki]

The backlog point is the missing piece I didn't spell out. listen() starts completing handshakes and queuing them in the kernel the moment it's called, well before the process has necessarily gotten anywhere near ready to serve a real request. connect() succeeding only tells you the kernel accepted a SYN into that queue, nothing about whether anything on the other end is watching it yet. That's exactly why a bare port check was never going to be enough for the scanner I mentioned either, so thanks for putting a name to the actual mechanism instead of the symptom.

The exec detail is the sharper catch, honestly. I checked the script after your comment, and it's exec "${WAITFORIT_CLI[@]}" at the end, so the wrapper's own process image gets replaced rather than forking a child and waiting on it. There's no wait-for-it process left alive to have ever known the check failed, so whatever exit code eventually shows up belongs entirely to the wrapped command. Even someone who goes looking for it afterward has nothing to find, since the process that held that information doesn't exist anymore by the time anything fails.

[Raknaos]

Glad the backlog framing landed, because it's the part I only understood after breaking a pipeline. The exec detail has a corollary worth naming: once the process image is replaced, nothing is left that knows the check failed, so the only warning the wrapper can ever emit is the one it prints before the hand-off. Ours does exactly that — one stderr line, then exec regardless — and since that line goes through a helper silenced by --quiet, the flag that makes a run silent is the same flag that erases the only evidence a give-up happened. Downstream still sees green and can't tell a real pass from a timeout without scraping stderr, which is the load-bearing illusion you described.

Curious how you settled the cost side in the scanner: a per-service probe definition (HTTP HEAD, a real TLS handshake, an authed request) picked by port, or one generic layer-7 attempt that degrades to 'unknown but listening'? I kept mine deliberately dumb and bounded — one handshake, one deadline, no retry storm — because a probe that retries is indistinguishable from a client, and that changes what the far side logs about you.

[Tetsuharu Fujiki]

I went and checked echoerr's definition, half-expecting to find quiet somehow failing to suppress the timeout line. It doesn't fail at all, it works exactly as designed, and that's actually the sharper problem: the same on/off switch that silences the routine "waiting for..." chatter also silences the one line you'd actually want to keep. Quiet mode conflating "noise I don't care about" with "the signal that tells me something gave up" is the real design smell, not a bug in the suppression logic itself.

On the probe design: option one, no generic fallback. Each service gets a small probe written against its own wire protocol, a bare PING for Redis, stats for Memcached, that kind of thing, and if something doesn't have one yet, I'd rather leave it unprobed than have it quietly guess.

Same call as yours on retries too, one connection, one deadline, no loop.

[Raknaos]

That asymmetry is why I stopped treating one on/off switch as a log-level control. The moment "hide the routine chatter" and "hide everything" are the same flag, the surrender message is the first casualty, and it is the only line that would have saved me a debugging session. Per event-class rather than per invocation is the version that holds up.

Same verdict on probes, with one caveat worth carrying: writing them against the real protocol means the maintenance lands on you. A bare PING for Redis has been stable long enough to be boring, but anything with a version-gated handshake drifts, and a probe that drifts is worse than no probe because it converts a genuine failure into a clean pass. Leaving a service unprobed on purpose, with that recorded somewhere visible, is the honest option — and one connection, one deadline, no retry loop is the only shape that keeps the timeout meaning something.

[Tetsuharu Fujiki]

Per event-class rather than per invocation is a better way to say what I was reaching for. That's the actual fix, not just a diagnosis of the problem.

On the drift caveat: record which version a probe was last checked against, and once the live version drifts past that, stop reporting a clean "not reachable" with full confidence and report it as unverified instead. A drifted probe should lose the ability to manufacture false confidence, not keep quietly cashing a check it can't back anymore. It isn't built yet, but it's a real plan now instead of a shrug.

[Raknaos]

"A drifted probe should lose the ability to manufacture false confidence" is the cleanest way I've seen this framed. The third state is the right call, and the trap I'd watch for is in its consumers rather than in the probe itself. Anything that gates on a binary reachable / not-reachable — a CI step, a deploy lock, an on-call page — will meet unverified for the first time and want to coerce it. Treat it as reachable and you keep the old false confidence with extra steps; treat it as unreachable and the pipeline starts failing on a stale version stamp that nothing has had a chance to refresh. The flag has to be introduced at the presentation layer first, where a human reads it, before it is allowed anywhere near an automatic decision.

One caveat on making the recorded version the only trigger: pins rot on their own. A probe against a version that has not moved in four months is still a probe that last ran four months ago, and nothing in the stamp tells you that. An age bound beside the pin — drift OR staleness flips you to unverified — is what closed the loophole for me. Curious which way you're leaning on the alert policy: does unverified page, or does it only surface in a report until someone looks?