Between October 14 and October 18, three failures on my machine didn't stop the step that came after them. A stats collector died with exit code 2, and the scheduler logged 0. A check script crashed on an encoding error, and the shell said 0. A test run failed and even printed rc=1, and the commit and the push ran anyway.
The failures themselves were ordinary. The trouble sat between each command and the next step: twice a layer we had added to make a job quieter or politer, and once a single character.
I'll go through all three, and at the end I'd like to hear where that gap sits in your own setup.
Some context first. This is the same Windows machine where about twenty agent sessions share one git repo and a set of scheduled jobs. Sessions report "passed" or "failed" to each other, and those reports are supposed to come from exit codes. A fake 0 doesn't stay local.
The wrapper our own rules recommended
Since September 30 our shared rules had said that long-running Python jobs should run at below-normal priority, and they listed start /belownormal as one way to do it. The collector's scheduled slot did exactly that from Git Bash (the doubled slashes stop Git Bash from turning /c into a path):
cmd //c start //belownormal //wait //b python collector.py
On October 14 a stats collector hit a 403 on an item it wasn't allowed to read. It stopped after 2 of 42 items and exited with code 2. The scheduled slot recorded 0.
That suggestion had been in the rules for fourteen days. I haven't dug into why start drops the child's code. The observed result was enough to change the rule. The default is now to lower the priority inside the script:
psutil.Process().nice(psutil.BELOW_NORMAL_PRIORITY_CLASS)
If a job has to be wrapped, the wrapper hands the code back explicitly:
cmd //v:on //c "start /belownormal /wait /b python collector.py & exit !errorlevel!"
The collector itself was changed too, to skip items it can't read and exit non-zero instead of dying. The next morning it ran through the new wrapper. It skipped six items, and the slot recorded 4. That's a non-zero code, and for once a true one.
Two tails on the same day
On October 15 | tail fooled us twice.
A long background job was launched as job | tail -60 with the result going to a file. The file sat at 0 bytes, because tail prints nothing until its input ends. The job looked dead. It wasn't, and it nearly got restarted mid-run.
The same day, the coordinating session ran a check script through | tail. The script crashed with a cp949 encoding error, Windows' Korean code page. The pipeline exited 0, because a pipeline reports the status of its last command, and tail had done its job fine.
That day the lesson went into our lessons file: write long output straight to a file, and use tail only to read that file afterwards. It went in as a candidate. It did not go into the rules every session reads first.
Three days later, a semicolon
On October 18 a session working on one of our apps did what the lesson said. It sent the test output to a file and printed the exit code:
npx vitest run > vt.log 2>&1; echo rc=$?; grep "Tests " vt.log; git add ... && git commit ...; git push
The screen said rc=1. One test out of 429 had failed: a heavy check that runs 3,000 readings went past vitest's default 5-second timeout in the full run. The commit was joined to the test with ;, not &&, so it ran anyway, and so did the push. The exit code was asked for and printed. Nothing was gated on it.
A minute later the next commit gave the heavy checks a 60-second limit, and that one went out through vitest > file && typecheck && commit. The October 15 lesson was about where output goes. It said nothing about what has to be true before the next command runs.
The same afternoon a session on a different app committed with three type errors. That one is simpler: no check ran at all, so nobody asked for an exit code.
Since October 19 this is in the shared rulebook every session reads:
npm test > test.log 2>&1 && git commit -m "..." -- <paths>
Output goes to a file, and the commit hangs off &&. A report can say "passed" only when the exit code was 0.
What the three have in common
The failing commands never lied. The collector, the check script and the tests all exited non-zero. What went wrong sat between that code and the next step: a priority wrapper, a | tail, a ;. The first two were added to make something nicer. The third is one character. Each let the next step run as if nothing had happened.
Bash has set -o pipefail for the pipe case, and set -e stops a script at the first failing command. Neither is in our rules. A line that ends in && git commit is easier to check by reading it once, though I'm not sure that holds up as scripts get longer.
A question
Where in your setup does a failure still come back as 0, or get printed and ignored? A wrapper, a pipe, a ; where && should be, an || true in CI, a retry script that swallows the last error? I'm collecting the places I haven't looked yet.
Sources
- Bash manual, Pipelines (a pipeline's exit status is the last command's unless
pipefailis set): https://www.gnu.org/software/bash/manual/html_node/Pipelines.html, accessed 2026-10-19 - Bash manual, The Set Builtin (
-eand-o pipefail): https://www.gnu.org/software/bash/manual/html_node/The-Set-Builtin.html, accessed 2026-10-19 - Vitest config,
testTimeout(default 5,000 ms in Node.js): https://vitest.dev/config/testtimeout, accessed 2026-10-19 - Everything else: my own lessons log, job logs and commit history from October 14 to 18.
This post was drafted with AI help. The numbers and facts come from my own logs and repo.
Top comments (0)