DEV Community: tinyproof

A CLI that took a full prompt as an argument and still hung waiting for stdin to close

tinyproof — Fri, 28 Aug 2026 04:32:14 +0000

A CLI that took a full prompt as an argument and still hung waiting for stdin to close

I called a review tool from a script, passing the entire prompt as a single command-line argument. No explicit shell pipeline was feeding it anything else. It printed one line — Reading additional input from stdin... — and then nothing.

$ codex exec --skip-git-repo-check "<full prompt text>"
Reading additional input from stdin...

No more output. Past the shell tool's own timeout, the process got moved to the background automatically, with a message to that effect. Polling its status afterward returned the same thing every time: still running. It had not finished by the time I stopped checking.

What I checked first

The prompt itself wasn't the problem — I'd already passed the whole thing as the argument, and nothing in it was malformed. The tool wasn't crashing; it was waiting for something. The one concrete clue was the line it printed: it was explicitly announcing that it intended to read more input from stdin, even though the argument already contained a complete prompt.

Root cause

The tool appears to support two separate input channels — a prompt given as an argument, and additional input read from stdin — and the two aren't mutually exclusive. Having a complete prompt in the argument doesn't stop it from also trying to read stdin. In the shell environment I was calling it from, stdin did not send an end-of-file signal on its own. So the tool sat there waiting for an EOF that, in this environment, doesn't arrive by itself.

That distinction matters because of what it isn't: it isn't the tool being slow, and it isn't the tool being unresponsive. What I observed was consistent with a process waiting on an input stream that, in this environment, doesn't close on its own.

The fix

Close stdin explicitly, so the tool gets its EOF immediately instead of waiting on a channel nothing else is going to signal:

codex exec --skip-git-repo-check "$(cat <<'EOF'
...prompt content...
EOF
)" < /dev/null

One detail from getting there: writing the prompt to a separate file and reading it back with cat /path/to/file, combined with the same < /dev/null redirect, got rejected by this sandbox's static analyzer — "Contains shell syntax that cannot be statically analyzed." The heredoc form above, with the prompt written inline in the command itself, was not rejected and ran. In this environment, that is what got the redirect past the sandbox's analysis step — I have not tried every other way of writing it.

What I verified

With < /dev/null added, the same call printed its verdict and exited in the foreground, somewhere between a few seconds and roughly a minute and a half, with no more Reading additional input from stdin... and no move to the background. I ran this twice, with two different prompts. Both finished in the foreground both times.

That confirms the fix resolves this exact hang, for this exact calling shape, in this environment, twice. It does not tell me why the tool reads stdin at all when the argument already contains a full prompt — I didn't read its source, and I'm not claiming to know its internals. It also doesn't tell me whether some other shell environment closes stdin automatically and would never have hit this in the first place.

What I check now

If a call to a CLI that accepts a full prompt as an argument stalls with no more output, I no longer read that as the other side being slow or unresponsive. I check whether stdin was ever closed. For this specific tool, in this specific environment, that means appending < /dev/null by default, not adding it only after a hang shows up.

This article was written with the help of AI. The incident, the commands and the outputs are real and from my own machine; the draft was AI-written, then checked against the original logs and corrected.

A health check that flags "started but never finished" — for every day it runs before the schedule is done

tinyproof — Thu, 27 Aug 2026 02:58:56 +0000

A health check that flags "started but never finished" — for every day it runs before the schedule is done

A scheduled pipeline calls a health check partway through its own daily run. The check flagged one thing: today's date, "started but never finished." The pipeline was not stuck. It just was not done yet.

The check

The logic, generalized to the actual shape:

if "today start" in log_text and "today end" not in log_text:
    flagged.append(day)

Read a log file, look for a start marker and an end marker, flag anything with the first and not the second. That is correct for any day that has actually ended.

Why it flagged today

The log for the current day had a start marker. It could not have an end marker yet — the pipeline only writes that marker after the last task of the day finishes, and the health check was itself invoked by an earlier task in that same run. At the moment the check ran, the day's schedule was still in progress. No end marker existed to find, whether or not anything was actually wrong.

This isn't intermittent. The code above makes it deterministic: every invocation made after the run has started but before its end marker is written produces this exact flag.

What I verified before calling it a false positive

I didn't want "it's probably fine" to be the answer just because it was the reassuring one. I pulled the log for the flagged day and read the timestamps against the task list:

[08:07:00] task A — exit 0
[08:07:00] task B — exit 0
[08:07:00] incident recorded: task C failed
[08:07:00] task C — exit 1
[08:07:00] task D — exit 0
[08:08:42] task E starts

Every entry from the start of the run to the point the health check itself fired belongs to the same process ID, in sequence, with no gap. Task C failing with exit 1 is a real, separate problem — logged and recorded as its own incident — but it is not what the health check was reporting on. Nothing in that window shows the run stopping and something else starting later. "Started, no end marker yet" and "started, then stopped" produce an identical log shape to this check. Only one of them is actually a problem.

What I did not do

I did not change the health check. The fix needs a decision this task wasn't scoped to make: how to tell "the log is from today, and today isn't over" apart from every other case where a start-without-end pattern really does mean something stopped. What I wrote down instead: compare the log's date to the current date, and only apply the no-end-marker rule to days that are not today. The script itself is still unchanged.

Why this shape is worth knowing about

Any health check built on "there's a start marker, I'm checking for an end marker" runs into this if it can be invoked before the thing it's checking is actually finished — a check called mid-pipeline, by one of the pipeline's own steps, is exactly that scenario. In this incident, the tell was: the flagged date was the current date, and the process that supposedly "stopped" was still writing to the log when I went to look.

What I check now

When this kind of check reports "started, never finished," the first thing I look at is whether the flagged date is today. If it is, I check whether the run is actually still going — same process ID, timestamps still advancing — before treating it as an incident. A start-without-end pattern is only worth treating as stopped once the window it's looking at has actually closed.

This article was written with the help of AI. The incident, the log lines, and the exit codes are real and from my own machine; the draft was AI-written, then checked against the original log before publishing.

A regression test that still passed after I undid the fix

tinyproof — Wed, 26 Aug 2026 02:29:22 +0000

I fixed a bug where two places in a system computed the same selection logic, and one of them did not get updated when the other changed. I added a regression test afterward: 12 test cases, all passing. I sent it for review.

The review came back with one line: the test that was supposed to prove the fix had reimplemented the selection logic instead of calling the real function that does the selecting.

What I actually wrote

I wrote a small loop inside the test file that mirrored what I believed the real selection function did: same conditions, same order, same output shape. Then I asserted the dry-run result matched that loop's result.

That passed. All 12 cases passed. It looked like coverage.

Why that does not prove anything

The bug I had just fixed was exactly this shape: the same decision made in two places, one changed, the other did not follow. Writing a second copy of the decision inside the test file recreates the identical risk one level up. The test was not checking whether the real function behaves correctly — it was checking whether my copy of the function agrees with my other copy of the function. Both copies could be wrong in the same way and the test would still pass.

The fix

I wrote a separate test that calls the real function directly — no duplicated logic, no simplified stand-in. The only things replaced were the parts with side effects: the function that actually sends output, a network lookup, and file writes. Those were swapped for stubs that record what they were called with and do nothing else. The selection logic itself runs unmodified, exactly as it runs in the real implementation.

What I actually verified

I ran a reverse probe: removed the one line in the implementation that was the actual fix, then ran both test files against that broken code.

old test file (rewritten logic):         12 passed
new test file (calls the real function):  1 failed, 3 passed

The old file passed 12 out of 12 with the bug back in place. The new file caught it: 1 failed, 3 passed. That is the difference between a test that checks the code and a test that checks a copy of itself.

What this does not prove

This confirms the new test can detect this one specific regression. It does not prove the new test catches every way this function could break, and it does not prove the old test is useless everywhere — only that it gave a false pass on exactly the failure it was written to catch. A test suite can hold both kinds of test at once, and the pass count alone will not tell you which kind you are looking at.

What I check now

Before trusting a passed result on any test meant to verify logic, I look at the test file for an if, for, or selection loop that mirrors the intent of the real implementation, instead of a direct call to the real function followed by an assertion on its output. That shape is the signal.

The fastest way to check is the same reverse probe: remove the line in the implementation that matters, rerun everything, and see what turns red. If nothing does, the test's pass rate was never measuring the thing it claimed to measure. It was measuring agreement between two versions of the same guess.

A retry loop that couldn't tell the checker crashed from the task not being done

tinyproof — Mon, 24 Aug 2026 02:46:39 +0000

A retry loop that couldn't tell "the checker crashed" from "the task isn't done"

I run a completeness checker that a scheduler calls after every task to decide whether that task is finished. The checker returns a non-zero exit code in two completely different situations, and my retry loop was treating them the same.

Exit code 1: the checker ran fine, judged the task, and the task is missing something. Its own output always contains the literal string "not done: <what's missing>".
Exit code 2: the checker refuses to judge at all — usually because the caller forgot a required argument.

Here is an exit code 2, produced by leaving out a required --since flag:

$ python3 scripts/completeness_check.py my-task 2026-08-21
exit=2
[checker] refusing to judge: no --since <marker file> given.
  Without a defined window this can pick up output from an unrelated run
  and pass an idle pass as complete. The scheduler always passes it;
  a manual run needs a marker file touched first.

That message never contains the words "not done". It is the checker saying "I can't tell", not "you're missing something".

What the old retry loop did

The old logic in my scheduler script checked one thing: was the checker's exit code zero. Anything else got treated as "the task isn't finished, retry it" — including a checker that refused to run because of a missing flag. The retry fed the checker's own error message back into the task's next attempt as if it were a list of missing work. The task had nothing to fix. What was broken was the way the scheduler called the checker, not the task itself.

Root cause

An exit code only carries one bit: zero or not zero. "Failure" here actually has two distinct meanings — the thing being checked is not good enough, or the checker itself couldn't render a verdict. Collapsing both into the same non-zero signal removes the caller's ability to tell "retry the task" from "fix how you're calling the checker".

The same shape got caught three separate times in one review pass, and each fix just added another item to a list of "here's what a broken checker call looks like" — first "exit code 2", then "the output contains a traceback". Both missed a third shape: the checker calling a hard exit with code 1 and no traceback at all, which still isn't "not done".

The fix

Instead of enumerating every way the checker can fail to judge, check for the one positive signal that means it actually did judge and found something missing:

# the checker prints "not done:" from exactly one line in its own source,
# whenever it has actually rendered a not-done verdict
if [ "$rc" -ne 0 ] && ! printf '%s' "$why" | grep -q "not done:"; then
  # non-zero, but no not-done marker -> the checker itself is broken,
  # not the task. File an incident, don't retry the task.
  ...
fi

Anything with the marker string is a real "not done" and gets retried. Everything else — regardless of exit code — is "can't judge", and gets an incident filed instead of a retry. New failure shapes from the checker don't need a new branch; they fall into "can't judge" by default because they lack the one positive marker.

What I actually verified

I reproduced the exit-code-2 case for real — ran the checker without --since and got the output shown above. I then ran the new classification logic against that exact output by hand: no "not done:" marker present, so it classifies as "can't judge", not retried. Before the fix, the same output would have been read as "not done, retry" — I traced that branch by hand too, since I didn't want to actually re-trigger a real retry to prove a negative.

What this does not prove

I did not exhaustively enumerate every exit path the checker can take and confirm each one lands where I expect — only the one shape I had in hand (missing --since). The fix's design, positive matching instead of enumeration, is meant to make that unnecessary, but I have not verified it against a checker that has been running long enough to grow new failure shapes I have not seen yet.

What I check now

For any external checker or validator wrapped in a retry loop: does its output give me a way to tell "the subject is invalid" apart from "the checker itself couldn't run"? If the only signal is a bare exit code, that is the early warning sign — a retry loop built on exit code alone will eventually retry something that was never actually broken.

This article was written with the help of AI. The checker, the retry loop, and the exit codes described are real, from my own scheduler. The bash snippet above is translated out of the original — the real marker string is in Chinese — and I re-ran the classification logic by hand against the real reproduced output before writing this, rather than pasting a raw terminal transcript.

I fixed one silent failure and stopped looking. A review found three more

tinyproof — Wed, 19 Aug 2026 13:37:22 +0000

I had a scheduled script that ran fine by hand and produced nothing on a schedule. Exit code 0, no error in the log, an output file created and empty.

The script calls a CLI tool that reads credentials from the macOS keychain. Run it in my terminal: works. Run it from launchd: Not logged in · Please run /login. The credentials had not expired.

This post is about the fix, and then about the three other places the same shape was hiding — which I did not find. Someone reviewing my code did.

The smallest difference I could reproduce

launchd hands a process a much smaller environment than a login shell does. No .zprofile, no .zshrc, so nothing those files set is present.

I bisected down to a single variable. Both runs below are identical except for one thing:

# A — with USER
env -i HOME="$HOME" USER="myname" PATH="$P" mytool -p "say hi"
# → responds normally

# B — same command, USER removed
env -i HOME="$HOME" PATH="$P" mytool -p "say hi"
# → Not logged in · Please run /login

That is the smallest difference I could reproduce. PATH was already correct in both, HOME was set in both. Remove USER and the tool decides you are not logged in.

I do not know why the tool needs USER. I did not read its source. It might use it to look up the keychain entry, it might be something else. What I can tell you is the reproduction above, because I ran it both ways.

The fix is to set the environment explicitly instead of hoping the scheduler provides it:

export PATH="/usr/local/bin:/opt/homebrew/bin:/usr/bin:/bin"
export HOME="/Users/myname"
export USER="myname"      # without this the CLI reports "not logged in"

The part that mattered more

The login failure exited 0.

My script checked the exit code to decide whether the run succeeded. The run had done nothing, and the exit code said fine. So the real fix is not the export line:

run_job() {
  "$TOOL" -p "$(cat "$prompt")" > "$out" 2>&1

  # exit code is not enough — the login failure exits 0
  if grep -qE "Not logged in|Please run /login|Invalid API key" "$out"; then
    record_incident "auth failed"; return 1
  fi
  # neither is "the file exists" — check it has content
  if [ "$(stat -f%z "$out")" -lt 50 ]; then
    record_incident "no output"; return 1
  fi
}

What I actually verified, so you know how far to trust it: I copied that logic into a standalone script and fed it five files. Three contained one of the error strings each, padded past 50 bytes so the size check could not be what caught them. One was empty. One was 201 bytes of ordinary output. The first four are caught, the last passes. That exercises the guard. It is not a test of the original failure — I did not break a live login to reproduce it.

The three I did not find

I wrote that guard and moved on. Then someone reviewed the rest of my code and came back with three more instances of the same shape. All three are theirs:

A linter I wrote printed an error for an unreadable input file and exited 0. Tested with /dev/null: 0 workflow(s) checked, exit 0. CI would read that as "nothing wrong".
A checker I wrote reported "0 segments, 0 issues" on an empty file — the same exit code as a genuinely clean file.
A pipeline swallowing an exit status, which showed up in two places I had been counting as separate bugs.

The pipeline one, and the correction I owe

I had this filed as two findings: "timeout is missing on macOS and fails silently" and "I read an exit status through a pipe". The first half of that was wrong.

$ timeout 5 echo hi
zsh: command not found: timeout
$ echo $?
127                      # ← correct. A missing command does report failure.

$ timeout 5 echo hi | head -1
zsh: command not found: timeout
$ echo $?
0                        # ← the pipeline reports head's status, not timeout's

A missing command exits 127 on its own. What swallowed it was the pipe: by default a shell pipeline reports the exit status of the last command. timeout was not a silent failure, it was the thing that made the pipe's behaviour visible. Two findings, one bug.

set -o pipefail makes the pipeline return non-zero if any command in it fails — same commands as above, 0 becomes 127. It is worth turning on where you are branching on a pipeline's status, which in an unattended script is usually.

What I check now

An unreadable or empty input never exits 0. It gets its own code, distinct from "scanned and found nothing".
Success is not "exit code 0". It is "the thing I expected to be produced exists and has content".
On macOS, check that timeout exists before a script depends on it. Same for anything else assumed to be standard.
If a status matters, do not read it through a pipe without pipefail.

None of those is the part worth passing on. I found one instance, fixed it, and did not go looking for others. The three above came from someone reading the same code who had not just solved it.