DevSecOps builder creating practical security tools for solo founders and small teams shipping with AI—safer code, CI/CD pipelines, and agent workflows without enterprise overhead.
The machine-owned check is the right boundary. I’d also bind every successful check result to the exact Git tree it evaluated.
An exit code alone can become stale immediately: the agent can run the check, modify another file, then hand the card back using a result produced against different state. Recording the commit/tree hash, command, exit code and output digest makes the claim reproducible.
For dirty worktrees, either refuse completion or hash the relevant files before and after the check and fail if they changed. The useful invariant is not just “this command passed,” but “this command passed against the exact artifact now being marked complete.”
"This command passed against the exact artifact now being marked complete" — that's the invariant, said better than I've managed it.
Most of it's there: argv, exit code, duration, sha, dirty flag, anchored to the tree
the check ran on rather than HEAD at write time. And the agent never carries a result at all — done() runs the check itself.
But you've found something I hadn't. I read the head before the run and never again, so a tree that changes during the check is recorded against pre-run state. Your before-and-after hash is the fix, and I don't have it.
Fixed in 9feb4bd. You were right — the tree was only read before the check ran, so a pass could be recorded against a tree that had already changed. It's read on both sides now, and if it moved the card doesn't move either.
One thing your comment saved me from: it hashes the diff, not the filenames. A file edited twice with the same name would have slipped through otherwise.
Thanks. First bug on that board that didn't come from me or CI.
DevSecOps builder creating practical security tools for solo founders and small teams shipping with AI—safer code, CI/CD pipelines, and agent workflows without enterprise overhead.
Nice fix. Hashing the diff contents closes the “same filenames, different state” hole.
The remaining edge is the small TOCTOU window between the second tree read and the card-state write. If another process can modify the worktree there, I’d make the final transition a compare-and-swap: update the card only if the current tree still equals the verified hash. Alternatively, hold the same repository lock across verification and transition.
Then the invariant becomes atomic: either the exact verified tree is marked complete, or nothing moves.
Fixed in 13450bb — the promotion is a compare-and-swap now. Of your two fixes it was the only one this design could take: the check deliberately runs outside the write lock (waiters give up at 60s, and a suite can run for minutes), so holding the lock across verification was out. Instead done() takes a third tree reading while it holds the lock, and the card moves only if the digest still equals the one the check was verified against. A mismatch lands in the same bucket as the last fix — absence of evidence, neither pass nor fail, the card stays in progress. The test stages your scenario literally: a second process holds the lock, the tree moves while the write waits on it, and the card must not move with it. Which is your closing sentence as an invariant: either the exact verified tree is marked complete, or nothing moves. Three findings now, each a strictly smaller window than the last. The acknowledgements section is developing a habit of your name.
@peterbuildssecure Postscript, a week on: all three of your findings shipped, and it's on npm now — npx shipward setup ~/code/your-project --seed-from-branches — so the thing a stranger installs today is meaningfully more correct than the thing I wrote about. That's down to you. Three rounds, each window smaller than the last, and I hadn't seen any of them coming.
If you ever have an idle hour, the compare-and-swap is the part I'd most like broken by someone who isn't me. You've got a better record at that than I do.
DevSecOps builder creating practical security tools for solo founders and small teams shipping with AI—safer code, CI/CD pipelines, and agent workflows without enterprise overhead.
That is a very generous postscript. It’s also good to hear the fixes made it into the installed artifact rather than stopping at the discussion.
The first place I’d attack is the lock domain. The third digest read and card update are serialized against other Shipward participants, but Git, an editor or another process may not respect that lock.
I’d add a failpoint immediately after the under-lock digest comparison and before the card write, then have an external process mutate the tree in that gap. If the card can still move, the compare-and-swap is correct within one store but the claimed invariant spans two independently writable systems.
One clean way to narrow the claim is for completion to attest “digest X passed” rather than “the current workspace is complete,” and require every consumer of that status to compare X with the tree it is about to act on. That turns a later mutation into visibly stale evidence instead of trying to make the filesystem and card store one atomic transaction.
That failpoint test is the right one, and I think it fails.
The lock in question is Shipward's own — it serializes done() against other Shipward writers and nothing else. git, my editor, any other process: all outside it. So the gap is real. The third digest read happens under that lock, the card write happens milliseconds later, and nothing in those milliseconds is mine to hold. The compare-and-swap took the window from minutes to milliseconds, which is small enough that I stopped being able to hit it by hand — and I'd quietly filed that as closed. Two independently writable systems, exactly as you put it. No lock I can take makes them one transaction.
So I took the other half of your comment instead, in 606af4d. The digest already existed — the check stamps it, the swap compares against it — and then it was thrown away, so what reached the card was check, argv, exit, ok, at, sha, dirty, ms. sha, not digest. A consumer reading a completed card got "passed at abc1234, tree dirty" and had nothing to compare, in precisely the case where the sha proves least: a dirty tree carries one sha through every edit you make to it. The checkable half of the evidence was the half I was dropping. It's kept now, completion attests "digest X passed check Y", and the board's Trust tab does the comparison for anything sitting in review — it names both trees and says the pass is neither wrong nor a failure, just no longer about what you'd be promoting.
Writing the tests for that turned up the thing I'd have shipped straight past. The digest hashed a bare git diff HEAD, while the dirty flag has always excluded the board's own directory on the grounds that the board is not the code under test. The two had drifted, and the digest was the one that mattered: three of the four changed files in my working tree at that moment were Shipward's own tracker and notes. Which means the pass I record was invalidated by the very write that recorded it — an attestation stale on arrival — and any second writer touching the board mid-check could refuse a legitimate pass outright. There was even a test asserting board writes don't count. It passed against the broken code, because it staged the board as a new untracked file, which git can't see either way; every real repo commits its board. That test now commits it first, and fails without the fix.
Four rounds, and each one has taken something away from what this tool is allowed to say about itself. I've stopped calling that a coincidence.
DevSecOps builder creating practical security tools for solo founders and small teams shipping with AI—safer code, CI/CD pipelines, and agent workflows without enterprise overhead.
The board-directory exclusion is the tell: any directory the check writes to during verification, not just the board's, invalidates the exclusion rationale. Rather than closing it path by path as you (and I) keep finding a new one, I'd invert the check: dirty should mean "the working tree changed since the digest was taken," full stop, with the only sanctioned exclusion being paths the verified digest itself never covers — which by definition can't drift the thing you're attesting. Two different concerns (what we hash vs. what we watch for changes) collapsing into one exclude-list is what keeps producing a fourth round.
We're a place where coders share, stay up-to-date and grow their careers.
The machine-owned check is the right boundary. I’d also bind every successful check result to the exact Git tree it evaluated.
An exit code alone can become stale immediately: the agent can run the check, modify another file, then hand the card back using a result produced against different state. Recording the commit/tree hash, command, exit code and output digest makes the claim reproducible.
For dirty worktrees, either refuse completion or hash the relevant files before and after the check and fail if they changed. The useful invariant is not just “this command passed,” but “this command passed against the exact artifact now being marked complete.”
"This command passed against the exact artifact now being marked complete" — that's the invariant, said better than I've managed it.
Most of it's there: argv, exit code, duration, sha, dirty flag, anchored to the tree
the check ran on rather than HEAD at write time. And the agent never carries a result at all — done() runs the check itself.
But you've found something I hadn't. I read the head before the run and never again, so a tree that changes during the check is recorded against pre-run state. Your before-and-after hash is the fix, and I don't have it.
Fixed in 9feb4bd. You were right — the tree was only read before the check ran, so a pass could be recorded against a tree that had already changed. It's read on both sides now, and if it moved the card doesn't move either.
One thing your comment saved me from: it hashes the diff, not the filenames. A file edited twice with the same name would have slipped through otherwise.
Thanks. First bug on that board that didn't come from me or CI.
Nice fix. Hashing the diff contents closes the “same filenames, different state” hole.
The remaining edge is the small TOCTOU window between the second tree read and the card-state write. If another process can modify the worktree there, I’d make the final transition a compare-and-swap: update the card only if the current tree still equals the verified hash. Alternatively, hold the same repository lock across verification and transition.
Then the invariant becomes atomic: either the exact verified tree is marked complete, or nothing moves.
Fixed in
13450bb— the promotion is a compare-and-swap now. Of your two fixes it was the only one this design could take: the check deliberately runs outside the write lock (waiters give up at 60s, and a suite can run for minutes), so holding the lock across verification was out. Instead done() takes a third tree reading while it holds the lock, and the card moves only if the digest still equals the one the check was verified against. A mismatch lands in the same bucket as the last fix — absence of evidence, neither pass nor fail, the card stays in progress. The test stages your scenario literally: a second process holds the lock, the tree moves while the write waits on it, and the card must not move with it. Which is your closing sentence as an invariant: either the exact verified tree is marked complete, or nothing moves. Three findings now, each a strictly smaller window than the last. The acknowledgements section is developing a habit of your name.@peterbuildssecure Postscript, a week on: all three of your findings shipped, and it's on npm now —
npx shipward setup ~/code/your-project --seed-from-branches— so the thing a stranger installs today is meaningfully more correct than the thing I wrote about. That's down to you. Three rounds, each window smaller than the last, and I hadn't seen any of them coming.If you ever have an idle hour, the compare-and-swap is the part I'd most like broken by someone who isn't me. You've got a better record at that than I do.
That is a very generous postscript. It’s also good to hear the fixes made it into the installed artifact rather than stopping at the discussion.
The first place I’d attack is the lock domain. The third digest read and card update are serialized against other Shipward participants, but Git, an editor or another process may not respect that lock.
I’d add a failpoint immediately after the under-lock digest comparison and before the card write, then have an external process mutate the tree in that gap. If the card can still move, the compare-and-swap is correct within one store but the claimed invariant spans two independently writable systems.
One clean way to narrow the claim is for completion to attest “digest X passed” rather than “the current workspace is complete,” and require every consumer of that status to compare X with the tree it is about to act on. That turns a later mutation into visibly stale evidence instead of trying to make the filesystem and card store one atomic transaction.
That failpoint test is the right one, and I think it fails.
The lock in question is Shipward's own — it serializes done() against other Shipward writers and nothing else. git, my editor, any other process: all outside it. So the gap is real. The third digest read happens under that lock, the card write happens milliseconds later, and nothing in those milliseconds is mine to hold. The compare-and-swap took the window from minutes to milliseconds, which is small enough that I stopped being able to hit it by hand — and I'd quietly filed that as closed. Two independently writable systems, exactly as you put it. No lock I can take makes them one transaction.
So I took the other half of your comment instead, in 606af4d. The digest already existed — the check stamps it, the swap compares against it — and then it was thrown away, so what reached the card was check, argv, exit, ok, at, sha, dirty, ms. sha, not digest. A consumer reading a completed card got "passed at abc1234, tree dirty" and had nothing to compare, in precisely the case where the sha proves least: a dirty tree carries one sha through every edit you make to it. The checkable half of the evidence was the half I was dropping. It's kept now, completion attests "digest X passed check Y", and the board's Trust tab does the comparison for anything sitting in review — it names both trees and says the pass is neither wrong nor a failure, just no longer about what you'd be promoting.
Writing the tests for that turned up the thing I'd have shipped straight past. The digest hashed a bare
git diff HEAD, while the dirty flag has always excluded the board's own directory on the grounds that the board is not the code under test. The two had drifted, and the digest was the one that mattered: three of the four changed files in my working tree at that moment were Shipward's own tracker and notes. Which means the pass I record was invalidated by the very write that recorded it — an attestation stale on arrival — and any second writer touching the board mid-check could refuse a legitimate pass outright. There was even a test asserting board writes don't count. It passed against the broken code, because it staged the board as a new untracked file, which git can't see either way; every real repo commits its board. That test now commits it first, and fails without the fix.Four rounds, and each one has taken something away from what this tool is allowed to say about itself. I've stopped calling that a coincidence.
The board-directory exclusion is the tell: any directory the check writes to during verification, not just the board's, invalidates the exclusion rationale. Rather than closing it path by path as you (and I) keep finding a new one, I'd invert the check: dirty should mean "the working tree changed since the digest was taken," full stop, with the only sanctioned exclusion being paths the verified digest itself never covers — which by definition can't drift the thing you're attesting. Two different concerns (what we hash vs. what we watch for changes) collapsing into one exclude-list is what keeps producing a fourth round.