On 27 July I opened a project I'd been building with Claude Code and found three things true at once:
a card had carried a null commit for two da...
For further actions, you may consider blocking this person and/or reporting abuse
The part I like is making the check an argv array owned by config. That sounds small, but it removes the agent's easiest escape hatch. I would still want the tracker to store the raw command output too, because a green exit code without the transcript is hard to debug later.
Dirty passes keep the output now — 9feb4bd.
Clean ones still don't, and I'm leaving that: a clean sha can be checked out and re-run to get the log back, so storing it forever costs every future session for nothing. A dirty tree can't be re-run, which is exactly the case you spotted.
Failures keep the output — exit code, duration, head and tail of the log with the
elision stated. Passes don't: just the check name, argv, exit, duration, sha, and
whether the tree was dirty.
The reason is that the note is the memory. Every future session re-reads it, so a
full test log is a cost paid forever, not just by the run that produced it. And a
green run at a clean sha you can re-derive — check it out, run the same argv, the
transcript comes back.
The dirty tree is where you're right. A pass over uncommitted changes isn't
reproducible from the sha, so there the transcript was the only copy. That I
should fix.
The machine-owned check is the right boundary. I’d also bind every successful check result to the exact Git tree it evaluated.
An exit code alone can become stale immediately: the agent can run the check, modify another file, then hand the card back using a result produced against different state. Recording the commit/tree hash, command, exit code and output digest makes the claim reproducible.
For dirty worktrees, either refuse completion or hash the relevant files before and after the check and fail if they changed. The useful invariant is not just “this command passed,” but “this command passed against the exact artifact now being marked complete.”
"This command passed against the exact artifact now being marked complete" — that's the invariant, said better than I've managed it.
Most of it's there: argv, exit code, duration, sha, dirty flag, anchored to the tree
the check ran on rather than HEAD at write time. And the agent never carries a result at all — done() runs the check itself.
But you've found something I hadn't. I read the head before the run and never again, so a tree that changes during the check is recorded against pre-run state. Your before-and-after hash is the fix, and I don't have it.
Fixed in 9feb4bd. You were right — the tree was only read before the check ran, so a pass could be recorded against a tree that had already changed. It's read on both sides now, and if it moved the card doesn't move either.
One thing your comment saved me from: it hashes the diff, not the filenames. A file edited twice with the same name would have slipped through otherwise.
Thanks. First bug on that board that didn't come from me or CI.
Nice fix. Hashing the diff contents closes the “same filenames, different state” hole.
The remaining edge is the small TOCTOU window between the second tree read and the card-state write. If another process can modify the worktree there, I’d make the final transition a compare-and-swap: update the card only if the current tree still equals the verified hash. Alternatively, hold the same repository lock across verification and transition.
Then the invariant becomes atomic: either the exact verified tree is marked complete, or nothing moves.
Fixed in
13450bb— the promotion is a compare-and-swap now. Of your two fixes it was the only one this design could take: the check deliberately runs outside the write lock (waiters give up at 60s, and a suite can run for minutes), so holding the lock across verification was out. Instead done() takes a third tree reading while it holds the lock, and the card moves only if the digest still equals the one the check was verified against. A mismatch lands in the same bucket as the last fix — absence of evidence, neither pass nor fail, the card stays in progress. The test stages your scenario literally: a second process holds the lock, the tree moves while the write waits on it, and the card must not move with it. Which is your closing sentence as an invariant: either the exact verified tree is marked complete, or nothing moves. Three findings now, each a strictly smaller window than the last. The acknowledgements section is developing a habit of your name.Who has the authority to assert this is exactly the right question, and it generalizes past task trackers. The same gap exists for architectural claims: an agent says this follows the pattern we agreed on, and unless something with actual authority checks that against a rule, you've got a self-report system for architecture too, not just for task status.
Agreed, and task status is actually the easy case — git doesn't care what the
agent claims, the commits and passing tests are either there or they're not.
Architecture is harder because most of the time the rule was never written
down in a checkable form in the first place. It's a paragraph in a design doc
or something agreed in a meeting, and you can't verify a claim against that.
Tools like dependency-cruiser or ArchUnit cover the parts you can encode
(imports, layer boundaries), but there's always a portion that won't encode.
I think the honest position is that this portion stays self-reported, and
the job is to keep it small.
If the agent can select which declared check to run, what stops it from picking the cheapest one? A card handed back against the lint check instead of the test suite would still get the green status. Curious whether checks are bound to specific cards or any declared check satisfies any card.
This makes me think of agent tracking less as a logging problem and more as an authority-design problem. If the same system can perform an action, declare it successful, and define the criteria for success, the feedback loop is inherently biased. Independent sources of truth become much more valuable as agent autonomy increases.
That's the design position, yes — and the one refinement I'd make is that the fix isn't only independent sources of truth, it's removing the agent from the reporting path entirely. The agent here doesn't run the check and carry back the result; it can only request the transition, and the tracker runs the check itself, under the same lock that moves the card. An agent that never holds the evidence can't bias it. The criteria are out of its hands too — a check is declared by the human, as an argv array in config, and a card can only name one, never define one. What's left to the agent is the part you'd actually want an agent for: doing the work and saying what it thinks it did.
The “who has the authority to assert this?” framing is a really useful way to think about agent reliability. In our AI work at IT Path Solutions, we’ve found that different claims often need different sources of evidence and different validation windows. A Git commit can confirm that code landed, for example, but it doesn't establish that the behavior remains correct after dependencies or surrounding logic change. Giving each claim a clear evidence owner, scope, and freshness boundary could make these systems much more trustworthy.
The freshness boundary is the part of this the tracker actually has an answer for. Every evidence entry is stamped with the sha it ran against, and every time a session re-reads the board the drift since that sha is computed and surfaced — commits since, and whether any of them touched the files the entry names. So a green check doesn't quietly stay green: it visibly ages. What it can't do is tell you which later changes invalidate it — that would need the dependency knowledge you're describing, and I don't think a tracker should pretend to have it. "This passed at that sha, and 14 commits have landed since, 2 touching files it names" is a claim git can back. "It still holds" isn't, and the honest move is to keep those two sentences apart.
The authority-owner framing maps directly to agent deployment too. A worker should not be allowed to both perform a tool call and declare its outcome: the durable ledger or provider response has to establish that claim. I use a drain boundary where new claims stop, in-flight effects become terminal or UNKNOWN, and a replacement can reconcile from the idempotency key. Otherwise a self-reported “done” can hide the same gap as a stale tracker card.