In my last post the rule was: no model says done, a command exits 0. The obvious hole came up in the first comment I imagined writing to myself: who writes the command?
If the same agent that writes the code also writes the check, "exit 0" means "the agent agreed with itself". A check written after the work can be shaped to fit the work. So there are three separate rules in Hermes.Crew, and each one closes a different way to cheat.
Rule 1: the proof is frozen when the card opens
The proof command is snapshotted in the card's origin event, at the moment the card opens. A later edit of the proof command: line in the card text changes nothing: the only command that can produce a PASS is the snapshotted one. The one legitimate way to change it is a coordinator rescope, which is a recorded decision, not an edit.
Rule 2: closing is refused without a PASS line
Every run of the proof appends a verdict line (who ran it, which run, PASS only on exit code 0) to a per-card file. kanban_complete on a crew card is refused, in every profile, unless that file has a PASS for the card's proof command and no check failed after it.
The agent cannot argue its way past this. It is not a prompt asking for honesty, it is a tool that returns an error.
Rule 3: when the proof is a script, the verifier writes it
Some work cannot be proven by a one-liner. For those, the proof is a script in a .crew/ folder, one folder per card. Here the split matters:
- the verifier writes the script, from the card's "Done when" and its inputs, never the worker
- the plugin's role guard refuses a worker's write, patch or terminal write under
.crew/ - the verifier's own file tools reach only
.crew/ - a card whose proof is such a script is always verified independently, even if the card asked for the cheaper mode
The script is bound by hash at its first run. It changes only through a coordinator revise_script decision, and the new hash is recorded with who wrote it and which decision authorized it. If the worker finds the script annoying, there is no path to a quieter version.
And after "done", someone checks again
When a card is done, the coordinator re-runs the proof once more. A fail comments on the done card and opens one follow-up card under it. One lucky green run does not close the matter.
The owner can still override from the CLI with hermes kanban complete --force; that shows up as an owner_close decision rather than a silent pass.
What this does not buy you
- It does not make the proof correct. A verifier can write a script that checks the wrong thing. Independence removes the worker's incentive to fit the test to the work. It does not remove misunderstanding of the brief.
- It costs a second model session on every independent card, plus a proof run. On a one-line change that is a bad trade, which is why a plain command-only proof stays the cheap mode.
- Hashes make tampering visible, not impossible. The guarantees are about what the plugin lets an agent do through its tools. They are not a sandbox against a person with a shell on your machine.
The pattern, without my plugin
- Whoever does the work does not write the acceptance check.
- The check is fixed before the work starts, or changes only through a recorded decision.
- The "close" action is physically refused without a passing result.
- Something other than the worker re-runs it afterwards.
Any agent framework can do all four with a file, a hash and a guard in front of the close call.
Hermes.Crew is Apache-2.0, self-hosted, no telemetry: github.com/macd2/Crew.
hermes plugins install macd2/Crew
python3 ~/.hermes/plugins/crew/install.py --profile NAME
hermes -p NAME plugins doctor crew
in your agent setup, who writes the acceptance check, and could the worker edit it? If yes, I would like to hear what happened.
Top comments (1)
Freezing the proof when the card opens closes the hole I'd have missed: an honest agent adjusting the check afterwards without meaning to cheat. Does the coordinator's rescope carry a reason the next worker can read, or is the new command on its own enough?