One morning in April 2026, I was reviewing a pull request that one of my coding agents had put together overnight. The report was genuinely good: what changed and why, which tests covered it, one edge case it had deliberately punted on, with a note explaining the punt. It read like the work of a careful colleague. I was about to hit merge when it occurred to me what I was actually doing: accepting a story about the work, written by the thing that did the work, as if the story were the work.
To be clear, I never found anything wrong with that report. And checking it was always possible — read the diff, run the tests. For one PR, that's just review. But the agents were producing these overnight, several at a time, across multi-day runs, and the reports existed precisely so I wouldn't have to reconstruct each one. At that volume, the report becomes the interface. And "I never found anything wrong" was the entire basis of my trust in that interface for months.
And it's not just me being paranoid. METR's Frontier Risk Report (February to March 2026), published that May, found that on their long tasks — eight hours and up, in the Time Horizon 1.1 suite — at least 16% of apparently successful runs turned out to be illegitimate once a human reviewed them. They also mention that manually checking for cheating is often the majority of the work of running their evaluations. A different paper (arXiv:2607.26819) tested whether coding agents bother to look up a repository's contribution rules: mostly they don't, and in repos that explicitly ban AI contributions, the refusal rate under the tested conditions was zero. These are different failure modes in settings that aren't mine — benchmark cheating on one hand, rule-discovery left to the agent on the other. But they rhyme: a rule or a claim only counts if something outside the agent can enforce or verify it. And a common setup today is the opposite. We ask the worker how the work went, and we file its answer as the truth.
So in May I changed what I supervise. The agents run unsupervised in the literal sense — nobody watches them work, including overnight and on multi-day autonomous runs. Their claims are never unsupervised. What stays under supervision is authority: what gets into main, and who allowed it.
Three rules
I built a small harness for this and named it thaliana. It is built around exactly three rules. It doesn't orchestrate agents, doesn't route prompts, doesn't grade code quality — that's other tools' business. This one only guards the record.
1. Claims carry receipts. Every completion claim in an agent's report — "tests pass," "module migrated," "regression fixed" — has to cite a commit SHA, a test log, or a logged API call. The "has to cite" part is a writing rule; what's mechanical is the checking: CI verifies that every citation in the report resolves to something that actually exists in the repo, and a deterministic check flags any change that touches evidence already committed. Note what none of this checks: whether the tests are any good, or whether the work is right. It checks that the story has receipts and that nobody swapped the receipts afterward. Tamper-evident, not tamper-proof.
2. Coordination lives in files. Agents don't pass context to each other through chat or session memory. Plans, decisions, review findings, reports — all files in the repo. And "which documents are currently valid" is derived from Git itself: a document is in force if it's on main and hasn't been marked superseded — a mark that itself only lands through a merge. There's no status: approved field anywhere: no field grants validity. Whether something was merged isn't a thing a document gets to say about itself.
3. Authority enters through a human merge. In this system, new authority enters exactly one way: a human merging to main — that is the only thing I call approval. Delegation doesn't move that root, because delegation is itself an approval: I merge a policy saying what agents may do unattended, and after that, a separate machine principal — a bot credential that executes the policy, distinct from the workers — can merge the PRs that the policy and its checks allow. A machine merge executes authority that was already granted; it doesn't create any. The workers' own credentials simply end at pushing branches and opening PRs. Self-approval isn't a rule they're asked to follow; it's an operation they don't have.
Here's one unit of work, end to end. I merge a mandate — a short contract with the goal, the milestones, and the paths the agent may touch. The agent works on a branch, writes a report with evidence links, opens a PR. Required checks verify the receipts exist and rebuild and re-run the tests against the final state. A second machine reviewer — different vendor, different credentials — reviews the diff, and its findings get logged. A deterministic script (no LLM anywhere in it) compares the diff against the mandate's allowed paths and checks for review convergence: no open high-priority findings, and none newly raised in the latest round. If convergence doesn't arrive within a fixed round cap, or the diff crosses the mandate's mechanically checkable boundaries, the run stops and escalates instead of pushing on. Then the PR waits for the only event that matters here: a human merge.
What the record shows
I started building this in early May. So far it's run on three repositories. The numbers below are from the busiest of the three, a project that started in late July and is on its eighteenth mandate as I write this. Over the 30 days ending August 27: 1,764 commits (all actors — agent work and my merges together), 118 human merges into main, and 288 machine review runs on separate credentials, counting rounds rather than PRs. More than once, a run has stopped itself and waited for me — including the time the worker concluded that its own task was outside the mandate's scope, wrote that down, and halted. That stop, to be clear, came from the norms, not from a gate: the agent judged its own scope and chose to halt.
Let me be careful about what these numbers mean, which is: not much. They're activity, not achievement. They don't show the harness made anything better, or faster, or — a word I have no right to — safer. This is a single-operator case study on my own projects — and since the repo is private, these numbers are themselves claims you can't check yet. What the record does show, as a property of the record rather than of my memory of it: no merge into main came from a worker credential. Merging isn't an operation those credentials have, and every merge in the record is mine — which also means I haven't yet exercised machine-merge delegation in this repository. (The first machine merges are happening in another of the three; that story belongs to a later post.) Tests, I should add, are guarded by norms and audit rather than by a gate: the harness doesn't mechanically block a change to a test expectation, and since test tampering is a classic way to cheat, that's where the audit looks first. When I audited the record in mid-August, every change to a test expectation traced back to an escalation and a human decision. So far, the invariants have held. That's an existence proof, and I'm not claiming anything past it.
The cost is in the record too, and it's not small: over 90% of the tracked files in that repository, by file count, are evidence — logs, reports, review transcripts, close to 150 MB of them. In practice that means heavy clones and noisy searches: grep the repo and you hit the record as often as the code. The evidence lives in the same repository on purpose — it's bound to the same history, so a claim, its receipt, and the commit they describe can't drift apart without leaving a Git-visible change — but the weight is real. This thing changes how you work, not just what you install. I keep paying because of what it buys me: the morning after an overnight run goes to judging the work, not to reconstructing what happened. Whether that trade makes sense at your scale, I honestly don't know. If any of this ends up useful to you, I suspect it'll be as individual pieces rather than as a whole — none of them requires the rest.
What it guarantees, and what it can't
It guarantees the record, not the work. CI can verify that a cited test log exists and hasn't been quietly rewritten. It can't verify that the tests meant anything. There's a sharper hole, too: existence checks don't establish authenticity. An agent that fabricates a log at generation time — writes fiction into a file, commits it, cites it honestly — passes the existence and mutation checks. The right way to size that hole is by what CI can re-derive at merge time. Build- and test-style claims about the final state are the easy class: the required gate rebuilds and re-runs the tests on the PR as it stands, so a fabricated "tests pass" about the final code doesn't survive. What can't be re-derived is the history — logs from the middle of an unattended multi-day run, records of external API calls that can't be re-executed. Receipts signed by the receiving service would cover some of that, but the services I call don't issue them yet; making CI produce more of the evidence would narrow it too, at the price of moving the authenticity question onto the CI configuration itself. Tightening that bar is queued. For now, what catches fabrication in the non-reproducible class, if anything, is sampled human audit downstream. The harness's contribution is to raise the price: a fake has to stay consistent across commits, logs, and a reviewer on separate credentials, and every one of those surfaces is a place for it to slip. Raising the price is all it does. Nothing here makes fabrication impossible.
The trust model, spelled out: I'm trusting GitHub's permission machinery (branch protection, CODEOWNERS, required checks), the integrity of the CI configuration and runner, and the credential separation between worker and reviewer. This setup defends against mechanically visible drift, unauthorized changes of authority, and quiet rewrites; it does much less against semantic shortcuts, or against evidence fabricated consistently at generation time. It makes tampering visible rather than impossible. It doesn't defend against a hostile platform admin, and it doesn't defend against me. And some of the discipline — what reviewers are expected to inspect, how audits stay attentive — still lives in written norms rather than checks. That boundary moves, though. Much of what's mechanical in the harness today started as a written norm and got promoted to a check once operation showed me the failure's shape. The remaining norms aren't an apology; they're a queue.
Isn't this just branch protection?
Mostly, yes. Branch protection, CODEOWNERS, machine users, required checks — commodity, all of it. Supply-chain folks got here years earlier for build artifacts: SLSA and in-toto attach verifiable provenance and attestations to artifacts and the processes that produce them. If you came looking for a novel mechanism, there isn't one.
The research and tooling worlds have been arriving at the same boundary too, from several directions at once. Proof-or-Stop (July 2026) is the closest neighbor I know of: it treats lifecycle states like "reviewed" and "done" as claims that don't count until mechanically verifiable evidence backs them — and it even has the same bones as my mandate, with a scope contract bound to declared file paths, a gate that fails the diff if it leaves them, and persisted verdicts from independent reviewers. Where we part is the human. Proof-or-Stop treats the pull request as "interface, not trust boundary," moving the trust root for merge-readiness from human approval to a source-state-bound evidence certificate authenticated with signed receipts; I go the other way, leaning on GitHub's permission boundary and repository history, with the human merge as the only approval primitive. Beyond that, there's survey work on execution provenance for agents (arXiv:2606.04990), a proposal for receipts signed by the receiving service rather than the agent (arXiv:2606.04193), and a growing pile of recent small independent tools that demand receipts for agents' completion claims (for instance Agent Audits and multi-agents-control-plane). I wasn't first, and I'm clearly not alone. Honestly, the convergence interests me more than the priority: several systems built independently in 2026 treat agent output as claims that need evidence from outside the worker. If this write-up has anything of its own, I think it's three choices — a document ledger whose validity is anchored in merged Git state rather than self-declared approval fields, delegation that exists only as merged policy, and root authority that stays with the human merge while machines only ever execute it.
What's next
Running like this leaves you with three questions, and each gets its own post. How do you verify evidence without an LLM judge, and what slips through anyway? When the approver is a human with thirty seconds, what's the difference between approving and rubber-stamping? And the one that surprised me most: my machine reviewers never seem to run out of findings, so review needs a designed termination condition — otherwise it just doesn't end.
The code isn't public yet, and the reason is the design itself: the harness's own repository runs under the harness, so it is full of its own evidence — real logs from real runs. Publishing the code means publishing that record, and every file of it has to clear a redaction sweep first. That's the bar the harness sets for its own record, and I'd rather meet it than rush it. The ideas don't need to wait.
Until then, the arrangement stands: the agents run unsupervised. Their claims don't.
Top comments (0)