Two loops and a human

Six weeks of running coding agents on real projects, and what the design looks like now

Status, stated up front. The slow loop in this piece has run unattended since mid-September: daily digest, weekly sweep, monthly dreaming, each ending in a pull request a human approves. The fast loop has run too, on one project, for about ten days in September, and taught most of what follows. It is not running today. Every project has since moved to a different orchestrator, and the headless shape for it is undecided. I would rather say that than describe a diagram as live when it isn't.

---

The problem this solves

One person, several projects, and agents doing most of the typing. The failure mode isn't that the agents are bad at code. It's that they are confident, fast, and have no memory of last week. Left alone, they will re-derive a rule you already learned, write into a directory you meant to keep clean, call a half-finished job "complete", and do it all in well-formed prose.

So the design question was never "how do I make the agent smarter". It was: where does judgment live, where does state live, and what is the one thing a human must do?

The answer that held up is two loops with one gate between them.

The shape

• A slow loop runs in a hub repository that holds decisions, rules, and the distilled memory each project reads. Its jobs are unattended. They may capture, route and propose. They never merge, never delete, never edit the human's working copy. Every run ends in a PR, or in a message saying why it couldn't.
• A fast loop runs inside each project, phase by phase: define, plan, build, review, ship, with human gates at define, plan and ship. It ends in a PR too.
• The gate is the same for both: a protected main branch, one approval that any later push invalidates, one required check, and no one who can go around it. Who approves whom is the interesting part, below.
• The channel between them is one-directional each way. The hub exports a read-only snapshot (rules, skills, one context "pack" per project) that projects read. Projects write findings into a shared drop, one file per item, which the slow loop files into the hub by PR. Projects can't reach the hub checkout at all; the hub never writes into a project except through that export.

The human sits on the gate. Everything else is machinery to make the gate cheap.

The slow loop

Job	When	Proposes	The human does
Machine checks	daily	a status file: every check pass / fail / unknown; a missing check is unknown, never clean	reads it through the digest
Daily digest	daily, on a hosted runner	one comment: open PRs each with a one-line mergeable-or-blocked verdict, decisions falling due, open blockers	answers batched decisions in one reply
Weekly sweep	Sundays	drains the drop, routes notes, regenerates packs, refreshes the export, and proposes a verdict table for every aged follow-up	says go or go except …
Monthly dreaming	1st of the month	rewrites the distilled memory file and nothing else	approves one PR

Two design choices matter more than the schedule.

Prompts are files. Every job's prompt lives in the repo and runs through the agent's command line, headless. Changing how the sweep behaves is a reviewed PR, with the same gate as code.

Jobs propose tables, not questions. The first version of the sweep listed aged items and asked "close or park?" per item. That is a chore, and chores don't get done; the list grew to 22. The current version proposes a verdict per item with the evidence, prints a hash of the table, and the human answers once. The apply step refuses if the table changed after approval. Every "the system asks me the same thing every week" became a table with a hash.

The gate: who approves whom

The hosting platform won't let an identity approve its own PR. That constraint became the design:

• The human's PRs need an approval from the automation's reviewer: a model review pass that always runs from the trusted branch's copy of its own definition, so a PR can't bring its own reviewer.
• The automation's PRs (sweep, filing, dreaming) need the human's approval.
• The reviewer withholds approval on exactly one signal: who authored the PR. An earlier version withheld on file paths and made two human PRs unapprovable; both merged on recorded exceptions, which is the thing the rule exists to avoid.

A PR is mergeable when four things hold at the same commit: a standing approval from one of those two identities, no commits since it, CI green (no checks at all is not green), and a clean security pass. One script prints that as one line, and the digest prints it for every open PR. "Is this ready?" stopped being a question.

Two smaller rules carry weight. A scope ratchet: after the first review verdict, later commits may change what is already in the diff but not widen it, unless a human says so explicitly. And a bootstrap exception: a PR that changes the reviewer itself can't earn the reviewer's approval, so it takes a one-time exception only the human grants, recorded where it can't be lost.

The channel

Out. The export is generated from a committed revision, never a working tree. That rule came from finding the export four days and 174 commits behind the main branch because it had been copied from someone's checkout. Each export carries a manifest the projects can read but not write; a project's copy of the rules is verified against it, so a project can't vouch for its own copy. Every number the verdict uses comes from files only the exporter wrote. Both of those came from review findings on the same PR, and the fix for the second was to delete the "grace period" that trusted the project's copy at all.

In. The drop. One file per item, written as .<name>.partial and renamed when complete. Intake moves files into a hub-owned staging directory before reading them, and skips byte-identical repeats by hash. A failed move is never logged as "vanished". Another lesson from a run that printed "drop is clear" over 58 unfiled files.

The fast loop, honestly

The first fast loop wrapped a phase-based orchestrator so a phase could run headless on a project's own runner, with exactly one human decision in the middle: approve the plan. It ran on one project, a small game, for about ten days. In that time:

• A resume path never armed the cut between planning and executing, so an unapproved plan executed for 78 minutes before anyone noticed. Rule: the cut is armed from the run's own state, at the shared entry point, with no opt-out.
• The orchestrator printed "Phase 14 is already complete" and the wrapper, not recognizing the sentence, parked for a human. The next day a reworded banner did it again. Rule: finished is a state, not a sentence; on an unknown stop, query the tool's own state three times, and promote to completion only if they agree. Never demote.
• Blocker issues opened and never closed: seven on one phase, no way to tell which was live. Rule: the run that resumes a phase closes the blocker that parked it.
• Every stop was classified into a named shape with a named action and retry budget, and the bias was written down: escalate when unsure, and never send "you decide" to the agent.

Then the orchestrator changed. A newer one shipped its own gates, its own pre-approval flags, and its own disk state. Rebuilding the wrapper around it would mean two state machines over one pipeline. So the fast loop is being redesigned as: a thin environment-owning wrapper, the orchestrator's own gates as the stops, and the hub only watching. That design exists on paper. The first headless milestone under it hasn't run. The first supervised milestone on the new orchestrator hit an upstream bug on its own patch route, and is parked until the fix lands upstream, which I've filed and offered to test.

That parked milestone is also the clearest statement of the discipline: nothing in the project's state was hand-edited to get around it.

The autonomy ladder

"Autonomous" was a mode a project was either in or not. The headless runs showed that was the wrong shape: every phase was a series of stops, and whether a machine could answer a stop depended on one thing, what the project's tests decide without a human.

So autonomy became a per-project level, derived from two measured numbers:

• Oracle coverage. The project declares its layers in one file. A layer is covered when its oracle is decisive (red means wrong, the run stops) and proven (made to fail on a deliberate break, with the date and commit recorded). Coverage is counted per layer, not per test. A layer declared "none, human review" counts against coverage; hiding it fails the gate.
• Escalation rate. Stops the machine couldn't answer from the pack, the budget, or the approved plan, per phase, over the last N phases.

The ladder: L0 unassessed; L1 supervised phase (the human answers every stop as it happens); L2 supervised plan (the human approves the plan, then the phase runs unattended); L3 unattended phase; L4 unattended milestone; L5 unattended roadmap (the project proposes its own next phase). L5 is deliberately unspecified. Nothing in the repo has evidence for what would make it safe, and writing criteria from imagination is how a gate becomes a formality.

The level is a standing state the last window of phases supports, never the highest a project ever reached, and it caps how a phase may run.

Why dreaming is monthly

The distilled memory is one file, under 200 lines, that a session carries in. Daily and weekly jobs change the hub constantly; why rewrite the memory only monthly? ("Dreaming" is the hub's name for that rewrite.)

Because it is a full rewrite reviewed as a diff, and a full-rewrite diff is only reviewable when enough has changed to matter. Weekly, most of the diff would be noise and the review would become a rubber stamp, which is the one thing a memory rewrite can't afford: a bad rewrite deletes what you knew. Monthly keeps each review real and each revert cheap. The fast-changing state already has a home: the digest carries today's, the sweep carries the week's. Dreaming is for what stayed true long enough to be worth remembering.

The repo's own record of the choice is less principled than that: it was sequenced last among the unattended jobs because the lowest-frequency job is the slowest to reveal a fault. The test of whether monthly is right is simple and not yet run: measure how much the file changes per rewrite, and whether the digest catches it contradicting itself mid-month. Small diffs, no contradictions: monthly holds. Otherwise, a weekly staleness check that proposes lines but rewrites nothing.

The security angle

The threat model isn't a malicious agent. It's a fallible one with write access and confidence, plus the ordinary supply chain around it. The rules fall into five groups, and every one came from an incident:

1. Identity. Automation acts under its own identity, never as the human. Approval is keyed on who authored a change, the one signal an actor can't forge. The automation can't change the definitions of the jobs it runs; those changes are the human's to make, permanently.
2. Credentials. The step where the model works holds no write credential; a separate step publishes. Credentials are short-lived and issued by a service the runners can only ask, which fails closed. The earlier design, where processes under one account shared a channel, produced a batch of findings that were all one class, and was replaced. No credential store lives in any repository.
3. The reviewer can't be captured. Reviews always run from the trusted branch's copy of the review job, with the model told to ignore the reviewed tree's own settings. A static check on the job definitions runs on every PR, with every suppression carrying its reason inline.
4. Sessions are boxed. A session never disables its sandbox. The human's checkout is write-denied to every session; work happens in worktrees and lands by PR. Hub sessions never write to projects; project sessions never write to the hub except through the drop.
5. Inputs are untrusted, including the hub's own copies. Drop files are moved into hub-owned staging before being read. A project's copy of the rules is verified against a manifest it can't write. And the rule that closed two findings on one PR: no number a verdict depends on may come from a file the checked party can write.

The pattern across all five: every check that could be satisfied by the thing it checks got rewritten until it couldn't.

Five rules, with the incidents behind them

1. A check must be seen to fail. Eleven checks that could not run read as passes; three edits removed controls; all found by people looking back. Now a retargeted assertion carries a negative control in the same commit, and a "sabotage run" (remove the rule, show the test fails) is part of the PR.
2. A failed step never renders as complete. "Drop is clear" over 58 unfiled files. Every job prints pass / fail / unknown (could not run) per check, against a declared roster.
3. Derive, don't restate. A workflow re-written from prose beside its reference missed five things, the last one on the parity test meant to catch them. Second copies are derived from their source and tested for equality.
4. Fix the project or file upstream; never duct-tape around the orchestrator. Before changing anything to get past a stop: read the tool's own doc for the state you're in; ask whether the change would be right if the tool had no bug; never write the tool's state by hand; a stop you can't route is reported verbatim and the milestone waits.
5. Propose, don't ask. Every recurring question the human was asked became a table with a hash and a single go.

What isn't working

• The fast loop isn't running, and its shape on the new orchestrator is a proposal.
• Several rules exist in name only: a named test that was never committed, a frontmatter round-trip check that was designed and not built, seven onboarding rulings not yet in the template that new projects read. The sweep now reports these; it didn't for a month.
• No project is above L0 on the autonomy ladder yet; one is the first candidate for L1 now that its readiness gate passes. The escalation record that moves a project up comes from the fast loop, which is the thing being rebuilt.
• One milestone is parked on an upstream bug.

I'm publishing anyway because the slow loop is real and the lessons are real. The design's main claim is modest: put the human on one gate, make everything else propose, and make every check unfakeable by the thing it checks. Six weeks in, that part holds.

---

I'm Shai Kappel, founder of Indie Desert Studio. If you're running agents on real projects and want to compare notes, find me on LinkedIn.