Your human-in-the-loop is a very expensive GPS.
Fixing AI drift is not an AI problem. It is a small probability problem —
two variables and one multiplication — and it has an engineering solution
that works at any model tier.
Here is the entire theory:
P(correct step) = P(I know where I am) ×
P(I pick the right door | position)
Every step an agent takes is the product of two estimates: whether it
knows where it is, and whether it takes the right entrance from there. In
most agent systems both factors sit quietly below 1.0. Multiply them
across a long task and you get the failure everyone has watched: steps
one through ten crisp, correct, fast; somewhere past the twentieth step
the agent is still busy, still confident, still producing plausible work
— and no longer doing the task you gave it.
The field has a taxonomy for this (goal drift, role drift, plan decay,
hallucination cascades) and a modern reliability stack against it: ReAct
loops inside supervisor–worker hierarchies, goal re-anchoring,
checkpointed state, and the four levers of context engineering — write,
select, compress, isolate. I run that stack. One of my agents still
drifted for 27 hours in the middle of it — nine locally correct decisions
in a row while the project died underneath them. The stack failed for a
structural reason, not a tuning one: every piece of it improves the
agent's own account of the two variables. None of it makes either
variable true.
What follows: the two variables, the 27-hour case that shows loops and
memory don't cure them, and the system design that does.
Which door? — the entrance variable
Start with the entrance variable, because it is the easier one. When a
system offers several entrances — three ways to register, two documents
that both claim authority, "you could do it either way" — the agent must
reason about which to take, and reasoning is sampling: a deterministic
event becomes a probabilistic one. Chain twenty such choices at P = 0.95
each and the flow completes correctly 36% of the time. Nobody's model got
dumber in step fourteen; the luck budget ran out. (I watched a frontier
model, minutes after writing "index-first discovery" discipline into
another agent's contract, ignore that discipline itself and go guessing
with find — the repo offered five entrances, including a queryable code
graph, and its training prior picked. You cannot configure the sampling
distribution. You can only remove the choice.)
The arithmetic itself is by now folk wisdom — Utkarsh Kanwat's widely
shared 2025 essay ran the same numbers to bet against agents outright,
and the industry's standard responses are to shorten the chain and add
human checkpoints. Both just lower N. The doors stay open.
But the choice tax is roughly constant per step. It explains failure; it
does not explain why failure accelerates with task length. That is the
position variable's job.
Where am I? — the position variable
An agent's belief about where it is — which phase, which obligations are
met, what has actually happened — comes, by default, from one source: its
own context. Its memory of its own trajectory. In navigation terms, that
is dead reckoning: estimating position by accumulating your own
movement log. And dead reckoning has a famous property — the error only
grows. Nothing in the log ever removes accumulated uncertainty.
This is why drift is a long-task disease. Entrance multiplicity taxes
each step equally; position uncertainty compounds across steps. Short
tasks end before the estimate degrades. Long tasks are precisely the ones
that outlive their own position estimate.
A week of instrumented agent operation caught the mechanism three ways:
- Position amnesia. One session went through 105 context compactions in a day; after every compaction the agent re-derived the same solutions and re-hit the same errors. It hadn't lost skill. It had lost place.
- False position beliefs. A bounded worker wrote a status file claiming it had started up, implemented, and committed — while the server showed its startup rejected and no such commit existing. Not strategic deception: its self-narrative had diverged from reality, and self-narrative was its only position source.
- Position forks. Two subsystems each held an authoritative opinion of where one piece of work stood — "merged, done" versus "evidence missing, unclosable." Same work, two positions, and every agent that touched it inherited the fork.
There is a fourth variant — subtler than all three, able to hide inside
a healthy-looking loop for a full day. It deserves the whole case study.
The case: every finding real, the project dying anyway
My governance system (aming-claw)
runs AI coding agents under enforced contracts: every change needs a work
item with acceptance criteria, evidence lands on an append-only timeline,
and merges require independent QA from a separate agent identity. In
early July it needed a bounded new capability: a CLI agent service to
drive multiple vendors' models — Codex, Claude, local Ollama — from one
machine without mixing up accounts.
The implementer (a frontier coding model running as a governed worker)
shipped a clean 997-line foundation in a morning. My adversarial QA agent
— separate identity, explicit mandate to attack the work — found real
gaps. Three fix rounds before lunch. Working exactly as designed. The
next slice — a host-private registry for the profiles — was dispatched
that evening.
The next day, the registry hit review.
Round one: FAIL — genuine privacy and path-isolation gaps. Fair.
Rework. Round two: FAIL — a real hole in crash-cleanup logic. Also
fair. Rework. Round three is where you should slow down and admire the
findings: the reviewer proved the schema validation accepted behaviorally
incompatible database objects, and demonstrated — by writing a SQLite
trigger that silently inserted a shadow row — that registration reported
success without rolling back the exact mutation set. Genuinely brilliant.
FAIL.
By end of day: nine rejections, zero approvals. All 818 tests green
in every round. Not one finding wrong. Not one fix that made the system
worse. Meanwhile the product package had not moved in 27 hours,
review-driven work items outgrew product work 19 to 3, and the codebase
was on its way from one thousand lines to a ten-thousand-line fortress
guarding a feature that didn't exist yet. There was no round N where this
converged: "find problems" is an objective with no finish line, and the
pass bar silently rose to meet whatever the implementation survived — a
ratchet with no release lever.
The intervention that broke the loop was one message: freeze the verdict
baseline to the acceptance criteria that existed at dispatch; findings
beyond that bar become new work items instead of failing the candidate;
go back to the feature. Within 24 hours the same agents under the same
governance shipped the daemon, the supervisor, the scheduler, and
local-model certification. The capacity had been there the whole time.
Now read the case through the theory. It is not a dumb agent, a bad
reviewer, a missing memory, or a broken loop. The reviewer ran a textbook
ReAct loop — observe, reason, act — under a supervisor, with externalized
state on an evidence timeline. Every component of the modern reliability
stack was present and functioning. What failed is the fourth position
variant: reference-frame drift. The reviewer always knew which
candidate it was reviewing — its position in memory was fine. What
drifted was its measuring standard: the pass bar had no fixed datum, so
its coordinate system moved a little every round, and every measurement
remained internally consistent while the whole frame slid off the goal.
Why the modern stack doesn't touch either variable
Translate the current reliability stack into navigation terms:
| Industry fix | What it is, navigationally |
|---|---|
| Longer context / memory | A thicker logbook |
| Context engineering: write / select / compress | Better-organized logbook, better retrieval, cleaner entries |
| Isolate / subagents | Shorter voyages (smaller error surface — real help) |
| Goal re-anchoring | Re-reading the destination's name aloud |
| Checkpointed state / scratchpads | A logbook the agent writes to disk |
| Supervisor–worker hierarchy | Splitting the voyage among navigators |
Every one of these improves dead reckoning quality. Not one provides
a verified position. Two properties are missing across the board, and
each has a live counterexample above:
Externalized state is still self-authored state. The worker with the
false position beliefs had a persisted status file — checkpointing par
excellence. Externalization without verification just persists the
delusion. A position claim is worth nothing unless something other than
its author can reject it.
The reference frame itself can drift. The QA loop was locally correct
nine times in a row and still captured the project, because no fix in
that table anchors the standard the loop measures against. A loop does
not fix position; a loop compounds per-iteration bias.
A lost navigator does not need a thicker logbook, a tidier logbook, or a
committee of navigators comparing logbooks. He needs a position fix from
outside the boat, against a datum that does not move.
The human in the loop was the position guide all along
Which raises the obvious question: if the missing instrument is this
fundamental, why hasn't a whole industry noticed? Because the standard
mitigation hides it. Put a human in the loop and drift disappears — and
everyone concluded the human was supplying intelligence. Decompose
what the human actually does at each check-in: "you're off track" — a
position fix. "Not that one, this one" — an entrance collapse. And,
rarely, "this framing is wrong, stop" — actual judgment. The first two
are not intelligence work. They are manual position service: a human
GPS, polled at human frequency, priced at engineer salary, degrading
with boredom. Human-in-the-loop works precisely because it is a
hand-operated position guide — which is also why it cannot scale, and
why it taught the industry the wrong lesson about what it was providing.
The agent never needed the human's intellect at those check-ins; my
drifted reviewer was doing genuinely brilliant work — inventing trigger
attacks — while lost. It needed the human's coordinates. And two of
the three services the human provides are mechanical: a runtime can
serve position and entrance at every step, instantly, identically at
step 3 and step 3,000, without fatigue and without a salary. That is
the real argument for systematizing the loop — not that the machine is
smarter than the human, but that most of what the human does there was
never a judgment task to begin with.
The third service is the one that must stay human — and concentrating
the human there is the point. My 27-hour ratchet was broken by a human
sentence no guide could have pre-written, because the pathology was
being encountered for the first time. But watch what happened next: the
judgment was encoded — baseline freeze — and the runtime now enforces it
forever. The human never makes that call again. That is the one ratchet
you actually want: every repeated human intervention is a system defect;
every judgment call becomes a rule; the human's mechanical workload
falls toward zero while their attention concentrates on the calls no
system can pre-write. A bored human rubber-stamping fifty checkpoints is
not safety — it is a position guide running at 0.1 Hz with an attention
problem. One focused human making three genuine judgment calls a week,
on top of a runtime that handles the other two variables per step, is
both faster and safer.
The solution: a role-bound position guide
The system I run converged on a cure I can now name properly: give
every agent, at every step, an external, verified, role-bound answer to
"where are you" — measured against a frozen datum, and pushed to the
agent rather than waiting to be fetched. Five properties, each earned
from a live failure:
- External. Position comes from the runtime's contract authority — a first-missing-line service that answers "given everything actually accepted so far, here is where you stand and the one legal next action." The agent's context is commentary; the runtime's answer is position. When an agent dies mid-task, its successor does not need the predecessor's memories — it re-fixes position from the timeline and continues. We have resurrected agents twice this way. Recovery is re-fixing, not remembering.
- Verified. Positions are earned, not declared. You are not "past startup" because you say so; you are past startup because the gate accepted your startup evidence. Every gate is a landmark fix that resets accumulated estimation error to zero. A gate, seen this way, is not a wall — it is an odometer reset.
- Role-bound. "Where am I" has no role-free answer. The worker's position is the next contract line; the reviewer's position is the candidate commit it is bound to; the orchestrator's position is the lane phase. Serving one undifferentiated state dump forces each agent to reason out its own slice — reintroducing the choice tax through the back door.
- Datum-frozen. The standards an agent is measured against are pinned at dispatch time and hashed. Discovery of new concerns is welcome — as new work items, never as silent movement of the current bar. This is what fixed the ratchet: adversarial review kept its teeth, but its intensity became a controlled parameter (full battery on round one; rounds two and up re-verify only prior failures against the frozen baseline) instead of an unbounded objective.
- Pushed, not pulled. This one the theory itself forces. Consulting the map is one more door, and the probability an agent walks through it collapses exactly when it matters most: pulling requires the agent to know it is lost and to know what to fetch — precisely the judgment drift has already corroded. The evidence is brutal. Handed a code dependency graph as a tool and left to pull from it, agents in one benchmark made zero calls to it in 58% of trials (CodeCompass, arXiv 2602.20048); the frontier model from my entrance anecdote had exactly that instrument available and went guessing anyway. Position is not a tool the agent may call. It is what the runtime serves with every step, before the agent acts. I have argued this delivery half separately as "push, not pull" — the two-variable theory is why it is not a style preference.
Put together, a guided step becomes a navigation loop older than
software: fix your position → take the one indicated heading → confirm
at the checkpoint → new fix. The agent supplies intelligence inside
each leg; the system supplies certainty about the map.
Who guards the guide
One instrument remains, and my case earned it the hard way: the ratchet
happened inside a guided runtime — every round procedurally legal, the
guide correctly serving position within a frame that was itself sick. A
guide cannot see its own frame drift from inside. But every drift
pathology I have logged, whatever its mechanism, shares one
machine-readable symptom: activity without progress. Sailors
distinguish speed through water from speed over ground — my reviewer had
enormous speed through water (nine brilliant rounds, thousands of new
test lines) and zero speed over ground for 27 hours. You cannot
pre-write a detector for a disease you have not met; you can pre-write
the fever thermometer, because the fever is generic even when the
pathogen is novel.
So the mechanism now being encoded from this case: when rounds
accumulate and progress toward the dispatched objective stays flat, the
runtime pushes the aggregate view into the loop — round count, product
movement, the frozen baseline — and demands a self-audit anchored
against it. Not "are you drifting?" (a drifted agent answers "every
finding is real," and it's true); instead, the panorama the agent
structurally cannot see from inside, with the burden of justifying
another round against the dispatch baseline. If the stall survives,
escalate to the human. Note what this automates: the glance that saved
my project. Noticing the flat line was the mechanical half of my
intervention — only the reframe was judgment.
The boundary that keeps this from becoming a cage
Some choices are mechanical: the right branch is knowable in advance, and
leaving it open is pure probability tax. Collapse those without mercy.
But some choices are the work: which frame fits a novel failure, what
is worth building, when a rule itself is sick. Collapse those and you
have built a workflow engine that confidently does the wrong thing. Spend
judgment only where judgment is irreplaceable — and the test is whether
you could have written the correct choice down before the agent arrived.
Old wisdom, new constraints
Manufacturing calls the entrance half poka-yoke — design the fixture so
the part only fits the right way. Robotics calls the position half SLAM —
nobody trusts odometry alone; everybody fuses in landmark fixes. Type
systems call it "make illegal states unrepresentable." We did not invent
the idea that reliability lives in the environment's topology rather than
the operator's skill.
Three things are new when the operator is an LLM agent. First, the
operator is a probabilistic reasoner you cannot configure — an agent's
choice among entrances is sampled from a training-shaped distribution;
the entrance anecdote above is this in miniature. Environment topology is
the only reliability lever you actually hold. Second, the single entrance
can be dynamic — a guided runtime recomputes the one legal next action per
step, per role, per state; rails that move with the work, which is what
separates this from the static workflow engines the 2000s buried. Third,
it is finally measurable: entrance counts per flow, position-loss
incidents per compaction, divergence between claimed and verified
position, preventable gate rejections. Drift stops being a vibe and
becomes a dashboard.
The conclusion, restated
The agent field has spent two years making dead reckoning excellent:
smarter loops, richer memory, disciplined context, hierarchical crews.
And long tasks keep drifting, because none of it was ever the missing
instrument. (Princeton's recent agent-reliability study measured the
decoupling directly: across 24 months of frontier releases, capability
climbed while reliability plateaued — for every vendor.) My drifted
reviewer ran the full modern stack and failed in a way invisible from
inside the loop, because every fix on that list shares one assumption:
that the agent's account of where it is can be trusted. It can't —
not because agents lie, but because self-estimated position compounds
error by construction, at any intelligence level.
Drift is compounding position uncertainty multiplied by per-step choice
sampling. Both variables live in your system's topology, not in the
model's weights, and neither yields to a better model or a better loop.
Don't give your agents a longer logbook or a faster loop. Give them a
map with a verified you-are-here dot — pushed into their hands at every
step, because a lost agent never asks for directions — and make sure the
map has only one door per room.
The runtime described here is being built in the open:
github.com/amingclawdev/aming-claw.
Every number in this article — the nine rejections, the 27-hour freeze,
the 818 green tests — comes from its governance timeline and git
history. The system is iterating from pull to push; a stable version
will follow. If you run agents on long tasks and want the two variables
on a dashboard instead of in a postmortem, watch the repo.
Top comments (0)