What Advisory Rules Actually Do in an Agent Loop

Advisory rules in my 19-agent system produced 0/3 compliance in the three runs right after the rule was canonized — the hook warned, the agent proceeded anyway. I replaced the warning with a deny-by-default gate plus a one-shot, logged override: over two months, 9 blocked tool-calls, 4 authorized overrides, 0 unauthorized writes. Small, well-delimited evidence, reproducible by script — but the direction is clear: enforcement has to live in the substrate, not in prose the model weighs against its objective.

The hook fired. My root agent read the warning telling it that plan.md belongs to the planner subagent, acknowledged it, and wrote the file anyway. Then it did it again on the very next step. The audit report for that run records it drily:

"plan.md root direct ×2 (step 1 + 2), planner NO invocado. hook PreToolUse advisory flagged 2x consecutivos; contenido aprobado verbatim inline → root procede."

Root authored the planning file directly, twice; the advisory hook flagged it twice in a row; root proceeded anyway, logging "aprobado verbatim inline" — approved verbatim inline — as its justification. Two runs later, a third instance with exactly the same shape: "plan.md root direct 3ª instancia consecutiva. Hook PreToolUse advisory disparó + root ack con razón 'aprobado verbatim inline'."

That third one is what stopped me. I was reading the audit reports as they came in, expecting the new rule to have changed something — instead I was watching my own system politely log the same violation, run after run.

Those were the three runs immediately after I canonized the rule. The hook fired every single time. Compliance: 0/3.

The system

Some context for that scene. I run a personal multi-agent system on top of Claude Code: 19 specialized agents — planners, implementers, reviewers, domain specialists, independent auditors — under a root-driven, single-writer architecture. One main thread is the only physical invoker of subagents, and coordination happens through on-disk artifacts rather than shared mutable state: plans, hypothesis files, reports. The system has accumulated 171 documented runs, each with its own directory, plan, and audit trail. That number is scale and nothing more — raw transcripts from the early era are gone to retention, so I cannot measure rule compliance across all of it, and I won't pretend otherwise. The piece of the division of labor that matters for this story: the planner subagent owns plan.md. Root coordinates, but it is not supposed to author the plan itself. A sanitized snapshot of the whole thing is public at github.com/lexosi/multiagent-system-lex.

Why advisory rules fail

Here's the mistake I made first, and I don't think it's an exotic one: I wrote the rule down and assumed that was enforcement. The advisory hook was technically real — a PreToolUse hook, introduced 2026-05-21, that fired on every root write into planner territory. But it always exited 0. It warned; it never blocked. Five days later, on 2026-05-26, I canonized the rule in the system's constitution: root must not author plan.md; the planner owns it. The three runs I opened with are the three runs immediately after that canonization.

A warning is just more context. The model isn't being malicious when it walks past one; it's optimizing for the task in front of it. When a soft rule competes with the immediate objective — produce the plan, keep the run moving, the content is already approved anyway — the rule loses. In my data, it lost every time. The failure mode isn't disobedience; it's incentives. The path of least resistance runs straight through the warning, and "approved verbatim inline" isn't a rebellion — it's a locally reasonable justification generated by a system that was never actually stopped. Anthropomorphizing this as the agent "ignoring the rules" misses the mechanism and, worse, suggests the wrong fix: a sterner warning is still a warning.

Once I saw the pattern for what it was, I stopped tweaking the wording and went looking for where enforcement could actually live — somewhere the model couldn't weigh it against the task. The answer was one layer down.

The cutover

Six days after canonizing the rule, the warning was gone. On 2026-06-01 (commit b3efe00) the advisory hook was deleted and replaced by a deny-by-default PreToolUse hook: any write into a guarded territory returns permissionDecision=deny unless the caller is the owning subagent — the hook discriminates callers by agent_type — or a valid one-shot sentinel file authorizes exactly one override, with a short TTL. The previous design's session-level environment-variable escape hatch died in the same commit: an override that lives for a whole session is an override that outlives its justification.

The numbers, verbatim from the eval report — which is regenerated by script, not written by hand:

• Advisory compliance after the rule was canonized (2026-05-26, plan.md territory): 0/3 runs.
• Hard-block era (2026-06-02..2026-08-04): 9 blocked tool-calls (attempted root writes to planner-owned files), across 7 runs.
• 4 writes authorized via the one-shot sentinel — each logged with a non-empty reason.
• Unauthorized writes to guarded territories: 0.

I want to label this evidence carefully, because the labeling is half the point. The advisory side is a natural experiment, n=3: three consecutive runs immediately after a single dated rule change, one territory, with frozen citations into on-disk audit reports that the eval script re-verifies. The hard-block side is a mechanical log: an append-only deny-ledger covering two months, parsed by script, no human or agent judgment in the count. And the honest limit: the hook logs denials and overrides, not allows — so "0 unauthorized writes" is an invariant by construction (the hook denies guarded territories unless the caller is the owner or consumes a valid sentinel), corroborated by the logs, not an exhaustive census of every write that ever happened. This is not a large-scale study and I'm not dressing it up as one. It is a small, well-delimited experiment that anyone can re-run against the repository with python eval/run_eval.py.

Deny, deny, override — as designed

The pattern I care most about in the ledger is deny → deny → override. Root tries to write a guarded file: denied. Tries again: denied. Then one of two things happens. Either root routes the work to the owning subagent — which is the intended outcome, and the common one — or I look at the situation, decide the exception is legitimate, and drop a sentinel file that authorizes exactly one write, which the hook consumes and logs. Four of those in two months. All deliberate, all recorded, all with a stated reason.

That flow is the design, not a failure of it. A hard block with no exit teaches the system — and its operator — to fight the wall, and fighting the wall is where the really creative workarounds come from. What I actually want is a gate: violations impossible by default, exceptions possible, authorized, and logged. The one-shot property is what keeps it honest. A sentinel authorizes one write, not a mood.

Where this fits

The reason I think this matters beyond my setup: everyone is building loops right now. Boris Cherny, head of Claude Code at Anthropic, said it on stage in June 2026: "I don't prompt Claude anymore. […] My job is to write loops" (Acquired Unplugged) — outer programs that invoke the agent loop as a subroutine and decide what happens next. And the recent evidence says those loops need harder gates than most of us are giving them. SlopCodeBench (UW-Madison/MIT, March 2026) had agents extend their own code under changing specs and found structural quality eroding iteration after iteration even while core tests kept passing — best strict rate 17.2%, agent code 2.2× more verbose than comparable human repositories. LoopsBench (Microsoft, August 2026) found the best outer-loop configuration solved 25.00% of its tasks, with visible regression events across every loop profile it measured — though it's days old and unreplicated, so hold that one loosely.

The shared reading of both: objective verifiers like tests are necessary but insufficient — what passes the gate can still rot. My result is a small datapoint in the same direction, one layer down. Even the process rules — who is allowed to write what — don't hold as good-faith conventions inside a loop. If a boundary matters, it has to live in the substrate the agent can't reason its way around: the tool-call layer, before the write happens, not prose the model weighs against its objective and discounts.

Touch it

I extracted the enforcement layer into a standalone package, loopward: a reliability layer for multi-agent systems — anti-loop limits, human stop-gates, per-run audit trails — enforced structurally rather than by convention. pip install git+https://github.com/lexosi/loopward.git, then loopward-demo runs a deterministic offline demo with zero API keys; this isn't an announcement, it's the touchable version of this post.

Enforcement by construction, not good faith

That's the one-line version of two months of logs. Rules the agent can weigh get weighed away; gates the agent cannot pass produce routing instead of violations, and the exceptions become visible, deliberate acts instead of silent drift.

What I'd do differently, and what's still open — because a write-up that ends in triumph is usually hiding the interesting part. I'd log allow decisions from day one, so "0 unauthorized" could be a census instead of an invariant plus corroboration. I'd keep raw transcripts — retention ate the advisory era, and n=3 is what survived, not what I designed. I wouldn't canonize a rule five days before I could enforce it. And open questions remain: n=3 is small even if it's clean; plan.md is one territory, and the same claim for the others is only partially measured; and I don't yet know how the gate scales as guarded territories multiply, because every new wall adds override friction and friction is a budget you spend from. What I do know is that the direction has stopped feeling arguable: if I need an agent to respect a boundary, I don't ask. I make the boundary the default, and the exception a logged, deliberate act.