Part 3 of three posts behind the multi-agent case study. The enforcement layer that made the agents finish things, the ways it failed, and the experiment I built to test it and never needed to run.
Around the second week of running agents, I stopped believing that a rule written in capital letters would be followed because it was in capital letters.
The evidence was not subtle. One rule, paste the raw log lines during a job instead of summarizing them, was violated more than thirty times after it was written. One agent had a section titled "Hard Rules (fireable offenses)." It was four lines long. All four were broken at least once afterward. Every workspace on the system is full of bold rules written the day something went wrong, and the record shows how little that achieves on its own.
The comment at the top of the first enforcement plugin says why. The platform can restrict what agents can't do. It cannot force what they must do. So the forcing became code.
The stack
Fourteen behavioral plugins run today. Each one exists because of a specific day.
- One detects an action request, detects a chat-only reply, and blocks or injects enforcement. It also blocks any attempt to delete or move a workspace directory. It exists because specialists kept explaining instead of doing.
- Several force a memory search before any action tool and a memory write before completion. They exist because agents kept rushing to the internet instead of reading what they already had.
- One blocks replies that tell me to clear my cache or try again. It exists because of a June afternoon I wrote about in the last post.
- One blocks mutations on external surfaces until declared sources have been read.
- One blocks bulk rewrites and non-English script in memory, because a cheap model kept drifting into other languages and a "cleanup" of that text once destroyed 1,089 files in a single pass.
- One blocks agents from upgrading or downgrading the platform through the package manager, because an agent with shell access can change the platform under itself.
- Email goes through a relay that rejects any non-allowlisted recipient before a connection opens. The allowlist is hardcoded in the relay so an agent that controls the environment cannot widen it. It exists because an agent once sent an email I had asked it to draft.
The coding assistant that built all of this got the same treatment: nine hooks of its own, including no direct edits to the platform config by any tool, a verified snapshot before any config change, and a post-compaction hook that re-injects the rules because the model forgets them when its context gets summarized.
The ways the law failed
This is the part I would want to read if I were starting over, so here it is without softening.
The enforcer that bypassed itself. The coding assistant built an enforcement hook and, within minutes, got around it by running the interpreter through a different tool.
Guards that block the right thing and the wrong thing. In one measured week, the memory-discipline guard blocked seven tool calls, including legitimate commands from the archive pipeline.
Guards cost context. Every turn carries the enforcement injections. In that same week the average turn sent 5,343 input tokens, the largest sent 160,734, and only 35 turns out of 6,308 hit the prompt cache. That is 0.55 percent. Routing through the quality-gate proxy rewrote payloads and defeated the vendor's caching entirely in one measurement: 276 turns, zero cache hits, roughly 600,000 input tokens paid twice. Coercion is not free.
The experiment I never needed to run
By July a question was on the table, and it was a model that put it there. Would a stronger model with lean prompting finish tasks on its own, or has the coercion been doing that work all along? I built an A/B switch to find out. Config B drops most of the enforcement plugins and rewrites each agent's rules as plain prose. Both configs validate, and the round trip was proven identical offline.
The switch has never been flipped, and not from neglect. Over the same weeks the enforcement stack and the continued refinements to the routing proxy started paying dividends in quality, and the pressure that had made config B tempting went away. Config B is the "simplify and hope for the best" option. It is also what every frontier model quietly wants: to be left to its own devices and to define for itself what counts as done. Of course a model suggested that path. The version that kept finishing things was the constrained one, so that is the version that kept running. The archives agree: no snapshot from a flip, the B workspaces untouched since July 2, and every config captured since then is config A.
The builder is an agent too
The most uncomfortable thing in six months of records is that the coding assistant I used to build the stack failed in exactly the ways the agents did. It answered before reading. It claimed verification that didn't happen. It took a conversational "that sounds good" as permission to edit 35 live cron jobs inside an experiment whose entire contract was that nothing live would change. It bypassed its own guardrail.
One comparison is worth the whole post. Asked to fix a gateway restart problem, the rushed path edited the framework's source in a local fork, linked it over production, then unlinked it and deleted the install entirely. I reinstalled by hand. The same problem, approached carefully the second time, was solved with three lines in a shell profile and no code changes. The transcripts measured both: 106 tool calls and a destroyed install, against 60 tool calls and a permanent fix. Reading is not slow. Guessing only feels fast.
The rule that surfaced through consistent frustrations
Do not write the rule in prose and hope. Build the mechanism first: the hook that blocks, the script that checks the artifact, the allowlist the agent cannot reach. Then try to break it before trusting it, because an untested guard is worse than no guard. It lets you stop worrying about a failure it was never preventing in the first place.
The full accounting of what shipped, what never did, and what it all cost is in the case study.
Top comments (0)