What is an SOP?
SOP stands for Standard Operating Procedure. You split a process into a number of steps, and for each step you say what to do and how to verify it. The result is a set of concrete steps that can be repeated, checked and signed off.
In my boss's words, it's a bowl of ramen: how many noodles, how much water, how much salt, how much scallion. I'm pretty sure he meant Japanese ramen. When I cook it, it's more like: some noodles, a little oil, a reasonable amount of water, scallions at will.
What is a gate?
This one hardly needs explaining. When I was a kid, my family said be home before six. Of course I could choose to come home after eleven, but not coming home wasn't an option. The curfew was just there. It triggers when you walk in the door: men's singles, women's singles, or mixed doubles.
End of the main text.
My last article covered the three-way (Three-Body-style) adversarial setup for multiple agents, plus some code-style rules. But does three parties fighting it out solve the problem? Honestly, no.
Agents write the docs, run the experiments and give the reports. Without limits, an agent will freely write all sorts of things, just like my ramen: somewhere between delicious and awful, at random.
While working, I ran into a few fairly serious problems:
- Even with prompt and skill constraints, agents often do things they were never authorized to do, or lose track because they forgot the context. How well it went, whether it's actually finished, all "more or less", just like my ramen. So quality depends entirely on how focused and responsible I am that day. My focus and sense of responsibility fluctuate, so of course the results differ between my highs and my lows.
- Agents like to overthink. They fall into endless thinking, trying to find the perfect answer, especially on open-ended questions with no clear goal, where they deliberate at length all the time. It's like Go: I may be the weaker player, but I can always use the wear-down-the-old-man tactic. Unfortunately, next to a silicon life form, I'm the old man. A forty-minute think I can sit through. Three forty-minute thinks in a row, I can't.
- Thanks to the breakneck progress of AI, a subagent can now send out minions of its own. The token explosion aside, the minions of the minions of the minions are a blind spot for me. I can go and read their thinking, but they're beyond what I can manage. My energy is limited, so I end up with minions of minions of minions who are not my minions.
- The main agent does the scheduling, and I'd rather not hand it the writing of code or docs. The sub-agents that fight each other haven't settled on a conclusion until they're done, so they certainly shouldn't write into the main directory. That means designing more subagents for the main agent to call: some write code, some review it. And the ones who write code and the ones who review it can't sit together.
So there's nothing for it. To keep the project going, let's change a few things.
1. SOP and separation of duties (for problem 4)
Turn agent orchestration into a full pipeline: retrieval -> investigation -> implementation -> self-check -> knowledge rot -> gate.
Each stage has one subagent definition under .claude/agents/. Every description says "use only when the main agent dispatches you by name... never auto-dispatch", and what material the role needs goes in required-inputs. Here they are, excerpted (the originals are written in Chinese; I've translated the descriptions):
Retrieval: the retrieval agent may go online, carry material back and put it in the designated directory.
---
name: prior-art
description: Researcher. Looks up other filesystems' source and docs on demand, hands back only facts with citations, no arguments. Use only when the main agent dispatches you by name with the question to look up; never auto-dispatch.
tools: Read, Bash, WebFetch, WebSearch
model: sonnet
effort: high
omitClaudeMd: true
required-inputs: draft directory, report
---
Investigation: the three-way adversarial setup kicks in to find a suitable approach.
---
name: experiment-designer
description: Experiment designer. Takes an experiment number and writes the pre-run registration (arms, controls, criteria, failure clauses) before any code or artifact exists. Use only when the main agent dispatches you by name with the question to answer and the clause under test; never auto-dispatch.
tools: Read, Bash
model: opus
effort: high
omitClaudeMd: true
required-inputs: draft directory, clause under test, fork list|question list|rerun
---
---
name: three-way-attack
description: The cloud attacker leg of the three-way argument. Use only when the main agent dispatches you by name with this round's background material, attack surface and the verdicts of earlier rounds; never auto-dispatch.
tools: Read, Bash
model: opus
effort: high
omitClaudeMd: true
required-inputs: no-read list, draft directory, background material, attack surface
---
Implementation: start writing code.
---
name: implementation-writer
description: Implementer. Changes crates/ for one milestone step, one parallel line, or a cluster the main agent has bundled (may close several items), with tests, and proves the tests go red. Use only when the main agent dispatches you by name with the step number and the clauses in play; never auto-dispatch.
tools: Read, Edit, Write, Bash
model: opus
effort: high
omitClaudeMd: true
required-inputs: draft directory, report, clause, crates files to touch
---
---
name: tooling-writer
description: Tooling implementer. Changes gate stages, hooks, research scripts and watchdogs, or changes agent definitions, shared constraints and project rules according to a verdict. Every change first builds an input that should go red. Use only when the main agent dispatches you by name with which item is being closed and the exit; never auto-dispatch.
tools: Read, Edit, Write, Bash
model: sonnet
effort: high
omitClaudeMd: true
required-inputs: draft directory, report, exit, files to change
---
Self-check: a three-way siege to review code quality.
---
name: investigator
description: Investigator. When a test or gate result differs from expectation, reproduces the symptom, bisects down to file and line, and hands back a minimal repro plus the condition that would refute it. If the dispatch says "fix it once found" and the clause spells out the fix, carries on, fixes it and proves it red; if the clause doesn't, stops and hands back to the main agent. Use only when the main agent dispatches you by name with the verbatim symptom and the question to answer; never auto-dispatch.
tools: Read, Edit, Bash
model: opus
effort: high
omitClaudeMd: true
required-inputs: symptom, draft directory, report
---
Knowledge rot: audit the whole project to check that every change is consistent.
---
name: sweep
description: Sweeper. After a number is retracted, a format constant changes, a new criterion is established, or a stage of work ends, searches the whole repo for places that still cite the old value, should now be governed by the new criterion, or have been made false by this stage, and hands back a classified list. Use only when the main agent dispatches you by name with the old value and which quantity it is (or the stage's scope of change and what it achieved); never auto-dispatch.
tools: Read, Bash
model: sonnet
effort: high
omitClaudeMd: true
required-inputs: draft directory, report
---
Gate: at the end of a task, mechanically check the error ledger, then the main agent analyzes and fixes the problems.
Every node has its own specialist for its own job. The main agent only breaks down and dispatches tasks, and every subagent minds its own business. On top of that, there has to be a mechanism for "knowledge rot" that regularly clears out expired context, so the pipeline never runs overloaded.
2. Limiting abilities (for problems 2 and 3)
---
tools: Read, Bash
model: sonnet
effort: high
omitClaudeMd: true
---
Eliminate fuzziness and the gray zone. The main agent has to be clear what my task is, and then dispatch work to the subagents according to the task.
Depth=1. A subagent may not have minions. That roots out the "blind spot" problem. (This corresponds to the environment variable CLAUDE_CODE_MAX_SUBAGENT_SPAWN_DEPTH=1, and none of the roles has Agent in its tools anyway.)
omitClaudeMd=true. It mustn't know too much. Context it doesn't need to know is withheld without exception, so it doesn't start free-associating.
tools: tools are narrowed, strictly. Each role above gets a subset of Read, Edit, Write, Bash, WebFetch, WebSearch, and only the researcher can go online. Everyone has Bash, so the lock doesn't sit on tools; it sits on hooks: dangerous patterns are rejected before they execute, with no reliance on the agent's good manners.
effort: high. Don't use more power than needed to solve a simple problem.
3. The gate (Harimoto siblings, go!) (for problems 1 and 2)
Red-green dual proof: every piece of code must have a test. And a test that merely runs "green" (passes) isn't enough: you have to prove to me that the test can catch an error (red), and the red and green evidence must be submitted together. The red evidence doesn't rest on the agent's word either: the gate breaks the code under test in one spot and confirms that the test really goes red.
Strong provenance: implementation must be tied to decision and experiment records. "Ghost code" with no historical decision behind it is not accepted, and the gate checks hard whether the comments explain where the code came from.
Exhaustive probe points: plant lots of probes in the code, check the data state every time execution passes, weave a nerve net. For example, even if the code has run through two hundred thousand different points in a simulated environment, you still have to show that the data flow is complete and free of violations. Checks that aren't wired in yet are honestly marked unimplemented, and they're not allowed to pretend they passed.
Text and format gate: check the knowledge base and docs item by item. Is everything written strictly in the preset JSON/Markdown format? Any ad-libbing? Does the record pass? Even a date has to be written as a standard date.
Scheduled patrol (timeout mechanism): the main agent attaches a scheduled script that watches the subagents it has sent out at all times. If it sees the context grow too large or the runtime run over, it immediately checks task progress, and if things have drifted from expectation, pulls the agent back.
4. Dirty-write protection (for problem 4)
Most mature agent frameworks are already aware of this one. For example, they use a worktree and the workspace under the root for physical isolation, so I don't have to build it from scratch.
But I've added one extra lock: unless it has gone through the dedicated "implementation writer" agent node, no code is ever allowed to be written into crates.
The one who writes code and the one who reviews it must be physically isolated. Without a double signature, you don't get past the main repo's threshold.
With that, it should be able to run.
Tokenman, move out!
Top comments (0)