I ship a data product on my own, and AI sessions do most of the typing. In Context, harness, the loop — a map of the architecture I run my project on with AI
I split it into three parts: the context (what the model always sees), the harness (what it
can reach and what stops it), and the loop the work runs in. This is the harness, in four points.
1. First, what it can reach
Before you can stop a model from doing something, it has to be able to do it. So it's worth being
concrete about what mine can actually do. A session I open can run shell commands, read and write any file in the project, browse the web, and start sub-sessions that work on separate things at the same time.
One file adds to that: .mcp.json, in the project root. It connects the model to my production
database through a small server, so a session can query the database itself instead of me running
the query and pasting the result back into the chat.
That's a lot for something that types on its own, which is why everything below is about shaping it.
2. The three things that shape it
Procedures are files — .claude/commands/, one markdown file each. No code inside, just plain
instructions: read these documents first · do these steps in this order · run these checks · stop
here and ask me. I type the file's name and the model executes what's inside it: /punto (where
does the project stand), /esco (plan a run that happens while I'm out), /redteam (attack this
design cold) and others.
They're versioned, so I can see how one changed and why. But the real point is this: when a
procedure fails, I fix the file — not "I'll remember better next time." That is the whole
difference between a workflow and a habit.
Workers are the same model with a different tool set — .claude/agents/, one file each. Mine
are misuratore (checks a claim against the real data), verifica-a-freddo (attacks a finished
artifact), corsia-zero (asks whether the thing is even the right thing) and costruttore-isolato
(writes code, in its own copy of the repo). What separates them is what they're allowed to read.
Gates are programs that inspect and refuse. The useful ones run before the model acts, not
after. After the fact, all you get is a report telling you something already went wrong. Before it,
you get a refusal: the model doesn't get to decide whether it was allowed — it finds out it wasn't.
3. The principle everything else follows from
The worker whose job is to measure something against real data is not allowed to read my design
documents. Not asked nicely in a prompt — a program refuses the read. If it never sees what I expect
the answer to be, it can't drift towards agreeing with me. The same goes for the reviewer whose job is to attack a piece of work: it gets the work and the code, and no explanation from me, because an explanation is the fastest way to make a reviewer agree with you.
An instruction is a wish. A program that refuses is a fact.
4. The ladder
Every rule you have sits on one of these four rungs. The first question is just: which one? That
turns "I have a rule about that", which nobody can check, into "that one's a ②", which anyone can.
The second question is the one that costs: does the check actually refuse?
There's a well-known way to answer it, and it isn't mine: mutation testing, an idea from the
1970s. Break your own check on purpose and see whether anything notices. Say the rule is never
commit an API key, and the check is a script that scans a commit for something key-shaped: change what it searches for to a word that can never appear, then run your tests. One of them has to go red. If they all stay green, that test was never watching your check — it was only confirming that a commit went through, which would have happened anyway. Put the check back, confirm green, done.
Decades old, and it works.
The interesting part is where you can point it, because in an AI setup the answer is: not at most
of your rules. Most of mine are sentences in a markdown file — stop and hand back when a sub-task ends, go read the relevant document before doing anything expensive. There is nothing in a
sentence to break. No threshold to move, no pattern to blank out. You can watch a model follow it
and feel reassured, but you can't get a refusal out of prose, and a rule you can only observe is a
rule you're trusting.
That's what the ladder is really for. Not ranking rules from bad to good, but telling you which
of yours can be proven at all — and how much of your setup is resting on the ones that can't.
Of my checks that were provable, three had no real test behind them, and they were exactly the
three that run when I'm not there: inside a sub-session, or overnight. That isn't bad luck. You test
what fails in front of you.
The three lines, if you take nothing else:
- Put the checks that matter in front of the action, not behind it.
- Separate your workers by what they're allowed to read.
- For every rule, ask what would actually stop it — and count how many of your answers are just a sentence in a file.
Top comments (0)