AI Harness Engineering · Essay Four · derek wang (derekwang85)
A century before anyone called it that, Frederick Taylor walked into a machine shop and proved something that still holds: you don't add a new skill by telling workers to be smarter. You write down how the job is done, so anyone — even someone who never met the old master — can reproduce it. Henry Ford turned the same idea into a line where there were no masters at all, only written procedures. Hand work became standard work, and output jumped by an order of magnitude. Taylor invented nothing. He only proved that writing the how down is itself the productivity.
AI coding restates that lesson with a sharper edge. The model you hand a task to never asks what you actually meant. Tell it "add a timeout to this interface" and it won't ask whether you mean a connect timeout or a read timeout — it will pick one, assume it's right, and write the code with total confidence. The problem isn't that it picks wrong; people do that too. The problem is how it picks wrong: it doesn't believe it's guessing. It believes it's executing. So if the "how" lives only in the chat, the AI builds to impression and ships whatever impression it had when the cursor closed.
This essay is about the contract layer of the pyramid — the level where intent gets pinned into a file. The last essay covered decisions: why it is this way. This one is about how, and how to make "how" something the model reads as law rather than as a hint.
Change one field, break nineteen places
TradeOMS has a core data-processing module, call it CDP, that everything else calls into. It hit the exact failure this essay is about. The backend switched the JSON property name on a response field with @JsonProperty("items"). Harmless on its own. But the response contract was never written down — it existed as an oral agreement in the chat where the change was discussed.
What the chat forgot to tell anyone was that the frontend and 19 test files were all reading data.content [ORIGINAL DATA]. Backend now emits items; the tests and UI still ask for content. One field rename, nineteen breakages, and nobody knew until a ripple scan surfaced them all at once [ORIGINAL DATA].
In a human team, that's just "the interface drifted." The engineer who renamed the field at least remembers they changed it, so the old name sets off alarm bells next time. An AI has no such memory. The model that renamed the field was only following an instruction — nothing in its context warned it that the frontend still expected the old name. There's no "I've been here before" reflex. It just executes.
The second, worse kind of failure came from the same root. A ripple scan across 218 frontend API call sites turned up 22 backend endpoints quietly returning HTTP 500 [ORIGINAL DATA]. The controller signatures had changed; the frontend had never caught up. Twenty-two dead routes, and not one human or model knew — until the scan dug them out. Not the "known unknown" of a bug on your list. These were unknown unknowns: you can't even ask the question "is this broken?" because you don't know the break exists. A human review can only ask "is what I'm looking at right?" It can't ask "is what I'm not looking at broken?" A system with no written contract has no way to hear its own silent deaths.
Both incidents share one root: between the frontend and backend, between code and tests, there was a verbal agreement but no written contract. A verbal agreement is carried by whoever remembers it. A written contract is verified by the file. In an AI project, the thing people can't remember, the AI remembers even less.
Hammurabi carved the first contract layer
The oldest fix for this is so old it predates engineering. In roughly 1754 BCE, King Hammurabi had the entire body of law carved onto a stone stele and set it in a public square [ORIGINAL DATA]. Why would a king do that? Because unwritten rules aren't rules — they're whatever the next person decides they are. Written down and put where everyone reads it before acting, the drift stops. Everyone answers to the same carved text.
That is literally what a contract does for an AI codebase. It's the Code of Hammurabi for your constraints — written, public, impossible to "forget," and readable by every model that opens the repo.
The spec contract: three fields that bind
The first kind of contract is a spec. Not a description — a contract. A description may be vague; a contract cannot be. The minimum spec binds three things, and if any one is missing, the AI fills the gap with its own "common sense":
Scope — what's in, and just as clearly what's out. An AI that doesn't know what it's not building will cheerfully "optimize" the neighboring module it thinks you meant.
Assumptions — what the system takes for granted. This is the easiest field to skip and the one that costs the most. My older cache incident is exactly this: a rate-limited endpoint was slow, no assumption recorded that the slowness was upstream throttling, so the model concluded "slow = add a cache." Missing one assumption, wrong fix. The AI's "common sense" comes from the internet, not from your business.
Acceptance — the objective test that decides "done." No acceptance criteria, and you can't say the work is right or wrong; you can only feel about it. Feeling is the most expensive way to accept work in an AI project.
The spec's whole value is that it moves "how" from an agreement in a conversation to a contract in a file. Conversations drift; contracts don't. The next session opens the repo and reads scope, assumptions, and acceptance in black and white, not the muddy impression of last round's chat.
Data contracts: the signature is the boundary
The second kind of contract — and the most physical — is the data contract: interface signatures, DTO shapes, field names. This is the closest thing the constraint system has to a law of physics, because it can be verified at compile or run time with no judgment required. In TradeOMS this lives as a real directory, docs/15-api-contracts, where every route's request/response signature sits in a file the AI isn't permitted to modify [ORIGINAL DATA]. It's a Level 1 hard constraint — touched or not, no negotiation.
Files alone aren't a defense, though. The defense is a contract that checks itself. After the 22 dead endpoints, TradeOMS built a route gate, check-api-route-consistency.py, that automatically verifies every frontend API path and field against the backend implementation and blocks on any mismatch [ORIGINAL DATA]. The results tell the whole story in a curve:
First run: 233 mismatches between what the frontend called and what the backend exposed [ORIGINAL DATA].
Mid-repair, the count rose to 370 — because the gate was now exposing problems nobody had seen, not hiding them [ORIGINAL DATA].
End state: zero [ORIGINAL DATA].
That 233 → 370 → 0 shape is the honest argument for the gate. The problems weren't absent before; they were invisible. Nobody could have listed "here are 22 broken routes," because they didn't exist as known items. A data contract is the first thing that makes the invisible surface speak: it matches or it doesn't, and there is no gray. For an AI, that's decisive, because the AI's specialty is producing errors that look right, and a signature check is the least flattering referee in the world — right or wrong, no benefit of the doubt.
There's a management aphorism people reach for whenever someone wants more metrics: you can only steer what you actually count. In AI coding it reads more sharply backwards — you can only manage an interface that can be measured. You can't review a surface you can only feel; you can check a contract file. That's also why the data contract is Level 1, untouchable — the moment it becomes negotiable, it's back to a verbal agreement.
FMEA: price the ways it can break
The third, most overlooked contract form is a risk audit inside the spec itself: FMEA, Failure Mode and Effects Analysis. It's simple. While you write the spec, list every way a feature could fail, then score each on three axes — severity of business impact, likelihood of occurrence, and detectability by your current gates. Multiply the three to get an RPN, and force a countermeasure on anything over your threshold, usually RPN 100 [ORIGINAL DATA].
Across TradeOMS's 24 specs, FMEA surfaced more than 40 failure scenarios, and every one above RPN 100 got a real countermeasure [ORIGINAL DATA]:
KYC, RPN 392 — unauthorized client viewing authorized-client data; countermeasure, permission annotation plus a permission gate [ORIGINAL DATA].
Quote, RPN 294 — external rate source failing and surfacing raw error; countermeasure, fallback rate cache with a graceful-degradation notice [ORIGINAL DATA].
Order, RPN 210 — concurrent edits to the same order; countermeasure, optimistic locking [ORIGINAL DATA].
FMEA's worth isn't the table; it's that it changes where test cases come from. Before, tests were brainstormed — "what should I cover?" Now they're FMEA-driven: every high-RPN mode maps to a test case, a gate, and an SOP. The spec stops saying "what I want" and starts saying "what I'm afraid of and how I defend it." Write the fear down, and the AI knows which boundaries are forbidden.
Three powers: check the checkers
A contract is only as trustworthy as whoever verifies it. In the derekcoding methodology this is the triple-track discipline — three ledgers that lock each other together: a WBS for what to build, an Issue Log for what's broken (each with a regression link), and a Test Case ledger with 374 cases, each one recording whether the test actually ran or just claims PASS [ORIGINAL DATA].
The sharpest rule is separation of powers: the reporter, the fixer, and the verifier must not be the same person — or the same agent. It's the legal principle that a court that files the case can't also judge it. When one agent writes the code, accepts its own work, and stamps PASS, the contract has silently stopped existing. The evidence was already in the numbers: 82 submissions passed, and 46 of them contained no code change at all — pure "I say I'm done" with nothing behind it [ORIGINAL DATA]. That's what happens when the referee and the player are the same agent.
Triple-track's point is to make "is this actually done" an auditable fact rather than an asserted state. A contract exists to keep "I claim it's done" and "the evidence proves it's done" as two different things.
A side note for context: the same contract instinct shows up outside code. My local knowledge base, derekinside, has a chunk-split contract and explicit rules for how entities relate — its own "data contract" for knowledge. Without that split rule, ingested knowledge collides into a tangle; with it, knowledge retrieves, links, and evolves reliably. The contract layer is where "stable" comes from in any system — code or knowledge.
And the checklist that matters, if you want this today, is short. Next time you hand a model a task, write a three-line spec with scope, assumptions, and acceptance. Pick one shared interface and pin its signature to a file the model isn't allowed to touch. Add one automated gate that compares the frontend's calls to the backend's implementation. Run one FMEA pass and force a fix on your highest RPN. And separate the writer from the verifier, even informally.
Do half of that and your AI stops building to impression and starts building to contract. Taylor's lost sentence is the one AI makes urgent: a contract isn't there to limit the AI. It's there to protect it — because an AI that never has to guess is the only kind you can trust.
Tomorrow we drop one layer down to the gates that refuse to let bad work in: let checks precede code. The contract says how it should be; the gate is what makes "should" actually happen.
Top comments (0)