DEV Community: DerekWang

An AI builds to the contract it can read, not the one you meant

DerekWang — Mon, 21 Sep 2026 04:12:00 +0000

AI Harness Engineering · Essay Four · derek wang (derekwang85)

A century before anyone called it that, Frederick Taylor walked into a machine shop and proved something that still holds: you don't add a new skill by telling workers to be smarter. You write down how the job is done, so anyone — even someone who never met the old master — can reproduce it. Henry Ford turned the same idea into a line where there were no masters at all, only written procedures. Hand work became standard work, and output jumped by an order of magnitude. Taylor invented nothing. He only proved that writing the how down is itself the productivity.

AI coding restates that lesson with a sharper edge. The model you hand a task to never asks what you actually meant. Tell it "add a timeout to this interface" and it won't ask whether you mean a connect timeout or a read timeout — it will pick one, assume it's right, and write the code with total confidence. The problem isn't that it picks wrong; people do that too. The problem is how it picks wrong: it doesn't believe it's guessing. It believes it's executing. So if the "how" lives only in the chat, the AI builds to impression and ships whatever impression it had when the cursor closed.

This essay is about the contract layer of the pyramid — the level where intent gets pinned into a file. The last essay covered decisions: why it is this way. This one is about how, and how to make "how" something the model reads as law rather than as a hint.

Change one field, break nineteen places

TradeOMS has a core data-processing module, call it CDP, that everything else calls into. It hit the exact failure this essay is about. The backend switched the JSON property name on a response field with @JsonProperty("items"). Harmless on its own. But the response contract was never written down — it existed as an oral agreement in the chat where the change was discussed.

What the chat forgot to tell anyone was that the frontend and 19 test files were all reading data.content [ORIGINAL DATA]. Backend now emits items; the tests and UI still ask for content. One field rename, nineteen breakages, and nobody knew until a ripple scan surfaced them all at once [ORIGINAL DATA].

In a human team, that's just "the interface drifted." The engineer who renamed the field at least remembers they changed it, so the old name sets off alarm bells next time. An AI has no such memory. The model that renamed the field was only following an instruction — nothing in its context warned it that the frontend still expected the old name. There's no "I've been here before" reflex. It just executes.

The second, worse kind of failure came from the same root. A ripple scan across 218 frontend API call sites turned up 22 backend endpoints quietly returning HTTP 500 [ORIGINAL DATA]. The controller signatures had changed; the frontend had never caught up. Twenty-two dead routes, and not one human or model knew — until the scan dug them out. Not the "known unknown" of a bug on your list. These were unknown unknowns: you can't even ask the question "is this broken?" because you don't know the break exists. A human review can only ask "is what I'm looking at right?" It can't ask "is what I'm not looking at broken?" A system with no written contract has no way to hear its own silent deaths.

Both incidents share one root: between the frontend and backend, between code and tests, there was a verbal agreement but no written contract. A verbal agreement is carried by whoever remembers it. A written contract is verified by the file. In an AI project, the thing people can't remember, the AI remembers even less.

Hammurabi carved the first contract layer

The oldest fix for this is so old it predates engineering. In roughly 1754 BCE, King Hammurabi had the entire body of law carved onto a stone stele and set it in a public square [ORIGINAL DATA]. Why would a king do that? Because unwritten rules aren't rules — they're whatever the next person decides they are. Written down and put where everyone reads it before acting, the drift stops. Everyone answers to the same carved text.

That is literally what a contract does for an AI codebase. It's the Code of Hammurabi for your constraints — written, public, impossible to "forget," and readable by every model that opens the repo.

The spec contract: three fields that bind

The first kind of contract is a spec. Not a description — a contract. A description may be vague; a contract cannot be. The minimum spec binds three things, and if any one is missing, the AI fills the gap with its own "common sense":

Scope — what's in, and just as clearly what's out. An AI that doesn't know what it's not building will cheerfully "optimize" the neighboring module it thinks you meant.
Assumptions — what the system takes for granted. This is the easiest field to skip and the one that costs the most. My older cache incident is exactly this: a rate-limited endpoint was slow, no assumption recorded that the slowness was upstream throttling, so the model concluded "slow = add a cache." Missing one assumption, wrong fix. The AI's "common sense" comes from the internet, not from your business.
Acceptance — the objective test that decides "done." No acceptance criteria, and you can't say the work is right or wrong; you can only feel about it. Feeling is the most expensive way to accept work in an AI project.

The spec's whole value is that it moves "how" from an agreement in a conversation to a contract in a file. Conversations drift; contracts don't. The next session opens the repo and reads scope, assumptions, and acceptance in black and white, not the muddy impression of last round's chat.

Data contracts: the signature is the boundary

The second kind of contract — and the most physical — is the data contract: interface signatures, DTO shapes, field names. This is the closest thing the constraint system has to a law of physics, because it can be verified at compile or run time with no judgment required. In TradeOMS this lives as a real directory, docs/15-api-contracts, where every route's request/response signature sits in a file the AI isn't permitted to modify [ORIGINAL DATA]. It's a Level 1 hard constraint — touched or not, no negotiation.

Files alone aren't a defense, though. The defense is a contract that checks itself. After the 22 dead endpoints, TradeOMS built a route gate, check-api-route-consistency.py, that automatically verifies every frontend API path and field against the backend implementation and blocks on any mismatch [ORIGINAL DATA]. The results tell the whole story in a curve:

First run: 233 mismatches between what the frontend called and what the backend exposed [ORIGINAL DATA].
Mid-repair, the count rose to 370 — because the gate was now exposing problems nobody had seen, not hiding them [ORIGINAL DATA].
End state: zero [ORIGINAL DATA].

That 233 → 370 → 0 shape is the honest argument for the gate. The problems weren't absent before; they were invisible. Nobody could have listed "here are 22 broken routes," because they didn't exist as known items. A data contract is the first thing that makes the invisible surface speak: it matches or it doesn't, and there is no gray. For an AI, that's decisive, because the AI's specialty is producing errors that look right, and a signature check is the least flattering referee in the world — right or wrong, no benefit of the doubt.

There's a management aphorism people reach for whenever someone wants more metrics: you can only steer what you actually count. In AI coding it reads more sharply backwards — you can only manage an interface that can be measured. You can't review a surface you can only feel; you can check a contract file. That's also why the data contract is Level 1, untouchable — the moment it becomes negotiable, it's back to a verbal agreement.

FMEA: price the ways it can break

The third, most overlooked contract form is a risk audit inside the spec itself: FMEA, Failure Mode and Effects Analysis. It's simple. While you write the spec, list every way a feature could fail, then score each on three axes — severity of business impact, likelihood of occurrence, and detectability by your current gates. Multiply the three to get an RPN, and force a countermeasure on anything over your threshold, usually RPN 100 [ORIGINAL DATA].

Across TradeOMS's 24 specs, FMEA surfaced more than 40 failure scenarios, and every one above RPN 100 got a real countermeasure [ORIGINAL DATA]:

KYC, RPN 392 — unauthorized client viewing authorized-client data; countermeasure, permission annotation plus a permission gate [ORIGINAL DATA].
Quote, RPN 294 — external rate source failing and surfacing raw error; countermeasure, fallback rate cache with a graceful-degradation notice [ORIGINAL DATA].
Order, RPN 210 — concurrent edits to the same order; countermeasure, optimistic locking [ORIGINAL DATA].

FMEA's worth isn't the table; it's that it changes where test cases come from. Before, tests were brainstormed — "what should I cover?" Now they're FMEA-driven: every high-RPN mode maps to a test case, a gate, and an SOP. The spec stops saying "what I want" and starts saying "what I'm afraid of and how I defend it." Write the fear down, and the AI knows which boundaries are forbidden.

Three powers: check the checkers

A contract is only as trustworthy as whoever verifies it. In the derekcoding methodology this is the triple-track discipline — three ledgers that lock each other together: a WBS for what to build, an Issue Log for what's broken (each with a regression link), and a Test Case ledger with 374 cases, each one recording whether the test actually ran or just claims PASS [ORIGINAL DATA].

The sharpest rule is separation of powers: the reporter, the fixer, and the verifier must not be the same person — or the same agent. It's the legal principle that a court that files the case can't also judge it. When one agent writes the code, accepts its own work, and stamps PASS, the contract has silently stopped existing. The evidence was already in the numbers: 82 submissions passed, and 46 of them contained no code change at all — pure "I say I'm done" with nothing behind it [ORIGINAL DATA]. That's what happens when the referee and the player are the same agent.

Triple-track's point is to make "is this actually done" an auditable fact rather than an asserted state. A contract exists to keep "I claim it's done" and "the evidence proves it's done" as two different things.

A side note for context: the same contract instinct shows up outside code. My local knowledge base, derekinside, has a chunk-split contract and explicit rules for how entities relate — its own "data contract" for knowledge. Without that split rule, ingested knowledge collides into a tangle; with it, knowledge retrieves, links, and evolves reliably. The contract layer is where "stable" comes from in any system — code or knowledge.

And the checklist that matters, if you want this today, is short. Next time you hand a model a task, write a three-line spec with scope, assumptions, and acceptance. Pick one shared interface and pin its signature to a file the model isn't allowed to touch. Add one automated gate that compares the frontend's calls to the backend's implementation. Run one FMEA pass and force a fix on your highest RPN. And separate the writer from the verifier, even informally.

Do half of that and your AI stops building to impression and starts building to contract. Taylor's lost sentence is the one AI makes urgent: a contract isn't there to limit the AI. It's there to protect it — because an AI that never has to guess is the only kind you can trust.

Tomorrow we drop one layer down to the gates that refuse to let bad work in: let checks precede code. The contract says how it should be; the gate is what makes "should" actually happen.

A decision you didn't write down isn't a decision

DerekWang — Sun, 20 Sep 2026 04:09:00 +0000

AI Harness Engineering · Essay Three · derek wang (derekwang85)

The most expensive failure in AI coding isn't a wrong decision. It's a decision that was made, then forgotten, then quietly unmade by the next generation. Here's the raw version of that law: a decision that lives only in conversation does not persist. It evaporates the moment the context window closes. If making it stick matters at all, it has to leave the chat and land in a file every session reads.

That file is the architecture layer of the pyramid — the ADR set. In Essay One I gave you the skeleton; in Essay Two the top. This essay is the layer that stops the drift no one notices until it's cost a week.

The decision that survived a lunch break

I've lived the failure too many times to count. You and an agent go forty exchanges deep and finally converge: "OK, hexagonal pattern, the domain layer never touches the database directly. Locking that in." It is, in fact, locked in — as chat history. Locked in about as securely as a candle against a hurricane.

Next session opens fresh. The model regenerates, nothing tells it the hexagon was decided, so it wires the domain layer straight to a repository. Nobody spots it until the deviation has spread across twenty files. By then the why is gone — the reasoning that justified the pattern is buried in a conversation nobody will scroll back through.

What you're really watching is this: a conversation is not a contract. It's a one-time instruction stream — no history, no archive, no power to bind the next session. A decision that stays in the chat thread is as good as undecided. Only a decision that lands in a file anyone — especially any future model — must read is actually in force.

Three kinds of rot, one hole

When decisions have nowhere to live, three distinct failures follow, and I've seen all three in real projects:

Memory drift. The humans remember a decision; the AI doesn't, because the decision was never written where the model could read it. So every regeneration quietly re-argues a question you thought was settled.
Architecture drift. Each generation makes a small, innocent-looking deviation from the design. Twenty regenerations later the code sits half a project away from intent, and no single change is identifiable as the point of failure. The boundary didn't vanish — it just stopped being pinned and started being random.
Repeated-argument drift. Because nothing is recorded, the same debate happens again with every new face and every new agent. Sometimes the same team talks itself into a different answer than it landed on last time — deciding against its own past decision.

All three share a single root cause: there is no authoritative place where "we chose X, and here's why" lives. Close that hole and all three lose their fuel at once.

The single source of truth, pointed at the model

The fix has a name engineering has trusted for decades: the single source of truth. One decision, one authoritative home, everything else derives from it or cites it. In the AI era this stops being good hygiene and becomes load-bearing — because the thing reading your files is a literal-minded repeater. A human engineer reading a sparse architecture note fills in the gaps with judgment and context. A model doesn't. If the file doesn't say "don't touch this," then to the model, touching it is allowed. You can't rely on the reader's discretion; the file has to pin the boundary by itself.

That's exactly what an ADR — an architecture decision record — is: a written choice, kept once, that acts as the judge for disputes rather than a note for readers. AI coding makes it indispensable precisely because the previous failure modes are invisible until too late.

The seven fields that make a record hold

You don't need a thesis. A workable ADR is a short card holding everything a future engineer — or future model — must know without re-litigating the question:

Status — proposed, accepted, superseded, deprecated. Never blank; a decision that can't be marked superseded will be silently ignored.
Decision-maker — who owned it. Even a record of "delegated to the agent" counts.
Context — the situation that forced the choice.
Decision — the choice itself, stated without hedging.
Rationale — why this beats the alternatives.
Consequences — what it costs, now and later.
Mitigation — what you'll do when those costs show up.

Set that way, the ADR is a self-contained slice of history. The next session reads decision plus rationale and understands not just what was chosen but why — so it doesn't silently reopen it. In my methodology project, ADR-0003 does this for the rule system itself: it records how the constitution gets amended, proving the AR format can govern even the rules that govern the project. The counts are real at the actual scale I work at, measured from my methodology repo's adr/ directories: TradeOMS holds 20 ADRs pinning trade-domain rules agents must not improvise past [ORIGINAL DATA], and SmartQuant opened with 11 the day it started, recording architecture as it stood up rather than after the fact [ORIGINAL DATA]. The point isn't volume; it's that every directional choice has a paper trail.

The loop comes back as Swarm-Fed ADR

Here's the AI-era twist that turns ADRs from a top-down cage into a learning record. A Swarm-Fed ADR is written by the working swarm: when a group of agents resolves an edge case during delivery — a decision reached across models and roles, each attacking it from a different angle — that resolution gets promoted into a formal ADR and fed back into the single source of truth. The choice made in the heat of the job becomes a rule the next generation obeys. This is the return channel from Essay One made concrete: the architecture layer constrains from the top, but it also accumulates what the system learns from below. Decisions stop being ivory-tower declarations and become a living sediment of resolved problems.

Written is not enforced

The objection worth taking seriously: an ADR nobody maps back to the code is a diary with nice formatting. I've watched teams file fifty ADRs in a quarter and still drift, because the records were approved and then ignored — never wired to a gate, never surfaced to the AI, read only at the retrospective.

Records are inert without enforcement. An ADR binds only when something checks whether the code follows it. A Swarm-Fed ADR binds only when retrieval actually surfaces it before the next generation. And the riskiest habit in AI coding isn't changing a rule — it's changing it for no recorded reason, so a future model reads an un-sourced edit the way it reads an un-sourced comment: as permission. So the habit to build is the counterintuitive one: before you move a boundary, write the ADR. It looks like an extra step. It's actually the thing that makes the change legitimate. Write it down in the moment, and you give every future conversation a coordinate origin that never disappears.

Tomorrow, one layer down: the contract layer — how you take a spec and make an AI promise, and prove it kept the promise.

A constitution is a behavior contract, not a description

DerekWang — Sat, 19 Sep 2026 04:05:00 +0000

AI Harness Engineering · Essay Two · derek wang (derekwang85)

Before you tell an AI what to build, you have to tell it who it is, where its lines are, and how it's allowed to work. Get that wrong first, and no amount of clever prompting downstream will save you. This essay builds the top layer of the constraint pyramid: the strategy charter — the one file that decides whether your AI is doing the right thing at all.

The one file that sets the coordinate system

A prompt tells the model a task. A constitution tells it who it is. For an AI-governed project, the constitution is a behavior contract every participant — human and model — must load before starting work. It answers three questions:

Who are we? One line that pins the project's essence. In one of my projects the charter's first sentence is "a general-purpose coding-engineering methodology and multi-agent orchestration system." In another it's "a futures-and-physical trading management system, Vue 3 front end, Java 17 + MySQL 8." The sentence matters less for its content than for being the first thing the model reads — it sets the whole coordinate system for everything the model generates after.
Where are the boundaries? What you refuse to do matters more than what you do. A clear "not accepted here" list — which stacks we won't touch, what directions we won't expand, what quality bar we won't let through — keeps the model from drifting down any direction that merely sounds reasonable.
How do we work together? The rules between humans and the AI: which processes are mandatory, what the model may not touch, and how the two sides hand work to each other.

If these three aren't settled in a file the model reads before it starts, your AI is improvising against a void. If they are, one file is enough — no pile of READMEs can make up for a charter that never asked who it was.

Code rules tame the output. Behavior rules tame the producer.

Teams usually reach for code conventions first — naming, layering, interfaces. Those govern the artifact. They don't touch how the model behaves before it writes anything: how it checks, how it verifies, how it owns mistakes. That's where the accidents actually live, so this is where I spent my hardest-won rules.

The behavior layer in my methodology project is written as a list of ten rules, and none of them is about syntax:

Read, then write. Never edit a file from memory.
State your assumptions. Every uncertain design call carries its assumption and reasoning out loud.
Check the ripple. Before changing A, look at B, C, D for damage.
Scalpel edits. Smallest possible diff; never touch unrelated files "while I'm here."
Verify first. Fix a bug by writing a failing test, watch it fail, then fix.
One responsibility. A function does one thing; mixed concerns breed "fix one, break three."
Trust the disk, not memory. Code state comes from the files on disk, never from what the conversation remembers. This one cost me real time: an agent once insisted a file was an old version because that's what the chat memory held, while the file on disk had long since changed.
Fail safe. On error, land in a safe state; refuse beats allow; degrade visibly, never silently.
Write the lesson back. After a fix, record it; a knowledge base that doesn't grow decays.
Three strikes, replace the tool. When the same kind of failure happens a third time, change the tooling — don't write a fourth patch.

None of these depend on a specific model or framework. Whatever agent you use, have it read the list before it works, and it's money in the bank.

The clause only the AI era needs: anti-sycophancy

A human organization has no equivalent for this one, because it polices a defect human teammates eventually get called out on and a model never does. A language model is a probability machine that leans toward whatever keeps the conversation comfortable — a plausible answer instead of a checked one, "good idea" instead of a challenge, "you're right" when it has no evidence. It has no shame, so it can run flattery to perfection. Left alone, it will cheerfully build you a confident wrong thing.

The charter answers this with a written clause. The model asks itself three questions before it outputs: Am I giving up my position to keep the conversation pleasant? Am I conceding a point without asking for evidence? Is this answer too smooth, too eager to please? If so, it strips the polish, or writes "I don't know." There are hard companion rules: accuracy beats pleasing, so no "great question!" or "happy to help!" — help directly, answer directly, disagree directly; and when uncertain, say so in the first sentence, no preamble.

Paired with it is the confidence label. On anything that matters, the model tags every core claim — [HIGH] (mapped in my methodology's data contract to ≥80% confidence [ORIGINAL DATA]), [MED], [LOW], [GUESS]. This is the clever part: it forces a model not to hide its quantitative nature to seem human. A human expert rarely labels their own certainty because their track record backs them; an AI has no track record, so the label is the only honest evidence of trust. The moment it has to stamp a claim [GUESS], it can't counterfeit certainty anymore. That's the one thing agentic AI quietly tries to take from you — the ability to know how much to trust the output — handed back.

One idea, two instances

The same pattern shows up across my projects in two shapes, which conveniently demonstrates the "names vary, layers hold" point from Essay One:

Project	Charter file	Shape
The methodology framework	`constitution/` five-piece set	pre-execution manifest, coding standards, behavior commandments, cognitive discipline — deliberately framework-agnostic
The trading system	`CONSTITUTION.md`	a single file that funnels AGENTS.md + SOUL.md + rule files; written tight because money is at stake

Both obey one hard priority that keeps the top layer honest: charter > any agent's own AGENTS.md > skill docs. If an agent's self-rules could override the charter, the charter is wastepaper. The charter has to be the first file every agent is forced to load. The complexity of the charter should match the complexity of the collaboration — one focused file for a solo project, separated behavior-and-cognition files for a multi-agent team, framework-neutral for a methodology meant to be reused. Thicker is not better.

A living document, or nothing

A charter that never changes is a gravestone. Once the team treats it as finished, the model outgrows it, the business moves past it, and the file becomes a museum piece people nod at and ignore.

What keeps it honest is the same thing that keeps any contract honest: make change visible and cheap. Every amendment goes through the repo like any other change — a diff, a review, a record. In my methodology project the amendment path itself is pinned by an architecture decision record (ADR-0003), which records why the constitution exists and how it gets changed — so revision is a documented motion, not a political fight. When I moved the rule system from one-way top-down to bidirectional feedback, I wrote the ADR first, then rewrote the charter. You don't just change a rule; you file the decision and propagate the change everywhere it touches — behaving, in that sense, like a responsible person who owns the results of their own change. A constitution that can't be amended dies; one that can stays in the present tense.

The honest limit

Let me steelman the skeptic. A team writes a beautiful long charter, the AI nods along, and everyone treats the file as protection. It becomes an amulet.

The truth: a constitution is worthless until it's wired to the layers beneath it. The confidence markers only bind if the gates check for them. The boundaries only hold if the contracts and gates enforce them. That's why it's a pyramid — a charter floating above nothing is a politely worded wish. And a charter you write but never revise is worse than none: the model reads it as a suggestion, not a contract. A charter isn't written into existence. It grows — you fall in a hole, you add a rule; you find a rule gone dead, you amend it. The apex stands, or nothing below it does.

The system lives in your files, not your prompts

DerekWang — Wed, 16 Sep 2026 23:58:36 +0000

AI Harness Engineering · Essay One · derek wang (derekwang85)

Ask a senior engineer where "the system" lives and they'll point at the code, or the org chart. After shipping enough AI-governed projects, I'd point somewhere else: the system lives in the files that fence in what a model is allowed to generate. The longer I watch teams steer language models, the surer I am that the difference between a project that drifts and one that holds a line is decided here, in a file the model reads before it writes anything.

This is the first layer of the idea I call the constraint pyramid. I'll keep it grounded in the one shape I've actually shipped more than once.

Prompts are memory. Files are architecture.

The mistake I made first, and that most teams still make, is trying to govern an AI with better prompts. "Just tell it the constraints in the chat." A prompt is a memory, not an architecture. It sits in a context window, one refresh from being washed away, one regeneration from being quietly ignored. You can't verify that the model obeyed it, and the same prompt produces different results across models and contexts. Prompt engineering buys you a conversation, not a system.

A file is the opposite. The model's output is bounded by what it can read, and a file it is told to read before every generation is not a suggestion the model has to remember to honor — it's ground truth the model hits before it produces a single line. Write your boundaries once, into a README that gates the project, and you've turned a hope into a constraint.

That shift — from words-in-chat to rules-in-files — is the whole premise of this series.

Five layers, one skeleton

The pyramid is a name for the set of files, arranged as five layers, where each layer answers one question and translates the one above it:

Strategy      —— README / constitution  : what this is, its boundaries
   ↕
Architecture  —— ADR set                : where the limits are, what won't be revisited
   ↕
Contracts     —— spec + runbook         : what exactly must be delivered
   ↕
Gates         —— pre-submit checks      : what is not allowed to become code
   ↕
Implementation— constrained output      : the model's work, fenced in from above

Read top to bottom, it moves from "why" to "how," each layer narrowing the space the layer below is allowed to work in. Read bottom to top, it's the audit trail: every line of generated code should be traceable up through a gate, a contract, an architecture decision, and finally a strategic intent. Constraint flows down; accountability flows up.

The layer names aren't the point — in different projects this same skeleton is called a README, a constitution, a skill set. What holds is the arrangement: fewer, heavier files at the top; more, freer output at the bottom.

Not all constraints weigh the same

The skeleton alone doesn't make the system work. What makes it executable is a second dimension I had to learn the hard way: constraint strength, in three tiers.

L1 — hard constraints, enforced by code and gates. A pre-submit script that fails a build over a naming violation isn't a request; it's a wall. Schema, shared DTOs, dependency files, the gate script itself all live here, and the model is not allowed to touch them.
L2 — architecture constraints, carried by accepted ADRs and file layout. They don't fail a build, but they define what "following the design" even means; changing one requires a new ADR, not a silent edit.
L3 — execution suggestions, the loose end. Runbook steps, skill definitions, checklist guidance — the model may improve on these, but it has to record what it changed.

The whole game is making sure the model knows which tier a rule sits in. Without that distinction, an AI treats every file as polite advice and rewrites whatever it likes — and then the constraint system exists only on paper. A rule placed too hard chokes the system on trivia; a rule placed too soft is decoration. The pyramid earns its keep precisely by letting you put each rule where it belongs.

What actually happened at 636 lines

This isn't a diagram I drew on a whiteboard. In a real project — the AI-generated operations system I refer to throughout this series — the strategy file started at two vague lines: "an issue-tracking system, Java backend, Vue frontend." No module boundaries, no naming rules. The model's output drifted weekly: controllers called TicketController in one file and TicketApi in another, business logic shoved into controllers in one module and into services in the next. After a few weeks each style looked correct in isolation, and unifying them stopped being a bug fix and became a rewrite — heavier than starting over, because now we had baggage.

So I stopped editing the prompt and started editing the file. Over the life of the project that README grew to 636 lines [ORIGINAL DATA] — module boundaries, path conventions, naming, layering, each rule pinned down one by one. When the model read it before every generation, the week-to-week style drift that had been a constant nearly vanished. The model didn't follow the rules because I asked nicely. It followed them because the rules were the file it was told to read first.

That's the whole difference between constraint and lecture. A lecture says "please write consistently." A constraint says "you can only write inside this boundary." The first one a model tunes out; the second one it can't miss.

Constraints go down. Deviations come back up.

One more piece, because people leave it out: the pyramid is not a one-way funnel. The model doesn't only obey constraints — its output should change them.

When generated code keeps violating a naming rule, that rule escalates from a soft suggestion to a gate check. When a gate keeps catching the same error, it's a sign the contract layer is missing a step, so you add the step. When a rule fights you no matter how it's phrased, that usually means the boundary itself is wrong — which becomes a new ADR, and sometimes a revision to the strategy document itself. Deviation flows up; the constraints get sharper for it. A healthy constraint system isn't rigid, it's alive — it has a return channel, a trace of what previous projects taught it, so it doesn't re-learn the same lesson in week four. I'll open that mechanism properly in a later essay.

Nothing defends a wall that's never checked

I can steelman the skeptic easily: files rot too. A README that says one thing while the code does another is worse than none, because it sells you the illusion of governance. I've watched teams paste a six-hundred-line spec, feel safe, and drift all the same.

The only honest defense is uncomfortable: the system has to be run, not written. A gate that never executes is a wish. An ADR nobody maps back to the code is a diary. The pyramid works only when its layers are wired to one another and fired in the pipeline. The moment it degrades into static documentation, it deserves exactly what it replaced.

The first thing to try today is the smallest possible version: find the place your model reads before every generation. If it's a chat, move it to a file — a file that says what the project is, its module boundaries, and its naming rules. Two lines is a start; six hundred is a system. Then sort every rule you currently carry in your head into one of the three tiers, and let the model see the difference.

In the next piece I'll show you the top layer — how to write the project's constitution before the code, so an AI is willing to live under it and, just as important, stops flattering you while it does.

AI Harnessing: AI writes code fast. Now what?

DerekWang — Mon, 31 Aug 2026 10:00:00 +0000

AI Harness Engineering · Prologue

derek wang (derekwang85)

In 1968, a roomful of the world's best programmers admitted something nobody wanted to say out loud: writing code was quietly outrunning the human ability to manage it. Projects slipped. Budgets blew. Quality collapsed. It wasn't the exception anymore. It was the norm.

That meeting gave the industry a name for the disease — the software crisis — and over the next half-century, discipline after discipline tamed it: structured programming, object-oriented design, test-driven development, continuous integration. Chaos, ground down into order. The field settled on a shared belief: quality, speed, and cost pull against each other, and you can't get something for nothing. Every choice is a trade-off made with your eyes open. Consumer internet optimizes for experience. Industrial control software optimizes for uptime. Resource-and-process software optimizes for how fast it can absorb change. Each picks a different priority. None of them walks away from speed.

So two philosophies emerged for buying back time. One wins by reducing rework — design it carefully up front, and you don't have to rebuild it twice (waterfall). The other wins by harvesting feedback fast — iterate quickly, and the market tells you where you're wrong before you've over-invested (agile).

When you stack requirements management (what to build), methodology (how to build it), architecture (how to wall off the boundaries), and an engineering team that can actually carry the work (who gets it done), we once believed the software crisis was a solved problem.

Then LLMs and agents showed up.

And the crisis flipped. The problem is no longer that code is written too slowly. It's that code is written too fast — faster than we can understand it, trust it, or maintain it. The software crisis is back, except this time what needs taming isn't code written by humans. It's code written by AI. And taming it needs a harness.

1. The flip side of speed: trust, and what you lose

In the past year, almost every software engineer has had that moment where the speed is unsettling. You give the AI one sentence. Forty seconds later it hands you a complete feature: coherent logic, comments in place, tests already written.

Then the doubt creeps in. You stare at the code. Why was it designed this way? Does it respect the conventions of the neighboring module? Did it handle the edge cases? Will it silently collapse on some transaction, at three in the morning, when nobody's watching?

The more I use AI coding, the more I believe managing AI-native software is less a technical problem than a management problem. When your repository fills up with AI-generated files, that old comfortable feeling — "I typed every line, I know what it does" — is gone. In its place is something harder to name. You look at a screen full of code and feel hollow: is any of this actually useful?

Drucker said it sharply, and I'll quote him directly:

There is nothing so useless as doing efficiently that which should not be done at all.

AI is doing exactly that — at absurd efficiency, it generates huge volumes of code that should never have been written in the first place. It has accelerated the writing to the limit and handed the burden of understanding, trusting, and maintaining back to humans, untouched — and doubled.

My own proof point: over the last six months I shipped several projects with AI coding. The first one, in hindsight, was a learning exercise as much as a delivery. It was an internal intelligent-ops system, upgraded with AI — codename aITMS. The original ITMS was live in production, but it never felt "smart" the way modern agents do. The goal was to internalize an OpenClaw-style interaction layer.

Five weekends of spare time (roughly ten working days), using AI agents powered by OpenClaw, produced: 18 controllers, 35 request DTOs, 38 response DTOs, 1,557 lines of bilingual i18n config, 39 architecture decision records, and file-level changes scattered across hundreds of files. In retrospect, none of that is remarkable. In early 2026, it was startling — almost frightening.

Nice numbers. But those AI-coding wins didn't turn into a disaster, and that wasn't because the models were strong. It was because I kept iterating alongside them and built a constraint system behind them.

One clarification before I go further, to avoid confusion: aITMS is the original carrier of the methodology and the practice project. Nearly everything downstream — the constraint pyramid, the gates, the maturity model — evolved from it. But the test of whether these count as a methodology isn't "it worked in aITMS." It's "will it hold up in a different project?" I'll draw that line explicitly in Section 5.

2. The paradigm shift: from writing code to taming it

Traditional software engineering assumes, from day one, one thing: code is written by humans.

Every quality tool we built takes that shape — compile checks, code review, test coverage. They all exist to catch the mistakes after a human produces the code. Underneath is an unspoken management logic: you trust the engineer's process, but you don't trust every line they write — so you back them up with process and tooling.

But once the producer of code becomes an LLM or an agent, that assumption breaks. And so does the management logic built on it.

Dimension	Traditional engineering	AI Harness Engineering
Who writes the code	Humans	LLMs + agents (humans become supervisors)
Output speed	Medium	High — AI generates in parallel
Primary risk	Logic bugs	Hallucination + inconsistency + architectural drift
Constraint mechanism	Compile checks + review	File-system constraints + gates + contracts
Quality assurance	Test coverage	Constraint coverage + contract validation + observability
Management focus	Managing how people write code	Managing how AI is constrained

The change isn't really about speed. It's somewhere deeper: the center of gravity moved from "managing how people write" to "managing how AI is constrained." Recall the four pillars from the opening — requirements, methodology, architecture, the engineering team. In the AI era, the first three change meaning entirely. Requirements shifted from "talking to people" to "writing files the AI can read." Methodology shifted from "how people write code" to "how you constrain AI writing code." Architecture shifted from "the design in someone's head" to "the boundaries written in files." This isn't a tweak to a technical detail. It's a shift in the entire paradigm of software-engineering management.

Embracing AI coding is a kind of democratization for non-programmers — it spreads the ability to author software. Yet ironically, the most capable software engineers are the most eager users of AI coding. If AI is the more trusted, more efficient option, why do professional engineers still insist on constraining and managing it?

Maybe because an LLM generates tokens by probability — it's fundamentally a stochastic process. And maybe because the code it produces is still bounded by its memory system (context length). AI coding today isn't perfect, and because the limits live in the base model's internals, neither algorithmic nor engineering breakthroughs will arrive cheaply in the short term. So AI coding is feasible in principle — but hiding in the details are subtler defects:

Data hallucination — the model "decides" a field should be named X, names it X, and it doesn't match the real column in the database.
Concept drift — the same concept is called A in one file, B in the next, and C on the tenth regeneration.
Architectural drift — each generation introduces an "innocent" little deviation, and after thirty rounds, the codebase has drifted away from your design without you ever noticing.

None of these three is a "code quality" problem in the traditional sense. They are all management problems. You didn't govern the data alignment. You didn't govern the conceptual consistency. You didn't govern its improvisation. So the output drifted from the intent.

Software is an engineering problem for two reasons: the direction has to be steered dynamically — that's a management problem — and the precision has to be measured strictly — that's a math problem. A few hundred lines for a throwaway page is a different universe from hundreds of thousands or millions of lines. The gap is the difference between digging a hole with a plastic shovel on the beach and boring through a mountain with a tunnel-boring machine.

So can I solve this new problem with the old toolbox? Can code review hold the line?

If humans do the review, it only catches a fraction. AI generation has already outrun human review — in the time you review one PR, the AI regenerates three. Trying to out-speed AI with human reviewers is a losing war of attrition, like fighting an assembly line with manual inspection. Not because the inspectors aren't trying, but because the line never gives you the window.

So I stopped chasing it. I moved the defense line forward — before the AI generates, I constrain its behavioral space so the risk never materializes. That's the real subject of this series:

When you can't hold quality by "looking fast enough," you hold it by "fencing the boundaries in advance."

3. The two faces of the harness: steering and restraint

With the defense line moved forward, you need the right tool — one that fences in the AI's behavioral space without choking its productivity. There's now a popular name for it: the AI harness.

The word harness originally means the rigging you put on a horse — the straps, the bits, the tack — that lets a small human control a far stronger animal. We developed a mature, reliable harness because we wanted to wield the speed and mobility of an animal far beyond our own strength. It took roughly six thousand years of trial and error — from taming the first wild horse to finding a structure that steers the power without hurting the animal. Generations of accumulated trial-and-error wisdom. And that taming process is exactly what today's engineers are doing for LLMs. Every constraint adjustment is a product experiment answering one question: does this actually work?

Like a horse harness, an AI harness serves you in two directions:

Steer — channel the force: make the AI "pull" in the right — or at least roughly correct — direction.
Restrain — bound it: keep it from bolting, keep it from throwing a tantrum.

Is restraint even good? Doesn't it strip away the model's native capabilities? The AI's power is real and large. Without a harness, it tears through your codebase, leaving a mess behind. That self-doubt from Section 1 — staring at a screen full of files, unable to tell which are useful and which are hallucinated — is precisely what happens with no harness. Riding a wild horse is more exciting, but it comes with an unpredictable temperament and an unknowable destination.

AI Harness Engineering is the engineering methodology for fitting that harness. It is not a tool. It is not a prompt trick. It is a system: file-system constraints + gates + contracts + a maturity model. It may well be the seed of a new software-engineering methodology — one day, a body of practice that helps us solve the AI-era software crisis.

The core thesis, applied to AI coding, condenses to this:

When AI writes the code, you no longer control quality by writing code. You control it by building a constraint system that guides the AI's behavioral space. In other words: writing fast isn't the point. Being able to control it is.

4. The spec system: a constraint pyramid

AI Harness Engineering is sometimes translated as something like "steering engineering," and I think that is more an aspiration than a description. The practical constraint, the one that actually shows up in projects, lives in a series of files. In the project docs you'll often see them described by different names across agents — charters, PRDs, architecture docs, methodology docs, script docs, the code itself. Don't get caught up in the naming. Spend your attention on the constraint layers instead. That's where the return is.

The core of the spec system is one structure — the constraint pyramid:

Strategy   —— README.md (1 file: boundaries / naming / topology)
   ↕
Architecture —— ADR set (39 files: one record per consequential decision)
   ↕
Contracts  —— Runbook + Skill (repeatable steps + reusable workflows)
   ↕
Gates      —— pre-submit scripts (automated checks before commit)
   ↕
Implementation —— constrained AI-generated code (N files)

Each layer answers one question. Strategy answers what and what-it-is. Architecture answers where the boundaries are. Contracts answer persistence and stability. Gates provide the quality-assurance system. Implementation realizes the goals of Strategy, Architecture, and Contracts — and submits to the gates.

These five layers weren't dreamed up. They were walked into being. In aITMS, the early README was two vague lines, and the AI's code style changed every single week. Then the README grew to more than six hundred lines, indexing the other layer files — module boundaries, naming, layering rules, pinned down one by one. The style drift nearly vanished. The pyramid's point isn't a pretty document structure. It's a fence around the AI's unconstrained improvisation. Put the fence up, and the style stabilizes.

This constraint system has an endpoint. It isn't about locking AI down. It's about getting somewhere further away — to a place where the system learns to improve itself. There's a recurring character in this series: a local knowledge-brain, a "co-brain" that shares coding capability across projects, called derekinside. It has a knowledge wing and entity nodes. It's not a static knowledge base; it's an AI-native system that remembers, retrieves, and evolves on its own. In essay nine, I'll open it up and show you what it remembers, how it retrieves, and what drives its self-evolution.

5. The series map: twelve essays

Twelve essays plus this prologue — thirteen in all. Every essay follows the same recipe: a methodology, a real-project proof, and a copyable checklist.

The Constraint Pyramid: files as architecture — the five-layer spec system in full; why "files" are the blueprint of the AI era
Strategy: charter & blueprint — what to build and what it is; the project charter and AI's behavioral floor
Architecture: decisions as boundaries — why "why" must live in a file; how decisions hold consistency together
Contracts: from spec to code — spec contracts, data contracts, and repeatable governing specs
Gates: let checks precede code — the only automatically-enforced constraint layer; evolution from five gates to fifteen
Implementation: idempotency & degradation — interface idempotency, degradation contracts; the business doesn't stop when AI is down
The pre-execution manifesto: rules before code — the Manifest's six questions and the order files get created
Multi-agent orchestration: from a single point to division of labor — swarm decisions and multi-model consensus
Memory & knowledge: a system that evolves itself — how a co-brain that remembers, retrieves, and grows actually works
Debugging & attribution: beneath the iceberg — iceberg PDCA and hypothesis-driven debugging
Self-growing tests: let regression run itself — full regression, failure-pattern matching, and a self-growth engine
The maturity model: self-diagnosis & evolution — where your project stands, and how the constraint system measures it

The methodology's experience comes from four projects on my desk: a trading system (TradeOMS), a coding-methodology framework (derekcoding-framework), a quantitative-research system (SmartQuant), and a local co-brain (derekinside). Different industries. Different maturity. But all four grew out of aITMS practice — aITMS is the origin, not the whole.

The constraint pyramid you've previewed, and the maturity model to come, were extracted from live practice. They're deep case studies, they're implementation templates, and they're the methodology thinking about itself. There is certainly room to improve them. I'm putting them out here to provoke deeper thought among my peers — a brick thrown to attract jade.

6. A self-check: five first-day questions

Before you start, here's a checklist you can run today. If your project is using AI coding, go through it item by item:

Does a README define module boundaries and naming conventions? (Not a one-paragraph "intro." A blueprint AI must read before it generates.)
Are architectural decisions written down — or are they living in a chat history, one refresh away from being washed away?
Is there a pre-submit script that checks automatically — or is everything riding on manual review?
For AI-generated interfaces: have you thought about what happens when the same request is submitted twice?
Which parts hard-depend on AI responding instantly? If it goes dark tonight, is there a degradation path or a manual fallback?

If you can't — or aren't sure you can — answer one or two of these five, this series is for you.

Putting a harness on AI isn't about holding its power back. It's about pointing that power where you want it to go. Fast isn't fast if you can't control it.

Next up, from the first foundation stone: The Constraint Pyramid: files as architecture — why files are the real blueprint of the AI era.