TL;DR — When the spec is good, implementation needs less model. I started using a top-tier model to write the spec and a cheaper, faster one to implement it — still using the strong model, just spending it on the spec instead of the implementation. The gain isn't some magic prompt phrasing; it's the context: explicit business rules, audited project constraints, a defined output contract. That's systems engineering — the discipline of anyone who's kept real software alive, whatever their stack.
Every backend dev knows the scene: the Swagger is out of date, the last hotfix shipped without a unit test, and the README.md documents a command nobody's used in six months. The code works. The docs lie. And the gap between the two is exactly where AI — and we — start to go wrong.
I've spent the last few months developing with AI for real inside production projects, not tutorial greenfield. My takeaway was less about which model to use and more about a shift that already has a name: the move from prompt engineering to context engineering.
The difference isn't semantic. Prompt engineering treats the problem as writing — finding the magic phrase. Context engineering treats it as what it always was: a systems engineering problem. And it's where my backend background applied most directly — though anyone who's kept a real system alive has the same instinct.
The experiment that convinced me
Let me start with the evidence, because that's what made me take this seriously.
My reflex, for a long time, was to reach for the strongest model for everything — more expensive, smarter, fewer errors. Makes sense on paper.
In practice, I saw something else. When the task's specification is well done — explicit business rules, audited project constraints, a defined output format — the model capability needed for implementation drops sharply. Enough to split the work by stage: I started using a top-tier model (currently Opus) to write the spec, and a cheaper, faster model (Sonnet) to implement from it. I didn't drop the strong model — I moved it: off the implementation, onto the spec.
The comparison that actually matters isn't cheap implementer vs. expensive implementer. It's strong model implementing with no structured spec vs. strong model writing the spec, cheaper model implementing from it — and the second is both cheaper and more accurate. The accuracy isn't coming from the cheap model — it's that the spec carries the precision. With unlimited budget you could run the strong model on both stages, and that's the ceiling; the useful finding is narrower: good context makes a cheaper implementer beat the strong model left to implement without it — while saving on the highest-volume stage.
Be precise about what that does and doesn't show, though. Sonnet isn't a weak model — it's a notch below Opus, not a tiny one. So this isn't evidence that some minimal model can implement anything; it's that a good spec lowers the capability ceiling a task demands, and in my case that ceiling dropped below top-tier. How far it drops depends on the task.
I want to be honest about the scope of this claim, because it easily turns into misleading clickbait. It's not a universal recipe. The exact split depends on task complexity, codebase maturity, and how good your patterns already are. Apply this to a messy project with no specs and the cheap model will hallucinate, and you'll conclude I was wrong — when what was missing is exactly the half that makes the claim true.
What is generalizable is the principle behind it:
Context quality lowers the model capability a task requires.
The Opus→Sonnet split is just concrete evidence that the principle works. And it explains, as a bonus, why the hype migrated so fast from "prompt engineering" to "context engineering": the marginal gain from investing in context is bigger than from investing in the phrasing.
What counts as "good context"
Here's a trap I also fell into: thinking more context is better context. It isn't. Dumping the whole repo into the window doesn't help the model infer better — it just inflates cost and dilutes signal. Good context isn't volume, it's precision. In practice, what moves the needle:
- What the project does — one sentence. Sounds obvious; it's almost never written down.
- The stack, the architecture, and the patterns — because LLMs are excellent at replicating. That's their core strength. A project with consistent, documented patterns gives the model a mold to copy, and that's where hallucination drops.
-
Clear commands in the
README.md— silly but real example: I had a test breaking, asked the agent to fix it, and it ran the test command itself to see the error before touching anything. It only managed that because the command was documented. - The output contract — defining the input and output format of functions and methods up front gives the model a target instead of a guess. One caution before calling this "TDD as executable spec," though: in my flow the same model writes both the test and the implementation, so the test isn't an independent oracle — it's the model checking its own reading of the task against itself. What actually anchors things is the contract in the spec I review by hand; the test only operationalizes it, and its assertions still have to survive Check 2 of the verifier below.
Notice none of this is "prompt technique." It's engineering hygiene. AI just made the cost of skipping that hygiene visible and immediate.
The system: a pipeline of skills
A thesis without a method is a motivational talk. So here's the how.
I treat every repeatable step of my flow as a skill: a structured prompt with three fixed parts — Instructions, Constraints, and Output Format. They live versioned in .md files, reachable from both the editor and the agent. When I catch myself asking the model for the same thing a third time across different projects, I formalize it into a skill.
The flow of a task in an existing project looks like this:
extract task description
→ spec-writer (generates the spec from the informal ticket)
→ [review the spec] (human in the loop — the step I don't delegate)
→ implement
→ ai-output-verifier (red-team what the AI generated)
→ code-reviewer (senior-level critical review)
→ commit-message (Conventional Commits from the diff)
→ pr-description (description in the team's format)
→ push
→ review by another model on the PR
Three of these skills do the heavy lifting. Let me open all three.
1. The one that generates context: spec-writer
The heart of the pipeline. It turns an informal ticket (natural language, bullets, a Slack thread) into an implementation-ready spec. The detail that makes the difference isn't the template — it's the Constraints Audit: before describing what the code should do, the skill forces an inspection of what the project already has configured that could invalidate the model's output.
# Skill: Spec Writer
Turns an informal ticket description into an implementation-ready SDD spec.
## Instructions
1. Read the provided description.
2. Identify ambiguities BEFORE writing the spec. If there's at least 1
critical doubt, STOP and ask.
3. Project Constraints Audit (mandatory). Before describing what the code
should do, audit what the project ALREADY HAS configured that could
invalidate the output. Run concrete inspection — don't assume:
- Schema (DB writes): inspect the table. NOT NULL, defaults,
UNIQUE/FK, encoding. Sample existing data to see the expected
format of special fields.
- Transpile/syntax (JS/TS): Babel rules may differ across
.js, .jsx, .ts, .tsx. Confirm modern syntax (?., ??) IN THE SAME
file type that will be modified.
- TypeScript config: target, strict, noImplicitAny.
- Lint config: rules that could reject the output.
- Dependencies in use: suggest libs ALREADY present, don't add
new ones without justification.
- Existing patterns: grep for a similar implementation and follow the style.
4. Suggest at least 3 non-obvious edge cases from the original ticket.
## Constraints
- DON'T invent business rules. If info is missing, mark it as TODO.
- DON'T assume default stack/config. Use what the Constraints Audit
found. If transpile doesn't support ?. in .jsx, the spec says
"use traditional conditionals in .jsx".
- Keep the spec short: one screen max.
Why this matters: most of the hallucinations I've seen weren't the model "inventing" from nothing. They were the model assuming a reasonable default that the specific project didn't follow — ?. in a file that project's Babel doesn't transpile, an insert that violates a NOT NULL nobody mentioned. The Constraints Audit kills that entire class of error before the first line of code.
2. The one that fights sycophancy: ai-output-verifier
Sycophancy is the tendency of LLMs to agree with, flatter, and validate the user at the expense of accuracy — the model tells you what you want to hear instead of acting as a neutral truth-seeker. In a flow where AI generates a lot of code, this is the hidden tax: the code looks right, passed the tests (which the same AI wrote), and nobody checked whether the tests test anything.
This skill formalizes distrust into a checklist. I run the checks in order and stop at the first red flag:
# Skill: AI Output Verifier
Red-team of AI-generated output. Fights sycophancy, fabrication, and the
"code that looks right but isn't" pattern.
### Check 1: Fabricated references
Did the AI cite file:line, a function, or an API? Open each one and confirm
it exists AND does what it claimed. Red flag: a plausibly-named but
nonexistent function, a lib version different from the installed one.
### Check 2: Tests that don't test
Does each expect validate BEHAVIOR, or just that it "ran without throwing"?
Would the test fail if you commented out the implementation? Red flag:
expect(result).toBeDefined(), mocks that return the expected result
instead of simulating real behavior.
### Check 3: Happy path syndrome
Does it work with empty input (null, undefined, "")? Under concurrency?
With the external integration down? Red flag: no try/catch on I/O, no
input validation.
### Check 4: Adjectives without evidence
Did the AI use "robust", "performant", "secure", "production-ready"? Is
there a benchmark or technical argument to back it? Red flag: "more
performant" with no comparison.
### Check 5: Sycophancy detection
Did you push the AI in a direction and it changed its mind too fast,
without a new technical argument? If so, ask explicitly: "what's the
evidence for this new position?"
### Check 6: Plan vs delivery
Did the AI reference files/specs/paths as if they exist — in summaries,
conclusions, or next steps? For each path, run `ls` or open the file: does
it exist? Watch verb tense — "X becomes the destination in Y/Z/" is a plan
dressed as delivery. In the final summary, separate "I created" (verified
by `ls`) from "I suggest creating" (just a plan).
### Check 7: Confidence on inferences
Did the AI make claims it couldn't have verified directly? It must not give
the same affirmative authority to what it SAW in the code and what it
INFERRED from folder structure. Demand a level: High (directly observed) /
Medium (inferred from structure) / Low (speculative).
## Constraints
- DON'T mark "verified" if any check landed on "maybe". Go back to the code.
- DON'T rely on the AI's tests alone. They came from the same source that
might have the bug.
- DON'T skip a check because "it's a small change". Sycophancy scales —
small changes are where AI cheats most.
The metric that started to matter to me isn't "how much code per day." It's how much bad code I kept from reaching production. This skill is the instrument for that.
3. The one that gives the verdict: code-reviewer
Short and direct. Critical review at a senior backend Node/TS level, focused on correctness and security — not style, which Prettier handles.
# Skill: Code Reviewer
Critical code review at a senior backend Node/TS level. Focus on
correctness, security, maintainability — not style.
## Instructions
1. Read the provided diff.
2. Classify each issue as Blocker, Major, or Minor.
3. For each Blocker/Major, give a concrete example of the bug (input that
breaks it, race condition scenario).
4. Suggest a refactor ONLY when it reduces real complexity. Not for aesthetics.
5. End with a Verdict: Ship / Fix and ship / Block.
## Constraints
- Assume Node 20+, TypeScript strict, ESLint configured.
- Don't comment on formatting.
- Consider concurrency, DB transactions, idempotency, promise leaks,
silent error handling.
- If it's an HTTP endpoint: input validation, authz, rate limit, PII logging.
Notice the pattern across all three: Instructions + Constraints + Output Format, always. The Constraints are the part that matters most, because that's where you encode the "do NOT" — and the "do NOT" is what keeps the AI on the rails.
Where this points
The Achilles' heel of all this is documentation aging. A spec is a snapshot of the past; the code keeps changing. The direction that feels right to me is living documentation — generated and maintained on every change, not in a heroic quarterly effort. Tools are already aiming exactly at this, and while they mature, you can approximate it: take a snapshot of the project and ask the model to generate a context file from the code + README.md.
How I see this evolving inside a repo: alongside the README.md, a _Skills/ folder with the team's structured prompts and anchor files (CLAUDE.md, CODEX.md) that list the available skills and point to the project's CONTEXT.md. The model enters the repo and already knows what to read, in what order.
And there's a layer beyond this one. Context engineering is about what you put into a model call; the moment you wrap those calls in a system — quality gates, retries, an independent reviewer, persistent memory — you've crossed into what the field now calls harness engineering: the discipline of everything that isn't the model. The skill pipeline above is already a harness in miniature. That's where I go next.
None of this is exotic. It's the old engineering discipline — document decisions, keep patterns, write tests that test — with a new, impossible-to-ignore incentive: now the cost of skipping those steps shows up immediately, in the form of an agent that hallucinates. AI didn't change what good development is. It just stopped letting us pretend that shortcuts are free.
This is the flow I run today. The skills evolve every week as I catch the AI failing in new ways — if you run something similar and hit different walls, I'd want to know which.
Top comments (9)
This lines up with what I see building RAG and agent systems. The leverage is almost never the prompt wording, it is the context contract: what the model is allowed to assume, what the output must conform to, and what happens when a precondition is missing. The spec-versus-implementation split is a nice framing. I would add that the same discipline applies at retrieval time: if you feed an agent stale or loosely scoped context, a stronger model just fails more confidently. Treating retrieved context like an API response, with schema and validation, caught more of my failures than any model upgrade. How do you keep the spec from drifting out of date, the same way the Swagger you mentioned does?
Yeah, the aging is the part that actually bites in production. What has worked for me is treating the context contract like code: it lives in version control, has a schema, and a small eval set fails CI when the contract drifts from what the retrieval layer really returns. For the retrieved parts I pull from the source of truth at request time instead of caching a snapshot, plus a staleness flag so the model is told a field might be old rather than quietly guessing. It does not remove the aging, it just makes it visible and reviewable instead of silent.
The staleness flag is the part I'm stealing — telling the model "this field may be stale" instead of letting it guess silently is such a clean way to make aging visible. Lifting that.
The eval-set-fails-CI piece is where you're ahead of me. I have the manual version — a red-team checklist I run on AI output by hand (fabricated refs, tests that don't test, confidence on inferences vs. verified facts) — and a separate eval harness for structured outputs. What I haven't done is wire the contract check into CI as a gate that actually blocks the merge. That's the obvious next step and you just made it concrete.
That CI gate is where this starts paying off, I think. Manual red-team checklists are useful, but they usually run after someone already believes the change is fine.
The version that worked best for me was small: 20-40 frozen cases per contract, each tied to one failure mode, and the gate only blocks on regressions in those cases. Not a full benchmark, more like unit tests for the context boundary. When the spec changes intentionally, the test diff becomes the review artifact.
For structured outputs, the nice part is that you can separate schema validity from semantic claims. The model can be perfectly valid JSON and still invent the wrong thing, so I like keeping those as two different failure buckets.
Awesome! Much smaller and more shippable than the scored-dataset harness I'd been picturing, and the "test diff is the review artifact when the spec changes on purpose" part is the bit I was missing.
The schema-vs-semantic split is the exact design I'm building around. I've got an WIP project — zollama, a typed Ollama SDK — where Zod already owns the schema-validity bucket: shape, types, enums, rejected at the boundary. The semantic bucket — deterministic checks on top of valid JSON, for precisely your "perfect JSON, invented the wrong thing" case — is what I'm building now. Your two-bucket framing is basically the spec for it. Early days, though.
Nice, good luck with zollama. The semantic bucket is the harder half by a wide margin in my experience. Schema validation catches maybe a third of the real failures. The part that actually moves the needle is usually a small set of deterministic checks tied to specific fields, not a general semantic scorer, since a general scorer just becomes another model you have to trust. Curious whether you're going field-by-field or trying for something more general purpose.
Field-by-field, deterministic. Same reason you said. So the semantic bucket is concrete per-field checks: enum membership, referential checks (does this ID actually exist), cross-field consistency — pass/fail, no judgment model in the loop.
Where I'd draw the line: that's for the gate, the in-band blocking check. A graded LLM-judge still earns its place offline, over a dataset, where "roughly better or worse" is the actual question and a soft score is fine.
Exactly. I’d also keep the deterministic checks versioned with the schema, so a field change cannot quietly weaken the gate. Offline judges are most useful when their disagreement clusters feed back into new deterministic rules or explicit abstain paths.
Spot on — "a stronger model just fails more confidently" is the whole thing in one line.
Honest answer: the spec is a snapshot and it does age, same as the Swagger. I haven't solved that — living docs that regenerate on every change is the direction I pointed at, not something I run today. Not yet.
What I don't let drift is the technical-constraints layer, and it's basically your API-response idea moved to authoring time. The spec-writer doesn't trust any stored doc for verifiable facts — it re-inspects the live project on every spec: queries the actual schema, greps the actual code for the patterns in use, reads the actual package.json/tsconfig. So those constraints are re-derived from the source of truth each time, never cached. I don't ask a document what the schema is; I ask the schema.
The part that still drifts is intent/business logic — and that's where a stale spec is every bit as dangerous as your stale context. No clean answer yet. That's the honest gap.