The new LLM shipped last week, promising superior intelligence. You pointed your agents at it, and the results got worse.
Your engineers shrugged: the hype outran reality again. Here is the alternative: the model is fine. It is reading instructions your team wrote a year ago to keep a weaker model out of trouble, and unlike that weaker model, it found every one of them and obeyed.
I call this instruction debt: the cost of instructions you wrote down and never retired. The decision in front of you is whether your standing instructions, the full set of system prompts, agent rules files, skills, and guardrails your agents read, get treated as versioned infrastructure with a deprecation path, or stay write-once files that quietly tax every model upgrade you will ever make. With frontier release gaps compressed to around 40 days by Theo Browne's count, "wait and see" means paying that tax every six weeks.
The Better the Model, the Worse Your Stale Instructions
Every's review of GPT-5.6, (by the way I love @every's content) ressonate that pattern. They initially found GPT-5.6-Sol worse than its predecessor at customer support work, until they deleted defensive rules written for older models. The conclusion: "Sol had found the instructions and followed them; the instructions were the problem."
Frontier Models, are better than their predecessors at the exact thing that makes stale instructions dangerous: it does the reading. It searches files, reads standing instructions, and checks connected tools before asking for help. A weaker model that ignored your rules file was accidentally protected from that file's staleness. A stronger one is not.
And this is not one reviewer's experience. Anthropic's own migration guidance for its newest model says it plainly: skills developed for prior models "are often too prescriptive" and "can degrade output quality," and capability improvements "are a good prompt to re-evaluate which instructions, tools, and guardrails are still needed." When a vendor states in an official document that your existing instructions can make its new model worse, the debate about whether this problem exists is over. That is the inversion: staleness used to be free because weak models either needed or ignored your rules. It is now different
Dead Weight and Active Harm
Audit for two failure modes.
Dead weight is a instruction the new model no longer needs: "think step by step," enumerated lists of behaviors, role-play framing like "you are a senior engineer." Anthropic now says you can steer most behaviors with a brief instruction; the enumeration itself is a leftover from weaker models, wasting context and burying the rules that still matter.
Active harm is instruction the new model obeys to its own detriment. The defensive rules that dragged GPT-5.6-Sol down for Every are one case. Theo Browne supplied another from his own repository: T3 Code's agent rules file sat unchanged for two months, still calling the shipped project "a very early WIP" and still directing "sweeping changes that improve long-term maintainability," an open invitation for a capable model to rewrite architecture nobody asked it to touch. No one noticed, because nothing broke.
The Industry Is Carrying This Debt at Scale
The data point that changed my mind about the scale came from Drew Breunig's analysis of prompt debt: Datadog's State of AI Engineering report found the most-used model in observed traffic in March 2026 was GPT-4o, a model nearly two years old by then. Breunig adds that several large inference providers privately put GPT-4o-vintage models above half of all calls, a secondhand figure, but directionally consistent. A Berkeley-led study he cites found enterprises stay pinned to older models because newer ones break their existing agents. Half an industry frozen on old models is not nostalgia. It is aggregate instruction debt made visible: the upgrade breaks the instruction base, so the upgrade waits.
Breunig also documents why the debt keeps growing. Most standing instructions exist to fight a specific model's training: ChatGPT's image pipeline told its model eight times not to reply when an image was returned; Claude Code told Opus seven times to parallelize tool calls. Each repetition is a dated workaround for one model generation, waiting to confuse the next. And holding still is not an escape. Goedecke again: a delicately prompted harness built around an old model will always lose to a bare-bones harness built around the current one.
The Maintenance Bill on My Own Advice
A fair objection: I've been telling you for the last 2 years about the importance of writing these files. The opening episode's playbook argued for guardrails, curated context, and reusable instructions, and later episodes doubled down with skills and context packs. Some people advocate that we should let the tools the more vanilla as possible and have nothing but the original harness provided by the vendors. I stand by all of it, specially considering the harness can help you you to save cost, and steer smaller and less capable models to perform better, so, my point is:
Instruction debt is not an argument against writing instructions; it is the maintenance bill on that advice.
It completes a pattern. Addy Osmani gave us intent debt, the cost of intent you never wrote down, and comprehension debt, the gap between the code that exists and the code you understand. Instruction debt is the third member: intent you never wrote, code you never read, instructions you never retired.
The Playbook: Give Every Instruction a Deprecation Path
1. Inventory your instruction surface. You cannot retire what you cannot see. List every place standing instructions live: system prompts, agent rules files, skills, MCP tool descriptions, prompt templates in environment variables. TechDebt.works lists the warning signs: prompts living in Slack messages, reliability described as "works most of the time," one person who owns all the prompts. The concrete handle is a registry, even a single spreadsheet: file, owner, model it was written for, last reviewed. Pitfall: auditing the main system prompt while per-repo rules files sprawl unowned. Signal: every instruction has a named owner and a "written for" model version.
2. Version prompts like code. Prompts belong in git, next to the service, changed through pull requests. Add one question to the PR template: which model weakness does this instruction compensate for? Pitfall: treating version control as an archive nobody reads. Signal: prompt changes get the same review scrutiny as code changes.
3. Eval-gate model upgrades like dependency updates. Before switching models, run your evaluation suite against the new one, compare side by side, and hold the upgrade if a critical flow degrades. Tooling exists: promptfoo for CI-style prompt evals, and research tools like RETAIN built for regression-testing prompt migration between models. Pitfall: having no evals, which turns upgrade day into debugging in production. If you cannot score outputs automatically, fix that first. Signal: a model upgrade is a branch plus an eval report, not a leap of faith.
4. Audit and prune on every upgrade. Two questions per instruction: does the new model still need this, and does the new model now obey this to our detriment? Strip old-generation behavioral steering immediately. Keep rules files to concrete project facts (where files live, the test framework, the never-touch list), never behavior coaching. Pitfall: pruning only when something visibly breaks; this debt does not break things visibly. Signal: the instruction base shrinks or holds flat across upgrades while eval scores rise.
What This Costs, and the Kill Metric
The inventory and the pruning habit cost days, not quarters. The real investment is the eval suite, the same verification infrastructure this series keeps arriving at from different directions. Fund first: the instruction registry and evals for one critical agent workflow. Fund later: prompt-compilation tooling. The kill metric fits in one experiment: on your next model upgrade, run the new model against your eval set twice, once with the current instruction base and once with a pruned one. If pruned wins, you have measured your instruction debt directly, in eval points. If it does not, your instruction base is healthy and the audit cost you a day.
Start, Stop, Continue
Executives
Start: requiring model upgrades to pass an eval gate the way dependency updates pass tests; asking one question in your next AI review, who owns the standing instructions and when was one last pruned.
Stop: booking "the new model is disappointing" as vendor failure before anyone has audited the instructions it was given; funding prompt writing with no retirement process attached.
Continue: incident-driven guardrails, with the added rule that every new guardrail names the model weakness it compensates for.
Engineers
Start: adding a "written for model X" header to every rules file you own; deleting old-generation steering ("think step by step," role-play framing) this week.
Stop: letting agents generate their own rules files without review; carrying any instruction you cannot explain the reason for.
Continue: writing skills and context packs. The answer to instruction debt is a lifecycle, not abstinence.
Strategic Takeaway
A versioned, eval-gated, pruned instruction base compounds: every model upgrade lands at close to full value in week one, and each audit makes the next one cheaper. A hand-tended one plateaus, then turns against you: every defensive fix deepens the lock-in, until you are one of the organizations still running two-year-old models because the upgrade breaks your agents. The gap is brutal because it is invisible: the competitor gets the new model's full capability immediately, while you spend a quarter concluding the hype outran reality.
So run the test on yourself. Take your most important agent workflow and ask: if a better model shipped tomorrow, would that workflow get better, or would the model read your instructions first and get worse? If you cannot answer, that is the debt.
I expect pushback on this one, especially from teams whose prompt libraries feel like hard-won assets. Mine currently is.
Top comments (0)