DEV Community: Anisa

Your model didn't get worse — the wrapper around it did (and you can control that)

Anisa — Sat, 11 Jul 2026 00:18:24 +0000

My GPT got dumber after the update" gets blamed on the model regressing, or on you prompting worse. Both are unfalsifiable, and both send you to fix the wrong layer. The layer that actually moved is the one you can pin.

"The model" is two layers. The weights — the trained network, slow to change, and when they do change it's announced under a new name. And the wrapper — the router that picks which model answers, the system prompt, the default reasoning effort, verbosity caps. The wrapper changes silently, on its own schedule, per product. It's almost always what moved under you.

So stop re-tuning prompts to chase it. Pin the wrapper:

Force the route. Don't leave it on Auto — set Thinking (or say "think hard") so the router can't quietly demote your prompt to a faster, weaker model. OpenAI's own GPT-5 launch post describes exactly this router (it scores prompts "simple" vs hard); after the backlash they put the picker back (Auto/Fast/Thinking — TechCrunch, Aug 2025).
Pin the version. If you build on a model, call its exact versioned ID via the API. A model ID's weights don't change — new versions ship under new IDs — so router and system-prompt churn can't reach you.
Own the harness. Running agents? Set the system prompt, reasoning effort, and verbosity yourself instead of inheriting a default. Anthropic's own April 23 post-mortem is the proof: six weeks of "Claude Code got worse" traced to three wrapper changes (a reasoning-effort downgrade, a reasoning-history bug, a verbosity cap their ablations put at ~3% quality) — API weights never touched.

A real weights change — a new model — will still move behavior. But that's announced, and you choose when to adopt it. The silent stuff is all wrapper, and the wrapper is the part you can pin.

Sources: OpenAI GPT-5 launch (router + "think hard"); TechCrunch, Aug 2025 (model picker reinstated); Anthropic April 23 post-mortem (anthropic.com/engineering/april-23-postmortem); InfoQ and VentureBeat (corroboration); Claude platform docs (model IDs & versions).

Your agent stops obeying your rules halfway through the session. Here's the structural reason — and the fix.

Anisa — Tue, 30 Jun 2026 18:11:42 +0000

You've seen this one. You give your coding agent a clear rule: "Don't edit files I didn't ask you to touch." It behaves. Twenty minutes and a dozen tool calls later, it edits the three files you never mentioned, and you find out when the test suite goes red.
The rule never left the prompt. The agent just stopped following it. No error, no warning — it quietly aged out.
This isn't a smarts problem and it isn't bad luck. It's structural, it's measurable, and once you see the mechanism you can stop it for good. Here's the failure, the fixes that don't work, the real cause, and the rebuild.
The failing example
Say your CLAUDE.md (or .cursorrules — same story) opens with:

Never edit files outside the src/payments/ directory without asking first.

Early in the session the agent is perfect. It asks before it strays. Then you send it on a multi-step task — read these files, run the tests, fix the failures, refactor the helper. Each step piles tool calls and output into the context. Around the fifteenth tool call it edits src/auth/session.ts to "fix an import" — a file you never put in scope — and keeps going like nothing happened.
A developer on the Cursor forum said it cleaner than I can: "If we can't trust it to use our rules, how can we rely on any output?"
The fixes that don't work
Most people reach for one of three, and all three fail for the same reason.

Write the rule louder. Bold it. ALL CAPS. Add "IMPORTANT" and three exclamation points. This feels like it should help. It doesn't — stronger wording is subject to the same attention dynamics as the original line (codeongrass, 2026).
Repeat the rule five times in the system prompt. Now you've spent more of the prompt and bought a few more tool calls of compliance before the same decay sets in.
Buy a bigger context window. The most expensive non-fix. Chroma's Context Rot study (July 14, 2025) tested 18 frontier models — GPT-4.1, Claude 4, Gemini 2.5, Qwen3 — and every one got less accurate as the input grew, with the steepest drop between 100K and 500K tokens. A million-token window doesn't fix this; it just delays the point where you notice. The structural cause The model attends to every token in context, but not equally. Attention is shaped by recency and density. Your rule was written once, at the very top. Since then the context has filled with tool calls, file dumps, and the agent's own reasoning — thousands of tokens, all newer than the rule. Past roughly fifteen tool calls, the system prompt is a small, old island in a sea of recent tokens. Its effective weight drops and the constraint stops steering. One engineer documented the threshold directly: Claude agents reliably break system-prompt constraints past about fifteen tool calls, from attention dilution at high context depth — format requirements ignored, approval gates skipped, autonomy rules bypassed, all with no error (codeongrass, 2026). It scales past coding, too. One widely-cited 2025 analysis attributed nearly 65% of enterprise agent failures to context drift and memory loss during multi-step reasoning — not to the model being incapable. The failures are silent: the agent produces confident, plausible output while running on a rule it has effectively forgotten. So the rule isn't being disobeyed. It's being out-voted by everything you've added since. The rebuild If the problem is that the rule is old and far away, the fix is to make it recent and close — at the exact moment it's needed. Stop treating the rule as something the model must remember. Make it a step the model must perform, re-stated right before the action that tends to break it.
Move the constraint to the point of action. Before: # CLAUDE.md Never edit files outside src/payments/ without asking. After: # CLAUDE.md Before EVERY file edit, output this gate and stop if it fails:
- Files in scope:
- File I'm about to edit:
- In scope? If no, ask before proceeding. The rule is now the most recent thing in context at the moment of the decision, not the oldest.
Make the good behavior a required action, not a forbidden one. "Don't edit out-of-scope files" is a thing not to do — easy to drop. "State the file and confirm it's in scope before editing" is a thing to do — it leaves an artifact you can see in the transcript. Forbidding decays; requiring leaves a trace.
Re-inject at the boundary. If your harness allows it, repeat the one rule that matters in the last user turn before a risky step. The last message gets the highest effective attention — use that slot for the constraint you can't afford to lose. It's the same reason "put the important instruction at the end" keeps working. The pattern underneath all three: don't ask the model to hold the rule across a growing context. Re-present the rule at the moment of the decision, every time. Same model, different shape — the rule holds at tool call fifty the same as tool call five. The 4-point checklist

Is every must-hold rule a step it performs, not a line it remembers? Stated once at the top = it will age out.
Does the rule live next to the action it governs? Distance from the decision equals decay.
Is compliance visible? A rule the agent has to answer for in the transcript beats a rule it silently should follow.
Did you test past tool call fifteen? Rule-drop mostly shows up deep in a session. A five-call demo proves nothing.

One more thing
If you've got a prompt that behaves at the start and drifts by the end, that's the kind of failure I rebuild. I send one short teardown a week — a real flaky prompt, the structural reason it broke, and the rebuild that holds:dm me.

Why your AI coding agent ships confident, slightly-wrong code (and why rewording the prompt never fixes it)

Anisa — Sun, 28 Jun 2026 21:25:45 +0000

Your AI coding agent writes something that looks right. It compiles in your head. Then you notice it called user.getProfileById() — a method that doesn't exist anywhere in your codebase.

You didn't ask it to make that up. It invented it confidently, in the middle of otherwise-fine code. And that's the worst kind of wrong: not obviously broken, just quietly incorrect in a way you have to catch.

If you've run Claude Code, Cursor, or any agent on a real repo, you know this one. Here's why it happens — and why the obvious fix doesn't work.

The fix everyone tries first (and why it fails)

You reword the prompt.

You add "Don't make up functions." It behaves… for one file. Then it does it again. So you add "Only use methods that exist in the provided code." Better for a bit. Then two more sentences — and now your prompt is fifteen rules long and it still invents a method the moment the task gets complex.

Here's the part nobody tells you: rewording treats a structural problem as a vocabulary problem.

A prompt isn't a contract the model reads once and obeys. It's something the model has to hold in working memory while it reasons about your actual task. A flat list of fifteen rules is unholdable. As the work gets harder, the model spends its attention on the code and quietly drops whichever rule wasn't front-of-mind. "Don't invent methods" is usually rule #11. Under load, it falls out.

You can't out-word that. A sixteenth rule just gives it one more thing to drop.

The actual cause: shape, not wording

The agent invents a method because nothing in the prompt's structure requires it to check. You told it what not to do. You never changed what it actually does, step by step.

So stop forbidding the bad thing. Remove the opportunity for it.

Instead of a rule it has to remember, make grounding a required step it has to perform.

Before — a pile of rules:You are an expert engineer. Write clean code.
Follow our conventions. Don't make up functions.
Only use methods that exist. Handle errors.
Don't leave TODOs. Write tests. ...

After — an ordered procedure:1. First, list every method or API you plan to call.
For each, quote the file and line where it's defined.

If you can't find where something is defined, stop and tell me — do not write code that uses it.
Only once that list is complete, write the code, using only the methods you grounded in step 1. Same model. No new rule about "don't hallucinate." But now the agent can't invent getProfileById() — the structure makes it find the real method before it's allowed to write the call. And if the method genuinely doesn't exist, step 2 forces it to say so instead of guessing.

Why this holds when rewording didn't

Here's the part that matters.

When you reworded, you were patching one instance — this invented method, in this file. The next one slips through in a slightly different shape.

When you change the structure, you close the whole class: every invented method, in every file, because the model now can't reach the "write the call" step without grounding it first. You didn't fix a symptom. You removed the cause.

That's the real test of a prompt fix: does the same mistake come back wearing different words? If it does, you patched wording. If it doesn't, you fixed structure.

(One aside, because it's the same principle: this is also why cramming prompts into rigid JSON schemas often lowers quality. When you make a model fill fields, it stops reasoning and starts form-filling. Reason in prose and an ordered procedure; save strict JSON for the final hand-off, after the thinking is done.)

A checklist for your next prompt

Forbid less, structure more. Every "don't do X" is a rule the model can drop. Turn it into a step it has to perform.
Make verification a step, not a hope. "Quote where this is defined" beats "use real methods."
Front-load grounding. Read before write — make the model gather the real facts before it generates.
When it can't ground something, it should stop and ask. A question is cheaper than a confident wrong answer.

Do those four and most of the "confidently wrong" problem disappears — without a single extra rule.

If you want this in a reusable form, I packaged the checklist plus a tiny self-updating skill (it keeps these rules current as you add your own) as a free download — [link]. Take it; it's genuinely useful on its own.

And if you've got a prompt that keeps doing this and you'd rather just hand it off — rebuilding flaky prompts into structures that hold is the thing I do. I'm ready to help