The latest articles on DEV Community by Andrew Kew (@thegatewayguy). https://dev.to/thegatewayguy https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F3895707%2Fc20fb912-7c14-4930-97fc-5894d1c3e4c5.png https://dev.to/thegatewayguy en Andrew Kew Fri, 24 Apr 2026 14:51:12 +0000 https://dev.to/thegatewayguy/harness-bugs-not-model-bugs-1f4e https://dev.to/thegatewayguy/harness-bugs-not-model-bugs-1f4e <p>For six weeks, developers have been complaining that Claude got worse. You've seen the posts — "Claude Code is flaky", the AI-shrinkflation discourse.</p> <p>Yesterday, Anthropic shipped a <a href="https://www.anthropic.com/engineering/april-23-postmortem" rel="noopener noreferrer">postmortem</a>. Three unrelated bugs, all in the Claude Code <em>harness</em>. The model weights and the API were never touched.</p> <p>That distinction is the whole point.</p> <h2> What actually broke </h2> <ol> <li> <strong>Default reasoning effort got dialled down.</strong> A UX fix dropped Claude Code's default from "high" to "medium" in early March. Users noticed it felt dumber. Reverted April 7.</li> <li> <strong>A caching optimisation dropped prior reasoning every turn.</strong> Supposed to clear stale thinking once per idle session; a bug made it fire on every turn. Claude kept executing without memory of why. Surfaced as forgetfulness and repetition. Fixed April 10.</li> <li> <strong>A verbosity system prompt hurt coding quality.</strong> "Keep responses under 100 words." Internal evals missed a 3% regression on code. Caught by broader ablations. Reverted April 20.</li> </ol> <p>None of this was a model change. The weights didn't move. The API was never in scope.</p> <h2> The lesson </h2> <p><strong>The model is not the product.</strong> What your users experience is <code>model + harness + system prompt + tool wiring + context management + caching</code>. Each layer has its own bugs. When someone says "Claude got worse," the weights are usually the last thing that changed.</p> <p><strong>API-layer products were unaffected.</strong> If you're building directly against the Messages API, none of these bugs touched you. This is why "am I on Claude Code, or am I on the raw API?" matters.</p> <p><strong>"Eval passed" ≠ "no regression."</strong> The verbosity prompt passed Anthropic's initial evals. Only a broader ablation — removing lines one at a time — caught the 3% drop. Fixed eval suites miss behavioural drift; ablations catch it.</p> <h2> What to actually do </h2> <ul> <li> <strong>On Claude Code?</strong> Update to v2.1.116+. You're already through it. Usage limits got reset as an apology.</li> <li> <strong>On the API directly?</strong> Nothing to do. Stay on whatever model you were on.</li> <li> <strong>Shipping your own harness on top of a frontier model?</strong> Read the postmortem twice, then audit your prompt + caching + context-management pipeline for the same silent-failure modes. The bugs Anthropic described are exactly the ones every harness reinvents.</li> </ul> <p><em>The meta-lesson is boring and important: most of the quality variance lives between "the model" and "the thing your user sees." Ship good harnesses.</em></p> <p><em>✏️ Drafted with KewBot (AI), edited and approved by Drew.</em></p> ai claude anthropic postmortem