Anthropic released Claude Opus 5.5 at 16:31 UTC on 22 September. By 07:08 UTC the next morning, Driftproof had six sealed receipts answering the question every skill author has on a release day: does my skill still help on the new model?
What we ran
Three public skills from addyosmani/agent-skills (code-review-and-quality, git-workflow-and-versioning, documentation-and-adrs), one case each, the same cases as our Report 009. Each skill ran with and without its SKILL.md, on Opus 5.5 and on a fresh Opus 5 arm beside it. Same runner, same judge, same grading template, and the same SKILL.md bytes and suites, pinned by hash in every receipt.
The first surprise: the old Claude Code could not call the new model
Report 009 ran on Claude Code 2.1.272. That version stopped at its first call to Opus 5.5 with an error asking for 2.1.280 or newer. So on release day, a model upgrade was also a runtime upgrade. We ran both models on 2.1.280, which keeps the runtime fixed when comparing models, and published Opus 5 on the old and new versions as a separate table.
What it found
With the skill, the two models did not separate on any of the three skills: +0.009, -0.023 and -0.143 on a 0 to 1 scale. On two of the three, these draws could not have told either way. For git-workflow-and-versioning, the runner puts the draws needed at 16 per arm, against the 3 we drew. For documentation-and-adrs, the spread between draws is wide enough that no number of draws at that spread would resolve it. The report says which is which, because "no separation" and "nothing changed" are different claims.
What moved instead
Without the skill, Opus 5.5 scored well below Opus 5 on two tasks. Its commit messages dropped the conventional fix: prefix (0.300 against 0.765), and in two of its four code reviews, some findings sat under category headings with no severity label (0.680 against 0.862). So on those two skills, the skill's lift grew on the new model; on documentation-and-adrs it shrank. With one case per skill, that is an observation to test, not a finding.
Is the judge biased toward its own model?
Every draw was judged by Opus 5, the same family as one of the two models. So we re-graded every answer with Opus 5.5 as the judge, published as Amendment 1 on the report. No verdict changed. 50 of 51 answers stayed on the same side of the pass line, with a mean absolute difference of 0.029 from the original grades, against 0.013 when Opus 5 graded the same answers a second time. On average Opus 5.5 scored its own model's answers 0.020 lower than the control did, and Opus 5's 0.012 lower; skill by skill, the differences are small and point both ways.
What it cost
204 model calls, 0.00 dollars metered (Claude Code on a subscription), about 10 dollars at API prices.
What this is not
Three cases and few draws is a release-day check, not a verdict on Opus 5.5. Every figure in the report is read from a receipt you can validate yourself with npx driftproof validate <file>.
Full report, receipts and raw evidence: https://driftproofhq.com/reports/011/
Disclosure: I build and maintain Driftproof (open source, Apache 2.0).
Top comments (0)