skills.sh ranks agent skills by installs. Installs tell you a skill is popular. They don't tell you whether it changes what the agent actually writes.
So we took the top 20 skills from the skills.sh snapshot of 17 Oct and ran each one twice on the same tasks: once with the skill loaded, once without. Two answering models, Claude Sonnet 5.5 and Claude Opus 5.5, with Claude Haiku 5.5 as the judge. 40 results in all.
The scoreboard
| Sonnet 5.5 | Opus 5.5 | |
|---|---|---|
| Clearly helped | 12 | 12 |
| Clearly hurt | 4 | 2 |
| Too few answers to tell | 4 | 6 |
11 of the 20 skills clearly helped on both models.
What hurt
- find-skills (vercel-labs/skills), number one by installs: clearly hurt on both models. Both times it was the same task, a search request, where answers scored about 0.75 with the skill against 0.92 to 0.95 without. find-skills is a router that points the agent at other skills. We tested its written answer, not the skills it hands off to.
- reddit-automation (flowkit-labs/skills): clearly hurt on both models, on the ranking task, and on the reply task with Sonnet.
- grill-with-docs and hyperframes: clearly hurt on Sonnet only.
What helped most
- hyperframes-registry (heygen-com/hyperframes): 0.43 without to 0.98 with on Sonnet, 0.62 to 0.97 on Opus.
- prototype (mattpocock/skills): 0.36 to 0.93 on Opus.
- triage (mattpocock/skills): a perfect 1.00 with the skill on both models, against about 0.7 without.
The ones we couldn't call
10 of the 40 results came back "too few answers to tell". grill-me, tdd and Anthropic's own frontend-design landed there on both models.
That is not "no effect". It means answers varied too much from one draw to the next to say either way, even after up to 10 draws per arm. We'd rather print that than a number that flips on a re-run.
How it works
- For each skill, three text tasks written from the skill's own description and README, fixed before any run.
- Each task is answered with and without the skill, 3 to 10 times per arm, and each answer is scored 5 times.
- "Clearly helped" means that on at least one task the two score ranges (mean plus or minus one standard deviation) don't overlap and the gap is at least 0.05, and no task went clearly the other way. A clear drop on any task wins over a gain elsewhere.
Limits
- Three text-only tasks per skill, written by us, not by the authors. No tool calls, generated media or deployed services were tested.
- One judge model.
- Popularity here is reported installs, not quality.
If you think a task is unfair or a result is wrong, every skill page has a link to ask for a re-run, and every receipt, task suite and retained answer is public.
- Full table, one page per skill: https://driftproofhq.com/skills/
- Report 015: https://driftproofhq.com/reports/015/
Disclosure: I maintain Driftproof, the open-source tool that produced these results.
Top comments (0)