DEV Community

Cover image for Nearly one in ten published Claude Code skills does not load
Kynth
Kynth

Posted on • Originally published at dev.to

Nearly one in ten published Claude Code skills does not load

Cross-posted from The State of Agent Tooling. Dataset: DOI 10.5281/zenodo.21936490, CC BY 4.0.

43,199 of 445,348 published Claude Code artefacts fail a structural check and will not load as published. That is 9.70% of every skill, subagent, plugin and marketplace we could find on GitHub and skills.sh, read from source on 2026-08-14.

Nobody has published this number before, so there is nothing to compare it against yet. That is the reason to publish it.

What was measured

Every artefact was fetched and checked for one thing: whether an agent runtime could register it. Not whether it is good, popular, or maintained.

Kind Listings Fail Rate
Skills 347,382 27,398 7.89%
Subagents 71,430 15,669 21.9%
Plugins 20,212 33 0.16%
Marketplaces 6,324 99 1.57%

Subagents are the broken half

The aggregate hides the finding. Subagents fail at 2.8x the rate of skills, and the reason is structural rather than cultural.

A subagent's name and description are Required: Yes in Anthropic's frontmatter reference. A skill's are not: a skill falls back to its directory name, and to the first paragraph of its body. The same careless file is a working skill and a dead subagent.

4,119 subagents have no name. 1,799 have no description.

Almost all of it is one defect

38,183 of the 43,199 failures — 88.4% — are a YAML block at the top of a file that does not parse.

This is the cheapest class of bug in software, and it is sitting in front of the most-copied artefacts in a fast-growing ecosystem, because nothing in the publishing path checks it. There is no registry gate, no CI convention, and no error surfaced to the author. The file is committed, the repository is starred, and the artefact silently never loads.

Cause Listings
Frontmatter does not parse 38,183
Subagent has no name 4,119
Subagent has no description 1,799
Manifest is not valid JSON 128
Marketplace lists no plugins 99

Four things this number is not

Not a count of distinct artefacts. It counts listings. The 445,348 come from 14,233 distinct repositories, an average of 31.3 apiece, because this ecosystem is overwhelmingly forked and vendored and the same file is counted once per repository carrying it. The failure rate is the citable figure.

Not an execution result. Nothing was run. Every check is static. A listing that passes can still be wrong, useless or dangerous. Passing means the file parses into something a runtime could register: a floor, not a verdict.

Not a quality judgement. No model scored anything and no human curated the list. Every fatal cause above is a specific, checkable, arguable defect, and the row-level data is published so the argument can be had against the evidence.

Not complete. Discovery is bounded by GitHub's search API, which is rate-limited and does not return everything. What discovery truncated is logged rather than hidden.

A correction, published with the first edition

Preparing this census turned up a defect in the index that produced it.

The index carried a missing-references flag — a skill naming a bundled file that is not in its repository — and it fired on 139,623 listings, 40.2% of every skill. It was wrong roughly seven times in eight, from three bugs in one regular expression:

  1. The extension alternation put js ahead of json, and JavaScript alternation is first-match. So every package.json was captured as a missing package.js, and every res.json(data) inside a fenced code block became a missing file called res.js. 12,178 and 6,946 listings respectively.
  2. Nothing scoped the match to the skill's own bundle. The most-reported "missing files" in the entire index were the framework names Next.js (21,014) and Node.js (19,383), the skill's own SKILL.md (27,002), and CLAUDE.md (11,371) — a file that lives in the user's project and never in the bundle.
  3. A reference written ./run.sh could never resolve against the repository tree, so 1,498 listings were reported missing a file that was sitting right there.

Corrected, 18,657 skills (5.37%) name a bundled file that is genuinely absent.

The flag is a score deduction and was never a load verdict, so the 9.70% headline is unaffected — it comes from the fatal causes above and nothing else.

This is in the report rather than in a changelog because an index that publishes other people's broken metadata has to publish its own.

The data

Published in full under CC BY 4.0, including every failing row, so the headline can be recomputed by anybody who wants to check it.

  • census.json — every aggregate, plus the exact rule that decides a failure
  • broken-listings.csv — 43,199 rows, one per failing listing, with its repository, path and cause
  • listings.csv.gz — the full population, 445,348 rows
  • The measurement vocabulary — load rate, drift, shipping status and skill decay, each defined as a formula over a named population
  • The method, written down before the numbers

Cite as: Kynth Studios (2026). The State of Agent Tooling 2026.1. Zenodo. https://doi.org/10.5281/zenodo.21936490

Top comments (0)