192 skills have no frontmatter at all
An agent picks a skill by reading its description and deciding whether the request matches. No description, no selection — whatever else is in the file.
I drew 2,465 skills at random from a public registry and ran a reference-integrity and portability check over them. 192 of them (7.8%) ship without a single line of YAML frontmatter. No name, no description. They are published, downloadable, and cannot be chosen the way the specification says skills get chosen.
I should be careful about what that does and doesn't say. It is a statement about the files, not about any particular runtime — what a given agent does when the string is missing is something I did not measure, and inferring runtime behaviour from files is a mistake I made elsewhere in this same audit (below). What the count says is that in 192 published skills, the input that selection is supposed to run on is not there.
The rest of the breakdown
1,424 of 2,465 skills (57.8%) have at least one error-level finding. Percentages are the share of sampled skills with at least one finding of that kind, not the share of findings:
| skills | of sample | |
|---|---|---|
frontmatter name differs from the registry slug |
720 | 29.2% |
| a back-quoted path or link that does not resolve in the package | 445 | 18.1% |
| ships with no YAML frontmatter at all | 192 | 7.8% |
| an absolute path that resolves only on the author's machine | 94 | 3.8% |
malformed allowed-tools
|
58 | 2.4% |
| a provider API key read straight from the environment | 52 | 2.1% |
| an external CLI the skill never declares | 31 | 1.3% |
One caveat I want to put next to the top row rather than in a footnote: that 29.2% is not 29.2% broken. The registry deliberately keeps the routable slug separate from the stored display name, and says so in its own docs, so a real share of those are working as intended. What remains worth knowing is that on nearly a third of published skills, the name the agent registers under is not the name the registry shows you.
The sample is 2,465 drawn uniformly from 69,265 enumerated, seed 20260804. Those 69,265 are the most-recently-updated slice of the registry, not all of it, so every rate above is a rate inside that frame. The harness is published and the seed is fixed; re-run it and you should get the same draw.
The number that mattered was my own
Separately from the registry sample, I audited all 46 skills bundled inside openclaw/openclaw — a repository with 385,000 stars — at a pinned commit.
The first run produced 219 findings. After six precision fixes to my own tools, the same 46 skills produced 59. Every genuine defect appears in both runs. So the 160 that vanished were all mine.
Publishing before that work would have made the census wrong by nearly a factor of four.
What I had wrong
The big one, 139 of 197 path findings: I resolved references against the skill's own folder instead of the repository it lives in. A skill sitting inside a repo and pointing at packages/ or docs/ got reported as pointing at nothing.
The rest, listed plainly because the list is the useful part:
-
openai/gpt-5.4read as a file path, because.4looks like an extension. - An artifact excused on the line that creates it, but reported as a missing reference on the line that reads it back — 16 of 80.
-
/home/YOUR_USER/…treated as an author's real home directory. - Indented frontmatter parsing to nothing, so a skill with both required keys was reported as having neither.
- Sample output inside a fenced code block checked as if it were a live reference — 10.8% of path findings.
Number 5 I found by dogfooding: a skill written to document these linters could not pass them, because it quoted their own example output.
Number 1 has a detail I keep thinking about. The guard that was supposed to stop model identifiers being read as paths already existed, with a comment explaining exactly why it existed. The example in that comment ended in a letter. Anything ending in a version number sailed straight through. The comment was right and the code under it was not, and the comment is what stopped me looking.
A linter dies of false positives, so I treated the false positives as the bugs.
The finding that looks textbook and isn't
One bundled skill hardcodes /Users/steipete/openclaw — a maintainer's home directory, in a file shipped to everybody. That is the textbook portability finding, and it is not a bug. That skill is an explicit single-machine runbook for one canonical checkout, and the path is correct there. It was deliberately left out of the upstream report.
This is the part a percentage cannot carry. A linter produces candidates. Whether a candidate is a defect depends on what the document is for, and that judgement did not survive automation.
One report was closed as not planned, and closing it was right
I filed two issues upstream. One was rejected, and I agree with the rejection.
I had argued that a skill could not read the files bundled alongside it. It can: the prompt that hands the skill list to the model spells out the relative-path resolution rule. I had read one file, found no rule in it, and stopped.
That is worth more than the finding would have been. Choosing a resolution rule is choosing a definition of "broken," and that definition has to match runtime behaviour, not the filesystem. Mine was tuned to the permissive side afterwards, which is why every number above is a floor rather than a ceiling.
The other report held up: two bundled skills pointed at files deleted in an earlier refactor, so the reads return nothing and the work proceeds without the rules it was supposed to apply — silently, as usual. The fix is merged and the issue is closed as completed.
What I did not publish, and why
- The corpus. 2,465 third-party skill files are other people's work. Re-fetch them with the harness instead.
- The 94 author paths. Reported as a count, never as a list, with home-directory segments redacted in the example output. Nobody was contacted. Mass-filing issues against individuals, on the strength of a rule with a known false-positive history, would be the wrong move — see the entire middle of this post.
Reproducing
The harness, the seed, and the results are in agent-skill-audit. Data is CC0, harness is MIT.
The rates above are tied to the tool versions used on the run: tenken 0.2.0, skills-lint 0.7.1, carrylint 0.2.2, reflint 0.8.3. All of them have moved since — precision fixes in some places, new rules in others — so a re-run today would not reproduce the table exactly. That is the reason to publish the harness and not only the numbers.
Corrections to the numbers themselves are welcome, and I would rather have them in public than not.
Top comments (2)
This is a useful audit because selection is the real contract for skills. A skill that validates on paper but cannot be chosen correctly by the agent is effectively broken at runtime.
Agreed on the framing — selection is the contract, and the description is the only input it runs on.
The one place I'd hold back is "broken at runtime." That's the exact inference I got wrong in this audit: I filed an upstream issue claiming a skill couldn't read its bundled files, on the strength of having read the files, and it was closed as not planned because the resolution rule lives in the prompt and not on disk. So what I can say about the 192 is that the input selection runs on is missing from the file. What each runtime does with a missing description — fall back to the slug, refuse to load, load it anyway — I didn't measure.
The row that bothers me more, for the same reason, is the 29.2%: those skills all validate. Nothing in the file looks wrong. And the name the agent registers under still isn't the name the registry shows you. It only surfaces at selection time.