We downloaded the SKILL.md files of nine public agent skills and ran our open-source linter over them. Seven passed. One warned. One failed.
This is not a benchmark of those projects — it is a check of one thing: whether a skill is structurally usable by an agent that has to decide when to load it.
What we ran
skilllint is a small local linter for agent skills. Eleven rules, no network, no telemetry. It reads a SKILL.md, parses the frontmatter, and answers three questions:
- Can an agent tell when to use this skill? (name, description, trigger phrasing)
- Is the package self-consistent? (dead markers, references to files that do not exist, oversized files)
- Does it behave like an instruction, or like a trap? (unlimited autonomy, embedded secrets, prompt-injection patterns)
Every verdict comes with the SHA-256 of the exact bytes checked, so the result is reproducible.
The skills we checked
| Skill | Verdict | Score | Findings |
|---|---|---|---|
anthropics/skills → claude-api
|
WARN | 78 | 5 |
anthropics/skills → doc-coauthoring
|
PASS | 90 | 2 |
anthropics/skills → pdf
|
PASS | 95 | 1 |
upstash/ratelimit-js |
PASS | 95 | 1 |
upstash/qstash-js |
PASS | 95 | 1 |
upstash/workflow-js |
PASS | 95 | 1 |
better-auth/better-icons |
PASS | 100 | 0 |
xmtplabs/xmtp-agent-examples |
FAIL | 50 | 4 |
kid-sid/claude-spellbook (go) |
PASS | 95 | 1 |
16 findings in total. Nine of ten planned downloads succeeded; the tenth path returned 404 and we dropped it instead of quietly replacing it with something easier.
What the WARN found
anthropics/skills/claude-api is a large, careful document — and the linter still flagged five things:
- No trigger phrase in the description. The description says what the skill is, but not when to use it. An agent reading only the description has to guess.
-
A dead marker (
deprecated) left in the text. -
A referenced file that was not in the download (
agent-openai.py). - 105 KB in a single file — over our 60 KB soft limit, and heavy to inject into context.
- Unbounded autonomy wording: the text asks the agent to proceed "without asking". For a skill that touches an API with keys, that is the line worth reviewing before loading it.
None of that makes the skill bad. It makes it unverified.
What the FAIL found
xmtplabs/xmtp-agent-examples has no frontmatter at all. No name, no description.
That is not a style complaint. A skill without a description cannot advertise its trigger, so an agent has no reliable way to know the skill exists or when to reach for it. The package may still be excellent as documentation — as an agent skill it is unreachable.
Honest caveats
-
We fetched only
SKILL.md, not the whole package. Every "referenced file not found" finding (5 of the 16) is therefore not conclusive — the file may exist in the repo and simply was not in our download. We are reporting the raw output rather than editing it into a nicer story. - Eleven rules is an opinionated minimum, not a security assessment. A PASS means "structure is sound", never "safe".
- These are static checks. They cannot tell you whether the instructions work; only execution can.
Why this matters more every week
Skills are executable instructions that get injected into an agent's context. Two failure modes dominate:
- Silent non-loading. No description, no trigger → the skill simply never fires. Nothing errors. You just never get the capability.
- Silent over-loading. The description fires on anything → the agent drags a 105 KB file into every relevant context and spends your budget on it.
Both are invisible until you measure them. That is the same reason we built the rest of the stack: a signed receipt for the artifact, a mandate check before the action, and an actual-spend ledger afterwards.
Reproduce it
skilllint is MIT-licensed and runs locally:
python3 -m skilllint path/to/skill # verdict + sha256 + findings
Or try it on your own skill before publishing it — 20 checks a day, free, no account, no upload:
Top comments (1)
The per-rule breakdown is what makes this checkable, so I pulled the same file. Two of the five findings look like rule artifacts.
The claude-api description carries a block starting "TRIGGER" that enumerates when to read it, and a second starting "SKIP" with the overriding conditions. The rule looks like it is matching surface phrasing rather than whether an agent could route.
Your caveat covers the agent-openai.py finding, but it is stronger than "not conclusive": that string sits inside a sentence listing filename patterns to look for in the user's project, as a marker that the file belongs to another provider. There is no such bundled file to miss, so a full-package download would not clear it either. Our pull is 105,146 bytes, which lines up with your 105 KB, so we are on the same version.
That puts 2 of the 5 findings behind the 78 on rule behaviour. Worth printing which rule fired per finding, since the score is what people will quote.