Only 17.7% of the catalog is popular enough to be graded, 1 in 32 graded skills is unsafe, and the risk lives in the long tail — plus a new agent-native attack surface.
We Security-Graded 117,854 AI Agent Skills. Here's What We Found. | Agent Skills Hub
Only 17.7% are popular enough to be graded. Among graded skills, 1 in 32 is unsafe. The risk lives in the long tail.
agentskillshub.top
The uncomfortable part isn't the skills that are unsafe. It's how few have been checked at all.
Installing an AI agent skill or MCP server means handing untrusted code your shell, your environment variables, and increasingly your agent's own config and memory. Discovery is easy — there are tens of thousands to pick from. Knowing whether the one you found is safe to run is not.
So we scanned the whole catalog. Here's the honest picture.
📄 This is a cross-post. Canonical version (with charts): agentskillshub.top/blog/securing-117k-ai-skills
How we scanned
A rule-based scanner, modeled on SlowMist's Agent Security Framework and its 11 red-flag categories. It runs static checks over each skill's README and code, looking for concrete patterns: outbound data exfiltration (curl -d $(...)), credential harvesting (env | grep -i token), reading .env / .ssh / .aws, curl | sh install scripts, privilege escalation, persistence, and secret-exfil combos. Each skill gets a grade — safe / caution / unsafe / reject — plus the specific flags it tripped. Skills with no README or too new to fetch stay unknown.
This is deliberately a first layer: it catches patterns, not intent. At 117K scale, the pattern layer is what makes the catalog auditable at all.
Finding 1 — 82% of the catalog has never been graded
Of 117,854 indexed skills, only 20,853 (17.7%) clear 5 stars — the threshold where a skill is popular enough to be worth grading. The other ~97,000 are effectively unaudited.
"We have 117K skills" is not a feature. The number that matters is how many you can actually trust, and for the long tail the honest answer is: nobody has looked.
Finding 2 — Among graded skills, 1 in 32 is unsafe or worse
| Grade | Share |
|---|---|
| 🟢 safe | 85.5% |
| 🟡 caution | 5.3% |
| 🔴 unsafe | 3.0% |
| â›” reject | 0.1% |
| ⚪ unknown | 6.1% |
8.4% carry a security concern. 3.1% — about 1 in 32 — are unsafe or reject. At this catalog's size that's ~650 graded skills you genuinely should not run blind, sitting in the same search results as everything else.
Finding 3 — Popularity predicts safety. The risk lives in the long tail.
| Stars | Unsafe / reject |
|---|---|
| 5–20★ | 4.1% |
| 20–100★ | 3.7% |
| 100–1,000★ | 0.9% |
| 1,000★+ | 0.4% |
The skill you've heard of is almost certainly fine. The danger is the obscure 7-star repo you'd grab from a search for a niche task — exactly the moment a directory is supposed to help, and usually doesn't.
Finding 4 — The red flags include a new, agent-native attack surface
Most common flags among a sample of 1,000 flagged skills:
| Flag | Count |
|---|---|
| sudo usage | 483 |
| background service install | 152 |
| curl | shell | 99 |
| agent config theft | 87 |
| tunnel service | 66 |
| eval() | 52 |
| sensitive env vars | 34 |
| agent memory theft | 23 |
| backdoor install | 11 |
The classic shell risks dominate. But look at agent config theft (87) and agent memory theft (23): skills that read your agent's configuration and memory files. That's not a server exploit — it's a new attack surface that only exists because you're running an agent. Your Claude/MCP config, your stored context, your credentials-by-proxy. The threat model moved, and most directories haven't noticed.
What to do about it
Check the trust signal before you install, from where you already work:
npx @agentskillshub/cli search "postgres mcp" --safe
npx @agentskillshub/cli audit owner/repo
Every result carries its grade and the specific flags it tripped. --safe hides anything unaudited or worse.
The honest caveats (because that's the whole point)
- Our 3% is a floor, not a ceiling. Academic deep-analysis (Liu et al., 2026, arXiv:2601.10338) puts the agent-skill vulnerability rate at 26.1%, because they analyze semantics, not just patterns. Our rule-based first pass deliberately under-claims. Read 3% as the lower bound of a bigger problem.
- ⚪ unknown is not "probably fine." It means no one has checked. 97K of the catalog is unknown. We label it gray and don't dress it up.
- All numbers are reproducible. Every grade is visible on the site and via the CLI. Re-derive them yourself.
A trust layer that only told you the good news wouldn't be one. The most useful thing we can say about 97,000 skills is that we don't yet know — and we'll tell you that to your face.
Full writeup with charts: We security-graded 117,854 AI agent skills. Check any skill before you install: npx @agentskillshub/cli audit owner/repo.
Top comments (7)
The skill grading angle is important because agent safety is starting to look like package security. It is not enough to know a tool exists; teams need provenance, permissions, maintenance signals, and failure modes before letting an agent run it.
"Agent safety is starting to look like package security" is the frame I keep landing on too. The difference is blast radius: a bad npm package runs in your build; a bad skill runs inside an agent that already has your shell, your tokens, and increasingly your context. So provenance / permissions / maintenance / failure modes aren't nice-to-haves, they're the npm audit equivalent we don't have yet. That's what the grade + quality score are reaching for (security grade = permission/risk surface, quality = maintenance + docs), though "failure modes" is the one nobody has a clean signal for yet — open problem.
Yes, that npm audit analogy is exactly the missing mental model. The tricky part is that agents make permission boundaries dynamic: the same skill can be harmless in a read-only run and dangerous when the agent also has shell, tokens, or repo context. I would like to see skill registries expose that risk profile as metadata, not as a paragraph in the README.
The caveat you put at the bottom is the most important line in the whole post: 3% is a floor because pattern-matching catches syntax, not intent. We hit the exact same wall scanning MCP servers at mcpsafe.io. A regex finds curl -d $(...), but it sails right past a tool description written to read as an instruction to the agent — which is basically the MCP-native version of your "agent config theft" flag. The dangerous payload there is plain English sitting in a description field, and there's no pattern to grep for.
The way we ended up handling it was a 5-model consensus panel instead of a pure rule set: run the server through several LLMs and only surface a finding when they agree, specifically to recover the semantic stuff your Liu et al. citation points at without drowning in false positives. Different cost profile than a static scanner — slower, no way you sweep 117K with it — but that's the tradeoff for catching intent. Across ~650 public MCP servers our grade curve looked a lot like yours: most popular servers land fine, and the risk concentrates in the long tail.
Finding 3 is the one I'd want every directory to internalize. "Popularity predicts safety" is true, and it's also exactly why discovery-by-stars quietly launders risk — the obscure 7-star repo is both the riskiest and the one a search surfaces when you're doing something unusual, which is the only moment you actually needed the directory to protect you.
And respect for "unknown is not probably fine." Labeling 97K gray instead of dressing it up green is the part most catalogs won't do, and it's the thing that makes the other numbers trustworthy.
That's exactly the wall, and you put it better than I did: the dangerous payload is plain English in a description field, and there's no pattern to grep for. The "instruction smuggled into a tool description" case is the one that scares me most, because it reads as documentation to a human reviewer too.
Same direction here, different layer: the rule scan is the free catalog-wide floor (the only thing cheap enough to run on all 117K), and the semantic pass runs only on what the floor already flags — same cost wall you describe, no way I sweep the whole catalog with LLMs. Your 5-model consensus panel is a sharper version of that than what I've got; single-LLM drowns in false positives exactly like you'd expect, and "only surface when they agree" is a clean way to buy precision back. Going to steal that framing.
The grade curves matching across two independent datasets (~650 servers vs the catalog) is the part I find most reassuring. And "discovery-by-stars launders risk" is the whole thesis in five words. the long-tail repo is both the riskiest and the one a search hands you at the exact moment you needed the directory to have your back. Would love to compare notes on the prose-injection category sometime.
Great analysis! Scanning 117k AI agent skills and finding that most are barely evaluated is eye-opening.
Really appreciate the clear breakdown on long-tail risks and agent-native attack surfaces. The tool looks super useful — I’ll definitely try the CLI.
Thanks for the valuable insights! 🚀
Thanks, appreciate it. If you try it, npx @agentskillshub/cli audit owner/repo is the fastest way in — and I'd genuinely want to hear if a grade looks wrong on something you know well. The long-tail accuracy is where it needs the most eyes.