Who's Auditing the Skills Your AI Agent Runs?

A lot of people have recently started installing "skills" into Claude Code, Codex, and similar agents. Skills are convenient — but behind that convenience, a pile of assets nobody is inspecting keeps growing.

So I built skilltrust, a CLI that inventories your installed skills in one command. In the first half of this post I'll explain why "read everything before you use it" is not a defense, and in the second half I'll show the CLI and its design.

sunnydachs / skilltrust

Deterministic, offline, stdlib-only CLI that audits an AI agent's trust surface (skills, hooks, trusted dirs, gh token scopes)

skilltrust

Audit your AI-agent trust surface in one command: skill inventories, trust settings, and token scopes — deterministic, offline, stdlib-only. It never judges skill content; it inventories facts so you can decide by provenance and permissions.

English | 日本語

Why it exists

Agent skills (Claude Code SKILL.md, OpenClaw) are prompt-payload packages: a skill's text IS an instruction to your agent, inherited alongside your shell, filesystem, and stored credentials. Research in 2026 quantified the risk — the study behind NVIDIA's SkillSpector ("Agent Skills in the Wild", Liu et al.) analyzed 31,132 of 42,447 collected skills and found 26.1% carry at least one vulnerability and 5.2% show likely malicious intent; the USENIX Security '26 paper "Do Not Mention This to the User" behaviorally confirmed 157 of 98,380 analyzed skills as malicious. Snyk's ToxicSkills scan found 36.8% of 3,984 marketplace skills defective, 76 of them outright malicious payloads. An audit of the…

View on GitHub

What a skill actually is: a package that runs with your permissions

An agent skill is a package that ships the prompt itself. Once installed, it becomes part of your agent's reasoning, and when it runs it touches your shell and your files — it inherits, exactly, the permissions you granted your agent.

If a skill's contents get swapped, it becomes someone else's code running on your machine.
Someone else's code that reads your files, can reach your credentials, and runs without asking you.

The question stops being "I installed it from a trusted source, so it's fine" and becomes "what is on this machine right now?"

This is not theory — it has been measured

Not a scare story: 2026 research and audits have the numbers. The foundation study behind NVIDIA's SkillSpector collected 42,447 skills and analyzed 31,132 of them: 26.1% contained at least one vulnerability, and 5.2% showed high-severity patterns suggesting malicious intent.

The USENIX Security '26 paper analyzed 98,380 skills and confirmed 157 of them as clearly malicious — deliberate bundles of 632 vulnerabilities across 13 attack techniques. Snyk's ToxicSkills report found defects in 36.8% of 3,984 marketplace skills, 76 of them outright malicious payloads.

In the ClawHavoc campaign, an audit alone found 341 malicious skills — most distributing a credential stealer called Atomic Stealer.

In other words: skills that are dangerous before anyone reads them are already circulating, in volume, as installed inventory.

"Just read every skill before you use it" — why that doesn't work

At this point the standard advice appears: read every skill before you use it. That advice has two practical limits.

First, it doesn't scale — reviewing every skill in a pile that keeps growing is impossible for one person. Second, and this is the deeper problem:

The text you are reading can itself be the payload.

A sentence that steers your agent can sit inside instructions that read as completely safe, and the human eye will not catch it. Static scanners share the same weakness against intent written in natural language.

So what is an effective defense? My answer: know your provenance and blast radius. Concretely, four things:

• What is installed
• Which directories you have marked as "trusted"
• Whether hooks (commands that fire when a session starts) are enabled
• Whether over-broad GitHub tokens are reachable from your agent

skilltrust: that inventory in one command

skilltrust turns those four questions into one command. What it does inside:

Installed skills        ──┐
Trusted directories     ──┼→ RiskMarkers ──→ ranked checks ──→ one action
GitHub token scopes     ──┤  (derived only)    ↑ first hit      (just one)
Claude hooks on/off     ──┘

It gathers the four inputs, boils them down to derived values called RiskMarkers, walks a fixed priority ladder, and prints exactly one line: what to do next.

A sample run (from the README; counts vary by machine):

AI-agent trust-surface scan — 2026-10-09

[Skill inventory]
  Claude Code : 12 skills (2 script-bundled)
  .agents     : 12 skills (2 script-bundled)

[Trust settings]
  Trusted project dirs: 1 (/home/you/projects/example)
  Claude hooks: no

[Tokens]
  gh: 'workflow', 'delete_repo'  ← workflow, delete_repo overbroad

[Do this one thing]
  → narrow the gh token scope (5 min)

Note: .agents is the convention for a skill location shared across agents.

The only part you need to read is the last section: the single highest-priority action, in one line. In this example it suggests narrowing an over-broad GitHub token scope — a five-minute fix.

A workflow-scoped token can rewrite CI and read the secrets of every repository you own, so it is worth killing first. The full sample output is in the README.

Why not let an LLM judge the contents

By now, someone is thinking: why not just have an LLM decide whether a skill's contents are dangerous? I deliberately don't. Two reasons.

First, an LLM's content verdict cannot be called trustworthy. As covered above, concluding "this text is safe" from reading the text is itself attackable — and if the judge can't be trusted, you can't make decisions on its output.

Facts — provenance and permissions — are the opposite: you can act on them.

Second, determinism has operational value. With the criteria fixed in code, no LLM and no network calls, the same environment produces the same result every time — so you can run it casually at install time, on pull, or any moment something might have changed.

Honest limitations

• It does not judge whether skill contents are dangerous. It looks only at the presence and count of bundled scripts — a marker with a measured 2.12x higher vulnerability rate.
• If a supported agent changes its config file format, skilltrust needs a tracking update.
• Without the gh command installed, the token section reports "unavailable".
• It never prints token values. Scope names only.

What it's like to run (measured)

I ran it on WSL2 with Python 3.14, scanning an ordinary personal dev environment with a handful of config files. What that looks like, with measured numbers:

• Runtime: 0.01 s (three measurements — it feels instant)
• Tests: 23, all passing (CI runs the same set on Python 3.11–3.14)
• Implementation: 814 lines, zero dependencies (Python standard library only)

All it does is read a few config files, so it should finish within seconds on your machine too.

Summary: inventory before you read

Skill safety is not decided by "did I read all of it" — it is decided by whether you know your provenance and blast radius. skilltrust does that check in one command.

Knowing where to look first when something breaks matters. But the bigger value is keeping the inventory current before anything breaks — if it is always up to date, you notice the moment it differs from your usual list.

Code: skilltrust on GitHub — bug reports and ideas welcome in issues.

This is a personal OSS project with no warranty. If you hit bugs or have suggestions, GitHub issues are the best way to reach me.