DEV Community

hao li
hao li

Posted on

Your Claude Code Plugin Passed Validation. That Proves Nothing.

Your Claude Code Plugin Passed Validation. That Proves Nothing.

claude plugin validate (without --strict) exits 0 on a broken skill.
I measured it. A green validate is not evidence the plugin works — and it is
certainly not evidence the plugin is safe.

That sentence is the reason frontmatter-guard v0.2 exists.

What v0.1 already did

v0.1 was a semantic linter for skill/plugin frontmatter. The official
validator checks structure — "will this plugin load?" — but never asks
whether every key you wrote actually does something. I learned this the
embarrassing way: I shipped effort: high in a command frontmatter,
claude plugin validate passed it silently, and effort turned out to be a
Codex concept that Claude Code ignores completely. My setting silently fell
back to the session default and nothing told me.

frontmatter-guard v0.1 catches that class of mistake: unknown keys with
"did you mean…?" suggestions, misspelled hook events that would never fire,
missing required fields, bad versions, wrong types.

The second blind spot: the manifest

But frontmatter is only one dimension. Probing the official validator showed
a second, scarier gap: it never checks whether the manifest's claims match
reality. A plugin can:

  • claim an official-sounding name (official-superpowers, anthropic-toolkit),
  • declare zero permissions,
  • ship hook scripts the manifest never mentions,

…and the official check still waves it through. Official validate is not a
working guarantee, and it is definitely not a security guarantee.

v0.2: manifest audit

v0.2 adds a second layer that audits .claude-plugin/plugin.json against
what is actually on disk, and emits a verdict — pass, review, or fail
— with a fix suggestion for every finding:

Rule Severity What it catches
impersonating-name error Official-sounding names (claude-code, official-*, anthropic-*)
typosquat-name warning Names suspiciously close to official ones
undeclared-hooks error Hook scripts on disk, no hooks declared in the manifest
undeclared-capability error Real capability (shell from hooks, network from hook commands or mcpServers) with no declared permissions
over-declared-permissions warning Permissions claimed for code that isn't there
missing-provenance warning No author / repository / license / homepage — nobody to hold accountable
pip install frontmatter-guard
frontmatter-guard check ./my-plugin
Enter fullscreen mode Exit fullscreen mode

Real output on a malicious-looking fixture:

my-plugin/.claude-plugin/plugin.json:1: error [impersonating-name] Plugin name 'official-superpowers' uses the authority-borrowing prefix 'official'.
    fix: Drop the prefix and pick a name that does not borrow authority from Anthropic or from 'official' status.
my-plugin/.claude-plugin/plugin.json:1: error [undeclared-hooks] Hook scripts exist on disk but the manifest declares no 'hooks'.
    fix: Declare every hook in the manifest 'hooks' key (event -> commands), or remove the hook files. Undeclared hooks run code the manifest never admits to.
my-plugin/.claude-plugin/plugin.json:1: error [undeclared-capability] Plugin can do network, shell but declares no matching permission.
    fix: Add the capability to the manifest 'permissions' list (e.g. "permissions": ["Bash"]), or remove the code that needs it. Hidden capabilities are the classic supply-chain move.
frontmatter-guard: 3 error(s), 1 warning(s) in 1 file(s)
frontmatter-guard: manifest verdict: fail
Enter fullscreen mode Exit fullscreen mode

A fail verdict always includes at least one error-level finding, so it
fails CI on its own. JSON output (--format json) carries the verdict for
automation.

Details

If you install Claude Code plugins from anywhere other than first-party
sources, run this before you trust them. The official green checkmark is not
doing that job for you.

Top comments (0)