DEV Community

Cover image for Skill Inspector: ESLint for your AI agent skills (SKILL.md)
Aditya
Aditya

Posted on

Skill Inspector: ESLint for your AI agent skills (SKILL.md)

#ai #python #opensource #productivity

Agent skills are turning into config files. A SKILL.md has some frontmatter, a few instructions and a folder of scripts and references, and like most config it breaks in dull ways:

  • The description says what the skill does but never when to use it, so the agent never activates it.
  • A referenced scripts/helper.py doesn't exist.
  • The body runs 9,000 tokens, and every activation pays for that context.
  • The name has a double hyphen, which the spec forbids.

You normally find out at runtime, when the agent starts acting strangely. I wanted to catch these earlier, so I built Skill Inspector.

What it is

Skill Inspector is a web-based validation studio and a CLI for agentskills.io-format skill packages. It works as a linter, formatter and test generator for SKILL.md.

Skill Inspector demo

What it checks

Static validation covers every rule in the agentskills.io spec: name format and length, required fields, description limits, body size warnings and unknown frontmatter fields. Each finding has a rule ID and a severity, FAIL or WARN.

Package resolution starts when you point it at a skill directory with --base-dir. It resolves every referenced file under scripts/, references/ and assets/, and flags missing files, empty files and path traversal attempts (../).

Token economics gives a live breakdown of the catalog footprint, the activation body, and the peak on-demand context if the agent reads every reference. You can see what a skill costs before you ship it.

The LLM-as-a-Judge scores four dimensions from 0 to 100: description quality, instruction clarity, scope and boundaries, and token efficiency. It works with OpenAI, Anthropic or a local Ollama model, so you can run it without an API key.

For repairs, fix shows a unified diff and applies deterministic fixes to frontmatter and naming. An optional LLM pass refactors the instructions themselves.

It also generates 10 to 16 synthetic test scenarios (happy path, adversarial, tool failure, scope boundary) and exports them to Pytest, JSON or Markdown.

The trajectory profiler takes an agent execution trace and flags redundant file reads, search loops, error ping-pong and context bloat.

A Claude Code / plugin linting mode accepts Claude Code-specific frontmatter and list-style allowed-tools.

Try it 

# install the CLI
curl -sSL https://raw.githubusercontent.com/neo7337/skill_inspector/main/install/install.sh | bash

# static checks
skill-inspector validate path/to/SKILL.md

# full package audit, Claude Code fields allowed
skill-inspector validate path/to/SKILL.md --base-dir path/to/skill --mode claude-code

# preview and apply auto-fixes
skill-inspector fix path/to/SKILL.md --dry-run
skill-inspector fix path/to/SKILL.md --apply

# generate a pytest suite of edge cases
skill-inspector generate-tests path/to/SKILL.md --format pytest --output tests/test_skill.py
Enter fullscreen mode Exit fullscreen mode

If you prefer a UI, clone the repo, run make install && make dev, and open http://localhost:5173. docker compose up -d works too.

Why I built it

A good skill is closer to an API contract than a prompt. The description is your routing logic, the body is your runtime cost, and the references are your dependencies. Code gets lint, diffs, tests and profiling, and I think skills should get the same.

It's MIT-licensed and still early (v0.0.8), so feedback matters. If you write skills, tell me which checks you wish existed.

Repo: https://github.com/neo7337/skill_inspector

Stars, issues and PRs are welcome.

Top comments (0)