DEV Community

Mohammad SHEIKHI
Mohammad SHEIKHI

Posted on

I scanned 4,078 AI agent packages on my own machine. Here's what I found.

AI coding agents don't just read config — the skills, MCP servers, and rules files you install are instructions and code that run with your agent's privileges: your shell, your files, your tokens.

I kept installing community skills without reading them. So I built agentscan — think npm audit, but for the stuff you install into your coding agent.

npx @meedo01/agentscan ./some-skill
Enter fullscreen mode Exit fullscreen mode

What it looks for

  • Prompt injection — "ignore previous instructions", hidden/bidi Unicode, "don't tell the user"
  • Data exfiltration — credential reads combined with network sends within a few lines
  • Dangerous commandscurl | bash, unscoped rm -rf, reverse shells, shell-startup persistence
  • MCP tool-poisoning — hidden directives in tool descriptions, exec tools on auto-approve
  • Over-broad permissions--dangerously-skip-permissions, wildcard tool grants
  • Hardcoded secrets — AWS/OpenAI/GitHub/Slack keys

Zero runtime dependencies, fully offline, runs in milliseconds. Nothing you scan leaves your machine.

What I found on my own machine

Running it over my home directory found 4,078 agent packages (skills, plugins, marketplaces):

  • The overwhelming majority graded A/A+ — clean.
  • A handful had dozens of critical findings: real rm -rf and curl | bash in shipped scripts, permission-bypass flags, SSH-key access. Not necessarily malicious — but exactly what you'd want to know before an agent executes them.

The hard part: false positives

The first version flagged ~2,000 "high/critical" findings on my machine. Almost all noise. Three releases of tuning later it's ~80, and the lessons were interesting:

  1. Documentation is not code. A curl | bash in a README is an example; in a shipped script it's critical. agentscan demotes danger patterns in prose to INFO.
  2. Test fixtures are not instructions. Security tools' own test files are full of scary strings.
  3. Auth is not exfiltration. Sending a Bearer token to the API it belongs to is normal; reading ~/.ssh/id_rsa and POSTing it to webhook.site is not. The rule now only pairs local credential-file reads with network sends.
  4. Grade per package. One risky skill shouldn't tar 4,000 clean ones with an aggregate F — --by-package grades each skill/plugin separately, worst first.

Try it

npx @meedo01/agentscan ~/.claude --by-package
Enter fullscreen mode Exit fullscreen mode

MIT licensed: https://github.com/Mohammad1sh/agentscan

It's heuristic static analysis: treat findings as "look here", not proof — and a clean report as "no known-bad patterns", not a guarantee. Rule ideas and false-positive reports are very welcome.

Top comments (0)