DEV Community

Cover image for Is your agent's grep tool a shell command?
Vasu Dalal
Vasu Dalal

Posted on • Originally published at agentx-core.com

Is your agent's grep tool a shell command?

#ai #typescript #security #agents

When you give an LLM a tool, you hand it a real function and let it choose the arguments. Those tools are everything your agent can do to a real system: read a file, write to your database, send an email, run a shell command, delete data. They are your risk surface, and most teams have never looked at it in one place.

So we did. We ran scan across a batch of popular open-source TypeScript AI agents. A few of the things it found, none of them exotic:

  • A coding agent whose grep and glob tools, which sound read-only, actually shell out through execSync. Its bash tool passes a model-chosen string straight to spawn. Arbitrary command execution, behind three innocuous names.
  • A query tool that fires an HTTP DELETE. A "query" that deletes.
  • A calculator that runs eval on whatever the model types, in a widely-used agent framework. Arbitrary code execution behind the friendliest name in the box.
  • A send-email tool that posts to an array of recipients, so the model chooses who gets mailed.
  • The single most common finding, in almost every agent we scanned: a fetch tool aimed at whatever URL the model supplies. That is a door to your internal network (an SSRF surface).

Notice the pattern. The dangerous tools are not named dangerous. They are named grep, query, calculator. A name is a claim. The code is the evidence.

See your own agent's tools

scan reads that evidence. One command, no install, no signup, no code change:

npx @agentx-core/scan .
Enter fullscreen mode Exit fullscreen mode

It lists every tool the model can call and ranks each one by what it can do, from read-only up to destructive:

🔍 AGENTX SCAN        (TypeScript · 3 files · 5 tools)
===========================================================================

  RISK   TOOL                GUARD              WHY
  ----   ----                ------   ------------------------
  high   calculator          yes      calls `eval`
         lib/tools/compute.ts:4
  high   grep                yes      calls `execSync`
         lib/tools/system.ts:8
  med    sendEmail           yes      calls `mailer.send`
         lib/tools/io.ts:5
  med    fetchUrl            yes      outbound req to agent-controlled host (SSRF)
         lib/tools/io.ts:11

  2 of 5 tools can take destructive or batch actions.
  Nothing here is guarding them: no AgentX in this project's dependencies.
Enter fullscreen mode Exit fullscreen mode

Every ranking carries its evidence, so you can check it instead of trusting it. Scan reads the tool's body, not its name: a tool called tidyUp that deletes files ranks high, and a scary-sounding tool whose code is actually clean goes to "review," not the top. When it cannot tell, it says so. On one agent, all 32 tools came back "nothing here to guard" because they run elsewhere, and scan said exactly that instead of inventing risk.

What it does not do

It does not run your code, and it does not decide whether a tool is exploitable. It sorts an inventory so the dangerous end is the first thing you see. Expect roughly a third of a real codebase to land in "review." That is the honest cost of not guessing.

Guarding what it finds

Scan reports. It does not change your code. To actually guard what it finds, put the AgentX gateway in front of your agent, or wrap your MCP server in one line. Both are language-agnostic, so there is no SDK to adopt.

It covers TypeScript agents built on the Vercel AI SDK today. Python agent? The scanner is TypeScript for now, but the guard it points to, the gateway and the MCP wrapper, is language-agnostic. Run it on yours:

npx @agentx-core/scan .
Enter fullscreen mode Exit fullscreen mode

This matters more as models get better, not less. A more capable agent does more with an unguarded tool.

Try it

Top comments (1)

Collapse
 
komo profile image
Reid Marlow •

A useful way to phrase this is: the tool name is documentation, not a boundary. I’ve been burned by letting the model see a polite verb while the implementation had shell/file/network reach behind it. The scanner is the right first pass, but I’d still want the CI gate to fail on capability drift: if grep stops being read-only, somebody has to review that diff before an agent can touch it.