DEV Community

Avery Wang
Avery Wang

Posted on

Your AI Coding Assistant Writes Shell Commands. Do You Actually Test Them Before They Run?

There's a popular conversation going around DEV right now about giving AI agents more tools and what happens when the boundaries fail. Most of those posts talk about the problem abstractly: prompt injection, over-permissioned agents, runaway automation. I want to get concrete about the smallest, most common version of it.

Almost every AI-assisted coding workflow ends with the model suggesting a shell command. A migration. A cleanup script. A find ... -delete. And the failure mode isn't usually malicious — it's plausible-looking but wrong. The command is syntactically valid, confidently phrased, and subtly destructive on your actual filesystem layout.

So I built a tiny habit: every destructive or irreversible command an AI assistant suggests goes through a dry-run harness before it touches a real directory. Here's the harness, the test plan, and where this approach breaks down.

The core idea: assert before you execute

The trick is to stop treating AI output as a command and start treating it as a proposal that must pass assertions. For file operations, most commands have a dry-run or list-equivalent mode:

  • rm -rf path → first find path -type f | head and check the count and scope
  • mv/cp batches → run with -n (no-clobber) and echo first
  • git clean -fd → always git clean -nd first
  • SQL migrations → run inside a transaction you roll back

I wrapped this into a small shell script I keep in ~/bin/ai-guard.sh:

#!/usr/bin/env bash
# ai-guard.sh — inspect a proposed command before running it.
# Usage: ai-guard.sh <sandbox_dir> -- <command...>
set -euo pipefail

SANDBOX="$1"; shift
[ "$1" = "--" ] && shift

if [ ! -d "$SANDBOX" ]; then
  echo "Sandbox dir '$SANDBOX' does not exist. Refusing." >&2
  exit 1
fi

# Rule 1: never let the command reference anything outside the sandbox.
for arg in "$@"; do
  case "$arg" in
    /*|*..*) 
      echo "BLOCKED: argument '$arg' escapes the sandbox." >&2
      exit 2 ;;
  esac
done

# Rule 2: snapshot file list before, run inside sandbox, diff after.
cd "$SANDBOX"
find . -type f | sort > /tmp/before.txt

echo ">>> Running inside $SANDBOX: $*"
"$@" || true

find . -type f | sort > /tmp/after.txt
echo ">>> Files changed:"
diff /tmp/before.txt /tmp/after.txt || true
Enter fullscreen mode Exit fullscreen mode

The workflow: I create a throwaway copy of the directory structure the command is meant to operate on (just the layout plus a few sentinel files — cp -r --parents or a small fixture script works), run the AI's proposed command through the guard, and read the diff. Only if the diff matches my intent do I run the real thing — and even then, with the dry-run variant first.

A concrete failure this catches

Last month an assistant suggested this to clean up nested build output:

find . -name "dist" -type d -exec rm -rf {} +
Enter fullscreen mode Exit fullscreen mode

Looks fine. But run in my actual repo root, it also matched packages/e2e/fixtures/dist — a checked-in fixture directory, not a build artifact. The sandbox harness flagged it immediately because my fixture copy included that directory and the diff showed deletions I didn't intend. The fix was adding -not -path "*/fixtures/*", which I would never have thought to check if I'd just pasted the command into my terminal.

That's the pattern worth internalizing: the AI's command was correct for the repo it imagined, not the repo I have. A sandbox diff forces the mismatch to the surface.

Where the free-tier tooling fits

One practical note: this habit multiplies how often you iterate with the model. I rarely accept the first suggested command — I ask for a safer variant, a dry-run version, or an explanation of edge cases, and each of those is another round trip. Iterating like that on a paid meter adds friction, which is honestly why people skip it. I've been running this loop with MonkeyCode, which offers free model access plus a free server option, so the extra "give me the dry-run version first" prompts don't cost anything. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The harness above doesn't depend on any specific tool, though — it works the same whether the command came from an assistant, a teammate, or a Stack Overflow answer from 2014.

Test plan you can steal

If you want to adopt this, here's the minimal checklist I run for any AI-suggested command that deletes, moves, overwrites, or migrates:

  1. Scope check: Does the command reference absolute paths or ..? If yes, rewrite before anything else.
  2. Fixture replay: Rebuild the target structure in /tmp with sentinel files, including edge cases (fixtures, symlinks, dotfiles).
  3. Dry-run first: Use -n, --dry-run, git clean -nd, or EXPLAIN/BEGIN; ... ROLLBACK; equivalents. No dry-run mode exists? That's itself a red flag.
  4. Diff review: Compare before/after file lists. Any unexpected path in the diff = stop.
  5. Reversibility question: If the command is wrong on the real run, what's my recovery? No answer (no backup, no VCS) = don't run it yet.

Limitations, honestly

  • This harness is a safety net, not a validator. It tells you what a command did to files; it can't tell you whether a migration produced semantically correct data.
  • It doesn't help with commands whose damage is non-local: API calls, CI triggers, package publishes, anything touching shared state. Those need environment-level isolation (a staging project, a scoped token), not a directory sandbox.
  • Path-based blocking is naive. A determinedly weird command can do damage without a suspicious-looking argument. Treat the guard as a tripwire, not a proof.
  • It adds a minute or two per command. That's the point — but it means you'll be tempted to skip it "just this once" for commands that look simple. In my experience the simple-looking ones are exactly where the fixtures get deleted.

Who should skip this

If your AI assistant never generates shell/SQL/infra commands for you — say you only use it for in-editor completions — this harness is overhead. And if your team already has a proper ephemeral-environment setup (devcontainers per task, preview environments, disposable VMs), use that instead; it's strictly stronger than my little script.

But if you're a solo dev or on a small team and you've been copy-pasting suggested commands straight into your terminal: build the fixture habit before the day an assistant confidently deletes something with no git history behind it. If you're already iterating with a model daily, try routing that iteration through a free tier — MonkeyCode's free models and free server are one option — and spend the saved budget on being more paranoid, not less.

Top comments (0)