DEV Community

Dakota Liu
Dakota Liu

Posted on

Never Interpolate Tool Args Into a Shell. Exec argv or Fail the Job.

The model can name a tool. It cannot name a shell. That is the rule I actually enforce now.

If you take a tool call, glue it into a string, and hand that string to bash -lc, you already lost. Quotes lie. Newlines lie. $HOME expands. And the "tiny helper" the model invented is a pipeline you never reviewed. Sound familiar?

I stopped treating tool calling as "the model talks to bash." I treat it as "the host maps a JSON object onto a frozen argv vector." No pipes. No &&. No $PATH scavenger hunts. If the mapper cannot build argv, the job dies. The model does not get a retry so it can "fix the command." Retries are how interpolation bugs become incident reports.

This is a from-zero tutorial. Freeze a catalog. Validate args. Exec with shell=False. Prove each stage with a command you can rerun. I am not selling a vibe. I am giving you a runner.

0. What you are building

A host-owned loop. The model only proposes a structured call. The host does everything else.

  1. Model returns {"tool": "...", "args": { ... }}.
  2. Host checks the tool name against a frozen catalog.
  3. Host validates args against that tool's JSON Schema.
  4. Host maps args onto argv. Never onto a string.
  5. Host execs, captures a bounded result, increments a step file the model cannot write.

Fail closed. Not "ask the model to try again." Fail the job. Why so harsh? Because the second attempt is usually a sneakier string.

1. Freeze the tool catalog

Do not describe tools in the prompt and hope the model respects the paragraph. Prose is not a contract. A file is.

Create tools.json. The model may read a summary. The model may not edit this file. If you catch yourself adding a run_command tool, sit on your hands.

{
  "version": 1,
  "tools": {
    "pytest_one": {
      "description": "Run one pytest node id",
      "schema": {
        "type": "object",
        "additionalProperties": false,
        "required": ["nodeid"],
        "properties": {
          "nodeid": {
            "type": "string",
            "minLength": 1,
            "maxLength": 200,
            "pattern": "^[A-Za-z0-9_./:\\[\\]-]+$"
          }
        }
      },
      "argv": ["python", "-m", "pytest", "-q", "{nodeid}"]
    },
    "git_status": {
      "description": "Read git status porcelain",
      "schema": {
        "type": "object",
        "additionalProperties": false,
        "properties": {}
      },
      "argv": ["git", "status", "--porcelain=v1"]
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

What is missing on purpose? shell. bash. npm run with a free-form script name. curl with a URL the model invented. If a task cannot be expressed as a named tool with a tight schema, it does not run in this job.

Verify this stage:

python -c "import json, pathlib; json.loads(pathlib.Path('tools.json').read_text()); print('catalog_ok')"
git check-ignore -q tools.json && echo 'catalog should be committed, not ignored'
Enter fullscreen mode Exit fullscreen mode

If json.loads throws, you do not start the agent. You fix the catalog. If tools.json is gitignored, you do not start the agent either. A catalog that is not in version control is a rumor.

2. Validate args before you think about exec

JSON Schema is the gate. Not a regex over the whole blob. Not "the model promised it was safe." Extra keys are a fail. Wrong types are a fail. A nodeid that looks like ../../etc/passwd is a fail.

Label this as a proposed runner, not a production metric dump. Drop runner.py next to the catalog.

# runner.py — proposed host-side mapper. Not an SDK.
from __future__ import annotations

import json
import os
import re
import subprocess
import sys
from pathlib import Path

try:
    import jsonschema
except ImportError:
    print("pip install jsonschema", file=sys.stderr)
    raise

CATALOG = Path("tools.json")
STEP_FILE = Path(".agent_step")
PLACEHOLDER = re.compile(r"^\{([A-Za-z_][A-Za-z0-9_]*)\}$")
MAX_STEPS = 8
TIMEOUT_S = 30
ENV_ALLOW = ("PATH", "LANG", "LC_ALL", "TERM")


def load_catalog() -> dict:
    data = json.loads(CATALOG.read_text())
    if data.get("version") != 1 or "tools" not in data:
        raise SystemExit("catalog_invalid")
    return data["tools"]


def validate_call(tools: dict, call: dict) -> tuple[str, dict]:
    if set(call) != {"tool", "args"}:
        raise SystemExit("call_shape")
    name = call["tool"]
    if name not in tools:
        raise SystemExit(f"unknown_tool:{name}")
    jsonschema.validate(call["args"], tools[name]["schema"])
    return name, call["args"]


def map_argv(template: list[str], args: dict) -> list[str]:
    out: list[str] = []
    for part in template:
        m = PLACEHOLDER.fullmatch(part)
        if not m:
            if "{" in part or "}" in part:
                raise SystemExit("argv_template_corrupt")
            out.append(part)
            continue
        key = m.group(1)
        if key not in args:
            raise SystemExit(f"missing_slot:{key}")
        value = args[key]
        if not isinstance(value, str):
            raise SystemExit(f"slot_not_string:{key}")
        if any(ch in value for ch in " \t\n\r\"'`$|&;<>()"):
            raise SystemExit(f"slot_metachar:{key}")
        out.append(value)
    return out
Enter fullscreen mode Exit fullscreen mode

See the last check? Even after schema, I still refuse shell metacharacters in a substituted slot. Schema is necessary. It is not sufficient. Why both? Because tomorrow you will loosen a pattern "just this once."

Verify this stage with fixtures. Do not skip the extra-key case. That is the one models love.

cat > /tmp/good.json <<'EOF'
{"tool":"pytest_one","args":{"nodeid":"tests/test_api.py::test_ok"}}
EOF
cat > /tmp/extra.json <<'EOF'
{"tool":"pytest_one","args":{"nodeid":"tests/test_api.py::test_ok","cmd":"rm -rf /"}}
EOF
cat > /tmp/meta.json <<'EOF'
{"tool":"pytest_one","args":{"nodeid":"tests/test_api.py::test_ok; id"}}
EOF

python - <<'PY'
import json, pathlib, runner
tools = runner.load_catalog()
for path, expect_fail in [("/tmp/good.json", False), ("/tmp/extra.json", True), ("/tmp/meta.json", True)]:
    call = json.loads(pathlib.Path(path).read_text())
    try:
        name, args = runner.validate_call(tools, call)
        argv = runner.map_argv(tools[name]["argv"], args)
        failed = False
    except SystemExit:
        failed = True
    except Exception:
        failed = True
    print(path, "fail" if failed else "ok", "argv=" + str(argv) if not failed else "")
    if failed != expect_fail:
        raise SystemExit(f"fixture_mismatch:{path}")
print("schema_gate_ok")
PY
Enter fullscreen mode Exit fullscreen mode

If /tmp/extra.json succeeds, your schema allowed additionalProperties. Stop. You do not have a catalog. You have a suggestion box.

3. Exec with a list. Never with a string

subprocess has two personalities. The list form is argv. The string form is a shell waiting to happen. Guess which one I allow.

def allowed_env() -> dict[str, str]:
    return {k: os.environ[k] for k in ENV_ALLOW if k in os.environ}


def read_step() -> int:
    if not STEP_FILE.exists():
        return 0
    raw = STEP_FILE.read_text().strip()
    if not raw.isdigit():
        raise SystemExit("step_corrupt")
    return int(raw)


def bump_step(current: int) -> None:
    nxt = current + 1
    if nxt > MAX_STEPS:
        raise SystemExit("max_steps")
    tmp = Path(".agent_step.tmp")
    tmp.write_text(str(nxt) + "\n")
    tmp.replace(STEP_FILE)


def exec_argv(argv: list[str]) -> dict:
    if not argv or argv[0] in {"bash", "sh", "zsh", "fish", "cmd", "powershell"}:
        raise SystemExit("shell_binary_refused")
    proc = subprocess.run(
        argv,
        cwd=Path.cwd(),
        env=allowed_env(),
        capture_output=True,
        text=True,
        timeout=TIMEOUT_S,
        shell=False,
        check=False,
    )
    return {
        "argv": argv,
        "code": proc.returncode,
        "stdout_head": proc.stdout[:2048],
        "stderr_head": proc.stderr[:2048],
    }
Enter fullscreen mode Exit fullscreen mode

Pinned cwd. Allowlisted env. Timeout. shell=False. And a hard refuse if argv[0] is a shell. The catalog should never have put it there. I still check. Trust the file, then distrust it anyway.

Verify this stage by proving the string form is unreachable and the list form is the only path.

python - <<'PY'
import inspect, runner
src = inspect.getsource(runner.exec_argv)
assert "shell=False" in src
assert "shell=True" not in src
assert "bash -lc" not in src
print("no_shell_in_exec")
PY

# Probe: git_status should run. A forged shell argv must not.
python - <<'PY'
import runner
try:
    runner.exec_argv(["bash", "-lc", "echo pwned"])
    raise SystemExit("shell_probe_should_have_died")
except SystemExit as e:
    print("shell_probe_ok", e)
print(runner.exec_argv(["git", "status", "--porcelain=v1"]))
PY
Enter fullscreen mode Exit fullscreen mode

Did the bash probe print pwned? Then you are not running this file. You are running a different one. Find out which.

4. The step counter lives on disk the model cannot write

The model will happily emit {"tool":"pytest_one",...} forever. Loops are cheap for the model. They are not cheap for you. So the host owns a monotonic counter. The model never sees a tool that can echo 0 > .agent_step.

def handle(call_raw: str) -> dict:
    tools = load_catalog()
    call = json.loads(call_raw)
    step = read_step()
    bump_step(step)
    name, args = validate_call(tools, call)
    argv = map_argv(tools[name]["argv"], args)
    return exec_argv(argv)


if __name__ == "__main__":
    payload = sys.stdin.read()
    print(json.dumps(handle(payload), indent=2))
Enter fullscreen mode Exit fullscreen mode

Verify this stage by replaying the same call until the counter trips. Eight is arbitrary. Pick a number you can defend. Then defend it.

rm -f .agent_step
for i in 1 2 3 4 5 6 7 8 9; do
  echo "step_try=$i"
  printf '%s' '{"tool":"git_status","args":{}}' | python runner.py \
    && echo "ran" \
    || echo "stopped as expected at try=$i"
done
cat .agent_step
Enter fullscreen mode Exit fullscreen mode

If try 9 still runs, MAX_STEPS is decoration. Delete the runner and start over. A counter you do not enforce is a diary entry.

5. Where a spare model and a spare box actually help

You iterate the catalog on a throwaway machine so a bad argv cannot see your laptop SSH keys. That is the whole reason to leave localhost. Not because "cloud is magic." Because blast radius is a directory you can delete.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode's free model access and free server option are one way to iterate this runner without parking the exec loop on your laptop. I am not attaching model names, quotas, or hardware claims I cannot show you from a primary source. If you try it, start with the fixtures in section 2. Do not start by adding a shell tool.

The model is only proposing JSON. The server is only a box. The catalog still has to be yours.

6. Decision table for the next tool you are tempted to add

Ask these in order. If you cannot answer, the tool does not ship.

Question Pass Fail
Can I name the binary without $PATH guessing? argv[0] is a real executable you installed "whatever the image has"
Are args a closed object? additionalProperties: false and required keys listed leftover cmd, flags, extra
Does any slot need a shell to make sense? no pipes, globs, $(...), &&
Can I bound stdout without feeding raw bytes back unfiltered? head + cap, then a schema for the next model turn "just dump it"
What happens on unknown keys? job fails model retries

Copy the table into the PR. If a reviewer cannot mark Pass on every row, the tool stays out. Yes, even the "debug" tool. Especially the debug tool.

7. Limitations, and who should not use this

This runner does not sandbox syscalls. It does not replace seccomp, a container, or a network policy. python -m pytest can still import surprising modules if your tree is dirty. git status is read-shaped, not proof that git is harmless in every repo. JSON Schema will not save you if you write a schema that accepts command as a free-form string. That is you interpolating again, with extra steps.

Do not use this approach if your product is a general-purpose shell agent. You would be lying to your users. Do not use it if you cannot pin the catalog in git. Do not use it as a substitute for code review on the argv templates themselves. And do not use it to launder a bash -lc wrapper behind a friendlier tool name. I will notice. Your future self will notice too, usually at 2 a.m.

The point is smaller than the discourse around "agents." One JSON object. One argv vector. One host-owned counter. If that feels too tight, good. Tight is the point.

Top comments (0)