Human in the loop is not a switch: classify the commands

Human in the loop is usually implemented as a switch: approvals on, or approvals off. Both settings are wrong for an agent that runs shell commands, and you find that out the same way everyone does, which is by watching a long task die on its third ls.

The stall looks like this. Your agent is working through something multi-step, it needs to look at a directory, the harness raises an approval, and the run stops dead waiting for a human who has gone to make coffee. Multiply that by forty tool calls and the agent is no longer autonomous, it is a very expensive interactive shell.

So people reach for one of two fixes, and both make it worse.

The two fixes that are not fixes

Turning the gate off. Now rm -rf runs unattended too. The gate was not the problem, and you have removed the only thing standing between a plausible-looking command and an irreversible action.

Wrapping the tool in something else. This is the one I see most: the agent stalls on the built-in shell tool, so someone routes it through a different tool layer and the stall goes away. It goes away because the new layer does not ask. You have not removed the approval, you have moved who is responsible for asking, and usually the answer is now nobody.

The interrupt is not a bug. It is the gate doing its job. The bug is that the gate cannot tell ls from rm.

Classify by reversibility, not by tool

The useful axis is not which tool is being called, it is whether the call can be undone. Reads are recoverable by definition, since the worst case is that you read something and discard it. Writes, deletes, and anything that leaves the machine are not.

So the gate gets a classifier, and the classifier decides who has to wake up:

READ_ONLY = {
    "ls", "cat", "head", "tail", "wc", "stat", "file", "find", "grep", "rg",
    "git status", "git log", "git diff", "git show", "git branch",
}

RISKY_PREFIXES = ("rm", "mv", "dd", "chmod", "chown", "kill", "curl", "wget",
                  "git push", "git reset", "npm publish", "docker rm")

# Anything that lets one approved command smuggle in another.
SHELL_OPERATORS = ("&&", "||", "|", ";", "`", "$(", ">", ">>", "\n")


def needs_human(command: str) -> bool:
    cmd = command.strip()

    # A command containing shell operators is not one command, it is several.
    # Do not try to reason about the pieces. Ask.
    if any(op in cmd for op in SHELL_OPERATORS):
        return True

    if any(cmd == allowed or cmd.startswith(allowed + " ") for allowed in READ_ONLY):
        return False

    # Everything not explicitly known-safe requires a human, including
    # commands that merely look harmless.
    return True

Two things about that function matter more than the word lists.

It is an allowlist, not a blocklist. A blocklist is a bet that you thought of every dangerous command, and you did not. RISKY_PREFIXES above is there for logging and for showing the user why something is being escalated; it is not what makes the decision. The decision is made by the final return True, which is the deny-by-default line. Delete that line and the whole thing is decoration.

It refuses to parse compound commands. ls && rm -rf build starts with an allowlisted token. If your check is startswith, you just approved a delete. You can go and write a real shell parser, or you can treat the presence of an operator as an automatic escalation, which is a two-line rule that does not have parser bugs.

The asymmetry is what makes this safe

Every classifier is wrong sometimes, so the question is what its errors cost.

If the classifier calls a safe command risky, the user gets one unnecessary approval prompt. Annoying, bounded, visible.

If it calls a risky command safe, something irreversible happens with nobody watching. Not bounded, and often not visible until later.

Those two are not comparable, which is why the default branch has to be "ask" rather than "allow". You are not trying to build a classifier that is right. You are trying to build one whose mistakes all land on the cheap side, and then curating the allowlist until the cheap mistakes are rare enough to live with.

This is also the reason the allowlist should be boring and explicit rather than clever. Every entry you add is a small, deliberate decision that this exact thing can run while you sleep.

What this looks like in practice

We build recal, a local-first assistant on macOS, and its command tool works this way: read-only commands execute unattended, everything else raises an approval that carries the actual command text so the user is approving a thing rather than a category. Long tasks stopped stalling, and the approvals that remain are the ones worth reading.

The honest catch is that the allowlist is maintenance. New tooling shows up, someone's workflow needs jq or kubectl get, and the list has to grow. There is no version of this where you write the classifier once. What you get instead is a system where the cost of being wrong is a prompt instead of a restore from backup, and that trade has been worth it every time.

The other honest catch: this does nothing about a command that is individually safe and collectively terrible. Forty approved writes in the wrong directory is still forty approved writes. Reversibility classification bounds the blast radius of a single call, not of a plan.

If you are building one of these

Start with deny-by-default and an allowlist of four or five read commands, then add entries when a real task trips on one. Log every escalation with the command text, because that log is what tells you which entries to add next. And resist the urge to make the classifier smart. The value here is not intelligence, it is that the failure mode is a question rather than a deletion.

Written with AI assistance and edited by a human. The allowlist-over-blocklist and compound-command guidance follows the security notes in Anthropic's bash tool documentation; the rest is what we shipped and what it cost us.