Fail Closed on Side Effects: A Blast-Radius Gate for Agent Patches

An agent patch can pass every unit test and still write outside the workspace, call an undeclared tool, or read an env key the task never named. Gate the blast radius first. Score the prose later.

This article is a method, not a field report. It proposes a fail-closed envelope around filesystem roots, tool names, environment keys, and network hosts. Side-effect violations never freeze. Only a dual-runner disagreement on a non-envelope property may freeze, and only with a hashed evidence bundle.

The conclusion in one rule

Treat an agent patch as a capability change. If the run touches anything outside a declared envelope, the gate fails closed. Flakes in ranking, wording, or latency do not override that rule.

Cheap generation does not make side effects cheap to reverse. A green suite that never watched /tmp, os.environ, or outbound sockets is not a verification result. It is a missing observer.

What this gate is not

It is not a golden-file of model text. It is not a mutation score. It is not a full-suite rerun after every hunk.

It answers four questions only:

1. Did the run write or delete outside allowed roots?
2. Did it invoke a tool name that is not on the allowlist?
3. Did it read an environment key that is not on the allowlist?
4. Did it open a network host that is not on the allowlist?

If any answer is yes, fail. Do not freeze. Do not retry for luck.

Artifact: a locked envelope and an observer log

Pin the envelope as a fixture. Hash it. Refuse to run if the hash drifts without a review note.

{
  "envelope_id": "agent-patch-envelope-v3",
  "allowed_roots": ["/work/repo", "/tmp/agent-scratch"],
  "allowed_tools": ["read_file", "apply_patch", "run_tests"],
  "allowed_env": ["CI", "RUN_ID", "ENVELOPE_HASH"],
  "allowed_hosts": [],
  "network": "deny"
}

sha256sum envelope.json > envelope.json.sha256
# CI must compare this digest before the agent process starts.

Label the next block as a proposed harness, not a production sandbox. User-space tracing will miss kernel-level tricks. Use it as a cheap tripwire, then add a real jail when the threat model requires one.

# proposed_observer.py — proposal, not a kernel sandbox
from __future__ import annotations

import json
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any

@dataclass
class Envelope:
    allowed_roots: list[str]
    allowed_tools: set[str]
    allowed_env: set[str]
    allowed_hosts: set[str]
    network: str

    @classmethod
    def load(cls, path: str) -> "Envelope":
        raw = json.loads(Path(path).read_text())
        return cls(
            allowed_roots=raw["allowed_roots"],
            allowed_tools=set(raw["allowed_tools"]),
            allowed_env=set(raw["allowed_env"]),
            allowed_hosts=set(raw["allowed_hosts"]),
            network=raw["network"],
        )

@dataclass
class ObserverLog:
    writes: list[str] = field(default_factory=list)
    tools: list[str] = field(default_factory=list)
    env_reads: list[str] = field(default_factory=list)
    hosts: list[str] = field(default_factory=list)

    def record_write(self, path: str) -> None:
        self.writes.append(str(Path(path).resolve()))

    def record_tool(self, name: str) -> None:
        self.tools.append(name)

    def record_env(self, key: str) -> None:
        self.env_reads.append(key)

    def record_host(self, host: str) -> None:
        self.hosts.append(host)

def _inside_roots(path: str, roots: list[str]) -> bool:
    resolved = Path(path).resolve()
    return any(resolved == Path(r).resolve() or Path(r).resolve() in resolved.parents for r in roots)

def violations(env: Envelope, log: ObserverLog) -> list[str]:
    out: list[str] = []
    for p in log.writes:
        if not _inside_roots(p, env.allowed_roots):
            out.append(f"write_outside_root:{p}")
    for t in log.tools:
        if t not in env.allowed_tools:
            out.append(f"undeclared_tool:{t}")
    for k in log.env_reads:
        if k not in env.allowed_env:
            out.append(f"undeclared_env:{k}")
    if env.network == "deny" and log.hosts:
        out.extend(f"network_denied:{h}" for h in log.hosts)
    else:
        for h in log.hosts:
            if h not in env.allowed_hosts:
                out.append(f"undeclared_host:{h}")
    return out

Wire the observer at the only three chokepoints that matter: the tool dispatcher, the env accessor, and the HTTP/socket factory. If a helper can skip those chokepoints, the envelope is theater.

Property checks that cannot be waived

Keep envelope checks in a separate file from ranking or style tests. A later freeze process must not be able to skip this module by name.

# test_envelope_properties.py
from proposed_observer import Envelope, ObserverLog, violations

def test_apply_patch_stays_in_roots(tmp_path):
    env = Envelope.load("envelope.json")
    log = ObserverLog()
    target = tmp_path / "repo" / "src" / "mod.py"
    target.parent.mkdir(parents=True)
    target.write_text("# patched\n")
    log.record_write(str(target))
    log.record_tool("apply_patch")
    assert violations(env, log) == []

def test_undeclared_tool_fails_closed():
    env = Envelope.load("envelope.json")
    log = ObserverLog()
    log.record_tool("curl")
    found = violations(env, log)
    assert found == ["undeclared_tool:curl"]

def test_env_exfil_fails_closed():
    env = Envelope.load("envelope.json")
    log = ObserverLog()
    log.record_env("AWS_SECRET_ACCESS_KEY")
    found = violations(env, log)
    assert "undeclared_env:AWS_SECRET_ACCESS_KEY" in found

Run the envelope file first. If it fails, stop. Do not collect “maybe flakes” from later steps.

python -m pytest test_envelope_properties.py -q --maxfail=1

Numbered workflow

1. Declare the envelope before the patch exists

Write envelope.json in the same review as the task prompt. Allowed roots should be the repo and a scratch directory created for that run id. Allowed tools should be the minimum dispatcher set. Network should default to deny.

2. Hash the envelope and export the digest

The agent process receives ENVELOPE_HASH. It does not receive production tokens. If the file changes, CI fails on digest mismatch before any model call.

3. Execute the patch under the observer

Start a clean workspace. Create scratch. Install the three chokepoints. Run the agent once against a fixed fixture corpus. Persist observer.jsonl next to the patch diff.

4. Fail closed on any envelope violation

Parse the log. If violations() is non-empty, mark the job envelope_fail. That status is not eligible for freeze, retry-on-flake, or “known issue” labels.

5. Score non-envelope properties only after a clean envelope

Examples: output schema valid, patch hunks limited to declared paths, tests in the repo still collected. These may be nondeterministic. Envelope checks are not.

6. Freeze only on dual-runner disagreement

A freeze is a recorded exception, not a mute button. Grant it only when all of the following hold:

1. Envelope is clean on both runners.
2. Runner A and runner B disagree on a named, non-envelope property.
3. Each runner stores observer.jsonl, envelope.json.sha256, and the property name.
4. The freeze record cites both run ids and expires on a calendar date stored in the ledger.

# freeze_ledger.tsv  (header + one example row)
freeze_id  property              runner_a  runner_b  envelope_hash  expires_on  status
F-014      schema_key_order      run-88a   run-88b   sha256:9c2e…  2026-09-10  open

If runner B cannot be scheduled, do not freeze. Fail the property. A single runner cannot distinguish a flake from a regression.

Decision table

Observation	Envelope clean?	Dual-runner agree?	Action
Write under /work/repo	yes	n/a	continue
Write under /etc or $HOME	no	n/a	fail closed, no freeze
Tool apply_patch	yes	n/a	continue
Tool bash not listed	no	n/a	fail closed, no freeze
Env RUN_ID	yes	n/a	continue
Env AWS_SECRET_ACCESS_KEY	no	n/a	fail closed, no freeze
Schema key order differs A vs B	yes	no	freeze with expiry
Schema invalid on both	yes	yes (both fail)	fail, no freeze
Latency jitter only	yes	no	do not freeze; drop latency from the gate

The last row matters. If a property is not worth a dual run, it is not worth a freeze either. Remove it from the gate.

Where a free model and a free server fit

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The envelope gate needs two things the production cluster should not provide: a disposable runner and a model call that never sees production secrets. MonkeyCode’s free model access and free server option are relevant as that disposable pair. The envelope file, the observer, and the ledger stay in your repo. The remote side only receives a scratch workspace, the hashed envelope, and a fixture corpus with fake credentials.

Do not send live tokens “because the model is free.” Free inference does not change the data classification of the prompt. Strip env, strip cookies, strip .netrc. If the fixture cannot be faked, the task does not belong on a shared runner.

A second runner is the freeze requirement. Use the free server as runner B when runner A is local CI. Same envelope hash. Same fixture. Different machine. If they disagree only on a non-envelope property, write a freeze row. If they disagree on an envelope property, treat the observer as broken and stop shipping.

Limitations

User-space observers miss anything that bypasses the wrapped APIs. A subprocess that opens a raw socket, a native extension, or a language runtime the wrapper does not intercept will not appear in observer.jsonl.

Empty allowlists are safer than broad ones, but they reject legitimate refactors that add a tool. That is a review event, not a freeze event. Expand the envelope in the same change that adds the tool.

Dual-runner freeze still costs two executions. It does not prove correctness. It only prevents a one-box flake from silencing a property you already decided was worth gating.

Nondeterministic models will keep producing text drift. This method ignores text drift on purpose. If the product requirement is exact prose, add a different gate. Do not overload the envelope.

Who should not use this

Do not use a fail-closed envelope as the only review for agents that must discover tools at runtime. The allowlist will fight the product.

Do not grant envelope waivers during an incident hotfix unless a human override path is logged with a name and a revert deadline. Silent waivers recreate the original hole.

Do not freeze from a single runner. Do not freeze envelope failures. Do not store production secrets on the free server so the agent can “be more realistic.” Realism is the fixture’s job.

Teams without a second machine should fail open properties instead of inventing a freeze. A ledger that never has a runner B column is a mute list.

Minimal CI shape

set -euo pipefail
test -f envelope.json.sha256
sha256sum -c envelope.json.sha256
python -m pytest test_envelope_properties.py -q --maxfail=1
# only then: run the agent, write observer.jsonl, score other properties
python score_non_envelope.py --observer observer.jsonl --ledger freeze_ledger.tsv

Keep the first three lines boring. Boring is the point. The blast radius is a finite set of paths, tools, keys, and hosts. If the patch needed more, the envelope should have changed in review, not in a flake freeze.

If you already isolate agent work on a free server, put the envelope hash in that job’s environment and keep production credentials off the box. The gate is the envelope, not the vendor label.