Postmortem: Path Allowlist Matched the Client CWD, Not the Job Root

Remote path allowlists fail when job root and client cwd diverge. A reconstructed lab incident shows a fail-open matcher. The durable fix binds every write to a server-side job root.

Incident summary

An agent job trusted relative paths from a client allowlist. The matcher resolved those paths on the laptop. The remote host used a different workspace layout. Laptop-absolute entries did not exist on the server. The matcher treated a miss as allow. The job then wrote outside the repository tree.

This postmortem reconstructs the failure in a disposable lab. It is not a customer outage report. Figures below come only from that reconstruction.

Impact

The extra file landed in a CI artifact. Review tools scoped diffs to the git root. A sibling directory named scratch entered the tarball.

The lab fixture held no secrets at all. The same bug class can copy credentials. Path sandboxing remains a hard write gate. It must not be treated as review polish.

Timeline

Times below are lab clock values only. They are not a live incident clock.

1. T+0 min. A path allowlist is added on the laptop.
2. T+12 min. Agent jobs move to a remote workspace layout.
3. T+18 min. The client still sends laptop-absolute allowed paths.
4. T+19 min. The server finds none of those paths.
5. T+19 min. The matcher fail-opens and returns allow.
6. T+20 min. The agent writes ../scratch/notes.md.
7. T+27 min. CI packs scratch/ beside the repo.
8. T+41 min. A size check flags the sibling directory.

What broke

The allowlist used client cwd as the trust root. Remote jobs do not share that laptop filesystem. Absolute laptop paths do not exist on the job host.

The matcher treated "no matching rule" as allow. That default hid the host path mismatch. Relative writes then escaped with parent .. segments.

# reconstructed matcher — broken (lab example)
import os

def allowed(path, allowed_abs_from_client):
    abs_path = os.path.abspath(path)
    if not allowed_abs_from_client:
        return True  # fail-open
    return abs_path in allowed_abs_from_client

The abspath call follows the process cwd. On the server that cwd is the job workspace. The allowed set still listed the laptop paths. The allowed-path intersection was then fully empty. An empty match still meant full allow.

Contributing factors

• Client-absolute paths. Policy was computed only on the laptop.
• Fail-open default. Empty matches returned allow, not deny.
• Missing job root. The server never pinned JOB_ROOT.
• No canonicalization. Parent .. segments survived until write.
• Repo-scoped diffs. Review ignored sibling directories beside root.
• No write lock. Two jobs could interleave on one path.
• No content hash. The apply step did not verify bytes.
• Late CI size checks. The tarball was already built first.

Each factor is small in isolation. Together they turn a miss into a write. The durable fix has to close every gap.

Reproduction (lab only)

This section is a labeled lab-only procedure. Run these steps only in a throwaway directory. Do not point it at a real repository.

mkdir -p /tmp/agent-lab/repo /tmp/agent-lab/scratch
cd /tmp/agent-lab/repo
git init
mkdir -p src
printf 'ok\n' > src/app.py
git add src/app.py
git commit -m seed

Broken allowlist simulation:

python3 - <<'PY'
import os
client_allowed = {
    os.path.abspath(os.path.join("/Users/demo/project", "src/app.py"))
}
job_rel = "../scratch/notes.md"
job_path = os.path.abspath(os.path.join("/tmp/agent-lab/repo", job_rel))
print("client_allowed", client_allowed)
print("job_path", job_path)
print("match", job_path in client_allowed)
print("empty_means_allow", True)
print("escaped_outside_repo", job_path.startswith("/tmp/agent-lab/scratch"))
PY

Expected lab result: match prints false. empty_means_allow prints true. escaped_outside_repo prints true. That triple is the incident in miniature.

Apply the escaped write only inside the lab tree.

printf 'lab-only\n' > /tmp/agent-lab/scratch/notes.md
tar -C /tmp/agent-lab -cf /tmp/agent-lab/bundle.tar repo scratch
tar -tf /tmp/agent-lab/bundle.tar

The archive lists scratch/notes.md as expected. Reviewers watching only git diff never see it.

Durable fix

Pin all trust to the server job root. Resolve each candidate path against that root. Reject any path escape after that resolve. Fail closed on an empty write policy. Hash the new file bytes before apply. Lock the target path during the apply. Persist a write receipt for later CI.

1. Resolve against JOB_ROOT only

from pathlib import Path

class PathEscapeError(ValueError):
    pass

def resolve_in_root(job_root: Path, rel: str) -> Path:
    if rel.startswith("/") or rel.startswith("~") or rel.startswith("\\\\"):
        raise PathEscapeError(rel)
    root = job_root.resolve()
    target = (root / rel).resolve()
    try:
        target.relative_to(root)
    except ValueError as exc:
        raise PathEscapeError(rel) from exc
    return target

The client process never supplies the job root. The job host alone sets that value. Relative segments are joined only after that pin.

2. Fail closed and limit suffixes

ALLOWED_SUFFIXES = {".py", ".md", ".txt", ".toml", ".yml", ".yaml"}

def allow_write(job_root: Path, rel: str) -> Path:
    target = resolve_in_root(job_root, rel)
    root = job_root.resolve()
    if target.suffix.lower() not in ALLOWED_SUFFIXES:
        raise PermissionError(f"suffix blocked: {target.suffix}")
    if not target.is_relative_to(root):
        raise PathEscapeError(rel)
    return target

An empty write policy now denies writes. Unknown file suffixes now deny those writes. Absolute client paths now deny as well.

3. Lock and hash before apply

import hashlib
import os
import tempfile
import time
from contextlib import contextmanager

@contextmanager
def path_lock(target: Path, timeout=10.0):
    lock = target.with_name(target.name + ".write.lock")
    start = time.time()
    while True:
        try:
            fd = os.open(str(lock), os.O_CREAT | os.O_EXCL | os.O_WRONLY)
            os.write(fd, str(os.getpid()).encode())
            os.close(fd)
            break
        except FileExistsError:
            if time.time() - start > timeout:
                raise TimeoutError(str(lock))
            time.sleep(0.05)
    try:
        yield
    finally:
        lock.unlink(missing_ok=True)

def apply_patch(job_root: Path, rel: str, body: bytes, expected_sha=None):
    target = allow_write(job_root, rel)
    digest = hashlib.sha256(body).hexdigest()
    if expected_sha and digest != expected_sha:
        raise ValueError("content hash mismatch")
    target.parent.mkdir(parents=True, exist_ok=True)
    with path_lock(target):
        fd, tmp = tempfile.mkstemp(dir=str(target.parent), prefix=".part-")
        try:
            os.write(fd, body)
            os.fsync(fd)
            os.close(fd)
            os.replace(tmp, target)
        except Exception:
            try:
                os.close(fd)
            except OSError:
                pass
            Path(tmp).unlink(missing_ok=True)
            raise
    return {
        "path": str(target.relative_to(job_root.resolve())),
        "sha256": digest,
    }

os.replace keeps the apply atomic on the same filesystem. The exclusive file lock reduces lost write updates. The hash stops mixed bodies from two model runs.

4. Write a job receipt for CI

import json

def write_receipt(job_root: Path, records):
    receipt = job_root / ".agent-receipt.json"
    payload = {
        "job_root": str(job_root.resolve()),
        "writes": records,
    }
    receipt.write_text(json.dumps(payload, indent=2) + "\n")

CI must compare job_root with the checkout path. A root mismatch must fail the build. That check catches laptop-absolute policy reuse early.

python3 - <<'PY'
import json, os, pathlib, sys
root = pathlib.Path(os.environ["GITHUB_WORKSPACE"]).resolve()
data = json.loads(pathlib.Path(".agent-receipt.json").read_text())
if pathlib.Path(data["job_root"]).resolve() != root:
    sys.exit("job_root mismatch")
for item in data["writes"]:
    p = (root / item["path"]).resolve()
    p.relative_to(root)
PY

Test plan

Run these checks on every matcher change. Treat a skip as a failed gate.

1. Relative file inside the job root must allow.
2. A ../ escape must raise PathEscapeError.
3. An absolute laptop path must raise PathEscapeError.
4. A symlink pointing outside root must deny after resolve.
5. An empty allowlist must deny, never allow.
6. Suffix .lock or an empty suffix must deny.
7. A hash mismatch must keep the previous file.
8. Concurrent writers must wait on the lock or fail.
9. Receipt job_root must equal the CI checkout path.

from pathlib import Path
import hashlib
import tempfile

def test_escape_and_hash():
    with tempfile.TemporaryDirectory() as raw:
        root = Path(raw) / "repo"
        root.mkdir()
        (root / "src").mkdir()
        try:
            resolve_in_root(root, "../scratch/notes.md")
        except PathEscapeError:
            pass
        else:
            raise AssertionError("escape allowed")
        body = b"print(0)\n"
        rec = apply_patch(root, "src/app.py", body)
        assert rec["sha256"] == hashlib.sha256(body).hexdigest()
        try:
            apply_patch(root, "src/app.py", b"x", expected_sha=rec["sha256"])
        except ValueError:
            pass
        else:
            raise AssertionError("hash mismatch accepted")

# labeled lab test — run after pasting the helpers

Decision table

Signal	Action
Client cwd equals job root	Local resolve is acceptable
Job runs on another host	Ignore client absolute paths
Allowlist empty	Deny all writes
Path contains .. after join	Deny after resolve
Two jobs share one file	Lock or serialize
Receipt root != checkout	Fail CI
Model output lacks a hash	Refuse apply
Suffix not in the allow set	Deny

Remote jobs make the mismatch visible

Remote workspaces use paths the laptop never sees. That path gap is the core bug. The same gap is also a teacher. A policy that names /Users/... cannot match /var/jobs/....

MonkeyCode is an open-source coding-agent project today. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The project provides free model access and a free server option. Those two availability notes are the only product claims in this article.

Free model access can draft a candidate diff. The free server option can host the job root. The path matcher still belongs in CI. Language models do not enforce host filesystem policy.

Teams that need a remote job host may try that free server option. Keep this receipt check inside the CI job.

Limitations

The matcher does not replace human code review. It also does not parse patch semantics. Path.resolve() can still surprise on some network mounts. Windows drive letters still need extra path tests. The file lock is not cluster-safe yet. Use a real lock service across hosts.

SHA-256 of file bytes is not a capability proof. The digest only detects mixed concurrent file writes. It does not prove the model was correct.

This user-space gate does not block reads. It does not strip secrets already in the tree. It does not rate-limit the model either.

Who should not use this approach

Skip this workflow if the agent must write outside the repo. Skip it if the host lacks O_EXCL. Skip it if policy requires a kernel sandbox. This design remains a user-space gate only. It is not a seccomp sandbox profile. It is not a virtual machine boundary. It is not a legal compliance control layer.

Do not use the lab commands on a shared machine. Do not copy the broken matcher into production. Do not fail-open just to preserve speed.

Close

Bind every write to the job root. Fail closed on any missing write policy. Hash the bytes and lock the path. Keep the write receipt inside the CI job. Client cwd is not a security boundary.