DEV Community

Sam Yang
Sam Yang

Posted on

A Clean Exit Is Not a Patch: A Myth-Busting FAQ

The pull request opened with a cheerful summary, a green checkbox emoji, and a complete absence of changed files in the GitHub diff. The author pasted a chat in which a coding agent invoked an MCP write tool and printed JSON with ok set to true. Reviewers asked for the actual diff, yet the branch still pointed at the same tree hash as main. A quiet process exit had been treated as a mergeable artifact, which it never was.

That pattern is getting louder as teams bolt MCP servers onto agent loops and then read the loop as if it were a build system. Public threads this week keep arguing that many so-called agents are mostly control flow wearing a tool schema, and the useful part of that claim is operational. A schema is a handshake between a model and a process. A patch is a recorded mutation of one worktree, plus a test receipt another machine can replay without trusting the chat.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. The checks below mention MonkeyCode only because its free model access and free server option give you a second machine that is not your laptop. Strip those product lines out and the receipt protocol remains the whole point of the article.

Myth: structured tool success means the file landed in the clone you will merge. The evidence people cite is almost always a pretty JSON blob, because JSON looks like an API and APIs feel like side effects. The corrected model is narrower and less comforting: a tool result is a message about what some process claimed, not a git object. If the write happened on a rented workspace, a container that later recycled, or a path the agent invented, your clone can stay untouched while the transcript still reads like a victory.

You can watch the gap with ordinary git, which does not care how confident the model sounded. After any agent session that claims a write, capture identity before you argue about quality.

#!/usr/bin/env bash
# receipt_before.sh — run in the repo you actually intend to merge
set -euo pipefail
git rev-parse HEAD
git rev-parse --show-toplevel
git status --porcelain=v1
git diff --stat
find . -name '*.py' -print0 | sort -z | xargs -0 sha256sum | sha256sum
Enter fullscreen mode Exit fullscreen mode

If HEAD, the porcelain status, and the content fingerprint did not move, the handshake did not produce a patch in this tree. Teams argue about style at that point, which is wasted heat, because there is nothing to review except prose. Treat the chat as a design note and start the loop again with an explicit output path inside the repository you own.

Myth: MCP grounding stops hallucination at the tool boundary. The repeated claim is that once a model can list tools, it cannot invent the next step, because the catalog is a contract. Catalogs are closer to menus. A diner can still order a dish the kitchen does not serve, and a model can still emit a tool name, a relative path, or a success payload that no server executed. MCP reduces typing mistakes. It does not turn a probabilistic decoder into a transactional filesystem.

A small Python guard makes that distinction visible without requiring a full agent framework. The script below refuses to honor a tool payload unless three boring facts agree: the name is in an allowlist, the path stays inside the repo, and the bytes on disk change.

# tool_receipt.py
from __future__ import annotations

import hashlib
import json
from pathlib import Path

ALLOWED = {"write_file", "run_tests"}

def fingerprint(root: Path) -> str:
    h = hashlib.sha256()
    for path in sorted(p for p in root.rglob("*") if p.is_file()):
        rel = path.relative_to(root).as_posix()
        if rel.startswith(".git/"):
            continue
        h.update(rel.encode())
        h.update(path.read_bytes())
    return h.hexdigest()

def accept(payload: dict, root: Path) -> dict:
    before = fingerprint(root)
    name = payload.get("name")
    if name not in ALLOWED:
        return {"accepted": False, "reason": "name-not-allowlisted", "before": before}
    rel = payload.get("path", "")
    target = (root / rel).resolve()
    if root.resolve() not in target.parents and target != root.resolve():
        return {"accepted": False, "reason": "path-escapes-root", "before": before}
    if name == "write_file":
        target.parent.mkdir(parents=True, exist_ok=True)
        target.write_text(payload.get("contents", ""), encoding="utf-8")
    after = fingerprint(root)
    return {
        "accepted": before != after,
        "reason": "mutated" if before != after else "no-byte-change",
        "before": before,
        "after": after,
    }

if __name__ == "__main__":
    sample = json.loads(Path("tool_claim.json").read_text())
    print(json.dumps(accept(sample, Path(".")), indent=2))
Enter fullscreen mode Exit fullscreen mode

Label the sample payload as an unexecuted fixture unless you generated it from a real log. The interesting failure is not a Python exception. The interesting failure is accepted: false with no-byte-change, which is what you get when a model narrates a write and the disk stays still.

Myth: when the agent loop goes silent, the job is complete. Silence is an exit condition for a process, not a definition of done for a change. Loops stop because they hit a turn limit, a token budget, a cancelled stream, a parser error, or a model that learned to say "done" as a polite closer. None of those events produce a JUnit file. If you cannot point to a command, an exit code, and a log path, you watched a conversation end.

A reproducible minimum looks like this, and it is deliberately ugly.

#!/usr/bin/env bash
# receipt_after.sh — the agent does not get to skip this
set -euo pipefail
OUT="${RECEIPT_DIR:-.receipts}/$(date -u +%Y%m%dT%H%M%SZ)"
mkdir -p "$OUT"
{
  echo "host=$(hostname)"
  echo "pwd=$(pwd)"
  echo "head=$(git rev-parse HEAD)"
  echo "branch=$(git rev-parse --abbrev-ref HEAD)"
} > "$OUT/identity.txt"
git status --porcelain=v1 > "$OUT/status.txt"
git diff > "$OUT/worktree.diff"
set +e
python -m pytest -q --junitxml="$OUT/junit.xml"
echo "pytest_exit=$?" >> "$OUT/identity.txt"
set -e
sha256sum "$OUT"/* > "$OUT/MANIFEST.sha256"
echo "$OUT"
Enter fullscreen mode Exit fullscreen mode

The directory is the artifact. Chat text can go in the ticket as color commentary. If identity, diff, and junit are missing, you do not have a failing test or a passing test. You have a story about a loop that got tired.

Myth: two agreeing tool calls are a replica. Developers repeat this after an agent "double-checks" its own write by reading the file back through another tool. That is one process talking to itself, which is closer to a person rereading a draft than to a second machine. Replication starts when a different checkout, preferably on a different host, applies the diff and reruns the same command line. Agreement inside one session is echo.

This is the narrow place where a free remote server is relevant rather than decorative. Run the receipt on your laptop, then run the same scripts on a throwaway host that never held the original chat. MonkeyCode currently offers free model access and a free server option that can play that second-host role if you already work there. The product is not the proof. The proof is that the fingerprint and the pytest exit code match across machines, or that they fail in the same named way.

Myth: a live free workspace is source of truth while the session remains open. Live workspaces are scratch paper. They vanish, migrate, or get reused, and the reuse problem is worse when many users share a free pool. If the only copy of the change lives in that scratch tree, you do not have a change. You have a lease. Copy the diff and the receipt off the box before you celebrate, the same way you would copy notes off a whiteboard before the next meeting starts.

The corrected mental model fits in one sentence that teams can put in a contributing guide. An agent may propose work; only a hashed diff plus a replayable test log may claim that work happened. Everything else is drafting. That model stays useful when the tools are local, remote, paid, free, MCP-backed, or glued together with a shell loop that would look embarrassing in a architecture diagram.

Limitations follow from the same model and should be stated without theater. The receipt scripts do not detect semantic regressions, flaky tests, or a model that edited the tests until they smiled. They also do not prove that a free server is isolated, private, or appropriate for secrets, customer data, or production credentials. People who need a hardened runner, an air-gapped build, or a compliance trail should keep using the same CI they already trust and should not substitute a chat loop for that CI. People who only want a rubber stamp for an unreviewed dump of model output should not use this protocol either, because it will keep saying no.

If you already keep a spare remote session for agent work, run the receipt scripts there once and store the JSON beside the diff before anyone writes "LGTM" under a transcript.

Top comments (0)