DEV Community: Riley Xu

Migration Diary: Freeze a Public-Surface Snapshot Before You Leave a Paid Coding Agent

Riley Xu — Mon, 21 Sep 2026 13:49:21 +0000

The public surface of your repository is the cutover artifact you should freeze before any paid agent seat expires. Chat logs and clever prompts will not tell you whether the next host renamed an exported function overnight. A snapshot of CLI flags, HTTP paths, and Python exports gives you a meter that survives the vendor tab. This diary walks you through packing that snapshot, gating applies against it, and recording leftovers the product still owns.

Why the leftover is a surface map, not another transcript

Paid coding agents like to touch whatever file seems related to the latest prompt you typed. That habit is tolerable while a review UI sits between the model and your default branch. It becomes expensive when you move the loop onto a free host that writes straight into the tree. You need a frozen map of names callers already depend on, stored next to the code, not in a chat.

Informal memory fails as soon as the model generation changes under the same product name. A green unit test can still hide a renamed CLI flag that a shell script in another repository calls. The snapshot does not claim the agent wrote good code, and it should not be asked to. It only claims that public names did not vanish unless you already said they could vanish.

What you freeze, and what you ignore

Capture only names a downstream caller could import, invoke, or hit without opening a private helper. Leave private functions, test doubles, and generated caches out of the snapshot so noise does not hide breaks. Record the extractor command beside the snapshot so a later machine can rebuild the same map. If you cannot name that command, you do not have a snapshot yet, only a feeling about stability.

Do not freeze entire files as the surface, because a file-level hash will fire on comments. You will start ignoring the gate after a week of comment-only failures on otherwise harmless patches. Do not freeze vendor tool names either, because those names die with the paid subscription. The finished pack should make sense to a teammate who never opened the paid product.

Cutover plan

Work through these steps before you cancel, because the old review UI will not export this map for you. Keep the outputs in git so the destination host cannot quietly rewrite the contract during the first week. If a step fails, you still have a paid review screen, which is the only reason to wait.

Walk the last merged agent patches and write down every public name those patches were allowed to change.
Run the extractor on the current default branch and commit the JSON snapshot beside the code.
Install the apply gate as a local command that reads unified diffs before any host writes files.
Replay one known-good patch and one known-bad rename so you trust the gate before canceling.
Move generation to the free host only after the gate has failed a rename you planted by hand.
File a leftovers note for vendor-side refactors you never extracted, because those names are already gone.

Stop at step four if the planted rename still applies cleanly, and do not cancel while that hole remains. You do not have a gate yet, and a free host will not invent one. Fix the extractor or the regex before you spend another hour talking to any model.

Extractor you can run tonight

The script below is a proposal for Python modules that declare public names at module scope. Treat this extractor as unexecuted until you run it against your own repository tree. It ignores names that start with an underscore, which is a convention, not a security boundary.

#!/usr/bin/env python3
"""Dump a coarse public-surface snapshot. Proposal: adapt allowlists before you trust it."""
from __future__ import annotations

import ast
import json
import sys
from pathlib import Path

ROOT = Path(__file__).resolve().parents[1]
OUT = ROOT / "fixtures" / "public-surface.json"
SKIP_PARTS = {".git", ".venv", "venv", "node_modules", "fixtures", "tests"}


def is_skipped(path: Path) -> bool:
    return any(part in SKIP_PARTS for part in path.parts)


def public_functions(path: Path) -> list[str]:
    tree = ast.parse(path.read_text(encoding="utf-8"), filename=str(path))
    names: list[str] = []
    for node in tree.body:
        if isinstance(node, ast.Assign):
            for target in node.targets:
                if isinstance(target, ast.Name) and target.id == "__all__":
                    if isinstance(node.value, (ast.List, ast.Tuple)):
                        for elt in node.value.elts:
                            if isinstance(elt, ast.Constant) and isinstance(elt.value, str):
                                names.append(elt.value)
        if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef)):
            if not node.name.startswith("_"):
                names.append(node.name)
    return sorted(set(names))


def main() -> int:
    surface: dict[str, list[str]] = {}
    for path in ROOT.rglob("*.py"):
        if is_skipped(path):
            continue
        rel = str(path.relative_to(ROOT))
        names = public_functions(path)
        if names:
            surface[rel] = names
    OUT.parent.mkdir(parents=True, exist_ok=True)
    payload = {
        "extractor": "scripts/snapshot_public.py",
        "files": surface,
    }
    OUT.write_text(json.dumps(payload, indent=2, sort_keys=True) + "\n", encoding="utf-8")
    print(f"wrote {OUT} files={len(surface)}")
    return 0


if __name__ == "__main__":
    sys.exit(main())

mkdir -p scripts fixtures
python3 scripts/snapshot_public.py
git add fixtures/public-surface.json scripts/snapshot_public.py
git diff --cached -- fixtures/public-surface.json | head

You should commit the snapshot on a quiet branch that no coding agent may push. You want the first diff after cutover to be about product names, not about formatter churn. Tag that commit so you can reset the snapshot without digging through agent noise later.

Apply gate that reads the snapshot

The apply gate is the leftover the paid review UI used to give you for free. It reads a unified diff on stdin and then refuses patches that drop frozen snapshot names. It also writes a JSON verdict you can store beside the case for later audit. You should treat the following script as a local tool, not as a hosted policy engine.

#!/usr/bin/env python3
"""Refuse diffs that remove frozen public names. Proposal for a local apply gate."""
from __future__ import annotations

import json
import re
import sys
from pathlib import Path

ROOT = Path(__file__).resolve().parents[1]
SNAP = json.loads((ROOT / "fixtures" / "public-surface.json").read_text(encoding="utf-8"))
FILE_RE = re.compile(r"^diff --git a/(.+) b/(.+)$")
DEL_RE = re.compile(
    r"^-\s*(?:async\s+)?def\s+([A-Za-z_][A-Za-z0-9_]*)|^-\s*class\s+([A-Za-z_][A-Za-z0-9_]*)"
)


def verdict(diff_text: str) -> dict:
    dropped: list[str] = []
    current_b = ""
    for line in diff_text.splitlines():
        header = FILE_RE.match(line)
        if header:
            current_b = header.group(2)
            continue
        if not current_b or current_b not in SNAP.get("files", {}):
            continue
        match = DEL_RE.match(line)
        if not match:
            continue
        name = match.group(1) or match.group(2)
        if name in SNAP["files"][current_b]:
            dropped.append(f"{current_b}:{name}")
    return {
        "ok": not dropped,
        "dropped_public_names": dropped,
        "action": "apply" if not dropped else "hold",
    }


def main() -> int:
    diff_text = sys.stdin.read()
    row = verdict(diff_text)
    Path("fixtures/last-apply-verdict.json").write_text(
        json.dumps(row, indent=2) + "\n", encoding="utf-8"
    )
    print(json.dumps(row))
    return 0 if row["ok"] else 2


if __name__ == "__main__":
    sys.exit(main())

git diff HEAD | python3 scripts/apply_gate.py; echo exit:$?
# Plant a bad rename, then confirm the gate holds the patch.
git diff HEAD | python3 scripts/apply_gate.py || echo "hold as expected"

A hold is not proof of good design, and it is not a substitute for a real review. It is only proof that the next host did not delete a name you already published. You still review behavior, tests, and security the way you did before the agent existed.

Decision table for this leftover

Paid-product leftover	Freeze into the snapshot?	Reason you should care during cutover
Exported function or class name	Yes	Callers break without a compile error in other languages.
CLI flag or subcommand	Yes	Shell scripts fail late, usually in someone else's job.
HTTP path or public schema field	Yes	Mobile and partner clients will not read your chat log.
Private helper renamed inside a file	No	That churn should not block an otherwise valid patch.
Vendor chat title or prompt slug	No	You cannot replay those strings after billing stops.
Comment-only edits	No	Hashes on whole files will train you to skip the gate.
Tool traces from the old agent	No	They are not your public surface, and they are not portable.

Print the table into the leftovers file if your team argues about what counts as public. Inventory arguments stay cheaper than the incident that follows a silent rename into production clients. Revisit the table when you add a second language, because this extractor only reads Python.

Where a free host participates, and what it must not change

After the snapshot is in git, you still need a host that can propose patches without the paid seat. Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode is relevant here only as a destination with free model access and a free server option. You point the same apply gate at patches from that host, and you keep the snapshot file unchanged.

You should not record invented quotas, model names, or hardware claims against that option. Those numbers are not part of this diary, and they are not a substitute for last-apply-verdict.json. If a generated patch drops a frozen name, you hold the apply, regardless of which host wrote the hunk.

Leftovers the snapshot will never contain

The paid product still owns prompts you never copied, including the ones that already renamed things last month. Those renames will not appear as deletions in a fresh diff, because the default branch already absorbed them. Write them into leftovers as narrative, with the old name and the new name, so the next host does not thrash back.

Browser review clicks, team-shared threads, and vendor caches of rejected patches are also gone at cancel time. You will not reconstruct a rejected rename after cancel from an empty leftovers markdown file. Spend thirty careful minutes on that file while you still can open the old UI, then stop hoping the export button exists.

Limitations, and who should skip this

This gate is coarse, and it will miss several export styles your codebase may already use. It parses unified diffs with regular expressions, and it will miss metaprogrammed exports, star imports, and HTTP paths declared in YAML. A green verdict does not mean the patch is correct, safe, or even worth merging today. Free model access and a free server option help you regenerate candidate patches after you cancel.

They do not create an SLA, a quota you can quote, or a stable model identity you can cite. Skip this workflow when you ship no public names, such as a private notebook that never grows an API. Skip it when compliance forbids sending repository slices to any remote host, including a free one. Skip it when a published OpenAPI job already fails on breaking paths, and wire that job instead.

Teams that need guaranteed capacity should stay on a contract they can cite, not on leftover free access. The paid product can disappear tomorrow, and your callers will not care which model last touched the tree. Keep the snapshot in git, keep the gate on your path, and treat every new host as an untrusted patch source. Anything the old vendor still holds belongs in leftovers, not in a hopeful prompt you will not be able to open.

Migration Diary: Drain the Transcript Archive Before You Leave a Paid Coding Agent

Riley Xu — Sun, 20 Sep 2026 11:43:58 +0000

You should drain the transcript archive before you cancel a paid coding agent, because the leftovers live in those chats. Git keeps merged code, but it does not keep rejected paths, named constraints, or half-finished repairs. Those fragments sit in agent threads, and they vanish when the vendor session finally dies. This diary treats transcript drain as a required cutover step rather than optional nostalgia for old chats.

What the paid seat was actually holding

Paid coding agents store more than the patches you later merge into the default branch of a repo. They also store file lists you never committed, command outcomes you never logged, and design vetoes you never wrote down. When you export nothing, the next loop on cheaper infrastructure starts blind and repeats expensive mistakes. You then spend scarce budget rediscovering constraints that a paid agent thread had already settled for you.

A useful drain is not a full dump of every retry, joke, or stack trace from those sessions. You want a compact leftover ledger that lists paths, decisions, open defects, and commands that already failed. That ledger becomes the first context pack for whatever loop you run after cancel day. Without that ledger, free infrastructure only looks cheaper because it is silently repeating already billed work.

Cutover plan

Keep the paid seat alive until the leftover ledger survives a dry run against your tree. Work through the following numbered steps in order, and do not skip the redaction pass.

Export every agent thread that touched the target repository during the last full billing cycle.
Normalize those exports into UTF-8 markdown or JSONL so a local parser can read them.
Extract mentioned paths, decisions, failed commands, and unresolved todos into one leftover markdown file.
Redact tokens, hostnames, and private URLs before that file enters git or shared chat.
Replay the leftover file against a cheap loop and mark each item done, skipped, or blocked.

The export format will differ by vendor, so do not wait for a perfect official archive API. Copy visible threads into transcripts/raw/ as .md or .jsonl files, and record the export date in the filename. If the vendor offers a zip of chats, drop that zip in the same folder and unpack it once. Name those files after the repository and the thread title, not after the model nickname.

Build a leftover ledger from raw transcripts

The helper below is a proposed local script, not a claim about any vendor's private export schema. Save it as drain_transcripts.py and point that script at a directory of UTF-8 text files. It walks those files, pulls path-like tokens, decision sentences, failed command lines, and TODO markers, then writes CUTOVER_LEFTOVERS.md. Treat every extracted line as a candidate, then delete anything the regex should not have kept.

#!/usr/bin/env python3
"""Drain coding-agent transcripts into a leftover ledger.

Proposed local helper. Adjust regexes to match your export format.
"""
from __future__ import annotations

import re
import sys
from pathlib import Path

PATH_RE = re.compile(
    r"(?:^|[\s`'\"(])([A-Za-z0-9_./-]+\.(?:py|ts|tsx|js|go|rs|java|rb|md|toml|yml|yaml|json|sql))",
)
DECISION_RE = re.compile(
    r"^\s*(?:we(?:'ll| will)|decided|do not|don't|use .+ instead|avoid|must not)\b.*",
    re.I,
)
FAIL_RE = re.compile(
    r"^\s*(?:\$ |# )?(?:npm|pnpm|yarn|pip|pytest|cargo|go|make|git)\b.*",
    re.I,
)
TODO_RE = re.compile(r"\b(TODO|FIXME|BLOCKED|follow-?up)\b[:\s].*", re.I)
SECRET_RE = re.compile(
    r"(sk-[A-Za-z0-9]+|ghp_[A-Za-z0-9]+|AKIA[0-9A-Z]{16}|Bearer\s+[A-Za-z0-9._-]+)",
)


def redact(text: str) -> str:
    return SECRET_RE.sub("[REDACTED]", text)


def drain(raw_dir: Path) -> str:
    paths, decisions, fails, todos = set(), [], [], []
    for p in sorted(raw_dir.rglob("*")):
        if p.suffix.lower() not in {".md", ".txt", ".jsonl"}:
            continue
        text = redact(p.read_text(encoding="utf-8", errors="replace"))
        for line in text.splitlines():
            for m in PATH_RE.finditer(line):
                paths.add(m.group(1))
            if DECISION_RE.search(line):
                decisions.append(f"- {line.strip()[:240]} ({p.name})")
            if FAIL_RE.search(line) and re.search(r"error|fail|exit 1|denied", line, re.I):
                fails.append(f"- `{line.strip()[:200]}` ({p.name})")
            if TODO_RE.search(line):
                todos.append(f"- {line.strip()[:240]} ({p.name})")
    parts = [
        "# Cutover leftovers",
        "",
        "## Paths mentioned",
        *[f"- `{x}`" for x in sorted(paths)] or ["- none"],
        "",
        "## Decisions and vetoes",
        *(decisions[:80] or ["- none"]),
        "",
        "## Commands that already failed",
        *(fails[:80] or ["- none"]),
        "",
        "## Unresolved follow-ups",
        *(todos[:80] or ["- none"]),
        "",
        "## Replay status",
        "| Item | Owner | Status | Notes |",
        "| --- | --- | --- | --- |",
        "| (fill from lists above) | | done/skipped/blocked | |",
    ]
    return "\n".join(parts) + "\n"


def main() -> None:
    raw = Path(sys.argv[1] if len(sys.argv) > 1 else "transcripts/raw")
    out = Path(sys.argv[2] if len(sys.argv) > 2 else "CUTOVER_LEFTOVERS.md")
    out.write_text(drain(raw), encoding="utf-8")
    print(f"wrote {out} from {raw}")


if __name__ == "__main__":
    main()

Run the helper like this after you fill the transcripts/raw directory with exported threads.

python3 drain_transcripts.py transcripts/raw CUTOVER_LEFTOVERS.md
# Proposed check: fail the commit if whitespace or conflict markers slipped in.
git add CUTOVER_LEFTOVERS.md
git diff --check

Read the generated markdown before you commit it, because regex extraction is greedy and will catch noise. Delete anything that looks like a customer name, a hostname, or an access token that the redactor missed. Then fill the replay table so cancel day has a queue instead of a blank prompt.

Decision table for each leftover

Use this table while you walk the generated ledger so trivia never enters the next agent loop. It keeps rejected ideas available as vetoes without turning ordinary chat noise into fake requirements. Fill every row before you schedule cancel day, including the rows you already intend to discard.

Leftover type	Keep in git?	Replay on the next loop?	Discard when
Merged patch already on main	No, git already has it	No	Hash is on the default branch
Rejected approach with a named reason	Yes, as a veto note	Only if someone reopens it	The constraint is documented elsewhere
Failed command with a non-obvious error	Yes, in the fail list	Yes, after the environment is rebuilt	You reproduced a clean pass
Open TODO that still matches the tree	Yes	Yes	The file no longer exists
Secrets, cookies, private URLs	Never	Never	Immediately after redaction
Banter, retries, token-usage chatter	No	No	After the drain script runs

You should treat "keep in git" as a documentation choice, not as a backup of the vendor's chat product. The leftover file is a cutover artifact, and it should shrink every week until it is empty. If a row cannot be classified, mark it blocked and leave the paid seat up for that thread only.

Replay without the paid seat

Once the leftover ledger exists, you can replay open items without keeping a paid coding-agent subscription on the critical path. MonkeyCode is one option for that replay: it provides free model access and a free server option you can point at the same ledger. Disclosure: This article was prepared as part of MonkeyCode's product outreach. You still run redaction first, because a free server does not make a leaked secret cheaper.

A practical replay prompt should stay boring on purpose so the new loop cannot wander. Paste the relevant leftover section, the file contents you already listed, and a request for a patch that honors the veto notes. Do not paste the entire raw archive into the new loop, because drain exists to prevent that. Keep each replay focused on a single leftover row so later failures remain easy to attribute.

You are continuing a cutover, not starting a greenfield task.
Honor every veto in the leftovers. Do not reopen rejected libraries.
Work only on the paths listed. Return a unified diff and a test command.
Leftovers:
<paste one unresolved item>

If the free loop cannot finish an item, write blocked in the table and keep a narrow paid thread for that item alone. Canceling every paid thread at once is exactly how leftover context dies during a rushed cutover. Split the cancel work by leftover status, not by calendar pressure or the invoice date.

Limitations

This drain does not reconstruct tool traces, hidden system prompts, or any vendor-side index state. Regexes miss fenced paths, screenshots, and binary attachments, and they over-capture words that look like files. The script is not a compliance archive, and it is not a substitute for your company's retention policy. Vendor export buttons change without notice, so verify the current path on the vendor's own documentation. Do not treat a click path from this diary as a durable instruction for any vendor UI.

You also cannot prove behavioral parity with a leftover list alone, even a tidy one. Pair this ledger with tests you already trust, rather than treating extracted sentences as a specification. If two transcripts disagree, believe the repository and the test suite, not the more confident chat. When a leftover cites a file that no longer exists, drop the row instead of resurrecting dead paths.

Who should not use this approach

Do not use this drain if your transcripts contain regulated data you are not allowed to copy onto a laptop. Do not use it if your employer forbids export of vendor chats, even for migration. Skip this drain when the repository is a short spike with no rejected approaches worth keeping. Teams that already keep architecture decision records and a failing-test list may only need a secret scan, not a full parser.

If you cannot redact with confidence, do not move the files to any shared server, free or paid. The cheapest leftover you will ever manage is the sensitive line that you never copy elsewhere. Encrypted local storage is acceptable during drain week, but unredacted cloud sync is not acceptable.

What cancel day should look like

Cancel day should be a checklist, not a feeling that the new loop merely seems fine. You want an empty replay table, a redacted leftover file, and a vendor export that you could reconstruct from git. Keep one backup of the raw transcripts in an encrypted volume until the first week of cheap-loop work lands on main. Then destroy the raw export, because the leftover value has already been distilled into git.

If the leftover ledger already sits in git, replay blocked rows on a free loop first. Keep the paid seat only for those rows that still fail after the free replay.

Migration Diary: Rebuild the Diff Budget Before You Leave a Paid Coding Agent

Riley Xu — Sat, 19 Sep 2026 09:49:01 +0000

You should pin a numeric diff budget before you leave any paid coding agent. Paid tools often apply large patches in one turn without exposing the true blast radius. A self-hosted loop will hide that radius even less, because nobody ships a polished review pane. This diary freezes that budget, scores leftover paid diffs, and refuses oversized patches after cutover.

The failure you will hit on day one

You will paste a leftover transcript into the new loop and watch it rewrite half a package. The paid agent trained you to accept that volume because the UI grouped files into a single approve button. After cutover, you own every hunk, every import, and every test file the model touched. Without a numeric budget in the repo, those vibe-sized patches become your default engineering process.

Recent community threads keep splitting casual generation from engineering practice, and that split is useful here. Engineering, during a cutover, means you can reject a change for size before you debate style. You need numbers for files, hunks, and net lines, not a feeling that the patch looks focused. The paid product will not export those numbers in a durable form you can keep after canceling.

What a diff budget actually records

A budget is not a linter, and it is not a model quality score either. It is a hard ceiling on how much tree mutation one agent turn may propose. You record four integers: max files, max hunks, max lines added, and max lines deleted. You also record path prefixes that stay forbidden, because leftover paid runs love to rewrite lockfiles.

Store the budget beside the repository, not inside a vendor dashboard you are about to abandon. Treat formatter-only files as a separate class so a noisy lockfile cannot burn the whole budget. Keep the budget file committed so every teammate scores leftovers against the same numeric ceilings.

{
  "version": 1,
  "max_files": 6,
  "max_hunks": 18,
  "max_lines_added": 180,
  "max_lines_deleted": 120,
  "forbidden_prefixes": [
    "vendor/",
    "dist/",
    "pnpm-lock.yaml",
    "package-lock.json"
  ],
  "ignore_globs": ["**/*.snap", "**/*.min.js"]
}

Label the JSON above as a proposed contract, not a benchmark from a production fleet. Tune those integers against leftover paid diffs rather than against a blog post default. Commit the file as .diff-budget.json so later hosts inherit the same ceilings without a dashboard login.

Step 1: Inventory leftover paid patches

Export every leftover apply from the paid tool before you cancel the remaining seat. Prefer unified diffs stored on disk over screenshots copied from a vendor review pane. Name files with the original turn id so you can replay arguments later without the vendor.

mkdir -p leftovers/paid-diffs
# Proposed local layout; replace the export step with whatever your vendor actually provides.
find leftovers/paid-diffs -name '*.diff' | wc -l
git diff --numstat HEAD > leftovers/working-tree.numstat

If the vendor only keeps chat transcripts, reconstruct diffs by copying the last applied tree into a throwaway git worktree. You should not start the free-stack cutover until this directory is non-empty or you have proven there were no leftovers. Empty inventories are usually a smell, because paid agents almost always leave a trail.

Step 2: Parse unified diffs into budget counters

Write a small parser so you never have to eyeball another 400-line leftover patch. The script below is a proposed local harness that understands unified diffs only and ignores binary markers. Put it beside the budget file and treat exit status one as a hard apply refusal.

#!/usr/bin/env python3
"""Proposed scorer: pin this beside .diff-budget.json. Not a vendor integration."""
from __future__ import annotations

import json
import pathlib
import re
import sys
from dataclasses import dataclass, field

HUNK_RE = re.compile(r"^@@ ")
FILE_RE = re.compile(r"^diff --git a/(.+) b/(.+)$")


@dataclass
class DiffStats:
    files: set[str]
    hunks: int = 0
    added: int = 0
    deleted: int = 0
    forbidden: list[str] = field(default_factory=list)


def parse_unified(text: str) -> DiffStats:
    stats = DiffStats(files=set())
    current = None
    for line in text.splitlines():
        file_match = FILE_RE.match(line)
        if file_match:
            current = file_match.group(2)
            stats.files.add(current)
            continue
        if line.startswith("+++ ") or line.startswith("--- "):
            continue
        if HUNK_RE.match(line):
            stats.hunks += 1
            continue
        if current is None:
            continue
        if line.startswith("+") and not line.startswith("+++"):
            stats.added += 1
        elif line.startswith("-") and not line.startswith("---"):
            stats.deleted += 1
    return stats


def load_budget(path: pathlib.Path) -> dict:
    return json.loads(path.read_text(encoding="utf-8"))


def violates(stats: DiffStats, budget: dict) -> list[str]:
    reasons = []
    for path in sorted(stats.files):
        for prefix in budget["forbidden_prefixes"]:
            if path.startswith(prefix) or path.endswith(prefix):
                stats.forbidden.append(path)
    if len(stats.files) > budget["max_files"]:
        reasons.append(f"files {len(stats.files)} > {budget['max_files']}")
    if stats.hunks > budget["max_hunks"]:
        reasons.append(f"hunks {stats.hunks} > {budget['max_hunks']}")
    if stats.added > budget["max_lines_added"]:
        reasons.append(f"added {stats.added} > {budget['max_lines_added']}")
    if stats.deleted > budget["max_lines_deleted"]:
        reasons.append(f"deleted {stats.deleted} > {budget['max_lines_deleted']}")
    if stats.forbidden:
        reasons.append("forbidden " + ",".join(stats.forbidden))
    return reasons


def main(argv: list[str]) -> int:
    budget = load_budget(pathlib.Path(argv[1]))
    target = pathlib.Path(argv[2])
    failed = 0
    paths = [target] if target.is_file() else sorted(target.glob("*.diff"))
    for diff_path in paths:
        stats = parse_unified(diff_path.read_text(encoding="utf-8", errors="replace"))
        reasons = violates(stats, budget)
        status = "FAIL" if reasons else "PASS"
        if reasons:
            failed += 1
        print(
            f"{status} {diff_path} files={len(stats.files)} "
            f"hunks={stats.hunks} +{stats.added} -{stats.deleted}"
        )
        for reason in reasons:
            print(f"  - {reason}")
    return 1 if failed else 0


if __name__ == "__main__":
    sys.exit(main(sys.argv))

Run it against the leftover directory before you point any new agent at the tree.

python3 score_diff_budget.py .diff-budget.json leftovers/paid-diffs

You now have a leftover report that does not depend on the paid review UI. Keep the FAIL lines because they are the patches that trained you to accept too much. Commit the report beside the diffs so the cutover plan cites evidence instead of memory.

Step 3: Recalibrate ceilings from leftovers, not from hope

Do not copy the paid agent's largest leftover patch and treat it as your new normal. Sort FAIL rows and look for a cluster of PASS sizes that still shipped useful work. Set max_files just above that cluster, then force oversized work to split across turns. If every leftover fails, your paid workflow was already a rewrite machine, and cutover will hurt until you split tasks.

python3 score_diff_budget.py .diff-budget.json leftovers/paid-diffs \
  | awk '/^FAIL/ {fail++} /^PASS/ {pass++} END {print "pass=" pass, "fail=" fail}'

Write a one-page note that lists which leftover tasks must be split before cutover. That note belongs in the cutover plan, and it should not sit as optional documentation. Future you will otherwise reopen the paid seat just to finish the big refactor.

Step 4: Gate apply on the new loop

Wire the scorer in front of patch apply, even if apply is just git apply. A budget that only runs against leftovers will rot after the first quiet week. The snippet below is proposed glue you can drop in front of a local runner.

#!/usr/bin/env bash
set -euo pipefail
DIFF_FILE="${1:?usage: gate-apply.sh proposed.diff}"
python3 score_diff_budget.py .diff-budget.json "$DIFF_FILE"
git apply --check "$DIFF_FILE"
git apply "$DIFF_FILE"

If the new loop streams a patch, write it to a temp file and score it before git apply. Cancellation belongs here too: once the scorer fails, you should not keep streaming more hunks into the same turn. Split the original user task and start a fresh turn with a much smaller file list.

Where inexpensive inference actually helps

You will retune this budget several times while leftover paid tasks are replayed locally. That replay stays cheaper when model calls are not metered like another premium seat. Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode offers free model access and a free server option for this kind of replay. You can rerun leftover tasks against the same scorer without renting another vendor review pane. Keep the budget file in git so the host you choose cannot silently widen the blast radius.

Free model access still does not replace path allowlists, automated tests, or human code owners. It only makes it less painful to iterate the gate while you leave the paid agent. Leave those ceilings in the repository even if you later change hosts again.

Leftovers after the seat is gone

Paid diffs still sitting in chat history are leftovers, even after you exported a few files. Formatter churn that hid inside a giant approve click is another leftover you should score. Lockfile rewrites, snapshot updates, and generated clients are leftovers that will burn budget silently. Any turn that touched more than one service boundary is a leftover that should become two tickets.

Build a short leftover table now and close each row on purpose during cutover. The table is part of the cutover plan, not a retrospective slide. Close rows with diffs on disk, not with a memory of what the paid pane displayed.

Leftover	Risk if ignored	Close-out action
Unexported paid patches	You cannot score the old blast radius	Dump unified diffs before seat expiry
Mega-refactors in one turn	New loop copies the same habit	Split into path-scoped tasks
Lockfile and vendor edits	Reproducible builds drift	Add forbidden prefixes
Snapshot and minified files	Budget burned on noise	Add ignore globs
Missing apply gate	Free-stack patches land uncounted	Wrap `git apply` with the scorer

Limitations and who should skip this

This proposed harness understands unified text diffs and very little else besides that format. Binary assets, submodule pointer moves, and rename detection will undercount or overcount depending on git flags. A formatter that rewrites an entire file will exhaust max_lines_added even when the semantic change is one function. You must decide whether format-on-apply runs before or after the budget, and you must write that down.

Teams that already enforce tiny pull requests through code owners may not need another ceiling. Security-sensitive repositories should not treat a raw line count as a real authorization check. Generated codebases with huge protobuf dumps need a different contract, because the useful unit is a schema file, not a hunk. If your agent only answers questions and never writes trees, you can skip this diary entirely.

Do not use a diff budget as proof that the new model is as good as the paid one. Size control is only a safety rail, and correctness still needs tests you already trust. If you already exported paid-agent diffs, score those files first and keep the same ceilings on the next host you try. Leave the paid seat only after every FAIL row has an owner and a split ticket. The leftover directory is finished when new turns cannot exceed the budget without a conscious edit to the JSON.

Migration Diary: Rebuild a Tool-Call Ledger Before You Leave a Paid Coding Agent

Riley Xu — Fri, 18 Sep 2026 08:10:19 +0000

You do not lose the model first when you leave a paid coding agent; you lose the evidence. The vendor UI kept every tool call, argument blob, and result hash behind a session you cannot export. Rebuild that trail as a local, append-only log before you cut over, or later incidents become folklore. This diary walks a concrete cutover so the leftover history lives in your repo, not in a product you no longer pay for.

Recent discussion treats chat-driven coding as engineering, then wonders why regressions appear after the tab closes. Engineering here is not the prompt; it is a replayable record of tools, files, and outcomes. If you cannot answer which call wrote package.json, you are not operating a loop you can defend. A migration that keeps the chat and drops the tool timeline only relocates the same guessing.

What actually walks out the door

Paid coding agents usually hide three things you will miss on the second day after canceling. They hide the ordered tool timeline, the raw arguments, and the truncated results that still prove a side effect. They also hide retries that looked like one click inside the product timeline you cannot export. Your leftover is not a nicer theme; it is a missing forensic trail that used to live in their UI.

You should treat that trail as a migration artifact, the same way you treat lockfiles during a runtime move. Chat transcripts without tool envelopes cannot reconstruct a patch, because prose never names the exact side effect. Screenshots of the vendor timeline cannot feed a regression test, and they rot as soon as the seat disappears. Export what the loop did into a local ledger, then change vendors only after that file exists.

The artifact: an append-only JSONL ledger

Keep one line per tool event, and refuse pretty-printed multi-line JSON inside the same file. A line is a fact; a paragraph is a story you will fight during grep. Store the ledger beside the workspace, not inside the vendor cloud, and rotate it per branch.

Treat the following schema as a proposal until you have run it against a real failing command.

{"v":1,"ts":"2026-09-18T12:00:00Z","run_id":"cutover-014","seq":3,"actor":"agent","tool":"apply_patch","status":"ok","args_sha256":"ab12","result_sha256":"cd34","cwd":"/work/app","files":["src/auth.ts"],"exit_code":0,"duration_ms":842,"error":null}

You hash arguments and results so the log stays small when payloads include whole files. You keep files and cwd in clear text because that is what incident review actually queries. You keep run_id plus seq so retries and hidden follow-up calls do not look like independent work. You keep actor in every line so a human override cannot impersonate the model during blame.

1. Write the record at the tool boundary

Wrap every tool invocation in one function, and deny ad-hoc subprocess calls from prompt glue. If a helper can run a command, it must emit a ledger line even when the command fails. Failures are the events you will need first, so an exception path that skips the log is a defect.

# ledger.py — proposed wrapper, not a production SDK
import hashlib, json, os, time
from datetime import datetime, timezone
from pathlib import Path

LEDGER = Path(os.environ.get("AGENT_LEDGER", ".agent/tool-calls.jsonl"))

def _sha(data: bytes) -> str:
    return hashlib.sha256(data).hexdigest()

def record(run_id, seq, tool, args, result, status, files, cwd, exit_code, started):
    LEDGER.parent.mkdir(parents=True, exist_ok=True)
    row = {
        "v": 1,
        "ts": datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ"),
        "run_id": run_id,
        "seq": seq,
        "actor": "agent",
        "tool": tool,
        "status": status,
        "args_sha256": _sha(json.dumps(args, sort_keys=True).encode()),
        "result_sha256": _sha(result if isinstance(result, bytes) else str(result).encode()),
        "cwd": cwd,
        "files": files,
        "exit_code": exit_code,
        "duration_ms": int((time.time() - started) * 1000),
        "error": None if status == "ok" else str(result)[:500],
    }
    with LEDGER.open("a", encoding="utf-8") as handle:
        handle.write(json.dumps(row, separators=(",", ":")) + "\n")
    return row

2. Validate the log before you trust a cutover

A ledger that cannot fail a test will rot into comments nobody trusts during an incident. Run a structural check in CI on every agent branch, and block merge when a required field disappears. The test below is small on purpose, because a heavy fixture will not run on every throwaway agent branch.

# test_ledger.py
import json
from pathlib import Path

REQUIRED = {
    "v", "ts", "run_id", "seq", "actor", "tool", "status",
    "args_sha256", "result_sha256", "cwd", "files", "exit_code",
    "duration_ms", "error",
}

def test_ledger_lines_are_complete():
    path = Path(".agent/tool-calls.jsonl")
    assert path.exists(), "run the loop once so the ledger has a line"
    seen = set()
    for raw in path.read_text(encoding="utf-8").splitlines():
        row = json.loads(raw)
        missing = REQUIRED - row.keys()
        assert not missing, missing
        key = (row["run_id"], row["seq"])
        assert key not in seen, key
        seen.add(key)
        assert row["v"] == 1
        assert row["status"] in {"ok", "error", "denied"}
        assert isinstance(row["files"], list)

3. Redact before you hash, not after

Paid products often redact in the UI while still shipping secrets into their own telemetry. Your local log can be worse if you hash a token that also landed in files. Scan arguments for env-shaped strings, replace those values with names, and only then compute the hash. Never log the entire environment wholesale for debug during the first weekend on a free server.

import re

SECRETISH = re.compile(r"(?i)(api[_-]?key|token|secret|password|bearer)\s*[:=]\s*\S+")

def redact(payload: str) -> str:
    return SECRETISH.sub(r"\1=***", payload)

Cutover plan in numbered steps

Follow this order so you do not discover a missing field after the paid seat expires.

Open three recent paid sessions, list every visible tool name, and write those names into tools.txt before canceling.
Map each vendor tool name to a local function, including deny paths for shells you will not allow.
Record one golden run on the paid side and keep the vendor timeline only as a checksum against JSONL.
Replay the same task on the new loop and compare tool, files, and exit_code sequences, ignoring chat prose.
Fail the cutover if seq gaps appear, because gaps mean retries or hidden tools you have not wrapped.
Freeze the schema at v1, and delay additive fields until the first week of queries has settled.

Use the following commands during step four so the ledger is real before you argue about model quality.

mkdir -p .agent
export AGENT_LEDGER="$PWD/.agent/tool-calls.jsonl"
python -m pytest test_ledger.py -q
python - <<'PY'
import json
from pathlib import Path
from collections import Counter
rows = [json.loads(l) for l in Path(".agent/tool-calls.jsonl").read_text().splitlines()]
print(Counter(r["tool"] for r in rows))
print("runs", sorted({r["run_id"] for r in rows}))
PY

Where a free model path actually helps

After the ledger exists, swapping the completion backend is a configuration change rather than a memory loss event. Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode offers free model access and a free server option for running the same loop while you compare tool sequences. Point the same wrapper at that path only after step four passes on a disposable branch.

You are proving that apply_patch still emits files and that denied commands still land as status=denied. If the server is free, you still meter wall time in duration_ms, because free does not mean unbounded patience from your process table. Skip slogans here, and do not treat a green chat as evidence that the cutover is finished. If the ledger diverges, keep the paid seat one more week and fix the wrapper before you cancel.

Decision table: what belongs in the line

Use this table to keep the line factual when someone asks to log everything for safety.

Event	Log it?	Why
Tool start with hashed args	Yes	Reconstruct order
Tool result hash plus exit code	Yes	Prove side effects
Full file contents	No	Blow up disk and leak secrets
Model prose in a separate file	Optional	Mixes story with facts
Human approval or edit	Yes, `actor=human`	Stop false blame
Raw environment block	No	Credential spill
Dependency install command	Yes	Lockfile incidents hide here

Leftovers you should expect anyway

Even a clean JSONL file will not restore vendor-only features you were quietly depending on. You will still lack their hosted session search, their shared team timeline, and whatever policy pack lived behind the login. You will also lack their silent retries unless your wrapper records them as new seq values. Write those gaps on a leftover card so nobody calls the migration feature complete after one green test.

Context windows, ranking, and editor plugins are separate migrations and should not ride along in this schema. Do not fold them into the audit trail or the schema will become a junk drawer. Finish the evidence trail first, then migrate prompts and routing as their own cutovers.

Limitations and who should not use this

This approach assumes you can intercept tools at the boundary where commands actually run. If the paid product only exposes a chat box, you cannot honestly claim parity, and you should not delete the seat. This approach also stores hashes and file paths that may remain sensitive, so legal review comes before a shared .agent directory.

Do not treat this ledger as a security information event management system with real retention guarantees. It has no retention job, no access control, and no tamper-evident chain beyond whatever git already gives you. Do not use it to justify skipping tests; a successful exit_code is not a product specification. Do not point any free server at production credentials while you compare traces on a live workspace.

Teams that never grant tools to a model can skip this entire diary without losing an operational record. People who need guaranteed uptime, signed vendor BAAs, or air-gapped legal holds need a different cutover, with procurement in the room. A local JSONL file will not satisfy those constraints, and pretending otherwise wastes a week.

Close the loop

If the next incident cannot name the tool, the file, and the run_id, the migration is not done. Keep the paid timeline until your test reads a real ledger line from a failed command, not only from a happy path. When that failure is boring to replay from JSONL alone, you can leave the paid seat.

After the schema is frozen, you can exercise the same wrapper on MonkeyCode's free model access and free server option using a throwaway branch.

Migration Diary: Rebuild the Command Sandbox Before You Leave a Paid Coding Agent

Riley Xu — Thu, 17 Sep 2026 06:29:50 +0000

The leftover that actually hurts during cutover is not a missing model brand; it is the sandbox the paid product never exposed. You can copy prompts, tools, and stop rules and still ship an agent that runs destructive shell on a free server. Freeze a command policy before you leave, then let every new run fail closed against that policy. This diary walks through a local policy file, a validator, and a cutover checklist you can run in one evening.

The leftover is the fence, not the chat

Paid coding agents wrap every shell call in a host you never configured, so the allowlist lives in their UI, not your repo. After you cancel, that wrapper disappears, and your loop talks to a raw terminal or an unmanaged worker process. The leftovers are not chat transcripts; they are implicit denials, working-directory jails, and silent network blocks. If you skip this inventory, a free model on a free server will look more capable only because it is less fenced.

You should treat the paid product as a black box that already answered four questions you must now write down. Write down which binaries may start, which flags are forbidden, which directories may be the working tree, and whether network is allowed. Those answers form a contract your next runtime has to enforce even after you change generators or hosts. Store the contract in git before you export a single chat, because chats do not block a dangerous binary.

Capture the paid behavior before you pull the plug

Work through the capture steps while the paid agent can still run a harmless command in your project. Do not wait until the invoice cycle ends and the vendor sandbox evaporates without leaving a dump. Keep a plain text file beside the repository and record denials in the same words the product used. You are not collecting slogans from the settings page; you are collecting the commands that already failed.

Run one read-only command, one write command, one network command, and one process-spawn command.
Record whether each call was allowed, rewritten, timed out, or blocked without a clear error.
Note the working directory the tool actually used, not the directory you assumed from the prompt.
Copy any UI labels that mention sandbox, restricted mode, or approved commands into that text file.

A denial is the most honest documentation the vendor will ever give you during this migration. Save the four outcomes even when they look boring, because boring denials are the policy. If a command succeeded in a surprising directory, treat that success as a finding rather than a convenience. Those four notes become the first draft of the YAML file in the next section.

Artifact: a fail-closed command policy

Put a policy file in the repository so the next runtime cannot invent permissions during a helpful-looking retry. The example below is a proposed contract, not a claim about any vendor's hidden defaults. Keep fail_closed true, because an empty allowlist must mean deny rather than mean trust the model. Commit this file with the capture notes so a reviewer can see why each binary was admitted.

# sandbox_policy.yaml — proposed local contract
version: 1
fail_closed: true
working_tree:
  allowed_roots:
    - "."
  deny_path_prefixes:
    - "/etc"
    - "/home"
    - "~"
    - ".."
network:
  allow: false
process:
  max_runtime_ms: 15000
  max_output_bytes: 65536
commands:
  allow:
    - argv_prefix: ["git", "status"]
    - argv_prefix: ["git", "diff"]
    - argv_prefix: ["python", "-m", "pytest"]
    - argv_prefix: ["ls"]
    - argv_prefix: ["rg"]
  deny_substrings:
    - "rm -rf"
    - "sudo"
    - "curl "
    - "wget "
    - "ssh "
    - "chmod 777"

The YAML also carries timeout and output caps that your subprocess wrapper must enforce later. The checker in the next section only validates argv, which is the leftover most teams forget to write down. Do not treat prefix matching as a container, and do not add bash -c to the allowlist to make a demo pass. If a paid-agent command cannot be expressed as an argv prefix, that command is not ready to move.

Run the checker before any cutover

Pair the policy with a small checker that the agent loop must call before subprocess. Label this as a proposed local gate you should run on fixtures, not as a production security audit. Fail the process when a fixture disagrees, because a yellow log line will not stop a cutover on a Friday. Keep the fixtures in git beside the policy so model swaps cannot quietly widen the shell.

# check_sandbox.py — proposed local gate, unexecuted until you wire fixtures
from __future__ import annotations

import json
import shlex
import sys
from pathlib import Path

import yaml

DENY_DEFAULT = "command rejected by sandbox_policy.yaml"


def load_policy(path: Path) -> dict:
    data = yaml.safe_load(path.read_text(encoding="utf-8"))
    if not data or not data.get("fail_closed"):
        raise ValueError("policy must set fail_closed: true")
    return data


def argv_list(command: str) -> list[str]:
    return shlex.split(command, posix=True)


def is_allowed(command: str, policy: dict) -> tuple[bool, str]:
    argv = argv_list(command)
    if not argv:
        return False, "empty command"
    joined = " ".join(argv)
    for blob in policy["commands"].get("deny_substrings", []):
        if blob in joined:
            return False, f"denied substring: {blob!r}"
    if policy.get("network", {}).get("allow") is False:
        if argv[0] in {"curl", "wget", "ssh", "scp", "nc"}:
            return False, "network binaries are blocked"
    for root in policy["working_tree"]["deny_path_prefixes"]:
        if any(str(part).startswith(root) for part in argv[1:]):
            return False, f"path prefix blocked: {root}"
    for rule in policy["commands"].get("allow", []):
        prefix = rule["argv_prefix"]
        if argv[: len(prefix)] == prefix:
            return True, "allowed prefix match"
    return False, DENY_DEFAULT


def main() -> int:
    policy = load_policy(Path("sandbox_policy.yaml"))
    fixtures = json.loads(Path("sandbox_fixtures.json").read_text(encoding="utf-8"))
    failures = []
    for row in fixtures:
        ok, reason = is_allowed(row["command"], policy)
        if ok != row["expect_allow"]:
            failures.append({**row, "reason": reason})
    if failures:
        print(json.dumps(failures, indent=2))
        return 1
    print("sandbox fixtures passed")
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

Add fixtures that encode yesterday's paid-agent denials, not the commands you wish had been allowed. Include at least one path-escape case, one network case, and one extra git subcommand that should still fail closed. If a fixture is annoying to write, that annoyance is the migration working as designed.

[
  {"command": "git status", "expect_allow": true},
  {"command": "git diff --stat", "expect_allow": true},
  {"command": "git push origin main", "expect_allow": false},
  {"command": "python -m pytest tests/test_sandbox.py", "expect_allow": true},
  {"command": "rm -rf /", "expect_allow": false},
  {"command": "curl https://example.com", "expect_allow": false},
  {"command": "ls ../secrets", "expect_allow": false}
]

git diff --stat passes because the allow rule matches an argv prefix, not a full string. git push origin main fails because push was never admitted, which is usually what you wanted on a free server. ls ../secrets fails on the path prefix even though ls itself is allowed. Tune prefixes in review when a historical paid command must keep working, and do not tune them inside a prompt.

Cutover plan you can finish before the subscription lapses

Treat cutover as a change to enforcement, not a change to chat tone or model branding. Each step should produce a file or a command output you can paste into the pull request. If a step produces only a feeling, you have not finished it. Stop when the checker is green against the paid-agent leftovers, not when the new chat looks fluent.

Freeze sandbox_policy.yaml and sandbox_fixtures.json in the same commit so reviewers see denials next to code.
Route every tool-calling path through is_allowed and refuse to start a subprocess on a deny.
Log the reason string; a silent block will look like a model failure and waste a retry budget.
Replay yesterday's paid-agent commands against the fixtures until every historical allow still allows.
Enforce max_runtime_ms in the subprocess wrapper, because argv checks do not kill hung processes.
Only then point the loop at a free model runtime, keeping the same policy file unchanged.

Run the checker from the repository root before you start the agent loop on any host you actually operate.

python3 -m pip install pyyaml
python3 check_sandbox.py

If the checker exits nonzero, do not start the agent loop on that revision. Fix the policy or the fixture first, then rerun until the output is a single passing line. A red checker is cheaper than a surprise network call on a shared server. Keep the command in CI so a later prompt change cannot reopen the fence.

Map leftovers to local replacements

Paid products hide policy in toggles, so you need a table that names each leftover and the file that now owns it. Fill the table from your capture notes rather than from vendor documentation, because the UI often overstates isolation. Keep the table in the same commit as the YAML file so a reviewer can see both the claim and the enforcement. If a row has no local replacement, you are not ready to cut over.

Paid leftover	Local replacement	Failure if skipped
Restricted-mode toggle	`fail_closed: true`	unknown commands execute
Working directory jail	`allowed_roots` and deny prefixes	writes leave the repo
Blocked outbound HTTP	`network.allow: false`	tools exfiltrate context
Approved command palette	`argv_prefix` allowlist	unexpected binaries start
Hidden tool timeout	`max_runtime_ms` in the wrapper	hung processes on a free server

Use the table as a punch list during the last week of the paid seat. Check a row only when a file or a test proves the replacement exists. Do not check a row because a model promised it would be careful. Careful is not an enforcement mechanism.

What a free runtime changes, and what it does not

When you leave a paid coding agent, generation quality is the discussion everyone wants, but process isolation is the discussion you actually need. A free server that can run tools makes the missing sandbox concrete, because commands now execute on a machine you operate. Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode is an open-source coding workspace with free model access and a free server option you can point at after the allowlist is frozen.

Keep the same fixtures when you change generators, including when you try that free-model path. If a run needs a wider allowlist, change the file in review rather than widening it in a prompt. Model choice becomes a smaller decision once permission lives in git. That is the useful leftover of this diary, and it still holds if you never touch that workspace.

Limitations

A prefix allowlist is not a container, a seccomp profile, or a multi-user isolation boundary for shared machines. It will not stop a binary you allowed from reading files through a feature you forgot to deny. It will not stop prompt injection from requesting a command that happens to match an allowed prefix. You still need operating-system isolation if the server is shared, networked, or holds secrets from other projects.

You should also keep secrets out of the working tree the policy allows for search tools. An allowlist that permits rg over a directory of tokens is a credential leak with extra steps. Move secrets into an environment the tool cannot print, and deny pagers that dump those values. If a fixture cannot be run without a secret in the tree, rewrite the fixture before you cut over.

Who should not use this approach

Skip this diary if your agent cannot run shell tools and only edits buffers through a mediated API. Skip it if you already run every tool inside a locked-down container with a reviewed seccomp or AppArmor profile. Skip it if you need a compliance certification, because this checker is a migration aid rather than an audit artifact. Teams that cannot fail closed should not pretend a YAML file is a sandbox, and they should stay on a mediated vendor host.

Once the policy exists, swapping generators is a smaller decision than it felt inside the paid UI. You can change models without changing permission, and that separation is the actual deliverable of this cutover. Keep widening the allowlist only through review, and treat every new binary as a migration leftover rather than a convenience. Exporting the chat history can wait until this fence is tested against the fixtures in your repository.

Migration Diary: Build a Workspace Manifest Before You Leave a Paid Coding Agent

Riley Xu — Wed, 16 Sep 2026 04:23:58 +0000

You should freeze a workspace manifest before you leave a paid coding agent, because free-tier loops rarely match the vendor file picker. Paid SDKs often choose files, skip binaries, and cap bytes without showing you the exact set. After cutover, the model either under-sees tests or over-sees lockfiles, and both failure modes look like a free-model drop. The leftover is not the model quality drop; it is an undocumented context contract you never extracted.

What breaks when the picker stays implicit

A paid coding agent usually walks your tree, respects some ignore file, and injects a subset of paths into the prompt. You never see the glob list, the per-file byte cap, or the hash of what the model actually received. When you move the loop onto a free server, you inherit none of those silent rules. The first week then fills with missing fixtures, surprise Dockerfiles, and patches that never saw the test tree.

The failure stays reproducible only if you keep receipts for every packed path and digest. Two checkouts with the same commit can still feed different bytes when one path is a symlink, a submodule, or a generated protobuf. You need a manifest that records path, size, digest, and whether the file was packed or only pointed at. Without that receipt, you cannot tell a model regression from a context regression on the same SHA.

Treat this builder as a cutover gate rather than a nice-to-have logger beside the repo. If you cannot rebuild yesterday's packed set from git SHA plus the manifest recipe, you are not ready to leave the paid SDK. Write the hash into the same diary line as the prompt version and the git SHA.

The artifact: a workspace manifest plus a packer

The script below is a labeled example you can run against a local git worktree today. It does not call any vendor API, and it writes a JSON manifest plus an optional pack of truncated text. Each entry records whether the file was packed, pointed at, skipped, or cut to a byte cap. You keep that recipe in git so two engineers can rebuild the same selection from the same commit.

#!/usr/bin/env python3
"""Build a workspace manifest before cutting an agent loop over to a free host."""
from __future__ import annotations

import hashlib
import json
import os
from dataclasses import asdict, dataclass
from pathlib import Path

ALWAYS_INCLUDE = ("README.md", "pyproject.toml", "tests/", "src/")
ALWAYS_SKIP_SUFFIX = (".png", ".jpg", ".woff", ".lock", ".min.js")
MAX_FILE_BYTES = 32_768
MAX_TOTAL_BYTES = 400_000


@dataclass
class Entry:
    path: str
    size: int
    sha256: str
    packed: bool
    truncated: bool
    reason: str


def git_root(start: Path) -> Path:
    cur = start.resolve()
    while cur != cur.parent:
        if (cur / ".git").exists():
            return cur
        cur = cur.parent
    raise SystemExit("run this inside a git worktree")


def should_skip(rel: str) -> str | None:
    parts = rel.split("/")
    if any(p in {".git", "node_modules", "dist", "__pycache__", ".venv"} for p in parts):
        return "ignored_dir"
    if rel.endswith(ALWAYS_SKIP_SUFFIX):
        return "binary_or_lock"
    return None


def force_include(rel: str) -> bool:
    return any(rel == prefix.rstrip("/") or rel.startswith(prefix) for prefix in ALWAYS_INCLUDE)


def digest_bytes(data: bytes) -> str:
    return hashlib.sha256(data).hexdigest()


def build_manifest(root: Path) -> dict:
    entries: list[Entry] = []
    packed_total = 0
    for dirpath, dirnames, filenames in os.walk(root):
        dirnames[:] = [d for d in dirnames if d not in {".git", "node_modules", "dist", ".venv"}]
        for name in filenames:
            full = Path(dirpath) / name
            rel = full.relative_to(root).as_posix()
            skip = should_skip(rel)
            raw = full.read_bytes()
            sha = digest_bytes(raw)
            if skip and not force_include(rel):
                entries.append(Entry(rel, len(raw), sha, False, False, skip))
                continue
            truncated = len(raw) > MAX_FILE_BYTES
            chunk = raw[:MAX_FILE_BYTES]
            would_pack = packed_total + len(chunk) <= MAX_TOTAL_BYTES
            packed = would_pack and (
                force_include(rel)
                or full.suffix in {".py", ".md", ".toml", ".txt", ".json", ".yml", ".yaml"}
            )
            reason = "packed" if packed else ("budget" if not would_pack else "not_text")
            if packed:
                packed_total += len(chunk)
            entries.append(Entry(rel, len(raw), sha, packed, truncated and packed, reason))
    entries.sort(key=lambda e: e.path)
    blob = json.dumps([asdict(e) for e in entries], separators=(",", ":")).encode()
    return {
        "root": str(root),
        "file_count": len(entries),
        "packed_count": sum(1 for e in entries if e.packed),
        "packed_bytes_capped": packed_total,
        "selection_sha256": digest_bytes(blob),
        "entries": [asdict(e) for e in entries],
    }


def write_pack(root: Path, manifest: dict, out: Path) -> None:
    out.mkdir(parents=True, exist_ok=True)
    for entry in manifest["entries"]:
        if not entry["packed"]:
            continue
        src = root / entry["path"]
        dest = out / entry["path"]
        dest.parent.mkdir(parents=True, exist_ok=True)
        data = src.read_bytes()[:MAX_FILE_BYTES]
        if entry["truncated"]:
            data += b"\n\n# [truncated by workspace-manifest]\n"
        dest.write_bytes(data)
    (out / "MANIFEST.json").write_text(json.dumps(manifest, indent=2), encoding="utf-8")


if __name__ == "__main__":
    root = git_root(Path.cwd())
    manifest = build_manifest(root)
    write_pack(root, manifest, root / ".agent-pack")
    summary = {k: manifest[k] for k in manifest if k != "entries"}
    print(json.dumps(summary, indent=2))
    print(f"wrote {root / '.agent-pack'} selection={manifest['selection_sha256'][:12]}")

Keep the recipe next to the agent loop, and paste selection_sha256 into the cutover diary beside prompt version. If the hash moves while git SHA stays still, your ignore rules or walk order changed under you. That hash drift is a stop-the-line event during migration week, not a curiosity for later cleanup. Add .agent-pack/ to .gitignore so truncated copies never become a second source of truth in git.

Cutover plan in numbered steps

1. Capture the paid agent's hidden set

You start by collecting the path list the paid agent already used, even if the vendor never documented the picker. Run one representative task and save the trace, the prompt dump, or the UI file list into paid-seen.txt. If the product hides that list, extract path-like tokens from logs with a small parser and treat gaps as unknown rather than empty. You are gathering evidence for a diff, not trying to reimplement their ranking.

# Labeled example: extract path-like tokens from a saved trace.
python - <<'PY'
from pathlib import Path
import re
text = Path("paid-trace.txt").read_text(encoding="utf-8", errors="replace")
paths = sorted(set(re.findall(r"(?:src|tests|lib)/[A-Za-z0-9_./-]+", text)))
Path("paid-seen.txt").write_text("\n".join(paths) + "\n")
print(len(paths))
PY

2. Build the free-loop manifest on the same commit

Check out the same SHA you used for the paid run, then generate .agent-pack/MANIFEST.json with the script. Compare those path sets with a real set difference instead of scrolling two folders by eye. A single missing tests/conftest.py will look like a model quality drop after you switch hosts later. Record packed-only paths as well, because extra lockfiles can waste the entire remaining byte budget.

git rev-parse HEAD
python workspace_manifest.py
python - <<'PY'
import json
from pathlib import Path
paid = set(Path("paid-seen.txt").read_text().split())
man = json.loads(Path(".agent-pack/MANIFEST.json").read_text())
packed = {e["path"] for e in man["entries"] if e["packed"]}
print("paid_only", sorted(paid - packed)[:20])
print("pack_only", sorted(packed - paid)[:20])
PY

3. Decide pack, pointer, or skip for every leftover path

Paid pickers collapse three decisions into one silent include list, and you have to split them again. Pack is for source the model must edit and for tests it must not ignore during patching. Pointers are enough for binaries, lockfiles, and generated blobs whose digest still matters for audit. Skip is for secrets, editor junk, and vendor noise that previously snuck through the paid picker.

Leftover after leaving the paid SDK	Symptom in the free loop	Action
Vendor auto-injected tests you never listed	Free loop ships untested patches	Add `tests/` to always-pack
Vendor skipped `package-lock.json`	Free loop rewrites dependency pins	Skip lockfiles, pointer only
Large generated `api.pb.go` was silently truncated	Tool output becomes mid-file garbage	Pointer plus hash, do not pack
Symlink to a secret file was followed	Free server sees credentials	Refuse follow; record as skip
Submodule directory was omitted	Model invents missing adapters	Pack a stub README in the submodule

4. Gate the cutover on selection hash stability

Run the builder twice on a clean tree and require identical selection_sha256 before you trust it. Dirty one tracked test file and require the hash to change, or your walk is ignoring edits. If either check fails, ignore rules or directory order are nondeterministic, and the free loop will drift. Put those two tests in CI next to the agent diary, not in a personal notebook.

# tests/test_workspace_manifest.py
# Labeled example: not executed in this article.
from pathlib import Path
from workspace_manifest import build_manifest


def _tiny_repo(base: Path) -> Path:
    (base / "src").mkdir(parents=True)
    (base / "tests").mkdir()
    (base / ".git").mkdir()
    (base / "README.md").write_text("# demo\n", encoding="utf-8")
    (base / "src" / "app.py").write_text("print(1)\n", encoding="utf-8")
    (base / "tests" / "test_app.py").write_text(
        "def test_ok():\n    assert True\n", encoding="utf-8"
    )
    return base


def test_clean_tree_is_stable(tmp_path: Path):
    repo = _tiny_repo(tmp_path)
    assert build_manifest(repo)["selection_sha256"] == build_manifest(repo)["selection_sha256"]


def test_edit_changes_selection(tmp_path: Path):
    repo = _tiny_repo(tmp_path)
    before = build_manifest(repo)["selection_sha256"]
    (repo / "src" / "app.py").write_text("# change\n", encoding="utf-8")
    assert before != build_manifest(repo)["selection_sha256"]

5. Move the loop, keep the pack as the only context source

Point your free-tier agent at .agent-pack, not at the live worktree, until patch apply is separately gated. That isolation is the leftover paid SDKs hide behind a single index-the-repo toggle in settings. Live trees pick up unsaved buffers, user caches, and half-written refactors the model should not see. Keep the pack read-only during the first week so a runaway tool cannot enlarge its own context.

If you already have a free-tier host, you can run the packer there without changing ignore rules. MonkeyCode's free model access and free server option can run this same manifest builder against a checkout. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The script does not depend on that host; you can run it against any local worktree.

How to read the leftover report

After the set difference prints, rank paid-only paths by whether the last paid run actually edited them. If the paid agent edited a file you did not pack, add that glob before you blame the new host. If the free pack includes huge JSON fixtures the paid picker skipped, drop them to pointer status and keep the digest. Write those three decisions in the diary so the next cutover does not rediscover them blindly.

Leftovers you should expect in week one

You will still find files the paid agent saw through IDE unsaved buffers, because those buffers never hit disk. You will also find line-ending conversions on mixed Windows checkouts that change SHA without changing meaning. Normalize packer reads to LF when your team mixed editors, or the selection hash becomes a source of false alarms. Log git SHA, selection hash, and packed file count beside every free-loop run you keep.

Another leftover is prompt caching that used to key on vendor-side file indexes you cannot export. Your free loop should key cache entries on selection_sha256 plus prompt version, or you will reuse an answer against a different tree. Watch for submodule directories the paid SDK omitted while still answering as if those adapters existed. Pack a short stub in those directories so the model sees the hole instead of inventing files.

Limitations, and who should not bother

This recipe does not reconstruct a vendor's proprietary ranking of supposedly important files in your tree. It only makes your own ranking explicit, hashed, and testable on a clean checkout of the same SHA. It also does not prove the model read the packed bytes; it only proves you offered a stable set. If you need read receipts, log the later tool calls that open files after the first pack.

Do not use this approach for throwaway single-file scripts, or for repos already wrapped in a strict allowlist. Do not pack customer data, .env files, or anything your free server is not allowed to store. Raise the byte caps only after you measure real missing-context failures, not because a dashboard looks empty. Teams with a trusted vendor picker and no cutover plan should leave this gate alone for now.

The core conclusion stays the same after you delete every product name from this whole page. Freeze the files the model can see, hash that set, and refuse cutover until the hash is stable. The paid SDK made that contract invisible on purpose or by accident; your free loop cannot afford that. Ship the manifest first, then argue about model quality with evidence instead of leftover folklore.

Migration Diary: Build a Parity Notebook Before You Move an Agent Loop to a Free Tier

Riley Xu — Mon, 14 Sep 2026 21:52:47 +0000

Liquid syntax error: Unknown tag 'endraw'

Migration Diary: Rebuild Patch Apply Gates Before You Leave a Paid Coding Agent

Riley Xu — Sun, 13 Sep 2026 20:37:41 +0000

Paid coding agents feel complete because they edit files, run tests, and hide the apply step inside a vendor runtime. The leftover that actually breaks cutover is not the rented model, but the missing gate that owns disk writes. Rebuild that gate before you leave, because a cheaper loop will otherwise write faster than your review process.

What the paid runtime was doing for you

Most hosted coding agents do more than stream tokens into a chat pane during ordinary coding work. They parse model output into file operations, keep a workspace lock, and often run a hidden formatter first. You only see a green check after the vendor stack already decided the incoming patch was safe enough. That decision is product logic rather than model intelligence, and it will not travel with exported prompts.

When you move the same loop onto a cheaper runtime, the model still emits unified diffs and tool calls. Nothing in that text knows your allowlist, generated-code folders, secret files, or protected CI configuration. If you paste those diffs into a workspace through a naive apply helper, you copy the vendor speed without the vendor brakes. The first leftover of this migration is therefore an explicit apply gate that your repository can enforce.

The failure that shows up after cutover

Consider a coding agent asked to fix a flaky test that also rewrites workflow files and a secrets loader. On a paid agent that extra damage may be blocked by path policy or a human review modal. On a raw loop those edits land as ordinary writes, while a unit test still passes for the wrong reason. You cannot detect that leftover by comparing invoices, because token spend does not encode filesystem policy.

You detect it by replaying each patch against a dry-run and a path policy, then failing closed on violations. The remainder of this diary is a portable gate you can run before any model is allowed to touch the tree. Treat every vendor screenshot as untrusted until the same patch is frozen as a fixture in git.

Cutover plan

Keep the paid agent available while you harvest real patches, because synthetic diffs hide the ugly headers you must parse. Export traces that include file edits rather than chat answers, and store the raw model text beside the ticket. Classification comes next, and anything you cannot label as create, replace, delete, or command should be dropped.

Export five to ten traces that include file edits, not just chat answers, and save the raw model text beside each ticket.
Classify every edit as path create, path replace, path delete, or command run, and drop output you cannot classify cleanly.
Write an allowlist of directories the agent may touch, plus a denylist covering secrets, vendor trees, and generated blobs.
Parse unified diffs in a dry-run that never writes bytes, and record which files would change under the policy.
Require a local test command to pass after a real apply, and roll the tree back when that command fails.
Only then point the same gate at a free model runtime or a free server workspace that you actually control.

The order matters more than the tooling, because switching runtimes first mixes policy bugs with model drift. If both change on the same day, you will not know which layer failed when a patch lands incorrectly. Finish the gate against paid traces first, then swap the generator and keep the gate bytes identical.

Artifact: a local apply gate you can version

The following example is a worked local script, not a production agent, and you should run the tests before trusting it. It reads a unified diff from standard input, enforces path policy, and applies the patch only after a successful dry-run. Keep the file in version control next to your prompts, because boring gates survive model swaps better than clever ones.

#!/usr/bin/env python3
"""apply_gate.py — dry-run and policy-check a unified diff before it touches disk."""

from __future__ import annotations

import argparse
import re
import subprocess
import sys
from dataclasses import dataclass

DIFF_HEADER = re.compile(r"^diff --git a/(.+) b/(.+)$")
FORBIDDEN_PARTS = (
    ".env",
    "id_rsa",
    "credentials.json",
    "node_modules/",
    "vendor/",
    ".git/",
)

@dataclass(frozen=True)
class PatchFile:
    path: str


def parse_diff(text: str) -> list[PatchFile]:
    files: list[PatchFile] = []
    for raw_line in text.splitlines():
        match = DIFF_HEADER.match(raw_line)
        if not match:
            continue
        files.append(PatchFile(path=match.group(2)))
    if not files:
        raise ValueError("no diff --git headers found; refuse unstructured text")
    return files


def violates_policy(path: str, allow_roots: list[str]) -> str | None:
    posix = path.replace("\\", "/")
    if posix.startswith("/") or posix.startswith("../") or "/../" in posix:
        return f"path escapes workspace: {path}"
    for part in FORBIDDEN_PARTS:
        if part in posix:
            return f"path hits denylist fragment {part}: {path}"
    if allow_roots:
        allowed = any(
            posix == root or posix.startswith(root.rstrip("/") + "/")
            for root in allow_roots
        )
        if not allowed:
            return f"path outside allowlist: {path}"
    return None


def git_apply(diff: str, check: bool) -> subprocess.CompletedProcess[str]:
    flags = ["git", "apply", "--whitespace=nowarn"]
    if check:
        flags.append("--check")
    return subprocess.run(flags, input=diff, text=True, capture_output=True)


def main() -> int:
    parser = argparse.ArgumentParser(description="Policy gate for agent-generated patches")
    parser.add_argument("--allow", action="append", default=[], help="relative root the patch may touch")
    parser.add_argument("--apply", action="store_true", help="write the patch after dry-run")
    parser.add_argument("--test", default="", help="command to run after a real apply")
    args = parser.parse_args()
    diff = sys.stdin.read()
    files = parse_diff(diff)
    errors = [msg for item in files if (msg := violates_policy(item.path, args.allow))]
    if errors:
        print("APPLY_GATE_FAIL")
        print("\n".join(errors))
        return 2
    check = git_apply(diff, check=True)
    if check.returncode != 0:
        print("APPLY_GATE_FAIL")
        print(check.stderr.strip() or check.stdout.strip())
        return 3
    if not args.apply:
        print("APPLY_GATE_DRY_RUN_OK")
        for item in files:
            print(f"would_touch {item.path}")
        return 0
    apply = git_apply(diff, check=False)
    if apply.returncode != 0:
        print("APPLY_GATE_FAIL")
        print(apply.stderr.strip())
        return 4
    if args.test:
        test = subprocess.run(args.test, shell=True)
        if test.returncode != 0:
            subprocess.run(["git", "checkout", "--", "."], check=False)
            print("APPLY_GATE_FAIL")
            print("tests failed; tree rolled back with git checkout -- .")
            return 5
    print("APPLY_GATE_APPLIED")
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

Save that script at the repository root and treat a failed parse as a refused apply rather than a chat repair loop. Unstructured model text without diff headers must not reach git apply, even when the surrounding commentary looks confident. The gate is intentionally small so you can read every branch before you attach it to an unattended worker.

Commands you can run during the diary

Harvest a patch from the paid agent while you still have access, then freeze that text as a fixture under version control. A dry-run that prints APPLY_GATE_DRY_RUN_OK becomes your baseline, and anything else is a cutover defect you still own. Next, prove the denylist fails closed with a fixture that tries to edit an environment file outside the allowlist.

mkdir -p fixtures/patches
# paste a vendor diff into fixtures/patches/fix-flaky-test.diff

python3 apply_gate.py --allow src --allow tests \
  < fixtures/patches/fix-flaky-test.diff

cat > fixtures/patches/forbidden-env.diff <<'EOF'
diff --git a/.env b/.env
--- a/.env
+++ b/.env
@@ -1,2 +1,3 @@
 API_TOKEN=keep-me
+DEBUG_TOKEN=leaked
EOF

python3 apply_gate.py --allow src --allow tests \
  < fixtures/patches/forbidden-env.diff
echo exit:$?

You should see APPLY_GATE_FAIL and a nonzero exit status, which is now evidence rather than a vendor screenshot. When a later model emits the same class of edit, the gate does not care which runtime produced the text. That independence is the whole point of leaving a paid coding agent without leaving behind its hidden filesystem policy.

For an allowed apply that must survive tests, pass a command you already run in continuous integration. If the test command fails, the script attempts a coarse rollback with git checkout, which is a leftover not a transaction manager. It will not undo untracked files, so keep the agent workspace clean before each apply during the migration window.

python3 apply_gate.py --allow src --allow tests --apply \
  --test "python -m pytest -q tests" \
  < fixtures/patches/fix-flaky-test.diff

Dirty trees mix agent writes with your own edits, and the diary then cannot explain who changed a given path. Reset or stash first, then run the gate, then inspect git status as part of the same ritual. If untracked files appear mid-run, treat them as a failed cutover even when the exit code is zero.

A tiny test plan for the parser

Label the next block as a local example until you execute it against your own fixtures. It does not need a network, a vendor key, or a running agent. It only checks that policy fails closed before git apply is invited to speak.

# test_apply_gate.py — run with: python3 -m pytest -q test_apply_gate.py
from apply_gate import parse_diff, violates_policy

SAMPLE = """diff --git a/src/app.py b/src/app.py
--- a/src/app.py
+++ b/src/app.py
@@ -1,1 +1,2 @@
 print(1)
+print(2)
"""

def test_parse_reads_destination_path():
    files = parse_diff(SAMPLE)
    assert files[0].path == "src/app.py"

def test_denylist_blocks_env():
    assert violates_policy(".env", ["src"]) is not None

def test_allowlist_blocks_github_workflows():
    msg = violates_policy(".github/workflows/ci.yml", ["src", "tests"])
    assert msg is not None and "allowlist" in msg

python3 -m pytest -q test_apply_gate.py

If those three assertions fail, stop the migration and fix the gate before you argue about model quality. Parser bugs masquerade as agent regressions after you leave the paid runtime. Freeze the tests with the fixtures so a later prompt tweak cannot silently widen the write surface.

A decision table for leftovers

Fill this table with your own paths so the migration document lives in the repository instead of a vendor admin console. The script only enforces what you write down, and empty allowlists will fail closed or allow too much depending on flags. Prefer an explicit allowlist of application code and tests over a denylist that forever lags new secret filenames.

Leftover from the paid agent	What you freeze before cutover	Fail closed when
Hidden path allowlist	`--allow` roots checked into the repo	Diff touches `infra/`, `.github/`, or secrets
Review modal before write	Dry-run exit code in CI	`git apply --check` fails
Vendor formatter pass	Explicit format command after apply	Format diff is outside allowlist
Automatic test button	`--test` command you chose	Tests fail or never ran
Workspace isolation	One git checkout per agent job	Untracked files appear mid-run

Revisit the table after the first week on the new runtime, because free loops explore files the paid UI never showed you. Each new forbidden path should become a fixture, not a chat message, or the leftover will return on the next model bump. If you cannot name the test command in the last row, you are not ready to let any agent apply patches unattended.

Where a free model runtime fits

After the gate exists, the model becomes a patch generator instead of a filesystem owner, which is the cutover you want. If you are already evaluating MonkeyCode, its free model access and free server option can host the loop that emits diffs into this gate.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Keep the product at the edge of the design, because the gate still runs if you pipe patches from any other CLI. It still fails closed if the free server is slower, noisier, or missing a vendor-only formatter you used to ignore. Do not treat free capacity as a reason to skip fixtures, since cheap retries are how you replay those fixtures often.

A useful next step is to mount the same repository on the free server and keep the allowlist identical to your laptop. That single check prevents the leftover where a cheap machine can write anywhere because nobody copied the vendor policy. If the server workspace uses different absolute paths, fix the working directory first rather than widening the allowlist.

Limitations

This gate understands diff git headers and git apply, and it will refuse custom tool JSON or headerless whole-file rewrites. Binary patches, submodule tricks, and relative paths that escape through git apply in nested repos can still surprise you. The rollback is workspace-coarse and will clobber unrelated dirty files if you ignore the clean-tree rule described above.

The script also does not replace code review, because a patch inside src can still remove authentication or weaken assertions. Fail-closed path policy is necessary and not sufficient, so pair it with a human diff for anything that ships. Treat green tests after an agent apply as necessary evidence, not as proof that the change matches the original ticket.

Who should not use this approach

Do not use this approach if your agent must edit infrastructure as code without a second reviewer on the same change. Skip it if you lack git, if the workspace is not recoverable, or if untrusted models need syscall isolation rather than path checks. A path allowlist is not a container, and it will not stop network calls, credential theft, or prompt injection into later tools.

Teams that already have a mature policy engine around git apply should wrap that engine instead of copying this example verbatim. The diary is for people leaving a paid coding agent who never saw the apply step because the vendor UI hid it. If that is not your leftover, freeze a different artifact and keep this gate off the critical path.

What to leave behind

The paid coding agent can go when three artifacts live in your repo: frozen patch fixtures, the apply gate, and a one-line test command. Everything else is leftover interface, including review modals that never made it into a headless script you can run overnight. If a free loop cannot get through the gate, the migration is not ready, no matter how the invoice looks after the switch.

Migration Diary: Stamp Idempotency Keys on Tools Before You Leave a Paid Agent

Riley Xu — Sat, 12 Sep 2026 18:36:06 +0000

Paid agent runtimes retry mutating tools after timeouts, dropped streams, and loop restarts, and they often hide those retries from you. You should stamp an idempotency key on every side-effecting tool before you leave that paid loop. The leftover after cutover is not a missing system prompt; it is a duplicate pull request, comment, or deploy. Treat retry as a first-class protocol, not as an SDK quirk you hope the vendor already solved.

Why the second write appears only after you migrate

Paid coding agents wrap tool dispatch, HTTP clients, and filesystem writes behind one retry budget you cannot inspect. When a stream dies, the vendor may replay the last tool call while your local process also retries it. You only notice after two issues exist, two branches land, or two webhooks fire against production. Moving the loop to another runtime does not remove that race; it merely makes the second write cheaper and easier to miss in logs.

Read-only tools can be replayed without a key, but anything that creates, updates, or notifies needs an envelope. You should classify tools before you rewrite prompts, because prompt edits will not stop a double create_pr call. Keep the paid agent running in drain mode until every in-flight key either commits or expires in your store.

Step 1: Inventory side effects, not model names

Walk the tool list the paid agent actually invoked during the last week of traces, not the catalog in marketing docs. Mark each tool as pure, idempotent-if-keyed, or needs-compensation, and refuse to cut over while any mutating tool stays unclassified. Capture the natural key the backend already understands, such as a branch name, issue id, or object path.

Export recent tool traces from the paid runtime into a local JSONL file you control.
Tag every record with pure, keyed, or compensate using the decision table below.
Freeze new mutating tools in the paid product until those tags exist in version control.
Reject any cutover plan that only copies system prompts and leaves tool transport unchanged.

python3 - <<'PY'
import json, collections, pathlib
path = pathlib.Path("paid_tool_trace.jsonl")
counts = collections.Counter()
for line in path.read_text().splitlines():
    rec = json.loads(line)
    name = rec.get("tool") or rec.get("name") or "unknown"
    counts[name] += 1
for name, n in counts.most_common():
    print(f"{n:5d}  {name}")
PY

You now have a frequency list, which is more honest than a vendor tool catalog. High-frequency mutators are the leftovers that will clone themselves on the free host. Sort that list before you touch model routing, because routing changes amplify retries instead of removing them.

Step 2: Stamp a stable envelope around every mutating call

An idempotency key must be derived from intent, not from a random UUID generated after the HTTP client already fired. Hash the tool name, the normalized arguments, and the agent-run id so a replay of the same thought hits the same key. Store the first response beside that key, then return the stored payload when the model asks again with identical intent.

# Proposed local helper. Label it unexecuted until you point it at a real trace file.
from __future__ import annotations

import hashlib, json, sqlite3, time
from dataclasses import dataclass
from typing import Any, Callable

SCHEMA = """
CREATE TABLE IF NOT EXISTS tool_calls (
  idempotency_key TEXT PRIMARY KEY,
  tool_name TEXT NOT NULL,
  request_json TEXT NOT NULL,
  response_json TEXT,
  state TEXT NOT NULL,
  created_at REAL NOT NULL,
  updated_at REAL NOT NULL
);
"""

def intent_key(run_id: str, tool_name: str, args: dict[str, Any]) -> str:
    blob = json.dumps({"run_id": run_id, "tool": tool_name, "args": args}, sort_keys=True)
    return hashlib.sha256(blob.encode()).hexdigest()

@dataclass
class Envelope:
    key: str
    tool_name: str
    args: dict[str, Any]

class ReplayStore:
    def __init__(self, path: str = "tool_replay.sqlite") -> None:
        self.conn = sqlite3.connect(path)
        self.conn.execute(SCHEMA)
        self.conn.commit()

    def execute(self, env: Envelope, send: Callable[[dict[str, Any]], Any]) -> Any:
        now = time.time()
        row = self.conn.execute(
            "SELECT state, response_json FROM tool_calls WHERE idempotency_key = ?",
            (env.key,),
        ).fetchone()
        if row and row[0] == "committed":
            return json.loads(row[1])
        if row and row[0] == "inflight":
            raise RuntimeError(f"tool {env.tool_name} still inflight for {env.key[:12]}")
        self.conn.execute(
            "INSERT OR REPLACE INTO tool_calls VALUES (?,?,?,?,?,?,?)",
            (env.key, env.tool_name, json.dumps(env.args, sort_keys=True), None, "inflight", now, now),
        )
        self.conn.commit()
        try:
            result = send(env.args)
        except Exception:
            self.conn.execute(
                "UPDATE tool_calls SET state = ?, updated_at = ? WHERE idempotency_key = ?",
                ("failed", time.time(), env.key),
            )
            self.conn.commit()
            raise
        self.conn.execute(
            "UPDATE tool_calls SET state = ?, response_json = ?, updated_at = ? WHERE idempotency_key = ?",
            ("committed", json.dumps(result), time.time(), env.key),
        )
        self.conn.commit()
        return result

Keep run_id stable across stream retries inside one agent attempt, and mint a new run_id only when the user starts a distinct task. If you rotate run_id on every token timeout, you will stamp a new key and create the duplicate you were trying to prevent. Log the key prefix beside the tool name so later leftovers can be grepped without dumping argument blobs into chat.

Step 3: Decide retry, replay, or compensate with a table

Not every failure should call send() again. Some tools are safe to replay from the store, some must wait, and some need an explicit undo. Use this table during cutover reviews so engineers stop arguing from model folklore.

Tool shape	Example	On timeout	On identical retry	On confirmed extra write
Pure read	`get_file`, `search`	retry send	retry send	not applicable
Keyed create	`open_pr`, `create_issue`	wait, then replay store	return stored payload	close or comment the extra object
Keyed write	`put_file`, `update_status`	wait, then replay store	return stored payload	overwrite with the stored canonical body
Effect with no natural key	`charge_card`, `page_oncall`	do not auto-retry	refuse until a human key exists	run a documented compensating action

DECISIONS = {
    "get_file": "pure",
    "search_repo": "pure",
    "open_pr": "keyed",
    "create_issue": "keyed",
    "put_file": "keyed",
    "merge_pr": "compensate",
    "notify_slack": "compensate",
}

def classify(tool_name: str) -> str:
    if tool_name not in DECISIONS:
        raise ValueError(f"refuse unclassified tool: {tool_name}")
    return DECISIONS[tool_name]

If a tool cannot be classified, you do not migrate it. Paid runtimes love unclassified helpers because they look like convenience; after cutover they become silent duplicates. Put refuse unclassified tool in the dispatcher so a new vendor tool cannot sneak onto the free host during the first weekend.

Step 4: Prove replay with a test you can run offline

Do not trust a dashboard green check. Write a local test that sends the same envelope twice and asserts one underlying side effect. Keep the test free of network calls by injecting a fake send that increments a counter.

# Proposed test. Run it against the helper above before any production drain.

def test_second_call_does_not_resend(tmp_path):
    store = ReplayStore(str(tmp_path / "replay.sqlite"))
    hits = {"n": 0}

    def send(args):
        hits["n"] += 1
        return {"pr_url": "https://example.invalid/pr/1", "args": args}

    args = {"title": "meter retries", "head": "feat/keys", "base": "main"}
    env = Envelope(intent_key("run-22", "open_pr", args), "open_pr", args)
    first = store.execute(env, send)
    second = store.execute(env, send)
    assert first == second
    assert hits["n"] == 1

python3 -m pytest -q test_tool_replay.py

A passing test means the leftover path is closed for that tool shape, not that the paid vendor is gone. Add one test per mutating tool class, especially comment posting and file writes, because those are the duplicates readers will see first. If the test cannot run without the paid SDK, you have not extracted the protocol yet.

Step 5: Dual-run, then drain, then disable paid dispatch

Cutover is a drain problem, not a prompt-paste problem. You keep the paid loop alive only for runs that already hold inflight rows, and you start new runs on the destination host with the same envelope code. Compare keys, not prose summaries, when you declare the migration done.

Deploy the replay store beside both runtimes and refuse tools that cannot stamp a key.
Route new agent runs to the destination host while the paid host accepts only resume traffic.
Watch inflight rows until they become committed or failed; do not kill the paid process early.
Diff tool names and key prefixes across both logs and treat unmatched mutators as blockers.
Disable paid dispatch only after the unmatched set stays empty for a full drain window you chose in advance.

sqlite3 tool_replay.sqlite "SELECT state, COUNT(*) FROM tool_calls GROUP BY state;"
sqlite3 tool_replay.sqlite "SELECT tool_name, substr(idempotency_key,1,12), state FROM tool_calls WHERE state != 'committed';"

Empty inflight plus zero unclassified tools is your go signal. Fancy model quality scores are not. If a row sits in inflight longer than your tool's own timeout, you have found a hung side effect and you must inspect it by hand before disabling the paid dispatcher.

Once keys and a replay log exist, you can run the same envelope against another model host without changing tool contracts. Disclosure: This article was prepared as part of MonkeyCode's product outreach. You can replay a drained, key-stamped loop against MonkeyCode's free model access and free server option without renaming tools or inventing a second dispatcher.

Leftovers to hunt after the paid agent is quiet

Duplicates hide in objects the model never mentions again. Search for paired pull requests with the same head branch, paired issue titles with the same hash prefix, and paired bot comments posted within one retry window. Those records are the migration leftovers, and they will not appear in a prompt diff.

# Proposed git forensics. Adjust the since stamp to your drain window.
git log --since='2026-09-06' --pretty='%h %s' | awk 'seen[$2]++{print}'

If your hosting platform allows, list webhook deliveries grouped by idempotency header and delete or cancel the extras. Do not ask the model to "clean up duplicates," because that request is another unkeyed mutation. Cleanup belongs in a script that reads the replay store and the decision table together.

Limitations, and who should not use this approach

This envelope gives you at-most-once intent at a single worker, not a distributed exactly-once transaction across regions. A SQLite file will not protect two replicas that stamp keys at the same moment without a real lock service. Hashing arguments also fails when servers inject timestamps, unsigned maps, or absolute paths that change between retries.

Compensation is not magic. Closing an extra pull request is easy; reversing a customer-visible charge is not, and this diary does not claim otherwise. Clock-based expiry on keys can resurrect duplicates if you expire too fast, so prefer explicit run_id lifetime over short TTLs copied from web frameworks.

Skip this approach if every mutating backend already requires an idempotency header you already plumb end to end. Skip it if your tools only read state and never write. Skip it if you need multi-region consensus, because a local replay file will lie to you the first time two workers overlap. Skip it if you expect any free host to inherit the paid vendor's hidden retry budget, timeouts, or lease behavior; those budgets are what you are extracting, not what you should assume still exist.

The durable artifact is the key, the store, and the drain query, not a rewritten persona prompt. Leave the paid agent only after those three objects survive a replay test and an empty inflight list. The model can change on the next host. The side effects should not.

Migration Diary: Extract Loop Stop Conditions Before You Leave a Paid Coding Agent

Riley Xu — Fri, 11 Sep 2026 16:09:23 +0000

Paid coding agents usually fail after cutover because stop conditions still live inside the old vendor runtime. You cannot swap a model endpoint and expect the same number of tool rounds to occur. The hidden while-loop used to decide when work was done, blocked, or too expensive to continue. This diary treats those stop conditions as portable policy you extract, test, and carry yourself.

When you leave a paid coding agent, the loop does not shrink itself to match your new host. It either spins until the process is killed, or it exits after one timid tool call. Both outcomes look like model quality problems when they are actually missing brakes and missing continue signals. Own the budget before you drain the paid lease, not after the first runaway session.

The loop budget you never checked in

Hosted coding agents wrap a while-loop that your repository never contained as an explicit object. Each iteration may request a completion, dispatch tools, patch files, or ask the model to continue. The vendor then applies limits you cannot grep: max rounds, max tool calls, and a done heuristic. Those limits are not documentation trivia; they are the difference between a refactor finishing and a runaway bill.

If you only change the base URL, you inherit none of those brakes on the new runtime. You also lose the vendor's keep-going signal that let multi-file edits complete across many rounds. The leftover is therefore policy, not a secret config file sitting in an export archive. Treat every unnamed stop reason as an unpaid dependency on the old host.

A useful budget is small and boring. You need round caps, tool-call caps, output-token caps, consecutive tool-error caps, and a no-progress rule. You also need an explicit UNKNOWN_VENDOR reason so undocumented stops fail closed during replay. Anything vaguer than that will be reimplemented as folklore in chat threads.

Cutover plan: own the stop conditions first

Follow these five steps in order. Do not start DNS or key cutover until step five is green on a replay trace you actually captured.

Step 1 — Record one production loop as a trace

Pick a single coding task that used to succeed on the paid agent, including tool calls and file patches. Capture every turn as JSON: round index, tool names, token counts if present, and the final stop reason. Do not trust the vendor summary field alone, because that field often collapses several internal decisions. Store the trace next to your tests so later budget changes have something concrete to replay.

mkdir -p traces fixtures
curl -sS "$PAID_AGENT_TRACE_URL" \
  -H "Authorization: Bearer $PAID_AGENT_TOKEN" \
  | python3 -m json.tool > traces/prod-refactor.json
wc -l traces/prod-refactor.json

If the vendor has no trace export, log the loop yourself for one freeze window. Wrap the existing SDK calls, write one JSON line per round, and keep the paid runtime read-only. You are collecting evidence, not starting the migration.

Step 2 — Name every stop reason in your own enum

Map vendor strings onto names you control before you write a single retry. Unknown strings must not become TASK_COMPLETE, because that lie ships half-finished diffs. Keep the mapping in one module so product language cannot drift across services. If a reason cannot be mapped, fail closed and keep the paid loop alive for that task class.

VENDOR_STOP_MAP = {
    "completed": "task_complete",
    "max_iterations": "max_rounds",
    "tool_limit": "max_tool_calls",
    "length": "max_output_tokens",
    "cancelled": "user_cancel",
    "empty_assistant": "no_progress",
}

Step 3 — Encode a `LoopBudget` object the new host must honor

Put numbers in code, not in a slide. Start from the captured trace, then tighten one notch so the new host cannot outrun the paid loop. Label these defaults as a proposal until your replay suite has run against real traces. You should not copy a vendor dashboard number you cannot prove from logs.

Step 4 — Fail closed when the new host cannot honor a reason

If the replacement runtime cannot report output tokens, do not pretend the token cap still works. Disable that task class, or route it back to the paid agent until the meter exists. Partial budgets create a false sense of control and hide the exact leftover you are trying to delete. Record the gap in the leftover checklist rather than commenting it out.

Step 5 — Drain the paid runtime only after replay passes

Replay the golden trace against your loop object with tools stubbed and the model call injected. Assert the stop reason, round count, and tool-call count match the trace within a one-round tolerance. Only then move traffic, and move one task class at a time. Leave the paid key in a break-glass path until two quiet days pass.

Artifact: a replayable loop budget

The following module is a concrete starting point, not a benchmarked production runtime. It makes stop conditions testable without calling any vendor SDK. Save it as loop_budget.py and keep vendor mapping in a separate file so cutover diffs stay reviewable.

from __future__ import annotations

from dataclasses import dataclass, field
from enum import Enum
from typing import Optional
import hashlib
import json


class StopReason(str, Enum):
    TASK_COMPLETE = "task_complete"
    MAX_ROUNDS = "max_rounds"
    MAX_TOOL_CALLS = "max_tool_calls"
    MAX_OUTPUT_TOKENS = "max_output_tokens"
    USER_CANCEL = "user_cancel"
    TOOL_ERROR_BUDGET = "tool_error_budget"
    NO_PROGRESS = "no_progress"
    UNKNOWN_VENDOR = "unknown_vendor"


@dataclass(frozen=True)
class LoopBudget:
    max_rounds: int = 12
    max_tool_calls: int = 24
    max_output_tokens: int = 32_000
    max_consecutive_tool_errors: int = 3
    no_progress_rounds: int = 2


@dataclass
class LoopState:
    rounds: int = 0
    tool_calls: int = 0
    output_tokens: int = 0
    consecutive_tool_errors: int = 0
    observation_hashes: list[str] = field(default_factory=list)
    last_stop: Optional[StopReason] = None
    vendor_stop: Optional[str] = None


def observation_hash(text: str) -> str:
    return hashlib.sha256(text.encode("utf-8")).hexdigest()[:16]


def should_stop(budget: LoopBudget, state: LoopState) -> Optional[StopReason]:
    if state.vendor_stop and state.vendor_stop not in {
        "completed",
        "max_iterations",
        "tool_limit",
        "length",
        "cancelled",
        "empty_assistant",
    }:
        return StopReason.UNKNOWN_VENDOR
    if state.rounds >= budget.max_rounds:
        return StopReason.MAX_ROUNDS
    if state.tool_calls >= budget.max_tool_calls:
        return StopReason.MAX_TOOL_CALLS
    if state.output_tokens >= budget.max_output_tokens:
        return StopReason.MAX_OUTPUT_TOKENS
    if state.consecutive_tool_errors >= budget.max_consecutive_tool_errors:
        return StopReason.TOOL_ERROR_BUDGET
    if len(state.observation_hashes) >= budget.no_progress_rounds:
        tail = state.observation_hashes[-budget.no_progress_rounds :]
        if len(set(tail)) == 1:
            return StopReason.NO_PROGRESS
    return None


def apply_round(state: LoopState, *, tools: int, tokens: int, observation: str, tool_error: bool) -> LoopState:
    state.rounds += 1
    state.tool_calls += tools
    state.output_tokens += tokens
    state.observation_hashes.append(observation_hash(observation))
    if tool_error:
        state.consecutive_tool_errors += 1
    else:
        state.consecutive_tool_errors = 0
    return state


def replay_trace(path: str, budget: LoopBudget) -> StopReason:
    with open(path, encoding="utf-8") as handle:
        events = json.load(handle)
    state = LoopState()
    last_reason: Optional[StopReason] = None
    for event in events["rounds"]:
        apply_round(
            state,
            tools=len(event.get("tool_calls", [])),
            tokens=int(event.get("output_tokens") or 0),
            observation=event.get("observation") or "",
            tool_error=bool(event.get("tool_error")),
        )
        state.vendor_stop = event.get("vendor_stop")
        last_reason = should_stop(budget, state)
        if last_reason:
            state.last_stop = last_reason
            return last_reason
    return StopReason.TASK_COMPLETE

A minimal trace fixture keeps the tests honest. Notice the third round repeats the same observation, which should trip NO_PROGRESS if you set no_progress_rounds to 2.

{
  "task": "rename getUser to fetchUser across api and tests",
  "rounds": [
    {
      "tool_calls": [{"name": "read_file"}],
      "output_tokens": 400,
      "observation": "found 3 call sites",
      "tool_error": false
    },
    {
      "tool_calls": [{"name": "apply_patch"}, {"name": "apply_patch"}],
      "output_tokens": 900,
      "observation": "patched api.py; tests still import getUser",
      "tool_error": false
    },
    {
      "tool_calls": [{"name": "apply_patch"}],
      "output_tokens": 200,
      "observation": "patched api.py; tests still import getUser",
      "tool_error": false,
      "vendor_stop": "empty_assistant"
    }
  ]
}

python3 - <<'PY'
from loop_budget import LoopBudget, replay_trace, StopReason
reason = replay_trace("traces/prod-refactor.json", LoopBudget(no_progress_rounds=2))
assert reason in {StopReason.NO_PROGRESS, StopReason.TASK_COMPLETE, StopReason.UNKNOWN_VENDOR}
print(reason.value)
PY
python3 -m pytest -q test_loop_budget.py

# test_loop_budget.py — labeled unexecuted until you point it at a real trace
from loop_budget import LoopBudget, LoopState, StopReason, apply_round, should_stop

def test_unknown_vendor_fails_closed():
    state = LoopState(vendor_stop="policy_hidden_v3")
    assert should_stop(LoopBudget(), state) is StopReason.UNKNOWN_VENDOR

def test_repeated_observation_is_no_progress():
    budget = LoopBudget(no_progress_rounds=2)
    state = LoopState()
    apply_round(state, tools=1, tokens=10, observation="same", tool_error=False)
    apply_round(state, tools=1, tokens=10, observation="same", tool_error=False)
    assert should_stop(budget, state) is StopReason.NO_PROGRESS

Decision table for stop reasons

Use this table during review so engineers do not argue from screenshots. If a cell says fail closed, that task class stays on the paid agent until the gap is instrumented.

Stop reason	What you must observe	Safe action on the new host	Fail closed if missing
`task_complete`	Model or tool reports a terminal, verified result	Return the diff and close the loop	No verifier, only a chatty summary
`max_rounds`	Round counter in your process	Stop, emit partial log, do not retry blindly	Host has no round counter
`max_tool_calls`	Tool dispatcher increment	Stop and require a human or a smaller task	Tools fire outside your dispatcher
`max_output_tokens`	Tokenizer or vendor usage field you trust	Stop and split the task	Usage field is absent or delayed
`tool_error_budget`	Consecutive tool failures	Stop and surface the last stderr	Errors are swallowed as empty tool results
`no_progress`	Repeated observation hashes	Stop and ask for a narrower instruction	You only log model text, not observations
`unknown_vendor`	Unmapped vendor stop string	Keep paid path; do not guess	You alias it to `task_complete`

Leftovers after you leave the paid coding agent

Cutover leftovers hide in places that look unrelated to models. Walk this list before you revoke the paid key, because each item can restart the vendor loop without your budget object. Write the findings into leftovers.md so the next incident does not start from memory.

IDE extensions that still call the paid agent when a test fails, bypassing your round cap entirely.
CI helpers that retry the same coding task on timeout, multiplying max_rounds without a shared counter.
Webhooks that resume a thread after a tool result, with no copy of LoopState on the new host.
Cached vendor "continue" prompts that force another completion after you already stopped locally.
File-watchers that treat every patch as a fresh task and open a second loop on the same workspace.
Observability dashboards that still chart vendor iteration counts, hiding your new stop reasons from on-call.

rg -n "max_iterations|tool_limit|continue_on_error|agent.loop" \
  --glob '!node_modules' --glob '!.git'
rg -n "PAID_AGENT|CODING_AGENT_URL|RESUME_THREAD" .env.example deploy ci

If rg still finds a resume path, that path is a second control plane. Either delete it or pass the same LoopBudget into it. Two control planes means your leftover list is lying.

Replaying the budget without spending the paid gateway

Disclosure: This article was prepared as part of MonkeyCode's product outreach. After the budget object exists, you still need a place to replay traces while the paid coding agent is draining. MonkeyCode's free model access and free server option can host that replay worker so the golden traces do not keep billing the vendor you are leaving. Keep the worker read-only against production workspaces, and keep the paid break-glass path until replay reasons match the table above.

The replay worker should not become a second agent. It should load a trace, apply LoopBudget, stub tools, and write a stop-reason report. If you want a live model in the loop, inject it behind the same counters rather than asking the host to invent its own iteration policy. Free capacity is useful here because replay is repetitive, boring, and easy to accidentally run against a paid SDK.

Limitations and who should not use this

This approach assumes you can observe rounds, tool calls, and some usage signal. If your coding agent is a black box inside an IDE with no log hook, you cannot honestly claim a portable budget. If your tasks are single-shot completions with no tools, a loop object is ceremony and you should not add it.

Do not use these default numbers as capacity planning. They are placeholders until a real trace tells you otherwise, and they are not a performance claim about any host. Do not fail open on UNKNOWN_VENDOR to make a demo look finished. Do not run unconstrained file-editing tools on a shared server just because the model path is free.

Teams that already have a workflow engine with idempotency keys and explicit state machines may only need the mapping layer. Teams that treat the vendor UI as the system of record will fight this diary, because the leftover is the UI itself. In that case, extract traces first and delay any talk of leaving.

Stop conditions are the smallest artifact that makes a coding-agent migration reversible. Extract them, replay them, and only then let the paid loop drain. The model swap is the easy line in the runbook; the leftover is every unnamed reason the old while-loop used to stop.

Migration Diary: Normalize Stream Events Before You Leave a Paid Agent Gateway

Riley Xu — Thu, 10 Sep 2026 04:27:26 +0000

You should not cut a paid agent gateway until every client can consume a vendor-neutral stream. The model answer can look correct while leftover SSE shapes, partial tool deltas, and silent usage frames still break your UI. Capture a replay tape first, then normalize events, then dual-run the same fixture on a free runtime before you drain the paid path. This diary treats the stream contract as leftover infrastructure rather than a cosmetic logging concern you can fix later.

The leftover is the event grammar, not the paragraph

Paid agent gateways rarely emit a plain token stream that your existing client can treat as opaque text. They interleave text deltas, tool-call fragments, heartbeat comments, citation objects, and a final usage block that your client already parses. When you move the same prompt onto another runtime, the text can match while the event grammar changes under the client. You then spend the cutover week chasing "random" spinner stalls that are really missing done events.

You do not need a new agent framework to survive that cutover if you already own the client parser. You need a frozen tape, a tiny schema, and tests that fail when an unknown frame appears. The current industry chatter about agents being fancy loops over tools does not cancel this work. A loop still has to stream, and your frontend still has to render partial tool names without crashing.

Freeze four artifacts before you touch DNS or API keys

Do not start the migration by swapping base URLs or by rewriting the production streaming client. Freeze these four artifacts so the paid path and the free path can be compared without storytelling.

Capture a raw SSE tape from the paid gateway for three representative prompts you already ship.
Freeze a canonical event schema with a closed set of types your clients already understand.
Ship a normalizer that maps vendor frames onto that schema and rejects the rest.
Run a dual-run harness that replays the same prompts against the destination runtime.

You should store the tapes next to the schema, not in a personal Downloads folder. If the vendor frame names drift during your notice period, the tape is the only honest regression surface you will have. Commit the tapes with redacted headers so every pull request can replay the leftover frames.

Step 1: Record a paid-gateway tape you can replay

Run the client against the paid gateway with stream logging enabled, and write every frame including comments. Keep the HTTP status, headers, and body bytes, because some gateways hide heartbeats as SSE comments rather than JSON. Label each tape with the prompt id, tool list hash, and whether the run used tools.

Here is a small recorder you can drop in front of any SSE endpoint you already call. Treat the recorder as a proposal you should adapt rather than as a production proxy. Redact authorization headers before the tape file lands in git or in a shared fixture bucket.

# stream_tape.py — proposal: record SSE bytes for later normalization tests
from __future__ import annotations

import json
import time
from pathlib import Path

import httpx


def record_sse_tape(
    url: str,
    payload: dict,
    headers: dict,
    out_path: Path,
) -> Path:
    started = time.time()
    frames: list[dict] = []
    with httpx.Client(timeout=120.0) as client:
        with client.stream("POST", url, json=payload, headers=headers) as resp:
            resp.raise_for_status()
            event_name = "message"
            data_lines: list[str] = []
            for raw_line in resp.iter_lines():
                if raw_line == "":
                    frames.append(
                        {
                            "t": round(time.time() - started, 4),
                            "event": event_name,
                            "data": "\n".join(data_lines),
                        }
                    )
                    event_name = "message"
                    data_lines = []
                    continue
                if raw_line.startswith("event:"):
                    event_name = raw_line[6:].strip()
                    continue
                if raw_line.startswith("data:"):
                    data_lines.append(raw_line[5:].lstrip())
                    continue
                if raw_line.startswith(":"):
                    frames.append(
                        {
                            "t": round(time.time() - started, 4),
                            "event": "comment",
                            "data": raw_line[1:].lstrip(),
                        }
                    )
    out_path.write_text(json.dumps({"url": url, "frames": frames}, indent=2))
    return out_path

Record one tape that streams a tool call across several deltas and one tape that is text only. Record a third tape that errors after a partial answer so the client can show a stable failure. Those three shapes cover most leftover bugs you will actually meet on cutover night.

Step 2: Declare a canonical event schema your clients already speak

Do not invent a grand agent protocol just because the destination runtime uses different frame names. List the events your UI and your tool dispatcher already handle, then refuse everything else. A closed schema is the cutover valve: paid frames map into it, and destination frames must map into it too.

# canonical.py — proposal: closed event set for cutover
from __future__ import annotations

from typing import Literal, TypedDict


class CanonicalEvent(TypedDict):
    type: Literal[
        "text_delta",
        "tool_call_delta",
        "tool_call_finish",
        "error",
        "usage",
        "done",
    ]
    request_id: str
    text: str
    tool_name: str
    tool_args_delta: str
    tool_call_id: str
    error_code: str
    input_tokens: int
    output_tokens: int

You should keep the request_id field mandatory even if the destination runtime never emits one. You can mint a client-side id because leftover log joins and support tickets still key off that field. Treat empty strings and zeros as acceptable placeholders, but let missing keys fail the normalizer. That single rule keeps paid tapes and shadow tapes joinable in the same support query.

Step 3: Normalize vendor frames and fail on unknown leftovers

Write the mapper as pure functions over tape frames so you can unit-test without the network. Unknown event names, leftover citation objects, and vendor-only safety labels should raise in staging builds. In production you may log and skip, but the dual-run harness should fail closed so leftovers cannot hide.

# normalize.py — proposal: map SSE frames to CanonicalEvent
from __future__ import annotations

import json
from typing import Iterable

from canonical import CanonicalEvent


class LeftoverFrameError(ValueError):
    pass


def _base(request_id: str) -> CanonicalEvent:
    return CanonicalEvent(
        type="text_delta",
        request_id=request_id,
        text="",
        tool_name="",
        tool_args_delta="",
        tool_call_id="",
        error_code="",
        input_tokens=0,
        output_tokens=0,
    )


def normalize_frame(frame: dict, request_id: str) -> list[CanonicalEvent]:
    if frame.get("event") == "comment":
        return []  # heartbeats are leftovers; do not forward them
    raw = frame.get("data") or ""
    if raw == "[DONE]":
        event = _base(request_id)
        event["type"] = "done"
        return [event]
    try:
        body = json.loads(raw)
    except json.JSONDecodeError as exc:
        raise LeftoverFrameError(f"non-json frame: {raw[:80]}") from exc

    if "choices" in body:
        return _from_openai_shape(body, request_id)
    if body.get("type") in {"content_block_delta", "message_stop", "message_delta"}:
        return _from_anthropic_shape(body, request_id)
    raise LeftoverFrameError(f"unmapped leftover: {sorted(body.keys())}")


def normalize_tape(frames: Iterable[dict], request_id: str) -> list[CanonicalEvent]:
    out: list[CanonicalEvent] = []
    for frame in frames:
        out.extend(normalize_frame(frame, request_id))
    if not out or out[-1]["type"] != "done":
        raise LeftoverFrameError("tape ended without a canonical done event")
    return out

You should implement the vendor mappers against your real tapes rather than against documentation screenshots. Vendor docs omit the frames that appear only when a tool name is streamed one character at a time. Those partial tool names are the exact frames that freeze a naive streaming UI during cutover.

Add a focused test that replays the captured tool-call tape and checks argument reassembly. Keep the fixture path boring so a missing tape fails CI the same way a mapper bug fails CI. Label the test as unexecuted until the three paid tapes actually exist in the repo.

# test_normalize.py — proposal: fail if leftover frames appear
import json
from pathlib import Path

import pytest

from normalize import LeftoverFrameError, normalize_tape


def test_tool_call_tape_emits_deltas_then_done():
    tape = json.loads(Path("tapes/paid_tool_call.json").read_text())
    events = normalize_tape(tape["frames"], request_id="req_paid_001")
    types = [e["type"] for e in events]
    assert "tool_call_delta" in types
    assert types[-1] == "done"
    joined = "".join(e["tool_args_delta"] for e in events)
    json.loads(joined)  # partial args must reassemble


def test_unknown_vendor_frame_is_a_leftover():
    frames = [{"event": "message", "data": "{\"safety_ribbon\": true}"}]
    with pytest.raises(LeftoverFrameError):
        normalize_tape(frames, request_id="req_paid_002")

Run the tests from the repo root with a plain pytest invocation so the tape stays in source control. Keep secrets out of the tape by redacting headers in the recorder before you commit. If pytest cannot find the tape, the cutover is not ready, even when live demos look fine.

pytest test_normalize.py -q

Step 4: Dual-run the same prompts on a free model path

Keep the paid gateway as the source of truth until the normalizer emits identical canonical sequences for the destination. Identical text is not enough; event types, tool argument assembly, and the terminal done event must match. Where tokenizers differ, compare usage with a tolerance, and never block cutover on exact token counts.

A second runtime earns its keep in this window because you need a shadow stream, not a new vendor story. Disclosure: This article was prepared as part of MonkeyCode's product outreach. You can point the harness at MonkeyCode's free model access and free server option as a shadow destination. Store those shadow frames beside the paid tape so the mapper sees both grammars in one fixture. The product is relevant only as a second place to replay the fixture you already own.

# dual_run.py — proposal: compare canonical streams from two runtimes
from __future__ import annotations

from normalize import normalize_tape


def event_signature(events: list[dict]) -> list[tuple]:
    return [
        (e["type"], e["tool_name"], bool(e["text"]), bool(e["tool_args_delta"]))
        for e in events
    ]


def assert_shadow_compatible(paid_frames, dest_frames, request_id: str) -> None:
    paid = normalize_tape(paid_frames, request_id)
    dest = normalize_tape(dest_frames, f"{request_id}_shadow")
    if event_signature(paid) != event_signature(dest):
        raise AssertionError(
            f"shadow stream grammar drifted: {event_signature(paid)} vs "
            f"{event_signature(dest)}"
        )

If the destination cannot stream tools yet, you should not pretend that text deltas are enough. Park those prompts on the paid path, and cut only the text-only routes that already match. Partial cutovers that follow event signatures stay safer than one big-bang base URL swap.

Step 5: Drain leftovers that will not survive the paid contract

After the grammar matches, you still need to walk leftovers that are not stream events. Paid gateways often inject request ids, retry-after hints, citation bundles, and prompt-cache hashes that your operators still mention in runbooks. Write each leftover down, then assign an owner and a kill date before traffic moves.

Replace vendor request ids with client-minted ids in logs, traces, and support macros before you drain traffic.
Drop SSE comments that only existed to keep a vendor load balancer from closing the socket.
Stop parsing citation or retrieval objects unless you rebuilt that index on the destination side.
Treat prompt-cache hashes as vendor memory, and do not expect them to appear on the free path.
Rehearse a mid-stream disconnect, because the destination will not share the old idle timer.

The leftover that bites most teams is the spinner that waits for a usage frame. If your UI treats usage as the real done, a destination that omits usage will spin forever even though done already arrived. Flip that spinner dependency in the streaming client before you lower any paid traffic.

Decision table for the night you cut

Use this table on the night you cut instead of debating whether the answer looks fine. Each row is a leftover signal you can observe from the dual-run harness without reading the prose. If a prompt class does not fit a row, it stays on the paid gateway until you add one.

Signal on the shadow path	Action	Do not do
Canonical types match, text close enough	Cut that prompt class	Wait for token counts to match
Tool args reassemble, names differ	Freeze tool names in your dispatcher	Let the model alias tools
Unknown frame in destination tape	Keep paid path; extend the mapper	Strip the frame in the UI
No `done` event	Keep paid path; fix the adapter	Time out and show partial text
Usage missing but `done` present	Cut after the UI ignores usage	Block cutover for billing fields

Print the table next to the deploy checklist so the on-call person does not improvise. Arguments about polished paragraphs should lose to a matching row in that table. Keep the table in git beside the tapes so the next destination inherits the same valve.

Limitations, and who should skip this diary

This workflow assumes you own a streaming client and you can record production-like prompts without leaking secrets. If your product only uses unary completions, a stream normalizer is ceremony, and you should instead snapshot request and response JSON. If you cannot dual-run because of data-handling rules, stay on one runtime until you have a redacted fixture set.

The mapper above is labeled as a proposal, not as a certified adapter for every vendor. It does not claim compatibility with every vendor frame, and it does not measure latency, cost, or quality. Token counts will drift across tokenizers, so you should meter those separately rather than treating them as stream correctness. Teams that need vendor citations, built-in file search, or hosted memory should not treat a free model path as a full substitute.

What you should have in git after the cutover

You should have three tapes, one schema, one normalizer, and a dual-run assertion that still runs on pull requests. The paid gateway can then go quiet without taking your event grammar with it into shutdown. If a later destination appears, you record a new tape and extend the mapper instead of rewriting the UI. When those three tapes pass on a free runtime, you can cancel the paid stream path with a lot less theater.

Migration Diary: Rebuild Timeouts and Cancellation Before You Leave a Paid Agent SDK

Riley Xu — Wed, 09 Sep 2026 16:29:53 +0000

You should rebuild the timeout ladder and cancellation path before you leave a paid agent SDK. Paid runtimes hide connect, first-token, tool, and total budgets inside client defaults you never copied. If you cut the API key first, in-flight streams keep retrying the old endpoint while the new runtime looks idle. That leftover is not a prompt file; it is work that still bills and still mutates tools.

The leftover nobody inventories

Most cutover checklists copy prompts, tools, and conversation history, then swap the base URL. That sequence misses the hidden contract the vendor SDK enforced for you on every call. Connect timeouts, read timeouts, retry-on-429, and stream abort were product features rather than code you owned. After the old key dies, those retries become zombie traffic against a disabled project or an unmetered host.

You also lose cancellation semantics that felt automatic while the paid gateway was in the path. A vendor proxy often closes upstream work when the browser tab drops or the socket resets. A raw HTTP client against a new host will keep a tool running unless you thread an abort signal. Duplicate side effects then appear as two refunds, two tickets, or two deploys from one turn.

Treat this writeup as a migration diary rather than a glamorous model rewrite. You freeze generation, drain in-flight work, and install your own ladder before any DNS change. Only after abort is proven should you point any live traffic at the new runtime. The model name sitting in the diff is the least important line in the cutover.

What you extract before any DNS change

Walk the old client with a notebook and write down numbers instead of slogans from the vendor dashboard. You need four budgets and one cancel rule, because most SDKs collapse them into a single timeout= argument. If the console never showed those values, you measure them from real turns rather than copying a blog default. Guessed seconds will flap the moment the first slow tool appears.

Connect budget: how long the TCP handshake and TLS setup may take before you fail closed.
First-token budget: how long you wait for the first streamed chunk before you abort the turn.
Tool budget: how long a single tool may run, including HTTP child calls it makes itself.
Total budget: wall-clock limit for one user turn, covering both the model and every tool.
Cancel rule: who may abort, and whether abort is cooperative for tools or a hard kill.

Wrap one representative turn with timestamps and record p50 plus p95 for each rung of that ladder. Do not copy a thirty-second folklore value and call the result a policy you can defend. Your refund tool and your search tool have different tails, and the paid SDK was hiding that mix. Write the measured numbers into ladder.yaml so reviewers argue about evidence instead of vibes.

A timeout ladder you can actually test

Label the following adapter as a proposal, not production gospel you paste into a payment path. It fails closed, records why it failed, and refuses to start a tool after the parent turn is cancelled. You should run it against a mock model first, then against a non-paid endpoint used only for dry-run traffic. Keep the circuit beside the ladder, because retries without a breaker recreate the paid SDK's worst habit.

# proposed_timeout_ladder.py — proposed adapter, not a production SDK
from __future__ import annotations

import time
from dataclasses import dataclass
from enum import Enum
from typing import Optional


class CutoverFault(str, Enum):
    CONNECT = "connect_timeout"
    FIRST_TOKEN = "first_token_timeout"
    TOOL = "tool_timeout"
    TOTAL = "total_timeout"
    CANCELLED = "cancelled"
    CIRCUIT_OPEN = "circuit_open"


@dataclass(frozen=True)
class Ladder:
    connect_s: float = 3.0
    first_token_s: float = 8.0
    tool_s: float = 12.0
    total_s: float = 25.0


@dataclass
class TurnClock:
    ladder: Ladder
    started: float
    cancelled: bool = False

    def remaining(self) -> float:
        return self.ladder.total_s - (time.monotonic() - self.started)

    def raise_if_dead(self) -> None:
        if self.cancelled:
            raise TimeoutError(CutoverFault.CANCELLED.value)
        if self.remaining() <= 0:
            raise TimeoutError(CutoverFault.TOTAL.value)


class Circuit:
    def __init__(self, fail_limit: int = 3, cool_s: float = 15.0) -> None:
        self.fail_limit = fail_limit
        self.cool_s = cool_s
        self.fails = 0
        self.opened_at: Optional[float] = None

    def allow(self) -> None:
        if self.opened_at is None:
            return
        if time.monotonic() - self.opened_at >= self.cool_s:
            self.opened_at = None
            self.fails = 0
            return
        raise TimeoutError(CutoverFault.CIRCUIT_OPEN.value)

    def record(self, ok: bool) -> None:
        if ok:
            self.fails = 0
            self.opened_at = None
            return
        self.fails += 1
        if self.fails >= self.fail_limit:
            self.opened_at = time.monotonic()

A 429 storm against a free or cheap host is still a storm you caused during cutover week. Fail closed after a short burst, cool down, then send one probe instead of looping forever. The paid SDK made that loop look like resilience, and your bill made it look like progress. Your new runtime will not invoice you politely for the same mistake.

Numbered cutover plan

Follow these steps in order, because skipping drain is how you double-write tools after the key looks disabled. Each step produces an artifact a reviewer can open without sitting in your terminal session. If a step has no artifact, you are still improvising the cutover in production. Freeze writes early, even if that feels slower than swapping a base URL.

Snapshot old SDK timeouts from logs, environment variables, and client constructors, then store them in ladder.yaml.
Freeze new agent sessions on the paid runtime, and let current turns finish or abort through the vendor cancel API.
Install the ladder in a sidecar that still points at the paid base URL, and compare abort reasons for one quiet day.
Add an explicit cancel flag around tool dispatch so a cancelled turn skips every remaining tool, not only the model stream.
Point a canary at the new runtime behind the same ladder, and keep the paid key read-only for replay rather than live tools.
Drain until in-flight counters hit zero, and only then disable the old key in the vendor console.
Keep the leftover list: retry headers, stream heartbeats, and any auto-repeat of failed tools you did not reimplement on purpose.

# ladder.yaml — proposed starting point; replace with measured values
connect_s: 3.0
first_token_s: 8.0
tool_s: 12.0
total_s: 25.0
circuit_fail_limit: 3
circuit_cool_s: 15.0
fail_closed_on_unknown_abort: true
# cancel_sources: browser_drop, operator_abort, total_budget, circuit_open

Print ladder.yaml in the cutover ticket so the review is about numbers rather than a model beauty contest. If someone wants a longer first-token budget, they change a file instead of a folklore comment in chat. If someone wants retries, they have to explain how tools stay idempotent under that retry. That conversation is the migration.

Prove abort actually stops tools

A timeout that logs and then continues is decoration, not a drain plan you can ship. You want a test that starts a slow tool, cancels the turn, and asserts the tool never committed its side effect. Label the suite unexecuted until you wire a real tool behind the same raise_if_dead check. If committed is true after cancel, your cancel path is still a comment.

# test_cancel_drains_tools.py — proposed; wire a real tool before cutover
import threading
import time
import unittest

from proposed_timeout_ladder import Ladder, TurnClock


class FakeTool:
    def __init__(self) -> None:
        self.committed = False
        self.started = threading.Event()

    def refund(self, clock: TurnClock) -> None:
        self.started.set()
        deadline = time.monotonic() + clock.ladder.tool_s
        while time.monotonic() < deadline:
            clock.raise_if_dead()
            time.sleep(0.05)
        self.committed = True


class CancelDrainTest(unittest.TestCase):
    def test_cancel_before_commit(self) -> None:
        clock = TurnClock(Ladder(tool_s=2.0, total_s=5.0), time.monotonic())
        tool = FakeTool()

        def run() -> None:
            try:
                tool.refund(clock)
            except TimeoutError:
                return

        worker = threading.Thread(target=run)
        worker.start()
        self.assertTrue(tool.started.wait(1.0))
        clock.cancelled = True
        worker.join(2.0)
        self.assertFalse(worker.is_alive())
        self.assertFalse(tool.committed)


if __name__ == "__main__":
    unittest.main()

Run it with python -m unittest test_cancel_drains_tools.py before you touch DNS or keys. A slower destination will hit this path more often than a paid low-latency gateway ever did. That is useful pain during canary, not a reason to rip the ladder out. Fix the abort, then move the canary, and only then argue about prompt quality.

Decision table for leftovers

Reviewers argue about models while sockets keep doing the expensive work in the background. Put the leftovers in a table inside the ticket so they cannot hide behind a successful hello-world chat completion. You keep policy you can test, and you drop vendor magic you cannot see. If a row has no owner, that leftover will bill you after the celebration thread.

Leftover from the paid SDK	Symptom after cutover	What you keep	What you drop
Single `timeout=30`	Hang until a proxy kills the socket	Four-rung ladder	One magic number
Retry on every 429	Duplicate tool side effects	Circuit plus idempotency key	Blind retries
Browser disconnect ignored	Tool finishes long after the tab closed	Abort flag on drop	Fire-and-forget threads
Vendor stream heartbeats	False first-token timeouts	Idle ping versus first token	Treating pings as tokens
Auto-repeat failed tools	Double POSTs into your own API	Explicit replay queue	Hidden SDK replay

Heartbeats deserve a special note because they masquerade as liveness while starving first-token detection. If the vendor sent comment chunks or empty deltas to keep a load balancer happy, your new client may treat silence as death. Teach the first-token rung to ignore pings, and teach the total rung to ignore nothing. Those two rules disagree on purpose, and that disagreement is the policy.

Where a free runtime fits, and where it does not

After the ladder lives in your repo, you still need a destination that will not bill you for abort drills. Dry-run traffic is noisy: you cancel mid-tool, you trip the circuit, and you replay traces until the test is boring. That pattern is a poor use of remaining paid quota, and it is a good use of a sink you can afford to bruise. Keep production tools stubbed while the sink absorbs those bruises.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode's free model access and free server option can host that dry-run if you already want a non-paid place to exercise the adapter. Use it as a sink for canary turns and cancel tests, not as a claim that production latency will match the vendor you are leaving. You still own the ladder, the circuit, and the drain checklist. If the free path is slow, that slowness is a gift, because it forces first-token and total budgets to be honest.

Do not treat a free server as an infinite queue for orphaned tools you failed to cancel. Meter concurrent turns, because a missing timeout on your side will fill the box even when the invoice stays at zero. Unmetered free capacity is how leftovers become an outage instead of a line item. The cutover is finished when in-flight work hits zero, not when the new chat UI looks fine.

Limitations, and who should skip this

This adapter does not replace distributed tracing, exactly-once tool semantics, or a durable queue you can inspect after a crash. Threads plus monotonic clocks are a teaching artifact for one process, not a cluster plan. If you run many workers, use a process-wide cancel token and an idempotency store both tools can see. If your tools are not idempotent, no timeout policy will save you from double commits.

You should not use this cutover while a contract still requires production traffic through the paid gateway. You should not use it if you cannot freeze writes for the drain window. You should not point live refunds or payments at a free server while you debug abort behavior. Stub those tools, prove drain, then reattach the dangerous ones behind the same clock.

The approach also assumes you can measure the old SDK instead of guessing from memory of a dashboard screenshot. If you have no logs, you are inventing budgets, and invented budgets will flap under the first slow retrieval call. Spend a day on timestamps before you spend a week retuning prompts for a runtime that is still drowning in zombies. Prompts cannot cancel a refund that already started.

Close the diary

Rebuild the timeout ladder, prove cancel stops tools, and then drain the paid key with counters you can screenshot. The model swap is the easy line in the diff, and it is the line people will want to merge first. The leftover is the work that continued after you thought the session was dead. Copy numbers into ladder.yaml, run the unit test, and only then move the canary off the vendor SDK.

If you need a non-paid sink for those abort drills, you can try MonkeyCode's free model access and free server option as a canary host, then keep the ladder in your repo either way.