Migration Diary: Audit the Tool-Tied Leftovers Before You Switch AI Coding Hosts

Your cutover will not fail because the new host is weaker. It will fail because a file nobody on your team has opened in months is still telling the old tool how to behave.

The risky layer in an AI coding host migration is not model quality, latency, or price. It is the pile of repository files, editor configs, and CI secrets that quietly encode your previous provider's assumptions. Audit that layer first, translate what is worth keeping, and delete the rest only after a dual-run window closes. The workflow below runs in an afternoon on a mid-sized repository and needs nothing but Python and git.

One framing note before we start: hosting choices in this space churn constantly, so I am not going to quote adoption numbers or vendor comparisons here. Every rule below is about your repository, which is the part you actually control.

Step 1 — Freeze an inventory in one command

Before you change anything, produce a machine-readable list of every tool-tied file in the repository. Hand-grepping works until the third contributor adds a config file you forgot existed. Save the scanner in tools/leftover_audit.py and commit it, because you will rerun it at the end of the migration.

#!/usr/bin/env python3
"""leftover_audit.py - inventory tool-tied config before an AI host cutover.

Usage: python3 tools/leftover_audit.py . --json report.json
Exit code 1 when a secret-risk file exists, so CI can gate the cutover.
"""
import argparse, fnmatch, json, sys
from pathlib import Path

SKIP_DIRS = {".git", "node_modules", "vendor", "dist", "build", ".venv"}

RULES = [
    # (classification, glob, note)
    ("vendor_tied",  ".cursorrules",                      "old editor rule file"),
    ("vendor_tied",  ".cursor/*",                         "old editor project config"),
    ("vendor_tied",  ".github/copilot-instructions.md",   "host-specific prompt"),
    ("vendor_tied",  ".aider*",                           "CLI agent config"),
    ("vendor_tied",  ".continue/*",                       "assistant config"),
    ("translate",    "CLAUDE.md",                         "probably portable prose"),
    ("translate",    ".mcp.json",                         "tool-server endpoints"),
    ("portable",     "AGENTS.md",                         "host-neutral instructions"),
    ("portable",     "CONTRIBUTING.md",                   "human onboarding"),
    ("secret_risk",  ".env*",                             "check for embedded keys"),
]

def walk(root: Path):
    for path in root.rglob("*"):
        if any(part in SKIP_DIRS for part in path.parts):
            continue
        if path.is_file():
            yield path.relative_to(root).as_posix()

def main() -> int:
    ap = argparse.ArgumentParser()
    ap.add_argument("root", nargs="?", default=".")
    ap.add_argument("--json", dest="json_out")
    args = ap.parse_args()

    root = Path(args.root).resolve()
    findings = []
    for rel in walk(root):
        for kind, pattern, note in RULES:
            if fnmatch.fnmatch(rel, pattern):
                findings.append(
                    {"path": rel, "class": kind, "matched": pattern, "note": note}
                )

    findings.sort(key=lambda f: (f["class"], f["path"]))
    for f in findings:
        print(f'{f["class"]:<12} {f["path"]:<45} {f["note"]}')

    if args.json_out:
        Path(args.json_out).write_text(json.dumps({"root": str(root), "findings": findings}, indent=2))

    return 1 if any(f["class"] == "secret_risk" for f in findings) else 0

if __name__ == "__main__":
    sys.exit(main())

Run it and commit the report next to the code, so reviewers can diff the inventory later:

python3 tools/leftover_audit.py . --json docs/cutover/inventory-before.json
# non-zero exit means a secret-risk path exists; inspect before proceeding
echo "exit=$?"

The scanner is intentionally dumb, and that is a feature. It matches globs against repo-relative paths, never parses content, and never phones home. If your .env files live one directory deep, add a */.env* rule rather than rewriting the walker.

Step 2 — Classify every hit into three buckets

An inventory is only useful once each entry gets an action. Sort the findings into portable, translate, and retire, then assign an owner and a deadline to each one. The table below is the exact decision rule I use, and it is worth pasting into the migration issue.

Artifact	Default action	Failure mode if ignored
AGENTS.md, CONTRIBUTING.md	Portable — keep as the single source of truth	Duplicate instructions drift apart within a sprint
CLAUDE.md, .cursorrules	Translate into AGENTS.md, then retire	Two rule sets disagree and the new host follows the stale one
.mcp.json	Translate endpoints, re-test each tool server	Silent tool-call failures that look like model errors
.github/copilot-instructions.md	Move content out, keep file only if the old host stays	Reviewer confusion about which file is authoritative
.env*	Secret-risk — rotate anything committed	Keys survive the migration and outlive the vendor

Notice that retire is a real action rather than an optional cleanup. A retired file left in place keeps steering the old tool for anyone who runs it, including the teammate who did not read the migration issue. Delete it in the same pull request that lands the translated version.

Step 3 — Translate rules instead of copying them

Copying a rule file across hosts feels safe and is usually wrong, because the two systems weight instructions differently. Rewrite each rule as a testable statement about the repository, then delete the host-specific phrasing. A rule that says "prefer small diffs" becomes "change at most three files per commit unless the task says otherwise".

<!-- AGENTS.md (host-neutral) -->
## Repository rules
1. Touch only files named in the task; list every extra file in the PR body.
2. Run `npm test -- --runInBand` before proposing a change.
3. Never edit `generated/`; regenerate it with `make gen` instead.
4. Do not add dependencies without a linked issue.

Numbered rules survive translation because each one has an observable check. Prose rules do not, so resist the urge to carry over paragraphs of encouragement. Once the neutral file exists, git grep -n "cursor\|copilot\|claude" -- ':!docs/cutover' should return nothing in your instructions directory.

Step 4 — Dual-run the same task list on both hosts

A cutover without a comparison window is a guess. Build a small task list with an expected file set and a check command per task, then run both hosts against clean clones. The task list is the artifact you will argue about later, so keep it small and representative.

[
  {
    "id": "add-retry",
    "prompt": "Add bounded retry with jitter to src/http/client.ts and nothing else.",
    "expect_files": ["src/http/client.ts"],
    "check": "npm test -- --runInBand src/http"
  },
  {
    "id": "fix-gen-drift",
    "prompt": "Regenerate generated/api.ts with make gen and commit the result.",
    "expect_files": ["generated/api.ts"],
    "check": "make verify-gen"
  }
]

The runner below is a sketch, not a benchmark: the shell-out targets are placeholders for whatever CLI each host exposes, and I have not executed it against a specific vendor binary in this article.

# tools/dual_run.py - adapter sketch; replace the placeholder commands.
import json, subprocess, time

HOSTS = {
    "old": ["your-old-cli", "run", "--prompt", "{prompt}"],
    "new": ["your-new-cli", "run", "--prompt", "{prompt}"],
}

def run(host, task, workdir):
    cmd = [part.format(**task) for part in HOSTS[host]]
    started = time.monotonic()
    proc = subprocess.run(cmd, cwd=workdir, capture_output=True, text=True)
    touched = subprocess.run(
        ["git", "diff", "--name-only"], cwd=workdir, capture_output=True, text=True
    ).stdout.split()
    return {
        "host": host,
        "task": task["id"],
        "seconds": round(time.monotonic() - started, 1),
        "exit": proc.returncode,
        "unexpected": sorted(set(touched) - set(task["expect_files"])),
    }

Run each host in its own clone, on its own branch, and compare only exit, unexpected, and whether the check command passed. Wall-clock seconds are noise unless you run each task several times, so do not put them on a slide after one afternoon.

Step 5 — Put the comparison run somewhere neutral

Local machines drift, and a dual-run that only works on your laptop proves nothing about the next contributor. Run the same task list inside a clean container so both arms see the same Python version, the same lockfile, and the same missing system packages.

This is where a free server option earns its keep: it gives you a disposable, environment-neutral box for the new arm without provisioning anything permanent. MonkeyCode is an open-source project that offers free model access and a free server option, and the team currently advertises a ten-million-token allowance on the free tier; check the project's own repository for the current quota and its expiry rules, because those terms are the kind of thing that changes without notice.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Use it as a smoke environment rather than a throughput proof. A free box tells you whether the task list is environment-neutral; it does not tell you what a busy CI fleet will cost you, and treating one successful run as a capacity answer is how teams end up surprised in week two.

Step 6 — Close the window and re-audit

The migration ends when the inventory diff is empty of unresolved entries, not when the new host first produces a plausible diff. Rerun the scanner, diff the two reports, and delete every file still classified as retire. Then update CODEOWNERS so the neutral instruction file has a named owner instead of the original author's memory.

python3 tools/leftover_audit.py . --json docs/cutover/inventory-after.json
diff <(jq -r '.findings[].path' docs/cutover/inventory-before.json | sort) \
     <(jq -r '.findings[].path' docs/cutover/inventory-after.json | sort)

Anything still appearing in that diff is a leftover, and leftovers are the reason the old tool keeps winning arguments six weeks later.

Limitations and who should skip this

The scanner matches paths, not semantics, so it cannot see rules baked into CI secrets, editor extensions, or a global config under your home directory. The dual-run sketch needs an adapter for whichever CLI each host actually exposes, and the task list is only as representative as the tasks you pick. Nothing here measures model quality, and nothing here justifies picking one host over another on its own.

Skip this if you are a solo developer on a repository with no vendor-specific config, since the inventory will be empty and the harness will cost more than it saves. Skip it as well if the migration is already half-done and you cannot freeze a before-state, because a diff against an unknown baseline hides exactly the leftovers you are looking for.

If you run the audit and want a second opinion on the translation step, the MonkeyCode project's community channels are a reasonable place to compare reports with other people doing the same cutover. Bring the inventory file, not the question, and you will get a useful answer.