Migration Diary: The Decommission Step Nobody Puts on the Cutover Calendar

You can move a working AI workflow to a new host in an afternoon, but you will keep paying for the old one for months. The provider you stop calling does not stop calling you, because webhooks keep firing, CI jobs keep holding stale keys, and one forgotten cron job keeps a legacy account warm and billable. This entry is about the third phase of any migration that most runbooks quietly skip: decommissioning. Below is the audit I run before deleting anything, plus the four-bucket table that decides what actually gets revoked.

The conclusion first: teardown is a phase, not a cleanup task

Treating teardown as cleanup means it happens in fragments, whenever someone notices a stray 401 in a log or an invoice line they cannot explain. Treating it as a phase means it gets an owner, a start date, and an exit criterion you can point at. The exit criterion is not "we deleted the key" but "no request has hit the old endpoint for thirty consecutive days, and every reference in the repo is either removed or annotated."

That distinction matters more when the destination is cheap, because cheap destinations remove the financial pressure that used to force a tidy shutdown. When the old account cost real money every month, someone eventually chased it down. When the new side of the migration is a free tier, the old side can sit there for a year looking harmless right up until an audit or an incident asks why a retired vendor still holds credentials.

Step 1 — Inventory every outbound reference before you revoke anything

Start with three cheap searches that cover most of the surface area. Run them in the repository root and paste the output into the migration ticket, because the raw list is the evidence you will be asked for later.

# 1. Code, config, and docs that mention the retiring vendor
git grep -n -iE "oldprovider|legacy-llm" -- ':!*.lock' ':!node_modules'

# 2. SDK dependencies still pinned in your manifests
grep -n "old-sdk" requirements*.txt pyproject.toml poetry.lock package-lock.json 2>/dev/null

# 3. Secret names currently present in your shell (names only, never values)
env | grep -iE "(KEY|TOKEN|ENDPOINT|BASE_URL)" | cut -d= -f1

Those three commands miss things that live outside the repo, so the next artifact is a small read-only script that walks a directory tree and produces a single JSON report. It does not call any API, it does not modify files, and it is a starting point rather than a finished tool. Read it before running it, and edit OLD_MARKERS first.

#!/usr/bin/env python3
"""
teardown_audit.py - locate every reference to a retiring AI provider.

Read-only: writes one JSON report, touches nothing else.
Label: template script, adapt OLD_MARKERS and TEXT_SUFFIXES to your stack.
"""
import json
import re
from pathlib import Path

OLD_MARKERS = ["oldprovider", "legacy-llm", "old-provider.example.com"]

SKIP_DIRS = {".git", "node_modules", ".venv", "venv", "dist", "build", "__pycache__", ".next"}
TEXT_SUFFIXES = {".py", ".js", ".ts", ".tsx", ".go", ".rb", ".yml", ".yaml", ".toml",
                 ".json", ".env", ".md", ".sh", ".tf"}
ENV_RE = re.compile(r"\b([A-Z][A-Z0-9_]*(?:KEY|TOKEN|SECRET|ENDPOINT|BASE_URL))\b")


def iter_files(root: Path):
    for path in root.rglob("*"):
        if not path.is_file() or path.suffix not in TEXT_SUFFIXES:
            continue
        if SKIP_DIRS & set(path.parts):
            continue
        yield path


def main(root: str = ".") -> None:
    report = {"markers": OLD_MARKERS, "files": {}, "env_names": []}
    seen = set()
    for path in iter_files(Path(root)):
        try:
            text = path.read_text(encoding="utf-8", errors="ignore")
        except OSError:
            continue
        lowered = text.lower()
        hits = [m for m in OLD_MARKERS if m.lower() in lowered]
        if hits:
            report["files"][str(path)] = hits
        for name in ENV_RE.findall(text):
            if name not in seen:
                seen.add(name)
                report["env_names"].append(name)
    report["env_names"].sort()
    report["file_count"] = len(report["files"])
    Path("teardown_report.json").write_text(json.dumps(report, indent=2), encoding="utf-8")
    print(f"{report['file_count']} files reference a retiring marker")
    print(f"{len(report['env_names'])} candidate secret or endpoint names found")


if __name__ == "__main__":
    main()

The value is not the detection, it is the inventory becoming a reviewable list. You now have a file you can diff after every deletion, so progress is measurable instead of remembered. Run it on the main branch and again on your longest-lived feature branch, because long branches are where dead references survive.

Step 2 — Sort every hit into one of four buckets

A flat list of hits invites indiscriminate deletion, and indiscriminate deletion is how a migration turns into an outage. Sort each finding into one of four buckets, write the bucket next to the file path in the ticket, and let a second person agree before execution.

Bucket	Typical example	Action	Verification
Revoke now	Old API key in CI secrets	Delete secret, rotate at source	Pipeline fails loudly with 401, not silently
Repoint	LLM_BASE_URL still on the old host	Change value, keep the name	Health-check endpoint returns 200
Hold for N days	Inbound webhook receiver	Keep it, switch to log-only mode	Zero deliveries for N days
Never touch	Historical migration scripts	Leave it, add a dated comment	Comment states owner and reason

The "hold" bucket is the one people skip, and it is the one that catches surprises. An inbound webhook you did not know about will keep arriving after cutover, and if you delete the receiver you find out from a customer rather than from a log. Switch it to log-only first, then wait for the counter to reach zero.

The "never touch" bucket matters for the same reason in reverse: some references should stay because they document what happened. A migration script from two years ago that mentions the old host is archaeology, not debt, and deleting it costs you the ability to explain a historical data shape.

Step 3 — Rotate secrets, do not just delete them

Deleting a key from a secrets store removes your copy, not everyone else's. Keys leak into CI logs, shell history, screenshots in tickets, and the environment of a container image you built six months ago. Rotation is the only action that makes a leaked key worthless.

Sequence it so you never hold two valid generations longer than necessary: create the replacement, deploy it, confirm the new one is serving traffic, then revoke the old one, then check the logs for calls that still present it. That last check is the one that finds the job nobody remembered, and it is worth keeping for a full week rather than a single afternoon.

# After revoking, watch for stragglers presenting the retired credential
journalctl -u your-service --since "7 days ago" | grep -c "credential_revoked"

If that count is not zero after a week, you have a client you have not inventoried, and the audit script in Step 1 needs a new marker.

Step 4 — The thirty-day zombie check

Before you close the account, prove that nothing depends on it. Export the request logs and the billing detail for the last thirty days and confirm the call count is zero rather than merely small. A trickle of one request a day is more dangerous than a flood, because it never trips an alert and it keeps the billing relationship alive.

Write the closing date in the runbook next to the cutover date, and put the export somewhere the next person can find it. When the same question comes back in a year, the answer should be a file, not a memory.

Where a free destination changes the teardown math

Most stalled decommissions survive on one sentence: "keep the old account active in case we need to switch back." That sentence is expensive, and it is the main reason migrations never finish. If your destination offers free model access and a free server option, the incentive flips, because there is no monthly line item arguing for the zombie to stay alive.

I keep the rollback path open until the thirty-day check passes, then close it deliberately rather than letting it lapse. In that window, a destination such as the MonkeyCode open-source project is one option worth evaluating against your own workload, since free model access and a free server option mean the post-cutover period does not add cost. Confirm the current terms and limits on the project page before you plan around them, because availability claims are time-sensitive and quotas change.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

If you want to compare it against your existing setup, the useful test is not a chat window but your own audit script: point a sample of your real requests at the new host, keep the log-only window open, and see whether the four-bucket table still holds.

Limitations and who should skip this

This approach assumes you can enumerate your outbound references, which is false for teams with unmanaged shadow integrations. If multiple teams call the retiring provider outside your repo, the audit script finds nothing and gives you false confidence.

Skip the aggressive teardown entirely if the old provider holds artifacts you still need, such as embeddings, fine-tuned weights, or evaluation datasets that were never exported. Export first, verify the export, and only then start revoking.

Also skip it if you have legal hold or retention obligations that name the vendor, because decommissioning then becomes a compliance task with its own sign-off. And if you are a solo operator without on-call coverage, do the revoke steps during business hours; a silent 401 on a Friday night is a worse teacher than a planned outage on a Tuesday morning.

The pattern that generalises is simple: inventory, bucket, rotate, wait, close. It costs a few hours and it is the difference between a migration that finished and one that merely moved.