Migration Diary: Rehearse the Rollback Before the Free Endpoint Becomes Primary

The cutover is the easy half of a migration. The half that actually bites is proving you can go back inside your support window, with the same request IDs, the same stored history, and the same tool-call ordering you had an hour before the switch.

Most migration plans I have seen spend their detail budget on the forward path: who flips the flag, when DNS moves, which dashboard to watch. Almost none of them contain a scripted, repeatable rollback, so when the free endpoint starts behaving differently the team improvises under pressure. This entry is the artifact I now build before any cutover: a rollback drill you run twice, once while you still have a comfortable fallback.

1. Write down what "rolled back" means before you touch anything

A rollback is not a config change. It is a claim about state, and claims need definitions you can test. Fill in this inventory for your own system first, because the blanks are where migrations fail quietly.

State	Where it lives	Re-derivable from the fallback?
Trace / request IDs	Your gateway, provider logs	No — you must own the ID
Conversation history + tool calls	Your database	Partly — depends on your schema
Idempotency keys	Your ledger	Yes, if you froze the ledger
Prompt / rule pack version	Your repo, maybe a sidecar	Yes, if it is versioned
Evaluation baseline scores	Your eval store	No — snapshot before cutover
Rate-limit and quota budgets	Provider dashboards	No — record the numbers

Only the middle two rows are usually safe. Everything else needs an owner and a copy that lives with you, not with the provider you are leaving.

2. Put the switch in configuration, and make fallback loud

A rollback you cannot trigger from an environment variable is not a rollback, it is a meeting. Route every model call through one dispatch point, and make the default behaviour fail loudly instead of silently drifting to a different backend.

# backend.py - template, adapters omitted
import os

BACKENDS = {"primary": PrimaryAdapter(), "fallback": FallbackAdapter()}
ACTIVE = os.environ.get("AI_BACKEND", "primary")

if ACTIVE not in BACKENDS:
    raise RuntimeError(f"unknown AI_BACKEND={ACTIVE!r}")

def complete(request, *, backend: str | None = None) -> dict:
    name = backend or ACTIVE
    adapter = BACKENDS[name]
    try:
        return adapter.complete(request)
    except adapter.RetryableError:
        raise  # never fall back implicitly during a drill

The raise on retryable errors is deliberate. Automatic fallback turns your rollback into an untested code path that only executes during an incident, which is exactly when you cannot afford to discover it is broken.

The same rule applies on the server side: keep the old base URL, credentials, and worker image alive and addressable until the drill passes. Deleting the paid workspace on cutover day removes the only thing your rollback was going to talk to.

3. Build the drill harness in four steps

Step 1: Freeze a fixture set

Capture 30 to 50 real requests that cover your awkward cases: long context, tool calls, retries, malformed model output, and one deliberately invalid request. Strip user data, keep the structure, and commit the fixtures. These are the same inputs you will replay against both backends.

Step 2: Record artifacts, not just answers

For each fixture, capture the response text, the ordered list of tool names invoked, the error code if any, and the token counts you were billed. Answer quality is a human judgement; the artifacts above are machine-comparable.

# drill.py - template; normalization rules must be written by you
import json, pathlib

def normalize(resp: dict) -> dict:
    return {
        "tool_order": [c["name"] for c in resp.get("tool_calls", [])],
        "error": resp.get("error_code"),
        "shape": sorted(resp.get("content", {}).keys()),
        "prompt_tokens": resp.get("usage", {}).get("prompt_tokens"),
    }

def run(fixtures: pathlib.Path, backend: str) -> dict:
    out = {}
    for f in sorted(fixtures.glob("*.json")):
        req = json.loads(f.read_text())
        out[f.stem] = normalize(complete(req, backend=backend))
    return out

def diff(a: dict, b: dict) -> list[str]:
    return sorted(k for k in a if a[k] != b.get(k))

Step 3: Flip the switch and replay

Run the fixtures against the fallback first, then against the candidate. Run the drill three times, because a shared free tier will not give you identical latency or identical sampling twice.

Step 4: Assert, then time the flip

Compare the normalized artifacts and record how long the actual environment-variable change takes to reach every worker. That number is your real recovery time, and it is usually larger than the plan assumed because of cached config, container restarts, or a queue that drains slowly.

4. Score the result with a table, not a feeling

Check	Pass condition	If it fails
Tool-call order	Identical for every fixture	Block cutover
Error taxonomy	Every new code mapped to your retry policy	Block cutover
Response shape	Schema-valid after your parser	Block cutover
Token accounting	Counts present and attributable to a request ID	Warn, fix within a week
Text drift	Reviewed by a human on 10 fixtures	Warn, canary eligible

Note that latency is not in the pass column. Latency belongs in monitoring after cutover, not in a go/no-go vote, because a shared free tier is not a latency contract.

5. Inventory the leftovers before you celebrate

Every migration leaves residue, and naming it in advance stops it from becoming folklore six weeks later.

1. Credentials used during the drill. Rotate keys you created for testing, including on the throwaway server.
2. Scheduled jobs. Cron entries and queue consumers still pointing at the old base URL are the classic silent leftover.
3. Log and trace sinks. Assume the target keeps less than you are used to and export to your own storage.
4. Billing alarms. Keep them until the old workspace is genuinely deleted, then delete the alarms too.
5. Prompt cache warmup. Cold starts cost more tokens and produce different timing on day one.

6. Where the free tier fits in this drill

This is the part where the tooling matters. MonkeyCode offers free model access and a free server option, and the useful role for both in this workflow is a second environment: you can run the rollback drill against a scratch server without borrowing production capacity or paying for another workspace. The operator states that free access comes with a 10M-token allowance and that the server side is available at no cost; treat those numbers as current-at-time-of-writing and confirm the live terms before you plan a budget around them. The project is described as open source, so check the repository and license directly rather than trusting a diary entry.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

I would use it here specifically because a rollback drill is disposable work: it needs an environment for an afternoon, produces artifacts you keep, and should never touch real traffic.

Limitations and who should skip this

This drill proves state parity and switch mechanics. It does not prove quality parity, and it does not tell you whether a different model family will pass your evaluations. If your workload has a contractual latency SLO, a data-residency requirement in a specific region, or a regulated audit trail, a free shared tier is the wrong primary and you should stay on a paid agreement you can point at in a review.

If your migration is a single internal script with no stored history, the whole table above is overkill and a one-page checklist will do. And if nobody owns the fixture set, the drill will rot within a month, so assign that person before you assign the cutover date.

Start with the rollback, not the rollout. If you need a spare environment to rehearse in before you commit, the free server is a low-risk place to run the first three drill cycles.