Migration Diary: Stamp Idempotency Keys on Tools Before You Leave a Paid Agent

Paid agent runtimes retry mutating tools after timeouts, dropped streams, and loop restarts, and they often hide those retries from you. You should stamp an idempotency key on every side-effecting tool before you leave that paid loop. The leftover after cutover is not a missing system prompt; it is a duplicate pull request, comment, or deploy. Treat retry as a first-class protocol, not as an SDK quirk you hope the vendor already solved.

Why the second write appears only after you migrate

Paid coding agents wrap tool dispatch, HTTP clients, and filesystem writes behind one retry budget you cannot inspect. When a stream dies, the vendor may replay the last tool call while your local process also retries it. You only notice after two issues exist, two branches land, or two webhooks fire against production. Moving the loop to another runtime does not remove that race; it merely makes the second write cheaper and easier to miss in logs.

Read-only tools can be replayed without a key, but anything that creates, updates, or notifies needs an envelope. You should classify tools before you rewrite prompts, because prompt edits will not stop a double create_pr call. Keep the paid agent running in drain mode until every in-flight key either commits or expires in your store.

Step 1: Inventory side effects, not model names

Walk the tool list the paid agent actually invoked during the last week of traces, not the catalog in marketing docs. Mark each tool as pure, idempotent-if-keyed, or needs-compensation, and refuse to cut over while any mutating tool stays unclassified. Capture the natural key the backend already understands, such as a branch name, issue id, or object path.

1. Export recent tool traces from the paid runtime into a local JSONL file you control.
2. Tag every record with pure, keyed, or compensate using the decision table below.
3. Freeze new mutating tools in the paid product until those tags exist in version control.
4. Reject any cutover plan that only copies system prompts and leaves tool transport unchanged.

python3 - <<'PY'
import json, collections, pathlib
path = pathlib.Path("paid_tool_trace.jsonl")
counts = collections.Counter()
for line in path.read_text().splitlines():
    rec = json.loads(line)
    name = rec.get("tool") or rec.get("name") or "unknown"
    counts[name] += 1
for name, n in counts.most_common():
    print(f"{n:5d}  {name}")
PY

You now have a frequency list, which is more honest than a vendor tool catalog. High-frequency mutators are the leftovers that will clone themselves on the free host. Sort that list before you touch model routing, because routing changes amplify retries instead of removing them.

Step 2: Stamp a stable envelope around every mutating call

An idempotency key must be derived from intent, not from a random UUID generated after the HTTP client already fired. Hash the tool name, the normalized arguments, and the agent-run id so a replay of the same thought hits the same key. Store the first response beside that key, then return the stored payload when the model asks again with identical intent.

# Proposed local helper. Label it unexecuted until you point it at a real trace file.
from __future__ import annotations

import hashlib, json, sqlite3, time
from dataclasses import dataclass
from typing import Any, Callable

SCHEMA = """
CREATE TABLE IF NOT EXISTS tool_calls (
  idempotency_key TEXT PRIMARY KEY,
  tool_name TEXT NOT NULL,
  request_json TEXT NOT NULL,
  response_json TEXT,
  state TEXT NOT NULL,
  created_at REAL NOT NULL,
  updated_at REAL NOT NULL
);
"""

def intent_key(run_id: str, tool_name: str, args: dict[str, Any]) -> str:
    blob = json.dumps({"run_id": run_id, "tool": tool_name, "args": args}, sort_keys=True)
    return hashlib.sha256(blob.encode()).hexdigest()

@dataclass
class Envelope:
    key: str
    tool_name: str
    args: dict[str, Any]

class ReplayStore:
    def __init__(self, path: str = "tool_replay.sqlite") -> None:
        self.conn = sqlite3.connect(path)
        self.conn.execute(SCHEMA)
        self.conn.commit()

    def execute(self, env: Envelope, send: Callable[[dict[str, Any]], Any]) -> Any:
        now = time.time()
        row = self.conn.execute(
            "SELECT state, response_json FROM tool_calls WHERE idempotency_key = ?",
            (env.key,),
        ).fetchone()
        if row and row[0] == "committed":
            return json.loads(row[1])
        if row and row[0] == "inflight":
            raise RuntimeError(f"tool {env.tool_name} still inflight for {env.key[:12]}")
        self.conn.execute(
            "INSERT OR REPLACE INTO tool_calls VALUES (?,?,?,?,?,?,?)",
            (env.key, env.tool_name, json.dumps(env.args, sort_keys=True), None, "inflight", now, now),
        )
        self.conn.commit()
        try:
            result = send(env.args)
        except Exception:
            self.conn.execute(
                "UPDATE tool_calls SET state = ?, updated_at = ? WHERE idempotency_key = ?",
                ("failed", time.time(), env.key),
            )
            self.conn.commit()
            raise
        self.conn.execute(
            "UPDATE tool_calls SET state = ?, response_json = ?, updated_at = ? WHERE idempotency_key = ?",
            ("committed", json.dumps(result), time.time(), env.key),
        )
        self.conn.commit()
        return result

Keep run_id stable across stream retries inside one agent attempt, and mint a new run_id only when the user starts a distinct task. If you rotate run_id on every token timeout, you will stamp a new key and create the duplicate you were trying to prevent. Log the key prefix beside the tool name so later leftovers can be grepped without dumping argument blobs into chat.

Step 3: Decide retry, replay, or compensate with a table

Not every failure should call send() again. Some tools are safe to replay from the store, some must wait, and some need an explicit undo. Use this table during cutover reviews so engineers stop arguing from model folklore.

Tool shape	Example	On timeout	On identical retry	On confirmed extra write
Pure read	get_file, search	retry send	retry send	not applicable
Keyed create	open_pr, create_issue	wait, then replay store	return stored payload	close or comment the extra object
Keyed write	put_file, update_status	wait, then replay store	return stored payload	overwrite with the stored canonical body
Effect with no natural key	charge_card, page_oncall	do not auto-retry	refuse until a human key exists	run a documented compensating action

DECISIONS = {
    "get_file": "pure",
    "search_repo": "pure",
    "open_pr": "keyed",
    "create_issue": "keyed",
    "put_file": "keyed",
    "merge_pr": "compensate",
    "notify_slack": "compensate",
}

def classify(tool_name: str) -> str:
    if tool_name not in DECISIONS:
        raise ValueError(f"refuse unclassified tool: {tool_name}")
    return DECISIONS[tool_name]

If a tool cannot be classified, you do not migrate it. Paid runtimes love unclassified helpers because they look like convenience; after cutover they become silent duplicates. Put refuse unclassified tool in the dispatcher so a new vendor tool cannot sneak onto the free host during the first weekend.

Step 4: Prove replay with a test you can run offline

Do not trust a dashboard green check. Write a local test that sends the same envelope twice and asserts one underlying side effect. Keep the test free of network calls by injecting a fake send that increments a counter.

# Proposed test. Run it against the helper above before any production drain.

def test_second_call_does_not_resend(tmp_path):
    store = ReplayStore(str(tmp_path / "replay.sqlite"))
    hits = {"n": 0}

    def send(args):
        hits["n"] += 1
        return {"pr_url": "https://example.invalid/pr/1", "args": args}

    args = {"title": "meter retries", "head": "feat/keys", "base": "main"}
    env = Envelope(intent_key("run-22", "open_pr", args), "open_pr", args)
    first = store.execute(env, send)
    second = store.execute(env, send)
    assert first == second
    assert hits["n"] == 1

python3 -m pytest -q test_tool_replay.py

A passing test means the leftover path is closed for that tool shape, not that the paid vendor is gone. Add one test per mutating tool class, especially comment posting and file writes, because those are the duplicates readers will see first. If the test cannot run without the paid SDK, you have not extracted the protocol yet.

Step 5: Dual-run, then drain, then disable paid dispatch

Cutover is a drain problem, not a prompt-paste problem. You keep the paid loop alive only for runs that already hold inflight rows, and you start new runs on the destination host with the same envelope code. Compare keys, not prose summaries, when you declare the migration done.

1. Deploy the replay store beside both runtimes and refuse tools that cannot stamp a key.
2. Route new agent runs to the destination host while the paid host accepts only resume traffic.
3. Watch inflight rows until they become committed or failed; do not kill the paid process early.
4. Diff tool names and key prefixes across both logs and treat unmatched mutators as blockers.
5. Disable paid dispatch only after the unmatched set stays empty for a full drain window you chose in advance.

sqlite3 tool_replay.sqlite "SELECT state, COUNT(*) FROM tool_calls GROUP BY state;"
sqlite3 tool_replay.sqlite "SELECT tool_name, substr(idempotency_key,1,12), state FROM tool_calls WHERE state != 'committed';"

Empty inflight plus zero unclassified tools is your go signal. Fancy model quality scores are not. If a row sits in inflight longer than your tool's own timeout, you have found a hung side effect and you must inspect it by hand before disabling the paid dispatcher.

Once keys and a replay log exist, you can run the same envelope against another model host without changing tool contracts. Disclosure: This article was prepared as part of MonkeyCode's product outreach. You can replay a drained, key-stamped loop against MonkeyCode's free model access and free server option without renaming tools or inventing a second dispatcher.

Leftovers to hunt after the paid agent is quiet

Duplicates hide in objects the model never mentions again. Search for paired pull requests with the same head branch, paired issue titles with the same hash prefix, and paired bot comments posted within one retry window. Those records are the migration leftovers, and they will not appear in a prompt diff.

# Proposed git forensics. Adjust the since stamp to your drain window.
git log --since='2026-09-06' --pretty='%h %s' | awk 'seen[$2]++{print}'

If your hosting platform allows, list webhook deliveries grouped by idempotency header and delete or cancel the extras. Do not ask the model to "clean up duplicates," because that request is another unkeyed mutation. Cleanup belongs in a script that reads the replay store and the decision table together.

Limitations, and who should not use this approach

This envelope gives you at-most-once intent at a single worker, not a distributed exactly-once transaction across regions. A SQLite file will not protect two replicas that stamp keys at the same moment without a real lock service. Hashing arguments also fails when servers inject timestamps, unsigned maps, or absolute paths that change between retries.

Compensation is not magic. Closing an extra pull request is easy; reversing a customer-visible charge is not, and this diary does not claim otherwise. Clock-based expiry on keys can resurrect duplicates if you expire too fast, so prefer explicit run_id lifetime over short TTLs copied from web frameworks.

Skip this approach if every mutating backend already requires an idempotency header you already plumb end to end. Skip it if your tools only read state and never write. Skip it if you need multi-region consensus, because a local replay file will lie to you the first time two workers overlap. Skip it if you expect any free host to inherit the paid vendor's hidden retry budget, timeouts, or lease behavior; those budgets are what you are extracting, not what you should assume still exist.

The durable artifact is the key, the store, and the drain query, not a rewritten persona prompt. Leave the paid agent only after those three objects survive a replay test and an empty inflight list. The model can change on the next host. The side effects should not.