Migration Diary: Rehydrate Stored Chat History Without Orphaned Tool Calls

The first thing that breaks in a model migration is rarely the prompt; it is the stored transcript you replay into the new endpoint. Tolerant endpoints hide structural damage for months: tool results whose matching call vanished during last month's context trimming, call ids reused across turns, extra fields nobody ever validated. A stricter endpoint evaluates the same array on arrival, so your cutover dies on message seven with a schema error that describes the provider instead of your actual bug. You can prevent most of that with a small guard that validates and repairs history before rehydration, and this diary entry is the cutover plan for building it.

1. Write the invariants down before you compare vendors

You will not debug a migration by reading two documentation pages side by side, because the failure usually lives in your stored data rather than in their prose. List the structural invariants your transcript store must satisfy instead, then treat every endpoint difference as a test case to run rather than an argument to win.

1. Every tool result references a call id that appears in an earlier assistant message.
2. Every assistant tool call has exactly one result, except a call that is still pending at the tail.
3. Call ids are unique across the whole session, not merely unique inside a single turn.
4. Role ordering satisfies the strictest endpoint you might ever send stored history to.
5. Unknown fields are stripped at the boundary, so a rehydrated payload contains only fields you deliberately keep.

Invariants one and two fail most often in practice, and the cause is usually your own trimming logic. A tail-window trimmer keeps the newest messages and cuts the older assistant call that owned a tool result, leaving a result with no parent. One endpoint ignores that orphan silently, another rejects the request, and your logs end up blaming the new provider for a bug you shipped weeks earlier.

2. Decide what to keep, drop, and rewrite

Before writing any code, agree on a repair policy, because "normalize the history" means nothing to a teammate reading a diff at 2 a.m.

Leftover in the stored history	Verdict before rehydrate	Why
Tool result with no earlier call	Drop	It has no parent turn, so no endpoint can attach it to anything
Assistant tool call with no stored result	Drop the call, keep the text	The call is unresolvable; leaving it invites a strict validator to fail
Duplicate call id inside one session	Reassign new ids	Uniqueness lets you match results to calls after a repair
Unknown or provider-specific fields	Strip at the boundary	Strict validators reject payloads they never documented
Duplicated system prompt	Keep the first one	Two system turns change precedence in ways you cannot predict
Very long middle turns	Trim the middle, keep head and recent tail	Preserves the original task statement and the current state

Rewriting is not a synonym for dropping, and this is the part teams get wrong. When you strip an assistant's tool call, you must leave behind something a model can read, or the following turn looks as if the user answered themselves. Keep the stored transcript immutable and repair a copy, because someone will eventually ask why one session behaved differently on either side of the cutover.

3. The guard: validate first, repair second

The example below uses a plain list of message dictionaries, which maps cleanly onto most chat APIs. It has no third-party dependencies, so you can drop it into an existing service or run it as a standalone CLI over an export file.

# transcript_guard.py -- keep stored chat history structurally valid
# before you rehydrate it against a different model endpoint.

from __future__ import annotations

from dataclasses import dataclass
from typing import Any, Iterable

Message = dict[str, Any]
KEEP_AS_IS = {"role", "content", "tool_calls", "tool_call_id", "name"}


@dataclass(frozen=True)
class Problem:
    code: str
    index: int
    detail: str

    def __str__(self) -> str:
        return f"[{self.code}] message #{self.index}: {self.detail}"


def check_transcript(
    messages: Iterable[Message], *, allow_trailing_call: bool = True
) -> list[Problem]:
    """Return structural problems; an empty list means the history is rehydratable."""
    msgs = list(messages)
    problems: list[Problem] = []
    pending: dict[str, int] = {}
    answered: set[str] = set()

    for i, msg in enumerate(msgs):
        role = msg.get("role")
        if role == "assistant":
            for call in msg.get("tool_calls") or []:
                cid = call.get("id")
                if not cid:
                    problems.append(Problem("call_without_id", i, "tool call emitted with no id"))
                elif cid in pending or cid in answered:
                    problems.append(Problem("duplicate_call_id", i, f"call id {cid!r} reused"))
                else:
                    pending[cid] = i
        elif role == "tool":
            cid = msg.get("tool_call_id")
            if cid not in pending:
                problems.append(Problem("orphan_result", i, f"result {cid!r} has no earlier call"))
            else:
                answered.add(cid)
                pending.pop(cid)

    unresolved = sorted(pending.items(), key=lambda kv: kv[1])
    if allow_trailing_call and unresolved and unresolved[-1][1] == len(msgs) - 1:
        unresolved = unresolved[:-1]  # a call at the tail is a request for work, not damage
    for cid, idx in unresolved:
        problems.append(Problem("unanswered_call", idx, f"call {cid!r} never received a result"))
    return problems


def purge_orphans(messages: Iterable[Message]) -> list[Message]:
    """Drop both halves of every incomplete call/result pair. Returns a new list."""
    msgs = [dict(m) for m in messages]
    call_ids = {
        c.get("id")
        for m in msgs
        if m.get("role") == "assistant"
        for c in (m.get("tool_calls") or [])
    }
    result_ids = {m.get("tool_call_id") for m in msgs if m.get("role") == "tool"}

    repaired: list[Message] = []
    for m in msgs:
        role = m.get("role")
        if role == "tool":
            if m.get("tool_call_id") in call_ids:
                repaired.append(m)
            continue
        if role == "assistant" and m.get("tool_calls"):
            kept = [c for c in m["tool_calls"] if c.get("id") in result_ids]
            if kept:
                m["tool_calls"] = kept
            else:
                m.pop("tool_calls", None)
                if not str(m.get("content") or "").strip():
                    m["content"] = "[tool call dropped during migration: no stored result]"
        repaired.append(m)
    return repaired


def rehydrate(
    messages: Iterable[Message],
    *,
    keep_last: int = 8,
    max_chars: int = 20_000,
    strict: bool = True,
) -> list[Message]:
    """Produce a transcript that is safe to send to a stricter endpoint."""
    msgs = purge_orphans(messages)
    problems = check_transcript(msgs, allow_trailing_call=False)
    if problems and strict:
        raise ValueError("transcript is not rehydratable: " + "; ".join(map(str, problems)))

    system = [m for m in msgs if m.get("role") == "system"]
    rest = [m for m in msgs if m.get("role") != "system"]
    head, tail = rest[:1], rest[1:][-keep_last:]
    while tail and tail[0].get("role") == "tool":
        tail = tail[1:]  # never open the window on a tool result

    window = system + head + tail
    while len(window) > len(system) + 1 and sum(len(str(m)) for m in window) > max_chars:
        window.pop(len(system) + 1)
    return purge_orphans(window)


def strip_unknown_fields(
    messages: Iterable[Message], allowed: set[str] = KEEP_AS_IS
) -> list[Message]:
    """Run this last: it is the boundary that hides provider-specific fields."""
    return [{k: v for k, v in m.items() if k in allowed} for m in messages]


if __name__ == "__main__":
    import json
    import sys

    history = json.load(open(sys.argv[1], encoding="utf-8"))
    found = check_transcript(history)
    for p in found:
        print(p)
    print(f"{len(found)} problem(s) in {len(history)} messages")
    raise SystemExit(1 if found else 0)

Run it as a read-only gate over an export before you touch anything:

python transcript_guard.py transcripts/session-8890.json && echo "clean"

4. Tests that pin the leftovers

A validator without tests is a comment. These three cases encode the failures that actually reached production in migrations I would rather not repeat.

# test_transcript_guard.py
from transcript_guard import check_transcript, purge_orphans, rehydrate

CALL = {"id": "c1", "type": "function", "function": {"name": "get_rate", "arguments": "{}"}}


def test_orphan_result_is_reported():
    history = [
        {"role": "user", "content": "convert 40 EUR"},
        {"role": "tool", "tool_call_id": "c1", "content": "42.10"},
    ]
    assert [p.code for p in check_transcript(history)] == ["orphan_result"]


def test_purge_drops_both_halves_of_an_incomplete_pair():
    history = [
        {"role": "user", "content": "convert 40 EUR"},
        {"role": "assistant", "content": "", "tool_calls": [CALL]},
    ]
    repaired = purge_orphans(history)
    assert "tool_calls" not in repaired[1]
    assert repaired[1]["content"].startswith("[tool call dropped")


def test_trim_never_opens_the_window_on_a_tool_result():
    history = [
        {"role": "system", "content": "be terse"},
        {"role": "user", "content": "turn 1"},
        {"role": "assistant", "content": "", "tool_calls": [CALL]},
        {"role": "tool", "tool_call_id": "c1", "content": "42.10"},
        {"role": "assistant", "content": "about 42 EUR"},
    ]
    window = rehydrate(history, keep_last=2, max_chars=10_000)
    assert [m["role"] for m in window] == ["system", "user", "assistant"]

python -m pytest test_transcript_guard.py -q

The third test is the one that earns its keep, because it fails the moment someone "optimizes" the trimmer and reintroduces the orphan result bug.

5. The cutover sequence

1. Freeze writes to the transcript store for the length of the rehearsal window, or snapshot it so the export stays consistent.
2. Export a sample of real sessions, including at least one long agentic session with several tool calls per turn.
3. Run the checker in read-only mode and record problem counts grouped by code; that histogram tells you which invariant your own code has been violating.
4. Repair copies with rehydrate, then re-run the checker on the repaired copies and assert zero problems.
5. Send each repaired session once to the new endpoint and capture only structural outcomes first: rejected, accepted, or accepted with warnings.
6. Strip unknown fields at the boundary, enable the write path for a small percentage of sessions, and log every structural rejection with its problem codes.
7. Keep the previous endpoint reachable for one rollback window, then delete the old keys deliberately rather than letting them expire by accident.

Step three deserves emphasis: the histogram is your evidence. If every rejected session shares the same code, you have a bug in your own trimming rather than an incompatibility with a vendor, and you can fix it without touching the migration plan.

6. Using a second endpoint for the structural check

A migration diary is only useful if the replacement path is actually exercised, so the repaired copies need somewhere to go. MonkeyCode offers free model access and a free server option, which I used for the structural pass rather than for quality comparison. Disclosure: This article was prepared as part of MonkeyCode's product outreach. That let me point the validator at a second endpoint without opening a second invoice, and the free server option hosted the scheduled checker as a small job that reports problem counts per day. Availability, limits, and terms are set by MonkeyCode and can change, so read the current terms for the project and confirm what your workload needs before you route real traffic there. If you want to reproduce this on your own transcripts, running the guard against a free endpoint first is the cheapest way to learn which invariants your history has been quietly violating.

7. Limitations and who should not use this

Structural repair has a narrow, honest scope, and it is worth naming where it stops being useful.

• Stateless applications that store no history gain nothing; the guard only pays off when you persist and rehydrate conversations.
• Regulated or evidentiary transcripts must never be mutated in place. Repair into a separate store with its own retention policy, and keep the original byte-for-byte.
• Sessions that exceed the new context budget need summarization, not trimming. Cutting the middle changes what the model can answer, and that is a product decision rather than a migration chore.
• Quality regressions are invisible to this guard. A perfectly valid transcript sent to a weaker endpoint still produces worse answers, so keep an output-level check separate.
• Free endpoints and free servers can change limits or availability, so avoid making them the only path for a workflow with an SLA.
• Silent automatic rewriting on every request hides the problem you are trying to measure; run the guard once at the migration boundary, with logging.

Your transcript store is the one artifact that survives every provider change, every pricing decision, and every rewrite of your prompt templates. Ship the guard in continuous integration, fail the build when orphan results appear, and keep your migration energy for the failures that only show up in output quality.