Pin the Model, Record What Ran: Version Discipline for LLM Calls

"Which model answered this?" sounds like an easy question until someone asks it about a response from three months ago. If your code called a model by a moving name, read the version from a config file at startup, and logged only the output, the honest answer is often "probably the one we had configured at the time".

This post is a small, practical pattern for answering that question with a record instead of a guess: pin the identifier, record what was requested and what the provider reported, and plan for the day your pinned model is retired.

Aliases move, snapshots don't

Providers name models in two different ways, and the difference matters.

Anthropic's Model IDs and versioning page (no date shown; read 2026-11-05) says "Each Claude model ID identifies a pinned version of the model." It also describes convenience aliases for some earlier models: an alias such as claude-sonnet-4-5 resolves to the most recent dated snapshot for that minor version, for example claude-sonnet-4-5-20250929. For newer generations, the same page says a dateless ID such as claude-sonnet-4-6 is the snapshot itself, and that updated versions ship under a new ID.

Two details on that page are worth keeping in mind:

• The pinning guarantee covers model IDs, not the convenience aliases.
• Weights are fixed for a given ID, but the serving infrastructure around the model can change over time.

So pinning narrows what can change. It doesn't freeze everything, which is one more reason to record what you can observe.

Pinned models still retire

OpenAI's Deprecations page (no date shown; read 2026-11-05) states the other half of the problem: deprecation is "the process of retiring a model or endpoint", and after shutdown the model "will no longer be accessible". It lists minimum notice periods of at least 6 months for generally available models, and says preview models may be retired with much shorter notice, such as 2 weeks.

A pinned ID buys you predictability, not permanence. You will migrate, and the migration is itself a change you'll want to be able to see in your records later.

The pattern

  config / code                  provider
  -------------                  --------
  MODEL_ID (pinned) --request--> model
                                   |
  call record  <------response-----+
   - requested_model   (what you asked for)
   - reported_model    (what the response said, if any)
   - prompt_version    (your template version)
   - params            (temperature, max tokens, ...)
   - request_id, time

Three rules make it work:

1. Pin in code or reviewed config, never in an env var someone edits by hand. A model change should show up in version control with a reason.
2. Record both the requested and the reported model. If your provider's response includes a model identifier, store it next to what you asked for. If they ever differ, you want to know.
3. Version your prompt templates and parameters too. "Same model, different prompt" is still a different system.

A small implementation

Here is a provider-agnostic sketch. The call_model function stands in for whatever SDK you use.

import hashlib, json, time, uuid
from dataclasses import dataclass, asdict

MODEL_ID = "claude-sonnet-4-5-20250929"   # pinned snapshot, reviewed in PRs
PROMPT_VERSION = "summarize-v3"
PARAMS = {"temperature": 0.2, "max_tokens": 400}

@dataclass(frozen=True)
class CallRecord:
    request_id: str
    requested_model: str
    reported_model: str | None
    prompt_version: str
    params_hash: str
    started_at: float

def params_hash(params: dict) -> str:
    blob = json.dumps(params, sort_keys=True).encode()
    return hashlib.sha256(blob).hexdigest()[:16]

def run(call_model, prompt: str):
    started = time.time()
    resp = call_model(model=MODEL_ID, prompt=prompt, **PARAMS)
    record = CallRecord(
        request_id=str(uuid.uuid4()),
        requested_model=MODEL_ID,
        reported_model=resp.get("model"),
        prompt_version=PROMPT_VERSION,
        params_hash=params_hash(PARAMS),
        started_at=started,
    )
    if record.reported_model and record.reported_model != MODEL_ID:
        print("WARN model mismatch", asdict(record))
    return resp, record

And a test that fails loudly if someone swaps the pin for a moving alias:

import re

def test_model_is_pinned():
    # dated snapshot suffix, e.g. -20250929
    assert re.search(r"-\d{8}$", MODEL_ID), MODEL_ID

That regex fits a provider that uses dated snapshots. For a provider whose dateless IDs are already pinned, swap it for an allow-list of approved IDs. The point is that the check lives in your test suite, not in someone's memory.

Migrating without losing the thread

When a retirement notice arrives:

• Add the new ID next to the old one, don't overwrite it. Run your evaluation set against both and keep the results with the change.
• Ship the switch as its own change. Don't bundle it with a prompt rewrite, or you won't know which one moved your metrics.
• Keep old records readable. Records that say requested_model: claude-sonnet-4-5-20250929 should still make sense once that model is retired. Never rewrite history to the new name.

Limits

A call record shows what your application asked for and what the provider reported. It doesn't independently prove which weights ran, and it doesn't make the output correct. It does turn "probably" into a dated, queryable fact about your own system, which is where a good incident review starts.

Where this fits

Recording which model and prompt version produced a result is part of the problem we work on at Obligra. Obligra Verify™ is a system of record for AI-assisted decisions, in public preview, designed to retain the evidence your application selects. The habit above is useful with any storage you choose.

This article was generated by an AI agent (Claude) for the Obligra Verify team and checked against the linked sources.

Comments

[Alex Shev]

Pinning the model is necessary, but recording the whole effective request is what makes an LLM result explainable later. I would keep the model identifier, system prompt version, tool schema version, retrieval snapshot, generation settings, and a request correlation ID. That is enough to compare an unexpected output with the conditions that actually produced it.