Migrating an Internal Glossary of Model-Specific Terminology

#ai #machinelearning #llm #programming

After a migration your internal documentation contains two vocabularies that use several of the same words for different things. The fix is not a find-and-replace; it is a column that says what each word means where.

Why a word list is the wrong artifact

The usual response to “our docs are out of date after the migration” is to open the glossary page and swap the old provider’s nouns for the new one’s. That handles the harmless case — a term only one provider uses, which reads as obviously stale and gets corrected on sight the first time anybody trips over it.

It does nothing about the case that costs time, which is a word both providers use, for concepts that are similar enough that nobody notices they differ. Those entries survive a find-and-replace untouched, because the word is right. An engineer reads the sentence, applies the meaning they know, and writes code against a contract that does not hold. The glossary was not wrong; it was underspecified, and the migration is what exposed it.

The collisions that actually cause bugs

Four are worth working through, because each has produced a real class of defect.

“System prompt”. On the Anthropic Messages API it is a top-level system parameter sitting beside messages. On OpenAI’s Chat Completions it is a member of the message array with a role. Internal documentation that says “put the persona in the first message” is a correct instruction on one and produces a request that either fails validation or is silently treated as user text on the other.
“Max tokens”. Three identifiers for adjacent ideas: max_tokens on the Messages API, where it caps output and is required; max_completion_tokens on OpenAI Chat Completions, which supersedes max_tokens for reasoning models and counts reasoning tokens as well as visible ones; and max_output_tokens on the Responses API. A doc that says “set max tokens to 1000 for a short answer” means three different budgets. This one is worked through in full on the output-length page.
“Cache”. On one side an explicit, author-controlled thing you place breakpoints for, reported as cache_creation_input_tokens and cache_read_input_tokens. On another, an automatic behaviour you do not control, reported as prompt_tokens_details.cached_tokens. A runbook that says “add a cache breakpoint above the tools block” has no meaning against an API with no breakpoints, and a cost model that assumes caching is free to enable has no meaning against an API that charges for the write.
“Stop reason”. The field is stop_reason in one place and finish_reason in another, and the value sets are not a renaming of each other — one enumerates end_turn, tool_use, pause_turn, model_context_window_exceeded among others; the other stop, length, tool_calls, content_filter. Prose that says “check the stop reason is normal” is not actionable in either.

A fifth is worth adding because it is a unit rather than a field, and units are the hardest thing to catch. “Token” is provider-specific: each provider ships its own tokenizer, so a document that fits in a thousand tokens on one side may not on the other, and the gap widens on non-Latin scripts. Any internal number expressed in tokens — a chunk size, a truncation limit, a per-request budget, a cost estimate — is therefore a number with an implicit provider attached. The glossary entry for it should say so in one line, because the alternative is that somebody reads “chunks are capped at 800 tokens” and reasonably assumes it is a portable fact.

The pattern across all five: the word names a concept that exists on both sides, and the identifier, the control surface, the value set or the unit differs. That is exactly what a glossary should record and exactly what a one-line definition cannot.

A glossary row that carries enough

Give every entry five fields. The first is your own term — a name you control, which appears in your prose and in your code, so that documentation does not have to pick a provider’s word as the canonical one. The rest anchor it.

# glossary/output-cap.yaml
term: output cap
definition: >
  The maximum number of tokens the provider will generate for one
  response before truncating. Ours; used in prose and in code.
maps_to:
  provider_a: { field: max_tokens, required: true,  counts: "visible output only" }
  provider_b: { field: max_completion_tokens, required: false,
                counts: "visible output plus reasoning tokens" }
differs: >
  On provider_b a reasoning-heavy request can consume the whole budget
  before emitting visible text, returning empty content with a
  length-style finish reason.
used_in:
  - services/summarise/config.ts
  - docs/runbooks/truncation.md

The differs field is the one that earns its keep and the one people leave blank. If it is empty, either the concepts really are identical — in which case say so explicitly, which is itself useful — or nobody has looked hard enough. The used_in list is what makes the glossary maintainable: at the next migration it tells you which files to open, which is the difference between an hour and a week.

Keeping the docs from drifting back

A glossary corrected once and not enforced is a glossary that is stale again in two quarters, because the pressure that produced the mixed vocabulary — people writing docs while looking at a provider’s reference page — has not gone away. The cheap enforcement is a lint over your documentation tree.

Build the set of provider-specific identifiers from the maps_to blocks of every glossary entry. This is generated, not hand-maintained, so it cannot fall behind the glossary.
Scan Markdown files for those identifiers appearing in prose — outside inline code spans and fenced blocks. Inside code they are correct and expected; in a sentence they mean somebody wrote a provider’s word where your term belongs.
Fail with a message that names the glossary term to use instead, so the fix is obvious rather than a puzzle. A lint that only says “forbidden word” gets suppressed.
Allow an inline escape for the pages that genuinely are about one provider’s API surface, and require the escape to name which provider. Those pages exist and should not be fought.

Run it in CI on documentation changes only. It is a hygiene check, not a gate on shipping code, and treating it as the latter is how it ends up disabled.

What not to put in it

Two things belong elsewhere. Model names, context lengths and per-model capabilities are not glossary entries — they change on a different clock and belong in a capability matrix that is generated where possible; see migrating a capability matrix and the model reference doc. And prices do not belong here at all: they are data with an effective date, and putting them in prose is how a stale number ends up in a business case.

What belongs is the small set of words your team says out loud every day and means slightly different things by depending on which provider they were reading last. There will be a dozen or so. A glossary of a dozen entries that are all disambiguations is worth far more than a hundred entries that restate definitions available in any vendor’s documentation.