This post was created with AI assistance and reviewed for accuracy before publishing.
Every model provider retires models. The identifier you pass in the model field is a version string with a lifecycle, and unlike your npm dependencies nothing in your toolchain tells you when it is about to end.
The failure is undramatic and annoying. A string that has worked for a year starts returning errors, and the person on call has to find where it is configured, which turns out to be five places, three of them not in the application repository.
Find every place the string lives
Before anything else, find out how many there are. In most codebases the answer is more than expected:
grep -rn "claude-" --include="*.ts" --include="*.py" --include="*.env*" \
--include="*.yml" --include="*.tf" . | grep -v node_modules
The ones that hurt are outside the application code: a Terraform variable, a CI secret, a dashboard config, a seed script, a colleague's local .env. Each is a separate thing to update under time pressure, and the ones you forget fail later than the ones you find.
One module owns the mapping
Collapse them into a single place that maps a role to an identifier, and have the application refer to roles:
// models.ts is the only file that names a model.
export const MODELS = {
chat: process.env.MODEL_CHAT ?? 'claude-sonnet-4-5',
summarize: process.env.MODEL_SUMMARIZE ?? 'claude-haiku-4-5',
} as const;
// Everywhere else:
await client.messages.create({ model: MODELS.chat, ... });
The environment variable allows an override without a deploy, which is what you want during an incident. The default in code means a missing variable does not produce an empty model field and a confusing error.
Naming by role rather than by model is the part that pays off. MODELS.summarize stays meaningful across three model generations; CLAUDE_HAIKU_MODEL becomes a lie the first time you point it at something else.
Version the choice deliberately
Providers usually offer both a dated identifier and a moving alias. They fail in opposite directions.
| Form | Behaviour | Risk |
|---|---|---|
Dated, e.g. claude-haiku-4-5-20251001
|
Frozen | Sunsets on a schedule |
| Alias, e.g. a latest-style pointer | Moves | Behaviour changes without a deploy |
For production, the dated form is usually right, for the same reason you commit a lockfile: you want the thing you tested to be the thing that runs. The cost is that you own the upgrade, which is the point.
Aliases are fine in development and evaluation environments, where discovering a change early is useful rather than alarming.
Subscribe, then check on a schedule
Providers publish deprecation notices in documentation and changelogs, and send console or email notices to account owners. Two habits close the gap.
Route provider notification emails to a shared inbox rather than an individual. Deprecation notices have a way of arriving while the person who set up the account is on holiday, or after they have left.
Then check the deprecations page as a recurring task, roughly monthly. That sounds primitive next to automation, and it is more reliable than most automation, because a page layout change silently breaks a scraper while a calendar reminder does not.
Make the upgrade routine boring
When a new identifier is available, the path should already exist:
- Point staging at the new model via the environment variable.
- Run your evaluation set against it and compare outputs to the current model.
- Check cost and latency, since both frequently change.
- Promote by changing the production variable.
- Update the default in code so the variable is no longer load-bearing.
Step two is the one that requires prior investment. If you do not have a stored set of representative inputs with known-good outputs, every model change is a leap, and the only feedback is user complaints a week later. A few dozen cases is enough to catch a regression in format, tone, or refusal behaviour.
Cost and latency deserve their own check because they move independently of quality. A newer model that is better and three times the price is a product decision, not an automatic upgrade.
Fail loudly on an unknown model
Add a startup assertion that the configured identifier is one you recognise:
const KNOWN = new Set(Object.values(MODELS));
if (!KNOWN.has(configured)) {
throw new Error(`Unknown model: ${configured}`);
}
A typo in an environment variable otherwise surfaces as a runtime API error on the first user request, in whichever code path happens to run first. Failing at boot puts the error in the deploy log, where someone is already watching.
None of this is sophisticated. It is the same discipline you already apply to dependency versions, extended to a string that behaves exactly like one but arrived through a different door.
Top comments (0)