The Model You Built On Changed Overnight

Two stories hit Hacker News this week within a day of each other. One celebrates a model that lands "near-frontier intelligence for a fifth of the price." Another asks, plainly: "Has Opus 5.5 been nerfed yet?" Same week, a major provider had a partial outage, and a subscription tier shifted its limits again.

For a hobbyist, that's just the news cycle. For anyone who has welded a specific model into a live product — a support bot, a listing generator, a translation pipeline — every one of those headlines is a supply-chain event. And most builders don't treat them that way until something breaks in front of a customer.

Three ways your model changes under you

The model is not a fixed component. It is a moving part with a vendor on the other end whose incentives are not yours. It changes in three ways, and only one of them is loud:

1. Capability drift. A model gets quietly retuned. Output style shifts, refusals move, formatting changes. Nobody emails you. Your prompts, tuned against the old behavior, now produce subtly worse results — and you find out from a customer, not a changelog.

2. Pricing changes. "A fifth of the price" is a great pitch until the next pricing page. Costs per token move in both directions, and they move on the vendor's schedule. If your unit economics assume today's rate, you're running on a model you don't control.

3. Availability. Deprecations, outages, rate-limit changes, tier reshuffles. A five-minute outage is an inconvenience for a chat toy and a revenue hole for an order-processing pipeline.

Why this bites cross-border operators hardest

Cross-border automation tends to chain models together: translate → classify → draft → reply → log. Every link is calibrated to a model that was stable on the day you built it. When one link drifts, the error doesn't stay local — it propagates. A slightly worse translation feeds a slightly worse classification, which feeds a wrong reply to a customer in a language you don't read.

You won't notice a benchmark drop of half a percent. You will notice a 3% uptick in disputes, three weeks later, with no obvious cause.

Treat the model as replaceable, not foundational

The fix is a mindset shift: the model is the least stable part of your stack, so build as if it will be swapped.

• Put an abstraction layer in front. Call models through your own interface, not a vendor SDK sprinkled across your codebase. Swapping a model should be a config change, not a refactor.
• Pin, then canary. Pin the exact version. Evaluate a new one against your own tasks before it touches production.
• Build an eval harness for your work. Not public benchmarks — a golden set of real tickets, real listings, real translations. Re-run it every time a model changes. This is the only way "nerfed" becomes measurable instead of a feeling.
• Own the prompt and the data. Models are racing toward commodity. Your prompts, your examples, and your accumulated data are not. Those are the assets that survive a model swap.
• Budget guardrails and a kill switch. Meter spend, cap it, and keep a way to stop the pipeline cold. A model that changes price overnight should hit a ceiling you set, not your bank account.

The uncomfortable takeaway

The tempting move is to pick the best model this week and build around it. The durable move is to assume that model will be different — cheaper, worse, gone — by the time you're at scale, and to design so that none of those outcomes is fatal.

Models are the fastest-moving layer in your stack. Building on them like bedrock is how you end up rebuilding in public.

Build on your prompts, your data, and your evaluation. Rent the model — and keep the receipt.

Part of a series on running a cross-border store without getting captured by it.