The model you built on this quarter will not be the best model next quarter. Plan for that, or plan to keep rebuilding.
Teams building with AI right now are living through something software has not seen before: the single most important component of the system gets materially better, cheaper, or deprecated every few months, and none of that is under their control. A new frontier model lands and resets the quality bar. A price drop makes last quarter's architecture look wasteful. A provider sunsets the exact version you tuned everything around, with a migration deadline you did not choose. The ground moves, and it keeps moving.
The instinct is to treat each of these as a one-off event to react to. That is the mistake. The churn is not a series of surprises. It is the permanent condition of building on AI, and the teams that thrive are the ones who stopped being surprised and started designing for it.
The upgrade that broke everything
The pattern shows up the first time a better model ships and you discover you cannot actually use it. A new release is clearly stronger on every benchmark that matters, the team is excited, and then someone tries to swap it in. The carefully tuned prompts, written against the old model's quirks, now behave differently. The output format drifts. An edge case that was handled is suddenly not. What should have been a one-line change becomes a two-week regression hunt, and the upgrade quietly slips down the backlog.
So you stay on the old model. Not because it is better, but because moving is expensive and risky, and nobody can say with confidence that the new one will not break something in production. The improvement sits on the shelf. Meanwhile a competitor who built for portability ships the upgrade in an afternoon and pulls ahead on quality and cost at the same time.
The deeper version of this is deprecation, where staying is not even an option. The provider announces the model you depend on retires in ninety days, and now the migration you kept deferring is mandatory, urgent, and happening on someone else's schedule instead of yours. The work was always going to be necessary. The only question was whether you did it calmly, on your terms, or in a panic against a deadline.
Why the treadmill is permanent
It is tempting to believe this settles down, that the frontier stabilises and the churn was just the early turbulence of a new technology. It will not, at least not on any timescale you can plan around. The economics guarantee it.
Multiple well-funded labs are competing directly on capability and price, releasing on cadences measured in months. Each release pressures the others. Inference costs fall as hardware and techniques improve, which means the cost-optimal choice keeps shifting even when quality does not. And providers prune their own catalogues to control the cost of serving old versions, so the model you are on today has a finite, often undisclosed, lifespan regardless of how well it works for you.
None of these forces is slowing. The rational planning assumption is not "the model is stable." It is "the model is a depreciating, replaceable component with an expiry date I do not control." Once you hold that assumption squarely, the whole design problem changes. You stop optimising for the model you have and start optimising for the ease of replacing it, because replacing it is the one thing you know for certain you will have to do.
The lock-in you didn't notice
Most model lock-in is not a deliberate choice. It accumulates quietly, one reasonable decision at a time, until the cost of moving is high enough that you stop considering it.
It starts with prompts. A prompt tuned through dozens of iterations against one model is, in effect, a program written in that model's private dialect. It encodes assumptions about how that specific model interprets instructions, where it needs reinforcement, what format it reliably produces. Move to another model and that accumulated tuning does not transfer cleanly, because you were never really writing general instructions. You were writing to one reader.
Then it is the provider-specific features: a particular tool-calling format, a structured-output mode, a context-caching mechanism, a safety setting that behaves a certain way. Each one is convenient and each one is a thread stitching you to that vendor. Individually trivial, collectively a straitjacket. By the time a better or cheaper option appears, you are not comparing two models. You are comparing your current model against the full cost of unpicking every thread, and that comparison almost always ends in "not now."
The lock-in is invisible precisely because it never felt like a decision. That is why it has to be designed against deliberately. Portability does not happen by default. Dependence does.
The abstraction layer that actually helps
The obvious response is an abstraction layer, and the obvious abstraction layer is the wrong one. Wrapping every provider in a lowest-common-denominator interface that exposes only the features they all share throws away exactly the capabilities you are paying for, and leaves you with a system that is portable and mediocre everywhere.
A useful abstraction does something harder. It isolates the model behind a boundary your application code never reaches across, so that the model, its prompts, its parameters, and its provider-specific handling all live in one replaceable place. The rest of your system talks to a stable internal contract, "given this input, return this structured result," and does not know or care which model is behind it. Swapping the model means changing what lives inside that boundary, not touching the hundred places that consume its output.
The distinction that matters: you are not abstracting away the differences between models, you are containing them. The provider-specific tool-calling format still gets used, it just gets used inside the boundary, with an adapter per provider, rather than leaking into your business logic. Done well, this is not a heavy framework. It is a discipline about where model-specific knowledge is allowed to live, and the payoff is that "try the new model" becomes a bounded experiment instead of an architectural event.
Your eval harness is the insurance policy
Containment lets you swap the model. It does not tell you whether you should, or whether the swap broke anything. That is what an evaluation harness is for, and it is the single highest-leverage thing you can build for life on the treadmill.
An eval harness is a representative set of real tasks with known-good outcomes, run automatically, that tells you how any given model actually performs on your workload rather than on a public benchmark. With one, a new model release is a calm, mechanical exercise: point the harness at it, read the scores on your own tasks, see the regressions and the gains in numbers, and decide. Without one, every swap is an act of faith, which is exactly why teams without a harness freeze, and teams with one move in a day.
This reframes the harness from a testing nicety into the core asset that makes portability real. The abstraction layer gives you the ability to change the model cheaply; the eval harness gives you the confidence to actually pull the trigger. One without the other is not enough. A clean abstraction with no eval harness means you can swap but dare not, and a great eval harness bolted onto a tangle of model-specific code means you know you should swap but cannot afford to. You need both, and the harness is the one teams skip, because its value is invisible right up until the day a new model ships.
When to actually switch
Building for portability is not an argument for chasing every release. The ability to switch cheaply is precisely what lets you be disciplined about when you do, because the decision is no longer distorted by the pain of moving.
The trap is benchmark-chasing: a new model tops a public leaderboard and the reflex is to adopt it. But public benchmarks are not your workload. A model that is two points better on a general reasoning test may be no better, or worse, on your specific task, and the only thing that settles it is your own eval harness. Switch when the harness shows a real gain on the outcomes you care about, or when the cost falls enough to matter at your volume, or when your current model is being deprecated. Those are the reasons. "It is new and everyone is talking about it" is not.
There is also a real cost to staying, and portability lets you see it clearly rather than hide from it. Every quarter on a model that is now more expensive than its replacement is margin left on the table. Every quarter on a model that is now meaningfully weaker is quality you are choosing not to ship. When moving is cheap, those costs become visible decisions instead of comfortable defaults, and that visibility is worth as much as the flexibility itself.
Building for the next model, not this one
The shift is to stop treating the current model as a fixed foundation and start treating it as the most replaceable part of the system. That sounds like a loss of certainty. In practice it is the only durable certainty available, because the one thing you can rely on is that the model will change.
Concretely, it means a few commitments made early, while they are cheap, rather than late, under a deprecation deadline. Keep model-specific knowledge, the prompts, the parameters, the provider adapters, contained behind a boundary your application never reaches across. Write prompts as clearly as you can to the task rather than overfitting to one model's quirks, so they travel better. Build the eval harness before you think you need it, because you need it most on the day you did not plan for. And treat every provider-specific feature you adopt as a deliberate dependency with a known cost to remove, not a free convenience.
The teams that will struggle are the ones who built a beautiful system perfectly fitted to one model and called it done. The teams that will compound are the ones who built a system that treats the model as a swappable component and made swapping cheap and safe. The models will keep getting better, cheaper, and deprecated, on a schedule nobody controls. You cannot get off the treadmill. You can only decide whether you built something that runs comfortably on it, or something that trips every time the belt speeds up.
I write a field-note series on shipping AI that actually works in production. The original of this piece, and the rest, live on my LinkedIn. Originally published on LinkedIn.
Top comments (0)