Harvey, the legal AI startup, spent the first half of 2026 watching its gross margin do something no SaaS board wants to see: it went from roughly +50% to roughly -50%. Revenue nearly doubled, from about $190M to over $400M in annualized terms. That should have been a victory lap. Instead, every new customer was actively losing the company money.
The cause wasn't a pricing mistake in the traditional sense. It was an agent update shipped in March. Once Harvey's product could leave a diligence agent grinding through a data room overnight instead of answering one query at a time, token consumption went up roughly 20x over the year. The seat price didn't move. A partner running one query a week and an associate running an unattended agent all night were paying the exact same monthly fee, but costing the company wildly different amounts on the token bill.
This is not a Harvey-specific story. Bloomberg reported this week that Abridge, Ramp, and Rogo are making similar moves — either adopting open-weight models or training their own, specifically to cut loose from frontier-lab pricing. The pattern is consistent enough that it's worth naming: agentic usage broke flat-rate pricing, and the fix teams are reaching for is a model swap. If you're an engineer at a company that sells anything agentic, this is coming for your P&L whether or not it's hit yet.
What's actually interesting here isn't the economics. It's what Harvey did next, because it's a preview of the real engineering work behind every one of these migrations — and most teams attempting it are going to underestimate it badly.
The swap that wasn't a swap
The easy version of this story is "frontier models got expensive, so we switched to something cheaper." Harvey did switch — its new stack, Harvey Tenet, is built on Moonshot AI's open-weight Kimi K3, post-trained with Fireworks AI. Open-weight tokens run roughly 40% cheaper than Claude Opus 5 on a per-token basis. But if that were the whole story, this wouldn't be worth writing about, because a 40% token discount doesn't turn a -50% margin into a positive one on its own.
The number that actually mattered was cost-per-completed-task, not cost-per-token. Harvey reported roughly a 10x reduction in cost per cell on its Review Tables product and a 90% reduction in cost per query on Firm Knowledge. That gap between "cheaper tokens" and "10x cheaper outcomes" is entirely explained by post-training: a model fine-tuned on your actual task distribution completes work in fewer steps, needs less retry-and-repair, and stops burning tokens re-deriving context a specialized model would already have baked in.
That's the part that doesn't show up in the "we moved to open-weight" headline, and it's the part that determines whether your migration is a cost win or a quiet regression: the model swap is the easy 10% of the project. The post-training data and the evaluation harness that proves the new model is actually as good is the other 90%.
Three things every one of these migrations actually needs
If you're on a team looking at your own agent token bill and thinking "we should just move to [cheaper open-weight model]," here's what that decision quietly commits you to:
1. Domain-specific post-training data, not just a base checkpoint. A generic open-weight model is not a drop-in replacement for a frontier model on a specialized legal, financial, or clinical workflow — it's a starting point that needs the same thing frontier labs spend enormous effort on internally: curated supervised fine-tuning examples, preference-ranked outputs for the failure modes that actually matter in your domain, and correction data for the specific ways your agent gets stuck (wrong tool call, hallucinated citation, dropped context across a long agent trajectory). This is exactly the kind of work described on SyncSoft.AI's reasoning and human feedback data page — RLHF, preference ranking, agent trajectory correction, and SFT curation aren't nice-to-haves for a migration like this, they're the mechanism by which "cheaper base model" becomes "cheaper and just as good model."
2. A parity benchmark before you cut over, not after. The uncomfortable truth about most "let's switch models" projects is that teams eyeball a handful of outputs, feel good about them, and ship. That works fine until the new model's failure distribution turns out to be different in shape from the old one — not necessarily worse on average, but worse in ways that matter more (a legal citation hallucination is not equivalent in cost to a slightly clunky paragraph). Building an actual benchmark suite against your production task distribution, scoring outputs, and specifically hunting for new hallucination or error modes the old model didn't have is the only way to know if you're trading one problem for a worse one. This is squarely what SyncSoft.AI's model evaluation and QA work — benchmark datasets, response scoring, hallucination detection, red-teaming — is built for, and it's the step most migration post-mortems will eventually admit they skipped.
3. Monitoring for drift after you've shipped. A model that passes your parity benchmark on day one can still drift in behavior as your product surface changes, as your prompts evolve, and as the open-weight provider ships new checkpoints upstream. Teams that treat the migration as a one-time cutover instead of an ongoing evaluation loop are the ones who get paged three months later when a subtle regression finally surfaces in a customer escalation instead of a test suite.
None of this is exotic. It's the same lifecycle that any serious ML team already runs for a first-party model. What's changed is the number of companies who now need to run it, on a compressed timeline, because their unit economics forced the decision rather than a strategic model roadmap deciding it for them.
Why this is bigger than Harvey
Step back and the trend line is straightforward: agentic products expose you to token costs that scale with autonomy, not with the number of seats you sell. Flat-rate, per-seat SaaS pricing was built for software that waits for a human to click. An agent that runs unattended overnight breaks that assumption completely, and it breaks it in proportion to how good your product gets — the better the agent, the more it runs unsupervised, the more tokens it burns, the worse a flat price looks.
Two responses to that are on the table: reprice (usage-based billing, which is its own painful migration with customers who hate metered pricing), or re-architect the cost base underneath the product. Harvey, Abridge, Ramp, and Rogo all reached for the second option first, and it's a reasonable bet — it's the option that doesn't require a sales conversation with every existing customer.
But "re-architect the cost base" is doing a lot of work in that sentence. It means becoming, functionally, a company that owns model quality end-to-end rather than one that resells a frontier lab's API. That's a genuinely different skill set: it requires a pipeline for domain data collection and annotation, a preference-data and correction loop for post-training, and a QA function that can catch regressions before customers do. Companies that have spent the last two years treating "call the API" as their entire AI strategy are going to find that the hard part of this migration isn't picking a cheaper model — it's building the data and evaluation muscle to make a cheaper model safe to ship.
If your team is staring down a token bill that's growing faster than your revenue, the model you land on matters less than whether you've built the pipeline to prove it's actually as good as what it's replacing before your customers find out the hard way.
I work at SyncSoft.AI, where we build training data, RLHF pipelines, and evaluation/QA benchmarks for teams doing exactly this kind of model work. If you're mid-migration and want a second pair of eyes on your eval harness or post-training data, feel free to get in touch — happy to compare notes even if it doesn't turn into anything.
Top comments (0)