The feature that demos profitable and ships in the red
I have watched more than one AI feature win the room, get the budget, launch to applause, and then quietly lose money on every single call. Nobody set out to build a loss-maker. The demo looked cheap. The model was priced in fractions of a cent. The spreadsheet, if there was one, multiplied that tiny number by the expected volume and produced a figure the finance team could live with. Then production arrived, and the real bill showed up a month later, several times larger than the plan.
This is the gap I want to close for a fellow leader, because it is one of the most expensive blind spots in AI right now. The demo is priced on the happy path run once. Production is priced on everything the happy path leaves out. If you do not model that difference before you ship, you are not pricing a feature. You are guessing, and the market will correct your guess with your own margin.
The right unit is cost per correct answer, not cost per call
Start by fixing the unit, because the wrong unit is where the trouble begins. Most teams track cost per API call or cost per thousand tokens. Those are input prices, not outcome prices. The number that actually governs your P&L is the fully loaded cost of producing one result a user will accept.
The difference is not cosmetic. A single successful outcome in production often consumes several model calls, not one. There is the first attempt, the retry when it comes back malformed, the verification pass that checks the work, the context you had to stuff in to make the answer good, and the occasional escalation to a larger and pricier model when the cheap one is not sure. Add the failures that never produced a usable result at all and still cost money to attempt. Divide your total spend by the number of answers a customer actually kept, and you get cost per correct answer. That number, not the sticker price per call, is the one to put in the business case.
The retry tax nobody budgets for
The reason cost per correct answer runs so far ahead of cost per call is what I think of as the retry tax, and it compounds for the same reason reliability does. If a step succeeds most of the time but not always, you pay again for the times it does not, and you pay for the machinery that catches the failure.
It gets sharper with autonomy. A single-shot chatbot is cheap. An agent that plans, calls tools, checks itself, and fans out across subtasks can consume many times the tokens of that chat to produce one finished result, and engineering reporting from the labs has put multi-agent setups at an order of magnitude more token usage than a single conversation. Parallelism buys coverage and speed, and it is sometimes exactly right, but it is never free. Every extra step you add to make the system more capable is also a line item. The uncomfortable truth is that capability and gross margin pull against each other, and most roadmaps only draw the capability curve.
The levers that move the margin
None of this is an argument against shipping. It is an argument for treating cost as a design surface with real controls, the same way you treat latency or accuracy. A handful of levers do most of the work.
Cache aggressively, because a large share of real traffic is repetitive, and an answer you already paid for should not be bought twice. Route by difficulty, so the cheap, fast model handles the easy majority and the expensive one is reserved for the genuinely hard minority rather than being the default for everything. Spend context like money, because every token you pad the prompt with is paid on every call, and most prompts are carrying weight they do not need. Fail fast and fall back deliberately, so a struggling request degrades to a cheaper path instead of burning the full premium budget before giving up. Each of these is ordinary engineering. Together they are often the difference between a feature that scales profitably and one that gets quietly capped because finance noticed.
Pricing against a floor that keeps moving
There is a strategic twist that makes this harder than a one-time calculation. Model prices fall, and they fall fast. That sounds like pure good news, and for your cost base it is. But the same price drop that helps you also helps every competitor, and it steadily erodes the part of your product that was just a thin wrapper over a model call. The capability you are paying a premium for today becomes a commodity you can buy for pennies in a year.
So I do not price an AI feature as a fixed markup over this quarter's token cost. I design the margin to survive the floor dropping. That means knowing which part of the value is the model, which part is yours, and making sure the part that is yours, the data, the workflow, the reliability, the distribution, is what the customer is actually paying for. If your margin depends on the model staying expensive, you do not have a margin. You have a countdown.
What a leader insists on
I will not approve an AI feature for general availability without these, and I ask for them while the design is still cheap to change, because retrofitting cost discipline onto a live system is painful and slow.
Cost instrumented per successful outcome, not just per call, visible on a dashboard a non-engineer can read. A target cost per correct answer set before launch, treated as a requirement rather than a surprise. A budget ceiling per request, so a pathological input cannot run up an unbounded bill. Caching, routing, and context discipline designed in from the first version, not bolted on after the invoice shocks someone. And an alert on cost per outcome drifting the way you would alert on latency or error rate, because cost regresses silently and you want to know before the quarter closes.
None of this is exotic. It is the financial equivalent of a test suite, and the teams that treat it as optional are the ones who discover, too late, that their most popular feature is their least profitable one.
Where this goes
The industry conversation is still mostly about what these systems can do. The conversation that decides who is still standing in three years is about what they cost to do reliably, at scale, for a price a customer will pay. Capability is becoming abundant and cheap. Operational and financial discipline is becoming the scarce thing, and scarce things are where durable advantage lives.
The leaders who win will not be the ones who shipped the most impressive feature. They will be the ones who knew, to the cent, what each correct answer cost them, and built a business that still worked when the model got ten times cheaper and so did everyone else's.
I am still refining how I model this, and I would value comparing notes. If you are running AI in production, what is your real cost per successful outcome, and what surprised you when you first measured it properly.
I write a field-note series on shipping AI that actually works in production. The original of this piece, and the rest, live on my LinkedIn. Originally published on LinkedIn.
Top comments (0)