When an AI feature is scoped, the estimate that gets written down is usually an engineering estimate: how long to wire up the model call, how long to build the UI around it, how long until it's demo-ready. That estimate is almost never wrong about the build. It's almost always incomplete about everything that comes after the build.
Why does the initial estimate miss so much?
Traditional software has a cost profile engineers know well: build it, deploy it, and the marginal cost of another user is close to zero. AI features break that assumption in a way that isn't always obvious at scoping time. A model call isn't a fixed-cost function, it's a metered one. Every additional user, every additional session, every additional retry adds a real, variable cost that scales with usage rather than staying flat.
That single difference is why AI feature costs deserve their own line item instead of getting folded into general infrastructure spend.
What are the actual categories of cost?
Model cost is the most visible one, input and output tokens, the choice between a cheap fast model and an expensive capable one, fallback calls when the primary model fails or times out, and the token overhead of large prompts or retrieved context sent with every request. A workflow that calls a model three or four times to complete one user action multiplies this cost quietly.
Data and retrieval cost sits underneath most AI features that reference anything beyond the prompt itself, embeddings generation, vector storage, the search infrastructure that ranks and returns relevant chunks, and the ongoing engineering work of keeping that retrieved data synchronized with whatever source of truth it's pulled from. Retrieval quality doesn't stay good on its own; it degrades as source data changes and has to be actively maintained.
Product cost is the UI and UX work that a model call alone doesn't cover: the screen the output lives in, the onboarding that teaches users what the feature can and can't do, the correction flow for when the output is wrong, and the support documentation that has to explain probabilistic behavior to a support team used to deterministic bugs.
Operational cost covers what happens after launch, monitoring for silent failures, running evaluations against new edge cases as they appear, handling incidents when a model behaves unexpectedly, and updating prompts or swapping models as behavior drifts over time. This is the category most schedules underestimate, because it doesn't have a clear end date the way a build does.
Risk cost is the hardest to quantify and the most consequential when it lands: privacy exposure from data sent to a model, the business impact of an incorrect recommendation, and the compliance and reputational exposure that follows when either of those happens at scale.
How does this change the margin math on AI-powered products?
Traditional SaaS gross margin held steady around 70–80% for years because hosting costs stayed roughly flat as the customer base scaled. AI breaks that pattern because inference cost is tied to usage, not to the number of customers. ICONIQ's 2026 State of AI report found that AI product builders are now expecting average gross margins closer to 52%, a meaningful compression that reflects model and retrieval costs moving with every request instead of staying fixed.
That compression isn't evenly distributed across customers, either. A heavy user of an AI feature can generate materially more inference cost than a light user on the exact same plan, which is a problem traditional flat-rate SaaS pricing was never built to absorb. One company's gross margin swung from 36% to -14% after an AI feature's usage outpaced its pricing model, a reminder that "unlimited" AI access and unpredictable cost exposure are close to the same thing.
The practical fix most teams land on is tracking cost per request, cost per session, and cost per successful outcome rather than reviewing the total AI bill in isolation, which hides which specific features and which specific users are driving the spend.
What does the maintenance burden look like technically?
AI feature maintenance differs from traditional maintenance in one structural way: model behavior is probabilistic, so regression testing has to evaluate output quality and behavior drift, not just whether the code still executes correctly. A prompt that performed well against last quarter's edge cases can silently degrade as usage patterns shift or as the underlying model gets updated by the provider.
That means evaluation sets need to keep growing, integrations and data sources need active monitoring for schema or format changes, and someone on the team needs explicit ownership of quality, not just uptime, for as long as the feature stays live. Traditional feature ownership tends to fade after launch stabilizes. AI feature ownership can't, because the thing being owned keeps changing on its own.
How are teams deciding whether a feature is worth this overhead before building it?
The pattern showing up across engineering teams is a shift toward scoping the cost surface before generating any code, rather than discovering it after launch. That's part of why architecture-first planning has become more common in AI-assisted development, producing a requirements document, a data flow map, and an explicit list of what a feature will touch before a single line gets written. Tools like LangGraph and CrewAI push this at the agent-orchestration layer by forcing explicit definitions of what an agent can call and when. Platforms like 8080.ai apply a similar discipline at the application layer, generating the system requirements document and architecture diagrams as an approval step a team reviews before the build continues which surfaces the retrieval and data dependencies of a feature while it's still cheap to change scope, rather than after it's in production and carrying real usage cost.
The underlying question worth asking before committing engineering time is a narrow one: does this feature solve a problem frequent enough, and valuable enough, to justify a cost structure that scales with usage instead of staying fixed? If a simpler deterministic rule solves the same problem, it usually should, a rule doesn't need an evaluation pipeline, doesn't drift, and doesn't carry inference cost per invocation.
What happens to features that pass the initial bar but stop earning their cost?
This is the part most roadmaps skip: a plan for what happens when a feature's usage or trust erodes over time. Weak signals, the number of AI features shipped, the number of model calls made, total tokens processed, measure activity, not value. Stronger signals, task completion rate, the ratio of accepted to edited outputs, cost per successful outcome, and whether the feature actually reduces support tickets rather than generating new ones, measure whether the feature is still worth what it costs.
A mature AI feature lifecycle includes a retirement path as deliberately as it includes a launch plan: a minimum adoption threshold, a maximum acceptable cost per outcome, and a clear owner responsible for making the call to consolidate or sunset a feature that users have quietly stopped trusting.
The engineering takeaway
AI hasn't made features cheaper to run, it's made them cheaper to start. That distinction matters more than it sounds like it should, because the entire cost structure described above only becomes visible after a feature ships, by which point the sunk cost of building it is already spent. Scoping the ongoing cost, model, retrieval, product, operational, and risk, before committing engineering time is quickly becoming the difference between an AI feature that earns its place on the roadmap and one that quietly drains margin for a year before anyone notices.
Top comments (0)