A pricing table is useful.
It is also easy to overread.
When a new model family arrives with clearer tiers, faster options, and lower-cost paths, the first instinct is to compare input and output prices. That makes sense. Founders need to know whether a feature can survive real usage.
But the price per million tokens is only the first layer of AI cost.
The real product cost usually appears one step later:
- Which tasks use which model?
- How much output does the workflow generate?
- How often does the same context repeat?
- How many retries happen when the first answer is not good enough?
- How much human review still sits around the AI step?
- What happens when users depend on the feature every day?
This is why GPT-5.6 is interesting from an economics angle, not only a capability angle.
The model lineup gives teams more pricing choice. The product still needs a cost system.
What changed
OpenAI introduced GPT-5.6 with three model tiers:
- Sol: the strongest model, priced at $5 input and $30 output per one million tokens.
- Terra: a balanced model, priced at $2.50 input and $15 output per one million tokens.
- Luna: a faster and lower-cost model, priced at $1 input and $6 output per one million tokens.
OpenAI also introduced more predictable prompt caching for GPT-5.6 and later models, including explicit cache breakpoints and a 30-minute minimum cache life. Cache writes are billed at 1.25x the model’s uncached input rate, while cache reads receive a 90 percent cached-input discount.
That creates a practical question for teams building AI into SaaS products:
Should cost planning start with the model tier, or with the workflow?
The safer answer is workflow.
Why model price is not the full cost
A lower-cost model helps when the task fits it.
It does not automatically make the full product cheaper.
For example, imagine two AI workflows:
- A support-tagging workflow that classifies customer messages into a few categories.
- A technical review workflow that reads long context, reasons through multiple constraints, and produces a detailed recommendation.
The first workflow may work well with a fast, lower-cost model.
The second may need a stronger model, or at least a careful routing rule that sends only the hard cases to the stronger path.
If both workflows use the same model by default, one of two things usually happens:
- The simple workflow becomes more expensive than necessary.
- The complex workflow becomes cheaper at first, but creates review work, retries, or user corrections later.
Both are cost problems.
One is visible on the invoice.
The other hides inside operations.
The four cost layers founders should model
A founder does not need to turn every AI feature into a finance spreadsheet before testing it.
But once the feature moves toward customer-facing usage, four cost layers should be visible.
1. Model tier cost
This is the obvious one.
Input tokens, output tokens, reasoning effort, model tier, and provider pricing all matter.
But teams should not stop here. The cheapest model for one task may become expensive if it produces answers that require extra review, retries, or longer prompts.
2. Output shape
Output tokens are often where costs grow quietly.
A product that returns short classifications, status labels, or structured fields has a different cost profile from a product that generates long explanations, drafts, recommendations, or reports.
If a feature always asks for a long answer, the bill grows with every user action.
A better pattern is to design the output around the user decision:
- Does the user need a short answer?
- Does the user need a draft?
- Does the user need a reasoned explanation?
- Does the system need a structured object instead of prose?
- Can the full explanation appear only when requested?
The output format is not only UX. It is cost design.
3. Repeated context and caching
Prompt caching becomes valuable when a workflow sends the same large context repeatedly.
That may include:
- System instructions.
- Product rules.
- Policy text.
- Tool definitions.
- Reusable examples.
- Account-level configuration.
- Long documents or knowledge context that remains stable across requests.
Caching is not magic. It depends on reuse.
If the prompt changes constantly, the cache hit rate stays low. If static content is placed at the beginning and dynamic user content appears later, the chance of a useful cache hit improves.
This changes prompt design.
A production prompt should not be treated as one big text block. It should be structured so repeated content remains stable, measurable, and cacheable where the provider supports it.
4. Review, retry, and fallback cost
This is the layer many early AI demos miss.
The first API call may be cheap.
The full workflow may not be.
A customer-facing feature can create extra cost through:
- retries after weak answers,
- review queues,
- escalation to a stronger model,
- fallback paths,
- support tickets,
- manual correction,
- reprocessing failed jobs,
- longer latency windows,
- and customer confusion when the output is not clear.
Those costs do not always appear as tokens.
They appear as engineering time, support load, product complexity, and lower trust.
A better cost model for AI features
Instead of asking, “Which model is cheapest?” ask:
What is the cheapest reliable path for this workflow?
That question leads to a more useful structure.
Routine path
Use this for low-risk, repeatable tasks.
Examples:
- classification,
- extraction,
- short summaries,
- simple rewriting,
- intent detection,
- formatting,
- routing,
- and lightweight support assistance.
The goal is speed and predictability.
Escalation path
Use this for tasks where stronger reasoning changes the outcome.
Examples:
- complex code review,
- multi-step product analysis,
- policy-sensitive work,
- security review,
- technical planning,
- and decisions that affect customers or operations.
The goal is quality, not default low cost.
Cached path
Use this when long context repeats.
Examples:
- documentation assistant,
- policy review,
- product onboarding assistant,
- internal knowledge workflows,
- support copilots with stable business rules,
- and agent workflows with repeated tool definitions.
The goal is to avoid paying full input cost for the same context again and again.
Human-review path
Use this when the output has meaningful business impact.
Examples:
- legal-sensitive drafts,
- financial recommendations,
- healthcare-adjacent content,
- security-sensitive workflows,
- customer-facing automation,
- and high-value account decisions.
The goal is confidence, not automation for its own sake.
What developers should measure
A production AI feature should not be measured only by total API spend.
It should track cost by workflow.
Useful metrics include:
Cost per successful task
Not cost per API call. A task may require multiple calls.Output tokens per task type
Some prompts look cheap until the output becomes long.Cache hit rate
If caching is expected to reduce cost, measure whether it is actually being hit.Retry rate
A cheaper model that triggers more retries may not be cheaper.Escalation rate
How often does the workflow move from a low-cost model to a higher-capability model?Human correction rate
Manual edits, rejected outputs, or support follow-ups are part of the cost.Latency by path
A low-cost path that feels slow can still hurt the product experience.Cost by customer segment
Heavy users may behave very differently from the average demo user.
These metrics make the cost real.
Without them, the team is only guessing from the pricing page.
What founders should decide before launch
Before turning an AI workflow into a customer-facing promise, founders should model three usage levels:
1. Pilot usage
A small number of users.
The goal is to learn whether the workflow is useful and where quality breaks.
2. Normal usage
Expected steady product usage.
The goal is to see whether cost fits pricing, support capacity, and margin.
3. Growth usage
Higher adoption after the feature becomes popular.
The goal is to check whether the system still makes sense when customers actually use it.
This is where many AI features become clearer.
A workflow that looks affordable for 20 users may need routing, caching, batching, or limits before it works for 2,000 users.
The practical takeaway
GPT-5.6 gives teams more choices across capability, speed, and cost.
That is useful.
But the economics of an AI product will not be solved by picking the lowest-priced model.
The better move is to design the workflow around:
- task complexity,
- output length,
- repeated context,
- cache behavior,
- retry rate,
- review requirements,
- fallback paths,
- and customer dependency.
The cheapest model is not always the cheapest workflow.
The cheapest reliable workflow is the one that routes the right task to the right path, measures what happens after launch, and avoids turning every customer action into the most expensive possible AI call.
Founder action checklist
Before shipping an AI feature, ask:
- Which parts of this workflow are routine?
- Which parts need stronger reasoning?
- Which context repeats often enough to cache?
- What is the expected output length?
- What happens when the answer is not good enough?
- How often will a user trigger this workflow?
- What is the cost per successful task, not just per API call?
- Does the pricing still work at 10x usage?
That is where AI cost planning becomes useful.
Not at the pricing table alone.
At the workflow.
Top comments (4)
This is exactly why we abstracted cost planning into credits instead of exposing per-token math to the end user. The 'cost per successful task' framing is right, but most side-project devs don't want to build the routing/caching/retry tracking infrastructure described here just to ship a feature. Fixed credit costs per model push that complexity to us instead, so the dev just sees 'this call cost X credits' without modeling cache hit rates and escalation paths themselves. Tradeoff is less granular optimization, but for the target user that's the right tradeoff.
Well said, Lolo. That tradeoff is exactly where product design matters. For many builders, credits can make usage easier to understand without pushing workflow complexity onto the user. Granular optimization still has its place, but predictable cost visibility often creates a better product experience for the audience being served.
The “cost per successful task” point is the most useful framing here.
A cheap model is only cheap if it completes the workflow reliably. If it creates retries, longer prompts, human correction, or escalation to a stronger model, the pricing table stops telling the real story.
For SaaS teams, routing by task type probably matters more than picking one default model everywhere.
Appreciate the perspective, Suny. This is exactly the kind of framing we had in mind.
For SaaS teams, the right question is not just which model is cheaper on paper, but which setup helps the workflow move forward with fewer breaks, reviews, and handoffs. That is where model routing starts becoming a product decision, not only a pricing decision.