A customer asks an AI agent to complete a task.
From the outside, the interaction looks simple:
User Request
↓
AI
↓
Successful Result
One request went in.
One result came out.
If the model call cost $0.20, it is tempting to think of $0.20 as the cost of serving that request.
But production AI systems rarely behave like a single API call.
Look inside the same workflow and the execution may have looked more like this:
User Request
↓
Context Retrieval
↓
Model Call
↓
Tool Call
↓
Tool Failure
↓
Retry
↓
Second Model Call
↓
External API
↓
Validation
↓
Successful Outcome
The customer still experienced one task.
The infrastructure experienced an execution graph.
Retrieval consumed resources.
The first model call consumed tokens.
The failed tool call may have incurred cost.
The retry required additional execution.
A second model call consumed more tokens.
The external API may have charged for usage.
Validation required additional processing.
And only then did the customer receive the result.
Nothing about this makes the workflow inefficient by definition.
The retry may have been necessary.
The external service may have been essential.
The second model call may have materially improved the result.
But it reveals an important distinction.
The price of the successful model call and the cost of producing the successful outcome are not necessarily the same thing.
That difference becomes increasingly important as AI products move from simple model interactions to production workflows.
The Model Call Is the Easiest Cost to See
AI teams naturally pay close attention to model costs.
They monitor:
- Input tokens
- Output tokens
- Cached tokens
- Inference costs
- Provider pricing
- Cost per request
They should.
These metrics provide essential visibility into one of the largest variable cost components of many AI products.
They help teams compare models, detect unusual consumption, estimate infrastructure spend and understand how changes in prompts or context affect cost.
Token visibility is not becoming obsolete.
Model cost is not becoming irrelevant.
The limitation is simply that these metrics answer a specific question:
What did the model execution cost?
That may not be the same question as:
What did it cost to produce the customer outcome?
Consider an agent workflow that uses $0.30 of inference.
If that inference is the only meaningful variable-cost operation required to complete the task, then model cost may be an excellent approximation of execution cost.
But suppose the same $0.30 model call exists inside a workflow that also requires retrieval, two paid external APIs, a failed tool execution and a retry.
The model price has not changed.
The economics of the workflow have.
This distinction matters because optimizing the visible component does not necessarily optimize the entire execution.
Reducing inference cost by 20% may be valuable.
But if repeated tool failures are responsible for most of the workflow's unnecessary spending, changing models may have relatively little impact on the economics of the completed task.
The model bill tells you something important.
It just doesn't always tell you the whole story.
A Production AI Request Is Usually a Workflow
The simplest mental model for an AI product is:
Request
↓
Model
↓
Response
For some products, that remains reasonably accurate.
For many others, production execution increasingly looks more like a graph:
Input
↓
Retrieval
↓
Reasoning
↓
Tool Selection
↓
External Service
↓
Validation
↓
Potential Retry
↓
Outcome
An AI support system may retrieve account context before generating an answer.
A document-processing workflow may extract data, classify it, validate the result and send uncertain cases for review.
A research agent may perform searches, retrieve documents, invoke several tools and make multiple model calls before producing its final report.
A voice application may combine transcription, inference, synthesis and external services within a single customer interaction.
The user experiences a feature.
The infrastructure executes a workflow.
That difference changes how cost can be understood.
At the infrastructure level, individual operations still matter.
Teams still need to know what each model, API and service costs.
But when those operations exist primarily to produce a single customer-visible result, another economic unit becomes useful:
the workflow itself.
Instead of looking only at:
Model Call A $0.12
Model Call B $0.08
External API $0.04
we can also ask:
Workflow
↓
All meaningful execution required
to produce the outcome
↓
Total economic cost
This does not mean every infrastructure expense can or should be attributed perfectly to an individual workflow.
Shared compute complicates attribution.
Caches serve multiple executions.
Some APIs are purchased in bundles.
Background jobs may support several customers.
Storage costs accumulate over time.
Human intervention can be especially difficult to allocate precisely.
Economic attribution is inherently imperfect.
The goal is not accounting precision at arbitrary granularity.
The goal is enough visibility to understand where meaningful costs originate.
Because once an AI request becomes a workflow, optimizing only the most visible operation can leave much of its economics unexplained.
Failures Still Cost Money
In traditional application monitoring, a failed request is usually represented as an operational event.
Something went wrong.
The request failed.
The system may retry it.
Eventually, the operation either succeeds or surfaces an error.
AI workflows add another dimension to that sequence.
Work may already have been paid for before the failure occurred.
Consider a simplified execution:
Attempt #1
Model inference $0.14
Tool call $0.05
Result Failure
↓
Retry
↓
Attempt #2
Model inference $0.16
External API $0.07
Result Success
From the customer's perspective, one task was completed.
From the runtime perspective, two attempts occurred.
From an economic perspective, both attempts mattered.
The failed model call still consumed tokens.
The failed tool operation may still have generated a charge.
The successful attempt then required additional resources before the customer received the outcome.
If we measure only the final successful execution, part of the cost disappears from the economic picture.
This is one reason failure economics can matter in AI systems.
A failed execution does not necessarily produce zero value.
It may provide information used by the next attempt.
It may be an expected part of a resilient workflow.
And retries themselves are not inherently wasteful.
Retries exist because distributed systems fail.
Networks time out.
Providers become temporarily unavailable.
External services return transient errors.
Long-running workflows encounter conditions that cannot always be predicted in advance.
Removing retries simply to reduce cost would often make the system less reliable.
The more useful distinction is between resilience cost and execution waste.
A retry that recovers a valuable customer workflow may be economically justified.
A loop that repeatedly performs the same expensive operation because the underlying failure cannot recover is something very different.
Both appear as retries.
Their economics are not the same.
This means retry rate alone is not enough.
Teams may eventually need enough context to understand questions such as:
- Which workflows generate repeated attempts?
- Where in the execution path do failures occur?
- How much work happens before failure?
- Which retries eventually produce useful outcomes?
- Which failure patterns repeatedly consume resources without changing the result?
The goal is not to eliminate failed execution.
That is unrealistic.
The goal is to understand the economic consequences of failure well enough to distinguish useful resilience from repeated spending that produces little additional value.
A failed AI execution can still have a cost even when it never produces a customer-visible outcome.
Tool Calls Change the Cost Model
This becomes even more important as AI products move beyond model-only execution.
An agent may decide to search the web.
Query a database.
Call an enrichment provider.
Execute code.
Store a generated asset.
Send a message through a communication API.
Invoke another model.
Each operation expands what the workflow can accomplish.
It can also expand its cost surface.
Consider a research workflow:
User Request
↓
Model
↓
Search
↓
Retrieval
↓
Model
↓
External Data API
↓
Validation
↓
Final Response
The model calls may still represent the largest variable expense.
But they are no longer necessarily the only meaningful expense.
Suppose the model portion costs:
$0.35
The workflow may also generate:
Search / retrieval $0.06
External data API $0.18
Validation $0.04
Storage $0.02
The exact numbers are not important.
The structure is.
Looking only at inference would describe one component of the execution while leaving the rest outside the analysis.
In that sense:
Model Cost
≠
Complete Workflow Cost
But even that distinction needs qualification.
Workflow cost is not always something that can be calculated by simply adding every infrastructure event together.
Production systems contain shared resources.
A retrieval result may be cached and reused across customers.
A database cluster supports thousands of workflows simultaneously.
An external API may charge through a monthly commitment rather than per request.
Background processing may serve multiple execution paths.
Storage costs may persist long after a workflow completes.
Some infrastructure costs are marginal.
Others are allocated.
Others are effectively fixed within a certain capacity range.
Trying to attribute every cent precisely can create more complexity than useful insight.
The objective is therefore not:
Calculate the exact accounting cost of every workflow.
A more practical objective is:
Understand which execution components materially change the economics of the workflows you care about.
For one product, that may primarily be model inference.
For another, external APIs may dominate.
For an agentic product, tool loops and repeated model calls may matter more.
For document processing, human review may be the expensive exception path.
For voice AI, several services may contribute materially to a single interaction.
The relevant cost model depends on the architecture.
What matters is recognizing that once AI execution becomes a multi-step workflow, the provider's model bill is no longer guaranteed to describe the complete economic behaviour of the product.
It describes one important layer.
The workflow provides the context around it.
The Customer Doesn't Buy Tokens
Infrastructure teams naturally think in infrastructure units.
Tokens.
Requests.
GPU time.
API calls.
Compute.
These units matter because they describe what the system consumes.
But they are rarely what the customer is actually trying to buy.
A customer using an AI support product wants a ticket resolved.
A customer using document automation wants a document processed correctly.
A sales team may want a qualified lead.
A research user may want a completed analysis.
A creative tool may exist to produce a usable asset.
The customer experiences the outcome.
The company pays for the execution required to produce it.
That creates an important bridge between infrastructure economics and customer value.
Consider a support automation system.
Suppose it processes 1,000 conversations during a given period.
Its model bill is:
Model inference: $300
That number is useful.
Now suppose the complete workflows also involve retrieval, external services and retries:
Model inference $300
Retrieval $40
External services $85
Retry execution $35
───
Observed workflow cost $460
The workflow-level view provides additional context.
But there is still another question.
What did that $460 produce?
Suppose 700 conversations were resolved successfully without human escalation.
For this particular product, one potentially useful lens becomes:
Observed workflow cost
↓
Successful resolutions
↓
Cost per useful outcome
In this simplified example:
$460
÷
700 resolved conversations
=
$0.66 per resolved conversation
That number does not replace token cost.
It answers a different question.
Token cost helps explain the economics of inference.
Workflow cost helps explain the economics of execution.
Cost per useful outcome can help connect that execution to something closer to customer value.
For some AI products, that connection can be extremely useful.
For others, it may be difficult or even misleading.
What is the "useful outcome" of an AI coding assistant?
A generated function?
An accepted suggestion?
A merged pull request?
Developer time saved?
What is the useful outcome of a research assistant?
A completed report?
A correct decision?
Knowledge acquired by the user?
Not every product produces discrete outcomes that can be measured cleanly.
And even when an outcome can be identified, assigning business value to it may remain difficult.
So cost per useful outcome should not be treated as a universal AI metric.
It is better understood as an economic lens.
Where outcomes can be meaningfully defined, it can help teams move from:
What did the infrastructure consume?
toward:
What did we spend to produce something the customer actually valued?
That is a different level of understanding.
From Cost per Call to Cost per Workflow
There are several ways to observe the economics of the same AI system.
None needs to replace the others.
They answer different questions.
| Infrastructure View | Workflow View | Economic View |
|---|---|---|
| Model calls | Total execution path | Useful outcome |
| Tokens | Retries and failures | Cost to produce outcome |
| Inference | Tool execution | Revenue relationship |
| API calls | External services | Resulting margin |
| Compute | Operational overhead | Business value |
The infrastructure view asks:
What resources did the system consume?
The workflow view asks:
What did the complete execution require?
The economic view asks:
What did that execution produce, and was the relationship between cost and value healthy?
All three matter.
Without infrastructure visibility, teams cannot understand the underlying cost drivers.
Without workflow context, they may struggle to explain why apparently similar requests have very different costs.
Without economic context, they may know exactly what was consumed without knowing whether that consumption was healthy for the business.
The progression might look like this:
Model Events
↓
Execution Events
↓
Workflow
↓
Outcome
↓
Economic Context
This is not a maturity model.
A company does not necessarily need every layer of attribution.
A simple AI feature may be understood perfectly well through provider costs and aggregate margins.
Adding workflow-level instrumentation where it produces no useful decision would only create additional complexity.
The level of visibility should follow the economic questions the business actually needs to answer.
But as workflows become more complex, aggregate infrastructure metrics can become less explanatory.
A $0.10 model call inside a successful one-step workflow and the same $0.10 model call inside a five-attempt workflow are technically the same model expense.
They exist inside very different execution economics.
The objective is therefore not to move away from cost-per-call metrics.
It is to preserve them while adding enough context to understand the system around them.
Two Identical Customers Can Have Completely Different Economics
Workflow variance eventually appears at the customer level.
Consider two customers paying the same subscription price.
From the commercial system, they may look almost identical.
Customer A Customer B
Plan: Pro Plan: Pro
Revenue: $200 Revenue: $200
Status: Active Status: Active
Now look at how they actually use the product.
Customer A
Clean inputs
Short context
Standard execution path
Few retries
Predictable tool usage
Customer B
Messy inputs
Long context
Exception-heavy execution
Multiple retries
Expensive tool usage
The product may expose the same features to both.
Their visible usage may even appear broadly similar.
Their cost-to-serve can still be very different.
This is where workflow economics and customer profitability begin to intersect.
A customer who frequently triggers expensive execution paths may produce lower margins than another customer on the same plan.
But high cost does not automatically mean a bad customer.
A power user may also:
- Generate more revenue
- Purchase additional capacity
- Retain for longer
- Create strategically valuable usage
- Produce high-value outcomes
- Justify the infrastructure they consume
Similarly, a relatively low-usage customer may still be economically unattractive because of support overhead, pricing structure or other costs.
The relevant question is not:
Who uses the most AI?
It is:
What is the economic relationship between what a customer pays, how they use the product and what it costs to serve them?
That relationship is difficult to understand from subscription revenue alone.
It is also difficult to understand from token consumption alone.
The missing context often lives inside the workflows connecting the two.
This is why averages can hide important differences.
Two customers can generate the same revenue.
Two requests can consume the same number of tokens.
Two workflows can produce the same visible outcome.
And yet the economics underneath them can be materially different.
The problem is not necessarily the expensive customer, the expensive workflow or the expensive model.
The problem is making business decisions without being able to see those differences.
Measuring Usage Is Not the Same as Understanding Economics
Reliable usage data is foundational.
A production system should be able to answer questions such as:
- What executed?
- How much was consumed?
- When did consumption occur?
- Which customer initiated it?
- Was the operation recorded correctly?
If those answers are unreliable, any economic analysis built on top of them becomes unreliable too.
Suppose a retry is counted twice when it should have been counted once.
Or an execution occurs but never reaches the usage ledger.
Or consumption is attributed to the wrong customer.
Or a stale entitlement allows execution under commercial conditions that are no longer valid.
The resulting profitability analysis may look precise.
The underlying data is not.
This is why runtime correctness comes first.
Authorization needs to be trustworthy.
Consumption needs to be recorded correctly.
Retries need to preserve the intended semantics.
Commercial and application state need to remain aligned.
Usage accounting needs to describe what actually happened.
Only then can the business begin asking economic questions with confidence.
But correct metering still answers only part of the problem.
Imagine the runtime reports:
Customer: A
Execution: 8f21
Model calls: 3
Tokens: 48,200
Tool calls: 4
Retries: 1
Usage: Recorded correctly
That is valuable operational information.
It still does not necessarily tell us:
Which workflow was this?
Why did it require three model calls?
Which tool caused the retry?
What did the complete execution cost?
Did it produce a useful outcome?
What was the commercial relationship with the customer?
Was the resulting cost-to-serve economically healthy?
The usage data may be completely correct.
The economic picture may still be incomplete.
That distinction matters.
Runtime Correctness helps establish that execution and consumption happened according to the system's rules.
Economic understanding requires connecting those trusted events to additional context.
Trusted Runtime Events
+
Workflow Context
+
Cost Context
+
Outcome Context
+
Commercial Context
↓
Economic Understanding
Correct runtime data does not automatically produce correct economic conclusions.
But without correct runtime data, trustworthy economic conclusions become much harder to produce.
Visibility Comes Before Optimization
Once teams begin seeing workflow-level cost differences, optimization becomes tempting.
Use a cheaper model.
Reduce context.
Change retry policies.
Introduce stricter limits.
Modify packaging.
Adjust pricing.
Route certain workflows differently.
Some of those decisions may eventually be appropriate.
But optimization should not be the first step.
Understanding should.
Consider a workflow whose average execution cost suddenly increases.
A team looking only at provider spend might conclude that the model has become too expensive.
But the underlying cause could be very different.
Perhaps retrieval started returning substantially more context.
Perhaps a tool integration began timing out.
Perhaps a new customer segment naturally requires more complex workflows.
Perhaps an external API changed its pricing.
Perhaps retry frequency increased after a deployment.
Or perhaps customers are simply completing more high-value tasks.
Each situation could produce higher infrastructure spending.
They do not imply the same response.
Higher Cost
↓
Could Mean
↓
More Valuable Usage
Workflow Inefficiency
Failure Overhead
Customer Mix Change
Pricing Misalignment
Architecture Change
Cost is a signal.
Context gives the signal meaning.
Before attempting to optimize economics, teams need enough visibility to trace meaningful cost back through the system:
Where did the cost originate?
↓
Which workflow created it?
↓
Which customer triggered it?
↓
What happened during execution?
↓
Did the workflow produce a useful outcome?
↓
What economics resulted?
Again, the objective is not perfect attribution.
A company may never know the exact economic value of every execution.
It may not need to.
The objective is enough visibility to make better decisions.
Once that visibility exists, teams can reason more deliberately about:
- Architecture
- Model selection
- Retry behaviour
- Usage limits
- Packaging
- Pricing
- Workflow design
Without attribution, optimization risks solving the wrong problem.
A cheaper model does not fix an expensive retry loop.
A higher subscription price does not fix unnecessary tool execution.
A stricter usage limit may damage a high-value workflow whose cost is perfectly justified.
And reducing infrastructure spending is not automatically an improvement if it also reduces the value customers receive.
The goal is not minimum cost.
The goal is healthier economics.
Optimization without attribution is guesswork.
From Runtime Events to Runtime Intelligence
Production AI systems already generate much of the information needed to understand their behaviour.
The difficulty is that this information often lives in separate systems.
Payment infrastructure understands commercial state.
Authorization infrastructure knows whether execution was permitted.
AI providers expose model consumption.
Application telemetry describes execution behaviour.
Metering systems record usage.
Observability systems capture failures and retries.
Product systems may know whether a workflow completed successfully.
Finance sees revenue and aggregate spending.
Each system answers part of the question.
The economic picture emerges when those signals can be connected.
One way to think about the architecture is:
Commercial State
↓
Runtime Authorization
↓
Execution
↓
Cost Events
↓
Workflow
↓
Outcome
↓
Economics
The first layers tell us whether execution should happen and what actually happened.
The later layers help explain what that execution meant economically.
This creates an interesting infrastructure problem.
AI systems are producing enough runtime information to reason about more than consumption alone.
The challenge becomes connecting:
Commercial State
+
Runtime Execution
+
Economic Context
We can describe this longer-term architectural direction as Runtime Intelligence.
Not as an established market category.
And not as a claim that infrastructure can automatically determine the correct economic decision for every request.
Many decisions remain contextual.
Some outcomes cannot be valued precisely.
Some customers are intentionally served at lower margins.
Some workflows create strategic value that cannot be captured by immediate revenue.
And some economic signals only become visible after execution has completed.
Runtime Intelligence is therefore better understood as a visibility problem before it becomes an optimization problem.
The objective is to connect trusted runtime events with enough commercial and economic context to help teams understand questions such as:
What happened?
For which customer?
Inside which workflow?
Through which execution path?
At what meaningful cost?
Did it produce the intended outcome?
What economics resulted?
That information does not make the decision automatically.
It makes a better decision possible.
And that may become increasingly important as AI products move from simple model calls toward complex, variable-cost execution graphs.
Closing — What Did This Workflow Actually Cost?
Return to the workflow from the beginning.
From the customer's perspective, it looked like this:
User Request
↓
AI
↓
Successful Result
One request.
One result.
A simple interaction.
But inside the infrastructure, producing that result may have required:
User Request
↓
Context Retrieval
↓
Model Call
↓
Tool Call
↓
Tool Failure
↓
Retry
↓
Second Model Call
↓
External API
↓
Validation
↓
Successful Outcome
Every meaningful operation along that path contributed to the execution.
Some generated direct variable costs.
Some consumed shared infrastructure.
Some existed because failure is unavoidable in production systems.
Some may have been necessary to make the result useful.
Others may have added cost without adding meaningful customer value.
The final model response was only the visible end of that process.
This is why the easiest AI cost to measure may not always be the most useful economic unit to understand.
Model pricing remains important.
Token visibility remains important.
Inference optimization remains important.
But they describe one layer of a larger system.
A model call has a price.
A workflow has an economic cost.
A useful outcome has business value.
The interesting part of AI economics exists in the relationship between the three.
Model Call
↓
Price
Workflow
↓
Execution Cost
Useful Outcome
↓
Business Value
Understanding that relationship does not require perfect attribution.
It does not require assigning a precise financial value to every customer interaction.
And it does not mean every expensive workflow should become cheaper.
Sometimes expensive execution produces disproportionately valuable outcomes.
Sometimes retries are the correct price of reliability.
Sometimes a high-cost customer is also one of the healthiest customers in the business.
The objective is not to minimize execution cost at all costs.
It is to understand what the business is spending, why it is spending it, and what that spending produces.
That understanding begins with reliable runtime data.
If authorization is incorrect, usage is incomplete, retries are miscounted or execution events cannot be trusted, the economic conclusions built on top of them cannot be trusted either.
Runtime Correctness therefore remains the foundation.
The next challenge is connecting that foundation to the economics it produces.
That is the direction I'm currently exploring with Licenzy.
Licenzy is being developed around reliable AI Monetization Runtime infrastructure: keeping commercial state and AI execution correctly aligned.
Runtime Intelligence represents the evolving direction beyond that foundation — exploring how trusted runtime events can be connected with workflow, customer and economic context.
That direction is still being explored.
It is not an argument that infrastructure should automatically decide what is profitable, or that every economic decision belongs inside the runtime.
The more immediate goal is simpler:
Make the economics of execution easier to understand.
Because as AI products become more complex, knowing what the model costs is only part of the picture.
The broader questions are:
What did this workflow actually cost — and was the outcome worth it?
And eventually:
Is my AI business actually healthy — and why?
Top comments (0)