Everything Worked
Imagine a customer triggers an AI workflow in production.
The request reaches the authorization layer.
The customer has an active entitlement.
There is enough usage capacity available.
Authorization succeeds.
The workflow starts.
It retrieves the required context, calls the model, executes an external tool and produces the expected result.
Usage is recorded correctly.
The customer's balance is updated.
The result is delivered.
Observability shows green.
No exception was raised.
No duplicate execution occurred.
No usage event was lost.
No reconciliation issue appeared later.
From an engineering perspective, the system behaved exactly as designed.
Everything worked.
Now look at the same execution from a different perspective.
The workflow required several model calls.
The context was larger than usual.
One external API added another variable cost.
An intermediate step failed and had to be retried.
The final result required additional processing before it became useful to the customer.
None of this made the execution incorrect.
The customer was entitled to the workflow.
The infrastructure handled the retry safely.
Usage was measured accurately.
The customer was charged according to the product's rules.
And yet, after accounting for what it actually cost to produce the useful outcome, the execution generated less revenue than it consumed.
Nothing failed.
The execution simply wasn't economically healthy.
That distinction matters because production systems are usually designed to detect incorrect behaviour.
They are much less likely to tell you when technically correct behaviour is producing unhealthy economics.
Correct Execution Is Not the Same as Healthy Execution
AI infrastructure has become increasingly good at answering questions about runtime correctness.
Was the request authorized?
Was the customer entitled to use the feature?
Was consumption applied atomically?
Were retries handled idempotently?
Was usage recorded correctly?
Did application state remain synchronized with commercial state?
These are important questions.
Getting them wrong can produce duplicate consumption, incorrect access, lost revenue or broken customer experiences.
But even a system that answers all of them correctly still leaves another question unresolved:
Was this execution economically sustainable?
Consider two properties of the same workflow.
| Runtime perspective | Economic perspective |
|---|---|
| Request authorized correctly | Cost justified by the revenue or value produced |
| Entitlement valid | Customer remains healthy to serve |
| Usage recorded correctly | Usage cost remains sustainable |
| Retry handled safely | Total retry cost remains acceptable |
| Execution completed | Completed outcome remains economically viable |
| Customer charged correctly | Revenue adequately covers cost-to-serve |
The left side does not guarantee the right side.
A request can be perfectly authorized and still be expensive.
Usage can be perfectly metered and still represent an unhealthy consumption pattern.
A retry can be perfectly idempotent and still add legitimate cost to an already low-margin workflow.
A customer can be charged exactly according to plan and still cost more to serve than the revenue that plan generates.
This is not an argument against runtime correctness.
Quite the opposite.
Runtime correctness is foundational.
Without it, the economic picture cannot be trusted in the first place.
But correctness tells us whether the system behaved according to its rules.
It doesn't necessarily tell us whether those rules produced a healthy business outcome.
That is a different problem.
And AI makes that distinction increasingly difficult to ignore.
AI Makes This Problem Different
Traditional software has always had infrastructure costs.
Servers cost money.
Databases cost money.
Storage, bandwidth and third-party services cost money.
But for many SaaS products, the marginal cost of one additional customer action is relatively small.
A user opens another dashboard.
Creates another project.
Runs another database query.
Sends another internal request.
The infrastructure performs more work, but the economics of the business rarely depend on the profitability of that individual action.
AI changes this relationship.
A single customer request may trigger a chain of operations with directly measurable variable costs:
- Model inference
- Input and output tokens
- GPU workloads
- Vector search
- Retrieval pipelines
- External APIs
- Agent tool calls
- Image generation
- Voice processing
- Human review
And those costs are rarely uniform.
The same feature may be inexpensive for one execution and significantly more expensive for another.
A short support question may require one small model call.
A complex research task may require a long context window, multiple model calls, retrieval, several tools and repeated reasoning steps.
Both may appear in the product as the same feature.
Operationally, they are not the same workload.
Economically, they may be completely different.
This creates a tighter connection between architecture and business economics.
Model selection matters.
Context size matters.
Retry behaviour matters.
Workflow design matters.
Tool selection matters.
Exception handling matters.
Not because every expensive execution is necessarily bad.
An expensive workflow can be perfectly healthy if it creates enough value or supports sufficient revenue.
The important point is that execution now has an economic dimension that cannot always be inferred from whether the request succeeded.
The Cost of the Successful Request Is Not the Whole Cost
Suppose an AI workflow eventually completes successfully.
The final execution consumed $0.18 of model inference.
It would be tempting to record $0.18 as the cost of producing the result.
But production workflows rarely exist as isolated model calls.
The actual execution history may look more like this:
Customer Request
↓
Model Call
↓
Tool Call
↓
Tool Timeout
↓
Retry
↓
Second Model Call
↓
External API
↓
Human Correction
↓
Successful Outcome
The final model call may have cost $0.18.
But $0.18 is not necessarily what it cost the business to produce the outcome.
The failed attempt still consumed resources.
The retry still consumed resources.
The tool call may have generated a third-party charge.
The external API may have its own usage cost.
Human intervention may have introduced support or operational overhead.
The useful economic unit is therefore not always the final successful request.
Sometimes it is the entire path required to produce the successful outcome.
That distinction matters.
Imagine two executions producing the same customer-visible result:
Execution A
1 model call
1 successful tool call
No retry
Total execution cost: $0.24
Execution B
2 model calls
1 failed tool call
1 retry
1 successful tool call
Total execution cost: $0.61
From a product perspective, both workflows succeeded.
From a billing perspective, the customer may have paid exactly the same amount.
From an economic perspective, they are not equivalent.
This is where execution-level visibility becomes useful.
Not because every internal operation needs to become a financial metric.
That would create complexity of its own.
Attribution has a cost.
Instrumentation has a cost.
Human support can be difficult to allocate precisely.
Shared infrastructure costs rarely map perfectly to a single request.
The objective is not perfect accounting at arbitrary granularity.
It is enough visibility to understand where economically meaningful differences are coming from.
For some products, model and API costs may provide sufficient visibility.
For others, retries, workflow stages or human intervention may materially change the economics and deserve to be included.
The appropriate level of attribution depends on the product.
But the principle remains:
The cost of a successful AI outcome may include every meaningful attempt required to produce it—not only the request that finally succeeded.
This also changes how we think about failure.
A failed execution is not always financially neutral.
The customer may never receive an outcome.
The business may still have paid for the work performed before the failure.
At sufficient scale, that difference becomes important.
A technically correct usage ledger can tell you exactly what was consumed.
Understanding whether that consumption created sustainable value requires another layer of analysis.
Customer Profitability Is a Distribution Problem
Aggregate metrics are useful because they compress complexity.
Total revenue.
Average revenue per customer.
Average infrastructure cost.
Gross margin.
At a company level, these numbers can provide a reassuring picture.
But averages hide distribution.
And in AI products, that distribution can matter significantly.
Consider two customers on the same plan.
Both pay:
$200 / month
Both use the same product.
Both appear as equally valuable accounts in a revenue dashboard.
Their underlying economics, however, may look very different.
Customer A
Short context
Standard model
Predictable workflow
Few retries
Minimal support
Monthly cost-to-serve: $38
Customer B
Long context
Expensive model routing
Multiple tool calls
Frequent retries
Exception-heavy workflows
Regular support intervention
Monthly cost-to-serve: $176
The revenue is identical.
The economics are not.
Customer A contributes substantial margin.
Customer B may still be profitable, but only narrowly.
Add a few unusual execution paths, additional support requests or expensive failures and that account could become unprofitable without anything appearing wrong in the billing system.
The customer paid exactly what they were supposed to pay.
The product delivered exactly what was promised.
The problem exists in the relationship between revenue and cost-to-serve.
This is why customer profitability is fundamentally a distribution problem.
A portfolio can look healthy in aggregate while containing customers with radically different economic profiles.
For example:
100 Customers
↓
Average Gross Margin: 70%
↓
Looks Healthy
But underneath that average:
70 customers → high margin
20 customers → moderate margin
8 customers → low margin
2 customers → negative margin
The 70% average is not wrong.
It is simply incomplete.
It tells you about the portfolio.
It doesn't tell you where profitability comes from.
I explored this problem from the business side in Your AI Business Can Grow While Your Margins Shrink, where I looked at why averages can hide radically different customer and workflow economics.
That distinction becomes more important when AI consumption varies significantly between accounts.
It also means that high usage should not automatically be treated as a problem.
A power user may generate substantial cost while also generating substantial revenue, retention value or strategic importance.
Another customer may consume far less infrastructure but still be economically unattractive because of pricing, support overhead or inefficient workflows.
Usage alone doesn't determine profitability.
The relationship between revenue, consumption and cost-to-serve does.
The question isn't which customers use the most AI. It's whether the economics of serving them remain healthy.
Workflow Profitability May Matter More Than Model Cost
Customer-level profitability provides one useful perspective.
But it still leaves an important question unanswered.
Why is one customer more expensive to serve than another?
The answer often exists one level deeper.
Inside the workflows they execute.
AI engineering teams naturally monitor infrastructure metrics such as:
- Token consumption
- Model cost
- Latency
- API calls
- GPU utilization
- Retry rates
These metrics are valuable.
They help teams understand how the system behaves.
But they don't necessarily explain whether that behaviour creates enough value to justify its cost.
Consider two workflows.
Workflow A
Cost: $1.40
Outcome:
A qualified sales opportunity
Workflow B
Cost: $0.22
Outcome:
An internal summary that is rarely used
Workflow B is cheaper.
That doesn't automatically make it economically better.
The relevant question depends on what the product is trying to accomplish.
Customers don't usually purchase tokens, inference calls or tool executions as ends in themselves.
They purchase outcomes.
A resolved support ticket.
A processed document.
A completed research task.
A qualified lead.
A successfully completed agent workflow.
This suggests another useful unit of analysis:
Cost per useful outcome.
Instead of measuring only:
Cost per Model Call
some products may benefit from understanding:
Total Cost of Workflow
↓
Successful Outcome
↓
Cost per Useful Outcome
Suppose a support automation processes 1,000 conversations.
The infrastructure metrics show:
Total AI Cost: $420
That number is useful.
But imagine only 600 conversations are resolved without escalation.
Looking at successful outcomes provides additional context:
$420 total execution cost
÷
600 resolved conversations
=
$0.70 per resolved conversation
Now the infrastructure cost can be compared with something closer to business value.
That might help answer questions such as:
- Is automation still cheaper than the alternative?
- Which workflows create the most expensive successful outcomes?
- Do retries materially change outcome economics?
- Does changing models improve cost without reducing completion quality?
- Are some execution paths expensive but still economically justified?
This metric is not universally applicable.
Defining a "useful outcome" can be difficult.
Some products create value gradually rather than through discrete completions.
A research assistant may influence a decision without producing an easily measurable economic event.
An AI coding tool may save developer time that is difficult to attribute precisely.
Multi-step agent systems may produce value across several workflows.
Attribution can quickly become more complex than the insight it provides.
The objective is therefore not to force every AI product into a single profitability metric.
The architectural lesson is simpler.
Infrastructure cost becomes more useful when it can be connected to the customer activity and business outcome that produced it.
Token cost tells you what the model consumed.
Workflow cost tells you what the system consumed.
Outcome economics begin to tell you whether that consumption was worth it.
Those are different levels of understanding.
Visibility Before Optimization
Once teams discover that AI costs vary significantly across customers and workflows, the natural reaction is to optimize.
Use a cheaper model.
Reduce context size.
Introduce stricter limits.
Change pricing.
Route requests differently.
Reduce retries.
These may all be reasonable decisions.
But they assume something important:
That the team already understands where the economic problem actually is.
Often, it doesn't.
A rising model bill tells you that spending increased.
It doesn't tell you which customers caused the increase.
A high token count tells you that the system processed more context.
It doesn't tell you whether that context produced valuable outcomes.
A high retry rate tells you that executions required additional attempts.
It doesn't tell you whether those retries were necessary, wasteful or economically significant.
And a declining gross margin tells you that the business is becoming more expensive to operate.
It doesn't tell you why.
Before optimization, teams increasingly need enough visibility to connect several layers of the system:
Customer
↓
Workflow
↓
Execution
↓
Retries / Failures
↓
Usage
↓
Cost
↓
Outcome
The goal is not perfect attribution.
Perfect attribution may be impossible or unnecessarily expensive.
The goal is decision-quality visibility.
Enough information to distinguish between very different situations.
For example, suppose infrastructure costs increase by 20%.
Without additional context, the obvious response might be to move more traffic to a cheaper model.
But deeper visibility could reveal that the increase came primarily from one workflow experiencing repeated tool failures.
The model wasn't the problem.
Or perhaps one customer segment began using significantly longer contexts because of a new product behaviour.
The pricing model may need attention.
Or perhaps costs increased because customers are completing more high-value workflows.
In that case, higher infrastructure spending may be perfectly healthy.
The same cost increase can therefore represent:
Waste
or
Product Growth
or
Pricing Misalignment
or
Workflow Inefficiency
or
Healthy High-Value Usage
The number alone cannot tell you which one.
Context can.
This is why optimization without visibility is mostly guesswork.
Engineering Is Becoming Part of Economic Analysis
Once execution data is connected to customers and workflows, familiar engineering signals begin to acquire a second meaning.
A retry rate is no longer only a reliability metric.
It may also explain why a workflow costs more than expected.
A metering discrepancy is no longer only a data-quality issue.
It may distort the company's understanding of customer profitability.
A stale entitlement is no longer only an access-control problem.
It may allow expensive execution under commercial conditions that are no longer valid.
A reconciliation gap is no longer only an operational inconsistency.
It may prevent the business from knowing whether recorded revenue and actual consumption describe the same reality.
This does not mean engineering teams should become finance teams.
Nor does it mean every infrastructure decision should be reduced to margin.
Reliability, latency, quality and customer experience still matter independently.
The change is that these concerns can no longer always be evaluated in isolation.
In AI products, infrastructure behaviour increasingly influences the economic behaviour of the business.
Consider a model-routing decision.
Engineering might evaluate:
Latency
Quality
Reliability
Cost per call
Those are useful dimensions.
But once the same execution is connected to workflow outcomes, another comparison becomes possible:
Model A
Higher cost per call
Better completion rate
Fewer retries
Higher useful-outcome rate
versus:
Model B
Lower cost per call
Lower completion rate
More retries
More escalations
The cheaper model is not necessarily the cheaper workflow.
And the more expensive workflow is not necessarily the worse business decision.
The answer depends on the outcome economics.
This is where engineering telemetry begins to become business intelligence.
Not because infrastructure metrics suddenly become financial metrics.
But because connecting them provides context that neither side has independently.
From Runtime Correctness to Economic Correctness
Up to this point, we have been discussing two different properties of an AI system.
The first is familiar.
The system should behave correctly.
Requests should be authorized.
Consumption should be atomic.
Retries should be safe.
Usage should be accurate.
Commercial and application state should remain aligned.
Runtime authorization is one part of that problem. In The Most Expensive AI Request Is the One You Should Have Blocked, I explored why deciding whether an AI request should execute can itself become an economic decision.
We can think of this as Runtime Correctness.
It answers:
Did the system execute correctly?
But the examples throughout this article expose another question.
Suppose all of those guarantees hold.
The request was authorized.
The workflow executed correctly.
The retry was legitimate.
Usage was recorded accurately.
The customer was charged exactly according to plan.
And the completed outcome still cost more to produce than its economics could support.
The runtime was correct.
The business outcome was unhealthy.
A useful way to reason about this second property is Economic Correctness.
Not as an established industry category, but as a mental model.
It asks:
Did this technically correct execution also make economic sense?
The distinction looks simple:
| Runtime Correctness | Economic Correctness |
|---|---|
| Was execution authorized? | Was execution economically justified? |
| Was consumption correct? | Was cost-to-serve sustainable? |
| Was usage accurate? | Did usage produce sufficient value? |
| Were retries safe? | Did retries preserve healthy outcome economics? |
| Was commercial state respected? | Did the resulting execution remain commercially viable? |
Neither replaces the other.
Economic analysis built on incorrect runtime data cannot be trusted.
And technically correct execution without economic visibility can still produce an unhealthy business.
A sustainable AI product increasingly needs to understand both.
Correct Execution
+
Economic Understanding
↓
Healthier AI Economics
This does not mean every request needs a real-time profitability calculation before it executes.
That would often be impractical, unnecessary or based on incomplete information.
Some economics can only be understood after the workflow completes.
Some costs arrive later.
Some outcomes are difficult to quantify.
Some customers should rationally be served at lower margins for strategic reasons.
Economic correctness is therefore not a binary runtime rule.
It is a way of asking whether the technical behaviour of the system remains aligned with the economic objectives of the business.
And asking that question requires connecting information that traditionally lives in different systems.
The Missing Connection Is Runtime Intelligence
The signals required to understand AI economics already exist in many production systems.
The problem is that they usually exist in different places.
Payment systems understand commercial state.
Authorization systems understand whether execution is allowed.
AI providers report model consumption.
Application logs describe execution behaviour.
Metering systems record usage.
Observability platforms capture failures and retries.
Product systems may know whether the workflow produced a useful outcome.
Finance sees revenue and aggregate cost.
Each system contains part of the story.
Very few contain the whole story.
A simplified execution may therefore generate information across several layers:
Commercial State
↓
Authorization
↓
Execution
↓
Metering
↓
Retries / Failures
↓
Outcome
↓
Economics
Individually, these signals answer useful questions.
Commercial state tells us what the customer purchased.
Authorization tells us whether the operation was allowed.
Execution telemetry tells us what actually happened.
Metering tells us what was consumed.
Failure data tells us what additional work occurred.
Outcome data tells us whether the workflow achieved its objective.
Economic data tells us what serving that activity ultimately cost.
The harder problem is connecting them.
Suppose a customer generates unusually high model spend.
That fact alone tells us very little.
Perhaps the customer is unprofitable.
Perhaps they are one of the company's most valuable accounts.
Perhaps a specific workflow is inefficient.
Perhaps a provider started producing more failures.
Perhaps retries increased after a deployment.
Perhaps usage increased because the product is delivering significantly more value.
The cost signal becomes useful only when it can be interpreted in context.
This suggests an architectural direction that goes beyond traditional observability.
We can think of it as Runtime Intelligence.
Not as an established infrastructure category, and not as a system that automatically knows which business decisions are correct.
Rather, as the ability to connect runtime behaviour with the commercial and economic context surrounding it.
Instead of observing only:
What happened?
the system begins helping teams understand:
What happened?
For which customer?
Inside which workflow?
Under which commercial conditions?
At what cost?
After which failures or retries?
And with what outcome?
That additional context can make infrastructure data substantially more useful to both engineering and business teams.
It can help reveal patterns such as:
- Customers whose cost-to-serve changes significantly over time
- Workflows where retries materially affect economics
- Execution paths responsible for disproportionate infrastructure spending
- Features whose usage grows faster than the value they produce
- Expensive workflows that remain healthy because they generate high-value outcomes
- Differences between expected and observed consumption
The objective is not to automate every decision.
Visibility may lead to a pricing change.
Or a workflow redesign.
Or different model routing.
Or better retry handling.
Or a product decision.
Or no change at all.
The important step is understanding the system well enough to make that decision deliberately.
Runtime Intelligence, in this sense, is less about autonomous optimization and more about connecting technical behaviour to economic consequence.
Closing
Return to the execution from the beginning.
The customer was authorized.
The entitlement was valid.
The workflow executed successfully.
Retries were handled correctly.
Usage was recorded accurately.
The customer was charged exactly according to the product's rules.
Observability showed green.
Everything worked.
That remains an important achievement.
But we can now ask one more question.
What did it actually cost to produce the useful outcome?
Perhaps the answer shows a healthy execution.
Perhaps the workflow was expensive but generated enough value to justify that cost.
Perhaps the customer remains highly profitable despite heavy usage.
Or perhaps several technically correct behaviours combined to produce an outcome whose economics are difficult to sustain.
Runtime correctness alone cannot distinguish between those situations.
It was never designed to.
That is why the next challenge for AI infrastructure may not simply be making execution more reliable.
It may be making the economics produced by that execution more visible.
Not every company needs request-level profitability.
Not every workflow has an easily measurable outcome.
Not every expensive customer should be optimized for margin.
And not every economic decision belongs inside the runtime.
But as AI products become more dependent on variable-cost execution, the connection between infrastructure behaviour and business health becomes harder to ignore.
The question evolves from:
Is my AI system working correctly?
to something broader:
Is my AI business actually healthy — and why?
That is the problem I'm currently exploring with Licenzy.
Licenzyis being developed around the idea of an AI Monetization Runtime: infrastructure for keeping commercial state and AI execution correctly aligned.
The direction we're exploring beyond that foundation is Runtime Intelligence — connecting reliable runtime behaviour with better visibility into the customer, workflow and economic outcomes it produces.
That work is still an exploration.
But the underlying question is already useful regardless of which infrastructure a team ultimately uses.
Your AI system may be working exactly as designed.
The harder question is whether you understand the economics it is producing.
Top comments (0)