FinOps grew up managing one kind of cost: cloud infrastructure. Instances, storage, data transfer, the stuff on your AWS or GCP or Azure bill. That scope is now too narrow, because a new operational cost has shown up that behaves differently from everything FinOps was built for, and most teams have no idea how big it is: AI token spend.
The industry conversation this year has been blunt about it. Token usage has become a major operational expense, and organizations still lack a good way to connect that spend to developer productivity or business outcomes. That gap is the whole problem. Let me lay out why AI spend breaks the old FinOps model and what tracking it actually looks like.
Why token spend is not just another line item
You might think, it is on the cloud bill (Bedrock, Vertex) or an API invoice (OpenAI, Anthropic), so treat it like any other cost. It is not that simple, for three reasons:
1. It is usage-metered at a granularity nothing else is. An EC2 instance costs the same whether it does important work or spins idle. A token costs the same either way too, but the volume is driven by application behavior in a way instance-hours are not. One inefficient prompt, one chatty agent, one retry loop, and volume explodes. The cost is coupled to code behavior, not provisioned capacity.
2. It hides across invoices. Some AI spend is on the cloud bill, some is on a separate SaaS-style API invoice, some is buried in a vendor's platform fee. Unlike cloud cost, which at least lives in one console, AI spend is scattered, so nobody sees the total. The first task is often just adding it all up.
3. It resists attribution. Ask "which team's EC2 is this" and tags get you an answer. Ask "which feature generated this token spend" and most teams have nothing. The requests go through a shared API key, and the key does not know which product feature, which customer, or which team is behind each call.
What tracking AI spend actually requires
Bringing AI cost under FinOps is not exotic. It is the same muscles, retrained on a new resource:
- Aggregate it first. Pull every source of AI spend into one view: cloud-billed model usage, direct API invoices, platform fees. You cannot manage a number you have not assembled.
- Attribute it. Route requests so you can tag them by feature, team, or customer. In practice this means per-team or per-feature API keys, or a gateway that stamps metadata on each call. Without this, every optimization is guesswork.
- Right-size the model, not just the instance. The new rightsizing question is "which model does this request actually need." Most requests are over-modeled the way most instances are over-provisioned. Routing easy requests to a cheap tier and reserving the frontier model for hard ones is often the single biggest saving.
- Catch runaway usage. A retry loop, a caching bug, an agent stuck in a plan-execute cycle. These are the new "forgot to turn off the GPU box," and they show up as a token-spend anomaly before anyone files a ticket. Anomaly detection on token volume is now as necessary as anomaly detection on cloud spend.
The part the industry is stuck on: connecting spend to value
Here is where AI FinOps is genuinely harder than cloud FinOps, and where nobody has fully cracked it. With infrastructure, "is this worth it" is often obvious, the server runs the product. With AI, the question is sharper: is this feature's token spend producing proportional value? A summarization feature costing more in tokens than it saves users in time is a bad trade, but you only see that if you track spend and outcome together.
The teams getting ahead are the ones instrumenting both sides: cost per feature on one axis, usage or productivity or revenue impact on the other. That is unit economics applied to AI, and it is the direction FinOps has to go. The bill alone cannot tell you whether the spend is smart.
The practical starting point
You do not need a platform to begin. This week:
- Add up every source of AI spend into one number. Most people are surprised by the total.
- Put the top two or three AI features behind separate keys or a tagging gateway so you can attribute the biggest chunks.
- Set one anomaly alert on total token volume so a runaway loop does not become a month-end surprise.
- Pick your highest-volume request path and test whether a cheaper model handles it acceptably.
That is a week of work and it moves AI spend from "invisible and growing" to "measured and managed," which is the whole game.
The take
FinOps that stops at the cloud bill is now measuring the wrong boundary. AI token spend is an operational cost with its own behavior, scattered across invoices, coupled to code, and stubbornly hard to attribute, and it is growing fast. The same discipline applies, aggregate, attribute, right-size, alert, but you have to actually point it at tokens, and then take the harder step of tying that spend to value.
Are you tracking AI token spend as a first-class cost yet, with attribution, or is it still landing on invoices nobody reconciles? And has anyone here actually connected a feature's token cost to its value? That last part is where I keep seeing teams stall.
Top comments (0)