Keith MacKay

Posted on Sep 20 Originally published at tlcmentor.substack.com

Uber Burned Its Entire 2026 AI Budget by April. Is Your Turn Coming?

#ai #infrastructure #leadership

Uber Burned Its Entire 2026 AI Budget by April. Is Your Turn Coming?

Tokens are the new compute hours, and most teams are managing them like cloud compute -- from 2019. Here's your tokenomics primer.

You deployed an AI coding assistant to your engineering team six months ago. Usage lit up immediately. The developers loved it. Then the April invoice arrived. Someone in finance called. The entire year's AI budget was gone.

This is not a hypothetical. Uber rolled out Claude Code to roughly 5,000 engineers in December 2025. By April 2026, the full year's AI budget had been consumed [1]. Microsoft followed by canceling internal Claude Code licenses, citing runaway token bills [1]. Uber's own CTO has since confirmed the number publicly [2]. These are not companies that can't afford AI. They're companies that had no system for managing what AI actually costs.

Per-token inference costs fell roughly 75% year-over-year [3]. And yet enterprise AI bills keep climbing. Economists have a name for this: Jevons' paradox: as efficiency improves, total consumption expands to fill and exceed the savings [4]. That is what happens when nobody is watching the meter.

Welcome to the token economy -- time to get a handle on it.

Part 1: Trends -- What's Happening?

Token consumption is not following an S-curve. It's a staircase.

Each time a new architectural pattern becomes mainstream (from chat to retrieval-augmented generation to agents to reasoning models), token usage jumps, briefly stabilizes, then jumps again as the next pattern unlocks. No plateau in sight. Google now processes 1.3 quadrillion tokens per month, more than 20x growth in a single year [5].

The data is striking. Average prompt token length grew roughly 4x since early 2024: from about 1,500 tokens per request to over 6,000 [6]. Per-developer token consumption climbed even faster in absolute terms: engineering analytics firm Jellyfish measured an 18.6x increase over nine months, driven almost entirely by agentic coding features [7]. Simple queries, which accounted for 80% of enterprise requests in early 2024, had collapsed to 20% by year end as multi-step chains took over [8]. Enterprise generative AI spend went from $1.7 billion in 2023 to $11.5 billion in 2024 to $37 billion in 2025 [9]. Cloud bills overall are rising roughly 19% year-over-year, with AI workloads as the primary driver [10].

The biggest accelerant is agents. An agentic workflow burns 10 to 100 times more tokens than an equivalent chat session [11]. Why? Because agents re-send the full conversation context on every tool call. An agent making 20 tool calls during a single task isn't sending 20 separate prompts. It's sending the entire accumulated context 20 times. One audit traced 62% of total agentic costs back to re-sent context alone [11].

Reasoning models compounded this further. Once teams discovered that more powerful reasoning models improved accuracy on complex tasks, adoption cascaded fast. Reasoning-optimized models went from negligible to over 50% of all enterprise tokens by late 2025 [6]. Each one costs more per token (sometimes a lot more) than its predecessor.

The sticker-shock cycle runs like this: costs are invisible → access expands → bills spike → finance panics → governance gets imposed → per-unit cost falls through optimization → access expands again. Repeat until someone builds a system. The companies that manage this cycle deliberately are the ones that don't get the April phone call.

The severity has since escalated past the anecdotal stage. One company reportedly ran up a $500 million Claude bill in a single month after deploying access without usage caps [12]. Goldman Sachs projects global token usage will multiply 24x by 2030 [12] -- meaning the companies feeling the pain in April 2026 are, by that math, still near the beginning of the curve.

Part 2: Observability -- Where Are We?

Most organizations can't answer a basic question: which team, feature, or workflow is responsible for their AI costs?

Traditional FinOps tools miss 70 to 90% of real AI costs [13]. An LLM API bill arrives as a single line item. No workflow breakdown. No feature attribution. No customer-level view. The team that built the runaway agent has no idea they built a runaway agent. They just know the demos went well.

The FinOps Foundation found that 98% of organizations now actively manage AI spend, up from 31% just two years ago [14]. That jump didn't happen because teams built good systems. It happened because the bills arrived and nobody could explain them. The Foundation's 2026 survey put a number on the result: 73% of organizations reported that their AI costs exceeded original projections, some by more than 2.4x [15]. "Actively managing" AI spend and actually seeing where it goes are two different things -- and the 70-90% miss rate on traditional FinOps tools is the gap between them. The Wall Street Journal put the underlying dynamic plainly: "AI economics are brutal. Demand is the variable to watch" [16].

The observability gap runs deeper than tooling — it varies by how your organization buys AI. SaaS packaged tools abstract tokens entirely: you pay a seat fee and have no visibility into consumption efficiency. Direct API access makes tokens explicit but volatile: every design choice in prompt length, context window, and model selection shows up on the invoice. Owned or self-hosted infrastructure internalizes token economics fully, with costs flowing through GPU utilization, power, and storage rather than per-token billing. Most organizations are operating across all three simultaneously, with governance built for none of them.

Before you can govern what you're spending, regardless of buying mode, you need answers to some surprisingly basic questions:

What does each request actually cost, broken down by model, workflow, and team?
What's your cache hit rate? Are you paying full price for prompts you've already sent?
Which agents have token budgets and which are running unconstrained?
What triggers an alert before a runaway loop burns the month?

In most organizations, none of this exists. Engineers build, finance gets surprised, and leadership asks what happened after the fact.

As my colleague Michael Stricklen pointed out in a LinkedIn article today, this is not always the case -- for organizations gamifying token spend with leaderboards, the stats on who's using how many tokens are being captured. Tying that back to actual return on investment is not. Uber itself ranked engineering teams by token consumption before it imposed spending caps, which optimized for activity rather than outcomes. As Stricklen puts it, "a merged pull request is not evidence of value" [17] -- the enterprise built the numerator (cost per team, per engineer, per repo) without ever building the denominator (whether the code shipped was worth what it cost to generate).

Part 3: Controls and Planning -- Where Do We WANT to Be?

Token economics is the operating model of AI, not a detail of strategy. That requires a governance shift at the top: Technology leaders need to wear their finance hats, understanding cost flows and unit economics the way they'd think about cloud or datacenter spend. Finance leaders need to wear their tech ops hats, understanding that AI spend doesn't respond to traditional per-seat or per-core cost models.

The analogy that keeps surfacing is cloud FinOps. And it's the right one -- it's now getting institutional weight to match. The Linux Foundation formally launched the Tokenomics Foundation in mid-2026, a standards body explicitly modeled on FinOps, built to establish common definitions and cost-tracking practices for AI tokens the way FinOps did for cloud [12]. That's not a validation of a metaphor. It's an acknowledgment that the problem has outgrown ad hoc tooling.

In 2010, cloud spend was invisible. Developers provisioned VMs without cost context, and finance got surprised every quarter. The FinOps discipline emerged to close that gap: shared visibility, budget ownership at the team level, forecasting models, optimization levers like reserved instances and rightsizing, and chargeback systems that made costs legible.

Token economics needs the same treatment. The FinOps Foundation has started mapping LLM infrastructure optimization strategies to familiar cloud cost levers [14]. These aren't perfect analogues -- routing to GPT-5.5 instead of Claude Opus 4.8 because it's available or better suited to a task is less like rightsizing a VM and more like switching hypervisors -- but they provide useful starting points for teams already fluent in thinking about ways to manage cloud cost management:

Model tier routing ~ VM rightsizing
Prompt caching ~ Reserved Instances
Per-agent budget limits ~ cloud spend alerts
Chargeback by team ~ cloud showback
Agent runaway loops ~ orphaned compute instances

The goal isn't to minimize token spend. It's to understand and govern it. Optimization that kills legitimate use cases is the wrong target. Optimization that eliminates waste while preserving value is the right one. The data backs this up: Jellyfish's research found the heaviest token users were roughly twice as productive as light users, but consumed 10 times the tokens to get there [7]. Whether that trade is worth it depends entirely on whether the shipped code generates value proportional to the spend -- a question most companies still can't answer, because they haven't built the observability to ask it.

A mature TokenOps posture looks like this: teams know their token budgets, models are matched to task complexity, expensive reasoning is reserved for tasks that actually need it, finance has a forecast model they trust, and nobody learns about AI costs from a surprise invoice.

Part 4: Tooling, Team Responsibilities, Training, and Incentives -- How Do We Get There?

The good news: the tooling exists. Most teams just haven't wired it up.

Routing is the highest-leverage starting point. Between 60% and 70% of production queries are simple enough for the smallest model tier [18]. Defaulting every query to a flagship model is the AI equivalent of running every workload on your biggest VM. Routing layers like LiteLLM and Portkey automatically send simple queries to cheaper models, cutting blended costs 60 to 85% on routed traffic [18]. Teams typically start with rule-based routing ("if token count under 500, use the small model") and graduate to classifier-based routing as traffic diversifies. For teams willing to go further, open-weight models carry economics that closed APIs structurally can't match; the Linux Foundation's research on the hidden economics of open models quantifies just how large that gap becomes at scale [19].

Prompt caching is the second lever and one of the most underutilized. Anthropic's implementation reduces costs by up to 90% and latency by up to 85% on long cached prompts [20]. If your system prompt, policy block, or style guide is the same across thousands of requests, you're paying full price for it every single time you don't cache it. That's not a technical limitation. That's a configuration choice.

Observability layers (Langfuse, LangSmith, Helicone) give you per-request cost and latency tracing. Without them, routing is guesswork. With them, optimization becomes a data problem.

Budget guardrails close the loop. Gateways like LiteLLM support per-agent, per-user, and per-team spend caps. An agent that hits its monthly token budget stops before it burns the year's runway. This is not optional for agentic deployments. Vendors are starting to build this in natively rather than leaving it to third-party gateways: Anthropic shipped admin-level spend alerts and model entitlements directly into Claude Enterprise in July 2026, triggered at 75% and 90% of an org's spend limit -- a direct, if overdue, response to exactly the failure mode that hit Uber and Microsoft [21].

Tools are necessary. They are not sufficient. The cloud FinOps parallel holds here too: the discipline succeeded when it became a shared responsibility: finance, engineering, and product. Not a single team clutching a spreadsheet.

Specifically:

Engineering owns instrumentation, routing logic, and caching architecture
Product owns cost-quality trade-offs per feature (is reasoning-model accuracy worth 10x the cost for this use case?)
Finance owns forecasting models, budget allocation, and showback reporting
Leadership owns the incentive structure

That last one is where most organizations stop short. If engineers are measured purely on feature velocity with no cost visibility, they default to the most capable model for everything. Cost context changes behavior. But only if it's legible and connected to what teams actually care about.

Training matters in ways that are easy to underestimate. Most developers don't know that front-loading stable context unlocks up to 90% cache discounts [20]. They don't know that specifying output format constraints reduces completion token bloat. They don't know that batching non-urgent requests can cut costs nearly in half. These aren't advanced techniques. They're a page of guidance nobody gave them.

The Bottom Line

Token costs are falling. Token consumption is rising faster. The gap between unit price and total bill will keep widening until organizations treat token management as a discipline, not an afterthought.

The three-stage pattern plays out everywhere: Honeymoon (costs hidden in SaaS subscriptions), Reckoning (direct API bills, the surprise invoice), Factory (internalized economics, governed consumption). Most enterprise teams are stuck between stages one and two right now.

The companies that build TokenOps practices today (observability, routing, caching, budget governance, team accountability) will have a structural cost advantage over the ones waiting for the next April phone call. The stakes are higher than a bad quarter: Gartner projects that over 40% of agentic AI projects will be canceled by the end of 2027, and escalating, ungoverned cost sits alongside unclear business value as one of the top reasons why [22]. TokenOps isn't just a cost-control exercise. For a meaningful share of AI initiatives, it's the difference between shipping and getting killed.

References

How is your organization managing token costs? Are you in the Honeymoon, the Reckoning, or the Factory? I'd like to hear what's working. And what blew up.

Related free tool: For a model-agnostic LLM gateway that implements the routing, budget controls, and observability described in Part 4 — built in Rust, with per-individual/team/project token budgets, OTEL observability, hooks for DLP or evals, and full CLI for automation: modelrouter on GitHub

If this resonated, here are some related articles:

For why falling unit prices don't mean your AI costs are under control (the infrastructure economics behind the bill): On LinkedIn: AI Infrastructure Scarcity is Raising Costs, but AI Usage Will Still Provide Unbeatable ROI | On Substack | On Medium
For how providers quietly dial back model quality to manage their own cost pressures (the supply-side mirror to this article's demand-side story): On LinkedIn: AI Shrinkflation: Your AI Model Was Quietly Dialed Back | On Substack | On Medium
For why AI governance stalls at the director and VP layer, which is where token budget enforcement breaks down most often: On LinkedIn: Why Your AI Transformation Is Stalling at Middle Management | On Substack | On Medium
For what happens when token governance fails AND your vendor goes offline simultaneously — the Fable/Mythos event as a live case study in why a proxy layer with failover is non-optional: On LinkedIn: Your Cloud AI Has No Failover. Here's the Architecture That Does. | On Medium

Keith MacKay is a technology strategy consultant and CTO in EY-Parthenon's Software Strategy Group (SSG), specializing in AI disruption and technology diligence for private equity and corporate clients. SSG's AI Disruption Lab conducts rapid assessments of how AI transforms and threatens existing business models and value chains. Keith teaches at Northeastern University and writes about strategy, management, and AI/technology, with Claude Code and Codex as AI collaborators.

Top comments (2)

Dean Lee • Sep 20

The Jevons paradox framing explains the volume, but the balance sheet problem runs deeper. Chained tool use quietly transforms engineering from a predictable fixed labor expense into an unhedged variable cost.

When an engineer works manually, overhead is strictly bounded by payroll. When that same engineer prompts an autonomous agent to explore a legacy codebase, the organization essentially writes an open-ended call option on token consumption. The sixty-two percent context re-send figure illustrates why linear task growth produces quadratic billing growth. Without strict prefix caching and hard spend caps enforced at the proxy layer, finance will keep discovering that developer velocity is being funded by unbudgeted debt.

mthburnsbarber-web • Sep 20

"A merged pull request is not evidence of value" — Stricklen's point embedded here is the one that should be uncomfortable for anyone running agentic coding at scale. Uber built the numerator (token cost per team, per engineer) without the denominator (whether the code actually shipped value). That's the real failure, not the budget overrun.

The Jevons' paradox framing is exactly right and underused in this conversation. Per-token costs fell 75% year-over-year and total bills keep climbing because each cheaper token just unlocks more aggressive usage patterns. This is the cloud compute story replayed in compressed time.

The 62% of agentic costs traced back to re-sent context is the stat I'd lead with in any executive briefing. Not model pricing, not seat counts — the fact that an agent making 20 tool calls is sending the full accumulated context 20 times. That's the lever that no amount of model rightsizing fixes.

The three-stage pattern (Honeymoon → Reckoning → Factory) is a useful diagnostic. Most enterprise teams are exactly where you put them: between stages one and two. The routing + caching + observability + budget guardrails stack is the right prescription, but you're also right that tools alone aren't sufficient — it requires the incentive structure to change.

At Black Label we're building for enterprise clients who are hitting stage two right now.