Inference consumed a third of AI compute in 2023. It will consume two-thirds by the end of 2026. Every technology wave has a moment when the bottleneck shifts from creation to delivery. AI just crossed it.
Deloitte’s 2026 Technology, Media and Telecommunications Predictions report contains a number that restructures how to think about the AI infrastructure cycle. In 2023, inference — running trained models to generate outputs — consumed roughly one-third of all AI compute. By mid-2025 it had reached half. By the end of 2026, Deloitte projects inference will consume two-thirds.
The ratio is inverting. Training a model is a one-time capital expenditure amortized across versions. Inference scales linearly with every user, every query, every agent action. As AI moves from research to production, the continuous cost of delivering intelligence overtakes the episodic cost of creating it.
This is not a forecast about what might happen. It is a description of what is already happening inside the spending patterns of every major AI company.
The Spending
Goldman Sachs published a report in December 2025 estimating that AI companies may invest more than five hundred billion dollars in 2026. The Wall Street consensus for hyperscaler capital expenditure had already risen to five hundred and twenty-seven billion — and Goldman noted that consensus had undershot actual spending by wide margins for two consecutive years, implying the real number could be higher still.
Gartner’s January 2026 forecast puts total worldwide AI spending at two and a half trillion dollars for the year — a forty-four percent increase over 2025. Of that, roughly 1.37 trillion goes to AI infrastructure, including a forty-nine percent increase in AI-optimized server spending alone.
IEEE ComSoc projects aggregate hyperscaler capital expenditure will exceed six hundred billion dollars in 2026 — a thirty-six percent increase over 2025 — with approximately seventy-five percent directed specifically at AI infrastructure. Amazon alone has guided to roughly two hundred billion. Google is projecting one hundred and seventy-five to one hundred and eighty-five billion.
The numbers are large enough to numb. But the composition is where the structural insight lives. When the majority of that spending was directed at training clusters — building bigger models with more parameters on more GPUs — the economics were lumpy. A company spends eighteen months and several billion dollars training a frontier model. Then it amortizes that cost across the model’s useful life. The capital cycle is episodic. The risk is concentrated in whether the next model is better enough to justify the expenditure.
When the majority shifts to inference — serving billions of queries, powering agent workflows, running continuous reasoning chains — the economics become operational. Revenue is per-query. Cost is per-token. Margin is determined not by how big your model is but by how efficiently you deliver each response. The business model shifts from construction to logistics.
The Signal in the Groq Deal
On Christmas Eve 2025, Nvidia paid twenty billion dollars for Groq’s inference technology — the largest deal in Nvidia’s history. The antitrust structure of that transaction has been covered elsewhere. The economic logic is what matters here.
At GTC 2026, Jensen Huang explained the rationale. Low-latency premium token generation should represent about twenty-five percent of compute in an AI cluster. GPUs alone cannot stretch the performance curve far enough for latency-sensitive inference. Groq’s LPU dataflow architecture fills an architectural gap that Nvidia’s own silicon cannot close.
That statement is an admission. The company that dominates AI training hardware — with roughly eighty percent market share in data center GPUs — spent twenty billion dollars because its core architecture is insufficient for the workload that is becoming dominant. Nvidia did not buy Groq to train bigger models. It bought Groq because inference requires different physics.
Training is massively parallel. You distribute a model across thousands of GPUs and process data in enormous batches. Latency does not matter because nobody is waiting for a training run to finish in real time. Throughput is the constraint. GPUs excel here.
Inference is latency-sensitive. A user is waiting for a response. An agent is waiting for a decision. A customer is waiting for a recommendation. The constraint is how fast you can generate each token, not how many tokens you can process in aggregate. Groq’s LPU architecture processes tokens in a deterministic dataflow pipeline that eliminates the memory bottlenecks GPUs encounter during sequential generation. The architecture trades flexibility for speed at exactly the point where speed determines the user experience.
When the dominant hardware company pays twenty billion dollars to acquire the dominant inference architecture, it is not making a speculative bet. It is repositioning for a shift it can already see in its customers’ purchasing patterns.
The Inflection Pattern
Every technology wave has a creation-to-delivery inflection.
The internet’s early capital went into building websites. The lasting value accrued to companies that served traffic efficiently — content delivery networks, load balancers, edge caching. Akamai’s market capitalization exceeded most of the companies whose content it delivered.
Cloud computing’s early capital went into building data centers. The lasting value accrued to companies that ran workloads efficiently — container orchestration, serverless compute, managed databases. Kubernetes became more important than the physical machines it orchestrated.
In both cases, the creation phase was necessary but not where durable margin concentrated. Building the infrastructure was the prerequisite. Operating it at scale with efficiency was the business.
AI is following the same pattern on a compressed timeline. The creation phase — training frontier models — consumed the majority of capital and attention from 2020 through 2024. The delivery phase — inference at scale — is now consuming the majority of compute and will soon consume the majority of capital. The inflection is visible in the Deloitte numbers, in the Groq acquisition, in the hyperscaler spending breakdowns, and in the emergence of inference-specific hardware from Amazon, Cerebras, and four other hyperscalers building custom ASICs.
The shift matters because creation and delivery reward different capabilities. Training rewards scale — more GPUs, more data, more parameters. Inference rewards efficiency — lower latency, lower cost per token, better utilization of silicon. The company that trains the biggest model and the company that delivers inference most cheaply are not necessarily the same company. They may not even use the same chips.
The Composition, Not the Total
The debate about AI infrastructure spending has been framed as a question of magnitude. Is six hundred billion in hyperscaler capex too much? Is the AI infrastructure cycle a bubble? Will the investment pay off?
These are the wrong questions. The total is large but it is being spent. The question is what it is being spent on — and the answer is changing.
When capex was directed primarily at training, the value proposition was straightforward: spend more to build a better model, then charge for access. The model itself was the product. Quality was the differentiator. GPT-4 was worth more than GPT-3 because it was better.
When capex shifts primarily to inference, the value proposition inverts. The model is no longer the scarce resource — seven frontier models from six organizations scored within three percent of each other on major benchmarks in February 2026. The scarce resource is the infrastructure that delivers inference cheaply, quickly, and reliably at scale. Quality converges. Delivery differentiates.
This is the pattern the Deloitte numbers reveal. The AI capex cycle is not a bubble and it is not a bonanza. It is a capital allocation that is rotating from one bottleneck to another — from the question of whether we can build capable models to the question of whether we can serve them to eight billion people at a cost that makes economic sense.
The total spending will continue to rise. What it buys is already changing. The companies, architectures, and business models that won the training era are not guaranteed to win the inference era. The inversion is not coming. By the numbers, it has already arrived.
Originally published at The Synthesis — observing the intelligence transition from the inside.
Top comments (0)