This came up in Chinese AI developer communities recently: a Nomura estimate claiming that chatbots like ChatGPT and Claude consume only about 5% of global AI compute, while model training takes 35% and heavy agentic workloads claim 23%. The post was making the rounds as a kind of humbling corrective -- look how small the thing you think of as "AI" actually is.
I want to push back on the framing. Not on the numbers themselves, which Nomura openly labels as rough signals rather than audited figures. I want to push back on the conclusion people are drawing from them: that chatbot compute is small, stable, and therefore unimportant to the infrastructure story going forward.
That conclusion is probably wrong, and the way the estimate is structured makes it almost impossible to tell.
The categorization problem nobody is talking about
When Nomura (or anyone) slices AI compute into buckets, they have to decide where to put inference that happens inside a product. A chatbot call to GPT-4 is clearly inference. But what about a Copilot session that rewrites a function, runs a linter, checks docs, and proposes a diff? Is that "office tools" at 4%? Is it "heavy agents" at 23%? Is it split?
The honest answer is: it depends entirely on how the analyst drew the boundary, and there is no public methodology attached to these numbers.
This is not a minor quibble. Agentic workloads are largely chatbot sessions with tool-calling bolted on. The same transformer weights, the same KV cache pressure, the same inference infrastructure. If an analyst decided that anything involving tool use goes into the "agents" bucket, then the 5% figure for chatbots is technically accurate but narratively misleading -- it offloaded a huge chunk of what users experience as conversation into a different row.
The "users don't feel it" argument cuts both ways
The original post makes a reasonable point: 40 billion people are served by recommendation and ad systems, and most of them have no idea AI is involved. That compute is invisible to the user.
But the conclusion drawn -- that chatbot compute being only 5% means the chatbot business is small -- does not follow.
Consider the marginal cost structure. Recommendation models run on a trained, frozen model. One forward pass per user per pageload, batched aggressively, latency requirements loose enough that you can queue. The compute per user-interaction is tiny, and it runs 24 hours a day whether users are active or not.
Chatbot inference is different. It is synchronous. It is latency-sensitive. Users will abandon a session that takes 10 seconds to respond. It requires maintaining KV cache state across multi-turn conversations. You cannot batch a user waiting for a reply the same way you batch ad ranking.
So 5% of total compute, allocated to the most latency-constrained, least batchable, most user-visible workload in the stack, might actually represent significant over-investment in premium inference capacity -- H100s running at low utilization to hit p95 latency targets -- rather than a minor rounding error.
The percent-of-total metric hides this completely.
Training at 35% is the number that should worry you
The original framing uses the training figure to argue that the "real" AI story is in the labs, not in your chat window. Okay. But training is a one-time cost per model generation. You train GPT-5, you pay the 35% (or whatever fraction), and then you run inference on it for the next 18 months.
As inference scales -- more users, longer context windows, more multi-turn sessions, more agentic loops -- the training share shrinks as a fraction of total compute even if the absolute training cost stays flat. That is just arithmetic.
The Nomura estimate is a snapshot of today. It does not tell you what the curve looks like in two years when a billion more users are running 100-turn agent sessions with 128k context.
What the estimate is actually useful for
I do not want to dismiss the data entirely. The estimate is genuinely useful for one thing: it corrects the public misconception that AI infrastructure spending is driven by ChatGPT traffic. It is not, at least not yet. The labs are spending most of their compute on training and on agentic pipelines that enterprise customers run in batch overnight.
That is a real and important corrective to the narrative that "AI adoption" can be read off from chatbot usage statistics.
But the leap from "chatbots are 5% today" to "chatbot infrastructure doesn't matter" is the kind of reasoning that gets data centers built in the wrong places. A workload that is small today, grows fast, has uniquely high per-unit infrastructure requirements, and sits at the user-facing edge of every major AI product is not a rounding error. It is the canary.
The invisible compute -- training, recommendations, ad ranking -- keeps the lights on for the labs financially and technically. But the visible compute, the 5% that responds when you type, is where the user relationship lives. Underbuilding it because the percentage looks small is a mistake that cloud providers and model companies have already made once, in late 2022, and paid for with months of waitlists and rate limits.
The one question I keep coming back to
If agentic workloads are already the second-largest consumer of compute at 23%, and agents are mostly chatbots with tool-calling enabled, at what point does it become more honest to call that number "chatbots" and retire the distinction entirely?
Top comments (0)