AI infrastructure at enterprises is not just expensive, it's financially invisible. According to new survey data from VentureBeat, two-thirds of companies are now running AI in production, but fewer than half can rigorously track what that compute actually costs. The headline is that performance has overtaken total cost of ownership as a top buying criterion. The deeper story is that this isn't a strategic shift, it's a surrender. When you can't see the bill, you stop making decisions based on it.
The AI Speed Trap: Cost Blindness Became a Feature, Not a Bug
The survey of 170 enterprises reveals a stark reordering of priorities. When selecting an AI infrastructure provider, integration with the existing stack remains top at 40%. But performance (latency and throughput) now ranks second at 35%, and GPU availability third at 24%. Both now outrank total cost of ownership, which sits at 22%. Cost per million tokens is dead last at 16%.
On the surface, this is rational. For the 66% of firms with AI in production, uptime and speed are existential. As we reported in SpaceX Bleeds $1.5 Billion in AI Compute Rush, operational scale drives massive, unrelenting infrastructure demand.
But the logic curdles when paired with another data point: only 47% of enterprises rigorously track the cost and return of their AI compute. Even among the 29% running AI at scale, rigorous tracking only reaches 56%. Value for money is the lowest-rated satisfaction metric (3.87/5), precisely because it's the hardest to judge.
XOOMAR interpretation: This isn't a conscious trade-off of cost for speed. It's a default. Criteria that cannot be measured, like an opaque and fragmented cloud bill, naturally lose to criteria that can, like system uptime or developer velocity. Enterprises aren't choosing to ignore cost, they're choosing the only variables they can actually see. This turns cost blindness from a bug into a systemic feature of the AI buildout.
The Ghost Fleet: 69% of Enterprise GPUs Run at Half Capacity or Less
The waste is not theoretical. Among the 155 enterprises that operate their own GPUs, 69% report utilization of 50% or less. A full 26% run at 25% capacity or below. Only 23% clear the 50% utilization mark.
More troubling than the low utilization is the complete lack of measurement. 12% of GPU operators do not measure utilization at all. They are flying completely blind, unable to claim efficiency or detect waste. This idle capacity represents sunk capital running up power and cooling bills while enterprises simultaneously complain of GPU scarcity and plan their next hardware evaluation.
Inside the Strategic Blind Spot: Why FinOps Fails on AI Workloads
Traditional cloud cost management, or FinOps, was built for predictable, steady-state virtual machines and storage buckets. AI compute workloads are different: they are bursty, they toggle between expensive training and inference modes, and their most critical cost driver, GPU utilization, is often a black box.
The survey shows a self-reinforcing feedback loop. The invisibility of true costs leads teams to deprioritize cost metrics. Those metrics then get less investment in tooling, which perpetuates the blindness. It's easier for a team to feel the visceral impact of a model being down (51% prioritize uptime) or a delayed feature launch (39% prioritize developer productivity) than to decipher an abstract "cost per million tokens."
This operational myopia creates a direct financial risk. Companies are planning their next infrastructure move, 44% are evaluating specialized AI clouds, with a poor understanding of the efficiency, or lack thereof, of their current multi-million-dollar deployments.
The Great Hyperscaler Sleepover: Running Three Platforms Is a Bug, Not a Feature
The data shows the average enterprise runs three AI infrastructure platforms. OpenAI (49%), Google Gemini (48%), Microsoft Azure (47%), and Google Cloud (42%) are each present in nearly half of all stacks.
This isn't strategic multi-cloud brilliance. It's the symptom of a market in transition, where companies use model APIs from one vendor, host custom models on another cloud, and run legacy workloads on a third. Each platform brings its own billing complexity and observability gaps, making a consolidated view of AI spend nearly impossible.
This fragmentation explains the intense, almost desperate, interest in alternatives. AI-specialized clouds like CoreWeave and Lambda are the top planned evaluation area at 44%, despite each having a mere 3.5% current adoption. Companies aren't just shopping for more raw power, they're implicitly shopping for a simpler, more accountable stack.
A Vendor's Dream, A CFO's Nightmare: The Stakeholder Split
The survey data, combined with XOOMAR interpretation, paints a clear picture of diverging priorities within the enterprise:
- Engineering/MLOps View: "Our success metric is uptime (51%) and developer velocity (39%). If the model is down, the business stops. The invoice is a problem for Finance."
- Procurement/Finance View: "We approved the budget for GPUs and cloud instances, but we have no line-item visibility. 53% of our peers can't track costs rigorously. We're writing blank checks."
- Vendor/Sales View (Hyperscalers & Specialized Clouds): "Conversations are about integration (40%), performance (35%), and access to hardware (24%). They are buying on speed and availability. We are not having deep TCO conversations because they can't have them internally."
This disconnect is a goldmine for vendors in the short term. The lack of cost governance turns price into a secondary concern behind getting the infrastructure now. But it plants the seeds for a brutal backlash when finance teams eventually force the issue.
From Dot-Com Burn to AI Burn: A Parallel in Speculative Build
This phase echoes the speculative infrastructure overbuild of the dot-com era, where fiber optic cable was laid for theoretical future demand. Today, it's GPU capacity being provisioned for theoretical future AI models and use cases. The critical difference is that the cloud era that followed was built on granular metering and pay-as-you-go economics from day one.
AI compute has skipped that governance phase. The rush to adopt has leapfrogged the foundational step of measurement. The question is whether this "waste" phase is a necessary cost of a technological revolution or sheer financial negligence that will lead to a sharp correction.
Predictions: The Bill Always Comes Due
The VentureBeat data is a snapshot of a system under stress. The direction of travel points to a coming reckoning.
- The Rise of AI-Specific FinOps: A wave of startups will emerge focusing solely on the unique challenge of measuring GPU utilization, inferencing cost, and ROI for AI workloads. Their value proposition will be turning the invisible bill into a manageable one.
- Cost Per Inference Becomes King: When the next capital tightening cycle hits, whether from a funding winter or internal CFO mandates, the metric that matters will flip. "Cost per million tokens" will cease to be a last-place curiosity and become the paramount benchmark, eclipsing raw latency for all but the most critical applications.
- Specialized Clouds' Make-or-Break: The fate of providers like CoreWeave won't be decided by who has the most H100s. It will be decided by who can solve the visibility problem. The winner will offer not just raw hardware, but the tools to prove that using it is more efficient than the fragmented, underutilized hyperscaler stacks enterprises run today. As seen in the competition highlighted in Wall Street Bets $500 Billion on AI Over Crypto Compute, the flow of capital is fierce, but it will eventually seek accountability.
The takeaway for enterprises is counterintuitive: slowing down to measure might be the fastest way to win. The companies that survive the coming efficiency purge won't be the ones who bought infrastructure the fastest, but the ones who first learned to see what they were actually spending.
Impact Analysis
- Enterprises are making major financial decisions about AI infrastructure without visibility into costs, potentially overspending.
- The shift in priority from total cost to performance indicates a strategic risk, focusing on short-term speed over long-term financial sustainability.
- With two-thirds of companies in production, this cost blindness could scale into industry-wide inefficiency, affecting profitability and innovation.
Originally published on XOOMAR. For more news and analysis, visit XOOMAR.
Top comments (0)