A single ChatGPT conversation, roughly 20 to 50 queries, consumes about half a liter of water. That's the number that stopped me. Not the electricity, not the carbon, not the GPU shortage. The water. Because water is the one input that doesn't have a substitute, and the AI industry is drinking from a straw that's getting shorter every quarter.
Here's the claim I want to test: the binding constraint on AI scaling over the next five years is not compute, not chips, not capital. It's thermal management, and specifically the water and energy required to reject heat from data centers. And the solutions being proposed are mostly incremental patches on a problem that is structural.
Let's get specific. Not "AI uses a lot of energy." Specific numbers, specific physics, specific alternatives, and one sacred cow in the AI infrastructure narrative that deserves to be led out back.
The Numbers, Because the Discourse Is Vague
Data centers consumed roughly 460 TWh globally in 2022, about 2% of world electricity. The IEA projects that could double by 2026, driven largely by AI workloads. Some estimates put AI-specific data center demand at 3 to 4x growth by 2030.
That's the headline. Here's the part that matters.
A single H100 GPU draws up to 700W under load. A DGX H100 node with 8 GPUs pulls roughly 10.2 kW at the wall. A rack of 8 such nodes is over 80 kW. Traditional enterprise racks were 5 to 10 kW. We've increased rack density by an order of magnitude in three years.
A hyperscale AI training cluster of 100,000 H100s pulls roughly 70 to 100 MW continuous. That's a small city's worth of power, running 24/7 for months during training.
Water consumption for evaporative cooling is the invisible cost. A 100 MW data center using evaporative cooling consumes roughly 1 to 2 million liters of water per day. That's the annual water use of 1,000 to 2,000 households, per day. In drought-prone regions, this is not abstract.
PUE (Power Usage Effectiveness) is the industry metric for overhead. A PUE of 1.1 means 10% overhead. Hyperscalers claim 1.1 to 1.2. Older facilities run 1.5 to 2.0. The difference is billions of kWh annually.
Put it together and you get a physical constraint that no amount of capital can wave away: you have to reject the heat. Every watt that goes into a GPU comes out as heat, and that heat has to go somewhere. Air, water, or phase change. Those are the options.
Why Water Became the Default (and Why It's a Trap)
Evaporative cooling works because water has a high latent heat of vaporization. One liter of water absorbs roughly 2,260 kJ when it evaporates. That's a lot of heat rejection per liter. It's why data centers in dry climates have historically used cooling towers: cheap, effective, proven.
The trap is threefold.
First, water is local. A data center in Arizona competes for water with farmers, municipalities, and ecosystems. A data center in Ireland or the Netherlands competes with housing and agriculture. The water doesn't vanish. It moves. But it moves out of a watershed that other people depend on.
Second, water is political. In 2023, Chile's environmental regulator blocked a Google data center expansion in Santiago over water use concerns. In 2024, a Virginia county pushed back on new data center permits over groundwater depletion. These aren't edge cases. They're the leading edge of a pattern.
Third, water is a proxy for a deeper problem. The reason we need so much water is that we're rejecting enormous heat loads from dense racks. The water is a symptom. The heat density is the disease.
The Sacred Cow: "PUE Is the Right Metric"
Here's the belief I want to dismantle: PUE is the industry's primary efficiency metric, and it's mostly useless for understanding AI's environmental footprint.
PUE measures total facility power divided by IT power. A PUE of 1.1 sounds great. But PUE says nothing about:
Water consumption (WUE, Water Usage Effectiveness, is tracked by almost nobody publicly)
Carbon intensity of the grid the facility sits on
Embedded carbon in the GPUs, which is enormous and amortized over 3 to 5 years
The marginal load the facility adds to a strained grid, which is often fossil-fuel peaker plants
A data center with PUE 1.05 in a coal-heavy region is worse for the climate than one with PUE 1.4 in Iceland. PUE rewards engineering, not outcomes. And the industry has optimized hard for the metric that flatters it, while the metric that matters, total environmental cost per useful inference, remains mostly unmeasured.
If you're a builder or operator, the implication is direct: stop reporting PUE as your climate story. Report WUE, grid carbon intensity, and marginal emissions. If you can't report those numbers, you don't know your footprint.
The Real Alternatives, Ranked by Physics (Not Marketing)
Let's talk about what actually works, because the alternatives are not equal.
- Liquid immersion cooling. Direct-to-chip and full immersion cooling replace air with dielectric fluid or water-glycol loops. Heat rejection capacity per rack jumps from 20 to 30 kW to 100 to 200 kW. Microsoft's Project Natick demonstrated sealed underwater data centers with a PUE around 1.07 over two years. Immersion is real, it's shipping, and it's the most likely near-term replacement for high-density AI racks. The tradeoff: retrofit cost, fluid handling, and the fact that most data centers weren't built for it.
- Two-phase and phase-change cooling. Boiling a dielectric fluid at the chip level carries away far more heat per unit volume than single-phase liquid. Startups like CoolIT, ZutaCore, and Accelsius are shipping this. It's more efficient than single-phase, but it's harder to maintain and the fluids are expensive.
- Free cooling and climate siting. If you can reject heat to ambient air for most of the year, you don't need evaporative water at all. This is why data centers are moving to Iceland, Norway, Sweden, and parts of Canada. Free cooling is the cheapest option when the climate allows it, and it's why the "build in the desert" model is going to die.
- Heat reuse. Rejecting heat into district heating networks, greenhouses, or industrial processes. Stockholm's data centers already do this. It's the highest-leverage move because it turns a waste stream into a product. The constraint is geography: you need a customer for the heat, and most data centers aren't near one.
- Behind-the-meter generation. Small modular reactors, geothermal, and on-site solar-plus-storage. Microsoft's 2024 deal to restart Three Mile Island is the highest-profile version. SMRs are the most speculative, but the physics favors nuclear for 24/7 high-density loads. Grid interconnection queues are 3 to 7 years in most US markets, which is longer than the useful life of a GPU generation.
- Algorithmic efficiency. This is the one the industry underweights. Quantization, distillation, speculative decoding, and MoE routing can cut inference energy by 5 to 10x for the same quality. A 4-bit quantized 70B model running on the right hardware can match a 16-bit version on many tasks at a fraction of the power. The cheapest watt is the one you don't draw. Notice the pattern. Every alternative is a tradeoff between capital cost, operational complexity, and geographic flexibility. There is no free lunch. There is only a different bill.
The Physics Limit Nobody Wants to Say Out Loud
Here's the part that gets buried in every vendor whitepaper.
The second law of thermodynamics says you cannot reject heat without doing work. The higher the ambient temperature, the harder it gets. The denser the rack, the more work per unit volume. There's no technology that breaks this.
What that means practically: the AI industry's thermal problem gets worse, not better, as models scale. A 10x increase in compute at the same efficiency is a 10x increase in heat that has to go somewhere. Efficiency improvements buy you time. They don't buy you a different equation.
The only paths out are:
Stop scaling compute (politically and commercially unlikely)
Move to places with abundant cold and clean power (Iceland, Norway, Quebec, and a handful of others, and they're filling up)
Build generation on-site (SMRs, geothermal, which take a decade to deploy)
Make the compute itself radically more efficient (quantization, distillation, better architectures, which helps but doesn't eliminate the problem)
If you're betting on AI scaling, you're implicitly betting on one of those four. Most people building in this space have not thought about which one.
What To Actually Do
Three moves, concrete enough to act on.
- Add thermal and water metrics to your infrastructure monitoring. If you're running inference or training at any meaningful scale, instrument for:
# Minimum viable sustainability telemetry for an AI workload
metrics = {
"gpu_power_w": gpu_power_draw(), # per-GPU wall power
"facility_pue": total_facility_power() / it_power(),
"wue_l_per_kwh": water_liters() / energy_kwh(),
"grid_carbon_g_per_kwh": grid_intensity(region),
"marginal_emissions_g": marginal_grid_emissions(region, load_mw),
"tokens_per_joule": tokens_generated() / energy_joules(),
}
If you can't fill in every field, you don't know your footprint. Start there.
- Treat tokens-per-joule as a first-class metric. Most teams optimize for latency and cost per million tokens. Add energy per token to the dashboard. A 4-bit quantized model with speculative decoding can cut energy per token by 3 to 8x with minimal quality loss on most tasks. That's not a niche optimization. It's the single highest-leverage efficiency move available today.
- Site for physics, not for tax breaks. If you're choosing where to run workloads, weight cold climate, low-carbon grid, and water availability above tax incentives. The next decade of data center buildout will be constrained by interconnection queues and water rights, not by land cost.
The Mental Model
Here's the lens I want you to keep.
AI's environmental cost is not a side effect. It's a physical throughput constraint on the entire industry, and it's the one constraint that no amount of capital can engineer away in the short term. Compute scales with money. Water and heat rejection scale with physics. The mismatch is the crisis.
The interesting question is not whether AI can be sustainable. It's whether the industry will treat thermal management as a core engineering discipline, the way it treats model architecture and chip design, before the constraints force the issue. The companies that get this right will have a structural advantage for the next decade. The ones that don't will be buying water rights in a drought.
What's your read? If you've worked on AI infrastructure, what's the real bottleneck in your experience: power, water, chips, or something else entirely? Drop it in the comments. I'm especially curious whether the water story is as underdiscussed in practice as it looks from the outside.
Top comments (0)