Google Ironwood TPU (TPU7x) vs NVIDIA B200: the 2026 inference-cost decision

Summary. Google's Ironwood (TPU7x) and NVIDIA's B200 land in almost the same place on paper: 192 GB of high-bandwidth memory per accelerator, roughly 7.37 TB/s of HBM bandwidth on Ironwood against about 8 TB/s on the B200, and near-identical FP8 dense throughput of about 4.6 versus 4.5 PFLOPS. Ironwood went generally available from November 2025 as Google's seventh-generation TPU; NVIDIA's Blackwell B200 shipped across every major cloud through 2025. The real decision is not the chip. It is where you can rent it, what software you already run, and how you pay. Ironwood scales to a 9,216-chip superpod with about 1.77 PB of shared HBM and 9.6 Tb/s inter-chip links, but only on Google Cloud. The B200 rents from roughly $3.70 to $14.24 per GPU-hour across providers as of July 2026, and also runs on-premises. This article compares both on memory, throughput, scale, software lock-in and cost, and gives a decision matrix for AI teams, including Indian teams weighing region availability and data residency.

If you already track B200 versus H100 inference cost per token, this is the sibling question: what happens when the alternative is not another GPU but Google's custom silicon.

The short answer

Pick Ironwood TPU7x when your serving stack is already on Google Cloud, your models run on JAX or PyTorch through XLA, and you need a very large single memory pool for decode-heavy or Mixture-of-Experts inference. Pick the NVIDIA B200 when you need multi-cloud or on-premises placement, when your team is committed to CUDA and TensorRT-LLM, or when portability across providers matters more than squeezing the last unit of price-performance from one vendor.

Both are strong inference accelerators. The mistake is treating this as a benchmark contest. At this tier the hardware is close enough that switching cost, availability and utilisation decide the bill.

Spec-by-spec: Ironwood TPU7x vs NVIDIA B200

Ironwood is one chip built from two chiplets. Each chiplet carries one TensorCore, two SparseCores and 96 GB of HBM, so a full chip exposes two TensorCores, four SparseCores and 192 GB of HBM, per Google's TPU7x documentation (last updated 16 July 2026). The B200 is a dual-die Blackwell GPU with 208 billion transistors and fifth-generation Tensor Cores that add native FP4.

Specification	Google Ironwood (TPU7x)	NVIDIA B200 (Blackwell)
HBM per accelerator	192 GB	Up to 192 GB HBM3e (some HGX configs list 180 GB)
HBM bandwidth	~7.37 TB/s	~8 TB/s
Peak FP8 (dense)	~4.6 PFLOPS (4,614 TFLOPs)	~4.5 PFLOPS
Lower-precision peak	2,307 BF16 TFLOPs	~9 PFLOPS FP4 (dense)
Max scale-up domain	9,216 chips per superpod, ~1.77 PB shared HBM, 9.6 Tb/s ICI	72 GPUs per rack (GB200 NVL72)
Software	JAX and PyTorch (no TensorFlow), XLA, vLLM	CUDA, PyTorch, TensorRT-LLM, vLLM
Placement	Google Cloud only	AWS, Azure, Google Cloud, Oracle, neoclouds, on-prem

Two numbers matter for inference. Memory capacity decides how large a model, and how big a KV cache, you can hold without spilling. Both sit at 192 GB. Memory bandwidth decides decode throughput on memory-bound token generation, where the B200's roughly 8 TB/s edges Ironwood's 7.37 TB/s. On paper this is a narrow gap, not a generation apart.

Where Ironwood pulls ahead

The TPU story is about the pod, not the chip. Google connects up to 9,216 Ironwood chips into a single superpod linked by Inter-Chip Interconnect at 9.6 Tb/s, giving workloads access to about 1.77 PB of shared HBM in one domain. Optical Circuit Switching reconfigures the fabric around a failed link with no human intervention, which matters when a single job spans thousands of chips for days. NVIDIA's rack-scale answer, the GB200 NVL72, links 72 GPUs over NVLink; beyond the rack you scale over InfiniBand or Ethernet rather than one optical fabric.

Google reports Ironwood delivers a 10x peak performance improvement over TPU v5p and more than 4x better performance per chip than the previous v6e (Trillium) generation for both training and inference. On the software side, Google's GKE Inference Gateway load-balances across TPU servers and, per Google, cuts time-to-first-token latency by up to 96% and serving cost by up to 30%. Because vLLM now supports TPUs, teams can move an inference service between GPUs and TPUs with only minor configuration changes, which lowers the cost of testing the switch.

There is external validation of demand. Anthropic said it plans to access up to 1 million TPUs. "Ironwood's improvements in both inference performance and training scalability will help us scale efficiently while maintaining the speed and reliability our customers expect," said James Bradbury, Head of Compute at Anthropic.

Where the B200 pulls ahead

The B200 wins on reach. It rents on AWS, Azure, Google Cloud, Oracle and a long list of specialist "neocloud" providers, and it runs in your own data centre. Ironwood runs on Google Cloud alone. For a regulated enterprise that needs an on-premises deployment, or a team that refuses single-cloud lock-in, that difference ends the debate before specs enter it.

The second advantage is software gravity. A decade of CUDA, plus TensorRT-LLM and a deep tooling ecosystem, means most existing inference code, kernels and observability already target NVIDIA. Ironwood supports JAX and PyTorch through XLA but does not support TensorFlow, and moving custom CUDA kernels to a TPU is real engineering work. If your team ships on CUDA today, the B200 is the path of least resistance. The same portability logic runs through our analysis of the NVIDIA Rubin buildout and enterprise cloud budgets.

The cost model: how you actually pay

Sticker throughput is not the bill. Utilisation, idle time, networking and migration effort are. The two platforms even price in different units.

Dimension	Ironwood TPU7x (Google Cloud)	NVIDIA B200 (multi-cloud)
Billing unit	Per chip-hour, sold in 4-chip VMs (tpu7x-standard-4t)	Per GPU-hour
On-demand indicative	Published per chip-hour on the Google Cloud TPU pricing page; varies by region and commitment	~$3.70 to $14.24 per GPU-hour across providers (July 2026)
Spot / preemptible	Spot TPU VMs available	~$3.59 per GPU-hour reported on some providers
Discount levers	Committed-use (1 and 3 year), calendar-mode and long-term reservations, All Capacity mode	Reserved and committed GPU-hours
Smallest viable unit	4-chip VM (single host)	1 GPU
Lock-in risk	Google Cloud plus XLA/JAX	CUDA ecosystem

Google does not publish a single headline chip-hour figure the way GPU aggregators list a range, because TPU rates depend on region, commitment and capacity mode; the Google Cloud TPU pricing page is the source of record. B200 rental, by contrast, spreads widely: aggregators put on-demand pricing from about $3.70 per GPU-hour on lower-cost providers to $14.24 per GPU-hour on hyperscaler instances as of July 2026, with spot pricing near $3.59. That spread is the real lesson. The same B200 can cost four times as much depending on where you rent it, so provider selection often saves more than chip selection. Indian teams costing this in rupees should read our breakdown of GPU cloud rental pricing in India alongside the global ranges here.

The honest cost comparison is per useful token served at your target latency and utilisation, not per advertised PFLOP. A cheaper accelerator running at 40% utilisation loses to a costlier one at 85%. Model that before you commit.

What changes with the eighth-generation TPU

Ironwood is no longer Google's newest silicon. On 22 April 2026, at Google Cloud Next, Google announced its eighth generation as two specialised chips: TPU 8t for training and TPU 8i for inference, both scheduled for general availability later in 2026. TPU 8i is the one to watch for serving. Google pairs 288 GB of HBM with 384 MB of on-chip SRAM, roughly 3x the previous generation, to keep a model's active working set on-chip, doubles inter-chip bandwidth to 19.2 Tb/s for Mixture-of-Experts models, and claims 80% better performance-per-dollar than Ironwood, enough to serve nearly twice the volume at the same cost. Both eighth-generation chips claim up to 2x better performance-per-watt than Ironwood.

The practical read: if you are buying committed capacity now for a 2026 serving workload, Ironwood is the available, proven option today. If your timeline is flexible into late 2026, price the TPU 8i story into your plan rather than locking a long commitment against Ironwood at full rate. This cadence is exactly why we treat accelerators as a rolling decision in our guide to AI compute capacity planning.

A decision matrix

Your situation	Lean Ironwood TPU7x	Lean NVIDIA B200
Serving stack already on Google Cloud	Yes	Only if multi-cloud is required
Codebase is CUDA / TensorRT-LLM	Migration cost is high	Yes
Runs on JAX or PyTorch via XLA	Yes	Yes
Needs on-premises or multi-cloud placement	No	Yes
Very large single shared-memory pool	Yes (superpod)	Rack-scale via NVL72
Portability across vendors is the priority	No	Yes
Lowest achievable rental with provider shopping	Limited to Google Cloud	Yes (wide price spread)

India-specific considerations

Indian AI teams face two constraints beyond raw price: region availability and data residency. B200 capacity is available through multiple clouds and Indian GPU providers, which gives buyers negotiating room and the option to keep inference in-country. TPU availability is tied to Google Cloud regions, so confirm the specific region and capacity mode before committing.

Data residency is the second factor. Where inference touches personal data of Indian users, the Digital Personal Data Protection Act 2023 (DPDP) governs how that data is handled, so the deployment region and the provider's data-processing terms belong in the decision, not just the hourly rate. Public cloud demand in India is rising fast: Gartner forecast end-user public cloud spending in India would reach $17.5 billion in 2026, up 28.1% year on year, with infrastructure-as-a-service the fastest-growing segment as AI workloads scale. That demand is precisely what makes disciplined accelerator selection worth the effort.

FAQ

Is Ironwood TPU7x faster than the NVIDIA B200?

They are close. Ironwood delivers about 4.6 PFLOPS of dense FP8 against roughly 4.5 PFLOPS on the B200, with 192 GB of HBM on both. The B200 has slightly higher memory bandwidth at about 8 TB/s versus 7.37 TB/s. Real-world speed depends on your model, batch size and software more than peak specs.

When did Ironwood become generally available?

Google announced Ironwood, its seventh-generation TPU, on 6 November 2025 and made it generally available in the following weeks. The TPU7x documentation describes the shipping configuration, including 9,216 chips per superpod. It has been in production use since late 2025, including by Anthropic, which said it plans to access up to 1 million TPUs.

How much does a B200 cost to rent in 2026?

As of July 2026, B200 on-demand rental ranges from about $3.70 per GPU-hour on lower-cost providers to $14.24 per GPU-hour on hyperscaler instances, with spot pricing near $3.59. The wide spread means provider choice often affects your bill more than the choice between GPU and TPU.

Can I run the same model on both?

Often yes. Both support PyTorch, and vLLM now runs on TPUs and GPUs, so a serving stack can move between them with minor configuration changes. The friction is custom CUDA kernels, which do not port to TPUs directly, and TensorFlow, which Ironwood does not support.

What is the difference between a TPU chip and a GPU here?

An Ironwood chip is built from two chiplets, each with one TensorCore, two SparseCores and 96 GB of HBM, exposed to frameworks as two devices. A B200 is a single dual-die GPU. For inference planning, compare per accelerator: both offer 192 GB of HBM at this tier.

Should I wait for the eighth-generation TPU 8i?

It depends on timeline. TPU 8i, announced 22 April 2026 for general availability later in 2026, claims 288 GB of HBM and 80% better performance-per-dollar than Ironwood. If your deployment can wait, price it in. If you need capacity now, Ironwood is the proven, available option today.

Which is better for a regulated Indian enterprise?

If you need on-premises deployment or in-country data residency across providers, the B200 gives more placement options. Ironwood is Google Cloud only. Where inference touches personal data, the DPDP Act 2023 makes region and data-processing terms part of the decision, not just the hourly rate.

Does choosing the cheaper accelerator save the most money?

Not usually. Utilisation, idle time and migration effort move the bill more than the sticker rate. A costlier accelerator at 85% utilisation can beat a cheaper one at 40%. Compare cost per useful token served at your target latency, and shop providers, before committing to committed-use discounts.

How eCorpIT can help

eCorpIT is a Gurugram-based, CMMI Level 5 and ISO 27001:2022 certified engineering organisation, and a partner of AWS, Microsoft and Google. Our senior-led teams help AI-heavy companies benchmark inference on TPUs and GPUs against their own models, model cost per useful token at real utilisation, and choose a placement and commitment plan that fits both budget and data-residency requirements. If you are weighing Ironwood, the B200 or the coming eighth-generation TPUs, talk to our team for a workload-specific assessment.

References

1. Announcing Ironwood TPUs General Availability and new Axion VMs — Google Cloud Blog
2. TPU7x (Ironwood) architecture and specifications — Google Cloud Documentation
3. Our eighth generation TPUs: two chips for the agentic era — Google (The Keyword)
4. Inside the Ironwood TPU codesigned AI stack — Google Cloud Blog
5. Cloud TPU pricing — Google Cloud
6. Anthropic to expand use of Google Cloud TPUs and services — Google Cloud Press Corner
7. NVIDIA HGX B200 datasheet (PDF)
8. Comparing NVIDIA's B200 and H100 — Civo
9. B200 cloud pricing across providers (2026) — GetDeploying
10. NVIDIA B200 pricing, July 2026 — Thunder Compute

11. Gartner forecasts India public cloud spending to surpass $17.5 billion in 2026 — Gartner Newsroom

12. NVIDIA GB200 NVL72 — 72 Blackwell GPUs, rack-scale NVLink

Last updated: 27 July 2026.