Note: This article is adapted from Phala’s official blog post, “TEE is Really Fast on Nvidia Blackwell,” originally published on July 2, 2026. Full credit goes to the Phala research team for the original research and writing.
See it here: https://phala.com/posts/blackwell-gpu-cc-serialized-bridge
TEE is Really Fast on Nvidia Blackwell
Confidential computing has always come with a trade off. You get privacy, but you pay for it in speed. New research from Phala’s team shows that on Nvidia’s newest Blackwell chips, that trade off is starting to disappear, and the real story is more interesting than just it’s faster now.
The chip is no longer the problem
On Nvidia’s B300 Blackwell chips, running AI computation under confidential computing costs almost nothing. Matrix multiplication, the basic math behind every AI model, runs at 0.998x normal speed. On longer runs, it actually matched or slightly beat non-confidential speed. For years, people assumed privacy meant slow. That assumption no longer holds on this hardware. The chip itself is not where the slowdown lives anymore.
So where does the cost go instead?
It goes into the connection between the secure part of the system and the GPU. Every time data has to move across that connection, it moves one piece at a time and pays a real setup cost, even when the software thinks it is running in the background. This quietly breaks a lot of popular AI serving tools, including widely used ones like vLLM. These tools try to save time by overlapping tasks, but under confidential computing, that trick backfires and adds delay instead of avoiding it. In one test, simply turning off that default setting took performance from 3550 tokens per second to 4104. A deeper fix pushed it to 5518 tokens per second, landing within 8 percent of normal, non-confidential speed.
The cost depends on what you are running
Not every workload pays the same price. Light, simple serving barely notices any slowdown at all. Heavier, steady workloads lose around 13 to 14 percent. Mixture of experts models and jobs that rely heavily on cached data lose the most, up to 27 percent, because they move more data across that same connection. The pattern is consistent: more data movement means more cost.
Loading models shows the exact same pattern. The normal way of loading a large 120 billion parameter model took almost 5 minutes on Blackwell hardware. A smarter loading method cut that down to about 8 seconds. That difference decides whether confidential AI can actually be treated as something ready for real production use, or whether it stays stuck as a research demo. The research also confirmed that confidential workloads can run across multiple GPUs connected together, with very high data transfer speeds between them, over 510 gigabytes per second. That is a real step toward trusting an entire cluster of GPUs together, not just single chips on their own.
Why this matters, and why Phala stands out
Most teams working in confidential AI are focused on simply getting the technology to work at all. This research goes further. It explains exactly why confidential computing is slow in certain situations and shows how to fix the actual cause instead of just accepting the cost as unavoidable. That is a meaningful difference. It is the difference between a research demo and something developers and institutions can actually rely on for real workloads.
This kind of work directly shapes how confidential AI systems should be built going forward, from how workloads get scheduled, to how models get loaded quickly, to how cached data gets handled, to how entire GPU clusters can be trusted together. It is the foundation that will decide whether confidential AI stays a niche option or becomes the normal, expected way that AI models get served.
If you work with AI infrastructure, or you are evaluating confidential computing for your own systems, this research is worth reading directly. The full paper is called “The Serialized Bridge: Understanding and Recovering LLM Serving Performance under Blackwell GPU Confidential Computing,” and it goes much deeper into every result mentioned here.
Find the full paper here: https://arxiv.org/abs/2606.23969
Top comments (0)