Note: This article is Adapted from the official Phala Network blog post: https://phala.com/posts/private-ai-gateway-verified-private-ai-compute
Most people building with AI today assume their prompts are private because they’re using HTTPS. That assumption has a gap. Encrypting the connection protects data in transit, but it doesn’t tell you anything about what happens inside the server handling your request. Private AI Gateway is built to close that gap.
Private AI Gateway is a routing and verification layer for AI inference running inside Trusted Execution Environments. It sits above your model providers and makes sure every request travels a verified path from client to model and back, with proof attached at every step. The result is that developers get a familiar API surface, security teams get something concrete to inspect, and sensitive prompt content stays protected the entire time.
How the verification actually works
When you send a request through Private AI Gateway, the client first checks the gateway’s attestation report. That report confirms which workload is running and exposes the workload’s encryption key. The client then uses that verified key to encrypt sensitive parts of the request before it even leaves the client machine. Only the verified gateway workload holds the matching private key, so nothing in between can read the prompt content. Routing information like billing, model selection, and rate limits stays visible because the gateway still needs to do its job. The sensitive parts stay locked until they reach the verified boundary.
Every request also produces a signed receipt. That receipt records which model route handled the request, what verification was checked before the prompt moved forward, and a chain of hashes from the original request to the final response. It gives security teams an auditable record after the fact rather than a trust assumption made upfront.
Why fragmented compute is a real problem
TEE-based model capacity is spread across many providers right now, and they all handle verification differently. Some bind their TLS key to the attestation. Some expose a separate encryption key. Some rely on GPU-level attestation. If you are routing across multiple providers, you currently have to handle each of these verification styles manually, which is an operational burden that compounds as you add providers.
Private AI Gateway handles that by aggregating providers based on their proof properties. A route only becomes available when it has the right model, acceptable latency, and a verified state. The gateway pre-verifies providers in the background, enforces the right channel binding for each one, and fails the request if the required proof is missing rather than silently routing to an unverified fallback.
What this means for developers, builders, and institutions
The API surface is OpenAI-compatible, so integration does not require rewriting existing code. You add an attestation check and an encryption step, and the rest of your workflow stays the same.
For institutions handling sensitive data, the signed receipts solve a compliance problem that attestation alone cannot. Knowing a workload was verified at boot is useful. Knowing which verified route handled each specific request, with a hash chain to prove it, is what security and legal teams actually need when they review AI usage.
Read the full blog post above from where this article is adapted from.
Top comments (0)