Ask two engineers whether their agents need an MCP gateway and you will often get two reasonable-sounding, opposite answers. One says a gateway is overkill—agents can talk to MCP servers directly, and adding a layer just adds latency and operational burden. The other says a gateway is essential—without it, you cannot govern anything. Both are right, for different scales. The honest way to understand the gateway is not as a feature you either have or lack, but as the point where the direct approach stops paying for itself. This article compares connecting to raw MCP servers against routing through a gateway, dimension by dimension, so you can see exactly what changes as you grow and decide where your own project sits.
We will reference an MCP gateway for concreteness, but the comparison is about the two architectures, not a product. The goal is a clear-eyed view of the trade-offs, including an honest account of what the gateway costs.
Starting from the same place
Both approaches begin identically. You have an MCP server exposing some tools, and an agent that wants to use them. In the raw model, the agent is handed the server's address and a credential, and it connects straight through. In the gateway model, the agent is handed the gateway's address, and the gateway connects to the server on its behalf. For a single agent and a single server, these are nearly indistinguishable in behaviour, and the raw model is genuinely simpler—one fewer component, one fewer hop. If that is the entirety of your deployment and it will stay that way, the raw model is a defensible choice.
The interesting question is what happens as you add agents, servers, teams, and compliance obligations. Below, each dimension is examined under both models.
Connection topology
Raw: Each agent connects to each server it needs. The number of relationships scales with agents multiplied by servers. Ten agents sharing twelve servers can mean up to 120 separately configured connections, each with its own credential and failure handling.
Gateway: Each agent connects to the gateway; each server registers with the gateway. Relationships scale with agents plus servers—ten agents and twelve servers is twenty-two connections to the single hub, not 120 point-to-point links. The gateway becomes the hub of a star rather than the many edges of a mesh. This single topological difference is the root cause of most of the others: a sum is governable in ways a product is not.
Authentication and credentials
Raw: Every agent holds credentials for every server it calls. Rotating a key means updating it everywhere it appears. Revoking access when someone leaves means finding every place their access was wired in. There is no single inventory of who can reach what.
Gateway: Agents authenticate once against your identity provider or SSO at the gateway. The gateway brokers upstream credentials—OAuth two-legged and three-legged, API keys, token passthrough—so agents never hold them directly. Rotation and revocation happen in one place. Where a call should carry a specific person's permissions, auth overrides let that user supply their own upstream credentials, giving true per-user access instead of a shared service identity. Credentials move from being scattered secrets to being a managed, central concern.
Access control and policy
Raw: Policy, if it exists, lives inside each agent or each server. There is no central statement of what is allowed, and no way to answer a question like "which agents can call destructive tools?" without inspecting every deployment. Tightening a rule requires touching every place that implements it.
Gateway: Because all calls pass through one interface, access control is expressed and enforced centrally. A default-deny posture means nothing is callable unless explicitly permitted, which bounds exposure as the catalogue grows. Policy engines such as Cedar (used as default-deny access control for MCP tools) and Open Policy Agent (OPA) let you write rules as reviewable policy rather than embedded code. A rule change is one change, applied everywhere.
Discovery and least privilege
Raw: An agent pointed at a server sees whatever tools that server exposes—often far more than it needs. If a database server exposes forty tools and the agent needs three, the other thirty-seven are still reachable. There is also no central catalogue of what servers exist, so teams duplicate work and nobody holds a current inventory.
Gateway: A registry provides a curated, multi-tenant inventory of servers and tools. More importantly, virtual MCP servers let you compose a scoped interface that exposes only the minimum tools a given team or workflow needs—the agent that needs three tools sees three, and never learns the other thirty-seven exist. Discovery and least privilege are delivered by the same mechanism. New tools can be added through OpenAPI-to-MCP conversion (generating a server from an existing API spec, no custom code) or hosted stdio servers (CLI-style tools with managed command, arguments, environment, and credentials), all landing in the same governed catalogue.
Safety inspection
Raw: There is no consistent place to inspect traffic. Each server might implement its own checks, or none. Detecting secrets in outbound arguments, masking personal data in responses, or catching prompt injection has to be rebuilt per server. The agent-specific risk of tool poisoning—a malicious tool description that instructs the agent to misbehave—lands with full force, because the agent reads and tends to trust server-provided descriptions with nothing in the path to check them.
Gateway: Guardrails run pre-tool (before execution) and post-tool (before the result returns) on every call, uniformly. The gateway can block unsafe calls, detect secrets, and mask personal data in one place. Curated discovery shrinks the set of servers that can inject anything into an agent, and in-path inspection can catch the downstream effects of an injection before a call executes. You can even select guardrail policy per request:
curl -X POST "{GATEWAY_BASE_URL}/chat/completions" \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-H 'X-TFY-GUARDRAILS: {"llm_input_guardrails":["my-group/pii-redaction"]}' \
-d '{"model":"openai/gpt-4o","messages":[{"role":"user","content":"..."}]}'
The header references a named policy group the gateway enforces. The exact field name is one platform's convention; the pattern of attaching inspection policy to a request without changing application code is what to remember.
Observability and audit
Raw: Each connection logs independently, if at all. Reconstructing "which agent called which tool, on whose behalf, and what came back" means stitching together logs from many systems, assuming they exist and agree. For an audit or an incident, this is slow and often incomplete.
Gateway: Because every call flows through one interface, the gateway produces a unified record tying each call to an authenticated identity, a policy decision, arguments and results (with sensitive fields masked), and a timestamp. Audit becomes a query, not an excavation—the difference between being able to prove a control and merely claiming it.
An honest accounting of the gateway's cost
A fair comparison admits what the gateway costs. It is another component to deploy and operate. It adds a hop, and therefore some latency, to every call—well-designed gateways keep this small by keeping only the request-handling plane in the path and reading configuration from memory, but it is not zero. It introduces a dependency: if the gateway is down, agents cannot reach their tools, so it must be run with the availability you would give any critical infrastructure. And it requires someone to own policy, the registry, and guardrail configuration—governance is work, even when it is centralised.
These costs are real, and for a genuinely small, static deployment they can outweigh the benefits. The reason the gateway wins at scale is that its costs are roughly fixed while the raw model's costs grow with every connection. One gateway to operate versus 120 connections to secure is not a close call at 120; it is only a close call near the crossover, which for most organisations arrives sooner than expected—usually the moment a second team, a compliance requirement, or a sensitive tool enters the picture.
Choosing where you sit
The practical way to use this comparison is to locate your project on each dimension. If you have one team, a handful of servers, no sensitive data, and no audit obligations, the raw model's simplicity is a feature. The moment any of those change—more teams, more servers, personal or regulated data, an auditor, or agents acting on behalf of real users—the dimensions above start turning against the raw model one at a time, and a gateway stops being overhead and starts being the thing that keeps the estate governable. TrueFoundry's MCP gateway is one example of the architecture described here, offered within a broader control plane and with deployment options spanning SaaS, self-hosted, and air-gapped for data-residency needs; the comparison itself applies whichever implementation you evaluate.
Key takeaways
- Raw MCP servers and a gateway start identically; the difference emerges as agents, servers, teams, and compliance obligations grow.
- The root difference is topology: raw connections scale as agents times servers (a mesh), a gateway as agents plus servers (a star), and most other advantages follow from that.
- Across authentication, policy, discovery, safety inspection, and audit, the gateway centralises what the raw model scatters—one identity, default-deny policy with Cedar and OPA, scoped virtual servers, in-path guardrails, and unified audit.
- The gateway has honest costs: an extra component, some latency, a critical dependency, and governance work to own.
- Those costs are roughly fixed while the raw model's grow per connection, so the gateway wins once a second team, sensitive data, or an audit requirement appears—usually sooner than teams expect.
Top comments (0)