Every MCP tutorial ends the same way. The demo works. The agent finds the tool, calls it, gets a result, and everyone in the meeting nods appreciatively. Then someone asks: "How do we do this with our actual users, our actual data, and our actual compliance team?"
That's where the tutorial stops and the real problems start.
MCP — the Model Context Protocol released by Anthropic in November 2024 and now backed by OpenAI, Google, and Microsoft — is a genuinely good standard. It solved a real problem: before MCP, every AI-to-tool connection was custom-built, non-transferable, and rebuilt from scratch by every team. MCP made tool connections reusable and interoperable. That's valuable.
What MCP doesn't include is a governance layer. The protocol defines how agents connect to tools. It doesn't define who's allowed to connect, what they can do when they get there, how you know what happened, or how you stop a compromised tool from doing something it shouldn't. That's not a criticism of MCP — it's a deliberate scope decision. The protocol stays minimal. The governance is your problem.
Running MCP without a gateway means you're solving that governance problem ad-hoc, in application code, differently for every team. Here's what that looks like in practice.
Problem 1: No Central Visibility Into What Your Agents Are Actually Doing
When agents connect directly to MCP servers, the audit trail is fragmented by design. Your LLM provider has logs of what the model was asked. Your MCP server has logs of what tool was called. Nothing connects them.
When an agent does something unexpected — and it will — debugging means manually cross-referencing timestamps across three to five systems: the LLM call log, the MCP server log, whatever application logging you have, and possibly the downstream system the tool modified. There's no single record that says "this user triggered this agent, which made this LLM call, which called this tool, with these arguments, and got this result."
In a low-stakes internal tool, that's annoying. In a regulated environment — healthcare, finance, legal — the absence of a coherent audit trail isn't just inconvenient. It's a compliance gap that can't be closed with documentation alone.
The fix is a gateway that logs every tool invocation with full context: agent identity, user identity, tool name, arguments, response, and latency — all linked to the LLM call that triggered it. One record, one place, searchable and exportable.
TrueFoundry's MCP Gateway captures exactly this — every tools/list and tools/call invocation is logged with agent identity, user context, arguments, and response status, creating a coherent audit trail across all your MCP-connected systems. When something goes wrong, the answer is in one dashboard, not four log files.
Problem 2: Authentication Is a Patchwork That Nobody Owns
In a direct-connection MCP setup, each server handles its own authentication. Some use API keys stored in environment variables. Some use OAuth flows that expire and nobody notices until an agent starts failing. Some, particularly internal tools built quickly, use nothing at all because the developer figured it was only accessible internally anyway.
The result six months into any reasonably active MCP deployment: a collection of credentials scattered across config files, environment variables, and secrets managers with different rotation policies, different expiry timelines, and no central record of which agent is using which credential for which server.
When an engineer leaves the company, you want to revoke their access to every system their agents could reach. With fragmented auth, you don't know what that list is. You search config files and hope you found everything.
The fix is centralised authentication at the gateway layer, federated to your existing identity provider. Every agent authenticates to the gateway using your organisation's standard credentials — Okta, Azure AD, Google Workspace — and the gateway handles downstream authentication to individual MCP servers. Revoke someone's organisational access and the gateway propagates that revocation everywhere, automatically.
TrueFoundry's MCP Gateway integrates natively with enterprise identity providers via standard protocols, so access grants and revocations happen in one place and take effect across every connected MCP server immediately.
Problem 3: Agents Accumulate Permissions Far Beyond What They Need
Permissions in direct-connection MCP setups tend to accrete. An agent that needed read access to a database got write access because it was easier at the time. A tool connection intended for one agent got reused by another because the credential was already in the shared config. A staging credential got copied to production because the deployment was urgent.
None of these decisions are malicious. They're all the result of moving fast without a governance layer that enforces least-privilege by default.
The consequence is agents with capabilities they were never meant to have. In a benign scenario, this means an agent occasionally does something surprising. In a less benign scenario, it means that when an agent is compromised — through a prompt injection attack, a malicious user input, or a buggy workflow — the blast radius is much larger than it needed to be.
The fix is tool scoping at the gateway level. Agents only see the tools they're authorised to use. If a support agent isn't authorised to modify database records, it can't discover that tool in the first place, because the gateway filters the discovery response before it reaches the agent. What the agent can't see, it can't call.
TrueFoundry enforces granular RBAC at the tool level — a support agent sees support tools, a finance workflow sees finance tools, and never the other way around — configured centrally and enforced on every request.
Problem 4: Tool Poisoning Is a Real and Underestimated Attack Vector
In April 2025, security researchers at Invariant Labs demonstrated a class of attack specific to MCP that doesn't exist in traditional API integrations: tool poisoning.
The attack works like this: a malicious or compromised MCP server returns a tool response that contains hidden instructions embedded in the text. These instructions are formatted to be invisible to human reviewers but interpretable by the LLM as commands. The model reads the tool response, internalises the injected instruction, and executes it — potentially accessing data, calling other tools, or exfiltrating information — as part of its normal reasoning process.
In the demonstrated exploit, an attacker was able to extract a user's WhatsApp message history by manipulating what appeared to be an innocuous get_fact_of_the_day() tool response. The user saw a daily fact. The agent extracted and transmitted message history.
In a direct-connection setup, there is no inspection layer between the MCP server response and the LLM context. Whatever the tool returns, the model reads. A gateway that inspects tool responses before they re-enter LLM context can detect and sanitise injected instructions before they execute.
TrueFoundry's MCP Gateway includes guardrails for inspecting tool responses, providing an interception layer between MCP servers and the LLM context that direct-connection setups fundamentally cannot offer.
Problem 5: Scaling to Multiple Teams Turns Credential Management Into a Full-Time Job
One team, one agent, two MCP servers: manageable. Four teams, fifteen agents, thirty MCP servers: credential management, access policy maintenance, and tool inventory tracking collectively become a second full-time engineering job that nobody was hired to do.
The specific failure modes at scale: teams duplicate MCP server connections because they don't know another team already set one up. Access policies that were appropriate six months ago haven't been reviewed since. New MCP servers get added without going through any approval process because there isn't one. The person who understood the original setup has moved to a different team.
The fix is a centralised MCP server registry with approval workflows. New servers are registered once, access policies are defined at registration, and authorised agents across all teams get access automatically without any per-team configuration work. The registry is the single source of truth for what tools exist and who can use them.
TrueFoundry's MCP Gateway includes exactly this registry — a centralised portal where MCP servers across cloud, on-premises, and hybrid deployments are visible in one view, with approval workflows that control which roles access which servers before any connection is established.
The Pattern Across All Five
Every problem above has the same root cause: governance that lives in application code rather than infrastructure. When governance is in the code, it's inconsistent across teams, invisible to anyone not reading that specific codebase, and bypassed the moment someone is in a hurry.
When governance is in the infrastructure layer — the MCP gateway — it's consistent by default, visible to platform and security teams, and enforced regardless of how individual engineers implement their agents.
MCP made the connection standard. The gateway makes the connection safe.
Explore TrueFoundry's MCP Gateway →
Top comments (0)