Originally published at webofmike.com on 2026-09-15. The demo repo and every command in it were run before publishing.
I have some version of this conversation just about every week. A platform team is rolling out MCP across the company, and the plan looks the same every time: stand up one federated endpoint, wire every MCP server they can find behind it, and give every agent the whole catalog. One URL, a hundred-plus tools, call it done.
And honestly, I get it. One connection to manage, one place to do auth, one thing to point the agents at. If you spent the last decade doing API gateways, this feels like the obvious move.
But then it ships, and the same two problems show up. End users look at the tool list and ask "which of these three get_account tools am I supposed to use?" And the agents ask the same question, except their version of confusion is wrong tool calls, retries, and a token bill that grows with every tool you add to the pile.
I want to walk through why I think this happens, why the fixes people reach for first (Code Mode, tool search, semantic routing) don't actually fix it, and what I'd do instead. I also built a working example of the alternative, written up in Multi-Tenant MCP Federation with agentgateway. This post is the argument; that one is the proof.
More tools make your agent worse
Let's start with the part that's measurable, because it's the part that surprises people.
Anthropic published numbers on this in their advanced tool use post. On their internal MCP evals, Claude Opus 4 scored 49% when the full tool catalog was loaded into context. Same model, same tasks, with on-demand tool discovery instead: 74%. They also found real-world tool catalogs eating around 77K tokens of context before the agent had done any work at all.
Sit with that first number for a second. A frontier model failing half its tasks, and the variable wasn't the model or the tasks. It was how many tool definitions we made it read first.
This is the thing that has no analog in the microservices world. A service with 400 downstream APIs available to it doesn't get dumber because they exist. An agent does, because tool selection is a reasoning problem, and every tool in the list is another thing to reason about. When someone tells me they're putting 100+ tools behind one endpoint, this is the first thing I bring up, because it's the easiest to demonstrate and the hardest to argue with.
Haven't we been here before?
The token numbers get attention, but I don't think they're the deep problem. The deep problem is that we've built this system before and it had a name: the enterprise service bus.
The mega MCP endpoint has the same failure mode the ESB had. Some platform team ends up owning the aggregate. Every domain team's change has to route through them. The integration layer that was supposed to speed everyone up becomes the queue everyone waits in. We spent years unwinding that mess, and "smart endpoints, dumb pipes" became a mantra for a reason.
There's a twist this time though, and it's the part I keep coming back to. Conway's law says organizations produce designs that mirror their communication structures. With REST APIs, that leakage showed up in URL paths and payload shapes. Annoying, but survivable, because the caller was code and code doesn't care if your API vocabulary is weird.
With MCP, the interface contract is prose. Tool names and descriptions are literally what the model reasons over when it decides what to call. So whoever writes those descriptions leaks their vocabulary, their org structure, their view of the world, directly into the agent's decision-making. When the platform team bulk-generates tool descriptions from scraped OpenAPI specs, the agent gets infrastructure vocabulary where it needed business vocabulary. And then we wonder why it picks the wrong tool.
Said another way: your org chart used to shape your architecture. Now it shapes your agent's judgment. That's new.
(Christian Posta has written about how agents break the assumptions we carried over from microservices. This is another one of those assumptions: with agents, interface design and reasoning quality are the same concern.)
Doesn't Code Mode fix this? Tool search? Semantic routing?
This is usually the pushback I get, so let's take it seriously.
Tool search lazy-loads tool definitions instead of front-loading the whole catalog, and it works. That's where Anthropic's 49% to 74% jump came from. Code Mode goes further: instead of exposing N tools to the model, the gateway exposes a script executor, the model writes a bit of code that calls the underlying tools as functions, and only the final result comes back. Agentgateway supports both today as a config knob on the backend: toolMode: Standard | Code | Search | CodeSearch.
I like these features. They solve a real problem. But notice what they solve: context economics. The tokens. What they don't touch is everything else that's wrong with the pile. Nobody owns it. get_account still means three different things because three teams meant three different things. There's still no boundary to hang an authorization policy on, no per-domain cost attribution, and one blast radius when a credential leaks or a server misbehaves.
Code Mode arguably raises the stakes. When the model writes a script that fans out across a dozen tools inside a sandbox, those individual calls never round-trip through the model where a human or a supervisor might notice something off. The gateway enforcing policy on every call inside that loop isn't one control point among several anymore. It's the only one left.
Semantic routing is the strongest version of this pushback, so it deserves its own beat. The idea: don't make the model read anything up front. Embed the request, retrieve the handful of tools that semantically match, route to those. That attacks selection accuracy and tokens at the same time, and tool search is a narrow implementation of the same idea. But ask what the retrieval actually runs over: tool names and descriptions. If those are autogenerated platform vocabulary, your embedding space is exactly as confused as your agent was, and now the confusion is buried a layer deeper where it's harder to debug. Semantic routing isn't an alternative to domain boundaries. It's a downstream consumer of them. The teams getting good retrieval results are the ones whose descriptions were written by the people who own the domain.
There's a governance version of this too: you can't hang an authorization policy on cosine similarity. "The embedding picked it" is not a sentence anyone wants in an audit report. Retrieval selects among the tools a caller is allowed to see. Policy decides what that set is. And policy needs a boundary to be written against.
So: use tool search, use Code Mode, put semantic routing in front of big catalogs. Just don't confuse a retrieval strategy for an architecture.
Tool boundaries are business domains
Here's the thing the microservices era actually taught us, once we got past the "how small is a microservice" arguments: the boundaries that hold up are business domain boundaries. Domain-driven design, bounded contexts, teams owning their interfaces. That's the part worth carrying forward, and it maps onto MCP almost one to one.
An MCP server is a bounded context. Billing, support, inventory. Not "the database tools" or "the stuff the platform team wrapped."
Tool descriptions are the ubiquitous language of that context, which means the domain team should write them, in the domain's vocabulary. In my experience this is the single highest-impact thing a team can do for agent accuracy, and it's a product responsibility. Not something to autogenerate and forget.
And those get_account collisions? That's not a namespacing bug you fix with prefixes. That's the architecture telling you "account" means different things in billing and support and CRM. DDD practitioners have been using that exact signal to find context boundaries for twenty years. The mega-endpoint hides the one clue you most needed to see.
If you followed the data mesh conversation, this should feel familiar: domain-owned products, self-serve platform, federated governance. Same idea, new interface surface.
OK, but I still want one endpoint
Right. So did the mega-endpoint folks, and they were correct to want it. Agents shouldn't be juggling fifteen connections and fifteen credential flows. The trick is realizing that one endpoint and one flat namespace are separable. Federate the transport. Keep the domains.
This is what virtual MCP servers in agentgateway do. One backend, multiple MCP targets, each target a domain owned by its team:
apiVersion: agentgateway.dev/v1alpha1
kind: AgentgatewayBackend
metadata:
name: mcp-federated
namespace: agentgateway-system
spec:
mcp:
targets:
- name: billing
selector:
services:
matchLabels:
domain: billing
- name: support
selector:
services:
matchLabels:
domain: support
- name: inventory
static:
host: inventory-mcp.inventory.svc.cluster.local
port: 80
Agents connect once. But the domain boundary survives inside the gateway, and that turns out to be the thing everything else hangs on.
Because now you can write policy against it. The gateway validates the caller's JWT and then applies CEL rules that pare down what each caller can see, per tool or per entire domain, based on claims:
apiVersion: agentgateway.dev/v1alpha1
kind: AgentgatewayPolicy
metadata:
name: domain-access
namespace: agentgateway-system
spec:
targetRefs:
- group: agentgateway.dev
kind: AgentgatewayBackend
name: mcp-federated
backend:
mcp:
authorization:
action: Allow
policy:
matchExpressions:
# support agents see the support domain
- 'mcp.tool.target == "support" && "support" in jwt.groups'
# plus one specific billing tool they legitimately need
- 'mcp.tool.name == "billing_get_invoice" && "support" in jwt.groups'
# billing agents get their own domain
- 'mcp.tool.target == "billing" && "billing" in jwt.groups'
The part I'd draw your attention to is what this does to tools/list. A support agent hitting the federated endpoint doesn't get 400 tools with most of them off-limits. It gets the support domain plus exactly one billing tool, because that's what its identity permits. We started this post with two problems, the human asking "which tool?" and the agent spending context on tools it will never call. This is one mechanism solving both, and it only works because the domain boundary exists for the policy to reference.
Identity is the quieter issue underneath all of this. Mega-endpoints drift toward a single service token carrying the union of every scope in the company, because that's the path of least resistance when there's one pipe. With domain boundaries and token exchange at the gateway (the on-behalf-of pattern), the agent's effective permission becomes the intersection of the user, the agent, and the domain policy. And since every call crosses the gateway, you get the audit trail your governance folks have been asking for: which business capability, which agent, on whose behalf, at what cost. Per domain, not per pile.
If you want to see all of this running rather than argued, that's exactly what the multi-tenant federation example builds: three business domains, three customers on three identity providers, per-customer entitlements, quotas, and chargeback, with zero auth code in the MCP servers themselves.
Where I'd start
Not with fifteen domains and a steering committee. Two or three domains, the ones where you already know who owns what. Then let the friction teach you: every naming collision and every "wait, who owns this tool?" argument is the architecture showing you where the next boundary is. Those arguments are better consultants than any target-state diagram.
The bigger point is that agent architecture is still young enough for the inverse Conway maneuver. Your MCP topology is going to mirror your org structure whether you plan for it or not. Right now, before the mega-endpoint becomes infrastructure everyone depends on, you get to pick the boundaries and let ownership follow them.
What you end up with isn't one giant endpoint, and it isn't an unmanaged sprawl of servers either. It's a mesh of domain-owned tool surfaces behind a governed gateway. An agentic mesh, with the same shape that made service mesh work: domains own their interfaces, the platform owns the substrate, and policy lives at the boundaries.
Conway's law says your systems will end up with boundaries one way or another. The only real question is whether you chose them.
Want to try this? Start with the working example repo, then the virtual MCP, tool mode, MCP auth, and CEL authorization docs.
Frequently asked questions
Why not put every MCP tool behind one federated endpoint?
Because more tools measurably degrade the agent: on Anthropic's internal MCP evals, Claude Opus 4 scored 49% with the full tool catalog in context versus 74% with on-demand discovery, and real catalogs burned about 77K tokens before any work happened. A flat namespace also has no owner and no boundaries, so there is nothing to attach authorization, quotas, or cost attribution to.
Don't Code Mode, tool search, or semantic routing fix the mega-endpoint problem?
They fix context economics, not architecture. Tool search and Code Mode cut tokens, but nobody owns the pile, name collisions remain, and there is still no boundary to hang policy on. Semantic routing retrieves over tool names and descriptions, so autogenerated platform vocabulary makes the embedding space exactly as confused as the agent was, and you cannot hang an authorization policy on cosine similarity.
How should MCP federation be structured instead?
By business domain. An MCP server is a bounded context (billing, support, inventory), and its tool descriptions are the domain's ubiquitous language, written by the team that owns it. Federate the transport so agents connect once, but keep domain boundaries inside the gateway, where identity-based policy filters each caller's tool list per domain.
Canonical version, with machine-readable markdown at https://webofmike.com/conways-law-mcp-servers/index.md: https://webofmike.com/conways-law-mcp-servers/
Top comments (0)