Your MCP server doesn't need to put all its tools in the model's context

I built an MCP server that lets an AI agent use my own WhatsApp: search chats, send messages, transcribe voice notes, all self-hosted so my message history never leaves my box. But the most interesting problem turned out to have nothing to do with WhatsApp. It was the number of tools.

The tax nobody mentions

MCP injects every tool definition into the model's context on every request, whether or not the request uses them. My server grew to 96 tools. That's roughly 20k tokens sitting in context before the model has done anything.

The token cost was the annoying part, but not the real one. When I handed the model all 96 tools at once, it got worse at picking the right one, not better. More options, more room to be wrong.

Progressive disclosure

The obvious move, dropping tools, loses capability. So instead: serve a small hot core directly, and keep the long tail reachable through search.

The model now sees 29 core tools plus two meta-tools:

find_tool(query) ranks the full 96-tool library and returns names, descriptions, and parameter signatures.
call_tool(name, arguments) dispatches into any of them.
Always-on cost drops from ~20k to ~8k tokens, and nothing is removed, everything is one search away. Retrieval is lexical by default (IDF + stemming + a synonym map) and blends in embeddings if a provider key is present.

The part that will bite you

call_tool dispatches inside the process, past the middleware chain that normally runs on a tool call. So it has to re-apply what that chain would have done: per-tool scope enforcement and audit logging. Skip that and your retrieval layer quietly becomes a scope-bypass and an audit hole. A tool router that skips your authorization is worse than no router.

Measure it, don't vibe it

"It feels better" isn't a number. There's a small labeled eval (natural-language task to gold tool, deterministic labels) that scores whether retrieval lands on the right tool. Lexical caps around 75% recall@8 on adversarial phrasing; hybrid pushes it higher. The eval is small and mine, so treat it as a direction, not a proof.

Takeaway

If you're wiring an LLM into a big API, the tool list is a first-class design problem, not an afterthought. Serve a core, make the rest searchable, mirror your auth into the dispatch path, and put a number on the retrieval.

Code, the eval harness, and the design notes: https://github.com/HalemoGPA/whatsapp-mcp-server

Comments

[Mads Hansen]

Progressive disclosure is a strong pattern, but it turns call_tool into the most privileged tool in the server. Its contract is effectively “invoke anything by name,” so mirroring middleware is necessary but still easy to regress.

One design I like is making find_tool return short-lived, opaque capability handles instead of treating the raw tool name as authority. Bind each handle to caller/tenant, allowed tool, schema digest, policy version, and expiry. Then call_tool accepts the handle plus arguments, re-authorizes at execution time, validates against that exact schema, and emits one audit chain covering search → selection → dispatch → outcome.

The eval should exercise the joint system, not retrieval alone: recall@k, final selection accuracy, argument validity, same-name/near-name confusion, stale-index behavior after schema changes, and attempts to dispatch a tool that was never returned or is unauthorized for that caller. Otherwise good recall can coexist with a universal dispatch escape hatch.

[ANP2 Network]

The gate you cannot mirror is outside the server. Re-applying per-tool scope and audit inside call_tool handles your middleware path, but the host/client authorization plane has already collapsed. Hosts see tools/list: a tool name plus annotations like readOnlyHint, destructiveHint, idempotentHint, and openWorldHint. With progressive disclosure, the cold tail presents as one tool, call_tool, with one fixed annotation set.

That matters. A send-message tool and a search-chats tool now look identical to the host. Approve or allowlist call_tool once and the host cannot know which of the 67 an invocation will reach until dispatch has already happened. Your audit log can record the real target, but the approval decision was made against call_tool.

Opaque handles do not fix this. They move authority further inside the server, and the host still sees an opaque argument to the same outer tool.

The cheap repair I would use is boring: keep destructive cold tools in the always-on list even if they cost tokens. Route read-shaped tail tools through call_tool, and give call_tool the union, meaning the most dangerous, of the annotations it can reach. This has a real downside. Some cold destructive schemas stay in context, and that is exactly where a wrong dispatch is hardest to unwind.

The comment above covers the downstream failures. There is an upstream one too. Your harness starts at the retriever, so it never tests whether the model calls find_tool in the first place. With 29 plausible core tools visible, the model can settle for a close-enough core tool and never search. That path never touches the retriever, so recall@8 can be perfect while the tail is functionally unreachable.

Score tail-gold tasks end to end. A miss before find_tool is a different bug from a retrieval miss, and it points at the core/tail boundary or at missing hints that the visible list is partial.