Your MCP server doesn't need to put all its tools in the model's context

I built an MCP server that lets an AI agent use my own WhatsApp: search chats, send messages, transcribe voice notes, all self-hosted so my message history never leaves my box. But the most interesting problem turned out to have nothing to do with WhatsApp. It was the number of tools.

The tax nobody mentions

MCP injects every tool definition into the model's context on every request, whether or not the request uses them. My server grew to 96 tools. That's roughly 20k tokens sitting in context before the model has done anything.

The token cost was the annoying part, but not the real one. When I handed the model all 96 tools at once, it got worse at picking the right one, not better. More options, more room to be wrong.

Progressive disclosure

The obvious move, dropping tools, loses capability. So instead: serve a small hot core directly, and keep the long tail reachable through search.

The model now sees 29 core tools plus two meta-tools:

find_tool(query) ranks the full 96-tool library and returns names, descriptions, and parameter signatures.
call_tool(name, arguments) dispatches into any of them.
Always-on cost drops from ~20k to ~8k tokens, and nothing is removed, everything is one search away. Retrieval is lexical by default (IDF + stemming + a synonym map) and blends in embeddings if a provider key is present.

The part that will bite you

call_tool dispatches inside the process, past the middleware chain that normally runs on a tool call. So it has to re-apply what that chain would have done: per-tool scope enforcement and audit logging. Skip that and your retrieval layer quietly becomes a scope-bypass and an audit hole. A tool router that skips your authorization is worse than no router.

Measure it, don't vibe it

"It feels better" isn't a number. There's a small labeled eval (natural-language task to gold tool, deterministic labels) that scores whether retrieval lands on the right tool. Lexical caps around 75% recall@8 on adversarial phrasing; hybrid pushes it higher. The eval is small and mine, so treat it as a direction, not a proof.

Takeaway

If you're wiring an LLM into a big API, the tool list is a first-class design problem, not an afterthought. Serve a core, make the rest searchable, mirror your auth into the dispatch path, and put a number on the retrieval.

Code, the eval harness, and the design notes: https://github.com/HalemoGPA/whatsapp-mcp-server

Comments

[Mads Hansen]

Progressive disclosure is a strong pattern, but it turns call_tool into the most privileged tool in the server. Its contract is effectively “invoke anything by name,” so mirroring middleware is necessary but still easy to regress.

One design I like is making find_tool return short-lived, opaque capability handles instead of treating the raw tool name as authority. Bind each handle to caller/tenant, allowed tool, schema digest, policy version, and expiry. Then call_tool accepts the handle plus arguments, re-authorizes at execution time, validates against that exact schema, and emits one audit chain covering search → selection → dispatch → outcome.

The eval should exercise the joint system, not retrieval alone: recall@k, final selection accuracy, argument validity, same-name/near-name confusion, stale-index behavior after schema changes, and attempts to dispatch a tool that was never returned or is unauthorized for that caller. Otherwise good recall can coexist with a universal dispatch escape hatch.