MCP Tool Sprawl: Why 40 Tools Wreck Tool Selection Accuracy

I added a three-tool MCP server to an agent last quarter. The new tools worked. What broke was a tool that had been in the system for months: search_tickets started losing calls to the new query_issues. Nothing about search_tickets had changed — not its schema, not its description, not the prompt around it. The regression was entirely a side effect of a neighbor moving in.

That is the thing people miss about MCP tool sprawl: tool selection accuracy does not degrade uniformly as you add tools. It degrades locally and sharply, in the semantic neighborhood of whatever you just added, while your aggregate eval numbers barely move.

TL;DR

• Tool selection is a discrimination problem over natural-language descriptions, not a lookup over names. Adding a tool splits probability mass with its nearest semantic neighbors, so the tools that regress are the ones that already existed.
• Aggregate "task success" evals hide this. You need a per-tool confusion matrix and an explicit incumbent regression check run before and after every tool-set change.
• Tool descriptions should encode the decision boundary ("use when… do NOT use when… prefer X if…"), not the API contract. Prescriptive trigger conditions measurably raise should-call rate on recent Claude models.
• Above roughly 15–20 tools, stop adding to the always-on set and switch to progressive disclosure: defer_loading: true plus a tool-search tool, so schemas load on demand.
• Tool definitions render at the very front of the prompt prefix. Reordering or editing them invalidates the whole cache — sort deterministically and treat the tool block as frozen.

Why does adding MCP tools reduce tool selection accuracy?

Because the model isn't dispatching on a name — it's picking the maximum-likelihood continuation given a block of prose, and every tool you add is a competing hypothesis.

Concretely: your tool definitions are serialized into the prompt (order is tools → system → messages on the Anthropic API). The model reads N descriptions and emits a structured call. Selection is soft matching over those descriptions in context, so the relevant quantity isn't N — it's the margin between the intended tool's description and its nearest competitor.

That framing predicts the failure pattern I actually observe:

• Adding an unrelated tool (send_slack_message to a code-search agent) costs you almost nothing in accuracy.
• Adding a near-duplicate (query_issues next to search_tickets) can drop the incumbent's recall by double digits while the newcomer looks fine in isolation.

Two tools whose descriptions differ only in vocabulary and not in decision criteria are, from the model's point of view, one tool with a coin flip attached. You didn't add capability. You added noise to an existing decision.

What do tool selection failures actually look like?

Four distinct modes, which want different fixes:

1. Near-synonym collision. search_docs vs query_knowledge_base. Both say "search," both say "returns relevant results." Nothing in either description says which corpus, which freshness, or which one wins when both apply. Fix: name the sibling explicitly inside the description.

2. Scope leakage across a verb pair. get_issue(id) and list_issues(filters?) where list_issues has an optional id. The model can satisfy "show me issue 402" through either. Fix: remove the overlapping parameter, or make one description forbid the case.

3. Right tool, wrong enum. Selection was correct; the model guessed status: "open" where your API wants "OPEN". This is a schema problem masquerading as a selection problem — and it's the one that most often shows up as "the agent is dumb" in bug reports. Fix: enum with the exact literals, and strict: true on the tool definition so inputs are guaranteed to validate.

4. Descriptions auto-generated from OpenAPI. These say what the endpoint does and never say when to call it. They're the single largest source of sprawl damage, because every generated MCP server produces dozens of them at once with a uniform, low-margin writing style.

How should you write a tool description that wins the discrimination?

Write the decision, not the contract. The schema already carries the contract.

// ❌ Auto-generated from OpenAPI — states the contract, not the decision
{
  "name": "query_issues",
  "description": "Queries the issues index and returns matching issue records.",
  "input_schema": {
    "type": "object",
    "properties": {
      "q":      { "type": "string" },
      "id":     { "type": "string" },
      "status": { "type": "string" }
    }
  }
}

// ✅ Encodes the decision boundary and its nearest competitor
{
  "name": "query_issues",
  "description": "Full-text search over ENGINEERING issues (Jira). Call this when the user describes a problem in prose and you do not have an issue key -- e.g. 'the checkout flow 500s on retry'. Do NOT call this when the user gives an issue key like ENG-402; use `get_issue` instead. For CUSTOMER-reported tickets, use `search_tickets` -- this index does not contain them.",
  "input_schema": {
    "type": "object",
    "properties": {
      "q":      { "type": "string", "description": "Natural-language problem description. Not an issue key." },
      "status": { "type": "string", "enum": ["OPEN", "IN_PROGRESS", "CLOSED"] }
    },
    "required": ["q"],
    "additionalProperties": false
  },
  "strict": true
}

Three things changed and each does real work. The description states a trigger condition ("when the user describes a problem in prose"). It states a negative ("do NOT… use get_issue"). And it names the sibling that owns the adjacent case, which is the only way the model can learn a boundary that lives between two descriptions rather than inside one.

The overlapping id parameter is gone — mode 2 fixed structurally rather than by prose. strict: true with additionalProperties: false and a real enum kills mode 3.

This is not just my house style. On recent Claude models, which reach for tools more conservatively than earlier ones, prescriptive "call this when…" descriptions give a measurable lift in should-call rate over descriptions that only state what the tool does. If you're seeing under-triggering after a model upgrade, the description is usually the cheapest lever — cheaper than raising effort, cheaper than restructuring the loop.

How do you measure tool selection accuracy separately from task success?

End-to-end task success is the wrong instrument here. It's noisy, it's slow, and it conflates selection with execution — an agent that picks the wrong tool and recovers on turn three still scores as a pass.

Measure selection directly. Freeze a set of (query → expected_tool) pairs, force a single call, and diff:

import collections, json
import anthropic

client = anthropic.Anthropic()
MODEL = "claude-sonnet-4-6"

# Frozen fixtures. ~10 per tool, including the adversarial near-misses
# that live on the boundary between sibling tools.
CASES = [
    ("the checkout flow 500s on retry",        "query_issues"),
    ("what's the status of ENG-402",           "get_issue"),
    ("customer wrote in about a double charge","search_tickets"),
    # ...
]

def selected_tool(query: str, tools: list[dict]) -> str | None:
    r = client.messages.create(
        model=MODEL,
        max_tokens=1024,
        tools=tools,
        tool_choice={"type": "any", "disable_parallel_tool_use": True},
        messages=[{"role": "user", "content": query}],
    )
    return next((b.name for b in r.content if b.type == "tool_use"), None)

def confusion(tools: list[dict], trials: int = 5) -> dict:
    m = collections.Counter()
    for query, expected in CASES:
        for _ in range(trials):          # sampling is not deterministic; repeat
            m[(expected, selected_tool(query, tools))] += 1
    return m

before = confusion(CURRENT_TOOLS)
after  = confusion(CURRENT_TOOLS + [NEW_TOOL])

# The metric that matters is not aggregate accuracy.
for tool in {e for e, _ in before}:
    b = before[(tool, tool)] / sum(v for (e, _), v in before.items() if e == tool)
    a = after[(tool, tool)]  / sum(v for (e, _), v in after.items()  if e == tool)
    if a < b - 0.05:
        print(f"INCUMBENT REGRESSION  {tool}: {b:.0%} -> {a:.0%}")
        for (e, got), n in after.items():
            if e == tool and got != tool:
                print(f"    stolen by {got}: {n}")

Two design choices carry the weight. tool_choice: {"type": "any"} with disable_parallel_tool_use: true forces exactly one call, so you're measuring discrimination and not the model's willingness to act. And repeating each case means you're estimating a distribution, not sampling one draw — a 1-in-5 misroute is a real production defect and a single-shot eval will miss it four times out of five.

The INCUMBENT REGRESSION gate is the whole point. Aggregate accuracy across a 30-tool set can hold at 91% while one tool collapses from 94% to 61%, because the newcomer's own good numbers paper over the hole. Gate the tool-set change on per-tool deltas, and make the confusion counts visible so you can see which tool is doing the stealing — that tells you which two descriptions need a boundary drawn between them.

Does progressive tool disclosure fix it?

Past a certain size, yes — and it's the only fix that scales, because prose-level disambiguation gets harder combinatorially as the tool set grows.

The Anthropic API supports this natively: mark tools defer_loading: true and add a tool-search tool. Deferred tools are known to the request but their schemas don't enter the model's context until search surfaces them.

tools = [
    {"type": "tool_search_tool_regex_20251119", "name": "tool_search_tool_regex"},

    # Hot path: always loaded, small, mutually well-separated.
    {"name": "read_file",  "description": "...", "input_schema": {...}},
    {"name": "run_tests",  "description": "...", "input_schema": {...}},

    # Long tail: schemas load on demand.
    {"name": "query_issues",   "description": "...", "input_schema": {...},
     "defer_loading": True},
    {"name": "search_tickets", "description": "...", "input_schema": {...},
     "defer_loading": True},
    # ...30 more
]

One hard constraint: the search tool itself must never be deferred, and at least one tool must stay non-deferred, or the request fails with a 400 (All tools have defer_loading set). Keep a small always-on core — the tools used on nearly every turn — and defer the long tail.

The trade-off is a round trip on turns that need a deferred tool, plus a new failure mode: search misses. You've moved the discrimination problem from "40 descriptions in context" to "a retrieval query over 40 descriptions," which is easier but not free. Keep the eval harness above pointed at the deferred set specifically.

There's a caching wrinkle worth knowing. Tool definitions render at position zero of the prompt prefix, so adding, removing, or reordering a tool invalidates the entire prompt cache — tools, system, and messages. Tool search is well-behaved here because discovered schemas are appended rather than swapped in, preserving the prefix. Roll your own dynamic tool set and you'll pay full price on every turn where it changes. Sort your tool list deterministically and treat it as frozen for the conversation's lifetime.

The short answer

Adding MCP tools wrecks tool selection accuracy because the model discriminates over descriptions, not names — so each new tool competes directly with its nearest semantic neighbors and quietly steals their calls, while your aggregate metrics stay flat. The fix is three-layered: write descriptions that encode the decision boundary and name the sibling that owns the adjacent case; gate every tool-set change on a per-tool confusion matrix with an explicit incumbent-regression check; and once the always-on set passes roughly 15–20 tools, move the long tail behind defer_loading and a tool-search tool so schemas load on demand instead of competing for attention on every single turn.