Case Study: Progressive MCP Tool Routing Cut Agent Hallucinations by 40%

When a 47-tool MCP ecosystem forced 50K context tokens per query, hallucinations and latency soared. Discover how implementing progressive disclosure and semantic tool search reduced costs by 60% and errors by 40%.

The Problem: A Drowning Agent in a Sea of Tool Schemas

Imagine your AI agent is a skilled technician, but you've thrown every instruction manual for every tool in your warehouse onto the floor around them. That was our starting point. Our internal developer platform had integrated 47 powerful Model Context Protocol (MCP) tools for code analysis, deployment, and security scanning. The initial implementation was straightforward: dump every tool's JSON schema into the system prompt for every single user query.

The result was catastrophic. For a simple query like "Find SQL injection vulnerabilities in our auth service," the agent received a context payload exceeding 50,000 tokens—including schemas for container orchestration, cost optimization, and frontend asset bundling tools. The primary failure modes were stark: a 38% hallucination rate where the agent confidently referenced tool parameters that didn't exist, and an average end-to-end latency of 45 seconds for what should have been a 10-second task.

Case Study: The 47-Tool Enterprise Setup

Our subject was a platform serving 200+ engineers. The MCP tools were categorized but always presented flatly. The agent's context window became a bottleneck, not an asset. We logged a 30-day period with the full-schema approach and established a baseline:

• Average Prompt Token Count: 52,400 tokens
• Hallucination/Tool Misuse Rate: 38% (flagged by post-execution validation)
• Cost per Query (GPT-4 API): ~$0.78
• Successful Task Completion Rate: 61%

The core issue was agent context optimization. The model was wasting critical attention capacity filtering irrelevant tool information, degrading its reasoning on the actual task.

Implementing Progressive Disclosure & Semantic Search

We replaced the flat dump with a two-tier routing system built on the principle of progressive disclosure. The agent first interacts with a lightweight "router" that uses semantic tool search to fetch only the necessary tool schemas.

Step 1: The Router & Semantic Index. We pre-processed all 47 tool schemas, generating vector embeddings for each tool's description and primary use cases. The router is a smaller, faster model prompted to classify the user's intent and map it to relevant tool categories (e.g., `security_scan`, `code_quality`, `infra_manage`).

# Example: Router's simplified tool schema selection logic
user_query = "Deploy the auth service to staging with canary"
intent = router.classify(user_query) 
# Returns: {"primary_intent": "deployment", "secondary_intent": "traffic_management"}

# Semantic search only within the relevant categories
relevant_tools = vector_db.semantic_search(
    query_embedding=user_query_embedding,
    filter_categories=[intent.primary_intent, intent.secondary_intent],
    top_k=3  # Fetch only the top 3 most relevant tools
)

Step 2: Dynamic Context Assembly. The agent's main context is then constructed with only the user query, a brief system preamble, and the schemas for the 3-4 dynamically selected tools. For our SQL injection query, it might receive schemas for `static_code_analyzer`, `dependency_checker`, and `report_formatter`—completely omitting the 44 irrelevant tools.

Tool Routing in Action: From 50K to 8K Tokens

The transformation in the agent's workflow was immediate. Instead of a chaotic firehose of information, it receives a curated toolkit. The MCP tool routing layer acts as a librarian, not a megaphone. Let's examine the before-and-after context for a deployment query:

Before (Full Schema): Context included schemas for `kubernetes_deploy`, `terraform_apply`, `aws_lambda_update`, `frontend_bundle`, `database_migration`, and 42 others.
After (Progressive Routing): Context includes schemas for `kubernetes_deploy`, `load_balancer_update`, and `rollback_manager`—the precise tools needed for a canary release.

This isn't just about token savings. The semantic search ensures the agent gets the *right* tools, not just *fewer* tools. The routing logic uses the tool's semantic embeddings to find matches based on conceptual similarity to the user's task, not just keyword overlap.

Results: A 40% Hallucination Reduction and $150K Annual Savings

After a 4-week A/B test with 20 engineers on each approach, the metrics spoke for themselves. Progressive tool routing dramatically improved reliability and efficiency:

• Hallucination/Tool Misuse Rate: Dropped from 38% to 22%—a 42% relative reduction. The agent stopped inventing parameters for tools it never saw.
• Average Prompt Token Count: Slashed from 52,400 to 8,200 tokens per query—a 84% reduction.
• Cost per Query: Reduced to $0.31, a 60% decrease.
• Task Completion Rate: Rose from 61% to 89%.

Extrapolated across the platform, this optimization projected an annual API cost saving of over $150,000. More importantly, developer trust in the AI assistant increased, leading to 3x higher adoption rates.

Your Implementation Roadmap

Adopting progressive routing doesn't require rebuilding your MCP stack. Start here:

1. Categorize & Vectorize: Tag your tools with semantic categories and generate embeddings for their schemas.
2. Build a Router: Use a fast classifier (like a fine-tuned LLM or even a simple embeddings similarity model) to map user intents to categories.
3. Implement Tiered Context: First call the router, then dynamically assemble the agent's prompt with only the relevant tool schemas.
4. Log & Iterate: Monitor which tools are selected and whether task completion improves. Refine your categories and embeddings as you add new tools.

Ready to stop drowning your agents and unlock the full potential of your MCP toolkit? Discover how TormentNexus provides built-in intelligent tool routing and context optimization out of the box. Explore TormentNexus now.

---

Originally published at tormentnexus.site