Progressive MCP Tool Routing: How We Stopped Drowning Agents in 50K Tokens and Cut Hallucinations by 40%

Discover how progressive tool routing transforms agent performance in complex MCP setups. This case study details a real-world implementation that reduced token bloat by 89% and agent hallucinations by 40% using semantic search and staged disclosure.

The 47-Tool Cliff: When Your Agent Drowns in Capability

We recently engaged with a financial services client facing a critical bottleneck. Their investment analysis agent, powered by a sophisticated multi-agent system, had access to an MCP (Model Context Protocol) server exposing 47 distinct tools. These ranged from real-time market data APIs and portfolio analytics to document generators and compliance checkers. The problem? To even consider making a decision, the LLM had to ingest a system prompt detailing *every* tool's schema, parameters, and usage examples. The resulting context ballooned to over 50,000 tokens for every single interaction.

The consequences were severe. The agent's latency was unacceptable, costs per query were prohibitive, and most damagingly, it frequently hallucinated. It would "remember" tools that didn't exist for a given task or misapply parameters, leading to confidently wrong analyses. The root cause was clear: the agent was suffering from severe agent context optimization failure. By presenting the entire tool universe upfront, we were violating the principle of progressive disclosure, forcing the LLM to navigate an irrelevant cognitive landscape.

The Failed Naive Approach: Keyword Search and Static Lists

Initial attempts to fix this relied on a static, category-based tool filtering system. A mapping file grouped the 47 tools into domains like "Market_Data," "Analytics," and "Document_Prep." The agent would first predict a category, then receive a condensed list of tools within it. This helped slightly, but it was brittle. The LLM often miscategorized intent—for instance, a query about "portfolio risk under inflation" might get sent to the general "Market_Data" bucket instead of the specialized "InflationScenarioAnalytics" tool. The system remained rigid, with no understanding of the semantic relationship between a user's intent and a tool's purpose.

Implementing Semantic Tool Search: The Core of Progressive Routing

The breakthrough came from treating tool selection not as a lookup, but as a semantic tool search problem. We built a lightweight indexing service that, during MCP server startup, parsed every tool's OpenAPI schema (description, parameters, examples) and generated a dense vector embedding for it. These embeddings were stored in a vector database, creating a semantic map of our entire 47-tool toolkit.

Now, when a user query enters the system, it isn't compared against a keyword list. Instead, it's embedded and used to perform a nearest-neighbor search against the tool embeddings. This is the heart of progressive MCP tool routing. Here’s a simplified configuration from our TormentNexus-inspired orchestrator:

// Progressive Router Configuration
{
  "routing_strategy": "semantic_progressive",
  "tool_index": {
    "source": "mcp://financial-tools-server",
    "embedding_model": "text-embedding-3-small",
    "retrieval_k": 8,  // Initial candidate set
    "reranking_model": "cross-encoder/ms-marco-MiniLM-L-6-v2"
  },
  "disclosure_stages": [
    {
      "stage": "discovery",
      "token_budget": 1500,
      "description": "Top 3 semantically relevant tools with summaries only."
    },
    {
      "stage": "detail",
      "token_budget": 8000,
      "description": "Full schemas for the selected 3 tools, with usage examples."
    },
    {
      "stage": "execution",
      "token_budget": 2000,
      "description": "Tool execution result and optional follow-up tool schemas."
    }
  ]
}

From 50K Tokens to Strategic Disclosure: A Technical Walkthrough

The progressive router now intercepts every request. For the query "Analyze the tech sector's exposure to the upcoming interest rate hike," the system first embeds this phrase. The vector search identifies tools like "InterestRateSensitivityAnalysis," "SectorExposureDashboard," and "HistoricalRateImpact" as the top candidates. Instead of injecting all 47 schemas, the router enters the **discovery** stage. It builds a concise, 1,500-token prompt containing only a one-sentence summary of these three tools:

# Initial Context Injection (Discovery Stage)
You have access to financial analysis tools. Based on the user's query about interest rates and the tech sector, the following tools appear most relevant:

1. **InterestRateSensitivityAnalysis**: Quantifies portfolio sensitivity to benchmark rate changes.
2. **SectorExposureDashboard**: Breaks down portfolio holdings by GICS sector with risk metrics.
3. **HistoricalRateImpact**: Retrieves historical data on sector performance during past rate cycles.

Which tool(s) would you like to use? Request their full schema with the command: `get_tool_schema("tool_name")`.

The LLM, now facing a focused, relevant choice, reliably selects the right tool. It then requests the full schema, which the router provides in the **detail** stage. This staged disclosure ensures the LLM's context window is always optimized for the immediate decision, preventing information overload and reducing the probability of hallucinated tool usage.

Measurable Outcomes: 40% Fewer Hallucinations and 89% Less Bloat

The results after deploying the progressive routing system were transformative. We measured key metrics over 10,000 production queries:

• Context Size Reduction: Average token count per interaction dropped from 50,120 to 5,480—an 89% reduction in bloat.
• Hallucination Rate: Instances where the agent referenced non-existent tools or malformed parameters fell by 40%, from 12.1% to 7.3% of interactions.
• First-Tool Accuracy: The percentage of queries where the agent correctly selected the optimal tool on its first attempt increased from 64% to 91%.
• Latency & Cost: Average response time decreased by 58%, and API costs per query dropped by approximately 65%.

The core insight is that progressive disclosure is not merely a convenience—it's a fundamental requirement for scalable agent architecture. By leveraging semantic tool search to implement intelligent, staged context loading, we align the agent's cognitive load with its decision-making task. This is the future of effective agent context optimization.

Architecting for Scale: Your Next Steps

Implementing a system like this requires careful engineering: a robust vector indexing pipeline, a stateful routing layer, and a deep understanding of your MCP tool semantics. The payoff, however, is an agent that performs with clarity and precision even as your toolset grows to hundreds or thousands of capabilities. It stops being a blunt instrument and becomes a scalpel.

If your agents are struggling with tool sprawl, context fatigue, and costly hallucinations, it's time to rearchitect your routing layer. Move beyond static lists and keyword matching. Embrace a dynamic, semantic approach that grants your agent intelligence by strategic restraint.

Ready to implement progressive tool routing and optimize your agent's performance? Learn how the TormentNexus platform provides the foundational services for semantic indexing and dynamic context management at https://tormentnexus.site.

---

Originally published at tormentnexus.site