Once a conversational system gets wired up to more than one external tool, database lookup, calendar booking, payment processing, a new failure category appears that has nothing to do with whether any individual tool works correctly. The model has to first decide which tool a given request actually calls for, and that routing decision, often treated as trivial during development, turns out to be one of the more fragile parts of the entire system once real, slightly ambiguous user requests start arriving.
This category sits under what is generally discussed as tool calling or function calling in modern model architectures, where a model is given a set of available tools, each with a name, a description, and a defined input schema, and has to select the correct one based on the user's message before anything else happens. Most tool calling demonstrations use requests where exactly one tool is obviously correct, book an appointment clearly calls the booking tool, check my balance clearly calls the account lookup tool. Real conversations are rarely that clean, and the routing failures that matter happen specifically in the space between two tools that both look plausible for a given request.
Where Tool Selection Actually Breaks
A concrete version of this shows up constantly in systems that have both a general knowledge base search tool and a more specific structured lookup tool, for example a general FAQ retrieval tool alongside a dedicated order status tool. A user asking where is my order sits squarely in the order status tool's territory. A user asking what's your return policy for damaged items sits more ambiguously between the two, since it could reasonably be treated as a general knowledge question the FAQ tool should handle, or as something specific enough to warrant a more targeted lookup if the system happens to have one. Without explicit routing guidance, the model's choice between the two can be genuinely inconsistent, sometimes reaching for one, sometimes the other, on functionally identical requests phrased slightly differently.
The consequence of picking the wrong tool is rarely a dramatic failure, which is part of why this problem persists quietly in production longer than more obvious bugs. The wrong tool often still returns something, general information instead of the specific structured answer the user actually needed, or a structured lookup triggered for a question that really called for broader contextual explanation instead. The response is not wrong exactly, it is just narrower or less useful than what the correct tool would have produced, and that kind of subtly degraded output rarely generates the same clear error signal a complete failure would, making it hard to catch through normal monitoring.
A second, more consequential version of this involves tools that have real side effects, a booking action, a payment trigger, a database write. Ambiguous routing between a read only tool and a write capable tool is a meaningfully higher stakes problem than ambiguous routing between two read only tools, because a wrongly triggered write action cannot simply be quietly corrected the way a wrongly retrieved piece of information can. A user asking something like can I move my appointment to Thursday sits in genuinely ambiguous territory between a tool that checks availability and a tool that actually reschedules the booking, and a system that resolves that ambiguity by defaulting toward the more consequential action, rather than confirming intent first, creates exactly the kind of failure that erodes trust fastest.
Why Tool Descriptions Alone Rarely Solve This
The most common first attempt at fixing ambiguous routing is simply writing more detailed descriptions for each tool, expanding the explanation of what each one does and when it should be used. This genuinely helps to a point, but description quality alone tends to plateau against a specific class of problem, which is requests that sit in the actual overlap zone between two tools' legitimate use cases rather than clearly belonging to either one. No amount of more precise wording fully resolves an inherently ambiguous request, because the ambiguity is a property of the request itself, not a gap in how clearly the tools were explained.
What tends to work better is building an explicit disambiguation layer into the system prompt, separate from and prior to the tool selection step itself, instructing the model to recognize when a request plausibly maps to more than one available tool and to either ask a brief clarifying question or apply an explicit, stated tiebreaking rule, rather than silently picking one option and proceeding. A tiebreaking rule might look something like, when a request could reasonably be handled by either the general knowledge tool or the order specific tool, prefer the order specific tool whenever an order reference is present anywhere in the conversation, and fall back to the general tool only when no such reference exists. That kind of explicit, stated precedence rule resolves the ambiguity deterministically rather than leaving it to whatever the model's implicit judgment happens to favor on a given pass, which is what produces the inconsistency in the first place.
For the higher stakes case involving write capable actions specifically, the more important instruction is not actually about better disambiguation at all, it is about explicitly requiring confirmation before invoking any tool capable of a real side effect whenever the triggering request is even mildly ambiguous about intent, rather than trying to perfect routing accuracy to the point where confirmation feels unnecessary. This connects to a broader design principle sometimes discussed as tiered permission handling, where tools are not treated as uniformly equal in how confidently the model is allowed to invoke them, read only tools can be invoked on reasonable inference, while tools with real consequences require an explicit confirmation step baked into the instruction regardless of how confident the model's own routing judgment happens to be in that moment.
Testing For This Specifically
Tool routing reliability, like structured output reliability, resists casual conversational review, because a human reading through sample conversations naturally focuses on whether the final answer sounds right, not on which specific tool silently produced it. Catching routing inconsistency requires deliberately constructing test cases that sit in the actual ambiguous overlap zone between available tools, rather than only testing requests that clearly and obviously belong to one tool or another, since those clean cases will pass reliably regardless of whether the underlying routing logic is actually sound.
The Actual Lesson
Adding more tools to a conversational system multiplies its capability, and it also multiplies the number of decision points where the system can quietly go wrong before it ever gets to the part anyone is actually testing, the quality of the final response. Tool selection deserves its own explicit instruction layer, its own deliberate tiebreaking logic for genuinely ambiguous cases, and its own tiered confirmation requirements for anything with real consequence, rather than being treated as a simple, self evident routing step that naturally resolves itself once each individual tool is well described.
Specific client tool architectures and routing logic remain confidential given the nature of this work. Happy to discuss the general approach to tool selection and disambiguation design with anyone building multi tool conversational systems through the proper channel.
Written by Mohammad Farhan Habib Faraz
Senior Prompt Engineer and Prompt Team Lead at PowerinAI
www.powerinai.com
Top comments (0)