My AI Agent Kept Calling the Wrong Tool. I Blamed the Model for a Week — the Bug Was in 6 Lines of JSON

For seven days I was convinced I'd picked the wrong model.

My support agent — the one that looks up orders, checks refund eligibility, and escalates to me when it's out of its depth — kept doing something maddening: when a customer asked "where's my order?", it would call search_knowledge_base instead of get_order_status. It would return a polite, fluent, completely useless answer about shipping policies in general. The customer's actual order? Never looked up.

I did what any reasonable solo operator does: I blamed the model. I upgraded to a bigger one. I cranked the temperature down. I rewrote the system prompt three times. I added few-shot examples of correct behavior. The failure rate dropped a little, then plateaued at roughly 1 in 5 conversations going sideways.

The actual fix took about twenty minutes, cost $0, and lived nowhere near the system prompt. It lived in the tool definitions — six lines of JSON I'd written in a hurry and never revisited.

Here's everything that week taught me about the part of prompt engineering nobody talks about.

The part of the prompt you forgot you wrote

When you give an agent tools, you're not just giving it capabilities. You're giving it a menu with descriptions, and the model picks from that menu on every single turn. The tool schema — name, description, parameter definitions — is prompt text. It carries the same weight as your system prompt. Most of us treat it like API documentation we fire off and forget.

My original search_knowledge_base tool looked something like this:

{
  "name": "search_knowledge_base",
  "description": "Searches for information to help answer the customer's question. Use this tool whenever you need information you don't have.",
  "parameters": { "query": { "type": "string" } }
}

Read that description again, from the model's point of view, on the turn where a customer asks "where's my order?"

"Use this tool whenever you need information you don't have."

The model doesn't have the order status. It needs information it doesn't have. By my own instructions, this was the correct tool. I hadn't written a description — I'd written a catch-all magnet that hoovered up every ambiguous turn.

Meanwhile get_order_status was described as: "Gets order status." Three words competing against a sentence that promised to solve everything. Of course the flashy generalist beat the terse specialist. Models are not reading your tools as an API reference; they're reading them as competing sales pitches.

What I changed

Once I understood that tool descriptions are prompts, the rewrite was mechanical. Four rules, all earned the hard way:

1. Describe when NOT to use the tool. Negative space does more work than positive. search_knowledge_base became:

"description": "Searches general policy and product documentation (shipping times, warranty rules, how-tos). ONLY for general questions. NEVER use for anything about a specific customer's order, account, or refund — use get_order_status or get_refund_eligibility for those, even if the customer hasn't given an order number yet."

That "even if the customer hasn't given an order number yet" clause fixed more failures than everything else combined, because it named the exact ambiguity the model was resolving wrong.

2. Let tools declare their prerequisites. get_order_status became: "Looks up a SPECIFIC customer's order: current status, tracking link, and delivery estimate. ALWAYS use this first when the customer asks about their own order. If you don't have an order number or email, ask the customer for one — do not fall back to searching the knowledge base." Now the model knows the tool exists precisely for the question it's staring at, and knows what to do when an argument is missing.

3. Rename tools so the name does half the work. I had escalate and escalate_to_human in two versions of the agent. The model conflated escalate with "handle this seriously" and sometimes called it for routine questions. Renaming it transfer_to_human_agent_and_end_conversation made misfires nearly vanish — the name now spells out the consequence. Names are read every turn; make them self-documenting.

4. Constrain parameters with enums and formats, not prose. My priority parameter used to be "priority: string". The model invented values like "high-ish" and "URGENT!!". Now it's {"enum": ["low", "normal", "high"]} and downstream code stopped crashing on Tuesdays. Wherever you're parsing a tool argument yourself, an enum or a stated format ("ISO 8601 date, e.g. 2026-09-01") is cheaper than any amount of retry logic.

The failure that made me log tool calls in the first place

Honest part of the story: I only found any of this because of an earlier, worse failure.

For the first two weeks, the agent ran with no structured logging — just the raw conversation transcript. When a customer complained the bot was "useless," I'd read the transcript, see a fluent and plausible answer, shrug, and blame the customer's phrasing. It took a support thread where the customer pasted their side of the conversation — ten messages, order number given in message two, never used — for me to realize I was debugging blind.

So I added one log line per tool call: timestamp, tool name, arguments, and a one-line summary of the preceding customer message. Twenty lines of Python. The pattern jumped out within an hour: every wrong-tool call happened when the customer's message mentioned both a specific order and a general policy word ("when do refunds usually post? I ordered #4471"). The knowledge base tool's description contained the word "information" and the model pattern-matched to it.

If you take one thing from this post: log tool calls, not just transcripts. The transcript tells you the agent was wrong. The tool log tells you why, and "why" is the only thing you can fix.

What I'd tell myself a week ago

• Your system prompt is maybe 40% of the prompt your model actually sees. Tool schemas, tool results, and error messages from your own code are the rest. Audit all of it.
• Every tool description competes with every other tool description. Write them as a set, not one at a time. If two tools could plausibly answer the same query, say explicitly which one wins and why.
• Vague descriptions don't just fail quietly — they succeed loudly, with fluent wrong answers that look fine in a transcript.
• The model is not stupid. It's obedient. It did exactly what six lines of JSON told it to do.

The failure rate went from ~20% of conversations to under 2% on the day I shipped the rewrite. No model upgrade, no temperature tuning, no extra cost per call.

---

Tool descriptions are prompts, and prompts are worth versioning, testing, and stealing good ones from. All 100 prompts are in The Agent Prompt Vault — $3, lifetime updates. Steal the ones that fit your workflow.