A keyword list is the wrong shape for routing a chatbot to web search. It matches fragments instead of meaning, it breaks the moment you support more than one language, and it fails silently in the expensive direction: the question that needed a lookup and did not get one. Let the model make the call instead, with a short prompt built on an inverted default. Here is the bug that taught me this, and the prompt shape that replaced it.
Last updated: September 2026.
The bug
The chat had a cheap gate before the expensive part. If the message contained a trigger word, run a web search and answer from the results. Otherwise answer directly. The list held the obvious things: price, weather, today, score, plus the currency words for every language the product supports.
One of those was the Turkish word for exchange rate: kur.
The German word for "briefly" is kurz.
A user wrote "Sag kurz Hallo", roughly "say a quick hello". The substring match found kur inside kurz, the gate fired, and the system paid for a web search in order to answer a greeting.
That is a funny bug. The unfunny part is what it revealed.
Why the list was never going to work
Substring matching has no word boundaries across languages. You can bolt on tokenisation, but you are now maintaining locale-aware word splitting for something that was supposed to be a cheap shortcut.
It is always wrong in both directions. It fires on greetings and stays silent on "is my residence permit application still processed the old way", where being a year out of date is genuinely harmful. Adding words fixes today's miss and creates tomorrow's false positive. I went three rounds of this before admitting the shape was wrong.
It scales with languages, not with logic. Eleven languages meant eleven sets of triggers, and every new language meant another translation pass by someone guessing which words natives actually type.
It cannot see intent. "Explain inflation" and "what is inflation right now" share their meaningful words. Only one needs a lookup.
What replaced it
Let the model decide, and give it a shape it can follow. Two mechanically different options depending on your stack:
| Approach | How it works | Use when |
|---|---|---|
| Tool calling | Expose a search_web tool; the model either answers directly or calls the tool with a query it wrote itself |
Your model supports function calling |
| Cheap classifier hop | One small call returns either a search query or the literal token NO_SEARCH, then you branch |
No tool calling, or you want the router on a different model |
Both share the property that made the difference: the model reads the whole sentence instead of pattern matching on fragments. It also writes a better query than the user's raw message, which turned out to be a larger quality win than the routing decision itself.
The prompt shape that worked
This is the part I got wrong twice, so it is the part worth copying.
My first instinct was a detailed, categorised list of what deserves a search: prices, weather, sports, news, regulations, each with examples. The score went down. A small routing model gets lost in a long taxonomy.
What worked was shorter and inverted. Around two hundred words with three moves:
Set the default to search. Not "search when needed" but "search unless the question is on this closed list".
Keep the exception list closed and concrete. Arithmetic, greetings, translation, grammar, code, creative writing, summarising text the user pasted in, and textbook facts that are the same in every country and every year. Everything else is a search.
Give two decision tests instead of categories. Does the answer change depending on the user's country or the current year? And what does a wrong answer cost this person: money, health, legal standing, safety, a missed deadline? If either trips, search.
Then one closing line that matters more than it looks:
Search these even if you think you already know the answer.
The inversion is the trick. A list of things to search is infinite and you will always be adding to it. A list of things not to search is small, stable, and a model can hold it in mind.
Measure it, or you are guessing
Build a fixed evaluation set before you touch the prompt. Mine is thirty questions: fifteen that must search and fifteen that must not, spread across several languages and domains.
| Must search | Must not search |
|---|---|
| Family law question with a country in it | Greeting |
| Rent or tenancy rule | Arithmetic |
| Visa procedure | Translation |
| Drug interaction | Grammar correction |
| Live exchange rate | Code question |
| A specific local business | Write me a poem |
| Product comparison | Textbook fact |
| Tax or fee amount | Summarise this pasted text |
Run the whole set after every prompt change. Two things this catches that spot checks never do: a change that fixes your new case and breaks two old ones, and a router that has quietly become search-happy, which always looks correct if you only inspect the questions that should search.
Both of my "improvements" scored worse than the version before them. I would not have known without the set.
For scale, in production this router now sends 18.1 percent of questions to a live search over a 60 day window of 5,977 answers. The remaining 82 percent are answered directly, which is the entire economic point of routing rather than searching on everything.
Two things I would not do again
Do not put a time budget inside the router. I briefly capped the routing stage so it could not slow the reply down. What it actually did was make the system give up early and fall through to the expensive path anyway. Correctness first, then optimise.
Do not let a strong style prompt fight the router. The main system prompt tells the model to be direct and expert. That instruction was competing with routing and pushing the model toward answering from memory. If the router shares a prompt with the personality, the personality wins. Give the routing decision its own explicit sentence rather than assuming the tool description covers it.
The general lesson
The keyword list was not too small. It was the wrong kind of thing. Any rule that pattern matches on fragments of language will fail across languages, and it will fail silently in the direction that costs the most.
If you are building anything that chooses between a cheap path and an expensive one based on what the user wrote, the model is a better judge than your list, provided you give it a short prompt with an inverted default and a closed set of exceptions.
And keep the greeting in your test set. That is how I found out.
This is from building GetAskAI. GetAskAI is a free, ad-supported AI chat that works without an account. It answers in 11 languages, searches the web and shows numbered sources when a question needs current information, and reads PDFs and images without storing them.
Top comments (0)