Most teams approach a multilingual support bot as a translation problem. Wire up a translation layer, send everything to one big model, done. We did roughly that the first time, and it mostly worked — until we looked at the bill and at which customers were actually being served well.
This is the field-note version of what we learned building support bots for businesses serving Southeast Asia: the "multilingual" part is not one problem, it's three, and conflating them is why these deployments quietly cost 3x and quietly fail the Tamil- or Malay-speaking customer.
Three problems hiding under "multilingual"
1. Language detection under code-switching.
A real SEA customer doesn't pick a language. They write "boleh tolong check my order ah?" — Malay and English in one breath. A monolingual pipeline treats that as an error condition. The first routing decision isn't "which model," it's "is this even one language," and most systems answer wrong. Miss it and you've already degraded the interaction before the model sees it.
2. Language is not model size.
This is the one nobody warns you about. English is relatively simple — SVO, minimal inflection — and a small model holds quality fine. Arabic, Malay, and Tamil carry morphological complexity that needs a larger model to stay accurate. Routing every language to the same frontier model leaves money and latency on the table for English while still under-serving the harder languages. We saw a 3.5B-class model hold English quality while a 7B-class tier was needed for the others. One published pipeline routes low-resource languages through translate-to-English before classification and lifts low-tier accuracy from ~0.46 to ~0.68 Macro-F1 — same idea: not every language deserves the same path.
3. Per-step routing inside a single turn.
"Check my order" is classify-and-lookup. "Explain why my transfer failed and draft a reply" is synthesis. Same customer, same language, two completely different model tiers — and most bots send both to the expensive one because the router only sees "it's a support message."
Then PDPA decides your topology
Singapore's PDPA (and Malaysia's, moving alongside it) wants clarity on where data is processed and stored, PII redaction, and no-train terms with the underlying providers. For a support bot handling customer PII, that means inference runs in-region.
Here's the part we didn't expect: the residency requirement and the latency requirement turn out to be the same decision. Keep inference in-region and you've satisfied the residency half of PDPA while staying under the ~300ms threshold for natural conversation. Push it overseas and you're violating one and degrading the other. "Where does the model run" stops being a cost footnote and becomes the architecture. We run inference in-region (SG-hosted, on Tencent Cloud, PDPA-aligned) partly because the regulation demands it and partly because it's the cleanest way the latency math works — the compliance team and the performance team ended up arguing for the identical topology.
Why this only works if the models are a dial
None of the above is a single-model story. The language-tier decision — 3.5B for English, 7B for the harder languages, a frontier model for the synthesis step — only pays off if switching tiers is a config change, not a migration. We sit 25+ models behind one OpenAI-compatible endpoint, so "route Malay to a bigger tier" is a line in a routing table, not a procurement cycle. That unglamorous fact is the reason routing works at all: the models are a dial, not a commitment. And per-language routing is also where most of the cost saving actually comes from — you stop paying frontier rates for English classification.
The part we're still getting wrong
We measure routing quality with one global score, but the failure modes are per-tier. English can be perfect while a low-resource language drifts for a week and nobody notices until a customer complains in Tamil. We're moving to Macro-F1 per language tier as the monitoring signal. Curious whether others running multilingual routing monitor drift per-language or still trust a single aggregate — that's the open question we haven't settled.
The whole point: a multilingual bot isn't a translation layer bolted onto one model. It's a router whose first dimension is language, and language is messier, more tiered, and more regulated than the translation mental model admits.
We're writing these up as we run them — more at tokenlat.com.
Top comments (0)