Most teams approach a multilingual support bot as a translation problem. Wire up a translation layer, send everything to one big model, done. We did roughly that the first time, and it mostly worked — until we looked at the bill and at which customers were actually being served well.
This is the field-note version of what we learned building support bots for businesses serving Southeast Asia: the "multilingual" part is not one problem, it's three, and conflating them is why these deployments quietly cost 3x and quietly fail the Tamil- or Malay-speaking customer.
Three problems hiding under "multilingual"
1. Language detection under code-switching.
A real SEA customer doesn't pick a language. They write "boleh tolong check my order ah?" — Malay and English in one breath. A monolingual pipeline treats that as an error condition. The first routing decision isn't "which model," it's "is this even one language," and most systems answer wrong. Miss it and you've already degraded the interaction before the model sees it.
2. Language is not model size.
This is the one nobody warns you about. English is relatively simple — SVO, minimal inflection — and a small model holds quality fine. Arabic, Malay, and Tamil carry morphological complexity that needs a larger model to stay accurate. Routing every language to the same frontier model leaves money and latency on the table for English while still under-serving the harder languages. We saw a 3.5B-class model hold English quality while a 7B-class tier was needed for the others. One published pipeline routes low-resource languages through translate-to-English before classification and lifts low-tier accuracy from ~0.46 to ~0.68 Macro-F1 — same idea: not every language deserves the same path.
3. Per-step routing inside a single turn.
"Check my order" is classify-and-lookup. "Explain why my transfer failed and draft a reply" is synthesis. Same customer, same language, two completely different model tiers — and most bots send both to the expensive one because the router only sees "it's a support message."
Then PDPA decides your topology
Singapore's PDPA (and Malaysia's, moving alongside it) wants clarity on where data is processed and stored, PII redaction, and no-train terms with the underlying providers. For a support bot handling customer PII, that means inference runs in-region.
Here's the part we didn't expect: the residency requirement and the latency requirement turn out to be the same decision. Keep inference in-region and you've satisfied the residency half of PDPA while staying under the ~300ms threshold for natural conversation. Push it overseas and you're violating one and degrading the other. "Where does the model run" stops being a cost footnote and becomes the architecture. We run inference in-region (SG-hosted, on Tencent Cloud, PDPA-aligned) partly because the regulation demands it and partly because it's the cleanest way the latency math works — the compliance team and the performance team ended up arguing for the identical topology.
Why this only works if the models are a dial
None of the above is a single-model story. The language-tier decision — 3.5B for English, 7B for the harder languages, a frontier model for the synthesis step — only pays off if switching tiers is a config change, not a migration. We sit 25+ models behind one OpenAI-compatible endpoint, so "route Malay to a bigger tier" is a line in a routing table, not a procurement cycle. That unglamorous fact is the reason routing works at all: the models are a dial, not a commitment. And per-language routing is also where most of the cost saving actually comes from — you stop paying frontier rates for English classification.
The part we're still getting wrong
We measure routing quality with one global score, but the failure modes are per-tier. English can be perfect while a low-resource language drifts for a week and nobody notices until a customer complains in Tamil. We're moving to Macro-F1 per language tier as the monitoring signal. Curious whether others running multilingual routing monitor drift per-language or still trust a single aggregate — that's the open question we haven't settled.
The whole point: a multilingual bot isn't a translation layer bolted onto one model. It's a router whose first dimension is language, and language is messier, more tiered, and more regulated than the translation mental model admits.
We're writing these up as we run them — more at tokenlat.com.
Top comments (7)
The aggregate Macro-F1 problem is the exact structural flaw that hides tail risk in multi-asset pricing.
When eighty percent of traffic is English, a single portfolio-level metric lets high-volume classification mask severe distribution shifts in low-resource buckets. You can have a stable 0.94 aggregate score while Tamil error rates double, because the volume weight washes the drawdown out. In risk modeling, evaluating heterogeneous exposures with a pooled average is just shorting tail variance. Segmenting drift monitoring by language tier treats each language pair as an independent loss distribution with its own drawdown limit.
The economic trap of the unified frontier deployment is similar. Paying top-tier token rates for simple English classification to subsidize occasional morphological complexity in Malay is buying an expensive insurance policy for routine operations. Slicing the pipeline into a fast 3.5B classification path and reserving heavier tiers for synthesis isolates the cost of variance to the transactions that actually generate it.
Your risk-modeling framing is the sharper version of the point: a pooled Macro-F1 is literally shorting tail variance. The 0.94 aggregate hiding a doubled Tamil error rate is the drawdown that never shows up in the portfolio number until it's realized — exactly the failure a single global score is built to mask.
So we monitor drift per language tier, each pair as its own loss distribution with its own limit. The same logic has to reach the router, though: the 3.5B English-classification path isn't cheap because the model is small, it's cheap because it only carries the variance English actually generates. Reserving the heavier tier for synthesis is allocating risk capital where the exposure is, not downgrading the model. The trap is treating the frontier tier as the baseline and discounting down from it.
Still open on our side: do you run the per-tier drawdown limit as a hard stop, or a weighted alert that still lets the aggregate drift? We've kept it soft for low-volume pairs, but your framing makes me think the hard limit is the point — it's what stops you buying insurance for the routine buckets.
A soft alert on low-volume buckets usually degenerates into no limit at all. Because sparse sample sizes produce noisy point estimates, teams naturally rationalize every breach as an anomaly until an outage proves otherwise.
The mechanics only hold if the circuit breaker is structural rather than discretionary. When a specific low-resource tier breaches its error budget, the router routes that slice immediately to the frontier model or human review, leaving the high-volume paths untouched. That bounds downside risk locally without penalizing the overall pipeline. Running twenty percent more compute on a thin tail costs pocket change compared to carrying an unhedged distribution shift in production.
The discretionary-alert failure mode is exactly why soft limits don't hold — once a breach gets rationalized as "anomaly," the budget is effectively gone. Routing the breached slice to frontier while leaving high-volume paths untouched is the precise operationalization of segmenting by loss distribution rather than pooling. One question: do you size each slice's error budget statically, or does the router widen it when the tail is actually moving?
Sizing the error budget statically is necessary to stop the router from trading away solvency during an incident. If you let the budget widen when the tail moves, the routing logic quietly takes on unhedged basis risk to preserve low-latency throughput. When variance expands, the threshold has to stay fixed so the overflow immediately hits the frontier tier or human review.
Code-switching also makes confidence more useful than a single language label. A router should be able to say “mostly Malay, English product terms, uncertain intent” and send only the ambiguous span through a stronger path. I would measure quality per language pair and task type, not per language alone; otherwise aggregate English volume can hide exactly the low-resource failures the routing design is meant to fix.
This is the right extension — a single language label throws away the information that actually drives routing. "Mostly Malay, English product terms, uncertain intent" is exactly the kind of structured signal a router should emit, and only the uncertain span should escalate.
It also lines up with the cost point: you stop paying frontier rates to process the 90% that's already deterministic. Measuring per language-pair × task-type (not per language alone) is the only way the routing design doesn't end up hiding the low-resource failures it was built to fix.