The first time I AI-translated a locale file, it looked perfect. Every string had a Spanish version, the tone was right, nothing was missing. I shipped it. Two days later a user reported the app was printing Hola {nombre} on screen, literally, braces and all. The model had helpfully translated the variable name {name} into {nombre}, and now the interpolation didn't match any variable my code was passing.
That's the thing nobody tells you about using an LLM for translation. It doesn't just translate the words you want translated. It translates everything that looks like words, and a surprising amount of what's inside a locale string isn't text meant for humans.
What actually breaks
Locale strings are full of machine parts that have to survive translation byte-for-byte. An LLM sees them as language and rewrites them. A few real failure modes I've hit:
Variable names get localized. Welcome, {name} becomes Bienvenido, {nombre}. The braces are intact, the sentence reads fine, and the string is broken because your code injects name, not nombre. This one is nasty because it looks completely correct in review.
Plural syntax gets mangled. ICU messages like {count, plural, one {# item} other {# items}} are a mini-language. The model translates item and items (good) but also sometimes rewrites one/other into the target language or drops the #, and now the whole message fails to parse at runtime.
Inline markup gets dropped or translated. A string like Read our <link>privacy policy</link> comes back with <enlace> tags, or with the tags silently removed, or reordered so your React component can't match them.
Printf-style tokens vanish. %s uploaded %d files is easy for a model to "clean up" into something that reads better and passes zero arguments correctly.
None of these throw at translation time. They throw in production, in a language you probably don't read, for users who mostly won't file a bug and will just leave. It's the exact silent-degradation problem as locale drift, except now you introduced it yourself by trying to fix drift.
Level 1: tell the model what not to touch
The first defense is the prompt. If you're calling an LLM directly, be explicit and give it the variable syntax you actually use:
Translate the values to {targetLang}. Do NOT translate or modify anything
inside {curly braces}, %placeholders, or <tags>. Keep them exactly as-is,
in the same order. Return the same JSON structure.
This cuts the error rate a lot. It does not get you to zero. Models still slip, especially on longer strings, plural blocks, and anything where reordering the sentence "wants" to move a token. Prompting reduces the odds. It doesn't give you a guarantee, and locale correctness is a place where you want a guarantee.
Level 2: validate the output, don't trust it
The move that actually works is to stop trusting the model and verify its output mechanically. Extract every placeholder from the source string, extract every placeholder from the translation, and assert the two sets match. Different order is fine. Different set is a bug.
// pull out {name}, {{count}}, %{item}, %s, %d, <tag>...</tag>
const PLACEHOLDER = /\{\{?[^}]+\}?\}|%\{[^}]+\}|%[sd]|<\/?[^>]+>/g;
function placeholders(str) {
return (str.match(PLACEHOLDER) ?? []).sort();
}
function validate(source, translated, key) {
const a = placeholders(source);
const b = placeholders(translated);
if (JSON.stringify(a) !== JSON.stringify(b)) {
throw new Error(
`Placeholder mismatch in "${key}":\n source: ${a.join(" ")}\n translated: ${b.join(" ")}`
);
}
}
That catches the {name} to {nombre} case, the dropped %s, and the missing <link>. For ICU plurals you want a real parser (@formatjs/icu-messageformat-parser) rather than a regex, so you can compare the message structure, arguments and plural categories, instead of just the tokens. But even the naive version above would have caught the bug that got me.
Run this over every translated key, fail loudly on mismatch, and the whole class of "the model quietly broke an interpolation" stops reaching production.
Level 3: put validation in the same step as translation
Level 2 works, but if translation and validation live in different places, someone eventually runs the translation without the check. So the real fix is to make them one operation: you can't get translated output that hasn't been validated, because validation is part of producing it.
This is what I ended up building into i1n. When it AI-translates a locale, placeholder and structure validation isn't a separate lint step you might forget, it runs on the output before anything gets written:
- Translation happens with the variable syntax and product context baked in, so the model knows
{name}is off-limits and that "Save" is a button, not a rescue. - Every translated value is checked against its source for placeholder parity and ICU structure. A broken interpolation never makes it into your files.
-
i1n checkruns the same validation in CI, so even a hand-edited translation or a value from an external tool gets caught the same way.
The nice part is that validation and translation reinforce each other. The context makes the model wrong less often, and the check means the times it's still wrong don't cost you a production incident. You review a diff of correct-by-construction translations instead of auditing raw model output for landmines.
The short version
- Prompting the model to leave placeholders alone helps, but it's odds, not a guarantee. Don't stop there.
- Extract placeholders from source and translation and assert they match. This is the check that actually holds. Use a real ICU parser for plurals.
- Make validation inseparable from translation so it can't be skipped, whether you wire that up yourself or let i1n do it (
npx i1n init, free tier, no card).
Raw LLM output is a great first draft and a terrible last word. The models are good enough that the words will be fine. It's the parts that aren't words that will page you.
Top comments (1)
This is a real failure mode and it slips through review because the translated string still looks fluent. The strongest part here is pushing validation into the same step as translation instead of treating it as a best-effort follow-up. For frontend teams, ICU/plural parsing and placeholder diff checks belong in CI the same way type-checking does, because broken locale syntax is effectively a runtime bug. I've seen this bite React apps in exactly the way you described.