Translating the interface is one of those jobs that sits on the list for a year, because the quote from an agency is real money and the effort of doing it yourself is real time, and neither of those is ever the most urgent thing this week. What I wanted was a number, so I could stop guessing: what does it now cost to put a product's interface through a model instead, and what does that quietly break on the way.
The material is the locale file out of a football quiz app I run, which is a perfectly ordinary Next.js codebase and not special in any way that matters here. All 430 strings went through four models into four languages. The cost turned out to be so small it barely deserves a paragraph, which I was not expecting, and the breakage turned out to include one bug I would have shipped without noticing, which I was not expecting either.
The first surprise was how little of the app was translatable at all
Before any of the interesting work there was a reality check, and I suspect it generalises well past my codebase. The locale file holds 430 strings and 4,593 words, which felt like most of the app until I went and counted how many components ever ask for a translation at all:
51 of 861 components import a translation hook
Six per cent. The rest render their text directly, and a regex hunting for prose sitting in raw JSX flags roughly 1,837 more text nodes that no translation layer will ever reach. That figure is a heuristic and I would not defend the exact number, but the shape of it is not in doubt, because you can go and look at lines like this one:
<Link href="/login">Sign In</Link>
Nothing is going to translate that. So the honest framing of everything below is that I translated one locale file, not an application, and the gap between those two things was bigger than I would have guessed for a codebase I wrote myself. If you are budgeting for internationalisation on the basis of your en.json, the file is probably telling you a comfortable lie.
What the tokens actually cost
Every run went through DigitalOcean's inference API, against DeepSeek v4 Pro, Llama 4 Maverick, GLM 5.3 and Mistral 3 14B. The same 430 strings, in batches of forty, into Bulgarian, Spanish, German and Japanese: 16 model-language pairs, 230 requests.
Measuring the cost through those batches turned out to be slightly wrong, because each response echoes the JSON keys back, and the keys are ASCII regardless of the target language, so they flatten the differences. A cleaner measurement is to hand each model the same paragraph and ask for nothing but the translation:
| language | output tokens, relative to English |
|---|---|
| Japanese | 1.66x |
| Bulgarian | 1.61x |
| Spanish | 1.38x |
| German | 1.34x |
All four models agreed on the ordering independently. The practical version of that table: if you bill per token, the same sentence costs you between a third and two thirds more to produce for a non-English user, and if you are streaming model output to users in Japanese or a Cyrillic language, that multiplier is sitting on every response you generate, not just the one-off translation job.
For the translation itself, the whole exercise — 430 strings, four languages, four models, everything above — came to a few hundred thousand tokens. It is a rounding error. The cost was never going to be the reason not to do this.
Quality, judged by a machine that does not speak the language
I do not read Japanese, and I was not going to eyeball 1,720 translations. So the gate is mechanical: a different model translates each string back into English, and DigitalOcean's bge-m3 embedding model scores how far the round trip landed from where it started.
| language | mean round-trip similarity | below 0.80 |
|---|---|---|
| German | 0.976 | 2% |
| Bulgarian | 0.973 | 0% |
| Spanish | 0.965 | 2% |
| Japanese | 0.944 | 7% |
That is, broadly, a pass. Machine translation of short interface strings is good now, and the rare failures were not random — every one of them was a terse label with the context stripped out of it:
'Tactics Brief · All-Time' -> 'Tactical Report · Historical'
'How do I play Conquest?' -> 'How do I play?'
'Retired-roster boards · grids
built from the legends' -> 'Legendary players only grid'
The middle one is the one to look at. Conquest is the name of a game inside the app, and the Japanese translation dropped it, so a question about one specific game became a question about the app in general. A proper noun that the model did not recognise as a proper noun simply evaporated, and the sentence that remains reads perfectly well, which is exactly what makes it dangerous.
The failure that would have reached production
Here is the one that matters, and the reason a similarity score is not enough on its own.
Three strings in the file use ICU MessageFormat plurals, the syntax every serious i18n library uses to get "1 attempt" and "5 attempts" right:
{count} {count, plural, one {attempt} other {attempts}}
Checking whether that construct survived translation, by language:
| language | ICU plurals intact |
|---|---|
| Bulgarian | 12 of 12 |
| German | 12 of 12 |
| Spanish | 12 of 12 |
| Japanese | 4 of 12 |
Two of the four models, translating into Japanese, returned this:
{count} 回
Which is correct Japanese. Japanese does not inflect nouns for number, so a plural construct is linguistically pointless, and the model made a sensible call. It also deleted a branch that the formatter requires, and a missing other branch is not a wording problem, it is a runtime error in the component that renders it.
Note what this does to the two gates. The round-trip similarity for those strings is fine, because the meaning genuinely is fine. Only a structural check catches it. The two gates are looking at different things, and if I had run only the clever one I would have shipped the break.
There were quieter versions of the same lesson. GLM 5.3 silently returned 40 fewer keys than it was given in one language, no error, just a shorter object. And between 0.2% and 2.8% of strings came back identical to the English input, which is sometimes right (ExtraTime should stay ExtraTime) and sometimes just a string the model skipped.
What I got wrong
My first placeholder checker reported that every model mangled placeholders in 4% of cases, consistently, across all sixteen runs. That consistency is what made me look: four different models failing at an identical rate on the same string is not how models fail, it is how a checker fails.
The string was the ICU plural above. My regex treated {attempt} and {attempts} as placeholders to be preserved verbatim, so when the models correctly translated them to {опит} and {опита}, it counted the correct answer as a defect. The real ICU problem, the one in the previous section, was sitting underneath that noise and only appeared once I wrote a check that understood the syntax instead of pattern-matching braces.
That is twice in a fortnight that my own instrument produced a tidy, publishable, wrong number. Both times the giveaway was the same: the result was too clean.
What to take from it
If you are thinking about doing this, the cost is not the question. A few hundred thousand tokens translates a locale file into four languages, and the per-language token inflation, real as it is, matters far more for the output you stream to users every day than for a one-time job.
The question is what to check afterwards, and it is not what I assumed. Check the structure first: that ICU constructs survive, that placeholders are intact, that the number of keys you got back equals the number you sent, which is the check that catches a model quietly truncating its own output. Then check meaning, with round-trip translation through a different model and an embedding score, because that catches the missing proper noun a structural check cannot see. And give the model your terse labels with some context attached, since every meaning failure I found was a four-word string with nowhere to stand.
Then go and find out how much of your interface is even reachable from your locale file. In my case it was six per cent of components, and that turned out to be the most useful thing I learned all day.
Top comments (0)