DEV Community

Multigrid
Multigrid

Posted on • Originally published at multigrid.ai

Why Translating a Prompt Word for Word Produces Worse Output

Running your prompt through a translation API and shipping the result is the cheapest possible localisation, and it produces a prompt that is semantically correct and pragmatically inert. The reason is not that the translation is bad. It is that a prompt is not content, and the thing that makes an instruction work is not the thing translation preserves.

The claim

An instruction works because it resembles instructions. When you write “Think step by step” or “Answer only from the provided documents”, the model is not parsing a specification; it is continuing text, and the continuation of text that looks like a careful instruction is text that looks like careful compliance. The phrasing is doing that work — it places the request inside a genre the model has seen a great deal of.

A word-for-word translation preserves the propositional content and discards the genre membership. The translated sentence means the right thing and reads like nothing anybody writes. That is the whole argument, and everything below is evidence for it.

Where the meaning survives and the force does not

Take a prompt line that is entirely ordinary in English: Be concise. Do not sugarcoat it. Bottom line up front.

“Sugarcoat” is a metaphor with no compositional meaning. A translation engine will usually find an idiomatic equivalent, but where it does not, the literal rendering is a sentence about confectionery. “Bottom line up front” is a piece of English business register — an abbreviation of a convention, understood by everyone who has read a memo, and meaningless when rendered as a sentence about lines and their bottoms. Even when the translation is idiomatic, the resulting phrase is often several registers away from where it started: the natural German equivalent of a punchy English imperative can land somewhere between bureaucratic and brusque, and the model imitates the register it was given.

The general form: any instruction containing an idiom, a metaphor, a register marker, or a domain convention loses exactly the part you were relying on. Prompts are unusually dense in all four, because prompt writing evolved as a compressed, punchy dialect. That density is why prompts translate worse than most other text.

Structure that does not transfer

Beyond vocabulary, three structural properties of English instructions fail to survive.

Imperative form

English has one bare imperative and it is neutral: “Summarise the document.” Japanese has several imperative forms with sharply different force. The bare imperative is harsh enough to read as an order shouted at somebody; the ordinary way to instruct is a -te kudasai request or a nominalised -koto specification of the kind used in written rules. A literal translation picks one, usually the wrong one, and the model answers in the register it was addressed in. Korean has the same problem with more levels. This alone changes the tone of every output.

Pronouns and ellipsis

“Summarise the document and translate it” relies on an English reader resolving it by proximity. In Japanese, Korean or Chinese the pronoun would ordinarily be dropped, and the literal translation inserts one where a native instruction would not, which is simultaneously unnatural and no less ambiguous. In a language with grammatical gender, the pronoun must agree with something, and the translation has to pick a gender for a referent it has not resolved.

Negation and scope

“Do not answer unless you are certain” is a conditional wrapped in a negation, and English tolerates that stacking with a particular word order. Languages that place the negation elsewhere in the clause, or that mark the conditional with a suffix, end up with a sentence whose scope is genuinely ambiguous where the English was not. Constraint prompts are full of this construction, and it is the one place where a translation error becomes a policy error.

Why this is a distribution argument

The mechanism is worth stating plainly, because it explains why the effect is not fixed by a better translation model. A language model produces a distribution over the next token conditioned on the context. Instruction-following is a behaviour learned from text where instructions were followed — documentation, tutorials, question and answer pairs, and post-training data. That training text exists in every language, but the instructions in it are the ones people in that language actually write.

A literally translated instruction is off that distribution. It is grammatical and it is comprehensible, but it is not a sentence anybody writes when instructing, so it sits in a thinly populated region of the model’s experience. The conditioning is weaker, and weaker conditioning shows up exactly where you would predict: the instruction is followed less reliably under pressure, the model reverts to its defaults on long inputs, and constraints are the first thing dropped.

There is a second-order effect that is easy to miss. Translationese in the prompt is a style signal, and the model matches the style of what it is given. A prompt that reads as translated tends to produce output that reads as translated, which is precisely the quality complaint that sent you looking for a fix.

What to do instead

There are two defensible positions and the machine-translated middle is neither of them.

  • Keep the instruction in English and localise around it. One canonical instruction, an explicit output-language directive, and examples in the target language. This is the right default for a team with no native speaker for a language: an English instruction is at least on-distribution as an instruction, and the target language is carried by the examples and the directive. Its failure modes are real but bounded, and they are catalogued in why a prompt that works in English fails silently elsewhere.
  • Have the instruction rewritten natively. Not translated — rewritten, from the intent, by somebody who writes instructions in that language. Give them the English prompt as a statement of requirements rather than as source text, and expect the result to differ in structure and not only in words. This is better when you have the person, and it is the only option for languages where the English instruction itself underperforms.

Whichever you pick, the examples are the part that must be native in both cases. Examples teach surface form directly, and a translated example demonstrates translated prose with total fidelity — see why few-shot examples belong in the target language.

The one thing not to do is ship a translated prompt unread. If nobody on the team can read the German prompt, then nobody can tell you whether the model is following it, ignoring it, or following a version of it that says something slightly different from what you meant. That is not a localisation strategy; it is an unmonitored change to your most important file.

Related

Top comments (0)