A translation job that has run for months starts refusing a handful of segments. The source text is unremarkable — a customer message, a novel, a support ticket. The refusal is caused by the translation itself: rendering an idiom literally produces a string that reads as a threat, and something downstream scores that string rather than the intent behind it.
The refusal, in each provider’s fields
The first job is to read the response properly, because three different systems can produce what looks like the same failure.
// Anthropic Messages API — the model itself declined
{ "stop_reason": "refusal", "content": [] }
// Azure OpenAI — a filter cut the completion short
{ "choices": [ { "finish_reason": "content_filter", "message": { "content": null } } ] }
// Gemini — HTTP 200, a candidate that produced nothing
{ "candidates": [ { "finishReason": "SAFETY",
"safetyRatings": [ { "category": "HARM_CATEGORY_DANGEROUS_CONTENT",
"probability": "MEDIUM" } ] } ] }
// OpenAI — the model complied in form but declined in prose
{ "choices": [ { "finish_reason": "stop",
"message": { "content": "I can't help with that request." } } ] }
The fourth case is the one that breaks pipelines. The status is 200, the finish reason is stop, and the refusal text lands in your database as the translation. In a bulk job nobody notices until a customer reads it.
Which layer refused
There are three candidates and each needs a different response.
- An input filter — a separate classifier scored the prompt before the model saw it. It is not an instruction-following model: it never sees “translate the following” as a task, only as more text in the blob it is scoring. Task framing helps it less than you expect.
- An output filter — the model produced the translation and a classifier scored the result. This one is structurally guaranteed to fire on faithful translations of violent source material, because the output really does contain what the source contained.
- The model itself — it read the task, understood it, and declined. This is the one that responds to framing, register instructions and examples, and it is the only one where prompt engineering is the right tool.
Determine which before changing anything. Sending the same segment to the provider’s moderation endpoint alone distinguishes filter from model in one call, and re-sending the segment as the model’s output — asking it only to echo the text — distinguishes input filter from output filter.
Two directions the misreading runs
Source to target. A phrase that is conventional hyperbole in the source language becomes, word for word, a description of violence or self-harm in the target. This is not a mistranslation of meaning so much as a mistranslation of register: the literal rendering is lexically accurate and pragmatically wrong, and the classifier reads lexically.
Target-culture reading. A term that is neutral in the source locale maps onto a loaded one in the target. A word for a regional or ethnic group, a historical event, a body part, or a political term can be routine in one locale and a slur or a taboo in another. Here the classifier is arguably right about the output string and the problem is upstream, in the choice of equivalent.
Both are made worse by literalness, which is why the standing fix is a translation instruction rather than a safety one. Word-for-word rendering is the operation that manufactures the offending string — why word-for-word prompt translation fails covers the same mechanism applied to prompts.
Worked examples
These are the classes that account for most real occurrences. All are ordinary speech in the source.
- Hyperbolic exhaustion and hunger. Japanese 死ぬほど 疲れた is “tired to the point of death”; Spanish me muero de hambre is “I am dying of hunger”. Rendered literally into English, both pattern-match the vocabulary a self-harm classifier is built around. Rendered idiomatically — “exhausted”, “starving” — neither trips anything.
- Mock threats between friends. German ich könnte dich umbringen and its equivalents in most languages are exasperation. Literally they are a stated intention to kill a named person, which is close to the canonical example in a threat classifier’s training data.
- Effort idioms built on risking life. Chinese 拼命 means “with everything one has”, literally “risking one’s life”. A literal rendering in a workplace context reads as a description of dangerous working conditions or worse.
- Religious and epic narrative. Scripture and epic describe war, sacrifice and killing as a matter of genre. This is the same collision worked through in safety filters and Hindi religious terms, and it applies to any tradition.
- Procedural domains. Butchery and charcuterie vocabulary, hunting regulations, veterinary euthanasia, funeral customs, surgical technique. Faithful translation of these produces text about killing and cutting because the source is about killing and cutting.
A retry policy that does not lose text
- Validate every response before storing it. Check the provider’s stop or finish field, check for a null or empty content field, and run a cheap refusal detector over the text for the fourth case above. A refusal must never be written to the same column as a translation.
- On a model refusal, retry once with an explicit register instruction: state that the text is colloquial, that hyperbole should be rendered idiomatically rather than literally, and that the task is translation of an existing document. Supply a short glossary of the idioms your corpus contains.
- On a filter block, do not reword — rewording is guessing at another model’s decision boundary. Where the provider exposes thresholds, adjust the specific category. Where it does not, route to a different provider, whose filter was trained separately and frequently does not fire on the same input.
- Segment smaller before you retry. A block on a 2,000-word chunk tells you almost nothing; the same content split into paragraphs usually isolates one sentence, which is both diagnosable and cheaper to route specially.
- After the retry and the fallback, surface the segment for human review with the original text, the category that fired and the provider. Do not fall through to an empty string, the source text, or a machine-translated approximation from a different system without marking it.
- Keep every refused segment in a regression set. Idiom collisions recur across a corpus, and a fixed glossary entry should be verified against the case that motivated it on the next model upgrade.
Refusal rates on borderline material move with model versions in both directions. A glossary and a fallback route are durable; a prompt tuned against one checkpoint’s boundary is not, and should be re-verified whenever the model behind the job changes.
Top comments (0)