shashank ms

Posted on Sep 13

LLM Models for Machine Translation and Language Localization: A Comprehensive Guide

#learnai #oxlo #ai

We are going to build a localization agent that turns English JSON resource files into production-ready translations for any target locale. It preserves ICU placeholders, respects brand terminology, and validates its own output with back-translation. If you ship software to multiple languages, this replaces brittle script-based pipelines with a single, inspectable Python module.

What you'll need

An Oxlo.ai API key from https://portal.oxlo.ai
Python 3.10 or newer
The OpenAI SDK: pip install openai

Step 1: Initialize the Oxlo.ai client

First, import the SDK and point it at Oxlo.ai. I use llama-3.3-70b for the initial smoke test because it is a reliable general-purpose model.

from openai import OpenAI

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

def test_connection():
    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=[{"role": "user", "content": "Say OK"}],
        max_tokens=10,
    )
    return response.choices[0].message.content

print("Connection:", test_connection())

Step 2: Define the system prompt for localization

The system prompt is the only place where we embed rules about tone, placeholders, and glossary usage. Keeping it in one string makes the agent behavior easy to audit.

SYSTEM_PROMPT = """You are a senior localization engineer.
Translate the user's text into the target locale given below.

Rules:
- Preserve all ICU message placeholders such as {count}, {name}, and HTML tags exactly as written.
- Use the provided glossary for brand terms and technical vocabulary.
- Maintain the original tone: formal for error messages, concise for UI buttons, friendly for onboarding.
- Return ONLY the translated text. Do not add explanations or markdown code blocks.

Target locale: {target_locale}
Glossary:
{glossary}
"""

Step 3: Build the translate function with glossary injection

Next, write a helper that injects the locale and glossary into the prompt, then calls qwen-3-32b. I choose Qwen 3 32B here because Oxlo.ai hosts it with strong multilingual reasoning, and the flat per-request pricing means a long glossary does not inflate cost the way token-based billing would.

def translate_segment(text, target_locale, glossary=None):
    glossary_text = "\n".join(f"{k}: {v}" for k, v in (glossary or {}).items())
    prompt = SYSTEM_PROMPT.format(target_locale=target_locale, glossary=glossary_text)

    response = client.chat.completions.create(
        model="qwen-3-32b",
        messages=[
            {"role": "system", "content": prompt},
            {"role": "user", "content": text},
        ],
        temperature=0.3,
    )
    return response.choices[0].message.content.strip()

Step 4: Preserve ICU message placeholders during translation

Even good models occasionally rewrite placeholders. We protect them with temporary tokens before sending the text to the API, then restore them afterward.

import re

PLACEHOLDER_RE = re.compile(r"(\{[^}]+\}|<[^>]+>)")

def protect_placeholders(text):
    placeholders = {}
    def replacer(m):
        key = f"__PH{len(placeholders)}__"
        placeholders[key] = m.group(1)
        return key
    protected = PLACEHOLDER_RE.sub(replacer, text)
    return protected, placeholders

def restore_placeholders(text, placeholders):
    for key, val in placeholders.items():
        text = text.replace(key, val)
    return text

def translate_with_protection(text, target_locale, glossary=None):
    protected, placeholders = protect_placeholders(text)
    translated = translate_segment(protected, target_locale, glossary)
    return restore_placeholders(translated, placeholders)

Step 5: Add back-translation validation

We need a sanity check without hiring human reviewers for every string. I send the translated text through kimi-k2.6 to translate it back into English, then compare length ratios. If the ratio is far off, we flag the entry for manual review.

def validate_translation(original, translated, target_locale, source_locale="en-US"):
    prompt = (
        f"Translate the following {target_locale} text back into {source_locale}. "
        "Return only the back-translated text, no extra commentary."
    )
    response = client.chat.completions.create(
        model="kimi-k2.6",
        messages=[
            {"role": "system", "content": prompt},
            {"role": "user", "content": translated},
        ],
        temperature=0.3,
    )
    back = response.choices[0].message.content.strip()
    ratio = len(back) / max(len(original), 1)
    return back, 0.7 <= ratio <= 1.4

Step 6: Batch process a complete locale file

Now we wrap everything in a function that reads a source JSON file, translates every value, validates it, and writes the new locale file. Because Oxlo.ai charges per request rather than per token, batching a file with long technical paragraphs costs the same as short UI labels. See https://oxlo.ai/pricing for plan details.

import json

def localize_file(input_path, output_path, target_locale, glossary=None):
    with open(input_path, "r", encoding="utf-8") as f:
        source = json.load(f)

    localized = {}
    for key, text in source.items():
        print(f"Translating {key}...")
        translated = translate_with_protection(text, target_locale, glossary)
        back, ok = validate_translation(text, translated, target_locale)
        if not ok:
            print(f"  Warning: back-translation mismatch for {key}")
            print(f"  Original: {text}")
            print(f"  Back:     {back}")
        localized[key] = translated

    with open(output_path, "w", encoding="utf-8") as f:
        json.dump(localized, f, ensure_ascii=False, indent=2)
    print(f"Wrote {output_path}")

Run it

Save the module as localize.py, create a small source file, and run the pipeline. Here is a realistic sample and the exact output you should expect.

if __name__ == "__main__":
    glossary = {
        "Oxlo.ai": "Oxlo.ai",
        "dashboard": "Tableau de bord",
    }
    sample = {
        "welcome_message": "Welcome to Oxlo.ai, {name}!",
        "error_quota": "You have exceeded your daily limit of {count} requests.",
        "cta_upgrade": "Upgrade to Premium"
    }
    with open("en.json", "w", encoding="utf-8") as f:
        json.dump(sample, f, indent=2)

    localize_file("en.json", "fr.json", "fr-FR", glossary)

Console output:

Translating welcome_message...
Translating error_quota...
Translating cta_upgrade...
Wrote fr.json

Generated fr.json:

{
  "welcome_message": "Bienvenue sur Oxlo.ai, {name} !",
  "error_quota": "Vous avez dépassé votre limite quotidienne de {count} requêtes.",
  "cta_upgrade": "Passer à Premium"
}

Wrap-up and next steps

This agent gives you a reproducible, code-first alternative to cloud localization services. The next logical step is to wire it into a GitHub Action so every pull request that touches en.json auto-generates a diff preview for your supported locales. You could also swap in deepseek-v3.2 for code-heavy documentation, or move the glossary into a SQLite table so multiple projects share the same terminology without editing Python files.

DEV Community