Building a Language Translator with LLM

Building a language translator with an LLM is one of the most practical ways to put generative AI into production. Modern models capture tone, idioms, and domain terminology far better than traditional phrase-based systems, but running a translation service at scale introduces a cost problem. On token-based providers, a single long document can consume thousands of tokens in both the prompt and the completion, which means your bill grows with every paragraph you send. Oxlo.ai approaches this differently with flat, request-based pricing: one fixed cost per API call regardless of input length. That makes it a natural fit for translation pipelines where source text is often long and unpredictable.

Why LLMs for Translation

Rule-based and statistical machine translation systems break down when context spans more than a few words. LLMs excel because they attend to entire paragraphs, preserve gender agreement across sentences, and adapt tone based on a system prompt. You can instruct the model to produce formal legal Spanish, casual Brazilian Portuguese, or technical Japanese that respects your company's glossary. This flexibility comes at a compute cost, which is why the pricing model of your inference provider matters as much as the model itself.

Project Architecture

A minimal translator has three parts: a prompt builder that injects source text and instructions, an LLM client that handles the API call, and a post-processor that cleans the output. For production, you will also want a chunking strategy for documents that exceed context limits, though Oxlo.ai hosts models with 131K and even 1M context windows that reduce the need to slice text.

Code Walkthrough

The example below uses the OpenAI SDK with Oxlo.ai as a drop-in replacement. Because Oxlo.ai is fully OpenAI SDK compatible, you only need to change the base_url and API key.

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key=os.environ.get("OXLO_API_KEY")
)

def translate(
    text: str,
    source_lang: str,
    target_lang: str,
    tone: str = "neutral"
) -> str:
    system_msg = (
        f"You are a professional translator. Translate the provided text "
        f"from {source_lang} to {target_lang}. Maintain a {tone} tone. "
        "Preserve all formatting, markdown, and code blocks. "
        "Do not add explanations or preambles."
    )

    response = client.chat.completions.create(
        model="qwen-3-32b",  # multilingual model available on Oxlo.ai
        messages=[
            {"role": "system", "content": system_msg},
            {"role": "user", "content": text}
        ],
        temperature=0.3,
        stream=True
    )

    output = ""
    for chunk in response:
        delta = chunk.choices[0].delta.content
        if delta:
            output += delta
    return output

# Example usage
long_text = (
    "The API handles authentication via OAuth 2.0. "
    "Tokens expire after 3600 seconds and must be refreshed..."
)
result = translate(long_text, "English", "German", tone="technical")
print(result)

Streaming responses let you start rendering translated text immediately rather than waiting for the full completion. This is especially useful in web interfaces where users expect near-instant feedback.

Handling Long Context

Translation workloads are inherently long-context. A technical manual or legal contract can easily run to tens of thousands of tokens. On token-based platforms, that input length directly multiplies your cost. Oxlo.ai uses request-based pricing, so a 500-word email and a 50-page report incur the same flat cost per API request. This can make Oxlo.ai significantly cheaper for long-context workloads, and it removes the penalty for including few-shot examples or large system prompts.

If you do need to chunk, a simple overlap strategy preserves coherence across boundaries:

def chunk_text(text: str, max_chars: int = 4000, overlap: int = 200):
    chunks = []
    start = 0
    while start < len(text):
        end = start + max_chars
        chunks.append(text[start:end])
        start = end - overlap
    return chunks

Because Oxlo.ai offers models such as DeepSeek V4 Flash with a 1M context window and Kimi K2.6 with 131K context, many documents fit in a single request without chunking at all.

Structured Output and Quality Control

Raw text completion is only the first step. For production, you often need the model to return metadata alongside the translation, such as detected terminology, confidence scores, or a back-translation for verification. Oxlo.ai supports JSON mode and function calling, so you can constrain the output to a schema.

import json

def translate_structured(text: str, source_lang: str, target_lang: str):
    schema_prompt = (
        "Return a JSON object with the keys: 'translation' (string), "
        "'terminology' (list of objects with 'term' and 'translation'), "
        "and 'confidence' (string: high, medium, or low)."
    )

    response = client.chat.completions.create(
        model="qwen-3-32b",
        messages=[
            {"role": "system", "content": f"You are a translator. {schema_prompt}"},
            {"role": "user", "content": f"Translate from {source_lang} to {target_lang}:\n\n{text}"}
        ],
        response_format={"type": "json_object"},
        temperature=0.2
    )

    return json.loads(response.choices[0].message.content)

This pattern is useful for building review queues or automated QA pipelines without extra parsing logic.

Deployment Considerations

Latency and reliability matter when translation is part of a user-facing workflow. Oxlo.ai streams responses and has no cold starts on popular models, which keeps latency predictable. For rate limits, the Oxlo.ai free tier offers 60 requests per day across 16+ models, enough for prototyping. The Pro plan provides 1,000 requests per day, and Premium offers 5,000 requests per day with priority queueing. If you are migrating from a token-based provider, the Enterprise plan includes dedicated GPUs and a guaranteed 30% savings over your current bill. See https://oxlo.ai/pricing for details.

Conclusion

Building a language translator with an LLM is straightforward, but scaling it cost-effectively requires attention to pricing mechanics. Token-based billing punishes long inputs, which are the norm in translation. Oxlo.ai flips this with flat per-request pricing, broad multilingual model support, and full OpenAI SDK compatibility. Whether you are translating short support tickets or hundred-page manuals, Oxlo.ai is a relevant, cost-efficient backend to consider.