Building a Language Translation System with LLM

Language translation remains one of the most practical applications for large language models, but production pipelines often stall on unpredictable token costs. Long documents, multilingual chat histories, and agentic pre-processing steps inflate input tokens quickly on traditional providers. Oxlo.ai approaches this differently with flat per-request pricing, making it straightforward to budget for high-volume translation workloads without scaling costs tied to prompt length.

Architecture of an LLM Translation Pipeline

A production translation system usually follows a simple pipeline: ingest source content, construct a system prompt with translation instructions, call an LLM, and parse the output. For long documents, you may split text into chunks, though Oxlo.ai hosts models with context windows up to 1 million tokens, so you can often send full pages or entire conversations in a single request. Because Oxlo.ai charges per request rather than per token, increasing prompt length to include glossaries, style guides, or prior conversation history does not change the inference cost. This predictability simplifies architecture decisions around context window usage.

Selecting a Multilingual Model

Model choice depends on language coverage, reasoning depth, and context length. On Oxlo.ai, several options stand out for translation tasks:

• Qwen 3 32B: Built for multilingual reasoning and agent workflows, it handles nuanced grammar and low-resource languages well.
• Llama 3.3 70B: A general-purpose flagship that delivers reliable results across common European and Asian language pairs.
• DeepSeek V4 Flash: An efficient mixture-of-experts model with a 1 million token context window, useful for translating entire documents or large codebases without chunking.
• Kimi K2.6: Supports 131K context and vision, making it suitable for mixed-modal translation pipelines.

All of these are accessible through the same OpenAI-compatible chat completions endpoint, so switching models requires changing a single string.

Setting Up the OpenAI SDK with Oxlo.ai

Oxlo.ai is a drop-in replacement for the OpenAI SDK. Point your client at https://api.oxlo.ai/v1 and use your Oxlo.ai API key.

from openai import OpenAI

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key="YOUR_OXLO_API_KEY"
)

response = client.chat.completions.create(
    model="qwen3-32b",
    messages=[
        {
            "role": "system",
            "content": (
                "You are a professional translator. "
                "Translate the user's text from Spanish to English, "
                "preserving tone and formatting."
            )
        },
        {
            "role": "user",
            "content": "El contrato entrará en vigor el próximo mes."
        }
    ]
)

print(response.choices[0].message.content)

The response streams back with no cold starts, so latency is consistent even under load.

Handling Long Documents and Batch Jobs

Translation workloads often involve long inputs: legal contracts, technical manuals, or chat logs. On token-based providers, these documents generate significant input charges before the model generates a single translated word. Oxlo.ai’s request-based pricing removes that penalty. A request containing 500 tokens costs the same as one containing 50,000 tokens, which can make long-context translation 10 to 100 times cheaper than token-based alternatives. For the largest inputs, DeepSeek V4 Flash’s 1M context window lets you pass a full book chapter in one shot. See https://oxlo.ai/pricing for current plan details.

Structured Output with JSON Mode

Production systems rarely need raw text alone. JSON mode lets you enforce structured output for downstream processing, such as extracting detected language, segment IDs, or confidence scores alongside the translation.

import json

response = client.chat.completions.create(
    model="qwen3-32b",
    messages=[
        {
            "role": "system",
            "content": (
                "Translate the text and return a JSON object with the keys "
                "'translated_text' and 'detected_source_language'. "
                "Do not include markdown formatting."
            )
        },
        {
            "role": "user",
            "content": "Bonjour, comment allez-vous aujourd'hui?"
        }
    ],
    response_format={"type": "json_object"}
)

result = json.loads(response.choices[0].message.content)
print(result["translated_text"])
print(result["detected_source_language"])

This pattern pairs well with function calling if you need to route translated segments to databases, notification queues, or other tools.

Extending to Audio and Vision

Translation is not limited to raw text. Oxlo.ai runs Whisper Large v3 and Turbo for audio transcription, so you can transcribe speech and then feed the resulting text into the same chat completions pipeline. For images containing foreign text, vision models such as Gemma 3 27B or Kimi VL A3B can perform optical character recognition and translation in a single request. These additional modalities use the same API key and base URL, keeping integration overhead minimal.

Why Oxlo.ai Fits Translation Workloads

Oxlo.ai gives translation developers three concrete advantages. First, flat per-request pricing eliminates the cost spikes that come with long-context or high-volume batch inference. Second, full OpenAI SDK compatibility means you can prototype with existing code and switch endpoints without rewriting clients. Third, the catalog of 45+ models, including multilingual specialists like Qwen 3 32B and long-context options like DeepSeek V4 Flash, covers most translation requirements without external providers. If you are currently on a token-based platform, the Oxlo.ai Enterprise plan also guarantees 30% off your current provider. To explore which plan matches your volume, visit https://oxlo.ai/pricing.