Building a Language Translation Model using LLM with Long Context

We are building a long-context document translator that ingests entire multi-page files in a single request and returns a faithful translation without chunking artifacts. This is useful for developers and localization teams who need to process technical manuals, legal contracts, or long-form articles while preserving formatting and terminology.

What you'll need

Before starting, grab Python 3.10 or newer, install the OpenAI SDK, and create an Oxlo.ai API key at https://portal.oxlo.ai. You will also need a sample long-form text to translate. I use a 1,200-character technical excerpt in the examples below, but the same code works for much longer documents.

pip install openai

Step 1: Initialize the Oxlo.ai client

I always start with a small sanity check to confirm the endpoint and API key are working. The following script sends a five-word phrase to the multilingual Qwen model and prints the French translation.

from openai import OpenAI

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key="YOUR_OXLO_API_KEY"
)

response = client.chat.completions.create(
    model="qwen-3-32b",
    messages=[
        {"role": "system", "content": "Translate the user's text into French. Output only the translation."},
        {"role": "user", "content": "The server responded with a 404 error."},
    ],
)

print(response.choices[0].message.content)

Step 2: Load a long-source document

Next, prepare a document that would normally require splitting on token-based platforms. I use a raw Python multiline string so newlines and markdown are preserved exactly. You can replace this with text read from a file.

SOURCE_TEXT = """
## Rate Limiting Policy

All API clients are subject to rate limits based on their subscription tier. Free-tier accounts may submit up to sixty requests per day. Pro-tier accounts are allowed one thousand requests per day, while Premium-tier accounts receive five thousand requests per day.

When a client exceeds its allocated quota, the API returns HTTP 429 Too Many Requests. The response includes a Retry-After header indicating the number of seconds to wait before the next valid request.

Enterprise customers may request dedicated throughput with custom limits. For details, contact our sales engineering team or review the Service Level Agreement appendix.
"""

print(f"Loaded source document: {len(SOURCE_TEXT)} characters")

Step 3: Define the translation system prompt

The system prompt is the only guardrail against unwanted commentary or formatting drift. I keep it strict: no explanations, no note sections, and markdown headers must remain intact.

SYSTEM_PROMPT = """
You are a professional technical translator. Your task is to translate the user's document into Spanish.

Rules:
1. Preserve all markdown syntax, including headers, bullet points, and code formatting.
2. Maintain the original tone: formal, concise, and technical.
3. Do not add explanations, notes, or translator commentary.
4. Do not translate proper nouns such as product names, HTTP status codes, or header names.
5. Output only the translated text.
"""

print("System prompt ready")

Step 4: Translate the full document in one request

Unlike token-based providers, Oxlo.ai charges per request, so the cost is the same whether the input is ten words or ten thousand words. I pass the entire SOURCE_TEXT as a single user message and use Kimi K2.6 for its 131K context window and strong reasoning. See https://oxlo.ai/pricing for current plan details.

from openai import OpenAI

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key="YOUR_OXLO_API_KEY"
)

SOURCE_TEXT = """
## Rate Limiting Policy

All API clients are subject to rate limits based on their subscription tier. Free-tier accounts may submit up to sixty requests per day. Pro-tier accounts are allowed one thousand requests per day, while Premium-tier accounts receive five thousand requests per day.

When a client exceeds its allocated quota, the API returns HTTP 429 Too Many Requests. The response includes a Retry-After header indicating the number of seconds to wait before the next valid request.

Enterprise customers may request dedicated throughput with custom limits. For details, contact our sales engineering team or review the Service Level Agreement appendix.
"""

SYSTEM_PROMPT = """
You are a professional technical translator. Your task is to translate the user's document into Spanish.

Rules:
1. Preserve all markdown syntax, including headers, bullet points, and code formatting.
2. Maintain the original tone: formal, concise, and technical.
3. Do not add explanations, notes, or translator commentary.
4. Do not translate proper nouns such as product names, HTTP status codes, or header names.
5. Output only the translated text.
"""

response = client.chat.completions.create(
    model="kimi-k2.6",
    messages=[
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": SOURCE_TEXT},
    ],
)

print(response.choices[0].message.content)

Step 5: Stream the output for real-time feedback

For documents that generate long translations, streaming prevents you from staring at a blank terminal. The code below is identical to Step 4 except for stream=True and a loop over response chunks.

from openai import OpenAI

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key="YOUR_OXLO_API_KEY"
)

SOURCE_TEXT = """
## Rate Limiting Policy

All API clients are subject to rate limits based on their subscription tier. Free-tier accounts may submit up to sixty requests per day. Pro-tier accounts are allowed one thousand requests per day, while Premium-tier accounts receive five thousand requests per day.

When a client exceeds its allocated quota, the API returns HTTP 429 Too Many Requests. The response includes a Retry-After header indicating the number of seconds to wait before the next valid request.

Enterprise customers may request dedicated throughput with custom limits. For details, contact our sales engineering team or review the Service Level Agreement appendix.
"""

SYSTEM_PROMPT = """
You are a professional technical translator. Your task is to translate the user's document into Spanish.

Rules:
1. Preserve all markdown syntax, including headers, bullet points, and code formatting.
2. Maintain the original tone: formal, concise, and technical.
3. Do not add explanations, notes, or translator commentary.
4. Do not translate proper nouns such as product names, HTTP status codes, or header names.
5. Output only the translated text.
"""

response = client.chat.completions.create(
    model="kimi-k2.6",
    messages=[
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": SOURCE_TEXT},
    ],
    stream=True,
)

for chunk in response:
    delta = chunk.choices[0].delta.content
    if delta:
        print(delta, end="", flush=True)

Run it

Save the finished script as translate.py and run it from your terminal. You should see a Spanish translation that mirrors the original markdown structure.

python translate.py

Expected output:

## Política de Limitación de Tasa

Todos los clientes de la API están sujetos a límites de tasa basados en su nivel de suscripción. Las cuentas de nivel gratuito pueden enviar hasta sesenta solicitudes por día. Las cuentas de nivel Pro pueden enviar mil solicitudes por día, mientras que las cuentas de nivel Premium reciben cinco mil solicitudes por día.

Cuando un cliente excede su cuota asignada, la API devuelve HTTP 429 Too Many Requests. La respuesta incluye un encabezado Retry-After que indica el número de segundos que debe esperar antes de la siguiente solicitud válida.

Los clientes Enterprise pueden solicitar rendimiento dedicado con límites personalizados. Para más detalles, contacte a nuestro equipo de ingeniería de ventas o revise el apéndice del Acuerdo de Nivel de Servicio.

Wrapping up

You now have a working long-context translator on Oxlo.ai. Two concrete next steps: first, extend the script to accept a --glossary flag that injects domain-specific terms into the system prompt for consistent terminology across large projects. Second, wrap the logic in a FastAPI endpoint so other services can submit documents and receive translations via webhook.