Text summarization is one of the most common production workloads for large language models. Whether you are condensing support tickets, legal documents, or research papers, the underlying cost structure matters just as much as model quality. Token-based billing penalizes long inputs, which makes summarization inherently expensive when you feed entire documents into the context window. Oxlo.ai takes a different approach with flat per-request pricing, so the cost of summarizing a 100-word email and a 10,000-word report is identical. This guide shows you how to implement text summarization using the OpenAI SDK with Oxlo.ai as your inference backend.
Why Request-Based Pricing Changes Summarization Economics
Most inference providers bill by the token. For summarization, the input is often the longest part of the prompt. A single long-context request on token-based platforms can cost as much as dozens of short queries. Oxlo.ai charges one flat rate per API request regardless of prompt length. For teams processing lengthy articles, transcripts, or agentic chains that repeatedly summarize memory buffers, this can reduce costs significantly compared to token-based providers such as Together AI, Fireworks AI, OpenRouter, Replicate, or Anyscale. You can see current plan details at https://oxlo.ai/pricing.
Setting Up the OpenAI SDK with Oxlo.ai
Oxlo.ai is fully OpenAI SDK compatible. You only need to change the base URL and API key. Install the official OpenAI Python package if you have not already.
pip install openaiConfigure the client to point to Oxlo.ai.
from openai import OpenAI
client = OpenAI(
base_url="https://api.oxlo.ai/v1",
api_key="YOUR_OXLO_API_KEY"
)Basic Summarization Example
A simple summarization prompt works without any changes to your existing code. Here is a minimal example using Llama 3.3 70B, the general-purpose flagship model on Oxlo.ai.
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{
"role": "system",
"content": "You are a concise summarization assistant. Extract the main points in three bullet points."
},
{
"role": "user",
"content": """Paste your long article or document text here..."""
}
],
temperature=0.3,
max_tokens=512
)
print(response.choices[0].message.content)Handling Long Documents
Summarization only works if the entire source fits in the context window. Oxlo.ai hosts several models with extended context lengths that are ideal for this task. DeepSeek V4 Flash supports a 1 million token context window and uses an efficient MoE architecture. Kimi K2.6 offers a 131K context with advanced reasoning and vision capabilities, so you can even summarize documents that contain embedded charts or screenshots. Because Oxlo.ai does not scale cost with input length, you can pass the full document in a single request instead of chunking it and losing coherence.
response = client.chat.completions.create(
model="deepseek-v4-flash",
messages=[
{
"role": "user",
"content": f"Summarize the following long document in one paragraph:\n\n{document_text}"
}
],
temperature=0.2
)Structured Output with JSON Mode
Production pipelines rarely want free text. They need structured data: a title, key points, and sentiment. Oxlo.ai supports JSON mode, which you can enable with a simple parameter. This is useful when feeding summarization output into downstream tools or databases.
response = client.chat.completions.create(
model="qwen3-32b",
messages=[
{
"role": "system",
"content": "You summarize articles into JSON. Use keys: title, summary, topics, sentiment."
},
{
"role": "user",
"content": article_text
}
],
response_format={"type": "json_object"},
temperature=0.2
)
import json
structured = json.loads(response.choices[0].message.content)Selecting the Right Model
Oxlo.ai offers more than 45 models across 7 categories. For summarization, the best choice depends on your input language, length, and reasoning requirements.
- Llama 3.3 70B: Reliable general-purpose summaries in English and major languages.
- Qwen 3 32B: Strong multilingual reasoning and agent workflows for cross-language documents.
- DeepSeek R1 671B MoE: Use this when the source material requires deep reasoning, such as technical specifications or legal contracts.
- Kimi K2.6: Advanced reasoning with a 131K context window and vision support for mixed-media reports.
- DeepSeek V4 Flash: The best option for very long inputs thanks to its 1M context and efficient MoE design.
All of these models are available with no cold starts, so the first request of the day returns just as quickly as the hundredth.
Streaming and Production Considerations
For user-facing applications, waiting for the full summary can feel slow. Oxlo.ai supports streaming responses, so you can display text as it is generated.
stream = client.chat.completions.create(
model="llama-3.3-70b",
messages=[{"role": "user", "content": f"Summarize:\n{text}"}],
stream=True
)
for chunk in stream:
if chunk.choices[0].delta.content:
print(chunk.choices[0].delta.content, end="")If you are building an agentic system, you can combine summarization with function calling. Have the model summarize a tool result before passing it to the next step, keeping context windows manageable without increasing your per-request cost on Oxlo.ai.
Next Steps
Text summarization is a workload where input length directly impacts cost on traditional token-based platforms. Oxlo.ai removes that variable with flat per-request pricing, making it a strong choice for teams that process long documents or run high-volume summarization pipelines. Sign up for a free account to get 60 requests per day and access to more than 16 free models, including a 7-day full-access trial. When you are ready to scale, the Pro and Premium plans offer 1,000 and 5,000 requests per day respectively, with priority queue access on Premium. Enterprise plans offer custom contracts with unlimited requests, dedicated GPUs, and guaranteed 30% off your current provider. Visit https://oxlo.ai/pricing to compare plans, and point your OpenAI SDK to https://api.oxlo.ai/v1 to start summarizing.
Top comments (0)