We are going to build a document summarizer that ingests long text files and returns structured, extractive summaries. It helps teams processing reports, transcripts, or research papers who need consistent output without managing token budgets. Because Oxlo.ai charges a flat rate per request, running a 5,000-character chunk costs the same as a 500-character chunk, so you do not pay more as the document grows.
What you'll need
Python 3.10 or newer, the OpenAI SDK (pip install openai), and an Oxlo.ai API key from https://portal.oxlo.ai. You can start on the free tier and upgrade once you move past the daily request limit.
Step 1: Initialize the Oxlo.ai client
I always verify the connection before I build logic. This snippet creates the client and runs a one-word sanity check.
from openai import OpenAI
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[{"role": "user", "content": "Say OK"}],
max_tokens=10
)
print(response.choices[0].message.content)
Step 2: Load and chunk the source document
In production you will read from PDF or markdown files. For this tutorial I embedded a sample technical report and split it on paragraph boundaries so no sentence gets cut in half.
DOCUMENT = """Project Aurora: Technical Post-Mortem
Introduction
In Q3 2024 the platform team migrated the primary ingestion pipeline from Kafka to Redpanda. The goal was to reduce p99 latency for event streaming under bursty load. This document reviews the architecture decisions, failure modes observed during rollout, and the remediation steps taken between August 12 and September 3.
Architecture Changes
We replaced the 12-node Kafka 3.5 cluster with a 6-node Redpanda 23.2 cluster using tiered storage backed by S3. Producers remained unchanged. Consumers were upgraded to use the rpk client with idempotent producers enabled. The topic count stayed flat at 340 topics, but replication factor dropped from 3 to 2 because Redpanda handles raft consensus differently.
Failure Modes
On August 19 traffic spiked to 4.2M messages per second during a partner data backfill. Two nodes entered an out-of-memory state because the default memory allocation for the wasm engine was uncapped. This caused a 7-minute partial outage in the us-east-1 region. Separately, we discovered that the S3 tiered-storage timeout was set to 5 seconds, which was too aggressive for cross-AZ transfers and caused log replication lag.
Remediation
We capped the wasm engine at 4 GB per node and added memory alerts at 80 percent utilization. The S3 timeout was raised to 30 seconds and the tiered-storage upload concurrency limit was reduced from 64 to 32 to prevent thundering herd against object storage. After these changes p99 latency stabilized at 12 ms, down from 45 ms under Kafka.
Conclusion
The migration succeeded but required two weeks of tuning. The main lesson is that default Redpanda settings are not safe for multi-tenant burst workloads without explicit resource guardrails."""
def chunk_text(text, max_chars=6000):
paragraphs = text.split("\n\n")
chunks, current = [], ""
for p in paragraphs:
candidate = current + "\n\n" + p if current else p
if len(candidate) <= max_chars:
current = candidate
else:
chunks.append(current)
current = p
if current:
chunks.append(current)
return chunks
chunks = chunk_text(DOCUMENT)
print(f"Created {len(chunks)} chunks")
Step 3: Define the summarization system prompt
I keep the prompt strict and structured. It asks for JSON so downstream tools can parse the result without regex.
SYSTEM_PROMPT = """You are a technical summarizer. Your job is to read a document section and produce a structured summary.
Rules:
- Extract only facts that appear in the text. Do not infer or hallucinate.
- Identify up to 3 key points, 1 risk or issue, and 1 outcome or metric.
- Output strictly as JSON with this schema:
{
"key_points": ["string"],
"risks_or_issues": ["string"],
"outcomes_or_metrics": ["string"],
"one_sentence_summary": "string"
}
- Use concise, neutral language.
- If a field has no relevant information, return an empty list or an empty string."""
Step 4: Summarize each chunk
This is the map step. I call Oxlo.ai once per chunk. Because Oxlo.ai pricing is per request, not per token, sending a 5,000-character chunk costs the same as sending a 500-character chunk. I set temperature low to keep the output deterministic.
import json
def summarize_chunk(text):
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": f"Summarize this section:\n\n{text}"},
],
response_format={"type": "json_object"},
temperature=0.1,
)
raw = response.choices[0].message.content
return json.loads(raw)
chunk_summaries = [summarize_chunk(c) for c in chunks]
for i, s in enumerate(chunk_summaries, 1):
print(f"Chunk {i}: {s['one_sentence_summary']}")
Step 5: Reduce partial summaries into a final report
The reduce step feeds all partial JSON summaries back to the model and asks for a single merged summary. This two-stage approach keeps each call focused and avoids losing details that would otherwise be buried in a single massive prompt.
def reduce_summaries(summaries):
combined = "\n\n".join(
f"Partial summary {i+1}:\n{json.dumps(s, indent=2)}"
for i, s in enumerate(summaries)
)
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": f"Synthesize these partial summaries into one final summary. Preserve all unique key points, risks, and outcomes:\n\n{combined}"},
],
response_format={"type": "json_object"},
temperature=0.1,
)
return json.loads(response.choices[0].message.content)
final = reduce_summaries(chunk_summaries)
print(json.dumps(final, indent=2))
Run it
Putting it together, the full script reads the document, maps over chunks, and reduces the results. When I ran this against the sample post-mortem, the output looked like this:
Created 2 chunks
Chunk 1: The platform team migrated from Kafka to Redpanda in Q3 2024 to reduce p99 latency, replacing a 12-node Kafka cluster with a 6-node Redpanda cluster using tiered S3 storage.
Chunk 2: After remediation steps including capping wasm memory and adjusting S3 timeouts, p99 latency stabilized at 12 ms and the migration was considered successful with lessons learned about default settings.
{
"key_points": [
"Migrated ingestion pipeline from Kafka to Redpanda in Q3 2024",
"Replaced 12-node Kafka 3.5 cluster with 6-node Redpanda 23.2 cluster with tiered S3 storage",
"Capped wasm engine memory at 4 GB per node and raised S3 timeout to 30 seconds"
],
"risks_or_issues": [
"Two nodes entered OOM during a 4.2M messages per second spike due to uncapped wasm engine memory",
"S3 tiered-storage timeout was too aggressive for cross-AZ transfers"
],
"outcomes_or_metrics": [
"p99 latency stabilized at 12 ms, down from 45 ms under Kafka",
"7-minute partial outage occurred in us-east-1 before remediation"
],
"one_sentence_summary": "The Q3 2024 migration from Kafka to Redpanda reduced p99 latency from 45 ms to 12 ms after resolving OOM and S3 timeout issues caused by unsafe default settings."
}
Next steps
Swap llama-3.3-70b for kimi-k2.6 or qwen-3-32b if you are processing multilingual documents or need the larger 131K context window. You can also replace the inline DOCUMENT string with a PyPDFLoader or unstructured parser to handle PDF and Word files in production.
Top comments (0)