Using LLM for Text Generation with Long Context

#learnai #oxlo #ai

We are building a long-context synthesis tool that reads an entire quarter of meeting transcripts and emits a structured project status report. If your team generates hours of notes that no one has time to reread, this automates the distillation. Because the tool sends the full text in a single request, it benefits from platforms that do not penalize long inputs.

What you'll need

Python 3.10 or newer
The OpenAI SDK: pip install openai
An Oxlo.ai API key from https://portal.oxlo.ai. The Free plan gives you 60 requests per day and a 7-day full-access trial.
A directory of markdown transcript files. I assume ./transcripts/.

Step 1: Load the transcript corpus

We walk a directory and concatenate every markdown file into one string with file headers. This gives us a single payload that can stretch to tens of thousands of tokens. That length is exactly where token-based billing hurts and Oxlo.ai's flat per-request pricing keeps the cost predictable.

import os
from pathlib import Path

def load_corpus(directory: str) -> str:
    files = sorted(Path(directory).glob("*.md"))
    parts = []
    for f in files:
        parts.append(f"--- {f.name} ---\n")
        parts.append(f.read_text(encoding="utf-8"))
        parts.append("\n\n")
    return "".join(parts)

corpus = load_corpus("./transcripts")
print(f"Loaded {len(corpus)} characters")

Step 2: Initialize the Oxlo.ai client

Oxlo.ai exposes fully OpenAI-compatible endpoints, so we use the official SDK and only change the base URL and model name. I use kimi-k2.6 here because its 131K context window handles the full transcript corpus without truncation.

from openai import OpenAI

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key="YOUR_OXLO_API_KEY"
)

Step 3: Define the system prompt

The system prompt constrains the model to output a structured report. Keeping it in a dedicated constant makes iteration easier.

SYSTEM_PROMPT = """You are a project intelligence analyst.
Read the entire transcript corpus below and produce a structured status report.
Output exactly the following sections:

1. Executive Summary (3 bullets)
2. Active Risks (list with severity: High/Medium/Low)
3. Completed Milestones (list)
4. Blocked Work (list with owner if mentioned)
5. Action Items (list with owner and due date if mentioned)

Be concise. Use only facts stated in the transcripts. Do not invent names or dates."""

Step 4: Build the synthesis call

This function assembles the full prompt and calls the model. Because Oxlo.ai charges per request rather than per token, stuffing the entire corpus into the user message costs the same whether it is ten pages or one hundred. That makes deep context analysis economically viable. See https://oxlo.ai/pricing for plan details.

user_message = f"Transcript corpus:\n\n{corpus}\n\nGenerate the structured status report."

response = client.chat.completions.create(
    model="kimi-k2.6",
    messages=[
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": user_message},
    ],
)

report = response.choices[0].message.content
print(report)

Run it

Save the script as synthesize.py, place your transcripts in ./transcripts/, and run:

export OXLO_API_KEY="sk-..."
python synthesize.py

With about 40K tokens of engineering standup notes, the output looks like this:

1. Executive Summary
- The backend migration to the new provider is on track for next Tuesday.
- Q3 hiring freeze has delayed the SRE expansion.
- The new API gateway passed load testing.

2. Active Risks
- High: Database failover script has not been tested in staging.
- Medium: Vendor contract for logging expires in 14 days.

3. Completed Milestones
- Migrated 3 legacy services to Kubernetes.

4. Blocked Work
- OAuth2 scope refactor (blocked on security review, owner: Dana).

5. Action Items
- Alex: provision failover environment by Friday.
- Dana: schedule security review for OAuth2 refactor by 2024-07-01.

Next steps

Wire the script into a weekly cron job and add response_format={"type": "json_object"} to emit machine-readable action items you can POST directly to your task tracker. If you want to expose this as a web service, move the client.chat.completions.create call into a FastAPI route and stream the response back with stream=True, which Oxlo.ai supports without cold starts.

DEV Community