DEV Community

shashank ms
shashank ms

Posted on

Building a Text Summarization App with LLM

We are building a command-line text summarizer that takes long articles and returns structured bullet summaries. I built this to process daily news digests and internal wiki pages without worrying about token-based pricing cliffs. Because Oxlo.ai charges a flat rate per request, a 10,000-word document costs the same as a tweet-sized prompt.

What you'll need

Step 1: Configure the Oxlo.ai client and system prompt

Create a file named summarizer.py. Start by importing the SDK and locking the system prompt so the model returns exactly three bullets with no fluff.

from openai import OpenAI

SYSTEM_PROMPT = """You are a precise summarization engine.
Read the user text and output exactly 3 bullet points.
Each bullet must start with a hyphen and be under 20 words.
Do not add preamble or commentary."""

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key="YOUR_OXLO_API_KEY"
)

Step 2: Build the core summarization function

This helper sends raw text to Llama 3.3 70B and returns the generated bullets. I keep temperature low to reduce hallucinations.

from openai import OpenAI

SYSTEM_PROMPT = """You are a precise summarization engine.
Read the user text and output exactly 3 bullet points.
Each bullet must start with a hyphen and be under 20 words.
Do not add preamble or commentary."""

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

def summarize(text: str, model: str = "llama-3.3-70b") -> str:
    response = client.chat.completions.create(
        model=model,
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": f"Summarize this text:\n\n{text}"},
        ],
        temperature=0.2,
        max_tokens=256,
    )
    return response.choices[0].message.content.strip()

Step 3: Handle long inputs with chunking

Real articles often exceed comfortable context limits, so I split on paragraphs and summarize each chunk. A second pass merges the partial summaries into a final set of bullets. This avoids truncation and keeps the cost flat on Oxlo.ai because each API call is one request regardless of length.

from openai import OpenAI

SYSTEM_PROMPT = """You are a precise summarization engine.
Read the user text and output exactly 3 bullet points.
Each bullet must start with a hyphen and be under 20 words.
Do not add preamble or commentary."""

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

def chunk_text(text: str, max_chars: int = 4000) -> list[str]:
    paragraphs = text.split("\n\n")
    chunks, current = [], ""
    for p in paragraphs:
        if len(current) + len(p) < max_chars:
            current += "\n\n" + p if current else p
        else:
            chunks.append(current)
            current = p
    if current:
        chunks.append(current)
    return chunks

def summarize_long(text: str, model: str = "llama-3.3-70b") -> str:
    chunks = chunk_text(text)
    partials = [summarize(chunk, model) for chunk in chunks]
    combined = "\n\n".join(partials)
    merge_prompt = (
        "Combine the following bullet summaries into exactly 3 final bullets. "
        "Remove duplicates and keep under 20 words each.\n\n" + combined
    )
    response = client.chat.completions.create(
        model=model,
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": merge_prompt},
        ],
        temperature=0.2,
        max_tokens=256,
    )
    return response.choices[0].message.content.strip()

Step 4: Add a CLI entrypoint

I use argparse to accept a file path and print the result. This makes the tool usable in shell pipelines and cron jobs.

import argparse
from openai import OpenAI

SYSTEM_PROMPT = """You are a precise summarization engine.
Read the user text and output exactly 3 bullet points.
Each bullet must start with a hyphen and be under 20 words.
Do not add preamble or commentary."""

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

def chunk_text(text: str, max_chars: int = 4000) -> list[str]:
    paragraphs = text.split("\n\n")
    chunks, current = [], ""
    for p in paragraphs:
        if len(current) + len(p) < max_chars:
            current += "\n\n" + p if current else p
        else:
            chunks.append(current)
            current = p
    if current:
        chunks.append(current)
    return chunks

def summarize(text: str, model: str = "llama-3.3-70b") -> str:
    response = client.chat.completions.create(
        model=model,
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": f"Summarize this text:\n\n{text}"},
        ],
        temperature=0.2,
        max_tokens=256,
    )
    return response.choices[0].message.content.strip()

def summarize_long(text: str, model: str = "llama-3.3-70b") -> str:
    chunks = chunk_text(text)
    partials = [summarize(chunk, model) for chunk in chunks]
    combined = "\n\n".join(partials)
    merge_prompt = (
        "Combine the following bullet summaries into exactly 3 final bullets. "
        "Remove duplicates and keep under 20 words each.\n\n" + combined
    )
    response = client.chat.completions.create(
        model=model,
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": merge_prompt},
        ],
        temperature=0.2,
        max_tokens=256,
    )
    return response.choices[0].message.content.strip()

if __name__ == "__main__":
    parser = argparse.ArgumentParser(description="Summarize text files with Oxlo.ai")
    parser.add_argument("file", help="Path to a text file")
    parser.add_argument(
        "--model",
        default="llama-3.3-70b",
        choices=["llama-3.3-70b", "qwen-3-32b", "kimi-k2.6", "deepseek-v3.2"],
        help="Oxlo.ai model to use",
    )
    args = parser.parse_args()

    with open(args.file, "r", encoding="utf-8") as f:
        content = f.read()

    if len(content) > 4000:
        result = summarize_long(content, args.model)
    else:
        result = summarize(content, args.model)

    print(result)

Run it

Save a sample article to article.txt, then run the script.

python summarizer.py article.txt --model llama-3.3-70b

Example output:

- Scientists discover a new species of deep-sea jellyfish near the Mariana Trench.
- The creature uses bioluminescence to lure prey in total darkness.
- Researchers plan to sequence its genome by the end of the year.

Wrap-up and next steps

Swap in qwen-3-32b for multilingual documents, or kimi-k2.6 when the source material mixes images and text. If you plan to process thousands of articles, the flat per-request pricing on Oxlo.ai keeps costs predictable regardless of article length.

Top comments (0)