DEV Community

shashank ms
shashank ms

Posted on

Unlocking the Power of LLM for Text Summarization

#learnai #oxlo #ai

We are going to build a CLI document summarizer that turns long articles, research papers, or meeting transcripts into structured bullet points and one-line takeaways. I use it to clear my reading queue without skipping the important details. It should take you about fifteen minutes to wire up.

What you'll need

pip install openai

Step 1: Initialize the Oxlo.ai client

Create a new file called summarizer.py. Oxlo.ai exposes an OpenAI-compatible endpoint, so we only need to change the base URL and plug in our key. I use Llama 3.3 70B as the default workhorse because it handles general text well and starts instantly.

from openai import OpenAI

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key="YOUR_OXLO_API_KEY",
)

# Quick connectivity check
response = client.chat.completions.create(
    model="llama-3.3-70b",
    messages=[{"role": "user", "content": "Say hello"}],
)
print(response.choices[0].message.content)

Step 2: Lock down the system prompt

The system prompt is the only logic we need. It forces the model into a strict output format: a one-line summary, bullet points, and named entities. Keeping it this tight prevents the model from adding fluff.

SYSTEM_PROMPT = """You are a precise document summarizer. Read the user text and produce exactly these sections:

1. One-liner: A single sentence summarizing the main point.
2. Key points: 3 to 5 bullet points covering the most important details.
3. Entities: A comma-separated list of key people, organizations, or technologies mentioned.

Use plain language. Do not include preamble or markdown outside the requested sections."""

Step 3: Build the core function

Wrap the API call in a clean function. It accepts a raw string, injects the system prompt, and returns the structured text. I set temperature low and max_tokens high enough to avoid truncation for typical articles.

def summarize(text: str, model: str = "llama-3.3-70b") -> str:
    if not text.strip():
        return "No text provided."

    response = client.chat.completions.create(
        model=model,
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": text},
        ],
        temperature=0.3,
        max_tokens=2048,
    )
    return response.choices[0].message.content

Step 4: Add a long-document mode

Token-based providers punish you for stuffing the entire prompt, but Oxlo.ai uses flat per-request pricing (see https://oxlo.ai/pricing). That means we can swap in a long-context model and pass the whole document in one shot without worrying about ballooning costs. For inputs up to 131K tokens, Kimi K2.6 handles the load.

def summarize_long(text: str) -> str:
    return summarize(text, model="kimi-k2.6")

If your text is shorter than a few pages, the standard Llama 3.3 70B route is fine. When you hit book-length transcripts or huge PDF pastes, flip to the Kimi route.

Run it

Here is the complete end-to-end script. I hardcoded a short article so you can run it immediately. Replace the ARTICLE string with a file read or stdin pipe for real use.

ARTICLE = """
Artificial intelligence researchers at the DeepMind lab have published a new paper on protein folding accuracy. The team, led by Dr. Sarah Chen, claims their updated algorithm reduces error rates by 18 percent compared to the previous state of the art. The work was funded by the BioTech Consortium and used a dataset of 240,000 protein structures. Critics note that the benchmark only covers soluble proteins and excludes membrane-bound structures. The code and weights were released under an Apache 2.0 license on GitHub.
"""

if __name__ == "__main__":
    print("=== Standard Summary ===")
    print(summarize(ARTICLE))

    print("\n=== Long-Context Summary ===")
    print(summarize_long(ARTICLE))

Example output:

=== Standard Summary ===
One-liner: DeepMind researchers led by Dr. Sarah Chen published an improved protein folding algorithm that cuts error rates by 18 percent, though critics highlight a limited benchmark scope.
Key points:
- The updated algorithm achieves an 18 percent lower error rate than prior state of the art methods.
- The research was conducted at DeepMind and led by Dr. Sarah Chen.
- Funding came from the BioTech Consortium.
- The dataset included 240,000 protein structures.
- The benchmark is criticized for excluding membrane-bound proteins.
Entities: DeepMind, Dr. Sarah Chen, BioTech Consortium, GitHub, Apache 2.0

=== Long-Context Summary ===
One-liner: DeepMind's team, led by Dr. Sarah Chen, released an improved protein-folding model with an 18 percent accuracy gain, funded by the BioTech Consortium and open-sourced on GitHub despite benchmark limitations.
Key points:
- The model reduces protein folding error rates by 18 percent over previous methods.
- Dr. Sarah Chen led the research at DeepMind.
- The BioTech Consortium provided funding.
- A dataset of 240,000 structures supported the study.
- The benchmark omits membrane-bound proteins, a noted limitation.
Entities: DeepMind, Dr. Sarah Chen, BioTech Consortium, GitHub, Apache 2.0

Wrap-up

Two concrete ways to push this further. First, pipe the output into a Slack bot or email handler so any shared link or pasted text gets summarized automatically. Second, flip on JSON mode in the Oxlo.ai request and parse the result with Pydantic for structured storage in a database or search index.

Top comments (0)