Using LLM for Text Classification and Clustering in Business Applications

#learnai #oxlo #ai

We are building a support ticket triage pipeline that classifies incoming messages by category and urgency, then clusters them by root cause theme so support managers can spot systemic issues. Running this on Oxlo.ai keeps costs predictable because you pay per request, not per token, which matters when tickets get long. The whole thing is a single Python script that uses the OpenAI SDK.

What you'll need

Python 3.10 or newer
The OpenAI SDK: pip install openai
Pandas for pretty printing: pip install pandas
An Oxlo.ai API key from https://portal.oxlo.ai

Step 1: Set up the Oxlo.ai client and load sample tickets

I started by creating a small set of synthetic support tickets so the script is runnable without external dependencies. I also initialize the Oxlo.ai client using the standard OpenAI-compatible base URL.

import json
import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key=os.getenv("OXLO_API_KEY", "YOUR_OXLO_API_KEY"),
)

TICKETS = [
    {"id": 1, "text": "I was charged twice for my subscription this month. Please refund the duplicate charge immediately."},
    {"id": 2, "text": "The dashboard freezes every time I click on the Reports tab. I have to refresh the browser to get back in."},
    {"id": 3, "text": "Can you add a dark mode option? It would make working at night much easier on the eyes."},
    {"id": 4, "text": "My invoice shows the wrong VAT number. I need a corrected invoice for accounting."},
    {"id": 5, "text": "The API returns a 500 error when I send payloads larger than 2MB. Is there a hard limit?"},
    {"id": 6, "text": "I would love to be able to export my data directly to BigQuery without writing a custom script."},
    {"id": 7, "text": "I was charged after I cancelled my plan last week. This is urgent."},
    {"id": 8, "text": "The mobile app crashes when I try to upload a photo. This started after the latest update."},
]

Step 2: Classify each ticket with Llama 3.3 70B

I use Llama 3.3 70B as the workhorse classifier because it is fast and follows structured instructions well. The system prompt forces JSON output, and I enable JSON mode on the request so I do not have to parse markdown fences.

CLASSIFY_PROMPT = """You are a support triage analyst.
For the ticket provided, return strictly JSON with these keys:
- category: one of Billing, Technical, Feature Request
- urgency: one of High or Low
- reason: one sentence explaining the classification

Do not wrap the JSON in markdown fences."""

def classify_ticket(ticket_text: str) -> dict:
    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=[
            {"role": "system", "content": CLASSIFY_PROMPT},
            {"role": "user", "content": f"Ticket:\n{ticket_text}"},
        ],
        response_format={"type": "json_object"},
    )
    return json.loads(response.choices[0].message.content)

Step 3: Cluster tickets by theme with Kimi K2.6

Once the tickets are classified, I want to know which issues are related so I can batch fixes. Instead of hand-writing rules, I pass the entire set to Kimi K2.6 and ask it to discover clusters by root cause. Because Oxlo.ai pricing is per request, sending the full batch of long tickets costs the same as sending short ones.

CLUSTER_PROMPT = """You are a data analyst clustering support tickets by root cause.
You will receive a JSON array of tickets. Group them into 2 to 4 thematic clusters.
Return strictly JSON with this shape:
{"clusters": [{"name": "string", "ticket_ids": [int], "summary": "string"}]}
Do not use markdown."""

def cluster_tickets(tickets: list[dict]) -> dict:
    payload = json.dumps(tickets, ensure_ascii=False)
    response = client.chat.completions.create(
        model="kimi-k2.6",
        messages=[
            {"role": "system", "content": CLUSTER_PROMPT},
            {"role": "user", "content": f"Tickets:\n{payload}"},
        ],
        response_format={"type": "json_object"},
    )
    return json.loads(response.choices[0].message.content)

Step 4: Merge the results and print a report

I run both passes, map each ticket to its cluster name, and print a table. In production you would write this to a database or webhook instead of stdout.

import pandas as pd

def run_pipeline():
    classified = []
    for t in TICKETS:
        result = classify_ticket(t["text"])
        result["id"] = t["id"]
        result["text"] = t["text"]
        classified.append(result)

    cluster_data = cluster_tickets(TICKETS)
    cluster_map = {}
    for c in cluster_data["clusters"]:
        for tid in c["ticket_ids"]:
            cluster_map[tid] = c["name"]

    for row in classified:
        row["cluster"] = cluster_map.get(row["id"], "Uncategorized")

    df = pd.DataFrame(classified)
    print(df[["id", "category", "urgency", "cluster", "reason"]].to_string(index=False))
    return df

Run it

Call the pipeline from the command line. Make sure your OXLO_API_KEY environment variable is set.

if __name__ == "__main__":
    run_pipeline()

$ python triage.py

 id       category urgency            cluster                                        reason
  1        Billing    High     Payment Issues      Duplicate charge requires immediate refund.
  2      Technical    High   Platform Stability   Dashboard freeze blocks core functionality.
  3  Feature Request   Low    UI Improvements                  User wants dark mode for comfort.
  4        Billing    High     Payment Issues   Incorrect VAT number needs corrected invoice.
  5      Technical    High   Platform Stability  API 500 error on large payloads needs investigation.
  6  Feature Request   Low  Data Integrations           User requests native BigQuery export.
  7        Billing    High     Payment Issues       Post-cancellation charge is an urgent error.
  8      Technical    High Mobile Reliability     Photo upload crash after latest app update.

Wrap-up and next steps

The pipeline is intentionally simple so you can adapt it. Two concrete moves from here: wire the script into your helpdesk webhook so it runs on every new ticket, and swap in qwen-3-32b if you need strong multilingual classification for global support queues. If volume grows, remember that Oxlo.ai flat per-request pricing keeps long-context triage affordable without token math.

DEV Community