We are building a support ticket clustering pipeline that groups unstructured customer feedback into labeled themes using an LLM. Unlike traditional embedding and KMeans workflows, this approach keeps all semantic reasoning in the model, so you do not lose nuance to dimensionality reduction. It is useful for support teams, product managers, or anyone who needs to make sense of hundreds of raw text snippets without maintaining a vector database.
What you'll need
- Python 3.10 or newer
- The OpenAI SDK:
pip install openai - An Oxlo.ai API key from https://portal.oxlo.ai
- A set of raw text items to cluster. We will generate synthetic support tickets in the script.
Every step below calls Oxlo.ai through the OpenAI-compatible client. Because Oxlo.ai charges a flat rate per request, you can send long batched prompts and make multiple labeling calls without watching token meters tick up. You can see the details at https://oxlo.ai/pricing.
Here is the system prompt we reuse across all calls. It forces strict JSON output so we can parse results programmatically.
SYSTEM_PROMPT = """You are a support analytics assistant. You analyze customer feedback and return only valid JSON. Do not wrap your output in markdown code blocks and do not add conversational text outside the JSON structure."""Step 1: Initialize the client and load feedback
First, point the OpenAI SDK at Oxlo.ai and create a small dataset of synthetic support tickets. In production you would load these from a CSV or your help desk API.
from openai import OpenAI
import json
from collections import defaultdict
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
tickets = [
"I can't log in after the latest update. It says password expired but I just changed it.",
"The export to CSV button is missing from the dashboard.",
"Login page freezes on Safari after entering 2FA code.",
"How do I change my billing address?",
"CSV export only includes the first 100 rows, need all data.",
"The mobile app crashes when I open settings.",
"Can I get an invoice for last month?",
"Settings page on Android crashes every time.",
"Where do I update my credit card?",
"Dashboard charts are blank on Firefox.",
"I want to cancel my subscription.",
"Blank charts in Firefox, but Chrome works fine.",
"Refund request for duplicate charge.",
"Two-factor authentication SMS not arriving.",
"Need to add seats to my team plan.",
"PDF report generation times out.",
"Team billing is confusing, how do I add members?",
"Timeout when generating large PDFs.",
"Is there a discount for annual billing?",
"PDF export fails for reports over 50 pages.",
]Step 2: Extract cluster themes
We send the full ticket list to the model once and ask it to propose a small set of clusters with descriptions. The response includes example ticket indices so we can verify coherence.
def extract_themes(tickets):
ticket_block = "\n".join(f"{i}: {t}" for i, t in enumerate(tickets))
user_message = f"""Here are support tickets (index: text):
{ticket_block}
Propose 3 to 5 clusters that capture distinct themes. Return a JSON object with a key "clusters" containing a list of objects. Each object must have:
- "id": integer
- "name": short string
- "description": one sentence
- "example_indices": list of integers from the ticket list above"""
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": user_message},
],
)
raw = response.choices[0].message.content.strip()
if raw.startswith("
```"):
raw = raw.split("\n", 1)[1]
if raw.endswith("```
"):
raw = raw.rsplit("\n", 1)[0]
return json.loads(raw.strip())
theme_data = extract_themes(tickets)
print(json.dumps(theme_data, indent=2))Step 3: Assign every ticket to a cluster
Now we pass the cluster definitions back to the model together with every ticket and ask for a strict assignment. The prompt is long because it contains the full context, but on Oxlo.ai the cost is the same flat per-request rate regardless of input length.
def assign_clusters(tickets, clusters):
ticket_block = "\n".join(f"{i}: {t}" for i, t in enumerate(tickets))
cluster_block = json.dumps(clusters, indent=2)
user_message = f"""Clusters:
{cluster_block}
Tickets:
{ticket_block}
Assign every ticket to the best fitting cluster. Return a JSON object with a key "assignments" containing a list of objects. Each object must have:
- "ticket_index": integer
- "cluster_id": integer or null if none fit
- "reason": one sentence explaining the choice"""
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": user_message},
],
)
raw = response.choices[0].message.content.strip()
if raw.startswith("
```"):
raw = raw.split("\n", 1)[1]
if raw.endswith("```
"):
raw = raw.rsplit("\n", 1)[0]
return json.loads(raw.strip())["assignments"]
assignments = assign_clusters(tickets, theme_data["clusters"])
print(f"Assigned {len(assignments)} tickets.")Step 4: Label and describe each cluster
Finally, we iterate over each cluster and call the model once per group to generate a concise title and a one sentence insight. This loop creates multiple API calls, but because Oxlo.ai uses request-based pricing, the total cost stays predictable even when every cluster contains dozens of tickets.
def summarize_cluster(cluster_name, tickets_in_cluster):
ticket_block = "\n".join(f"- {t}" for t in tickets_in_cluster)
user_message = f"""Cluster: {cluster_name}
Tickets:
{ticket_block}
Provide a JSON object with:
- "label": a concise 3 to 5 word title
- "insight": one sentence describing the underlying issue"""
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": user_message},
],
)
raw = response.choices[0].message.content.strip()
if raw.startswith("
```"):
raw = raw.split("\n", 1)[1]
if raw.endswith("```
"):
raw = raw.rsplit("\n", 1)[0]
return json.loads(raw.strip())
# Group tickets by cluster
cluster_tickets = defaultdict(list)
for a in assignments:
if a["cluster_id"] is not None:
cluster_tickets[a["cluster_id"]].append(tickets[a["ticket_index"]])
summaries = {}
for c in theme_data["clusters"]:
cid = c["id"]
if cid in cluster_tickets:
summaries[cid] = summarize_cluster(c["name"], cluster_tickets[cid])
for cid, s in summaries.items():
print(f"Cluster {cid}: {s['label']} -> {s['insight']}")Run it
Putting it all together, the main block below discovers themes, assigns tickets, and prints labeled clusters with their member tickets.
if __name__ == "__main__":
theme_data = extract_themes(tickets)
assignments = assign_clusters(tickets, theme_data["clusters"])
cluster_tickets = defaultdict(list)
for a in assignments:
if a["cluster_id"] is not None:
cluster_tickets[a["cluster_id"]].append(tickets[a["ticket_index"]])
for c in theme_data["clusters"]:
cid = c["id"]
if cid in cluster_tickets:
summary = summarize_cluster(c["name"], cluster_tickets[cid])
print(f"\nCluster {cid}: {summary['label']}")
print(f"Insight: {summary['insight']}")
print("Tickets:")
for t in cluster_tickets[cid]:
print(f" - {t}")Example output:
Cluster 1: Authentication and Login Failures
Insight: Users are experiencing repeated login and 2FA issues across browsers and after updates.
Tickets:
- I can't log in after the latest update. It says password expired but I just changed it.
- Login page freezes on Safari after entering 2FA code.
- Two-factor authentication SMS not arriving.
Cluster 2: CSV and Data Export Limitations
Insight: Customers need complete data exports and are hitting row limits or missing UI elements.
Tickets:
- The export to CSV button is missing from the dashboard.
- CSV export only includes the first 100 rows, need all data.
Cluster 3: Mobile App Crashes
Insight: The settings page crashes consistently on Android devices.
Tickets:
- The mobile app crashes when I open settings.
- Settings page on Android crashes every time.
Cluster 4: Billing and Subscription Questions
Insight: Users frequently request invoice changes, refunds, and team seat modifications.
Tickets:
- How do I change my billing address?
- Can I get an invoice for last month?
- Refund request for duplicate charge.
- Need to add seats to my team plan.
Cluster 5: PDF Report Timeouts
Insight: Large PDF exports fail or time out when handling sizable reports.
Tickets:
- PDF report generation times out.
- Timeout when generating large PDFs.
- PDF export fails for reports over 50 pages.Wrap-up and next steps
The pipeline above gives you fully labeled clusters without managing embedding models or vector indexes. If you want to productionize it, consider adding an outlier detection pass for tickets that map to null clusters, or schedule the script as a nightly job that pulls from your help desk API and posts the resulting themes to a Slack channel.
Top comments (0)