Topic modeling with traditional LDA leaves you staring at word lists that do not quite map to reality. In this guide we will build a working pipeline that feeds documents to an LLM, extracts human-readable candidate topics, consolidates them into a clean taxonomy, and assigns each document to its best match. It is useful for anyone organizing support tickets, research archives, or content feeds without hand-crafting a category list.
What you'll need
- Python 3.10+
pip install openai- An Oxlo.ai API key from https://portal.oxlo.ai
Step 1: Prepare the document corpus
I will use a hardcoded list of short support-style strings so you can run the script immediately without hunting for datasets.
DOCUMENTS = [
"My account password reset link expired after five minutes, can you extend it?",
"The mobile app crashes when I try to upload a photo over 10MB on iOS 17.",
"I was charged twice for my monthly subscription on March 3rd.",
"How do I export my data to CSV? I need it for my accountant.",
"The API returns a 502 error every time I batch more than 500 records.",
"I forgot my 2FA device and cannot log in to the admin panel.",
"Do you offer annual billing? I want to switch from monthly.",
"The dashboard chart for monthly active users is showing last month's data.",
"I need a refund for the duplicate charge on invoice #9921.",
"The webhook stopped firing after the latest release on Tuesday.",
"Can I limit user permissions to read-only for the billing section?",
"The PDF export feature cuts off the right margin on A4 pages.",
"I am trying to integrate the REST API but the OAuth token expires instantly.",
"Two of my team members never received the invitation email.",
"Is there a way to schedule automated reports to Slack?",
]Step 2: Define the topic extraction agent
The first agent reads raw documents and returns structured candidate topics. Because Oxlo.ai bills per request rather than per token, we can pack a large batch into a single prompt without watching the meter run on input length.
EXTRACTION_SYSTEM_PROMPT = """You are a topic modeling assistant. Your job is to read a batch of documents and extract the main topics discussed in each.
Rules:
- Output ONLY a valid JSON array. Do not wrap it in markdown code fences.
- Each element must be an object with keys: "document_index" (integer), "topics" (array of 1-3 short topic strings).
- Topic strings should be 2-4 words, lowercase, and generic enough to apply to similar documents.
- If a document is about billing or payments, use topics like "billing issue" or "payment error".
- If a document is about authentication, use topics like "account access" or "authentication failure".
Example output:
[
{"document_index": 0, "topics": ["account access", "password reset"]},
{"document_index": 1, "topics": ["mobile bug", "file upload"]}
]"""Step 3: Extract candidate topics
We format the documents into a numbered list, send them to Oxlo.ai in a single request, and parse the JSON response. I am using Llama 3.3 70B because it handles structured JSON reliably for extraction tasks.
import json
from openai import OpenAI
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
def extract_topics(docs):
user_content = "\n".join(f"{i}: {doc}" for i, doc in enumerate(docs))
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": EXTRACTION_SYSTEM_PROMPT},
{"role": "user", "content": user_content},
],
)
raw = response.choices[0].message.content
raw = raw.strip().removeprefix("
```json").removeprefix("```
").removesuffix("
```").strip()
return json.loads(raw)
candidates = extract_topics(DOCUMENTS)
print(json.dumps(candidates, indent=2))Step 4: Consolidate topics into a taxonomy
The raw extraction will produce overlapping labels like "billing issue" and "payment error". We feed the unique candidate topics back to the LLM and ask for a consolidated taxonomy of 5 to 7 canonical topics. This is the second major Oxlo.ai request, and again the flat pricing means we are not penalized for the long list of inputs.
CONSOLIDATION_SYSTEM_PROMPT = """You are a taxonomy specialist. You will receive a list of raw topic labels extracted from documents. Your task is to consolidate them into a clean, non-overlapping taxonomy.
Rules:
- Output ONLY a valid JSON array of strings.
- Return exactly 5 to 7 canonical topic names.
- Each topic name should be 2-4 words, title case, and mutually exclusive.
- Merge synonyms. For example, "billing issue" and "payment error" should become "Billing & Payments".
- Do not explain your reasoning. Output JSON only."""
def consolidate_topics(candidates):
unique = sorted({t for doc in candidates for t in doc["topics"]})
user_content = "Raw topics:\n" + "\n".join(f"- {u}" for u in unique)
response = client.chat.completions.create(
model="qwen-3-32b",
messages=[
{"role": "system", "content": CONSOLIDATION_SYSTEM_PROMPT},
{"role": "user", "content": user_content},
],
)
raw = response.choices[0].message.content
raw = raw.strip().removeprefix("```
json").removeprefix("
```").removesuffix("```
").strip()
return json.loads(raw)
taxonomy = consolidate_topics(candidates)
print("Canonical topics:", taxonomy)Step 5: Assign documents to canonical topics
Now that we have a fixed taxonomy, we run a final pass to tag every document with the single best matching topic. We send the taxonomy and the documents together in one request. On Oxlo.ai, this counts as one flat request regardless of how many documents we include.
ASSIGNMENT_SYSTEM_PROMPT = """You are a document classifier. You will receive a taxonomy and a list of documents. Assign each document to exactly one topic from the taxonomy.
Rules:
- Output ONLY a valid JSON array.
- Each element must be an object with keys: "document_index" (integer), "topic" (string matching the taxonomy exactly).
- Choose the single best topic. Do not invent new topics."""
def assign_topics(docs, taxonomy):
taxo_str = "\n".join(f"- {t}" for t in taxonomy)
docs_str = "\n".join(f"{i}: {doc}" for i, doc in enumerate(docs))
user_content = f"Taxonomy:\n{taxo_str}\n\nDocuments:\n{docs_str}"
response = client.chat.completions.create(
model="kimi-k2.6",
messages=[
{"role": "system", "content": ASSIGNMENT_SYSTEM_PROMPT},
{"role": "user", "content": user_content},
],
)
raw = response.choices[0].message.content
raw = raw.strip().removeprefix("
```json").removeprefix("```
").removesuffix("
```
").strip()
return json.loads(raw)
assignments = assign_topics(DOCUMENTS, taxonomy)Run it
The snippet below ties the three stages together and prints a readable report.
if __name__ == "__main__":
print("=== Extracting candidate topics ===")
candidates = extract_topics(DOCUMENTS)
print("\n=== Consolidating taxonomy ===")
taxonomy = consolidate_topics(candidates)
for t in taxonomy:
print(f" - {t}")
print("\n=== Final assignments ===")
assignments = assign_topics(DOCUMENTS, taxonomy)
for a in assignments:
idx = a["document_index"]
topic = a["topic"]
snippet = DOCUMENTS[idx][:60]
print(f"[{topic}] {snippet}...")Expected output looks like this:
=== Extracting candidate topics === === Consolidating taxonomy === - Billing & Payments - Account Access - Mobile App Bugs - API & Integrations - Data Export & Reporting === Final assignments === [Account Access] My account password reset link expired after five min... [Mobile App Bugs] The mobile app crashes when I try to upload a photo... [Billing & Payments] I was charged twice for my monthly subscription... [Data Export & Reporting] How do I export my data to CSV? I need it... [API & Integrations] The API returns a 502 error every time I batch... [Account Access] I forgot my 2FA device and cannot log in to the ad... [Billing & Payments] Do you offer annual billing? I want to switch... [Data Export & Reporting] The dashboard chart for monthly active use... [Billing & Payments] I need a refund for the duplicate charge on in... [API & Integrations] The webhook stopped firing after the latest re... [Account Access] Can I limit user permissions to read-only for the... [Data Export & Reporting] The PDF export feature cuts off the right... [API & Integrations] I am trying to integrate the REST API but the... [Account Access] Two of my team members never received the invitati... [Data Export & Reporting] Is there a way to schedule automated repo...
Wrap-up
You now have a fully LLM-driven topic modeling pipeline that runs in under a minute. Because Oxlo.ai uses flat request-based pricing, you can scale this to hundreds of documents per batch without token costs exploding, which makes it ideal for nightly ingestion jobs. Two concrete next steps: wire this into a cron job that reads from a support inbox or Slack export, or add a Gradio UI so non-technical teammates can drop in a CSV and download labeled results. For details on request-based pricing, see https://oxlo.ai/pricing.
Top comments (0)