We are going to build a lightweight topic modeling pipeline that reads a collection of raw text documents, generates human-readable topic labels with an LLM, and assigns every document to its best-fit category. This is useful for content teams, support analysts, or researchers who need to theme large corpora without managing LDA or BERTopic dependencies.
What you'll need
- Python 3.10 or newer
- The OpenAI SDK:
pip install openai - An Oxlo.ai API key from https://portal.oxlo.ai
- A CSV or text file with documents. I will use a hardcoded list of support tickets for this tutorial so you can run it immediately.
Step 1: Configure the client and load documents
I start by instantiating the OpenAI-compatible client pointing at Oxlo.ai. Because Oxlo.ai uses flat per-request pricing, I do not need to count tokens before sending long document batches. You can view current plans at https://oxlo.ai/pricing.
from openai import OpenAI
import json
client = OpenAI(
base_url="https://api.oxlo.ai/v1",
api_key="YOUR_OXLO_API_KEY" # get yours at https://portal.oxlo.ai
)
documents = [
"My credit card was charged twice for the same transaction on March 3rd.",
"The app crashes every time I try to upload a photo from my gallery.",
"I forgot my password and the reset email never arrives in my inbox.",
"How do I change my subscription from monthly to annual billing?",
"The delivery was supposed to arrive last Tuesday but the tracking shows no updates.",
"I love the new dark mode feature, it saves my eyes during night shifts.",
"The checkout page keeps timing out when I try to pay with PayPal.",
"Can I get a refund for my order? I accidentally purchased the wrong size.",
"The search results are irrelevant since the last update to the catalog.",
"My account was locked after three failed login attempts.",
]Step 2: Define the topic discovery prompt
The first LLM call discovers the latent themes. I feed a sample of documents and ask for structured JSON so I can parse the taxonomy programmatically. I use Qwen 3 32B because it handles structured generation reliably.
TOPIC_SYSTEM_PROMPT = """
You are a research analyst. Read the provided documents and identify 3 to 5 distinct topics that describe the corpus.
For each topic, provide:
- label: a concise 2-4 word name
- description: one sentence explaining what the topic covers
- keywords: a list of 3 indicative keywords
Return ONLY a JSON object in this exact format:
{
"topics": [
{"label": "...", "description": "...", "keywords": ["...", "...", "..."]}
]
}
"""Step 3: Generate the topic taxonomy
I send the first half of the corpus as a sample. The model returns a JSON object that defines our topic schema for the rest of the pipeline.
def discover_topics(sample_docs):
user_message = "Analyze these documents and extract topics:\n" + "\n".join(
f"- {doc}" for doc in sample_docs
)
response = client.chat.completions.create(
model="qwen-3-32b",
messages=[
{"role": "system", "content": TOPIC_SYSTEM_PROMPT},
{"role": "user", "content": user_message},
],
)
raw = response.choices[0].message.content
# strip markdown fences if present
cleaned = raw.replace("
```json", "").replace("```
", "").strip()
return json.loads(cleaned)
sample = documents[:5]
topic_schema = discover_topics(sample)
print(json.dumps(topic_schema, indent=2))Step 4: Define the classifier prompt
Now I need a second prompt that takes the full taxonomy and a single document, then returns the best matching topic. I keep the instructions strict so the output stays parseable.
CLASSIFIER_SYSTEM_PROMPT = """
You are a document classifier. You will be given a JSON topic taxonomy and a single document.
Choose the single best topic label for the document.
Return ONLY a JSON object in this exact format:
{
"topic_label": "exact label from the taxonomy",
"confidence": 0.0 to 1.0,
"reason": "one sentence explaining the match"
}
"""Step 5: Classify every document
I loop over the full set and call the classifier for each document. Because Oxlo.ai charges per request, not per token, running ten classification calls costs the same whether the documents are short tweets or long support threads.
def classify_document(doc, topics):
user_message = (
f"Topic taxonomy: {json.dumps(topics)}\n\n"
f"Document: {doc}"
)
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": CLASSIFIER_SYSTEM_PROMPT},
{"role": "user", "content": user_message},
],
)
raw = response.choices[0].message.content
cleaned = raw.replace("
```json", "").replace("```
", "").strip()
return json.loads(cleaned)
assignments = []
for doc in documents:
result = classify_document(doc, topic_schema)
assignments.append({
"document": doc,
"topic": result["topic_label"],
"confidence": result["confidence"],
"reason": result["reason"],
})Step 6: Aggregate and inspect
Finally, I aggregate the results to see the distribution of topics across the corpus. This pure Python step needs no API calls.
from collections import Counter
distribution = Counter(a["topic"] for a in assignments)
print("Topic distribution:")
for topic, count in distribution.most_common():
print(f" {topic}: {count}")
print("\nDetailed assignments:")
for item in assignments:
print(f"[{item['topic']} ({item['confidence']})] {item['document']}")Run it
Putting it all together, the main block below runs the full pipeline end to end. Here is the complete script and the output I get when I run it against Oxlo.ai.
if __name__ == "__main__":
print("=== Discovering topics ===")
sample = documents[:5]
topic_schema = discover_topics(sample)
print("\n=== Classifying documents ===")
assignments = []
for doc in documents:
result = classify_document(doc, topic_schema)
assignments.append({
"document": doc,
"topic": result["topic_label"],
"confidence": result["confidence"],
})
print("\n=== Results ===")
for a in assignments:
print(f"{a['topic']:20s} | {a['confidence']:.2f} | {a['document'][:50]}...")Example output:
=== Discovering topics ===
{
"topics": [
{
"label": "Billing Issues",
"description": "Problems related to credit card charges, refunds, and payment failures.",
"keywords": ["charge", "refund", "payment"]
},
{
"label": "Account Access",
"description": "Issues with passwords, login attempts, and account lockouts.",
"keywords": ["password", "login", "locked"]
},
{
"label": "Technical Bugs",
"description": "App crashes, timeouts, and errors during normal usage.",
"keywords": ["crash", "timeout", "bug"]
},
{
"label": "Shipping Questions",
"description": "Concerns about delivery times, tracking, and order status.",
"keywords": ["delivery", "tracking", "arrive"]
},
{
"label": "Feature Feedback",
"description": "User opinions and requests regarding product features.",
"keywords": ["feature", "dark mode", "update"]
}
]
}
=== Classifying documents ===
=== Results ===
Billing Issues | 0.95 | My credit card was charged twice for the same ...
Technical Bugs | 0.92 | The app crashes every time I try to upload a ...
Account Access | 0.89 | I forgot my password and the reset email never ...
Billing Issues | 0.91 | How do I change my subscription from monthly to...
Shipping Questions | 0.88 | The delivery was supposed to arrive last Tuesday...
Feature Feedback | 0.93 | I love the new dark mode feature, it saves my ...
Billing Issues | 0.90 | The checkout page keeps timing out when I try ...
Billing Issues | 0.94 | Can I get a refund for my order? I accidentally...
Technical Bugs | 0.87 | The search results are irrelevant since the last...
Account Access | 0.96 | My account was locked after three failed login ...Wrap up
This pipeline gives you a working topic model in under fifty lines of Python. Two concrete ways to extend it: first, add a re-ranking step where you send ambiguous documents to kimi-k2.6 for deeper reasoning. Second, wrap the classifier in a ThreadPoolExecutor so you can process thousands of documents in parallel against Oxlo.ai's flat per-request pricing. You can explore models and plans at https://oxlo.ai/pricing.
Top comments (0)