DEV Community

shashank ms
shashank ms

Posted on

LLM for Topic Modeling: Applications and Techniques

#learnai #oxlo #ai

We are going to build a zero-shot topic modeling pipeline that ingests raw support tickets, generates a concise taxonomy with an LLM, and labels every ticket. It runs entirely against Oxlo.ai's OpenAI-compatible API, so you do not need to host any local embedding or inference stack. If you are still grepping keywords or managing LDA hyperparameters, this gives you interpretable topics in under fifty lines of Python.

What you'll need

Step 1: Spin up the Oxlo.ai client and load raw documents

I keep a small list of synthetic support tickets directly in the script so the example is self-contained. We initialize the OpenAI client pointing at Oxlo.ai.

from openai import OpenAI

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

raw_docs = [
    "I was charged twice for my subscription this month and I need a refund.",
    "The mobile app crashes every time I open the settings menu on my Pixel 7.",
    "How do I export my data to CSV? I cannot find the button anywhere.",
    "Login fails with a 500 error after I enabled two-factor authentication.",
    "Your pricing page is unclear about the difference between Pro and Premium.",
    "The API returns a 429 rate limit even though I am well below my quota.",
    "Dark mode does not apply to the dashboard charts and it hurts my eyes at night.",
    "I tried to reset my password but the email never arrived, not even in spam.",
]

Step 2: Define the system prompt

The prompt is the only training signal we need. It forces structured JSON so we can parse the taxonomy without regex hacks.

SYSTEM_PROMPT = """You are a topic modeling analyst.
Given a set of raw documents, identify a concise taxonomy of topics that covers the corpus.
Respond with a single JSON object containing a key 'topics' whose value is a list of objects.
Each object must have the keys: 'topic_id' (integer), 'topic_name' (string, 2-4 words),
'description' (string, one sentence), and 'keywords' (list of 3-5 strings).
Do not wrap the JSON in markdown fences. Output valid JSON only."""

Step 3: Generate the topic taxonomy

I feed the first six documents to Llama 3.3 70B and ask it to propose the topics. You can swap in Qwen 3 32B if you need stronger multilingual support, but Llama 3.3 70B is a solid default for English structure.

import json

user_message = "Documents:\n" + "\n".join(f"{i+1}. {d}" for i, d in enumerate(raw_docs[:6]))

response = client.chat.completions.create(
    model="llama-3.3-70b",
    messages=[
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": user_message},
    ],
)

taxonomy = json.loads(response.choices[0].message.content)
print(json.dumps(taxonomy, indent=2))

Step 4: Classify the corpus

Now we label every ticket against the taxonomy. I use DeepSeek V3.2 for this bulk step because it is efficient and available on the Oxlo.ai free tier, which keeps the experiment cheap. We loop through the full list and collect the results.

LABEL_PROMPT = """You are a document classifier.
Given a topic taxonomy and a single document, assign the document to exactly one topic.
Respond with a single JSON object containing 'topic_id' and 'confidence' (0.0 to 1.0).
Do not wrap the JSON in markdown fences. Output valid JSON only."""

labels = []

for doc in raw_docs:
    tax_block = json.dumps(taxonomy, indent=2)
    user_message = f"Taxonomy:\n{tax_block}\n\nDocument:\n{doc}"

    response = client.chat.completions.create(
        model="deepseek-v3.2",
        messages=[
            {"role": "system", "content": LABEL_PROMPT},
            {"role": "user", "content": user_message},
        ],
    )

    result = json.loads(response.choices[0].message.content)
    labels.append(result["topic_id"])

print(labels)

Step 5: Aggregate and inspect results

A quick pandas pivot gives us the frequency table. No matplotlib required for this check.

import pandas as pd

name_map = {t["topic_id"]: t["topic_name"] for t in taxonomy["topics"]}
df = pd.DataFrame({"document": raw_docs, "topic_id": labels})
df["topic"] = df["topic_id"].map(name_map)

print(df[["topic", "document"]].to_string(index=False))
print("\n--- Counts ---")
print(df["topic"].value_counts())

Run it

Save the script as topic_model.py, export your key, and run it.

export OXLO_API_KEY="sk-oxlo.ai-..."
python topic_model.py

Typical output looks like this. The exact topic names will vary slightly because LLM outputs are stochastic, but the structure is always the same.

{
  "topics": [
    {
      "topic_id": 1,
      "topic_name": "Billing Issues",
      "description": "Problems related to charges, refunds, and subscription payments.",
      "keywords": ["charge", "refund", "subscription", "billing", "payment"]
    },
    {
      "topic_id": 2,
      "topic_name": "App Crashes",
      "description": "Reports of the application crashing or freezing during use.",
      "keywords": ["crash", "freeze", "bug", "error", "app"]
    },
    {
      "topic_id": 3,
      "topic_name": "Feature Requests",
      "description": "Questions or requests for new functionality or data exports.",
      "keywords": ["export", "feature", "CSV", "button", "functionality"]
    },
    {
      "topic_id": 4,
      "topic_name": "Account Access",
      "description": "Issues with login, authentication, or password recovery.",
      "keywords": ["login", "password", "authentication", "access", "error"]
    },
    {
      "topic_id": 5,
      "topic_name": "Pricing Questions",
      "description": "Confusion or inquiries about plan differences and costs.",
      "keywords": ["pricing", "plan", "cost", "Pro", "Premium"]
    },
    {
      "topic_id": 6,
      "topic_name": "API Rate Limits",
      "description": "Reports of unexpected throttling or quota errors.",
      "keywords": ["API", "rate limit", "429", "quota", "throttling"]
    },
    {
      "topic_id": 7,
      "topic_name": "UI and Themes",
      "description": "Feedback on visual design, dark mode, or dashboard layout.",
      "keywords": ["dark mode", "UI", "dashboard", "charts", "design"]
    }
  ]
}

[1, 2, 3, 4, 5, 6, 7, 4]

          topic                                           document
 Billing Issues  I was charged twice for my subscription this ...
   App Crashes  The mobile app crashes every time I open the ...
Feature Requests  How do I export my data to CSV? I cannot find...
  Account Access  Login fails with a 500 error after I enabled ...
Pricing Questions  Your pricing page is unclear about the difference...
 API Rate Limits  The API returns a 429 rate limit even though I ...
   UI and Themes  Dark mode does not apply to the dashboard charts...
  Account Access  I tried to reset my password but the email never...

--- Counts ---
Account Access      2
Billing Issues      1
App Crashes         1
Feature Requests    1
Pricing Questions   1
API Rate Limits     1
UI and Themes       1
Name: topic, dtype: int64

Wrap-up

Swap Llama 3.3 70B for Kimi K2.6 if you want to push longer documents through the classifier, or switch the taxonomy generator to Qwen 3 32B for multilingual corpora. The next concrete step is to wire this pipeline into a cron job or an Airflow DAG so new tickets are labeled as they arrive. If you are processing thousands of documents, remember that Oxlo.ai charges a flat rate per request, not per token, so long prompts with full taxonomies and lengthy tickets do not inflate your bill the way they would on token-based providers. See https://oxlo.ai/pricing for current plan details.

Top comments (0)