LLM Sentiment Analysis and Text Classification Guide

#learnai #oxlo #ai

Support teams drown in unstructured feedback. In this tutorial, we will build a working sentiment and topic classifier that labels customer messages in a single API call, then batch-processes a backlog using Oxlo.ai. The entire pipeline uses the OpenAI SDK with an Oxlo.ai base URL, so there is no new client library to learn.

What you'll need

Python 3.10 or newer. An Oxlo.ai API key from https://portal.oxlo.ai. The OpenAI SDK installed with pip install openai.

Step 1: Configure the Oxlo.ai client

I keep credentials in an environment variable so I do not accidentally commit keys. The Oxlo.ai client is a drop-in replacement for the standard OpenAI client.

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key=os.getenv("OXLO_API_KEY", "YOUR_OXLO_API_KEY"),
)

Step 2: Define the system prompt

The trick to reliable classification is telling the model exactly what valid JSON looks like. I use a strict system prompt so every response follows the same structure.

SYSTEM_PROMPT = """You are a classification engine. Analyze the user message and return ONLY a JSON object with this exact structure:

{
  "sentiment": "negative" | "neutral" | "positive",
  "topic": "billing" | "bug" | "feature_request" | "other",
  "confidence": 0.0 to 1.0,
  "reasoning": "one sentence explaining the label"
}

Rules:
- sentiment must be one of the three allowed strings.
- topic must be one of the four allowed strings.
- confidence is your certainty score.
- Do not include markdown, explanations, or text outside the JSON."""

Step 3: Build the classifier function

I wrap the API call in a small function so I can swap models later. I enable JSON mode to enforce valid output, and I default to llama-3.3-70b because it handles mixed instructions cleanly. If your data is multilingual, swap the model string to qwen-3-32b.

import json

def classify_message(text: str, model: str = "llama-3.3-70b"):
    response = client.chat.completions.create(
        model=model,
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": text},
        ],
        response_format={"type": "json_object"},
        temperature=0.1,
    )

    raw = response.choices[0].message.content
    return json.loads(raw)

Step 4: Process a batch of tickets

Most real workloads are not single messages. I simulate a small queue and run each item through the classifier, collecting results in a list. Because Oxlo.ai uses flat request-based pricing, long tickets do not inflate cost the way token-based metering does. See details at https://oxlo.ai/pricing.

tickets = [
    "I was charged twice last month and your dashboard is broken. Fix this now.",
    "Love the new export feature. Saves me an hour every week.",
    "How do I change my notification settings? I looked everywhere.",
    "The API returns a 500 error when I send payloads over 1 MB. Here is the curl...",
    "Please add dark mode. It is hard to use at night.",
]

results = []
for t in tickets:
    try:
        label = classify_message(t)
        results.append({"text": t, "label": label})
    except Exception as e:
        results.append({"text": t, "error": str(e)})

print(json.dumps(results, indent=2))

Step 5: Filter by confidence

Raw labels are not enough. I filter out anything below a confidence threshold so a human can review the edge cases.

CONFIDENCE_THRESHOLD = 0.85

flagged = []
clean = []

for r in results:
    if "error" in r:
        flagged.append(r)
        continue

    conf = r["label"].get("confidence", 0)
    if conf < CONFIDENCE_THRESHOLD:
        flagged.append(r)
    else:
        clean.append(r)

print(f"Auto-approved: {len(clean)}")
print(f"Needs review: {len(flagged)}")

for item in clean:
    print(f"[{item['label']['sentiment']}] {item['label']['topic']} -> {item['text'][:50]}...")

Run it

Save everything in classify.py, export your key, and run python classify.py. Here is what my last run looked like.

$ export OXLO_API_KEY="sk-oxlo.ai-..."
$ python classify.py

Auto-approved: 4
Needs review: 1

[negative] billing -> I was charged twice last month and your dashboard...
[positive] feature_request -> Love the new export feature. Saves me an ho...
[neutral] other -> How do I change my notification settings? I loo...
[negative] bug -> The API returns a 500 error when I send payloa...

Next steps

Two ways to extend this immediately. First, wrap the classify_message function in a FastAPI endpoint so your support stack can call it live. Second, if you are classifying long conversation threads instead of single messages, switch to kimi-k2.6 or deepseek-v3.2. Both handle extended context and reasoning well on Oxlo.ai.

DEV Community