Using LLM for Sentiment Analysis with Transformers

#learnai #oxlo #ai

We are building a sentiment analysis pipeline that classifies customer support tickets into positive, negative, or neutral categories with confidence scores. This gives support leads and product managers a quick signal for triage without the overhead of training and hosting a custom transformer model. Because Oxlo.ai uses flat per-request pricing and exposes an OpenAI-compatible endpoint, we can process long tickets and irregular batches without worrying about token length or cold starts.

What you'll need

Make sure you have Python 3.10 or newer installed. You will also need the OpenAI SDK and an Oxlo.ai API key.

pip install openai

Generate your key from https://portal.oxlo.ai. The examples below use llama-3.3-70b, which is Oxlo.ai's general-purpose flagship and works well for classification tasks.

Step 1: Configure the Oxlo.ai client

Instantiate the client with the Oxlo.ai base URL and your API key, then send a lightweight health check to confirm connectivity.

from openai import OpenAI

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key="YOUR_OXLO_API_KEY"
)

# Quick connectivity check
test = client.chat.completions.create(
    model="llama-3.3-70b",
    messages=[{"role": "user", "content": "Hello"}],
    max_tokens=10,
)
print("Client ready:", test.choices[0].message.content)

Step 2: Write the system prompt

I keep the prompt strict. It demands valid JSON with no markdown, defines the three required keys, and tells the model how to handle mixed signals.

SYSTEM_PROMPT = """You are a sentiment analysis engine. Analyze the user-provided text and return a JSON object with exactly these keys:
- sentiment: one of "positive", "negative", or "neutral"
- confidence: a float between 0.0 and 1.0
- reasoning: one concise sentence explaining the classification

Rules:
- If the text is mixed, choose the dominant sentiment.
- Confidence should reflect clarity, not positivity.
- Output only valid JSON with no markdown formatting."""

Step 3: Build the analysis function

This function sends the ticket text to Llama 3.3 70B with JSON mode enabled, then parses the response. I keep temperature low so repeated runs stay consistent.

import json

def analyze_sentiment(text: str) -> dict:
    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": text},
        ],
        response_format={"type": "json_object"},
        temperature=0.1,
    )
    
    raw = response.choices[0].message.content
    return json.loads(raw)

# Smoke test with a mixed ticket
sample = "The new dashboard is incredibly fast, but the export button is still broken."
result = analyze_sentiment(sample)
print(json.dumps(result, indent=2))

Step 4: Batch process feedback

Now we iterate over a list of support tickets and collect structured results. Because Oxlo.ai charges one flat rate per request, long tickets do not inflate the cost.

tickets = [
    "I love the new feature. It saved me hours last week.",
    "Export is failing again. This is the third time today.",
    "The update installed without issues. I will test more tomorrow.",
    "Terrible experience. Support never replied and I lost data.",
    "It is okay, nothing special but it works.",
]

results = []
for ticket in tickets:
    try:
        out = analyze_sentiment(ticket)
        out["source"] = ticket
        results.append(out)
    except Exception as e:
        print(f"Failed on ticket: {ticket[:50]}... Error: {e}")

# Print a summary table
for r in results:
    tag = r["sentiment"].upper()
    conf = r["confidence"]
    print(f"[{tag:8}] {conf:.2f} | {r['source'][:60]}...")

Run it

Save the full script as sentiment_pipeline.py and execute it. You should see connectivity confirmation, the smoke test JSON, and the final summary table.

$ python sentiment_pipeline.py
Client ready: Hello! How can I assist you today?

{
  "sentiment": "neutral",
  "confidence": 0.72,
  "reasoning": "Mixed feedback contains both praise and a bug report."
}

[POSITIVE ] 0.91 | I love the new feature. It saved me hours last week...
[NEGATIVE ] 0.89 | Export is failing again. This is the third time today...
[NEUTRAL  ] 0.68 | The update installed without issues. I will test more ...
[NEGATIVE ] 0.95 | Terrible experience. Support never replied and I lost...
[NEUTRAL  ] 0.55 | It is okay, nothing special but it works....

Wrap-up and next steps

This pipeline is ready to drop into a nightly cron job, a support webhook, or an ETL script. Two directions to take it next: swap in qwen-3-32b if you need to classify multilingual feedback, or prototype against deepseek-v3.2 on Oxlo.ai's free tier to keep experimentation cost-free. For details on request limits and upgrading, see https://oxlo.ai/pricing.

DEV Community