DEV Community

shashank ms
shashank ms

Posted on

Using LLMs for Text Analysis

#learnai #oxlo #ai

We are building a customer feedback analyzer that reads raw support messages, scores sentiment, extracts product issues, and flags urgency. It helps support teams prioritize tickets without manually reading every thread.

What you'll need

Oxlo.ai uses request-based pricing, so you pay per API call rather than per token. For long feedback threads or large batch jobs, this keeps costs predictable. See https://oxlo.ai/pricing for plan details.

Step 1: Connect to Oxlo.ai and verify the client

I start by initializing the client and making a quick health check call to confirm the key is active.

from openai import OpenAI
import os

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key=os.environ.get("OXLO_API_KEY", "YOUR_OXLO_API_KEY")
)

response = client.chat.completions.create(
    model="llama-3.3-70b",
    messages=[
        {"role": "user", "content": "Say 'Connection OK' and nothing else."}
    ],
    max_tokens=10
)
print(response.choices[0].message.content)

Step 2: Design the analysis schema and system prompt

Next, I define exactly what I want extracted from each message. The system prompt forces strict JSON and sets the classification rules.

SYSTEM_PROMPT = """You are a support feedback analyzer. Analyze the user message and return a single JSON object with these exact keys:

- sentiment: one of "positive", "neutral", "negative", "urgent_negative"
- product_mentioned: string or null
- issue_type: one of "bug", "feature_request", "billing", "account", "other", null
- urgent: boolean
- one_line_summary: string, max 12 words

Rules:
- Return only the JSON object. No markdown fences, no explanation.
- If the message contains words like "immediately", "asap", "broken completely", or "refund now", set urgent to true and sentiment to "urgent_negative"."""

Step 3: Build the analyzer function with JSON mode

I wrap the API call in a small function that accepts a feedback string and returns a Python dict. I use JSON mode to keep the output structured.

import json

def analyze_feedback(text: str) -> dict:
    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": text},
        ],
        response_format={"type": "json_object"},
        temperature=0.1,
        max_tokens=256
    )
    raw = response.choices[0].message.content
    return json.loads(raw)

Step 4: Process a batch of feedback records

Now I feed a list of real-looking support messages through the analyzer and collect the results.

feedback_batch = [
    "The export to CSV button does nothing on the dashboard. This is blocking our monthly report.",
    "Love the new dark mode. Much easier on the eyes during late night deploys.",
    "I was charged twice for the Pro plan last month. Please refund the duplicate immediately.",
    "Can you add webhook support for the audit log? Would help our compliance workflow.",
    "Login is completely broken for my team since this morning. We need this fixed ASAP."
]

results = []
for item in feedback_batch:
    try:
        parsed = analyze_feedback(item)
        parsed["original"] = item
        results.append(parsed)
        print(f"Analyzed: {parsed['one_line_summary']}")
    except Exception as e:
        print(f"Failed on item: {item[:40]}... Error: {e}")

print(f"\nSuccessfully analyzed {len(results)} items.")

Step 5: Generate a summary report

Finally, I send the aggregated findings back to the model to produce a human-readable stand-up summary for the support lead. I switch to qwen-3-32b here because it handles long context and multilingual reasoning well.

report_prompt = f"""You are a support lead reading the following analyzed feedback. Write a 3-bullet summary for the team stand-up. Each bullet should note the issue type, urgency, and action.

Analyzed feedback:
{json.dumps(results, indent=2)}

Stand-up summary:"""

summary = client.chat.completions.create(
    model="qwen-3-32b",
    messages=[
        {"role": "system", "content": "You write concise stand-up summaries for support teams."},
        {"role": "user", "content": report_prompt},
    ],
    temperature=0.3,
    max_tokens=300
)

print("\n--- Stand-up Summary ---")
print(summary.choices[0].message.content)

Run it

Save the full script as feedback_analyzer.py, set your API key, and run:

export OXLO_API_KEY="sk-oxlo.ai-..."
python feedback_analyzer.py

Expected output:

Connection OK
Analyzed: CSV export button not working
Analyzed: Dark mode praised by user
Analyzed: Duplicate Pro plan charge
Analyzed: Request for audit log webhooks
Analyzed: Login broken for entire team

Successfully analyzed 5 items.

--- Stand-up Summary ---
- Urgent bug: Login is completely broken for one team since this morning. Escalate to on-call engineer immediately.
- Urgent billing: Customer was double-charged for Pro and demands a refund. Route to billing team today.
- Non-urgent bugs and requests: CSV export failure and webhook request for audit logs. Schedule for next sprint review.

Next steps

Wire the analyzer into a Slack webhook so it runs on every new support form submission. If you process high volumes, consider Oxlo.ai's request-based pricing to keep long-context analysis predictable as you scale.

For higher accuracy on nuanced or multilingual feedback, swap llama-3.3-70b for kimi-k2.6 in the analyzer function. Kimi K2.6 excels at advanced reasoning and agentic coding tasks, and its 131K context window handles long ticket threads without truncation.

Top comments (0)