DEV Community

shashank ms
shashank ms

Posted on

Demystifying LLMs for Language Understanding: Best Practices and Use Cases

#learnai #oxlo #ai

We are building a customer message understanding agent that reads unstructured support tickets and extracts intent, entities, and sentiment as structured JSON. This replaces brittle regex pipelines and helps support teams route tickets automatically. I run this on Oxlo.ai so that long tickets do not inflate costs, since pricing is a flat rate per request rather than per token.

What you'll need

Step 1: Verify connectivity

Before building the parser, I make sure the client can reach Oxlo.ai and that my key is active. I send a minimal request to the general-purpose Llama 3.3 70B model.

from openai import OpenAI

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

response = client.chat.completions.create(
    model="llama-3.3-70b",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Reply with exactly: connection OK"},
    ],
)

print(response.choices[0].message.content)

Step 2: Write the system prompt

The system prompt is the contract between my code and the model. It defines the JSON schema and the language understanding task. I keep it strict so the model does not add conversational fluff.

SYSTEM_PROMPT = """
You are a language understanding engine. Analyze the user message and return a single JSON object with exactly these keys:

- intent: one of [refund_request, technical_issue, billing_question, general_feedback]
- entities: an object with any of these keys you detect: order_id, product_name, email, amount, urgency
- sentiment: one of [positive, neutral, negative]
- summary: a one-sentence summary of the user message

Rules:
1. Output ONLY valid JSON. No markdown, no explanation.
2. If a field is unknown, use null.
3. urgency should be "low", "medium", or "high" only if mentioned.
"""

Step 3: Build the parser function

Now I wrap the API call in a function that sends the system prompt plus the user message. I use JSON mode to lock the output format, then parse the result with the standard library.

import json
from openai import OpenAI

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

def understand_message(user_message: str) -> dict:
    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": user_message},
        ],
        response_format={"type": "json_object"},
        temperature=0.1,
    )
    
    raw = response.choices[0].message.content
    return json.loads(raw)

# quick sanity check
if __name__ == "__main__":
    test = "Hi, my order #12345 never arrived and I need a refund. This is frustrating."
    print(understand_message(test))

Step 4: Add validation and fallback

Models can hallucinate keys or return malformed JSON under edge cases. I add a small validator that enforces the schema and falls back to a safe default so the pipeline never crashes.

from typing import Optional

def safe_understand(user_message: str) -> dict:
    default = {
        "intent": "general_feedback",
        "entities": {},
        "sentiment": "neutral",
        "summary": user_message[:100]
    }
    
    try:
        result = understand_message(user_message)
    except Exception:
        return default
    
    # enforce expected keys
    for key in default:
        if key not in result:
            result[key] = default[key]
    
    return result

Step 5: Process a batch of messages

In production I do not call the API one by one. I loop over a list of tickets and collect the structured outputs. Because Oxlo.ai charges a flat rate per request, I know the cost of this batch upfront without counting tokens.

tickets = [
    "I was charged twice for my subscription this month. My email is alice@example.com.",
    "Love the new dashboard! Great work on the update.",
    "Order #99887 is missing the blue cable. I need it by Friday.",
    "How do I reset my password?"
]

results = []
for ticket in tickets:
    parsed = safe_understand(ticket)
    results.append(parsed)
    print(parsed)

Run it

Save the full script as understand.py, export your key, and run it.

export OXLO_API_KEY="sk-..."
python understand.py

When I run this against the batch above, the output looks like this:

{'intent': 'billing_question', 'entities': {'email': 'alice@example.com'}, 'sentiment': 'negative', 'summary': 'User reports being charged twice for subscription.'}
{'intent': 'general_feedback', 'entities': {}, 'sentiment': 'positive', 'summary': 'User loves the new dashboard update.'}
{'intent': 'technical_issue', 'entities': {'order_id': '99887', 'urgency': 'high'}, 'sentiment': 'negative', 'summary': 'Order is missing a blue cable and needed by Friday.'}
{'intent': 'technical_issue', 'entities': {}, 'sentiment': 'neutral', 'summary': 'User asks how to reset their password.'}

Wrap up

The agent is now a drop-in replacement for regex-based ticket routing. Two concrete next steps: wire the safe_understand output into your CRM webhook so tickets auto-route based on intent, or swap the model to qwen-3-32b if you need the same behavior across multilingual inputs without rewriting the prompt.

Top comments (0)