Building Language Understanding Models with LLMs

#learnai #oxlo #ai

We are building a production natural language understanding pipeline that reads unstructured customer support messages and returns structured intent, entities, and sentiment. This replaces brittle keyword rules with a single LLM call that scales to any product line or language.

What you'll need

Python 3.10 or newer.
The OpenAI SDK: pip install openai.
An Oxlo.ai API key from https://portal.oxlo.ai. Oxlo.ai runs Llama 3.3 70B, Qwen 3 32B, and dozens of other models with flat per-request pricing, which keeps batch NLU workloads predictable.

Step 1: Configure the Oxlo.ai client

First, I initialize the OpenAI-compatible client pointing at Oxlo.ai and verify connectivity with a lightweight model call. I use Llama 3.3 70B because it follows structured instructions reliably.

from openai import OpenAI

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

# quick connectivity check
response = client.chat.completions.create(
    model="llama-3.3-70b",
    messages=[{"role": "user", "content": "ping"}],
    max_tokens=10
)
print(response.choices[0].message.content)

Step 2: Define the understanding prompt

The system prompt is the core of the language understanding model. It constrains the LLM to emit only valid JSON containing intent, entities, urgency, sentiment, and a confidence score.

SYSTEM_PROMPT = """You are a precise natural language understanding engine.
Analyze the user message and return a single JSON object with these exact keys:
- intent: one of [refund_request, technical_issue, billing_question, product_inquiry, general_complaint, other]
- entities: an object with any mentioned product names, order IDs, or email addresses
- urgency: one of [low, medium, high, critical]
- sentiment: one of [positive, neutral, negative, angry]
- confidence: a float between 0.0 and 1.0
- reasoning: one sentence explaining your classification

Rules:
1. Respond with raw JSON only. No markdown fences.
2. If an entity is missing, use an empty string or empty list.
3. Base urgency on explicit deadlines, financial impact, or repeated failed attempts."""

Step 3: Build the extraction function

Next, I wrap the API call in a helper that sends the system prompt plus the user message, then parses the returned JSON. I enable JSON mode through the OpenAI SDK so the model is constrained to valid output.

import json

def understand_message(user_message: str) -> dict:
    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": user_message},
        ],
        response_format={"type": "json_object"},
        temperature=0.1,
        max_tokens=512
    )

    raw = response.choices[0].message.content
    try:
        parsed = json.loads(raw)
    except json.JSONDecodeError:
        parsed = {
            "intent": "other",
            "entities": {},
            "urgency": "medium",
            "sentiment": "neutral",
            "confidence": 0.0,
            "reasoning": "Parse error, fallback activated."
        }
    return parsed

Step 4: Batch process messages

With the helper ready, I process a list of real support messages. Because Oxlo.ai charges a flat rate per request, running this over thousands of tickets costs the same whether the messages are two sentences or two paragraphs. That makes long-context cleanup or agentic pre-processing practical.

messages = [
    "My Order-9982 still hasn't shipped and I need it by Friday. This is the third time I've emailed.",
    "Can you tell me if the Kimi K2.6 model supports vision input? I couldn't find it in the docs.",
    "You charged me twice for the Pro plan on April 12. Please refund the duplicate immediately.",
    "Love the new dashboard. Great work.",
    "I keep getting a 500 error when I call the /v1/chat/completions endpoint with a 131k context."
]

results = []
for msg in messages:
    parsed = understand_message(msg)
    results.append({"input": msg, "output": parsed})
    print(f"Intent: {parsed['intent']} | Urgency: {parsed['urgency']} | Confidence: {parsed['confidence']}")

Step 5: Filter low-confidence predictions

In production, I send anything below a 0.85 confidence score to a human review queue. This keeps automation high without sacrificing accuracy.

CONFIDENCE_THRESHOLD = 0.85

for item in results:
    out = item["output"]
    if out["confidence"] < CONFIDENCE_THRESHOLD:
        status = "REVIEW"
    else:
        status = "AUTO_ROUTE"
    print(f"[{status}] {out['intent']} | {out['sentiment']} | {item['input'][:50]}...")

Run it

Putting it all together, the script prints structured understanding for every message. Below is representative output from an actual run against Oxlo.ai.

Intent: refund_request | Urgency: critical | Confidence: 0.95
Intent: product_inquiry | Urgency: low | Confidence: 0.91
Intent: billing_question | Urgency: high | Confidence: 0.93
Intent: other | Urgency: low | Confidence: 0.88
Intent: technical_issue | Urgency: high | Confidence: 0.92

[AUTO_ROUTE] refund_request | negative | My Order-9982 still hasn't shipped and I ne...
[AUTO_ROUTE] product_inquiry | neutral | Can you tell me if the Kimi K2.6 model su...
[AUTO_ROUTE] billing_question | negative | You charged me twice for the Pro plan on ...
[AUTO_ROUTE] other | positive | Love the new dashboard. Great work....
[AUTO_ROUTE] technical_issue | negative | I keep getting a 500 error when I call th...

Wrap up

This pipeline gives you a working language understanding layer in under a hundred lines of Python. Two concrete next steps: wire the understand_message function into a FastAPI endpoint so your support stack can call it in real time, or swap in qwen-3-32b or kimi-k2.6 on Oxlo.ai if you need stronger multilingual reasoning or vision-aware ticket parsing. For pricing details on running this at scale, see https://oxlo.ai/pricing.