Building Customer Support Agents with LLM and NLU

#aiinfrastructure #oxlo #ai

Customer support agents built on large language models alone often hallucinate routing decisions or bypass structured workflows. Adding a dedicated natural language understanding layer gives you deterministic intent classification and entity extraction, while the LLM handles nuanced response generation and tool orchestration. The result is a hybrid system that is both controllable and conversational, with the NLU layer acting as a guardrail and the LLM layer acting as the interface.

The Problem with Pure LLM Support Agents

Pure LLM agents struggle with policy enforcement. When a user asks for a refund outside the return window, a generic chat model may apologize and offer unauthorized compensation. NLU removes this ambiguity by mapping utterances to structured intents before any action is taken. It also isolates sensitive entity extraction, such as credit card numbers or order IDs, into a constrained pipeline that is easier to audit.

A Hybrid NLU + LLM Architecture

A production support agent typically runs in three phases.

NLU phase: intent classification, entity extraction, sentiment analysis, and toxicity detection.
State and policy phase: a rules engine or retrieval layer that checks the extracted intent against business logic.
LLM phase: context-aware response generation, potentially invoking tools such as CRM lookups or refund APIs.

This separation lets you update refund policies without retraining a model, and it lets you swap the generative backend without rebuilding your intent taxonomy.

Implementing the NLU Layer

You do not need a massive model for intent classification. A midsize instruction-tuned model with JSON mode is usually sufficient. Oxlo.ai supports JSON mode and function calling across its chat models, so you can treat the NLU step as a structured API call.

import os, json
from openai import OpenAI

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key=os.environ["OXLO_API_KEY"]
)

INTENT_SCHEMA = {
    "intent": "one of: REFUND, ORDER_STATUS, TECH_SUPPORT, ESCALATE",
    "confidence": "float 0.0 to 1.0",
    "entities": [{"type": "ORDER_ID|EMAIL|PRODUCT", "value": "string"}],
    "sentiment": "NEGATIVE|NEUTRAL|POSITIVE"
}

def parse_ticket(text: str) -> dict:
    resp = client.chat.completions.create(
        model="qwen3-32b",
        messages=[
            {"role": "system", "content": f"Extract intent and entities as JSON matching this schema: {json.dumps(INTENT_SCHEMA)}"},
            {"role": "user", "content": text}
        ],
        response_format={"type": "json_object"},
        temperature=0.1,
        max_tokens=512
    )
    return json.loads(resp.choices[0].message.content)

For high-volume classification, you can also use Oxlo.ai embedding models such as BGE-Large to encode incoming tickets and compare them against a vector index of labeled examples. This gives you sub-second intent lookups without invoking a generative model at all.

Orchestrating the LLM Backend

Once the NLU layer returns a structured result, the orchestrator decides whether to call a tool, reject the request, or generate a response. If the intent is ORDER_STATUS, the agent should extract the ORDER_ID, call a lookup function, and then hand the result back to an LLM for phrasing.

Oxlo.ai exposes standard OpenAI SDK function calling, so the tool loop looks identical to other providers.

def lookup_order(order_id: str) -> dict:
    # Your internal API
    return {"status": "shipped", "eta": "2026-01-15"}

def run_agent(ticket_text: str) -> str:
    nlu = parse_ticket(ticket_text)
    messages = [
        {"role": "system", "content": "You are a support agent. Be concise. Use tool results to answer."},
        {"role": "user", "content": ticket_text}
    ]

    if nlu["intent"] == "ORDER_STATUS" and any(e["type"] == "ORDER_ID" for e in nlu["entities"]):
        order_id = next(e["value"] for e in nlu["entities"] if e["type"] == "ORDER_ID")
        result = lookup_order(order_id)
        messages.append({"role": "user", "content": f"Order lookup result: {json.dumps(result)}"})

    resp = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=messages,
        temperature=0.3
    )
    return resp.choices[0].message.content

If your workflow requires deep reasoning, for example diagnosing a complex multi-step technical issue, you can route the ticket to a reasoning model such as DeepSeek R1 671B MoE or Kimi K2.6 directly from the same client instance.

Why Inference Economics Matter for Support Workloads

Support tickets are inherently long-context objects. A single thread may contain a dozen prior emails, inline logs, or base64-encoded screenshots. Under token-based pricing, every extra paragraph increases cost linearly. For agentic workflows that iterate over multiple tool calls and reasoning steps, the bill compounds quickly.

Oxlo.ai uses request-based pricing: one flat cost per API request regardless of prompt length. For support agents that ingest long conversation histories or large documents, this can be significantly cheaper than token-based alternatives. You can read the exact structure on the Oxlo.ai pricing page.

Additionally, popular models on Oxlo.ai run with no cold starts, which keeps latency predictable for real-time chat interfaces. The platform is fully OpenAI SDK compatible, so the code above works as a drop-in replacement without rewriting your orchestration logic.

Extending to Vision and Embeddings

Modern support queues include screenshots and PDF attachments. Oxlo.ai offers vision models such as Gemma 3 27B and Kimi VL A3B that accept image inputs through the same chat completions endpoint. You can prepend a screenshot analysis step to your NLU pipeline to extract error messages from an image before the LLM formulates a response.

For knowledge-base retrieval, Oxlo.ai provides embedding endpoints via BGE-Large and E5-Large. Chunk your documentation, index the vectors, and retrieve relevant

DEV Community