DEV Community

shashank ms
shashank ms

Posted on

LLM for Natural Language Understanding in Chatbot Development

#learnai #oxlo #ai

We are building an intent classification and entity extraction pipeline for a customer support chatbot. This gives you deterministic routing and structured data without giving up the flexibility of natural language. I will walk through the exact code I run against Oxlo.ai's flat per-request API so long user messages do not inflate costs.

What you'll need

Step 1: Configure the Oxlo.ai client

I start by instantiating the OpenAI-compatible client against Oxlo.ai. I test with Llama 3.3 70B because it follows JSON instructions reliably.

from openai import OpenAI
import json
import re

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

# Verify connectivity
response = client.chat.completions.create(
    model="llama-3.3-70b",
    messages=[{"role": "user", "content": "ping"}],
    max_tokens=5
)
print(response.choices[0].message.content)

Step 2: Define the NLU system prompt

The system prompt is the contract between your code and the model. I keep it in a constant so I can tweak it without touching logic. It forces strict JSON output with intent and entities.

SYSTEM_PROMPT = """You are an NLU engine for an e-commerce support chatbot.
Analyze the user's message and return only a JSON object with this exact structure:
{
  "intent": "CHECK_ORDER_STATUS" | "RETURN_ITEM" | "PRODUCT_QUESTION" | "TALK_TO_HUMAN",
  "entities": {
    "order_id": string or null,
    "product_name": string or null,
    "date": string or null
  },
  "confidence": number
}

Rules:
- order_id must match # followed by exactly 6 digits, or be null.
- If the user asks for a person or says they are frustrated, intent is TALK_TO_HUMAN.
- Do not wrap the JSON in markdown."""

Step 3: Build the NLU parser

This function sends the user message to Oxlo.ai and returns a Python dict. I set temperature to 0.1 to keep outputs stable. I also strip markdown fences just in case.

def parse_message(user_message: str) -> dict:
    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": user_message},
        ],
        temperature=0.1,
        max_tokens=200
    )
    
    raw = response.choices[0].message.content.strip()
    
    # Remove markdown fences if the model added them
    raw = re.sub(r"^

```(?:json)?\s*", "", raw)
    raw = re.sub(r"\s*```

$", "", raw)
    
    return json.loads(raw)

Step 4: Add response routing

Now I wire the parsed intent to simple response logic. In production you might hit a database or hand off to a human, but the router stays clean because the NLU already did the hard work.

def route_response(nlu: dict, user_message: str) -> str:
    intent = nlu.get("intent", "TALK_TO_HUMAN")
    entities = nlu.get("entities", {})
    confidence = nlu.get("confidence", 0.0)
    
    if confidence < 0.7:
        return "I did not quite catch that. Could you rephrase?"
    
    if intent == "CHECK_ORDER_STATUS":
        order_id = entities.get("order_id")
        if order_id:
            return f"Looking up order {order_id} now. One moment."
        return "I can check that. What is your order ID? It should look like #123456."
    
    if intent == "RETURN_ITEM":
        product = entities.get("product_name", "that item")
        return f"I will start a return for {product}. A label will be emailed to you."
    
    if intent == "PRODUCT_QUESTION":
        product = entities.get("product_name", "this product")
        return f"Here are the specs for {product}."
    
    return "Transferring you to a human agent now."

Step 5: Handle multi-turn context with Qwen 3 32B

If the user replies with an order ID after being asked, the NLU should still classify it as CHECK_ORDER_STATUS. I keep the last few turns in memory and pass them to Qwen 3 32B, which handles agentic context well.

class ChatBot:
    def __init__(self):
        self.history = []
    
    def chat(self, user_message: str) -> str:
        self.history.append({"role": "user", "content": user_message})
        
        # NLU pass with recent history for context
        response = client.chat.completions.create(
            model="qwen-3-32b",
            messages=[
                {"role": "system", "content": SYSTEM_PROMPT},
                *self.history[-4:]
            ],
            temperature=0.1,
            max_tokens=200
        )
        
        raw = response.choices[0].message.content.strip()
        raw = re.sub(r"^

```(?:json)?\s*", "", raw)
        raw = re.sub(r"\s*```

$", "", raw)
        nlu = json.loads(raw)
        
        reply = route_response(nlu, user_message)
        self.history.append({"role": "assistant", "content": reply})
        return reply

Run it

This is the test script I run locally to verify the flow.

bot = ChatBot()

messages = [
    "Where is my order? I ordered a backpack last week.",
    "It is order #449201.",
    "Actually I just want to return it.",
]

for msg in messages:
    print(f"User: {msg}")
    print(f"Bot: {bot.chat(msg)}\n")

Example output:

User: Where is my order? I ordered a backpack last week.
Bot: I can check that. What is your order ID? It should look like #123456.

User: It is order #449201.
Bot: Looking up order #449201 now. One moment.

User: Actually I just want to return it.
Bot: I will start a return for that item. A label will be emailed to you.

Wrap-up

That is the core NLU layer I deploy for support bots. To push this further, connect the CHECK_ORDER_STATUS intent to your CRM API by swapping the string response for a real lookup, or add a confidence threshold that automatically creates a ticket instead of guessing. If you expect heavy traffic with long transcripts, Oxlo.ai's request-based pricing keeps costs flat regardless of context length, which makes multi-turn memory cheap to maintain. See https://oxlo.ai/pricing for plan details.

Top comments (0)