We are going to build a customer support chatbot that uses a lightweight NLU layer for intent classification and an LLM for natural response generation. This hybrid design keeps behavior deterministic on critical paths while letting the language model handle open-ended conversation. I run it on Oxlo.ai because flat per-request pricing keeps costs predictable even when I add an extra NLU call on every turn, and you can check https://oxlo.ai/pricing to see how it compares.
What you'll need
- Python 3.10 or newer
- An Oxlo.ai API key from https://portal.oxlo.ai
- The OpenAI SDK:
pip install openai
Step 1: Configure the Oxlo.ai client
Install the SDK and point it at Oxlo.ai. I use Llama 3.3 70B as the backbone because it follows instructions reliably for both the NLU and response stages.
from openai import OpenAI
import os
import json
import re
client = OpenAI(
base_url="https://api.oxlo.ai/v1",
api_key=os.environ.get("OXLO_API_KEY")
)
MODEL = "llama-3.3-70b"
def chat(messages, temperature=0.3):
response = client.chat.completions.create(
model=MODEL,
messages=messages,
temperature=temperature,
)
return response.choices[0].message.content
Step 2: Build the NLU parser
Before generating a response, I extract intent and entities with a dedicated LLM call. This keeps the main conversation model focused on tone and policy. I send a structured prompt and parse JSON from the output.
def parse_intent_and_entities(user_message: str):
nlu_prompt = f"""Analyze the customer message and return ONLY a JSON object with no markdown formatting.
Fields:
- intent: one of [ORDER_STATUS, REFUND_REQUEST, SUPPORT_ESCALATION, GENERAL_QUESTION]
- entities: dict with keys order_id (format ORD-12345), date, product_name
- urgency: low, medium, or high
Message: "{user_message}"
JSON:"""
messages = [
{"role": "system", "content": "You are an NLU engine. Output valid JSON only."},
{"role": "user", "content": nlu_prompt},
]
raw = client.chat.completions.create(
model="qwen-3-32b",
messages=messages,
temperature=0.1,
).choices[0].message.content.strip()
# Strip markdown fences if the model adds them
raw = re.sub(r"^
```(?:json)?\s*", "", raw)
raw = re.sub(r"\s*```
$", "", raw)
return json.loads(raw)
Step 3: Create the intent router
Each intent maps to a handler that returns context for the final response. I keep these as pure functions so I can unit test them without calling the LLM.
def handle_order_status(entities: dict) -> str:
order_id = entities.get("order_id", "UNKNOWN")
# In production, query your database here.
return f"Order {order_id} is currently in transit and expected by Friday."
def handle_refund_request(entities: dict, urgency: str) -> str:
order_id = entities.get("order_id", "UNKNOWN")
if urgency == "high":
return f"Order {order_id} qualifies for an expedited refund. I have initiated the process."
return f"Order {order_id} refund request has been logged. Processing takes 3 to 5 business days."
def handle_escalation(entities: dict) -> str:
return "I am connecting you to a human agent now. Estimated wait time is 2 minutes."
def handle_general(question: str) -> str:
return None
Step 4: Write the system prompt
The system prompt defines tone, guardrails, and how to incorporate the NLU context. I keep it explicit so the model does not hallucinate policies.
SYSTEM_PROMPT = """You are a customer support assistant for an electronics store.
Rules:
- Be concise, polite, and helpful.
- If NLU context includes a handler result, use it as ground truth. Do not contradict it.
- If the intent is GENERAL_QUESTION, answer based on store policies.
- Never promise delivery dates unless provided in the handler result.
- If the user asks for a human, always comply.
"""
Step 5: Wire everything into the chatbot
I assemble a small class that maintains conversation history, runs the NLU step, routes to the correct handler, and then asks the LLM to produce the final user-facing message.
class SupportBot:
def __init__(self):
self.history = [{"role": "system", "content": SYSTEM_PROMPT}]
def process(self, user_message: str) -> str:
# NLU step
nlu = parse_intent_and_entities(user_message)
intent = nlu.get("intent", "GENERAL_QUESTION")
entities = nlu.get("entities", {})
urgency = nlu.get("urgency", "low")
# Routing
handler_result = None
if intent == "ORDER_STATUS":
handler_result = handle_order_status(entities)
elif intent == "REFUND_REQUEST":
handler_result = handle_refund_request(entities, urgency)
elif intent == "SUPPORT_ESCALATION":
handler_result = handle_escalation(entities)
# Build augmented user message
augmented = f"[NLU: intent={intent}, urgency={urgency}]\n"
if handler_result:
augmented += f"[Context: {handler_result}]\n"
augmented += f"User: {user_message}"
self.history.append({"role": "user", "content": augmented})
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=self.history,
temperature=0.4,
).choices[0].message.content
self.history.append({"role": "assistant", "content": response})
return response
Step 6: Add multi-turn memory management
For a real deployment, I cap the history so the context window does not grow forever. I keep the system prompt plus the last 10 turns.
class SupportBot:
def __init__(self):
self.history = [{"role": "system", "content": SYSTEM_PROMPT}]
def trim_history(self, max_turns: int = 10):
if len(self.history) <= 1:
return
keep = 1 + (max_turns * 2)
if len(self.history) > keep:
self.history = [self.history[0]] + self.history[-(keep - 1):]
def process(self, user_message: str) -> str:
self.trim_history(10)
nlu = parse_intent_and_entities(user_message)
intent = nlu.get("intent", "GENERAL_QUESTION")
entities = nlu.get("entities", {})
urgency = nlu.get("urgency", "low")
handler_result = None
if intent == "ORDER_STATUS":
handler_result = handle_order_status(entities)
elif intent == "REFUND_REQUEST":
handler_result = handle_refund_request(entities, urgency)
elif intent == "SUPPORT_ESCALATION":
handler_result = handle_escalation(entities)
augmented = f"[NLU: intent={intent}, urgency={urgency}]\n"
if handler_result:
augmented += f"[Context: {handler_result}]\n"
augmented += f"User: {user_message}"
self.history.append({"role": "user", "content": augmented})
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=self.history,
temperature=0.4,
).choices[0].message.content
self.history.append({"role": "assistant", "content": response})
return response
Run it
Here is a short script that instantiates the bot and runs a few test messages. Because Oxlo.ai charges a flat rate per request, the extra NLU call in each turn does not surprise me with token costs.
if __name__ == "__main__":
bot = SupportBot()
queries = [
"Where is my order ORD-98765?",
"I want a refund for the bluetooth speaker. It arrived broken.",
"This is urgent, I need to speak to a person right now.",
]
for q in queries:
print(f"User: {q}")
reply = bot.process(q)
print(f"Bot: {reply}\n")
Example output:
User: Where is my order ORD-98765?
Bot: Your order ORD-98765 is currently in transit and expected by Friday.
User: I want a refund for the bluetooth speaker. It arrived broken.
Bot: I am sorry to hear the bluetooth speaker arrived damaged. Your refund request has been logged, and processing takes 3 to 5 business days.
User: This is urgent, I need to speak to a person right now.
Bot: I understand this is urgent. I am connecting you to a human agent now. Estimated wait time is 2 minutes.
Next steps
Replace the stub handlers with live API calls to your order database using Oxlo.ai function calling. You can also move the NLU step to DeepSeek V3.2 on the free tier to eliminate preprocessing costs while you experiment.
Top comments (0)