We are building a conversational support agent that handles order lookups, returns, and troubleshooting for an electronics store. It combines an LLM for dialogue with lightweight NLP for intent classification and entity extraction. I shipped a similar agent last quarter, and this is the minimal, runnable version I wish I had started with.
What you'll need
- Python 3.10 or newer
- The OpenAI SDK:
pip install openai - An Oxlo.ai API key from https://portal.oxlo.ai
Oxlo.ai uses request-based pricing, so adding a long system prompt or multi-turn history does not inflate your cost per interaction. That matters for conversational agents. See https://oxlo.ai/pricing for details.
1. Bootstrap the Oxlo.ai client
First, verify that your environment can reach Oxlo.ai and that your key is active. I like to run this sanity check before I add any application logic.
from openai import OpenAI
import os
client = OpenAI(
base_url="https://api.oxlo.ai/v1",
api_key=os.environ.get("OXLO_API_KEY", "YOUR_OXLO_API_KEY")
)
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": "You are a terse assistant."},
{"role": "user", "content": "Reply with exactly: Oxlo.ai client ready."},
],
)
print(response.choices[0].message.content)2. Write the system prompt
The system prompt is the contract that keeps the agent on task. I keep it in its own variable so non-engineers can edit it without touching code.
SYSTEM_PROMPT = """You are SupportAgent, a customer support bot for TechShop Electronics.
Your job is to help users with order status, returns, and basic troubleshooting.
Rules:
- Always ask for the order ID if the user mentions an order but does not provide one.
- For returns, confirm the item and reason, then offer a prepaid return label.
- For troubleshooting, follow the knowledge base steps exactly. Do not improvise.
- If you do not know something, say so. Do not guess.
- Keep responses under 80 words unless the user asks for detail.
Current date: 2025-01-15."""3. Add conversation memory
A support thread can last ten turns or more. We need to persist history and cap it so we do not hit context limits. I trim to the last twenty messages, which is usually five to ten turns.
class ConversationMemory:
def __init__(self, max_turns=10):
self.messages = []
self.max_turns = max_turns
def add(self, role: str, content: str):
self.messages.append({"role": role, "content": content})
# Each turn is a user + assistant pair, so 2 messages per turn
if len(self.messages) > self.max_turns * 2:
self.messages = self.messages[-(self.max_turns * 2):]
def build(self, system_prompt: str):
return [{"role": "system", "content": system_prompt}] + self.messages4. Add NLP for intent and entities
Before we ask the dialogue model to respond, we parse the user message for intent and structured data. I use a small, fast model on Oxlo.ai for this so the heavy model only has to reason, not parse. Because Oxlo.ai charges per request, splitting work across two cheap requests is often more predictable than one giant token-heavy call.
import json
def parse_user_message(user_message: str) -> dict:
prompt = f"""Analyze the message and return JSON with keys:
- intent: one of [order_status, return, troubleshooting, general]
- order_id: uppercase string or null
- product: string or null
Message: "{user_message}"
JSON:"""
response = client.chat.completions.create(
model="qwen-3-32b",
messages=[
{"role": "system", "content": "You are an NLP parser. Return only valid JSON."},
{"role": "user", "content": prompt},
],
temperature=0.0,
)
raw = response.choices[0].message.content.strip()
if raw.startswith("
```"):
raw = raw.split("\n", 1)[1].rsplit("```
", 1)[0].strip()
return json.loads(raw)5. Ground with retrieval
We simulate a product knowledge base and an order database. In production you would swap these dictionaries for a vector store and a SQL connection, but the interface stays the same.
KNOWLEDGE_BASE = {
"headphones wont charge": (
"1. Check the USB-C cable for fraying. "
"2. Use a 5V/1A wall adapter. "
"3. Hold the power button for 10 seconds to reset."
),
"router blinking red": (
"1. Reboot the modem. "
"2. Check the coax connection. "
"3. If still red, the ISP line may be down."
),
}
ORDERS_DB = {
"TS-8842": {
"item": "NoiseCancel Headphones",
"status": "delivered",
"delivered_date": "2025-01-10",
},
"TS-9911": {
"item": "Mesh Router X1",
"status": "in_transit",
"eta": "2025-01-18",
},
}
def fetch_context(intent: str, order_id: str | None, product: str | None) -> str:
chunks = []
if order_id and intent == "order_status":
record = ORDERS_DB.get(order_id.upper())
if record:
if record["status"] == "delivered":
chunks.append(
f"Order {order_id}: {record['item']}, delivered on {record['delivered_date']}."
)
else:
chunks.append(
f"Order {order_id}: {record['item']}, status {record['status']}, ETA {record['eta']}."
)
else:
chunks.append(f"Order {order_id}: not found.")
if intent == "troubleshooting" and product:
query = product.lower()
for key, answer in KNOWLEDGE_BASE.items():
if key in query or query in key:
chunks.append(f"Troubleshooting for {product}: {answer}")
break
return "\n".join(chunks)6. Assemble the agent
Now we wire the parser, retrieval, memory, and dialogue model together. I use Llama 3.3 70B on Oxlo.ai for the final response because it handles multi-turn context reliably and there are no cold starts.
class SupportAgent:
def __init__(self, client):
self.client = client
self.memory = ConversationMemory(max_turns=10)
def chat(self, user_message: str) -> str:
# NLP preprocessing
parsed = parse_user_message(user_message)
context = fetch_context(
parsed.get("intent", "general"),
parsed.get("order_id"),
parsed.get("product"),
)
# Compose system prompt with retrieved facts
system = SYSTEM_PROMPT
if context:
system += "\n\nRetrieved facts:\n" + context
# Update memory
self.memory.add("user", user_message)
# Generate response
messages = self.memory.build(system)
response = self.client.chat.completions.create(
model="llama-3.3-70b",
messages=messages,
)
assistant_msg = response.choices[0].message.content
self.memory.add("assistant", assistant_msg)
return assistant_msgRun it
Here is how to start a session and test two turns. The first turn asks about an order. The second turn asks for troubleshooting without an order ID, forcing the agent to follow the system rules.
agent = SupportAgent(client)
# Turn 1: order lookup
msg1 = "Where is my order TS-9911?"
print("User:", msg1)
print("Agent:", agent.chat(msg1))
# Turn 2: troubleshooting
msg2 = "My headphones won't charge, what should I do?"
print("\nUser:", msg2)
print("Agent:", agent.chat(msg2))Expected output:
User: Where is my order TS-9911?
Agent: Order TS-9911 (Mesh Router X1) is in_transit. ETA: 2025-01-18.
User: My headphones won't charge, what should I do?
Agent: Try these steps:
1. Check the USB-C cable for fraying.
2. Use a 5V/1A wall adapter.
3. Hold the power button for 10 seconds to reset.
Let me know if it still won't charge.Wrap-up
You now have a working conversational agent with memory, NLP preprocessing, and retrieval grounding running on Oxlo.ai. Two concrete next steps: swap the in-memory dictionaries for a real vector database like Milvus or pgvector, and add voice input by piping user audio through Oxlo.ai's Whisper endpoint before the text pipeline.
Top comments (0)