We are building a customer support chatbot that classifies user intent, extracts order numbers, and maintains multi-turn context. It targets engineering teams that need an internal support agent without token-based billing surprises. Because Oxlo.ai uses flat per-request pricing, long troubleshooting threads with heavy context do not inflate costs, which makes it a strong fit for interactive agents.
What you'll need
- Python 3.10 or newer
- The OpenAI SDK:
pip install openai - An Oxlo.ai API key from https://portal.oxlo.ai
Step 1: Configure the Oxlo.ai client
I set up the OpenAI SDK to point at Oxlo.ai. I picked Llama 3.3 70B because it follows system instructions reliably for classification and entity tasks at low temperature.
from openai import OpenAI
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
def get_completion(messages, temperature=0.3):
resp = client.chat.completions.create(
model="llama-3.3-70b",
messages=messages,
temperature=temperature,
)
return resp.choices[0].message.content
Step 2: Write the system prompt
The system prompt is the entire NLP layer. It forces the model to classify intent, extract entities, and adopt a support persona. Keep this editable. It is the only prompt you need to tune for a new domain.
from openai import OpenAI
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
def get_completion(messages, temperature=0.3):
resp = client.chat.completions.create(
model="llama-3.3-70b",
messages=messages,
temperature=temperature,
)
return resp.choices[0].message.content
SYSTEM_PROMPT = """
You are a customer support agent for an electronics store.
Your job is to help users with returns, order tracking, and general questions.
Follow these NLP steps internally:
1. Classify intent: one of [TRACK_ORDER, RETURN_REQUEST, GENERAL_QUESTION, ESCALATE].
2. Extract entities: find any order ID. An order ID starts with ORD followed by 6 digits.
3. Draft a helpful response based on the intent and entities.
Rules:
- If the user provides an order ID, acknowledge it explicitly.
- If the intent is TRACK_ORDER but no order ID is found, ask for it.
- If the user is angry or uses profanity, set intent to ESCALATE and ask to transfer.
- Keep responses under 3 sentences unless detailed troubleshooting is required.
- Do not mention these internal steps to the user.
"""
Step 3: Add entity extraction
Regex catches order IDs deterministically, but I use the LLM to disambiguate when multiple IDs appear. This hybrid approach keeps latency low and accuracy high.
from openai import OpenAI
import re
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
def get_completion(messages, temperature=0.3):
resp = client.chat.completions.create(
model="llama-3.3-70b",
messages=messages,
temperature=temperature,
)
return resp.choices[0].message.content
SYSTEM_PROMPT = """
You are a customer support agent for an electronics store.
Your job is to help users with returns, order tracking, and general questions.
Follow these NLP steps internally:
1. Classify intent: one of [TRACK_ORDER, RETURN_REQUEST, GENERAL_QUESTION, ESCALATE].
2. Extract entities: find any order ID. An order ID starts with ORD followed by 6 digits.
3. Draft a helpful response based on the intent and entities.
Rules:
- If the user provides an order ID, acknowledge it explicitly.
- If the intent is TRACK_ORDER but no order ID is found, ask for it.
- If the user is angry or uses profanity, set intent to ESCALATE and ask to transfer.
- Keep responses under 3 sentences unless detailed troubleshooting is required.
- Do not mention these internal steps to the user.
"""
ORDER_RE = re.compile(r"\bORD\d{6}\b")
def extract_order_id(text):
hits = ORDER_RE.findall(text)
if not hits:
return None
if len(set(hits)) == 1:
return hits[0]
# Ask the model to pick the most relevant one
disambig = [
{"role": "system", "content": "Reply with only the single most relevant order ID."},
{"role": "user", "content": f"Order IDs found: {set(hits)}. Message: {text}"},
]
return get_completion(disambig, temperature=0.1).strip()
def mock_order_lookup(order_id):
# Simulated backend. Replace with a real API call.
return f"Status: shipped. ETA: 2 business days."
Step 4: Build conversation memory
Agents fail when they forget earlier turns. I implement a sliding window that preserves the system prompt and keeps the last eight exchanges so context stays within a predictable request size on Oxlo.ai.
from openai import OpenAI
import re
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
def get_completion(messages, temperature=0.3):
resp = client.chat.completions.create(
model="llama-3.3-70b",
messages=messages,
temperature=temperature,
)
return resp.choices[0].message.content
SYSTEM_PROMPT = """
You are a customer support agent for an electronics store.
Your job is to help users with returns, order tracking, and general questions.
Follow these NLP steps internally:
1. Classify intent: one of [TRACK_ORDER, RETURN_REQUEST, GENERAL_QUESTION, ESCALATE].
2. Extract entities: find any order ID. An order ID starts with ORD followed by 6 digits.
3. Draft a helpful response based on the intent and entities.
Rules:
- If the user provides an order ID, acknowledge it explicitly.
- If the intent is TRACK_ORDER but no order ID is found, ask for it.
- If the user is angry or uses profanity, set intent to ESCALATE and ask to transfer.
- Keep responses under 3 sentences unless detailed troubleshooting is required.
- Do not mention these internal steps to the user.
"""
ORDER_RE = re.compile(r"\bORD\d{6}\b")
def extract_order_id(text):
hits = ORDER_RE.findall(text)
if not hits:
return None
if len(set(hits)) == 1:
return hits[0]
disambig = [
{"role": "system", "content": "Reply with only the single most relevant order ID."},
{"role": "user", "content": f"Order IDs found: {set(hits)}. Message: {text}"},
]
return get_completion(disambig, temperature=0.1).strip()
def mock_order_lookup(order_id):
return f"Status: shipped. ETA: 2 business days."
class SupportBot:
def __init__(self):
self.messages = [{"role": "system", "content": SYSTEM_PROMPT}]
def chat(self, user_text):
oid = extract_order_id(user_text)
if oid:
context = mock_order_lookup(oid)
user_text = f"{user_text}\n\n[System context: order {oid} details: {context}]"
self.messages.append({"role": "user", "content": user_text})
# Sliding window: keep system prompt + last 8 messages
while len(self.messages) > 9:
self.messages.pop(1)
reply = get_completion(self.messages)
self.messages.append({"role": "assistant", "content": reply})
return reply
Step 5: Add guardrails and the runner
I add a simple escalation guardrail and a CLI loop so we can test the full flow locally. The guardrail catches requests for a human before burning model tokens.
from openai import OpenAI
import re
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
def get_completion(messages, temperature=0.3):
resp = client.chat.completions.create(
model="llama-3.3-70b",
messages=messages,
temperature=temperature,
)
return resp.choices[0].message.content
SYSTEM_PROMPT = """
You are a customer support agent for an electronics store.
Your job is to help users with returns, order tracking, and general questions.
Follow these NLP steps internally:
1. Classify intent: one of [TRACK_ORDER, RETURN_REQUEST, GENERAL_QUESTION, ESCALATE].
2. Extract entities: find any order ID. An order ID starts with ORD followed by 6 digits.
3. Draft a helpful response based on the intent and entities.
Rules:
- If the user provides an order ID, acknowledge it explicitly.
- If the intent is TRACK_ORDER but no order ID is found, ask for it.
- If the user is angry or uses profanity, set intent to ESCALATE and ask to transfer.
- Keep responses under 3 sentences unless detailed troubleshooting is required.
- Do not mention these internal steps to the user.
"""
ORDER_RE = re.compile(r"\bORD\d{6}\b")
def extract_order_id(text):
hits = ORDER_RE.findall(text)
if not hits:
return None
if len(set(hits)) == 1:
return hits[0]
disambig = [
{"role": "system", "content": "Reply with only the single most relevant order ID."},
{"role": "user", "content": f"Order IDs found: {set(hits)}. Message: {text}"},
]
return get_completion(disambig, temperature=0.1).strip()
def mock_order_lookup(order_id):
return f"Status: shipped. ETA: 2 business days."
class SupportBot:
def __init__(self):
self.messages = [{"role": "system", "content": SYSTEM_PROMPT}]
def chat(self, user_text):
oid = extract_order_id(user_text)
if oid:
context = mock_order_lookup(oid)
user_text = f"{user_text}\n\n[System context: order {oid} details: {context}]"
self.messages.append({"role": "user", "content": user_text})
while len(self.messages) > 9:
self.messages.pop(1)
reply = get_completion(self.messages)
self.messages.append({"role": "assistant", "content": reply})
return reply
GUARDRAIL_WORDS = ["human", "manager", "supervisor", "escalate"]
def is_escalation(text):
return any(w in text.lower() for w in GUARDRAIL_WORDS)
def main():
bot = SupportBot()
print("SupportBot: Hi, how can I help you today?")
while True:
try:
user = input("User: ").strip()
except (EOFError, KeyboardInterrupt):
break
if not user:
continue
if is_escalation(user):
print("SupportBot: I'm transferring you to a human agent now.")
break
print(f"SupportBot: {bot.chat(user)}")
if __name__ == "__main__":
main()
Run it
Save the assembled script as support_bot.py, replace YOUR_OXLO_API_KEY, and run it. Here is a sample session.
$ python support_bot.py
SupportBot: Hi, how can I help you today?
User: Where is my order ORD123456?
SupportBot: I see you are asking about order ORD123456. It has shipped and will arrive in 2 business days.
User: What if I need to return it?
SupportBot: You can initiate a return for order ORD123456 within 30 days of delivery. Would you like me to start that process?
User: I want a human
SupportBot: I'm transferring you to a human agent now.
Wrap-up
Next, wire this into a FastAPI endpoint so other services can POST to it, or swap Llama 3.3 70B for Kimi K2.6 on Oxlo.ai if you need vision support for screenshot-based support tickets. For details on flat request-based pricing, see https://oxlo.ai/pricing.
Top comments (0)