We are building a stateful customer support agent that handles order lookups, refund requests, and escalation across multi-turn conversations. It is aimed at engineering teams who need to automate tier-1 support without losing context or ballooning inference costs. I will walk through the exact code I shipped, using Oxlo.ai for the LLM layer.
What you'll need
- Python 3.10 or newer
- The OpenAI SDK:
pip install openai - An Oxlo.ai API key from https://portal.oxlo.ai
Step 1: Verify connectivity to Oxlo.ai
I always start with a smoke test to confirm the endpoint and key are healthy before adding any logic.
from openai import OpenAI
import os
client = OpenAI(
base_url="https://api.oxlo.ai/v1",
api_key=os.environ.get("OXLO_API_KEY", "YOUR_OXLO_API_KEY")
)
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[{"role": "user", "content": "ping"}],
)
print("Endpoint ready:", response.choices[0].message.content[:20])
Step 2: Lock down the system prompt
The system prompt is the contract that keeps the agent from drifting. I keep it in a top-level constant so product teams can edit it without touching the router.
SYSTEM_PROMPT = """You are a tier-1 support agent for Acme Gadgets.
You help customers with order tracking, refunds, and basic troubleshooting.
Guidelines:
- Always ask for an order ID before looking up details.
- If a customer asks for a refund, check order status first.
- Never invent order data. Use the lookup_order tool.
- Keep responses concise, max two sentences.
- If the user mentions fire or safety hazards, reply 'ESCALATE-HUMAN'."""
Step 3: Build a minimal session manager
Dialogue lives or dies by state. This class appends user and assistant messages so the model sees the full thread on every turn.
class SupportSession:
def __init__(self):
self.messages = [{"role": "system", "content": SYSTEM_PROMPT}]
def user_says(self, text):
self.messages.append({"role": "user", "content": text})
def assistant_says(self, text):
self.messages.append({"role": "assistant", "content": text})
return text
Step 4: Define the tool schema and a stub backend
I expose one tool for order lookups. In production this hits your database, but a hardcoded dict is enough to verify the loop.
import json
TOOLS = [
{
"type": "function",
"function": {
"name": "lookup_order",
"description": "Retrieve current status of a customer order",
"parameters": {
"type": "object",
"properties": {
"order_id": {
"type": "string",
"description": "Order identifier, e.g. ORD-12345"
}
},
"required": ["order_id"]
}
}
}
]
def lookup_order(order_id: str):
db = {
"ORD-12345": {"status": "delivered", "item": "Mechanical Keyboard", "refund_eligible": True},
"ORD-67890": {"status": "in_transit", "item": "USB-C Hub", "refund_eligible": False}
}
return db.get(order_id, {"status": "not_found"})
Step 5: Wire up the turn handler with automatic tool execution
This is the core router. It sends the conversation to Oxlo.ai, detects if the model wants to call a tool, executes the function locally, and sends the result back for a final natural-language response.
def handle_turn(client, session, user_text):
session.user_says(user_text)
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=session.messages,
tools=TOOLS,
tool_choice="auto",
)
msg = response.choices[0].message
if msg.tool_calls:
session.messages.append(msg)
for tc in msg.tool_calls:
if tc.function.name == "lookup_order":
args = json.loads(tc.function.arguments)
result = lookup_order(args["order_id"])
session.messages.append({
"role": "tool",
"tool_call_id": tc.id,
"content": json.dumps(result)
})
follow_up = client.chat.completions.create(
model="llama-3.3-70b",
messages=session.messages,
)
reply = follow_up.choices[0].message.content
return session.assistant_says(reply)
return session.assistant_says(msg.content)
Step 6: Compress context when threads get long
After a dozen turns the thread gets noisy. Because Oxlo.ai charges per request instead of per token, I can afford to run a summarization pass without sweating the length. This keeps the model focused while capping the message list.
def maybe_compress(session, client):
if len(session.messages) > 14:
history = json.dumps(session.messages[1:-6])
summary_resp = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": "Summarize this support chat for an agent. Keep facts, drop fluff."},
{"role": "user", "content": history}
],
)
summary = summary_resp.choices[0].message.content
session.messages = [
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": f"Previous conversation summary: {summary}"}
] + session.messages[-6:]
Run it
Here is how I test the agent locally. I inject the compression check between turns to simulate a long session.
session = SupportSession()
print("Bot:", handle_turn(client, session, "Hi, where is my order? I think the ID is ORD-12345"))
maybe_compress(session, client)
print("Bot:", handle_turn(client, session, "Can I get a refund then?"))
Example output:
Bot: I found your order. It shows as delivered.
Bot: Yes, that order is eligible for a refund. I can start that process for you now.
Next steps
Swap in kimi-k2.6 for the summarization step if you want stronger reasoning over long context, or try deepseek-v3.2 on the free tier to prototype without spending quota. If you move to production, replace the lookup_order stub with an async call to your orders API so the tool handler does not block the event loop.
Top comments (0)