Today we are building a customer support agent that handles order questions, return policies, and troubleshooting over a multi-turn conversation. I will walk through the exact code I use to wire up memory, a knowledge base, and tool calling against Oxlo.ai's OpenAI-compatible API. Because Oxlo.ai charges a flat rate per request instead of per token, you can keep a long conversation history without the cost ballooning.
What you'll need
- Python 3.10 or newer
- The OpenAI SDK:
pip install openai - An Oxlo.ai API key from https://portal.oxlo.ai
Step 1: Scaffold the Oxlo.ai client
First I verify that the SDK can reach Oxlo.ai and that my key works. A single call to Llama 3.3 70B confirms everything is wired correctly.
from openai import OpenAI
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[{"role": "user", "content": "Say 'Connection OK'"}],
)
print(response.choices[0].message.content)
Step 2: Write the system prompt
The system prompt sets the agent's identity and guardrails. I keep it short so it leaves plenty of context window for the actual conversation.
from openai import OpenAI
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
SYSTEM_PROMPT = """You are a support agent for TechMart, an online electronics store.
You help customers with order status, return policies, and basic troubleshooting.
Rules:
- Be concise and friendly.
- If a customer asks about an order, you must use the lookup_order tool.
- Do not make up order details.
- Current date: 2024-05-20."""
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": "What is your return policy?"},
],
)
print(response.choices[0].message.content)
Step 3: Add memory and a knowledge base
Conversational AI needs state. I append a lightweight Python dictionary as context and maintain a messages list so the model sees the full history on every turn.
from openai import OpenAI
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
SYSTEM_PROMPT = """You are a support agent for TechMart, an online electronics store.
You help customers with order status, return policies, and basic troubleshooting.
Rules:
- Be concise and friendly.
- If a customer asks about an order, you must use the lookup_order tool.
- Do not make up order details.
- Current date: 2024-05-20."""
KB = {
"return_policy": "Returns are accepted within 30 days with original packaging.",
"warranty": "All electronics come with a 1-year manufacturer warranty.",
"hours": "Support hours are 9 AM to 6 PM ET, Monday through Friday.",
}
def build_context():
return "\n".join([f"{k}: {v}" for k, v in KB.items()])
messages = [
{"role": "system", "content": SYSTEM_PROMPT + "\n\nContext:\n" + build_context()},
{"role": "user", "content": "What are your support hours?"},
]
response = client.chat.completions.create(model="llama-3.3-70b", messages=messages)
assistant_reply = response.choices[0].message.content
messages.append({"role": "assistant", "content": assistant_reply})
# Second turn to test memory
messages.append({"role": "user", "content": "And the return policy?"})
response = client.chat.completions.create(model="llama-3.3-70b", messages=messages)
print(response.choices[0].message.content)
Step 4: Define the order lookup tool
To answer "Where is my order?" the agent needs real data. I expose a simulated order database through a function schema so the LLM can request lookups via Oxlo.ai's tool-use support.
from openai import OpenAI
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
SYSTEM_PROMPT = """You are a support agent for TechMart, an online electronics store.
You help customers with order status, return policies, and basic troubleshooting.
Rules:
- Be concise and friendly.
- If a customer asks about an order, you must use the lookup_order tool.
- Do not make up order details.
- Current date: 2024-05-20."""
KB = {
"return_policy": "Returns are accepted within 30 days with original packaging.",
"warranty": "All electronics come with a 1-year manufacturer warranty.",
"hours": "Support hours are 9 AM to 6 PM ET, Monday through Friday.",
}
def build_context():
return "\n".join([f"{k}: {v}" for k, v in KB.items()])
ORDERS = {
"TM-1001": {"status": "shipped", "item": "Wireless Headphones", "eta": "2024-05-22"},
"TM-1002": {"status": "processing", "item": "USB-C Cable", "eta": "2024-05-24"},
}
def lookup_order(order_id: str):
return ORDERS.get(order_id, {"error": "Order not found"})
tools = [
{
"type": "function",
"function": {
"name": "lookup_order",
"description": "Get the status and details of a customer order by ID.",
"parameters": {
"type": "object",
"properties": {
"order_id": {
"type": "string",
"description": "The TechMart order ID, e.g. TM-1001",
}
},
"required": ["order_id"],
},
},
}
]
messages = [
{"role": "system", "content": SYSTEM_PROMPT + "\n\nContext:\n" + build_context()},
{"role": "user", "content": "Where is my order TM-1001?"},
]
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=messages,
tools=tools,
)
print(response.choices[0].message)
Step 5: Build the conversation loop
Finally I wrap the logic in a small run_agent function. It appends the user message, sends the payload to Oxlo.ai, executes any tool calls locally, and sends the results back for a final answer.
from openai import OpenAI
import json
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
SYSTEM_PROMPT = """You are a support agent for TechMart, an online electronics store.
You help customers with order status, return policies, and basic troubleshooting.
Rules:
- Be concise and friendly.
- If a customer asks about an order, you must use the lookup_order tool.
- Do not make up order details.
- Current date: 2024-05-20."""
KB = {
"return_policy": "Returns are accepted within 30 days with original packaging.",
"warranty": "All electronics come with a 1-year manufacturer warranty.",
"hours": "Support hours are 9 AM to 6 PM ET, Monday through Friday.",
}
def build_context():
return "\n".join([f"{k}: {v}" for k, v in KB.items()])
ORDERS = {
"TM-1001": {"status": "shipped", "item": "Wireless Headphones", "eta": "2024-05-22"},
"TM-1002": {"status": "processing", "item": "USB-C Cable", "eta": "2024-05-24"},
}
def lookup_order(order_id: str):
return ORDERS.get(order_id, {"error": "Order not found"})
tools = [
{
"type": "function",
"function": {
"name": "lookup_order",
"description": "Get the status and details of a customer order by ID.",
"parameters": {
"type": "object",
"properties": {
"order_id": {
"type": "string",
"description": "The TechMart order ID, e.g. TM-1001",
}
},
"required": ["order_id"],
},
},
}
]
def run_agent(user_input: str, messages: list):
if not messages:
system_msg = SYSTEM_PROMPT + "\n\nContext:\n" + build_context()
messages.append({"role": "system", "content": system_msg})
messages.append({"role": "user", "content": user_input})
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=messages,
tools=tools,
)
msg = response.choices[0].message
if msg.tool_calls:
messages.append({
"role": "assistant",
"content": msg.content or "",
"tool_calls": [tc.model_dump() for tc in msg.tool_calls],
})
for tc in msg.tool_calls:
if tc.function.name == "lookup_order":
args = json.loads(tc.function.arguments)
result = lookup_order(args["order_id"])
messages.append({
"role": "tool",
"tool_call_id": tc.id,
"content": json.dumps(result),
})
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=messages,
tools=tools,
)
msg = response.choices[0].message
messages.append({"role": "assistant", "content": msg.content})
return msg.content
Run it
Call the agent with a few messages and watch it use the tool when needed. Because Oxlo.ai has no cold starts on popular models, the round trip is fast even with long context.
messages = []
print(run_agent("Where is my order TM-1001?", messages))
print(run_agent("Can I return it if it breaks?", messages))
Example output:
Your order TM-1001 (Wireless Headphones) has shipped and is expected to arrive on May 22nd.
Yes, you can return items within 30 days with the original packaging. Your Wireless Headphones also come with a 1-year manufacturer warranty.
Wrap-up
From here you can add streaming responses for real-time typing indicators, or swap in qwen-3-32b if you need multilingual support. If you plan to ship this to production, take a look at Oxlo.ai's request-based pricing at https://oxlo.ai/pricing. It stays predictable even when user sessions get long.
Top comments (0)