We are going to build a customer support triage agent that classifies incoming tickets by priority and drafts an initial response. I will show you the exact debugging hooks I add so that when the model misprioritizes a ticket or hallucinates a refund policy, I can trace the failure, patch the prompt, and verify the fix in under five minutes. We will run everything against Oxlo.ai so you are not burning tokens on unpredictable per-token bills.
What you'll need
- Python 3.10 or newer
- The OpenAI SDK:
pip install openai - An Oxlo.ai API key from https://portal.oxlo.ai
- A few sample support tickets to test with
Oxlo.ai uses flat request-based pricing, so you can iterate on long prompts without watching token counters tick up. See https://oxlo.ai/pricing for details.
Step 1: Scaffold the agent with Oxlo.ai
I start with a minimal triage function that sends a customer ticket to Llama 3.3 70B and returns the raw text. This gives us a baseline we can debug against.
SYSTEM_PROMPT = """You are a support triage agent for a SaaS platform.
1. Classify the ticket priority: low, medium, high, or critical.
2. Draft a brief, empathetic response.
3. If the user mentions a refund, cite only this policy: "Refunds are processed within 5-7 business days."
Do not invent features or policy details."""from openai import OpenAI
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
def draft_response(ticket_text: str) -> str:
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": ticket_text},
],
)
return response.choices[0].message.content
if __name__ == "__main__":
ticket = "I was double-charged this month and I need my money back immediately."
print(draft_response(ticket))Step 2: Enforce JSON output for predictable parsing
Free text is hard to regression test. I switch the model to JSON mode and tighten the system prompt so every reply contains exactly four fields we can assert against.
import json
from openai import OpenAI
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
SYSTEM_PROMPT = """You are a support triage agent for a SaaS platform.
Respond ONLY with a JSON object containing:
- priority: one of low, medium, high, critical
- reasoning: string, max 15 words
- response_draft: string
- refund_mentioned: boolean
If the user mentions a refund, cite only: "Refunds are processed within 5-7 business days."""def triage(ticket_text: str) -> dict:
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": ticket_text},
],
response_format={"type": "json_object"},
)
return json.loads(response.choices[0].message.content)
if __name__ == "__main__":
ticket = "I was double-charged this month and I need my money back immediately."
print(json.dumps(triage(ticket), indent=2))Step 3: Instrument every request with a local trace log
When a ticket is misprioritized in production, I need to see the exact prompt and response. I add a SQLite trace table that records every Oxlo.ai call with timing.
import sqlite3
import json
import time
from openai import OpenAI
DB_PATH = "triage_traces.db"
def init_db():
conn = sqlite3.connect(DB_PATH)
conn.execute("""
CREATE TABLE IF NOT EXISTS traces (
id INTEGER PRIMARY KEY,
timestamp REAL,
model TEXT,
messages TEXT,
response TEXT,
latency_ms INTEGER
)
""")
conn.commit()
conn.close()
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
SYSTEM_PROMPT = """You are a support triage agent for a SaaS platform.
Respond ONLY with a JSON object containing:
- priority: one of low, medium, high, critical
- reasoning: string, max 15 words
- response_draft: string
- refund_mentioned: boolean
If the user mentions a refund, cite only: "Refunds are processed within 5-7 business days."""def traced_triage(ticket_text: str) -> dict:
messages = [
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": ticket_text},
]
start = time.time()
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=messages,
response_format={"type": "json_object"},
)
latency = int((time.time() - start) * 1000)
result = json.loads(response.choices[0].message.content)
conn = sqlite3.connect(DB_PATH)
conn.execute(
"INSERT INTO traces (timestamp, model, messages, response, latency_ms) VALUES (?, ?, ?, ?, ?)",
(time.time(), "llama-3.3-70b", json.dumps(messages), json.dumps(result), latency)
)
conn.commit()
conn.close()
return result
init_db()Step 4: Build a prompt diff harness
Before I ship a new prompt, I want to see how it behaves against the same ticket side by side with the current version. I keep candidate prompts in a JSON file and run them through the same function.
import json
import os
from openai import OpenAI
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
BASELINE_PROMPT = """You are a support triage agent for a SaaS platform.
Respond ONLY with a JSON object containing:
- priority: one of low, medium, high, critical
- reasoning: string, max 15 words
- response_draft: string
- refund_mentioned: boolean
If the user mentions a refund, cite only: "Refunds are processed within 5-7 business days."""
DEFAULT_PROMPTS = {
"baseline": BASELINE_PROMPT,
"strict": BASELINE_PROMPT + "\nIf the user says 'down' or 'broken', priority must be critical."
}
def load_prompts(path="prompts.json"):
if not os.path.exists(path):
with open(path, "w") as f:
json.dump(DEFAULT_PROMPTS, f, indent=2)
with open(path) as f:
return json.load(f)
def run_variant(ticket_text: str, prompt_text: str) -> dict:
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": prompt_text},
{"role": "user", "content": ticket_text},
],
response_format={"type": "json_object"},
)
return json.loads(response.choices[0].message.content)
def diff_prompts(ticket_text: str):
prompts = load_prompts()
for name, text in prompts.items():
result = run_variant(ticket_text, text)
print(f"--- {name} ---")
print(json.dumps(result, indent=2))
if __name__ == "__main__":
diff_prompts("The API has been down for two hours and our checkout is broken.")Step 5: Lock in quality with a regression suite
I keep a CSV of tickets with expected priorities. After any prompt edit, I run the full suite and surface failures immediately. The CSV should have columns: ticket, expected_priority.
import csv
import json
from openai import OpenAI
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
SYSTEM_PROMPT = """You are a support triage agent for a SaaS platform.
Respond ONLY with a JSON object containing:
- priority: one of low, medium, high, critical
- reasoning: string, max 15 words
- response_draft: string
- refund_mentioned: boolean
If the user mentions a refund, cite only: "Refunds are processed within 5-7 business days."""
def run_eval(csv_path="eval_set.csv"):
passed = 0
failed = 0
with open(csv_path, newline="") as f:
reader = csv.DictReader(f)
for row in reader:
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": row["ticket"]},
],
response_format={"type": "json_object"},
)
result = json.loads(response.choices[0].message.content)
actual = result.get("priority")
expected = row["expected_priority"]
if actual == expected:
passed += 1
else:
failed += 1
print(f"FAIL: {row['ticket'][:50]}... expected {expected}, got {actual}")
print(f"Score: {passed}/{passed + failed}")
return failed == 0
if __name__ == "__main__":
run_eval()Run it
Putting the tracer and triage logic together, you can run a single ticket end to end and inspect the structured output.
if __name__ == "__main__":
init_db()
ticket = "I was double-charged this month and I need my money back immediately."
result = traced_triage(ticket)
print(json.dumps(result, indent=2))Example output:
{
"priority": "high",
"reasoning": "Billing error with refund request",
"response_draft": "I am sorry for the double charge. Refunds are processed within 5-7 business days.",
"refund_mentioned": true
}Wrap-up
You now have a triage agent with built-in observability and regression guardrails. Two moves I would make next. First, wire the SQLite tracer into FastAPI middleware so every production request is captured automatically. Second, swap in DeepSeek R1 671B MoE or Kimi K2.6 when you need advanced reasoning on ambiguous tickets, knowing Oxlo.ai's flat per-request pricing keeps the cost predictable even on long context windows.
Top comments (0)