We are building a support ticket triage agent that reads messy email threads, queries an internal knowledge base, and returns structured JSON drafts. It helps teams stop manually summarizing long conversations and guessing at policy answers.
What you'll need
- Python 3.10 or newer
pip install openai- An Oxlo.ai API key from https://portal.oxlo.ai
Step 1: Set up the client and handle long context
Real support tickets are long. Most providers charge by the token, so a thread with twenty messages gets expensive fast. Oxlo.ai uses flat per-request pricing, which means we can pass the full history into the prompt without the cost scaling. We start by instantiating the OpenAI-compatible client and loading a sample thread.
from openai import OpenAI
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
ticket_thread = """
Subject: Double billing
From: customer@example.com
Date: 2024-05-01
Hi, I was charged twice for my enterprise plan this month.
My finance team needs this resolved before quarter close.
On 2024-05-02, Support wrote:
We are looking into this.
On 2024-05-03, customer wrote:
Any update? We have an enterprise SLA.
"""
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": "You summarize support threads."},
{"role": "user", "content": ticket_thread},
],
)
print(response.choices[0].message.content)
Step 2: Define the system prompt
The system prompt is our production contract. It sets the role, the output schema, and a hard rule against inventing policy details. I keep it explicit and short.
SYSTEM_PROMPT = """You are a support triage agent.
Your job:
1. Read the full ticket thread.
2. Decide if you need a knowledge base article.
3. Produce a JSON object with exactly these keys:
- summary: string, max 20 words
- urgency: "low", "medium", or "high"
- needs_kb_lookup: boolean
- draft_reply: string, max 100 words
Rules:
- Do not invent policy details.
- If the ticket mentions refunds, SLAs, or billing terms, set needs_kb_lookup to true.
- Always respond with valid JSON and no markdown fences.
"""
Step 3: Enforce JSON mode
Free text breaks parsers in production. We force valid JSON so our downstream webhook can consume the output without regex hacks. Oxlo.ai supports OpenAI-style JSON mode, so we add response_format and remind the model in the prompt.
from openai import OpenAI
import json
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
SYSTEM_PROMPT = """You are a support triage agent.
Your job:
1. Read the full ticket thread.
2. Decide if you need a knowledge base article.
3. Produce a JSON object with exactly these keys:
- summary: string, max 20 words
- urgency: "low", "medium", or "high"
- needs_kb_lookup: boolean
- draft_reply: string, max 100 words
Rules:
- Do not invent policy details.
- If the ticket mentions refunds, SLAs, or billing terms, set needs_kb_lookup to true.
- Always respond with valid JSON and no markdown fences.
"""
ticket_thread = """
Subject: Double billing
From: customer@example.com
Date: 2024-05-01
Hi, I was charged twice for my enterprise plan this month.
My finance team needs this resolved before quarter close.
On 2024-05-02, Support wrote:
We are looking into this.
On 2024-05-03, customer wrote:
Any update? We have an enterprise SLA.
"""
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": ticket_thread},
],
response_format={"type": "json_object"},
)
output = json.loads(response.choices[0].message.content)
print(json.dumps(output, indent=2))
Step 4: Add a knowledge base tool
LLMs hallucinate policy details. We give the model a lookup_kb tool so it can request real articles before it claims anything about refunds or SLAs. Oxlo.ai supports OpenAI-compatible function calling, so the loop is identical to what you already know.
from openai import OpenAI
import json
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
SYSTEM_PROMPT = """You are a support triage agent.
Your job:
1. Read the full ticket thread.
2. Decide if you need a knowledge base article.
3. Produce a JSON object with exactly these keys:
- summary: string, max 20 words
- urgency: "low", "medium", or "high"
- needs_kb_lookup: boolean
- draft_reply: string, max 100 words
Rules:
- Do not invent policy details.
- If the ticket mentions refunds, SLAs, or billing terms, set needs_kb_lookup to true.
- Always respond with valid JSON and no markdown fences.
"""
def lookup_kb(query: str) -> str:
articles = {
"refund": "Refunds are processed within 5 to 7 business days after approval.",
"sla": "Enterprise SLA guarantees 99.9% uptime and 4-hour response time.",
}
return articles.get(query.lower(), "No article found.")
tools = [
{
"type": "function",
"function": {
"name": "lookup_kb",
"description": "Search the internal knowledge base by keyword.",
"parameters": {
"type": "object",
"properties": {
"query": {
"type": "string",
"description": "Keyword such as refund, sla, or billing.",
}
},
"required": ["query"],
},
},
}
]
messages = [
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": "Customer is asking for a refund on ticket #4922."},
]
resp = client.chat.completions.create(
model="llama-3.3-70b",
messages=messages,
tools=tools,
tool_choice="auto",
)
msg = resp.choices[0].message
messages.append(msg)
if msg.tool_calls:
for tc in msg.tool_calls:
if tc.function.name == "lookup_kb":
args = json.loads(tc.function.arguments)
result = lookup_kb(args["query"])
messages.append({
"role": "tool",
"tool_call_id": tc.id,
"name": tc.function.name,
"content": result,
})
resp2 = client.chat.completions.create(
model="llama-3.3-70b",
messages=messages,
tools=tools,
)
print(resp2.choices[0].message.content)
else:
print(msg.content)
Step 5: Stream the response
No one likes staring at a blank terminal while the model thinks. We enable streaming on the final completion so the draft appears word by word. On Oxlo.ai, popular models have no cold starts, so the first chunk arrives quickly.
from openai import OpenAI
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
SYSTEM_PROMPT = """You are a support triage agent.
Your job:
1. Read the full ticket thread.
2. Decide if you need a knowledge base article.
3. Produce a JSON object with exactly these keys:
- summary: string, max 20 words
- urgency: "low", "medium", or "high"
- needs_kb_lookup: boolean
- draft_reply: string, max 100 words
Rules:
- Do not invent policy details.
- If the ticket mentions refunds, SLAs, or billing terms, set needs_kb_lookup to true.
- Always respond with valid JSON and no markdown fences.
"""
messages = [
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": "Customer is asking for a refund on ticket #4922."},
]
stream = client.chat.completions.create(
model="llama-3.3-70b",
messages=messages,
stream=True,
)
for chunk in stream:
content = chunk.choices[0].delta.content
if content:
print(content, end="")
print()
Run it
Here is the complete agent. I run it against a realistic ticket that combines a long thread, a billing complaint, and a request for structured output. Because Oxlo.ai pricing is per request, the long thread does not inflate the cost.
from openai import OpenAI
import json
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
SYSTEM_PROMPT = """You are a support triage agent.
Your job:
1. Read the full ticket thread.
2. Decide if you need a knowledge base article.
3. Produce a JSON object with exactly these keys:
- summary: string, max 20 words
- urgency: "low", "medium", or "high"
- needs_kb_lookup: boolean
- draft_reply: string, max 100 words
Rules:
- Do not invent policy details.
- If the ticket mentions refunds, SLAs, or billing terms, set needs_kb_lookup to true.
- Always respond with valid JSON and no markdown fences.
"""
def lookup_kb(query: str) -> str:
articles = {
"refund": "Refunds are processed within 5 to 7 business days after approval.",
"sla": "Enterprise SLA guarantees 99.9% uptime and 4-hour response time.",
}
return articles.get(query.lower(), "No article found.")
tools = [
{
"type": "function",
"function": {
"name": "lookup_kb",
"description": "Search the internal knowledge base by keyword.",
"parameters": {
"type": "object",
"properties": {
"query": {
"type": "string",
"description": "Keyword such as refund, sla, or billing.",
}
},
"required": ["query"],
},
},
}
]
ticket_thread = """
Subject: Double billing and refund request
From: customer@example.com
Date: 2024-05-01
Hi, I was charged twice for my enterprise plan this month.
I need a refund for the duplicate charge.
This is urgent because my finance team is closing the quarter.
On 2024-05-02, Support wrote:
We are looking into this.
On 2024-05-03, customer wrote:
Any update? We have an enterprise SLA.
"""
messages = [
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": ticket_thread},
]
# First call: decide if we need KB
resp = client.chat.completions.create(
model="llama-3.3-70b",
messages=messages,
tools=tools,
tool_choice="auto",
response_format={"type": "json_object"},
)
msg = resp.choices[0].message
messages.append(msg)
if msg.tool_calls:
for tc in msg.tool_calls:
if tc.function.name == "lookup_kb":
args = json.loads(tc.function.arguments)
result = lookup_kb(args["query"])
messages.append({
"role": "tool",
"tool_call_id": tc.id,
"name": tc.function.name,
"content": result,
})
# Final call: answer with KB context
final = client.chat.completions.create(
model="llama-3.3-70b",
messages=messages,
tools=tools,
response_format={"type": "json_object"},
)
output = json.loads(final.choices[0].message.content)
print(json.dumps(output, indent=2))
else:
output = json.loads(msg.content)
print(json.dumps(output, indent=2))
Example output:
{
"summary": "Customer charged twice for enterprise plan, requests urgent refund.",
"urgency": "high",
"needs_kb_lookup": true,
"draft_reply": "Thanks for flagging this. We have confirmed the duplicate charge and initiated a refund. Per our policy, refunds are processed within 5 to 7 business days after approval. Your enterprise SLA response target is 4 hours, and we are well within it."
}
Wrap-up and next steps
The agent now handles long threads, structured output, tool use, and policy grounding. Two concrete next steps: wire the JSON output into your CRM webhook so high-urgency tickets open a PagerDuty incident automatically, and add a confidence threshold field so low-confidence drafts route to a human instead of the customer.
If your current provider bills by the token, moving this workload to Oxlo.ai removes the penalty for long context. You can explore the flat per-request pricing at https://oxlo.ai/pricing.
Top comments (0)