We are building a support ticket triage agent that reads unstructured customer messages, classifies intent, and drafts a first response. A traditional ML pipeline would need a labeled dataset, feature engineering, and retraining for every new category. We will skip all of that and use an LLM on Oxlo.ai for semantic understanding and structured output.
What you'll need
- Python 3.10+
pip install openai- An Oxlo.ai API key from https://portal.oxlo.ai
Step 1: Verify connectivity
I always start with a smoke test. Oxlo.ai exposes an OpenAI-compatible endpoint, so the SDK works without changes. I use Llama 3.3 70B because it is the general-purpose flagship and has no cold starts on Oxlo.ai.
from openai import OpenAI
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Say hello and confirm you are ready."},
],
)
print(response.choices[0].message.content)Step 2: Define the system prompt
Traditional classifiers need fixed vocabularies and one-hot encodings. An LLM only needs a natural language description of the job. This prompt asks for strict JSON so we can parse the result downstream.
SYSTEM_PROMPT = """You are a support triage agent. Analyze the user message and respond with a JSON object containing:
- category: one of [technical, billing, refund, spam]
- urgency: one of [low, medium, high, critical]
- reasoning: a one-sentence explanation
- draft_reply: a brief, professional first response
Rules:
- Respond with valid JSON only.
- Do not wrap the output in markdown code fences."""Step 3: Build the classifier
Now we wire the prompt into a reusable function. I use Qwen 3 32B because its multilingual reasoning and agent workflow strengths handle messy, informal user language better than a rigid TF-IDF plus SVM pipeline ever would.
import json
from openai import OpenAI
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
SYSTEM_PROMPT = """You are a support triage agent. Analyze the user message and respond with a JSON object containing:
- category: one of [technical, billing, refund, spam]
- urgency: one of [low, medium, high, critical]
- reasoning: a one-sentence explanation
- draft_reply: a brief, professional first response
Rules:
- Respond with valid JSON only.
- Do not wrap the output in markdown code fences."""
def classify_ticket(text):
response = client.chat.completions.create(
model="qwen-3-32b",
messages=[
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": text},
],
)
raw = response.choices[0].message.content
return json.loads(raw)
ticket = "hey i was charged twice last month and now my dashboard is blank. fix this asap."
print(json.dumps(classify_ticket(ticket), indent=2))Step 4: Inject internal context
A traditional model would need a separate retrieval step, tokenization alignment, and feature merging. With an LLM we can paste a runbook snippet directly into the prompt. I switch to Kimi K2.6 for its 131K context window and advanced reasoning, which lets me include troubleshooting notes without extra infrastructure.
import json
from openai import OpenAI
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
SYSTEM_PROMPT = """You are a support triage agent. Analyze the user message and respond with a JSON object containing:
- category: one of [technical, billing, refund, spam]
- urgency: one of [low, medium, high, critical]
- reasoning: a one-sentence explanation
- draft_reply: a brief, professional first response
Rules:
- Respond with valid JSON only.
- Do not wrap the output in markdown code fences."""
RUNBOOK = """
Known issues:
- Dashboard blank: usually resolved by clearing CDN cache (infra ticket #112).
- Double charges: refund only if a duplicate transaction_id exists in Stripe.
"""
def classify_with_context(text):
enriched = f"Runbook context:\n{RUNBOOK}\n\nUser ticket:\n{text}"
response = client.chat.completions.create(
model="kimi-k2.6",
messages=[
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": enriched},
],
)
raw = response.choices[0].message.content
return json.loads(raw)
ticket = "hey i was charged twice last month and now my dashboard is blank. fix this asap."
print(json.dumps(classify_with_context(ticket), indent=2))Step 5: Handle complex escalations with reasoning
Some tickets contain multiple overlapping issues that confuse simple classifiers. DeepSeek V3.2 is strong at coding and reasoning, and it is available on the Oxlo.ai free tier, so I use it as a secondary filter when the user mentions both billing and technical problems in the same thread.
import json
from openai import OpenAI
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
SYSTEM_PROMPT = """You are a support triage agent. Analyze the user message and respond with a JSON object containing:
- category: one of [technical, billing, refund, spam]
- urgency: one of [low, medium, high, critical]
- reasoning: a one-sentence explanation
- draft_reply: a brief, professional first response
Rules:
- Respond with valid JSON only.
- Do not wrap the output in markdown code fences."""
def classify_complex(text):
response = client.chat.completions.create(
model="deepseek-v3.2",
messages=[
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": text},
],
)
raw = response.choices[0].message.content
return json.loads(raw)
complex_ticket = "I tried to downgrade but got an error, then I saw two charges. I need this fixed before our demo tomorrow."
print(json.dumps(classify_complex(complex_ticket), indent=2))Run it
Here is the full script that routes a batch of tickets through the appropriate model based on simple heuristics. In production you could replace the heuristics with confidence scoring from the LLM itself.
import json
from openai import OpenAI
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
SYSTEM_PROMPT = """You are a support triage agent. Analyze the user message and respond with a JSON object containing:
- category: one of [technical, billing, refund, spam]
- urgency: one of [low, medium, high, critical]
- reasoning: a one-sentence explanation
- draft_reply: a brief, professional first response
Rules:
- Respond with valid JSON only.
- Do not wrap the output in markdown code fences."""
RUNBOOK = """
Known issues:
- Dashboard blank: usually resolved by clearing CDN cache (infra ticket #112).
- Double charges: refund only if a duplicate transaction_id exists in Stripe.
"""
def route_ticket(text):
if "error" in text.lower() and "charge" in text.lower():
model = "deepseek-v3.2"
elif "dashboard" in text.lower():
model = "kimi-k2.6"
else:
model = "qwen-3-32b"
enriched = f"Runbook context:\n{RUNBOOK}\n\nUser ticket:\n{text}"
response = client.chat.completions.create(
model=model,
messages=[
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": enriched},
],
)
raw = response.choices[0].message.content
return json.loads(raw)
tickets = [
"hey i was charged twice last month and now my dashboard is blank. fix this asap.",
"I tried to downgrade but got an error, then I saw two charges. I need this fixed before our demo tomorrow.",
"Do you offer discounts for nonprofits?",
]
for t in tickets:
print(f"Ticket: {t}")
print(json.dumps(route_ticket(t), indent=2))
print("---")Example output:
Ticket: hey i was charged twice last month and now my dashboard is blank. fix this asap.
{
"category": "billing",
"urgency": "high",
"reasoning": "User reports a double charge and a dashboard issue, both requiring immediate attention.",
"draft_reply": "I am sorry for the trouble. I have escalated this to our billing team and will check your dashboard status. We will update you within one hour."
}
---
Ticket: I tried to downgrade but got an error, then I saw two charges. I need this fixed before our demo tomorrow.
{
"category": "technical",
"urgency": "critical",
"reasoning": "Downgrade error combined with duplicate charges blocks the user's demo.",
"draft_reply": "I understand the urgency. I am investigating the downgrade error and the duplicate charges now and will prioritize a resolution before your demo."
}
---
Ticket: Do you offer discounts for nonprofits?
{
"category": "billing",
"urgency": "low",
"reasoning": "General pricing inquiry with no immediate impact on service.",
"draft_reply": "Yes, we offer nonprofit discounts. Please reply with your 501(c)(3) documentation and I will apply the discount to your account."
}Wrap-up
We replaced a brittle keyword pipeline with an LLM agent that reasons over context, parses structured JSON, and drafts replies. Because Oxlo.ai uses request-based pricing, the cost stays flat even when we stuff long runbooks into the prompt for Kimi K2.6's 131K context window. That makes this architecture significantly cheaper for long-context triage than token-based providers.
Next steps: wire the JSON output into your CRM via webhook, or add a tool-calling step so the agent can query your billing database directly before it replies. Both patterns work out of the box with Oxlo.ai's function calling support.
Top comments (0)