I recently shipped a context-aware support agent for a SaaS product that needed to handle messy, multi-turn customer threads without sounding robotic. In this tutorial, I will walk you through the exact version I built, using Oxlo.ai's OpenAI-compatible API and request-based pricing so long conversations do not inflate your bill. You can follow along with any of Oxlo.ai's chat models, though I used Llama 3.3 70B for the initial version.
What you'll need
- Python 3.10 or newer.
- The OpenAI SDK:
pip install openai. - An Oxlo.ai API key from https://portal.oxlo.ai.
Step 1: Test the connection with a single turn
Before building state management, I verify that the client reaches Oxlo.ai and that the model responds in a reasonable voice. I use llama-3.3-70b because it is a reliable general-purpose model for support dialogue.
from openai import OpenAI
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "user", "content": "My dashboard is blank after logging in. What should I do?"},
],
)
print(response.choices[0].message.content)
Step 2: Define the system prompt
The system prompt is the only place where you teach tone, boundaries, and escalation rules. I keep it in a dedicated variable so I can iterate without touching business logic.
SYSTEM_PROMPT = """You are a Tier-1 support agent for a SaaS analytics platform. Your name is Alex.
Tone and style:
- Be concise. Use no more than three sentences per reply unless the user asks for detail.
- Be friendly but professional. Do not use marketing language.
- Always confirm the user's issue before proposing a fix.
Boundaries:
- You can help with account access, dashboard bugs, and billing questions.
- If the user reports a security breach, data loss, or expresses anger with profanity, you must escalate.
- If you do not know the answer, ask one clarifying question. Do not guess.
Escalation:
- If the user needs human support, say exactly: "I am connecting you with a specialist now."
"""
Step 3: Manage multi-turn history
Dialogue falls apart if you drop context. I use a simple list that appends each turn and trims when it grows too large, which is affordable on Oxlo.ai because request-based pricing means deep history does not raise the per-turn cost.
class SupportAgent:
def __init__(self, client, model="llama-3.3-70b", max_turns=10):
self.client = client
self.model = model
self.max_turns = max_turns
self.history = [{"role": "system", "content": SYSTEM_PROMPT}]
def _trim_history(self):
# Always preserve the first system message, keep last N user/assistant pairs
if len(self.history) > (self.max_turns * 2) + 1:
self.history = [self.history[0]] + self.history[-(self.max_turns * 2):]
def chat(self, user_message: str) -> str:
self.history.append({"role": "user", "content": user_message})
self._trim_history()
response = self.client.chat.completions.create(
model=self.model,
messages=self.history,
)
reply = response.choices[0].message.content
self.history.append({"role": "assistant", "content": reply})
return reply
Step 4: Add escalation guardrails
I add a lightweight keyword guardrail that injects a reminder into the context when the user shows frustration. This avoids building a separate classifier and keeps the stack simple.
FRUSTRATION_SIGNALS = ["angry", "frustrated", "useless", "terrible", "refund", "lawsuit", "cancel everything"]
class SupportAgent:
def __init__(self, client, model="llama-3.3-70b", max_turns=10):
self.client = client
self.model = model
self.max_turns = max_turns
self.history = [{"role": "system", "content": SYSTEM_PROMPT}]
def _trim_history(self):
if len(self.history) > (self.max_turns * 2) + 1:
self.history = [self.history[0]] + self.history[-(self.max_turns * 2):]
def _is_frustrated(self, text: str) -> bool:
return any(signal in text.lower() for signal in FRUSTRATION_SIGNALS)
def chat(self, user_message: str) -> str:
self.history.append({"role": "user", "content": user_message})
if self._is_frustrated(user_message):
self.history.append({
"role": "system",
"content": "REMINDER: The user is frustrated. Follow escalation rules."
})
self._trim_history()
response = self.client.chat.completions.create(
model=self.model,
messages=self.history,
)
reply = response.choices[0].message.content
self.history.append({"role": "assistant", "content": reply})
return reply
Step 5: Return a structured hand-off summary
When the agent escalates, I want a JSON object instead of free text so my backend can open a ticket automatically. Oxlo.ai supports JSON mode, so I force the model to emit a structured payload without extra parsing logic.
import json
class SupportAgent:
def __init__(self, client, model="llama-3.3-70b", max_turns=10):
self.client = client
self.model = model
self.max_turns = max_turns
self.history = [{"role": "system", "content": SYSTEM_PROMPT}]
def _trim_history(self):
if len(self.history) > (self.max_turns * 2) + 1:
self.history = [self.history[0]] + self.history[-(self.max_turns * 2):]
def _is_frustrated(self, text: str) -> bool:
return any(signal in text.lower() for signal in FRUSTRATION_SIGNALS)
def chat(self, user_message: str) -> str:
self.history.append({"role": "user", "content": user_message})
if self._is_frustrated(user_message):
self.history.append({
"role": "system",
"content": "REMINDER: The user is frustrated. Follow escalation rules."
})
self._trim_history()
response = self.client.chat.completions.create(
model=self.model,
messages=self.history,
)
reply = response.choices[0].message.content
self.history.append({"role": "assistant", "content": reply})
return reply
def handoff_summary(self) -> dict:
summary_prompt = (
"Based on the conversation so far, produce a JSON object with exactly these keys: "
"escalate (boolean), reason (string), and summary (string). "
"The summary should be one sentence describing the user's issue."
)
temp_history = self.history + [{"role": "user", "content": summary_prompt}]
response = self.client.chat.completions.create(
model=self.model,
messages=temp_history,
response_format={"type": "json_object"},
)
return json.loads(response.choices[0].message.content)
Run it
The script below simulates a short conversation, prints each assistant reply, and outputs a structured hand-off when frustration is detected.
if __name__ == "__main__":
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
agent = SupportAgent(client)
turns = [
"My dashboard is blank after logging in.",
"I tried that already. It is still useless.",
"I want to cancel everything and talk to a human.",
]
for turn in turns:
print(f"User: {turn}")
reply = agent.chat(turn)
print(f"Agent: {reply}\n")
if "connecting you with a specialist" in reply.lower():
summary = agent.handoff_summary()
print("Hand-off payload:", summary)
break
Example output:
User: My dashboard is blank after logging in.
Agent: I am sorry to hear that. To help you, can you confirm whether you see any error message, or is the page completely blank?
User: I tried that already. It is still useless.
Agent: I understand your frustration. Let me connect you with a specialist now.
User: I want to cancel everything and talk to a human.
Agent: I am connecting you with a specialist now.
Hand-off payload: {'escalate': True, 'reason': 'Customer expressed frustration and requested human agent.', 'summary': 'User sees a blank dashboard after login and previous troubleshooting failed.'}
Next steps
Swap in qwen-3-32b or kimi-k2.6 if you need stronger agentic reasoning for tool use, such as looking up account status via function calling before replying. If you want to experiment at no cost, deepseek-v3.2 offers a free tier on Oxlo.ai and handles coding and reasoning tasks well enough for a prototype.
Top comments (0)