Overview
We are building a customer support agent that maintains natural conversational flow across multi-turn chats. It compresses stale context, detects when a user changes topics, and avoids the stiff, repetitive phrasing that makes LLM interactions feel robotic. If you run support automation or any agentic dialogue system, this is the scaffolding you actually need.
What you'll need
- Python 3.10 or newer
- An Oxlo.ai API key from https://portal.oxlo.ai
- The OpenAI SDK:
pip install openai
I will use kimi-k2.6 for the main conversational model because it handles long-context reasoning and agentic coding well. For lightweight classification tasks, I will call deepseek-v3.2, which is available on Oxlo.ai's free tier. If you need lower latency, swap in llama-3.3-70b or qwen-3-32b for the main agent. All calls use the same OpenAI-compatible endpoint with no cold starts.
Step 1: Scaffold the Oxlo.ai client and a single-turn call
First, I verify the connection to Oxlo.ai and make sure I can get a non-robotic greeting back. I hardcode a short system prompt and send one user message to confirm the end-to-end pipeline works.
from openai import OpenAI
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
SYSTEM_PROMPT = "You are a support agent for a SaaS analytics platform. Be concise, friendly, and never repeat the user's question back to them verbatim."
response = client.chat.completions.create(
model="kimi-k2.6",
messages=[
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": "My dashboard is not loading after the latest update."},
],
)
print(response.choices[0].message.content)
Step 2: Craft the system prompt that drives conversational flow
The system prompt is the single biggest lever for conversational quality. I write explicit constraints against robotic repetition and instruct the model to confirm understanding before offering solutions.
SYSTEM_PROMPT = """You are Atlas Support, a helpful agent for a B2B analytics platform.
Rules for conversational flow:
1. Never repeat the user's question back to them in your answer.
2. If the user changes topics, acknowledge the pivot briefly and switch context.
3. Ask one clarifying question at a time. Do not barrage the user with three questions in one turn.
4. Keep responses under three sentences unless the user asks for detail.
5. Do not use filler phrases like "As an AI language model" or "I hope this helps".
6. When you need data, ask for it. Do not hallucinate account details.
Current date: 2025-01-15."""
Step 3: Build a rolling memory buffer with compression
Conversations die when the context window overflows or when old turns dilute the prompt. I implement a FIFO buffer that keeps the last six exchanges. When the buffer fills, I call the model to summarize the oldest turns into a single sentence so the agent still knows what happened without carrying the full transcript.
from openai import OpenAI
from collections import deque
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
SYSTEM_PROMPT = """You are Atlas Support, a helpful agent for a B2B analytics platform.
Rules for conversational flow:
1. Never repeat the user's question back to them in your answer.
2. If the user changes topics, acknowledge the pivot briefly and switch context.
3. Ask one clarifying question at a time. Do not barrage the user with three questions in one turn.
4. Keep responses under three sentences unless the user asks for detail.
5. Do not use filler phrases like "As an AI language model" or "I hope this helps".
6. When you need data, ask for it. Do not hallucinate account details.
Current date: 2025-01-15."""
class ConversationBuffer:
def __init__(self, max_turns=6):
self.max_turns = max_turns
self.turns = deque(maxlen=max_turns)
self.summary = ""
def add_turn(self, user_msg, assistant_msg):
self.turns.append({"user": user_msg, "assistant": assistant_msg})
def build_messages(self):
messages = [{"role": "system", "content": SYSTEM_PROMPT}]
if self.summary:
messages.append({"role": "system", "content": f"Summary of earlier conversation: {self.summary}"})
for turn in self.turns:
messages.append({"role": "user", "content": turn["user"]})
messages.append({"role": "assistant", "content": turn["assistant"]})
return messages
def compress(self):
if len(self.turns) < 3:
return
old_turns = list(self.turns)[:-3]
transcript = "\n".join([f"User: {t['user']}\nAgent: {t['assistant']}" for t in old_turns])
prompt = f"Summarize this support conversation in one sentence:\n{transcript}"
resp = client.chat.completions.create(
model="deepseek-v3.2",
messages=[{"role": "user", "content": prompt}],
temperature=0.0,
max_tokens=60,
)
self.summary = resp.choices[0].message.content.strip()
self.turns = deque(list(self.turns)[-3:], maxlen=self.max_turns)
# Demo
buffer = ConversationBuffer()
buffer.add_turn("My dashboard is not loading after the latest update.", "I am sorry to hear that. Can you try a hard refresh and clearing your cache?")
buffer.add_turn("I tried that, but the spinner just sits there.", "Thanks for trying that. Are you seeing any error in the browser console?")
buffer.add_turn("It says CORS policy blocked the request.", "That is helpful. Are you behind a corporate VPN or proxy?")
buffer.compress()
messages = buffer.build_messages()
response = client.chat.completions.create(model="kimi-k2.6", messages=messages)
print(response.choices[0].message.content)
Step 4: Detect topic shifts so the agent pivots naturally
Nothing kills flow like an agent that keeps troubleshooting a forgotten issue. I add a lightweight classifier that compares the latest user message against the conversation summary. If the topic changed, I flush the stale context before the main model generates a response.
from openai import OpenAI
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
def detect_shift(summary, new_message):
prompt = f"""Conversation summary: {summary}
User's next message: {new_message}
Has the user shifted to a completely new topic? Reply with exactly one word: YES or NO."""
resp = client.chat.completions.create(
model="deepseek-v3.2",
messages=[{"role": "user", "content": prompt}],
temperature=0.0,
max_tokens=5,
)
return resp.choices[0].message.content.strip().upper().startswith("YES")
summary = "User had dashboard loading issues caused by a CORS error behind a corporate proxy."
new_msg = "Actually, can you help me reset my password?"
if detect_shift(summary, new_msg):
print("TOPIC SHIFT: reset buffer and summary")
else:
print("CONTINUE: keep context")
Step 5: Add response guardrails to prevent robotic output
Even with a strong prompt, models occasionally prepend phrases like "Certainly!" or echo the user. I add a thin post-processing layer that strips common filler prefixes and collapses extra whitespace before the response ever reaches the user.
import re
FORBIDDEN_PREFIXES = [
r"(?i)^certainly[.,!]*\s*",
r"(?i)^sure[.,!]*\s*",
r"(?i)^as an ai[ ,]*",
r"(?i)^i hope this helps[.,!]*\s*",
]
def clean_response(text):
for pattern in FORBIDDEN_PREFIXES:
text = re.sub(pattern, "", text)
text = re.sub(r"\n{2,}", "\n", text).strip()
return text
raw = "Certainly! I can help you with that. Please try restarting the service."
print(clean_response(raw))
Step 6: Assemble the complete conversational agent
Now I wire the buffer, classifier, and cleaner into a single class. The chat method orchestrates the flow: detect shifts, build context, call kimi-k2.6 on Oxlo.ai, then clean and store the result.
from openai import OpenAI
from collections import deque
import re
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
SYSTEM_PROMPT = """You are Atlas Support, a helpful agent for a B2B analytics platform.
Rules for conversational flow:
1. Never repeat the user's question back to them in your answer.
2. If the user changes topics, acknowledge the pivot briefly and switch context.
3. Ask one clarifying question at a time. Do not barrage the user with three questions in one turn.
4. Keep responses under three sentences unless the user asks for detail.
5. Do not use filler phrases like "As an AI language model" or "I hope this helps".
6. When you need data, ask for it. Do not hallucinate account details.
Current date: 2025-01-15."""
FORBIDDEN_PREFIXES = [
r"(?i)^certainly[.,!]*\s*",
r"(?i)^sure[.,!]*\s*",
r"(?i)^as an ai[ ,]*",
r"(?i)^i hope this helps[.,!]*\s*",
]
class SupportAgent:
def __init__(self, max_turns=6):
self.max_turns = max_turns
self.turns = deque(maxlen=max_turns)
self.summary = ""
def _detect_shift(self, new_message):
if not self.summary:
return False
prompt = f"Conversation summary: {self.summary}\nUser's next message: {new_message}\nHas the user shifted to a completely new topic? Reply YES or NO."
resp = client.chat.completions.create(
model="deepseek-v3.2",
messages=[{"role": "user", "content": prompt}],
temperature=0.0,
max_tokens=5,
)
return resp.choices[0].message.content.strip().upper().startswith("YES")
def _compress(self):
if len(self.turns) < 3:
return
old_turns = list(self.turns)[:-3]
transcript = "\n".join([f"User: {t['user']}\nAgent: {t['assistant']}" for t in old_turns])
prompt = f"Summarize this support conversation in one sentence:\n{transcript}"
resp = client.chat.completions.create(
model="deepseek-v3.2",
messages=[{"role": "user", "content": prompt}],
temperature=0.0,
max_tokens=60,
)
self.summary = resp.choices[0].message.content.strip()
self.turns = deque(list(self.turns)[-3:], maxlen=self.max_turns)
def _clean(self, text):
for pattern in FORBIDDEN_PREFIXES:
text = re.sub(pattern, "", text)
return re.sub(r"\n{2,}", "\n", text).strip()
def chat(self, user_message):
if self._detect_shift(user_message):
self.turns.clear()
self.summary = ""
if len(self.turns) == self.max_turns:
self._compress()
messages = [{"role": "system", "content": SYSTEM_PROMPT}]
if self.summary:
messages.append({"role": "system", "content": f"Summary of earlier conversation: {self.summary}"})
for turn in self.turns:
messages.append({"role": "user", "content": turn["user"]})
messages.append({"role": "assistant", "content": turn["assistant"]})
messages.append({"role": "user", "content": user_message})
resp = client.chat.completions.create(
model="kimi-k2.6",
messages=messages,
temperature=0.7,
max_tokens=300,
)
raw = resp.choices[0].message.content
cleaned = self._clean(raw)
self.turns.append({"user": user_message, "assistant": cleaned})
return cleaned
agent = SupportAgent()
print(agent.chat("My dashboard is not loading after the latest update."))
Run it
Here is a short script that simulates a realistic session. The user starts with a loading issue, then abruptly pivots to billing. Watch the agent acknowledge the shift instead of chasing the old CORS problem.
agent = SupportAgent()
session = [
"My dashboard is not loading after the latest update.",
"I tried a hard refresh, but the spinner just sits there.",
"Actually, can you help me reset my password?",
"I need the invoice for last month.",
]
for msg in session:
print(f"User: {msg}")
reply = agent.chat(msg)
print(f"Agent: {reply}\n")
Example output:
User: My dashboard is not loading after the latest update.
Agent: I am sorry to hear that. Are you seeing any specific error message in the browser console?
User: I tried a hard refresh, but the spinner just sits there.
Agent: Thanks for trying that. Are you on a corporate network or VPN?
User: Actually, can you help me reset my password?
Agent: No problem. I will pivot to account recovery. Do you still have access to the email associated with your account?
User: I need the invoice for last month.
Agent: Understood. I can help you locate that invoice. What is the email address on the account so I can pull up your billing history?
Next steps
Swap the support domain for your own by changing the SYSTEM_PROMPT and adding a tool-calling loop so the agent can query your internal APIs before responding. If you are running high-volume automation, Oxlo.ai's per-request pricing keeps long-context sessions predictable because the cost does not scale with input length. See the details at https://oxlo.ai/pricing.
Another concrete upgrade is streaming. Oxlo.ai supports streaming responses on all chat models, so you can yield tokens to the user interface as soon as they arrive instead of waiting for the full generation.
Top comments (0)