Building a production chatbot requires more than prompting a large language model. Without explicit dialogue management, state tracking, and tool integration, conversations drift, user intent gets lost, and API costs balloon across long sessions. A robust design separates conversational flow control from generation, giving you deterministic state machines wrapped around stochastic LLM outputs.
Why Dialogue Management Still Matters
LLMs excel at open-ended generation, but they are stateless. Each request is independent, so the model has no inherent memory of whether it already asked for a shipping address or confirmed a refund. Dialogue management provides the scaffolding: tracking slots, handling digressions, and enforcing business logic. This layer keeps the LLM focused on language understanding and generation while your code owns the conversation state.
A Practical Chatbot Architecture
A typical system has four layers. The interface layer handles user input and platform adapters. The NLU layer parses intent and extracts entities. The dialogue manager maintains state and decides the next action. The LLM layer generates natural language responses. In modern implementations, the LLM often subsumes traditional NLU through few-shot intent classification and structured JSON outputs, but the dialogue manager remains essential for guarding rails.
Implementing State Tracking
State can be as simple as a Python dictionary or as complex as a graph database. For most applications, an in-memory state object keyed by session ID is sufficient. The dialogue manager passes this state into the LLM prompt as a structured context block, then updates slots based on the model's JSON-mode output.
import json
from openai import OpenAI
client = OpenAI(
base_url="https://api.oxlo.ai/v1",
api_key="YOUR_OXLO_API_KEY"
)
def update_state(current_state, user_message, model_response):
try:
slots = json.loads(model_response.choices[0].message.content)
current_state.update(slots)
except json.JSONDecodeError:
current_state["last_error"] = "Invalid JSON from model"
return current_state
Tool Use for Actions
When a user says "Book me a flight to Tokyo on Friday," the LLM should not hallucinate a confirmation. Instead, it should emit a function call that your dialogue manager validates and executes. Oxlo.ai supports function calling and tool use across its chat models, so you can define schemas for booking APIs, CRM lookups, or database queries.
tools = [
{
"type": "function",
"function": {
"name": "search_flights",
"description": "Search available flights",
"parameters": {
"type": "object",
"properties": {
"destination": {"type": "string"},
"date": {"type": "string"}
},
"required": ["destination", "date"]
}
}
}
]
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=messages,
tools=tools
)
if response.choices[0].message.tool_calls:
for tool_call in response.choices[0].message.tool_calls:
if tool_call.function.name == "search_flights":
args = json.loads(tool_call.function.arguments)
result = search_flights(**args)
messages.append({
"role": "tool",
"tool_call_id": tool_call.id,
"content": json.dumps(result)
})
Managing Context Windows and Cost
Multi-turn conversations accumulate tokens fast. In token-based billing, a long dialogue history can make each user message exponentially more expensive. Oxlo.ai uses request-based pricing: one flat cost per API call regardless of prompt length. For dialogue-heavy applications with extensive system prompts, few-shot examples, and long conversation histories, this structure keeps costs predictable as session depth grows. You can afford to pass full state context every turn without watching token meters spin.
Choosing the Right Model
Not every turn needs a frontier reasoning model. Route simple intent classification to fast, lightweight models and reserve heavy reasoning for complex user requests. Oxlo.ai offers a range of options:
- Llama 3.3 70B for general-purpose chat and reliable instruction following.
- Qwen 3 32B for multilingual agents and tool-use workflows.
- DeepSeek R1 671B MoE or Kimi K2.6 when you need deep reasoning or advanced coding assistance within the conversation.
- DeepSeek V4 Flash for near state-of-the-art reasoning with a 1M context window, ideal for long document chats.
Because Oxlo.ai exposes all models through a single OpenAI-compatible endpoint, swapping models between turns requires only changing the model string.
Putting It Together
Here is a minimal but complete pattern: a dialogue manager that maintains state, selects tools, and generates responses via Oxlo.ai.
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.oxlo.ai/v1",
api_key=os.getenv("OXLO_API_KEY")
)
class SupportBot:
def init(self):
self.history = []
self.state = {"issue_type": None, "severity": None}
def system_prompt(self):
return (
"You are a support assistant. Update the conversation state using JSON. "
"If you need to look up an order, call the get_order_status function."
)
def process(self, user_text):
self.history.append({"role": "user", "content": user_text})
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[{"role": "system", "content": self.system_prompt()}] + self.history,
tools=[{
"type": "function",
"function": {
"name": "get_order_status",
"parameters": {
"type": "object",
"properties": {"order_id": {"type": "string"}},
"required": ["order_id"]
}
}
}]
)
msg = response.choices[0].message
self.history.append({"role": "assistant", "content": msg.content or ""})
if msg.tool_calls:
for tc in msg.tool_calls:
if tc.function.name == "get_order_status":
result = {"status": "shipped", "eta": "2 days"}
self.history.append({
"role": "tool",
"tool_call_id": tc.id,
"content": str(result)
})
final = client.chat.completions.create(
model="llama-3.3-70b",
messages=[{"role": "system", "content": self.system_prompt()}] + self.history
)
self.history.append({"role
Top comments (0)