DEV Community

shashank ms
shashank ms

Posted on

Building a Chatbot with LLM and Dialogue Management

#product #oxlo #ai

Chatbots built on large language models have moved beyond simple question-and-answer demos. To deliver reliable, task-oriented conversations, you need dialogue management: a layer that tracks state, handles context, and decides when to call external tools. In this guide, we will build a minimal but production-ready chatbot architecture that combines an LLM backend with explicit state tracking, and we will run it on Oxlo.ai to keep costs predictable and infrastructure minimal.

Architecture Overview

A robust conversational system separates language understanding from control logic. The architecture has four parts:

  • Natural Language Interface: The LLM interprets user intent from raw text.
  • Dialogue State Tracker: A structured store for conversation history, user slots, and system flags.
  • Policy / Action Executor: Decides whether to respond directly, ask a clarifying question, or invoke a tool.
  • Response Generator: The LLM renders the final output, conditioned on state and any tool results.

The LLM is stateless, so the dialogue manager must feed it the full relevant context on every turn. This design keeps the system testable and makes it easy to swap models or providers.

Why Oxlo.ai for Conversational AI

Most inference providers bill by the token. In a chatbot, every turn appends more messages to the context window, so token-based costs increase as the conversation progresses. Oxlo.ai uses flat per-request pricing, which means the cost of a turn stays the same whether you send a short prompt or a long conversation history. For support bots and agentic workflows that routinely carry multi-turn context, this makes spend linear and easy to forecast.

Oxlo.ai is also fully compatible with the OpenAI SDK. You can point your existing Python or Node.js client to https://api.oxlo.ai/v1 without rewriting request logic. There are no cold starts on popular chat models, so user interactions feel instant. The platform offers a range of models suited for dialogue, including Llama 3.3 70B for general-purpose chat, Qwen 3 32B for multilingual and agentic tasks, and DeepSeek V4 Flash for long-context conversations up to 1M tokens.

You can explore the exact tiers on the Oxlo.ai pricing page.

Project Setup

Install the official OpenAI SDK. It works out of the box with Oxlo.ai because the APIs are drop-in compatible.

pip install openai

Configure the client to point to Oxlo.ai. Use an environment variable for your API key.

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key=os.getenv("OXLO_API_KEY")
)

MODEL = "llama-3.3-70b"  # or qwen3-32b, deepseek-v4-flash, etc.

Managing Dialogue State

Because the LLM does not remember previous turns, we maintain a lightweight state object in memory. It stores the message history and any extracted slots, such as a delivery tracking number or user preference.

class DialogueState:

    def init(self, system_prompt: str):

        self.system_prompt = system_prompt

        self.history = []  # list of {"role": "user"|"assistant"|"tool", ...}

        self.slots = {}


def add_user_message(self, content: str):
    self.history.append({"role": "user", "content": content})

def add_ass

Top comments (0)