Chatbot Development with LLM and Intent Recognition

#aiinfrastructure #oxlo #ai

Building a production chatbot requires more than prompting a large language model. Without explicit intent recognition, a chatbot misroutes user requests, hallucinates policies, and wastes compute on irrelevant generation. A robust pipeline separates understanding from response generation. You classify the user’s goal first, then invoke the right model, prompt template, or tool to fulfill it. This two-stage pattern improves accuracy, simplifies auditing, and makes latency and cost predictable.

Intent Recognition Basics

Intent recognition is the process of mapping a user utterance to a structured goal. A message like "I was charged twice this month" should resolve to a billing_dispute intent before any response is drafted. Without this layer, a chatbot relies solely on the LLM's implicit reasoning, which can drift across sessions, misinterpret domain terminology, or ignore business rules. Explicit intent extraction gives you a deterministic hook for routing, guardrails, and analytics.

Two-Stage Architecture

A production chatbot benefits from separating comprehension from generation. The pipeline looks like this:

Receive user message.
Classify intent and extract slots (dates, IDs, categories).
Route to a handler: a prompt template, an API call, or a retrieval step.
Generate the final response through an LLM.

This design limits the LLM to language tasks it does well, while your application code enforces policy.

Classifying Intent with Oxlo.ai

You can implement intent classification with an LLM call. Because Oxlo.ai uses request-based pricing, a dedicated classification request costs the same flat rate regardless of how many few-shot examples or long definitions you include in the prompt. This makes it practical to treat classification as a first-class API call rather than a client-side heuristic.

The following example uses Oxlo.ai’s OpenAI-compatible endpoint with JSON mode to return structured intent data. You will need an API key from https://oxlo.ai/pricing.

import os
import json
from openai import OpenAI

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key=os.environ["OXLO_API_KEY"]
)

INTENTS = [
    "billing_question",
    "technical_support",
    "account_management",
    "general_chat"
]

def classify_intent(user_message: str) -> dict:
    response = client.chat.completions.create(
        model="qwen-3-32b",
        messages=[
            {
                "role": "system",
                "content": (
                    "Classify the user message into one of these intents: "
                    f"{', '.join(INTENTS)}. "
                    "Respond with a JSON object containing 'intent' and 'confidence'."
                )
            },
            {"role": "user", "content": user_message}
        ],
        response_format={"type": "json_object"}
    )
    return json.loads(response.choices[0].message.content)

Using JSON mode ensures you get parseable output without brittle regex extraction. If you need multilingual support, Qwen 3 32B on Oxlo.ai handles non-English queries well.

Generating Contextual Responses

Once the intent is known, you can select a specialized system prompt or trigger a tool. This keeps the generation model focused and reduces the risk of hallucinated policies.

SYSTEM_PROMPTS = {
    "billing_question": "You are a billing assistant. Cite policy ID 402. Be concise.",
    "technical_support": "You are a tier-1 support engineer. Ask one clarifying question.",
    "account_management": "You are an account coordinator. Never delete data.",
    "general_chat": "You are a helpful assistant."
}

def generate_response(intent: str, user_message: str) -> str:
    system = SYSTEM_PROMPTS.get(intent, SYSTEM_PROMPTS["general_chat"])
    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=[
            {"role": "system", "content": system},
            {"role": "user", "content": user_message}
        ]
    )
    return response.choices[0].message.content

For deep reasoning tasks, such as troubleshooting complex code in a technical support flow, you can route to DeepSeek R1 671B MoE or Kimi K2.6 directly through Oxlo.ai without changing your client code.

Full Integration Example

Wire the classifier and generator together with a confidence threshold and a fallback message.

def handle_message(user_message: str) -> str:
    result = classify_intent(user_message)
    intent = result.get("intent", "general_chat")
    confidence = result.get("confidence", 0.0)

    if intent not in INTENTS or confidence < 0.7:
        return "I'm not sure I understood. Could you rephrase that?"

    return generate_response(intent, user_message)

# Example usage
if __name__ == "__main__":
    print(handle_message("My invoice looks wrong"))

This pattern is easy to extend. You can add slots, conditional handoffs to human agents, or logging without touching the LLM client logic.

Cost and Latency Considerations

Chatbot workloads often involve long system prompts, multi-turn history, and large context

DEV Community