Building a Chatbot with LLM and NLP

#product #oxlo #ai

Building a production chatbot requires more than wrapping a language model in a REST endpoint. A robust system combines classical NLP techniques, structured dialogue management, and a capable LLM backend to handle open-ended generation. The goal is to balance determinism, where you need exact intent classification or entity extraction, with the flexibility of a large model for natural responses. Oxlo.ai provides the inference foundation for this stack, offering flat per-request pricing and an OpenAI-compatible API that drops into existing pipelines without refactoring.

Architecture Overview

A typical LLM-powered chatbot splits work across three layers. The input layer normalizes text and runs lightweight NLP tasks, such as intent recognition and slot filling. The reasoning layer, powered by an LLM, generates responses, handles ambiguity, and manages tool use. The memory layer persists conversation state, user profiles, and retrieval context. Decoupling these concerns lets you iterate on prompts, swap models, and add guardrails without rewriting the entire application.

Selecting a Base Model

Oxlo.ai hosts over 45 open-source and proprietary models across seven categories. For general conversational backends, Llama 3.3 70B is a reliable flagship. If you are building multilingual agents or need deep reasoning, Qwen 3 32B and DeepSeek R1 671B MoE are strong candidates. For coding assistants or agentic tool use, consider Kimi K2.6 or DeepSeek V3.2. Because Oxlo.ai is fully OpenAI SDK compatible, you can test these models by changing a single model identifier without touching your transport code.

Inference with Oxlo.ai

The platform exposes a standard chat completions endpoint at https://api.oxlo.ai/v1. There are no cold starts on popular models, so your chatbot responds immediately even after idle periods. The following Python snippet shows a minimal streaming request using the OpenAI SDK.

import openai

client = openai.OpenAI(
    api_key="YOUR_OXLO_API_KEY",
    base_url="https://api.oxlo.ai/v1"
)

response = client.chat.completions.create(
    model="llama-3.3-70b",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What is the status of my order?"}
    ],
    stream=True
)

for chunk in response:
    print(chunk.choices[0].delta.content or "", end="")

Structuring the NLP Pipeline

Even with a powerful LLM, explicit NLP preprocessing improves reliability. You can use the LLM itself for structured intent classification via JSON mode, or run a smaller specialized model for entity extraction. For retrieval-augmented generation, Oxlo.ai offers embedding endpoints with BGE-Large and E5-Large. You can generate embeddings, query a vector store, and inject retrieved context into the chat completion, all through the same API base URL.

The example below uses JSON mode to extract a structured intent before the main generation step.

response = client.chat.completions.create(
    model="qwen3-32b",
    messages=[
        {"role": "user", "content": "Book a table for two at 7 PM"}
    ],
    response_format={"type": "json_object"}
)

intent_json = response.choices[0].message.content
# Expected output: {"intent": "book_table", "party_size": 2, "time": "19:00"}

Memory and Multi-Turn Context

Chatbots need to maintain context across turns. On token-based platforms, long conversation histories inflate costs linearly because every token in the context window is billed on every request. Oxlo.ai uses flat per-request pricing, so a long multi-turn history does not increase the cost of the next request. This makes it practical to keep extensive dialogue state, few-shot examples, and RAG context in the prompt without worrying about the token meter spinning.

messages = [
    {"role": "system", "content": "You are a support agent. Be concise."}
]

def chat_turn(user_input: str) -> str:
    messages.append({"role": "user", "content": user_input})
    resp = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=messages,
        stream=False
    )
    content = resp.choices[0].message.content
    messages.append({"role": "assistant", "content": content})
    return content

For very long chats you still need truncation or summarization to fit context limits, but cost is no longer the bottleneck that forces aggressive early truncation.

Tool Use and Structured Output

For actions like checking order status or updating a CRM, define tools and let the model decide when to call them. Oxlo.ai supports function calling on compatible models, which lets you build agentic chatbots that interact with external APIs.

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_order_status",
            "description": "Retrieve the status of a customer order",
            "parameters": {
                "type": "object",
                "properties": {
                    "order_id": {"type": "string"}
                },
                "required": ["order_id"]
            }
        }
    }
]

response = client.chat.completions.create(
    model="kimi-k2-6",
    messages=messages,
    tools=tools
)

if response.choices[0].message.tool_calls:
    # Execute the tool and append results to messages
    pass

Cost and Scaling Considerations

Token-based providers bill for input and output tokens, which means long system prompts, RAG context, and extended chat histories directly increase your per-interaction cost. Oxlo.ai charges one flat rate per API request regardless of prompt length. For chatbots that pass large retrieved documents or maintain lengthy agentic context, this can reduce inference costs by an order of magnitude compared to token-based alternatives such as Together AI, Fireworks AI, OpenRouter, Replicate, or Anyscale.

You can explore the exact tiers on the Oxlo.ai pricing page. The Free plan offers 60 requests per day across more than 16 models, which is enough to prototype a multi-step pipeline. Paid plans scale to thousands of requests per day with priority queues, and Enterprise workloads can access dedicated GPUs.

Conclusion

A modern chatbot is a hybrid system. You get reliability from structured NLP and flexibility from a capable LLM. Oxlo.ai

DEV Community