Building a Chatbot with LLM: A Step-by-Step Tutorial

#product #oxlo #ai

Building a conversational interface around a large language model has become a standard project for developers exploring generative AI. A well-built chatbot needs more than just a prompt. It requires stateful message history, streaming output for responsiveness, and an API backend that stays predictable as conversations grow longer. Oxlo.ai provides a developer-first inference platform that is fully compatible with the OpenAI SDK, supports streaming and multi-turn contexts, and charges a flat rate per API request rather than billing by the token. That makes it a strong fit for chatbots where context windows expand with every back-and-forth.

Prerequisites

To follow this tutorial, you need Python 3.9 or later and an Oxlo.ai API key. If you do not have an account yet, you can sign up and start with the Free plan, which includes 60 requests per day across more than 16 models and a 7-day full-access trial. Install the official OpenAI Python client, which works as a drop-in replacement against Oxlo.ai endpoints.

pip install openai

Project Setup

Create a new file named chatbot.py. Import the OpenAI library and instantiate a client pointing to the Oxlo.ai base URL. Because Oxlo.ai exposes a fully OpenAI-compatible API, the only difference from a standard OpenAI script is the base_url parameter.

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ.get("OXLO_API_KEY"),
    base_url="https://api.oxlo.ai/v1"
)

Store your API key in an environment variable rather than hard-coding it.

Choosing a Model

Oxlo.ai hosts more than 45 models across seven categories. For a general-purpose chatbot, Llama 3.3 70B offers broad reasoning capabilities and reliable instruction following. If you expect multilingual users or agentic workflows, Qwen 3 32B is a solid alternative. For testing, DeepSeek V3.2 is available on the Free tier and handles coding and reasoning tasks well. For this tutorial, we will use Llama 3.3 70B, but you can swap the model identifier to experiment with others.

Basic Chat Loop with History

A conversational agent must remember prior messages. Maintain a messages list and append each user prompt and assistant reply. Start with a system message to define behavior.

MODEL = "llama-3.3-70b"

messages = [
    {"role": "system", "content": "You are a helpful assistant. Answer concisely."}
]

def chat(user_input):
    messages.append({"role": "user", "content": user_input})

    response = client.chat.completions.create(
        model=MODEL,
        messages=messages
    )

    assistant_message = response.choices[0].message.content
    messages.append({"role": "assistant", "content": assistant_message})
    return assistant_message

if __name__ == "__main__":
    print("Chatbot ready. Type 'exit' to quit.")
    while True:
        user_input = input("You: ")
        if user_input.strip().lower() == "exit":
            break
        reply = chat(user_input)
        print(f"Bot: {reply}")

Run the script, type messages, and watch the bot maintain context across turns. Because Oxlo.ai does not impose cold starts on popular models, the first response arrives quickly.

Adding Streaming Responses

Waiting for the entire completion before printing text creates a sluggish experience. Enable streaming so tokens arrive as they are generated. Oxlo.ai supports streaming responses natively.

def chat_stream(user_input):
    messages.append({"role": "user", "content": user_input})

    stream = client.chat.completions.create(
        model=MODEL,
        messages=messages,
        stream=True
    )

    print("Bot: ", end="", flush=True)
    assistant_message = ""
    for chunk in stream:
        token = chunk.choices[0].delta.content
        if token:
            print(token, end="", flush=True)
            assistant_message += token
    print()

    messages.append({"role": "assistant", "content": assistant_message})
    return assistant_message

Replace the chat() call in your main loop with chat_stream() to see tokens appear in real time.

Why Request-Based Pricing Matters for Chatbots

Chatbots accumulate context. Every prior question and answer gets prepended to the next request, so input length grows linearly with conversation depth. On token-based providers, longer inputs mean higher costs per turn. Oxlo.ai uses request-based pricing: one flat cost per API request regardless of prompt length. For long-context and agentic workloads, this can be significantly cheaper than token-based alternatives because your bill scales with the number of turns, not the number of tokens in the history. See the exact tiers on the Oxlo.ai pricing page.

Extending the Bot

Once the core loop is stable, you can add capabilities that Oxlo.ai supports out of the box:

Function calling: Define JSON schemas for tools and let the model decide when to invoke them. Oxlo.ai supports function calling and tool use on compatible models.
Vision: Pass image URLs or base64-encoded images in the messages array to build a multimodal assistant. Models such as Kimi K2.6 and Gemma 3 27B accept image input.
JSON mode: Force the model to return valid JSON by setting response_format={"type": "json_object"}, useful for structured logging or downstream processing.
Embeddings: If you want retrieval-augmented generation, generate document vectors with BGE-Large or E5-Large via the embeddings endpoint.

Conclusion

You now have a streaming, stateful chatbot built with the OpenAI SDK running against Oxlo.ai. Because the platform is fully OpenAI API compatible, you can migrate existing projects by changing two lines: the API key and the base URL. With request-based pricing, multi-turn applications stay cost-predictable even as context grows, and the Free tier gives you room to experiment before committing to a paid plan. Deploy your script as a web service, add tool use, or connect a vector store. The infrastructure underneath is already scaled for it.

DEV Community