Unlocking Conversational AI with LLMs: Best Practices and Strategies

We are going to build a customer support agent that handles order status lookups for an e-commerce store. This type of conversational AI reduces support volume by resolving the most common customer question instantly. I will use Oxlo.ai for inference because its flat per-request pricing keeps costs predictable even as conversation history grows, and its OpenAI-compatible API means we can use the standard SDK without changes.

What you'll need

• Python 3.10 or newer
• An Oxlo.ai API key from https://portal.oxlo.ai
• The OpenAI SDK installed via pip install openai

Step 1: Set up the client and database

Set up the client and a mock order database. I initialize the OpenAI SDK pointing at Oxlo.ai and create a hardcoded dictionary of orders so we have something to query.

import os
import json
from openai import OpenAI

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key=os.environ.get("OXLO_API_KEY")
)

orders_db = {
    "ORD-1001": {"status": "shipped", "eta": "2024-12-20", "item": "Wireless Headphones"},
    "ORD-1002": {"status": "processing", "eta": "2024-12-22", "item": "Mechanical Keyboard"},
    "ORD-1003": {"status": "delivered", "eta": "delivered 2024-12-15", "item": "USB-C Hub"},
}

Step 2: Craft the system prompt

Define the system prompt that gives the agent its role, constraints, and instructions for using the available tool. I keep it explicit so the model does not hallucinate policies.

SYSTEM_PROMPT = """You are a helpful customer support agent for an electronics store.
Your job is to assist customers with order status questions.
You have access to a function called check_order_status.
When a user provides an order ID, call the function.
Do not make up order details. If an order is not found, say so clearly.
Be concise and polite."""

Step 3: Define the tool schema

Create the tool definition so the LLM knows how to request an order lookup. Oxlo.ai supports the standard OpenAI tools format.

tools = [
    {
        "type": "function",
        "function": {
            "name": "check_order_status",
            "description": "Look up the current status of a customer order by ID.",
            "parameters": {
                "type": "object",
                "properties": {
                    "order_id": {
                        "type": "string",
                        "description": "The order ID, for example ORD-1001."
                    }
                },
                "required": ["order_id"]
            }
        }
    }
]

Step 4: Build the chat function

Build the core function that sends the conversation history to Oxlo.ai. I use llama-3.3-70b because it handles tool use reliably for support workloads, and Oxlo.ai serves it with no cold starts.

def chat_with_agent(messages):
    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=messages,
        tools=tools,
        tool_choice="auto",
    )
    return response.choices[0].message

Step 5: Handle tool execution

Implement the tool execution loop. When the model returns tool calls, I run the lookups against our mock database and append the results back into the message history.

def handle_tool_calls(message):
    tool_messages = []
    for tool_call in message.tool_calls:
        function_name = tool_call.function.name
        arguments = json.loads(tool_call.function.arguments)
        if function_name == "check_order_status":
            order_id = arguments.get("order_id", "").upper()
            data = orders_db.get(order_id, {"error": "Order not found"})
            tool_messages.append({
                "tool_call_id": tool_call.id,
                "role": "tool",
                "content": json.dumps(data)
            })
    return tool_messages

Step 6: Wire the interactive loop

Combine everything into a loop that maintains conversation state, handles tool calls, and prints the final response to the user.

def run_conversation():
    messages = [{"role": "system", "content": SYSTEM_PROMPT}]
    print("Support agent ready. Type 'exit' to quit.")

    while True:
        user_input = input("\nCustomer: ")
        if user_input.lower() == "exit":
            break

        messages.append({"role": "user", "content": user_input})
        message = chat_with_agent(messages)

        while message.tool_calls:
            messages.append({
                "role": message.role,
                "content": message.content,
                "tool_calls": [
                    {
                        "id": tc.id,
                        "type": tc.type,
                        "function": {
                            "name": tc.function.name,
                            "arguments": tc.function.arguments
                        }
                    } for tc in message.tool_calls
                ]
            })
            tool_results = handle_tool_calls(message)
            messages.extend(tool_results)
            message = chat_with_agent(messages)

        messages.append({"role": "assistant", "content": message.content})
        print(f"Agent: {message.content}")

if __name__ == "__main__":
    run_conversation()

Run it

Save the script as support_agent.py, set your API key, and run it.

export OXLO_API_KEY="sk-..."
python support_agent.py

A sample session looks like this:

Support agent ready. Type 'exit' to quit.

Customer: Where is my order ORD-1001?
Agent: Your order ORD-1001 for Wireless Headphones has been shipped and is expected to arrive on 2024-12-20.

Customer: Can you check ORD-9999?
Agent: I could not find an order with ID ORD-9999. Please double-check the order number and try again.

Next steps

Swap the mock dictionary for a real database connection or internal API so the agent queries live data. You can also add a second tool, such as escalate_to_human, so the agent can hand off complex issues when a customer asks for a refund or makes a complaint.