Building Conversational AI Models

#learnai #oxlo #ai

We are going to build a conversational support agent that answers questions about a SaaS API, checks user sentiment, and hands off to a human when frustration spikes. This pattern works for any company that needs to automate tier-1 support without writing fragile if/else logic. We will run the whole thing on Oxlo.ai using the OpenAI SDK so you only need to change the base URL to get started.

What you'll need

Python 3.10 or higher
An Oxlo.ai API key from https://portal.oxlo.ai
The OpenAI SDK installed with pip install openai

Step 1: Set up the client

Create a file named support_agent.py and instantiate the client pointing at Oxlo.ai. Send a single test message to confirm your key and network are working.

from openai import OpenAI

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

response = client.chat.completions.create(
    model="llama-3.3-70b",
    messages=[
        {"role": "user", "content": "Say 'Connection OK' and nothing else."},
    ],
)

print(response.choices[0].message.content)

Step 2: Define the system prompt

The system prompt is the only place we encode behavior. Keeping it in a dedicated variable makes iteration easy. It includes product facts, tone rules, and escalation criteria.

SYSTEM_PROMPT = """You are Tier-1 Support for AcmeApi, a payment processing API.

Knowledge base:
- Starter plan: $29/mo, 1,000 requests/day, email support only.
- Pro plan: $99/mo, 10,000 requests/day, priority email + chat support.
- Enterprise: custom pricing, unlimited requests, dedicated account manager.

Rules:
1. Answer in 2 sentences or less unless the user asks for detail.
2. If the user mentions "refund", "cancel account", "frustrated", or uses all-caps yelling, set escalate=True in your response.
3. Never make up pricing that is not listed above.
4. Always ask for the user's account email before sharing sensitive details.

Respond in this exact format:
thought: <your internal reasoning>
reply: <what you say to the user>
escalate: <True or False>
"""

Step 3: Build the request wrapper

We write a helper that prepends the system prompt, appends any prior conversation history, and calls the Oxlo.ai chat endpoint. For now it returns the raw text so we can parse it separately.

def ask_agent(user_message, history=None):
    if history is None:
        history = []

    messages = [{"role": "system", "content": SYSTEM_PROMPT}]
    messages.extend(history)
    messages.append({"role": "user", "content": user_message})

    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=messages,
        temperature=0.2,
    )

    return response.choices[0].message.content

Step 4: Parse structured output

The model replies with a structured block. We use a small regex parser to extract the fields into a dictionary our program can act on.

import re

def parse_response(text):
    result = {"thought": "", "reply": "", "escalate": False}
    thought_match = re.search(r"thought:\s*(.*?)(?=\nreply:|\nescalate:|$)", text, re.S)
    reply_match = re.search(r"reply:\s*(.*?)(?=\nescalate:|$)", text, re.S)
    escalate_match = re.search(r"escalate:\s*(True|False)", text, re.I)

    if thought_match:
        result["thought"] = thought_match.group(1).strip()
    if reply_match:
        result["reply"] = reply_match.group(1).strip()
    if escalate_match:
        result["escalate"] = escalate_match.group(1).lower() == "true"

    return result

Step 5: Add memory and the CLI loop

A conversational agent needs context. We keep a history list that grows with each turn, cap it at the last 20 messages, and break the loop when an escalation is detected.

def run_conversation():
    history = []
    print("AcmeApi Support Agent (type 'exit' to quit)\n")

    while True:
        user_input = input("User: ").strip()
        if user_input.lower() in {"exit", "quit"}:
            break

        raw = ask_agent(user_input, history)
        parsed = parse_response(raw)

        history.append({"role": "user", "content": user_input})
        history.append({"role": "assistant", "content": raw})

        if len(history) > 20:
            history = history[-20:]

        if parsed["escalate"]:
            print(f"Agent: {parsed['reply']}")
            print("[SYSTEM] Escalation triggered. Routing to human agent.\n")
            break
        else:
            print(f"Agent: {parsed['reply']}\n")

if __name__ == "__main__":
    run_conversation()

Run it

Save all the pieces into support_agent.py, export your Oxlo.ai key, and run the script. Here is a sample session:

$ export OXLO_API_KEY="oxlo_..."
$ python support_agent.py

AcmeApi Support Agent (type 'exit' to quit)

User: What is included in the Pro plan?
Agent: The Pro plan is $99 per month and includes 10,000 requests per day plus priority email and chat support.

User: I was charged twice this month and I am frustrated
Agent: I am sorry to hear that. Could you please provide the email on your account so I can look into the duplicate charge?
[SYSTEM] Escalation triggered. Routing to human agent.

Next steps

Swap in qwen-3-32b if you need multilingual support, or move the knowledge base out of the prompt and into a retrieval pipeline using Oxlo.ai's embeddings endpoint. If you deploy this to production, wrap the loop in a FastAPI endpoint so your frontend can stream responses.

DEV Community