Streamlining LLM Development with Oxlo.ai: A Tutorial

#learnai #oxlo #ai

I will build a support ticket triage agent that classifies urgency, tags product areas, and drafts replies in a single structured JSON response. This kind of tool is useful for any team that wants to cut first-response time without adding headcount. We will run it on Oxlo.ai because its request-based pricing keeps costs predictable even when tickets include long logs or chat history (details at https://oxlo.ai/pricing).

What you'll need

Python 3.10 or newer
An Oxlo.ai API key from https://portal.oxlo.ai
The OpenAI SDK: pip install openai

Step 1: Configure the Oxlo.ai client

The OpenAI SDK works as a drop-in replacement for Oxlo.ai. I set the base URL and API key, then create a shared client that the rest of the script will reuse.

from openai import OpenAI
import json

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key="YOUR_OXLO_API_KEY"
)

Step 2: Write the system prompt

The system prompt is the only training the agent gets. I keep it tight: define the role, the three classification tasks, and the exact JSON keys to return.

SYSTEM_PROMPT = """You are a support triage agent. Read the customer message and return a single JSON object with these exact keys:

- urgency: one of "critical", "high", "normal", "low"
- product_area: one of "billing", "api", "authentication", "ui", "integrations", "unknown"
- summary: a one-sentence summary of the issue
- reply_draft: a polite, concise first reply offering next steps

Rules:
- If the user mentions a service outage or security breach, set urgency to "critical".
- If the message is just a question, set urgency to "normal" or "low".
- Keep reply_draft under three sentences.
- Return only valid JSON, with no markdown code fences.
"""

Step 3: Enable JSON mode

To avoid parsing fragile text, I ask the model to reply with JSON and set response_format to json_object. Oxlo.ai supports this on Llama 3.3 70B and other chat models.

def triage_ticket(customer_message: str) -> dict:
    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": customer_message},
        ],
        response_format={"type": "json_object"},
        temperature=0.2,
    )
    raw = response.choices[0].message.content
    return json.loads(raw)

Step 4: Add error handling

Production code should not crash on a malformed response. I wrap the parser in a try/except block and log the raw output for debugging.

def triage_ticket_safe(customer_message: str) -> dict:
    raw = None
    try:
        response = client.chat.completions.create(
            model="llama-3.3-70b",
            messages=[
                {"role": "system", "content": SYSTEM_PROMPT},
                {"role": "user", "content": customer_message},
            ],
            response_format={"type": "json_object"},
            temperature=0.2,
        )
        raw = response.choices[0].message.content
        return json.loads(raw)
    except Exception as e:
        print(f"Failed to triage message: {e}")
        return {
            "urgency": "unknown",
            "product_area": "unknown",
            "summary": "Parse error",
            "reply_draft": "A human will review this shortly.",
            "raw": raw,
        }

Step 5: Process a batch of tickets

In practice, tickets arrive in a queue. I simulate a batch of three messages and print the structured results.

tickets = [
    "I was charged twice this month. My account ID is 9912. Please fix this immediately.",
    "How do I rotate my API key? I cannot find the settings page.",
    "The entire east region is down. None of our webhooks are reaching your endpoints and we are losing orders.",
]

for ticket in tickets:
    result = triage_ticket_safe(ticket)
    print(json.dumps(result, indent=2))
    print("-" * 40)

Run it

Save the script as triage.py, export your key, and run it.

export OXLO_API_KEY="your-key-here"
python triage.py

When I ran this against Oxlo.ai, the output looked like this:

{
  "urgency": "high",
  "product_area": "billing",
  "summary": "Customer reports a duplicate charge for account ID 9912.",
  "reply_draft": "I have located the duplicate charge on account 9912 and initiated a refund. You should see the credit within 3-5 business days."
}
----------------------------------------
{
  "urgency": "low",
  "product_area": "api",
  "summary": "Customer needs help locating the API key rotation settings.",
  "reply_draft": "You can rotate your API key under Settings > API > Regenerate Key. Let us know if you need help invalidating the old key."
}
----------------------------------------
{
  "urgency": "critical",
  "product_area": "integrations",
  "summary": "East region outage is blocking all webhooks and causing order loss.",
  "reply_draft": "We are treating this as a critical outage. Our on-call engineer has been paged and will update you within 15 minutes."
}

Next steps

To productionize this, wire the triage_ticket_safe function into your existing help-desk webhooks or email ingestion pipeline. If you need deeper reasoning for ambiguous tickets, swap the model string to kimi-k2.6 or deepseek-v3.2 on Oxlo.ai without changing any other code.

DEV Community