LLMs vs Traditional AI Models: Understanding the Difference

We are going to build a support ticket triage agent that classifies unstructured customer messages and drafts first-pass responses. To see why LLMs have displaced traditional NLP pipelines, we will first ship a brittle rules-based classifier, then swap it for an LLM via Oxlo.ai and compare the results on the same inputs.

What you'll need

• Python 3.10 or newer.
• An Oxlo.ai API key from https://portal.oxlo.ai. The free tier includes 60 requests per day, which is enough for this tutorial.
• The OpenAI SDK. Install it with pip install openai.

Step 1: Build the traditional classifier

Traditional AI models rely on feature engineering, regular expressions, and fixed vocabularies. We will implement a tiny keyword-based expert system so we have a concrete baseline to compare against the LLM.

import re

KEYWORD_MAP = {
    "billing": [r"refund", r"charge", r"subscription", r"cancel.*sub"],
    "account": [r"password", r"login", r"reset", r"sign.in"],
    "bug":     [r"crash", r"bug", r"error", r"broken"],
    "shipping":[r"ship", r"deliver", r"order", r"tracking"],
}

def traditional_classify(text: str):
    lowered = text.lower()
    scores = {}
    for category, patterns in KEYWORD_MAP.items():
        scores[category] = sum(1 for p in patterns if re.search(p, lowered))
    total = sum(scores.values())
    if total == 0:
        return "unknown", 0.0
    best = max(scores, key=scores.get)
    return best, scores[best] / total

Step 2: Define the LLM agent system prompt

LLMs do not need TF-IDF matrices or regex lists. We describe the task in natural language and let the model reason over raw text. Here is the system prompt our Oxlo.ai agent will use.

SYSTEM_PROMPT = """You are a support ticket triage agent.
Analyze the customer's message and produce a JSON object with exactly two keys:
  category: one of billing, account, bug, shipping, or other
  response: a polite, concise first-pass response to the customer
If the message is ambiguous, ask one clarifying question in the response field."""

Step 3: Wire up the Oxlo.ai LLM classifier

We replace the traditional model with a single API call to Oxlo.ai. Because Oxlo.ai uses flat per-request pricing, the cost does not grow when customers paste long logs or screenshots, which makes it predictable for agentic workloads.

from openai import OpenAI
import json

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

def llm_classify(text: str):
    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": text},
        ],
    )
    content = response.choices[0].message.content
    return json.loads(content)

Step 4: Compare both systems

Feed the same ambiguous tickets to both pipelines. The rules engine counts keywords and falls apart on multi-intent messages, while the LLM extracts intent and drafts a useful reply.

test_messages = [
    "I was charged twice last Tuesday and now I cannot log in either",
    "Your latest update made the whole thing unusable on my phone",
    "Where is my package? I need it before Friday.",
]

for msg in test_messages:
    trad_label, trad_conf = traditional_classify(msg)
    llm_result = llm_classify(msg)

    print(f"Ticket: {msg}")
    print(f"  Traditional -> {trad_label} (confidence {trad_conf:.2f})")
    print(f"  LLM         -> {llm_result['category']} | {llm_result['response']}")
    print()

Run it

Save the full script as triage.py, swap in your real Oxlo.ai API key, and run it from your terminal.

$ python triage.py

Ticket: I was charged twice last Tuesday and now I cannot log in either
  Traditional -> billing (confidence 0.50)
  LLM         -> billing | I see you have a duplicate charge and a login issue. I have escalated this to our billing team and will send a password reset link right away.

Ticket: Your latest update made the whole thing unusable on my phone
  Traditional -> bug (confidence 0.25)
  LLM         -> bug | I am sorry the latest update is not working on your device. Can you tell me which phone model and OS version you are running so we can investigate?

Ticket: Where is my package? I need it before Friday.
  Traditional -> shipping (confidence 0.25)
  LLM         -> shipping | I have pulled up your tracking details. Because you need it before Friday, I have flagged this with our logistics partner for expedited handling.

Wrap-up

You now have a concrete side-by-side view of why LLMs outperform traditional rules-based systems on unstructured language tasks. The LLM handles multi-intent tickets, vague wording, and context without any regex maintenance.

Two concrete next steps: wrap llm_classify in a FastAPI endpoint so new tickets hit the Oxlo.ai agent automatically, or add function calling to let the agent query your CRM before it responds. If you are moving from a token-based provider, Oxlo.ai request-based pricing can flatten your costs for long-context tickets. See https://oxlo.ai/pricing for plan details.