shashank ms

Posted on Sep 18

LLM vs Rule-Based Systems: Understanding the Differences

#learnai #oxlo #ai

We are going to build a support ticket router that classifies incoming messages into Billing, Technical, or Account departments. It starts as a rule-based classifier, then falls back to an LLM through Oxlo.ai for messages that do not match any keyword. This gives you the speed of regex for obvious cases and the reasoning of Llama 3.3 70B for everything else, without token costs that scale with ticket length.

What you'll need

Python 3.10 or newer
The OpenAI SDK: pip install openai
An Oxlo.ai API key from https://portal.oxlo.ai

Step 1: Scaffold the ticket router

I start by defining the departments and a simple dataclass for incoming tickets.

from dataclasses import dataclass
from enum import Enum
import re

class Department(Enum):
    BILLING = "billing"
    TECHNICAL = "technical"
    ACCOUNT = "account"

@dataclass
class Ticket:
    subject: str
    body: str

Step 2: Encode the rule-based classifier

Next, I map keywords to departments with compiled regex patterns. If a pattern matches, the ticket is routed immediately.

RULES = [
    (Department.BILLING, re.compile(r"\b(refund|charge|invoice|payment|billing)\b", re.I)),
    (Department.TECHNICAL, re.compile(r"\b(error|bug|crash|timeout|login fails|500)\b", re.I)),
    (Department.ACCOUNT, re.compile(r"\b(upgrade|downgrade|cancel|plan|subscription)\b", re.I)),
]

def route_rule_based(ticket: Ticket) -> Department | None:
    text = f"{ticket.subject} {ticket.body}"
    for dept, pattern in RULES:
        if pattern.search(text):
            return dept
    return None

Step 3: Collect the edge cases that break rules

Rules fail when the user does not use expected keywords or when multiple departments overlap. I keep a few samples that I know will fall through.

EDGE_CASES = [
    Ticket("Hand off workspace", "I am moving teams and need to transfer ownership to my manager."),
    Ticket("Login page shows charge error", "When I try to log in I see a message about an expired card."),
]

The first ticket contains no trigger words, and the second mixes billing vocabulary with a technical symptom.

Step 4: Write the LLM system prompt

For the fallback, I need a prompt that forces a single token of output so parsing stays trivial.

SYSTEM_PROMPT = """You are a support routing agent.
Analyze the ticket subject and body.
Classify into exactly one department: billing, technical, or account.
Respond with only the department name, lowercase, no punctuation."""

Step 5: Add the Oxlo.ai LLM fallback

Now I wire in Oxlo.ai. Because the platform uses flat per-request pricing, passing a long ticket thread to Llama 3.3 70B does not inflate the cost.

from openai import OpenAI

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

def route_llm(ticket: Ticket) -> Department:
    user_message = f"Subject: {ticket.subject}\nBody: {ticket.body}"
    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": user_message},
        ],
    )
    answer = response.choices[0].message.content.strip().lower()
    return Department(answer)

Step 6: Build the hybrid router

The final agent tries the cheap rule path first, then calls the LLM only when no rule matches.

def route_ticket(ticket: Ticket) -> dict:
    rule_result = route_rule_based(ticket)
    if rule_result:
        return {"department": rule_result.value, "method": "rule"}
    llm_result = route_llm(ticket)
    return {"department": llm_result.value, "method": "llm"}

Run it

I run a batch of four tickets through the agent to see where rules suffice and where the LLM takes over.

if __name__ == "__main__":
    tickets = [
        Ticket("Refund request", "I was double charged last month."),
        Ticket("Cannot compile project", "The build fails with error code 500 after the latest update."),
        Ticket("Hand off workspace", "I am moving teams and need to transfer ownership to my manager."),
        Ticket("Login page shows charge error", "When I try to log in I see a message about an expired card."),
    ]

    for t in tickets:
        result = route_ticket(t)
        print(f"{t.subject:25} -> {result}")

Example output:

Refund request            -> {'department': 'billing', 'method': 'rule'}
Cannot compile project    -> {'department': 'technical', 'method': 'rule'}
Hand off workspace        -> {'department': 'account', 'method': 'llm'}
Login page shows charge error -> {'department': 'technical', 'method': 'llm'}

Wrap-up

You now have a working router that compares both paradigms in a single pipeline. To push it further, try adding confidence scoring from the LLM so you can escalate truly ambiguous tickets to a human, or swap in qwen-3-32b for multilingual inquiries. Oxlo.ai makes either experiment simple because every request costs the same flat rate, so you can iterate without watching token meters. See https://oxlo.ai/pricing for the latest plan details.

DEV Community