LLM vs Rule-Based Systems: Comparison and Use Cases

#learnai #oxlo #ai

We are building a hybrid support triage agent that routes customer messages using hard rules for known issues and falls back to an LLM for everything else. This pattern keeps latency and cost low for predictable traffic while preserving flexibility for edge cases. Teams running high-volume support queues can use it to see exactly where rules end and LLMs begin.

What you'll need

Python 3.10 or higher
An Oxlo.ai API key from https://portal.oxlo.ai
The OpenAI SDK: pip install openai

You can prototype this on the Oxlo.ai free tier, which includes 60 requests per day and a 7-day full-access trial.

Step 1: Define the rule-based classifier

Start with a dictionary of keywords and canned responses. If a message contains "refund" or "password", we immediately tag it and return a static reply. No API call, no tokens, zero latency.

import re

RULES = [
    {
        "pattern": re.compile(r"\b(refund|money back|chargeback)\b", re.IGNORECASE),
        "category": "billing",
        "response": "I have forwarded your refund request to the billing team. You will hear back within 24 hours."
    },
    {
        "pattern": re.compile(r"\b(password|reset|forgot login)\b", re.IGNORECASE),
        "category": "account",
        "response": "You can reset your password at https://example.com/reset. Let me know if the link does not work."
    },
    {
        "pattern": re.compile(r"\b(shipping|tracking|delivery|package)\b", re.IGNORECASE),
        "category": "logistics",
        "response": "Please provide your order ID so I can pull the latest tracking details."
    }
]

def rule_classify(text):
    for rule in RULES:
        if rule["pattern"].search(text):
            return {
                "source": "rule",
                "category": rule["category"],
                "response": rule["response"],
                "confidence": 1.0
            }
    return None

Step 2: Configure the Oxlo.ai client and system prompt

When no rule fires, we route to an LLM. Oxlo.ai uses request-based pricing, so the cost is flat even if the user pastes a thousand-word rant. That makes it practical to send long, messy tickets to a capable model without counting tokens. See https://oxlo.ai/pricing for current plan details.

from openai import OpenAI

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

SYSTEM_PROMPT = """You are a support triage assistant. Analyze the customer message and output strictly valid JSON with these keys:
- category: one of [billing, account, logistics, technical, other]
- urgency: one of [low, medium, high]
- response: a concise, helpful reply written in the same language as the customer.
- reasoning: one sentence explaining why this category was chosen.

Rules:
1. If the user asks for a refund, category is billing.
2. If the user reports a bug or crash, category is technical.
3. Keep the response under three sentences.
"""

Step 3: Build the LLM fallback

This function calls Oxlo.ai using Llama 3.3 70B. Because Oxlo.ai is fully OpenAI SDK compatible, the code is a drop-in replacement. Just change the base URL and API key.

import json

def llm_classify(user_message):
    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": user_message},
        ],
        temperature=0.1,
    )
    raw = response.choices[0].message.content.strip()
    if raw.startswith("

```"):
        raw = raw.split("\n", 1)[1].rsplit("```

", 1)[0].strip()
    return json.loads(raw)

Step 4: Wire both layers into the hybrid agent

The agent tries rules first. If a rule matches, it returns instantly. If not, it calls the Oxlo.ai LLM. This is the architecture you actually ship: deterministic guardrails with a neural safety net.

class SupportAgent:
    def handle(self, user_message):
        result = rule_classify(user_message)
        if result:
            return result
        
        try:
            output = llm_classify(user_message)
            output["source"] = "llm"
            return output
        except Exception as exc:
            return {
                "source": "error",
                "category": "other",
                "urgency": "high",
                "response": "A human agent will review this shortly.",
                "reasoning": str(exc)
            }

Run it

Test the agent with three inputs: a rule-based hit, an ambiguous technical complaint, and a logistics question.

if __name__ == "__main__":
    agent = SupportAgent()
    
    tests = [
        "I want a refund for my order.",
        "The app crashes when I click export after updating to version 4.2.",
        "Where is my package?",
    ]
    
    for msg in tests:
        print("Input:", msg)
        print("Output:", agent.handle(msg))
        print()

Example output:

Input: I want a refund for my order.
Output: {'source': 'rule', 'category': 'billing', 'response': 'I have forwarded your refund request to the billing team. You will hear back within 24 hours.', 'confidence': 1.0}

Input: The app crashes when I click export after updating to version 4.2.
Output: {'source': 'llm', 'category': 'technical', 'urgency': 'high', 'response': 'Thanks for reporting this crash after the v4.2 update. I am escalating this to engineering immediately.', 'reasoning': 'The user describes a software crash triggered by a specific action after an update.'}

Input: Where is my package?
Output: {'source': 'rule', 'category': 'logistics', 'response': 'Please provide your order ID so I can pull the latest tracking details.', 'confidence': 1.0}

Next steps

Add confidence thresholds so borderline rule matches still trigger the LLM, or cache frequent LLM responses in Redis to avoid repeated API calls for similar tickets. If volume grows, Oxlo.ai request-based pricing stays predictable while token-based bills would scale with ticket length, so the hybrid design remains cost-effective.

DEV Community