LLM vs Rule-Based Systems: A Comparison

#learnai #oxlo #ai

We are building a refund triage agent that routes customer requests through hard rules first, then falls back to an LLM when the case is ambiguous. This hybrid approach keeps deterministic cases fast and cheap while letting the model handle edge cases that do not fit a decision table. I will walk through the exact Python module I shipped to production.

What you'll need

Python 3.10 or newer
The OpenAI SDK: pip install openai
An Oxlo.ai API key from https://portal.oxlo.ai

Step 1: Scaffold the rule engine

Rule-based systems are deterministic and cheap, but brittle once a case falls outside the decision table. I encode the non-negotiable constraints in plain Python so they run in microseconds without any external dependency.

from dataclasses import dataclass
from typing import Optional, Literal

@dataclass
class RefundRequest:
    days_since_purchase: int
    is_defective: bool
    customer_tone: str  # angry, neutral, polite
    explanation: str

def rule_based_decision(req: RefundRequest) -> Optional[Literal["approve", "deny", "escalate"]]:
    # Beyond 90 days: automatic deny
    if req.days_since_purchase > 90:
        return "deny"
    # Defective within 30 days: automatic approve
    if req.is_defective and req.days_since_purchase <= 30:
        return "approve"
    # Angry customer between 31 and 90 days: escalate to human
    if req.customer_tone == "angry" and req.days_since_purchase > 30:
        return "escalate"
    # No rule matched
    return None

Step 2: Set up the Oxlo.ai client

LLMs handle nuance well, but token-based billing makes long customer threads expensive. Oxlo.ai uses flat per-request pricing, so a verbose explanation does not inflate cost. You can view current plans at https://oxlo.ai/pricing.

from openai import OpenAI

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

Step 3: Write the system prompt

The system prompt acts as a programmable policy layer. It repeats the hard constraints so the model does not contradict them, and it asks for structured JSON so parsing is trivial.

SYSTEM_PROMPT = """You are a refund triage assistant. Your job is to decide whether to approve, deny, or escalate a refund request.

Policy:
- Approve if the item is defective and within 30 days.
- Deny if the request is beyond 90 days.
- Escalate if the customer is angry and the request is between 31 and 90 days.
- For all other cases, use your judgment. Consider the explanation, fairness, and company reputation.

Respond with a JSON object containing exactly two keys:
- decision: one of "approve", "deny", "escalate"
- reasoning: a short sentence explaining why
"""

Step 4: Wire rules and LLM together

The orchestrator tries rules first. Only if they return None does it call Llama 3.3 70B on Oxlo.ai. I set response_format to json_object so the output is machine-readable.

import json

def triage_refund(req: RefundRequest) -> dict:
    # Try deterministic rules first
    decision = rule_based_decision(req)
    if decision:
        return {
            "decision": decision,
            "reasoning": "Handled by rule engine.",
            "source": "rule"
        }

    # Build the user message from the request fields
    user_message = f"""Days since purchase: {req.days_since_purchase}
Defective: {req.is_defective}
Customer tone: {req.customer_tone}
Explanation: {req.explanation}"""

    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": user_message},
        ],
        response_format={"type": "json_object"}
    )

    result = json.loads(response.choices[0].message.content)
    result["source"] = "llm"
    return result

Run it

I test three tickets: a clear deny, a clear approve, and a fuzzy case that requires judgment.

if __name__ == "__main__":
    # Case 1: Hard deny
    req1 = RefundRequest(
        days_since_purchase=95,
        is_defective=False,
        customer_tone="neutral",
        explanation="I changed my mind."
    )
    print("Case 1:", triage_refund(req1))

    # Case 2: Hard approve
    req2 = RefundRequest(
        days_since_purchase=10,
        is_defective=True,
        customer_tone="neutral",
        explanation="Battery swells after one charge cycle."
    )
    print("Case 2:", triage_refund(req2))

    # Case 3: Ambiguous, hits the LLM
    req3 = RefundRequest(
        days_since_purchase=45,
        is_defective=False,
        customer_tone="neutral",
        explanation="Color faded after two washes despite following the care label exactly."
    )
    print("Case 3:", triage_refund(req3))

Example output:

Case 1: {'decision': 'deny', 'reasoning': 'Handled by rule engine.', 'source': 'rule'}
Case 2: {'decision': 'approve', 'reasoning': 'Handled by rule engine.', 'source': 'rule'}
Case 3: {'decision': 'escalate', 'reasoning': 'Product did not meet reasonable durability expectations; best handled by human agent.', 'source': 'llm'}

Wrap-up

This pattern separates concerns: rules handle volume, and the LLM handles exceptions. To push it further, log every LLM decision to SQLite and audit them weekly, or swap in DeepSeek R1 671B on Oxlo.ai when you need explicit chain-of-thought reasoning for regulatory compliance.

DEV Community