Designing an Escalation Gate: Deciding What Never Reaches a Model Call

Most support or triage traffic does not need a model at all, and the cheapest call you can make is the one you skip entirely. The workflow below puts a small deterministic gate in front of a model tier, so only the residue reaches it. Everything here is an explicitly hypothetical case study: the project does not exist, the code is illustrative and was not executed, and no metrics are reported as measured.

The hypothetical project and its constraint

Imagine a small open-source documentation site with a support inbox that receives short questions about password resets, invoices, and API errors. The maintainer wants draft replies, but cannot justify a paid inference bill for a side project. The hard constraint is therefore not answer quality alone; it is how few requests you can honestly push through a paid-capable tier.

The design goal is narrower than "build a support bot". You want a gate that classifies each incoming request into exactly one of three lanes, and you want that decision to be explainable after the fact. Read the lanes as a contract rather than a suggestion, because the whole cost argument depends on them being enforced in one place.

• Resolve — a deterministic rule or cache hit answers the request with a known documentation page.
• Escalate — the request is well-formed, in scope, and has no safe canned answer, so a model drafts a reply.
• Refuse — attachments, ambiguous one-word messages, billing disputes, or anything a human must own.

The important property is ordering: rules and cache run before any network call, and refusal runs before escalation. If you invert that order, you pay for requests you already knew how to reject.

The gate as a single decision function

The gate should be one pure function you can unit test without a model, a network, or a database. Keep it boring, keep it synchronous, and return a lane plus a reason string you can log. The listing below is illustrative and incomplete; it has no error handling and no persistence.

# gate.py — illustrative only. Not executed as part of this article.
from __future__ import annotations
from dataclasses import dataclass
from typing import Literal

Lane = Literal["resolve", "escalate", "refuse"]

@dataclass(frozen=True)
class Ticket:
    id: str
    text: str
    locale: str
    has_attachment: bool

# Ordered, auditable rules: substring -> canonical docs page.
RESOLVE_RULES: list[tuple[str, str]] = [
    ("reset password link expired", "docs/account/reset-password.md"),
    ("invoice vat number", "docs/billing/vat.md"),
    ("401 invalid api key", "docs/api/auth.md"),
]

def normalize(text: str) -> str:
    return " ".join(text.lower().split())

def gate(ticket: Ticket, cache: dict[str, str]) -> tuple[Lane, str, str]:
    """Return (lane, payload, reason). Pure: no I/O, no clock, no model."""
    if ticket.has_attachment:
        return "refuse", "", "attachment_requires_human"

    key = normalize(ticket.text)
    if key in cache:
        return "resolve", cache[key], "cache_hit"

    for needle, page in RESOLVE_RULES:
        if needle in key:
            return "resolve", page, f"rule:{needle}"

    if len(key.split()) < 4:
        return "refuse", "", "insufficient_context"

    return "escalate", "", "no_deterministic_answer"

Two details matter more than the code itself. First, the reason string is part of the interface, because a lane without a reason is impossible to audit when a user complains. Second, the function takes the cache as an argument instead of reading a global, which makes replay tests deterministic.

Keeping the escalated lane small and typed

Escalation should never be a free-form prompt. Give the model a narrow job: draft a reply that cites one documentation page from an allowlist, or return a structured "cannot answer" value. Validate that output before it reaches a human queue, and drop anything that fails validation back into the refuse lane. You are trading coverage for tractability on purpose.

Where the free tier fits

MonkeyCode offers free model access and a free server option, and those two pieces map onto this design in different places. The free server runs the gate, the replay harness, and a tiny queue; the free model access supplies the escalated lane only. That split keeps the gate available even when the model path is slow or unavailable.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Because availability terms, quotas, and licensing are exactly the kind of thing that changes without notice, read the project's own documentation before you size a design around it. Do not assume a free tier is permanent, and do not assume its throughput matches a paid endpoint. Treat the tier as a replaceable dependency behind an interface, which is what the next section is about.

# escalate.py — illustrative interface boundary; unexecuted.
from typing import Protocol

class Drafter(Protocol):
    def draft(self, question: str, allowlist: list[str]) -> str | None: ...

def escalate(ticket_text: str, drafter: Drafter, allowlist: list[str]) -> tuple[str, str]:
    answer = drafter.draft(ticket_text, allowlist)
    if answer is None or not answer.strip():
        return "refuse", "drafter_declined"
    return "escalate", answer

With that boundary in place, swapping providers is a constructor change rather than a refactor. It also gives you an honest fallback: if the drafter returns nothing, the ticket lands in a human queue instead of producing a confident wrong answer.

How you verify the design before trusting it

You cannot claim an escalation rate without a labeled set, so build one first and keep it small and boring. A hundred hand-labeled requests from your own inbox beats a synthetic benchmark you cannot defend. The harness below is illustrative, but the measurement plan is the part worth copying.

1. Assemble labeled.jsonl with fields text, locale, expected_lane, and a short note explaining the label.
2. Replay through gate() only, with the model stubbed out, and record lane agreement per request.
3. Inspect every disagreement by hand and decide whether the rule or the label is wrong before you touch either.
4. Measure the escalated share as a count, not an estimate, and compare it against your tolerance for model calls.
5. Re-run after any rule change, because a rule that fixes one ticket often silently reclassifies ten others.

# replay.py — illustrative; expects labeled.jsonl next to this file.
import json
from gate import Ticket, gate

def main() -> None:
    cache: dict[str, str] = {}
    agree = total = 0
    missed_escalation = 0
    with open("labeled.jsonl", encoding="utf-8") as fh:
        for line in fh:
            row = json.loads(line)
            total += 1
            ticket = Ticket("t", row["text"], row["locale"], False)
            lane, _, _ = gate(ticket, cache)
            agree += lane == row["expected_lane"]
            missed_escalation += row["expected_lane"] == "escalate" and lane == "resolve"
    print({"total": total, "agree": agree, "missed_escalation": missed_escalation})

if __name__ == "__main__":
    main()

The metric that should worry you most is missed_escalation, not raw agreement. A rule that answers a request it should have escalated is worse than a rule that escalates too often, because the failure is invisible to the user until it is a complaint. Set a hard threshold for it, and treat any regression as a release blocker.

A decision table you can argue about

Write the lane policy down before writing rules, so the arguments happen once instead of per ticket. The table below is a starting position, not a universal answer.

Request shape	Lane	Why	Owner
Known FAQ wording	Resolve	Deterministic page, no inference needed	Gate
Short how-to with docs coverage	Escalate	Needs phrasing, not new facts	Model draft
Refund or billing dispute	Refuse	Money and policy decisions	Human
Attachment or screenshot	Refuse	Unverifiable content in free tier	Human
One-word or empty message	Refuse	No context to classify honestly	Human

Limitations and who should skip this

Rule-based gates rot. Product pages get renamed, wording shifts, and a rule that fired last quarter silently stops matching. Budget review time for the rule set, and re-run the replay harness on a schedule rather than once.

Caching has its own failure mode: a stale page becomes a confident wrong answer for every matching request. Version your cache keys against the documentation commit, and invalidate on deploy rather than on a timer you will forget. Also keep personal data out of escalated prompts, since a free tier is still a third-party service.

Skip this approach if your requests are mostly high-stakes billing, legal, or medical questions, because refusal should dominate and the model lane adds risk without much value. Skip it too if you have no labeled sample and no willingness to build one, since you would be shipping rules you cannot evaluate. And skip it if your volume is tiny, because manual triage is cheaper than maintaining a gate.

If you want to stand up the escalated lane without committing a card, MonkeyCode's free model access and free server option are a reasonable place to start; verify the current terms in the project's own documentation first, then let your replay numbers — not the tier's marketing — decide whether the gate is worth keeping.