DEV Community

kanaria007
kanaria007

Posted on

Design DEGRADE (Defer) and Your Agent Becomes “Operations”

Most agents don’t break because they violated a rule.

They break because they kept going while missing the grounds to decide: approvals, evidence links, unambiguous IDs, confirmed state, dependency health, etc.

If your system only has two outcomes—

  • ACCEPT (do it)
  • REJECT (don’t)

—then real work quickly devolves into “humans rescue everything by interpretation,” and audits become “we can’t reproduce why this happened.”

So if you want an agent to become operable in production, the first thing you should design is not REJECT, but:

DEGRADE (DEFER): a safe, re-enterable stop.
Not failure—a controlled branch that turns “we can’t decide yet” into a runnable workflow.


0) Reminder: LLMs propose. Deterministic systems decide.

Treat the LLM as a proposer (plans, drafts, suggested steps).
Pin audit-grade grounds somewhere else:

  • input schema (what counts as admissible grounds)
  • policy_id + policy_version
  • deterministic rule-evaluation logs
  • evidence / trace IDs

Your execution system should run typed actions only, after verification (dry-run → approval → production).


1) Why “REJECT-only” collapses in practice

1) Missing grounds get mislabeled as “violations”

If every “unknown / missing” becomes REJECT, operators constantly override via interpretation. Over time, tacit knowledge grows, not the system.

2) The agent starts “filling in” what’s missing

When many fields are missing, an LLM will often move toward making the story coherent—because that “feels helpful.” That’s exactly how you get execution on fake grounds.

3) Exceptions never become a process

What you actually need is a durable record of:

  • what was missing (missing keys / approvals / evidence)
  • what you requested next (request list)
  • which gate to retry after (next gate)

Without that, postmortems become rituals, not learning.


2) The DEGRADE design patterns

DEGRADE is a mechanical branch: “we can’t decide safely → stop.”
The trick is making the stop implementable and operable.

Pattern A: Return reason codes + missing list (not prose)

When you DEGRADE, don’t return a question. Return:

  • category reason code (why we stopped)
  • missing requirements list (what’s missing, as machine paths)

Examples:

  • MISSING_APPROVAL: approvals.security_approved
  • MISSING_EVIDENCE: change_request.dashboard_id
  • STATE_UNKNOWN: current_flag_state_unavailable

Pattern B: Provide DEGRADE-specific typed actions

DEGRADE is not “do nothing.” It’s a standard next step:

  • request_more_info
  • escalate
  • open_task
  • attach_evidence

The LLM can draft human-facing messages—but the verifier decides what must be requested.

Pattern C: Deterministically separate “missing” from “violations”

Draw a hard line:

  • REJECT = confirmed rule violation (forbidden role, outside allowed window, missing required SLO gates…)
  • DEGRADE = missing grounds / uncertain state / dependency down / contradictions
  • ACCEPT = all prerequisites satisfied

Pattern D: Make DEGRADE progress (not an infinite hold)

A good DEGRADE must have exits:

  • auto-create a task
  • route to an approval workflow
  • request evidence links
  • auto-escalate on timeout

Pattern E: Design DEGRADE reason codes as category + granularity + SLO

DEGRADE will grow. That’s fine.

What’s bad is: “we don’t know what kind of DEGRADE this is,” “we don’t know how long it’s been stuck,” “we can’t improve it.”

Minimum requirements:

  1. Category: maps almost 1:1 to owner team / runbook / SLO
    Examples: MISSING_APPROVAL, MISSING_EVIDENCE, STATE_UNKNOWN, DEPENDENCY_UNAVAILABLE, CONFLICT_DETECTED, POLICY_AMBIGUOUS.

  2. Granularity: return a mechanical missing list (paths), not prose
    Examples: approvals.sre_approved, change_request.rollback_plan_id, etc.

  3. SLO metadata: DEGRADE is “a state with time,” so it needs timers
    Suggested fields:

  • retry_after_seconds
  • escalate_after_seconds
  • owners (who owns this)
  • deadline_at (optional hard stop)

Pattern F: Make DEGRADE re-enterable

This is the final move that makes DEGRADE “operations”:

If missing grounds get filled, the same case can be retried through the same verifier and produce a fresh verdict.

Key techniques:

  1. Fix a join key (request_id / change_id / case_id)
  2. Make validation “pure-ish”: doc in → verdict out (If external state is required, stop with STATE_UNKNOWN / DEPENDENCY_UNAVAILABLE.)
  3. Return a resume token + enforce idempotency in the executor
  4. Represent human approval as typed actions (approval.collect, approval.record, case.resume)
  5. Provide exactly one resume entrypoint (one API/job)

3) Example: Change management (production rollout) with DEGRADE

Production config changes are “small” but have huge blast radius—perfect for DEGRADE-first design.

3.1 Input schema (pin the grounds)

change_request:
  change_id: "CHG-2026-00112"
  service: "billing-api"
  environment: "prod"
  created_at: "2026-02-15T22:30:00+09:00"   # required time claim (missing => DEGRADE)
  change_type: "feature_flag_rollout"
  flag_key: "new_invoice_flow"
  to: { enabled: true, percent: 10 }
  window:
    start_at: "2026-02-16T01:00:00+09:00"
    end_at:   "2026-02-16T03:00:00+09:00"
  risk_level: "MEDIUM"
  rollback_plan_id: "rb-2026-0091"
  dashboard_id: "dash-billing-api"
  alert_policy_id: "alert-billing-slo"

approvals:
  owner_approved: true
  sre_approved: false

policy:
  policy_id: "prod-change-policy"
  policy_version: "2026-01-10"

guardrails:
  canary:
    step_percent: [10, 25, 50, 100]
    step_wait_minutes: 15
  slo_gates:
    - metric: "error_rate_5m"
      op: "<="
      threshold: 0.01
Enter fullscreen mode Exit fullscreen mode

3.2 Typed actions (including DEGRADE exits)

  • change.plan.publish
  • feature_flag.set_percent
  • slo_gate.check
  • rollback.execute
  • audit.append
  • change.request_more_info (DEGRADE)
  • change.block (REJECT)

3.3 Minimal verifier (stdlib-only) that prefers DEGRADE over “guessing”

Below is the core idea:

  • missing approval / missing rollback plan / missing observability → DEGRADE
  • missing SLO gates / invalid canary steps / outside allowed window → REJECT
  • on DEGRADE, return:

    • degrade_reasons (categories)
    • missing (mechanical paths)
    • SLO metadata (retry_after_seconds, escalate_after_seconds, owners)
    • resume_token
    • a typed action change.request_more_info

(Code below uses PEP 604 union types like DegradeMeta | None, so it requires **Python 3.10+.)

from __future__ import annotations

import hashlib
import json
from dataclasses import dataclass
from datetime import datetime
from enum import Enum, unique
from typing import Any, Dict, List, Sequence, Tuple


@unique
class VerdictLevel(Enum):
    ACCEPT = "ACCEPT"
    REJECT = "REJECT"
    DEGRADE = "DEGRADE"


@unique
class RejectReason(Enum):
    OUTSIDE_CHANGE_WINDOW = "outside_change_window"
    NO_SLO_GATES = "no_slo_gates"
    INVALID_CANARY_STEPS = "invalid_canary_steps"
    INVALID_STEP_WAIT = "invalid_step_wait"
    PROD_CHANGE_POLICY_MISSING = "prod_change_policy_missing"


@unique
class DegradeReason(Enum):
    # Categories that map to owners + SLOs
    MISSING_CHANGE_ID = "missing_change_id"
    MISSING_APPROVAL = "missing_approval"
    MISSING_ROLLBACK_PLAN = "missing_rollback_plan"
    MISSING_OBSERVABILITY = "missing_observability"
    MISSING_TIME_CLAIM = "missing_time_claim"
    MALFORMED_TIMESTAMP = "malformed_timestamp"
    DEPENDENCY_UNAVAILABLE = "dependency_unavailable"
    STATE_UNKNOWN = "state_unknown"
    CONFLICT_DETECTED = "conflict_detected"


@unique
class ActionName(Enum):
    CHANGE_REQUEST_MORE_INFO = "change.request_more_info"


@dataclass(frozen=True)
class TypedAction:
    name: ActionName
    params: Dict[str, Any]


@dataclass(frozen=True)
class DegradeMeta:
    missing: Tuple[str, ...]
    retry_after_seconds: int
    escalate_after_seconds: int
    resume_token: str
    owners: Tuple[str, ...]


@dataclass(frozen=True)
class Verdict:
    level: VerdictLevel
    reject_reasons: Tuple[RejectReason, ...] = ()
    degrade_reasons: Tuple[DegradeReason, ...] = ()
    degrade_meta: DegradeMeta | None = None
    normalized_plan: Tuple[TypedAction, ...] = ()


def _parse_iso8601(s: str) -> datetime:
    return datetime.fromisoformat(s)


def _sha256_hex(b: bytes) -> str:
    return hashlib.sha256(b).hexdigest()


def _make_resume_token(change_id: str, missing: Sequence[str], policy_version: str) -> str:
    payload = {"change_id": change_id, "missing": list(missing), "policy_version": policy_version}
    b = json.dumps(payload, sort_keys=True, separators=(",", ":")).encode("utf-8")
    return _sha256_hex(b)


# Category -> (retry_after, escalate_after, owners)
_DEGRADE_SLO: Dict[DegradeReason, Tuple[int, int, Tuple[str, ...]]] = {
    DegradeReason.MISSING_CHANGE_ID: (0, 10 * 60, ("owner",)),
    DegradeReason.MISSING_TIME_CLAIM: (0, 10 * 60, ("owner",)),
    DegradeReason.MISSING_APPROVAL: (0, 30 * 60, ("owner", "security", "sre")),
    DegradeReason.MISSING_OBSERVABILITY: (0, 30 * 60, ("sre",)),
    DegradeReason.MISSING_ROLLBACK_PLAN: (0, 30 * 60, ("owner", "sre")),
    DegradeReason.MALFORMED_TIMESTAMP: (0, 10 * 60, ("owner",)),
    DegradeReason.DEPENDENCY_UNAVAILABLE: (5 * 60, 30 * 60, ("sre",)),
    DegradeReason.STATE_UNKNOWN: (5 * 60, 30 * 60, ("sre",)),
    DegradeReason.CONFLICT_DETECTED: (0, 30 * 60, ("owner",)),
}


def _merge_degrade_meta(change_id: str, missing: List[str], reasons: List[DegradeReason], policy_version: str) -> DegradeMeta:
    retry = 0
    esc = 30 * 60
    owners: List[str] = []

    for r in reasons:
        r_retry, r_esc, r_owners = _DEGRADE_SLO.get(r, (0, 30 * 60, ("owner",)))
        retry = max(retry, r_retry)
        esc = min(esc, r_esc)
        for o in r_owners:
            if o not in owners:
                owners.append(o)

    token = _make_resume_token(change_id, missing, policy_version)
    return DegradeMeta(
        missing=tuple(missing),
        retry_after_seconds=retry,
        escalate_after_seconds=esc,
        resume_token=token,
        owners=tuple(owners),
    )


def _validate_canary_steps(steps: Any) -> bool:
    if not isinstance(steps, list) or len(steps) < 2:
        return False
    try:
        ints = [int(x) for x in steps]
    except Exception:
        return False
    if any(p <= 0 or p > 100 for p in ints):
        return False
    if any(ints[i] >= ints[i + 1] for i in range(len(ints) - 1)):
        return False
    return ints[-1] == 100


def validate_change_request(doc: Dict[str, Any]) -> Verdict:
    reject: List[RejectReason] = []
    degrade: List[DegradeReason] = []
    missing: List[str] = []

    policy = doc.get("policy", {})
    policy_id = policy.get("policy_id")
    policy_version = policy.get("policy_version")
    if not policy_id or not policy_version:
        return Verdict(VerdictLevel.REJECT, reject_reasons=(RejectReason.PROD_CHANGE_POLICY_MISSING,))

    change = doc.get("change_request", {})
    approvals = doc.get("approvals", {})

    change_id = change.get("change_id")
    if not change_id:
        degrade.append(DegradeReason.MISSING_CHANGE_ID)
        missing.append("change_request.change_id")
        meta = _merge_degrade_meta("(missing)", missing, degrade, policy_version)
        return Verdict(
            VerdictLevel.DEGRADE,
            degrade_reasons=tuple(degrade),
            degrade_meta=meta,
            normalized_plan=(TypedAction(ActionName.CHANGE_REQUEST_MORE_INFO, {
                "change_id": None,
                "missing": list(meta.missing),
                "resume_token": meta.resume_token,
                "owners": list(meta.owners),
                "retry_after_seconds": meta.retry_after_seconds,
                "escalate_after_seconds": meta.escalate_after_seconds,
            }),)
        )

    # Missing approvals => DEGRADE
    if approvals.get("owner_approved") is not True:
        degrade.append(DegradeReason.MISSING_APPROVAL)
        missing.append("approvals.owner_approved")

    risk = change.get("risk_level", "MEDIUM")
    if risk in ("HIGH", "CRITICAL") and approvals.get("sre_approved") is not True:
        degrade.append(DegradeReason.MISSING_APPROVAL)
        missing.append("approvals.sre_approved")

    # Missing rollback/observability => DEGRADE
    if not change.get("rollback_plan_id"):
        degrade.append(DegradeReason.MISSING_ROLLBACK_PLAN)
        missing.append("change_request.rollback_plan_id")

    if not change.get("dashboard_id") or not change.get("alert_policy_id"):
        degrade.append(DegradeReason.MISSING_OBSERVABILITY)
        missing.append("change_request.dashboard_id / alert_policy_id")

    # Missing SLO gates is a design violation => REJECT
    guardrails = doc.get("guardrails", {})
    slo_gates = guardrails.get("slo_gates", [])
    if not isinstance(slo_gates, list) or len(slo_gates) == 0:
        reject.append(RejectReason.NO_SLO_GATES)

    # Canary config must be valid => REJECT
    canary = guardrails.get("canary", {})
    if not _validate_canary_steps(canary.get("step_percent", [])):
        reject.append(RejectReason.INVALID_CANARY_STEPS)

    try:
        step_wait = int(canary.get("step_wait_minutes", 15))
        if step_wait <= 0:
            reject.append(RejectReason.INVALID_STEP_WAIT)
    except Exception:
        reject.append(RejectReason.INVALID_STEP_WAIT)

    # Window parsing failures => DEGRADE, outside window => REJECT
    try:
        start = _parse_iso8601(change["window"]["start_at"])
        end = _parse_iso8601(change["window"]["end_at"])
    except Exception:
        degrade.append(DegradeReason.MALFORMED_TIMESTAMP)
        missing.append("change_request.window.start_at/end_at")
        start = end = None

    if not change.get("created_at"):
        degrade.append(DegradeReason.MISSING_TIME_CLAIM)
        missing.append("change_request.created_at")
        now = None
    else:
        try:
            now = _parse_iso8601(change["created_at"])
        except Exception:
            degrade.append(DegradeReason.MALFORMED_TIMESTAMP)
            missing.append("change_request.created_at")
            now = None

    if start and end and now and not (start <= now <= end):
        reject.append(RejectReason.OUTSIDE_CHANGE_WINDOW)

    if reject:
        return Verdict(VerdictLevel.REJECT, reject_reasons=tuple(reject))

    if degrade:
        meta = _merge_degrade_meta(change_id, missing, degrade, policy_version)
        return Verdict(
            VerdictLevel.DEGRADE,
            degrade_reasons=tuple(degrade),
            degrade_meta=meta,
            normalized_plan=(TypedAction(ActionName.CHANGE_REQUEST_MORE_INFO, {
                "change_id": change_id,
                "missing": list(meta.missing),
                "resume_token": meta.resume_token,
                "owners": list(meta.owners),
                "retry_after_seconds": meta.retry_after_seconds,
                "escalate_after_seconds": meta.escalate_after_seconds,
            }),)
        )

    return Verdict(VerdictLevel.ACCEPT)
Enter fullscreen mode Exit fullscreen mode

3.4 The key point

When DEGRADE happens, the verifier returns:

  • degrade_reasons: what category of missing grounds this is
  • missing: exactly what is missing (paths)
  • normalized_plan: a typed “exit action” like change.request_more_info

Then the LLM can do what it’s good at: drafting messages and proposing next steps—bounded by a deterministic missing list.


4) DEGRADE turns exceptions into “operations”

DEGRADE doesn’t just reduce accidents. It converts exception handling into a process:

  • missing grounds are logged → improvable
  • requests become typed actions → automatable
  • “stopping” becomes normal → safer systems

If you can’t place DEGRADE, you can’t operate the agent—because most real incidents are “missing grounds,” not “clear violations.”


Summary

  • The most common real-world failure mode is missing grounds, not rule violations.
  • DEGRADE is not failure; it’s a safe, re-enterable stop.
  • Make DEGRADE operable with:

    • reason codes
    • missing lists
    • DEGRADE-specific typed actions
    • SLO metadata
    • re-entry (join key + resume token + idempotency)
  • Let the LLM do remediation assistance, not eligibility decisions.

If you can’t design DEGRADE first, your agent won’t be operable.

Top comments (0)