Logan for Waxell

Posted on Jul 20 • Originally published at waxell.ai

30 PRs Daily: Why HITL Approval Gates Break at Scale

#ai #agents #llm #governance

A tech lead at a mid-size SaaS company described his morning routine in a piece the Pydantic team published this week: thirty pull requests waiting, each one produced overnight by a teammate's AI coding agent, each one needing a snap judgment call before standup. The temptation to delegate the review itself to an AI was enormous. He resisted. But he said out loud what a lot of teams are only thinking: "At that point, what am I still doing here?"

The Hacker News thread it spawned — 115 points, 58 comments within hours — made clear this isn't an isolated complaint. It's a structural problem, and it has two versions.

The first version (the one in the article) is about developer experience: the cognitive load of reviewing AI-generated code at volume, the erosion of craft, the creeping suspicion that sustained vigilance over a high-volume approval queue is not actually a job that humans do well.

The second version is harder to talk about, because it shows up in production rather than in personal frustration. In enterprise AI deployments where agents aren't writing code but acting on live systems — sending emails, modifying records, initiating payments, calling external APIs — the same dynamic plays out with real business and regulatory risk attached to it. Approval queues designed as safety mechanisms quietly become rubber stamps. Not because the people reviewing are careless. Because the approval gate was built wrong.

Human-in-the-Loop Is Not the Problem. The Trigger Is.

Human-in-the-loop (HITL) in agentic systems means the agent pauses execution and waits for a human decision before proceeding. Used correctly, this is the right design for a meaningful class of actions: irreversible writes, external sends, high-cost operations, scope escalations. The problem is that most teams implement the trigger on action category rather than on risk signal.

Category-based HITL: "All write actions require approval."

Signal-based HITL: "Write actions that (a) touch externally-facing records AND (b) have not been previously authorized for this agent session AND (c) exceed the session's cost baseline require approval."

Category-based triggers create uniform queues. Every write — from updating an internal metadata field to deleting a customer's account — lands in the same approval inbox with the same priority. Signal-based triggers create meaningful decision moments: the human is only asked when the combination of factors actually warrants it.

The distinction sounds obvious. Very few teams implement the second version on the first try.

They implement category-based gates because they're easier to reason about, easier to build, and feel safer ("everything gets reviewed"). Then, six weeks later, the approval queue has 200 items, the average approval rate is 96%, and someone asks why HITL was implemented at all since nothing is ever rejected.

The 96% approval rate doesn't mean 96% of the actions were safe. It means the reviewer stopped reading. This is the agentic equivalent of security alert fatigue — a well-documented failure mode in which high-volume, low-information-content queues cause reviewers to stop making genuine decisions. The alert still fires. The human still clicks approve. The loop still technically contains a human. But the human is no longer a meaningful control.

Three Structural Mistakes in Enterprise HITL Design

Mistake 1: Triggers on action type, not risk profile.

Covered above. The fix is policy composition: define trigger conditions from multiple signals. Cost threshold crossed. Sensitive data pattern matched. First-time action type for this agent instance. External endpoint not previously called in this session. When two or more of these fire simultaneously, that's a meaningful approval moment. One signal alone, in most cases, isn't.

Mistake 2: Synchronous blocking for everything.

Some approvals genuinely need to block execution and wait. If an agent is about to send an email to 50,000 customers, blocking is correct — the cost of proceeding without approval exceeds the cost of waiting. But most approval-worthy actions don't have that profile.

A low-risk write that's uncertain enough to flag can be parked in an async queue while the agent continues work on non-dependent tasks. Treating every approval as a synchronous hard stop creates a latency tax on every operation, which creates legitimate pressure to route around the approval gate entirely.

The practical design splits approvals into two modes: blocking holds (agent cannot proceed until resolved) for genuinely irreversible high-risk actions, and deferred approvals (agent parks the action and continues other work) for lower-risk flagged operations. Deferred approvals remove the productivity argument for bypassing HITL.

Mistake 3: No escalation routing or time-bound defaults.

What happens when no one responds to an approval request? Most first-generation implementations default to one of two unsustainable states: block indefinitely (which creates operational paralysis over weekends and off-hours) or auto-approve after a timeout (which eliminates the safety guarantee).

The right default depends on the action's risk profile, which means the policy needs to specify it explicitly. Low-risk deferred: auto-execute after N minutes if no response. High-risk blocking: escalate to secondary reviewer after M minutes, then default to blocked if still unresolved. Indefinite blocking and blanket auto-approval are both wrong answers; they just fail in different directions.

The Human-on-the-Loop Spectrum

Human-on-the-loop (HotL) is typically presented as the alternative to human-in-the-loop: the agent operates autonomously within defined guardrails, and a human monitors rather than approves individual actions. This framing creates a false binary.

Production agents don't choose between HITL and HotL. They operate on a policy-controlled spectrum.

At the base layer, the agent runs autonomously: read operations, internal state changes, actions within established cost and scope budgets. In the middle layer, the agent operates with soft guardrails: signals are logged, thresholds generate warnings, anomalies trigger async review workflows. At the ceiling, the agent hard-pauses for explicit human approval: destructive actions, scope escalations, external sends above a defined risk score.

Most well-designed production agents live in the middle layer most of the time. HITL is a ceiling, not a mode. When it fires, it needs to mean something. The engineering work is calibrating where the ceiling sits for each action class in the agent's operating envelope.

How Waxell Runtime and MCP Gateway Handle This

The architecture described above isn't a theoretical design — it's directly how Waxell Runtime implements governed execution for production agent workflows.

Rather than blanket approval gates, Runtime exposes @agent, @workflow, and @decision decorators that let teams declare exactly which decision points warrant human approval. The policy context is defined in the same execution environment: cost ceilings, PII redaction scope, recursion bounds, approval scope. An agent running under a policy that includes Approval on writes — external sends doesn't pause on every write. It pauses when a write crosses the external send boundary and that scope hasn't been pre-authorized for the session. The distinction is structural, not behavioral.

For teams governing assistants they didn't build — Claude Desktop, Cursor, ChatGPT, and other MCP-compatible clients — Waxell MCP Gateway applies the same logic at the tool call layer. Risky actions park for human approval; the MCP connection stays open while the review happens. Policy changes propagate in 30 seconds, which matters when an action category changes risk profile and the team needs to update approval requirements without waiting for a deployment cycle.

Both draw from the same Waxell Observe policy engine — 50+ policy categories including dedicated Delegation, Control, Cost, and Safety categories, backed by over 1,000 pre-built policies — which means trigger logic can be composed from specific named conditions rather than built from scratch. The difference between a blanket approval gate and a calibrated one is largely a difference in how precisely the trigger conditions are expressed.

The framing Waxell uses for the broader problem is worth quoting directly: "A dashboard after the fact is not governance. It's an autopsy." HITL approval that fires on the wrong triggers — or fires so often it stops being read — is governance theater. Policy-triggered HITL is what governance actually looks like in production.

Why This Problem Is Getting More Urgent

The Pydantic article surfaced a developer experience problem: sustained human attention on high-volume AI output queues creates a specific kind of cognitive exhaustion that is different from the exhaustion of doing the underlying work. That insight applies directly to enterprise HITL governance, with the added dimension that the actions being approved aren't code diffs but live system operations.

Under NIST AI RMF's GOVERN function, meaningful human oversight requires that the human in the loop is making genuine decisions, not rubber-stamping a queue they've been trained by volume to auto-click. Under the EU AI Act's Article 50 transparency and logging requirements (in effect for deployers as of August 2, 2026), audit trails that document human oversight decisions need to reflect actual oversight rather than pro forma approval records.

Blanket category-based HITL generates exactly the kind of audit trail that satisfies the letter of these requirements while failing the intent: thousands of approval events with near-uniform approve decisions, logged timestamps showing sub-five-second review times, no documentation of the reasoning applied to any individual decision.

Policy-triggered HITL generates fewer approval events — because most low-risk actions are handled autonomously — and more meaningful ones, with the trigger conditions documented in the policy configuration and the decision record reflecting genuine human judgment applied to situations that actually warranted it.

The throughput is better. The audit trail is cleaner. And the human at the approval queue is making decisions rather than processing a workload.

FAQ

What is policy-triggered human-in-the-loop, and how is it different from standard HITL?

Policy-triggered HITL fires approval requests based on specific combinations of risk signals — cost thresholds crossed, sensitive data patterns matched, novel action types for the session, external scope escalations — rather than on action categories alone (e.g., "all write operations"). The result is a lower-volume approval queue where each item represents a genuine decision point, rather than a high-volume queue where reviewers stop engaging meaningfully.

What's the difference between human-in-the-loop and human-on-the-loop in AI agent governance?

Human-in-the-loop means the agent pauses execution and waits for explicit human approval before proceeding on a specific action. Human-on-the-loop means the human monitors the agent's behavior and can intervene, but doesn't approve individual actions. In production, most well-designed agents use both: automated execution for routine low-risk operations, and policy-triggered hard pauses for high-risk or anomalous actions. The two modes aren't alternatives — they're layers on the same governance stack.

What causes HITL approval fatigue in enterprise AI deployments?

Approval fatigue typically results from category-based HITL triggers ("all write actions require approval") rather than signal-based triggers ("writes that cross specific risk thresholds require approval"). High-volume queues with low information variance — where most items are similar and most decisions are the same — train reviewers to stop engaging with individual approval requests. The failure mode mirrors security alert fatigue: the alert still fires, the human still clicks, but the oversight stops being real.

What actions should always require human approval in AI agent workflows?

Irreversible external actions (emails sent to end users, payments initiated, records deleted in external systems), actions that escalate scope beyond the agent's established authorization, and actions that cross predefined cost or data sensitivity thresholds. The specific trigger conditions should be defined in governance policy rather than hardcoded by action type, so the threshold can be adjusted without redeployment as the agent's risk profile changes.

What should happen when no reviewer responds to a HITL approval request?

Time-bound defaults should match the action's risk profile. Low-risk deferred approvals can auto-execute after a defined timeout — the risk of proceeding is lower than the operational cost of blocking. High-risk blocking approvals should escalate to a secondary reviewer and default to blocked if still unresolved. Auto-approving high-risk holds and blocking indefinitely on low-risk holds are both wrong defaults; they fail in opposite directions.

Can human-in-the-loop be implemented without slowing down AI agents?

Yes, through asynchronous deferred approval for non-blocking action types. If an agent's flagged write operation doesn't block other tasks, parking it for async human review while the agent continues other work preserves throughput without eliminating oversight. Hard synchronous blocking should be reserved for actions where proceeding before approval would be irreversible or genuinely high-risk. Most HITL implementations block synchronously on everything because it's simpler to build — not because the workload requires it.