Last month I watched a teammate approve an agent's database migration in about four seconds. The UI showed a green checkmark, a confident one-line summary — "schema update, low risk" — and a single Approve button. What it didn't show: the agent had only verified the migration against a staging snapshot, the rollback script had never been executed, and two downstream services still pinned the old column name.
The migration was fine, luckily. But the decision mechanism was broken. The human in the loop was asked to decide with the evidence hidden, the consequence unstated, and no clear point of reversibility. That's the pattern I want to fix in this article — and I'll give you a concrete review card spec, a stop-condition table, and a research protocol you can run this week.
The decision that matters
Every human-in-the-loop approval has three properties worth naming before you design anything:
- Decision owner — who is accountable if this goes wrong? (Not "the team". A person or role.)
- Consequence class — reversible in seconds, reversible with effort, or irreversible?
- Point of reversibility — the last moment where stopping costs nothing.
The failure mode in most agent UIs is that all three are implicit. The approval button treats a config tweak and a production schema change identically: one click, one toast, done.
The pattern: an evidence-first review card
Instead of summary → approve, invert it: evidence → verdict → action. The agent's own conclusion comes last, visually and cognitively, so the reviewer forms an opinion before being anchored by the model's.
Annotated card fields
┌─────────────────────────────────────────────────────┐
│ ACTION REQUESTED │
│ Run migration 0042_add_index on production DB │
│ Consequence class: Reversible with effort │
│ Last safe stop point: before "Execute" (this card) │
├─────────────────────────────────────────────────────┤
│ EVIDENCE (what the agent actually checked) │
│ ✓ Dry-run against staging snapshot 2026-08-01 │
│ ✓ No lock conflicts in pg_stat_activity (sampled) │
│ ✗ Rollback script: NEVER EXECUTED │
│ ✗ Downstream services: 2 still reference old column │
├─────────────────────────────────────────────────────┤
│ AGENT VERDICT (collapsed by default) │
│ "Low risk, recommend proceeding." │
├─────────────────────────────────────────────────────┤
│ [Execute] [Run rollback test first] [Reject] │
└─────────────────────────────────────────────────────┘
Three deliberate choices:
-
Failed or missing checks render as first-class rows, not absence.
✗ Rollback script: NEVER EXECUTEDmust be louder than the checks that passed. Missing evidence should stop approval; decorative extra information should not. - The verdict is collapsed by default. Anchoring bias is the whole reason the card exists.
-
The middle action is recovery-shaped.
Run rollback test firstconverts the discovered gap into a next step instead of a binary yes/no.
Stop conditions: decide them before you ship
This is the artifact I'd hand to any team building an approval flow. Fill in the right column for your action classes:
| Missing or failing evidence | Approval behavior |
|---|---|
| Rollback path never executed (irreversible-class action) | Hard stop. Approve button disabled until rollback test passes. |
| Verification ran against stale/non-production data | Hard stop. Show snapshot age next to the check. |
| Agent confidence below your stated threshold | Require a second human reviewer, not just a warning badge. |
| Evidence coverage < 100% of declared checks | Approve button requires typed justification ("proceeding without X because…"), logged. |
| Extra context the agent volunteered but no check depends on | Show, don't block. This is noise — surfacing it as a blocker trains reviewers to ignore real stops. |
The last row matters as much as the first. If everything is alarming, nothing is.
A cheap way to prototype this: let the model draft, keep the human deciding
You don't need production infrastructure to test whether this card changes reviewer behavior. The evidence rows themselves can be drafted by an LLM from the agent's tool-call logs — the model summarizes what was checked; the card, not the model, decides what's missing.
For that prototype, I've been using MonkeyCode, which offers free model access plus a free server option, so a small team can stand up a review-harness endpoint and iterate on prompts without provisioning anything or putting a card on file. Disclosure: This article was prepared as part of MonkeyCode's product outreach.
A minimal, labeled-as-pseudocode sketch of the harness loop:
# Pseudocode — not executed; adapt to your stack.
log = agent_session.tool_calls # what the agent actually did
draft = llm.summarize_evidence(log, # free-tier model is fine here
schema=EVIDENCE_ROW_SCHEMA) # force structured output
card = ReviewCard(
evidence=merge(draft, run_deterministic_checks(log)), # checks re-run, not trusted
verdict_collapsed=True,
stop_conditions=load_policy("migration.yaml"),
)
if card.any_hard_stop():
card.disable("approve")
render(card)
Note the line run_deterministic_checks(log): the model drafts prose, but anything that gates the Approve button is re-verified by boring deterministic code. The LLM is a drafting assistant for evidence display, never the source of truth for whether evidence exists.
Research protocol: does the card actually change decisions?
Don't ship on vibes. Run this with 6–8 people who would realistically own these approvals:
- Scenario A (current UI): one-line summary + Approve. Measure time-to-decision and whether they catch the missing rollback test.
-
Scenario B (evidence-first card): same underlying situation, evidence rendered with the missing check as a
✗row. - Scenario C (noise test): same card plus three pieces of irrelevant volunteered context. Measure whether the real stop still lands.
Success measures: catch rate for the missing rollback check in B vs A; justification quality when participants override.
Stop measures for the study itself: if B doesn't beat A on catch rate, the card is decoration — change the design, not the participants. If C's catch rate collapses, your noise budget is too high and you should cut fields, not add color.
Separate what you observed from what you hypothesize. "Catch rate improved" is a finding. "Users trust the agent more" is a hypothesis until you measure trust directly.
Accessibility review (non-negotiable for approval UIs)
-
Don't encode check status in color alone. The
✓/✗glyphs carry meaning; pair them with text ("passed","not run") and ensure screen readers announce status, not just the label. - The collapsed verdict must be operable by keyboard and announced as expandable — and its collapsed state must be the default for everyone, not a mouse-only design detail.
-
Disabled Approve buttons must explain themselves. A grayed-out button with no reason is an accessibility and comprehension failure. Use
aria-describedbypointing at the failing check. - Focus management on stop: when a hard stop triggers, move focus to the explanation, not to a toast that disappears.
Limitations and who should not use this
- This pattern adds friction. For genuinely reversible, low-consequence actions (rename a label, regenerate a draft), an evidence-first card is over-engineering — use a lightweight undo instead.
- The evidence summary is only as good as the agent's tool-call logging. If your agent doesn't emit a faithful trace, fix observability first; no card design compensates for a model narrating checks that never ran.
- Free tiers — MonkeyCode's included — are appropriate for prototyping and evaluation harnesses, not as a stated commitment of quotas or permanence; don't bake them into a production approval path without your own capacity plan.
- If your approval flow is regulatory (medical, financial), you need audit-grade logging and a compliance review on top of everything here.
The question to bring back to your team
Before your next agent-approval design review, ask two things: which missing piece of evidence should stop approval outright, and which extra information would only add noise. If the room can't answer both quickly, that's the work — and the review card is just where the answers become visible. If you prototype the harness above (a free model and a free server are enough to start), I'd genuinely like to hear what your catch rates look like in the comments.
Top comments (0)