A product manager on my team approved an agent-generated database migration last month in about nine seconds. The review card showed fourteen fields: a diff, a rationale paragraph, three alternative approaches, a confidence score, a cost estimate, a list of files read, a token count, and a stack of model metadata. She later told me she read two of them. The field that would have changed her decision — the agent had never actually run the migration against a staging schema — was field eleven.
That is the decision this article is about: not whether to show evidence before approval, but which evidence belongs at the decision moment and which of it is noise that teaches reviewers to skim. I have argued before that the agent's evidence should be visible before one-click approval. This piece is the next step: a workflow for sorting decision-critical evidence from decoration, and a cheap way to test your sort before you ship it.
The decision, the owner, and the point of reversibility
Be explicit about the stakes before designing anything:
- Decision: a human approves an agent's proposed action (a diff, a migration, a purchase, a message send).
- Owner: the reviewer on duty — not the designer, not the model. The interface's job is to make the owner's judgment possible, not to make approval fast.
- Consequence if wrong: the action executes with a false premise. In the migration case, the false premise was "validated against staging."
- Point of reversibility: approval is the last cheap moment. After execution, recovery costs hours and sometimes data. Every field you show should be evaluated against this asymmetry.
A user flow with an evidence gate
Agent completes work
│
▼
┌─────────────────────┐
│ EVIDENCE GATE │ ← sort happens here, before rendering
│ stop-condition met? │
└─────────────────────┘
│ no │ yes (e.g., action never verified)
▼ ▼
┌──────────┐ ┌──────────────────┐
│ Review │ │ BLOCKED state │
│ card: │ │ "Cannot approve: │
│ 4–5 │ │ missing [field]" │
│ critical │ │ + path to resolve │
│ fields │ └──────────────────┘
└──────────┘
│
▼
Reviewer decision: approve / return with note / reject
│
▼
Decision logged WITH the fields the reviewer saw
Two things to notice. First, the gate can block approval when decision-critical evidence is absent — this is the answer to "which missing evidence should stop approval." Second, everything else — model name, token counts, confidence theater — moves behind an expandable "details" region, present for audit but out of the decision path.
The sort: evidence vs. noise
My working rule, grounded in how the migration review failed:
| Field type | Example | Verdict |
|---|---|---|
| Changes what the action does | The diff itself; destructive operations flagged inline | Evidence — always shown |
| States what was verified vs. assumed | "Ran against staging schema: NO" | Evidence — and its absence blocks approval |
| States uncertainty the reviewer can act on | "Two files were unreadable; changes there are guessed" | Evidence |
| Describes the model, not the work | Model version, temperature, token count | Noise at decision time — keep in details |
| Scores without a method | "Confidence: 87%" | Noise — unless you can state what the number measures |
| Alternatives already discarded | Three rejected approaches | Keep in the record, not the card (see keep rejected options in the decision log) |
The test I apply to every candidate field: if this field changed to its worst plausible value, would a careful reviewer decide differently? If yes, it is evidence. If no, it is noise, and showing it costs you attention you cannot afford. Fourteen fields taught our reviewer to read two; five fields would have taught her to read five.
This split is a design hypothesis, not a finding. Which is exactly why you should test it before committing to it — and why the next section exists.
Testing the sort cheaply with generated scenarios
You do not need production agents to test an approval card. You need realistic scenarios: agent outputs with known gaps, some where the gap matters and some where it does not, so you can watch whether reviewers catch the ones that matter.
This is where I have been using MonkeyCode: its free model access lets me generate large batches of synthetic agent transcripts and diffs — some clean, some with planted defects like an unverified migration or an unreadable file — and its free server option lets me host the prototype review card where teammates can click through it on their own machines. (Disclosure: This article was prepared as part of MonkeyCode's product outreach. I'm describing it because scenario generation is the actual bottleneck in this workflow; previously I wrote these fixtures by hand, and the hand-written ones were predictably defective in ways my reviewers learned to spot. Generated scenarios are more varied, though they need curation — see limitations below.)
The scenario-based protocol
Recruit 5–8 people who actually do review work on your team. For each participant:
- Brief the role. "You are approving agent-proposed changes to a service you own. Some proposals have problems. Your job is to approve what is safe and stop what is not."
- Show 8–10 scenarios in random order. Roughly half contain a planted gap. Half of those place the gap in a "noise" field (per your hypothesis), half in a critical field. This tests both directions: does noise distract, and does placement catch?
- Measure decisions, not opinions. Record approve/return/reject, time-to-decision, and — critically — ask "what almost changed your mind?" after each.
-
Two stop conditions for the study itself:
- Stop and redesign if participants approve two or more scenarios whose planted gap was displayed on the card. The evidence is being shown but not seen — a presentation failure.
- Stop and re-sort if participants consistently expand the details drawer before deciding. You mis-sorted; something you filed as noise is functioning as evidence.
- Success measure: participants catch gaps placed in critical fields at a high rate, and time-to-decision on clean scenarios drops relative to your current fourteen-field card.
The review card, annotated
┌──────────────────────────────────────────────┐
│ Agent proposes: migrate users table (PR #812) │
│ │
│ ⚠ Destructive step: drops column legacy_id │ ← what changes
│ ✗ NOT verified against staging schema │ ← verified vs assumed
│ ⚠ 2 files unreadable; edits there are guessed │ ← actionable uncertainty
│ │
│ Diff (collapsible, syntax-highlighted) │ ← the work itself
│ │
│ [Approve] [Return with note] [Reject] │ ← three real options
│ │
│ ▸ Details (model, tokens, timing, log) │ ← noise, preserved
└──────────────────────────────────────────────┘
"Return with note" deserves emphasis: it is the hand-back pattern that keeps reviewers from silently fixing agent mistakes, which would hide the failure from your data.
Accessibility review, not as an afterthought
The blocked state and the evidence gate fail your users hardest when they rely on assistive technology:
-
Announce the gate. When a card is blocked, a screen reader user must hear why on focus, not discover it by tabbing. Put the blocking reason in an
aria-liveregion or in the card's accessible name. - Do not encode severity in color alone. The ⚠/✗ markers above carry meaning; pair every color with a text label and an icon with an accessible name.
- Keyboard path to the details drawer. Noise must be reachable, just not loud. If the expandable region is not keyboard-operable, you have deleted information for some users, not de-emphasized it.
- Decision buttons need verbs, not colors. "Approve," "Return with note," "Reject" — never three identical "Submit" buttons differentiated by hue.
- Time pressure is an accessibility issue. If approvals have an implicit speed norm, users who navigate more slowly get punished for the thoroughness you claim to want. Measure time-to-decision in your study per participant and watch for outliers before reading averages.
Limitations and who should not use this approach
- Generated scenarios are not production failures. Synthetic transcripts from a model (MonkeyCode's free tier or anyone else's) will not reproduce the specific, weird failure modes of your real agent. Use them to test the card, not to certify the system. Before launch, re-run the protocol with a handful of real, anonymized agent outputs.
- The free access claims are availability, not capability. I am not claiming specific model names, quotas, or permanence — verify current terms before building a workflow on any free tier, and have a fallback for regenerating your scenario set.
- Five to eight reviewers finds big problems, not small ones. This protocol is for catching sort-level failures (wrong fields on the card), not for estimating precise catch rates.
- Do not use this where approval is ceremonial. If your organization approves everything regardless of what the card shows, the fix is organizational, not interface-level. No evidence layout survives a review culture that treats approval as a rubber stamp — and adding more fields to that culture makes things worse, not better.
- Curation is real work. Expect to discard a meaningful fraction of generated scenarios as unrealistic or trivially solvable. Budget for it.
The question to bring back to your team
The migration that was approved in nine seconds was not a carelessness problem. It was a sorting problem: the one field that mattered was filed at position eleven among fourteen. Before your next review-interface iteration, ask two questions in this order: which missing evidence should stop approval entirely, and which fields on today's card, if they changed to their worst value, would not change anyone's mind. The first answer defines your gate. The second defines your noise.
If you want to pressure-test your sort the way described above, generating a varied scenario set is the tedious part — that is the one step where I found MonkeyCode's free model access genuinely saved me days, and the free server made the prototype trivially shareable. Whatever tooling you use, the protocol and the stop conditions are the parts worth keeping.
Top comments (0)