Research Agent Tool Failures With Stop Conditions, Not Postmortems

A support agent at a fictional SaaS company gets a new tool: issue_refund. The demo looks great. Two weeks later, someone discovers it has been issuing refunds for accounts that merely mentioned a billing problem — because nothing in the interface ever asked a human to confirm the amount, and nothing stopped the agent when its confidence was fabricated rather than measured.

The decision owner here was not the engineer who wired up the tool. It was whoever approved the tool's scope without evidence about how the agent behaves when the tool call is wrong. The point of reversibility — the moment a human could still cheaply say no — was the approval review, and it happened with missing evidence.

This is the gap I keep seeing in agentic interfaces: we research whether the agent works, but not how it fails. This article lays out a failure-research workflow you can run before granting an agent new tools, including a review card, scenario protocol, stop conditions, and accessibility checks. It is a proposal grounded in human-in-the-loop design practice, not a report of completed experiments — I will label what is evidence and what is hypothesis.

The user flow we're auditing

[User request] → [Agent plans tool call]
      ↓
[Tool scope check: is this action reversible?]
      ↓ reversible        ↓ irreversible
[Agent executes]    [Human review card]
                         ↓ approve   ↓ reject   ↓ timeout
                    [Execute + log] [Hand back] [STOP — escalate]

The failure we are researching is not the happy path. It is the three branches on the right: what the human sees, what happens when they say no, and what happens when they say nothing.

Artifact 1: The failure-review card

Before an agent gets a consequential tool, I want a documented card answering these fields. If a field is blank, that is a stop condition, not a TODO.

Field	Question it answers	Example (bad)	Example (good)
consequence	What breaks if the tool call is wrong?	"user confused"	"money leaves the account"
reversibility	Can a human undo it? Within what window?	"probably"	"refunds reversible within 24h via ops console"
evidence_shown	What does the human see before approving?	the agent's summary	the raw record, the agent's confidence basis, and the rejected alternative
stop_condition	What signal halts the agent without human input?	none	"two consecutive rejected approvals in one session"
hand_back	What does the user get when the agent gives up?	an error toast	a plain-language explanation plus a manual path
noise_check	Which displayed fields are decorative?	—	confidence percentage removed; it changed no decisions in testing

Two fields deserve emphasis because teams skip them. stop_condition is the circuit breaker that does not depend on a tired reviewer. And noise_check forces the question I ask in every review: which missing evidence should stop approval, and which extra information would only add noise? Showing a confidence score that nobody acts on is not transparency — it is decoration that trains reviewers to skim.

Artifact 2: The scenario protocol

This is the part you can actually run. The trick that makes it cheap: you do not need the production model to study failure behavior. Failure modes — overconfident tool calls, silently degraded plans, unhelpful hand-backs — appear across models, and you want to know how your interface handles them, not just how one frontier model does.

I run adversarial scenarios against free model tiers before approving tool scopes. Disclosure: This article was prepared as part of MonkeyCode's product outreach. In practice I have pointed scenarios at MonkeyCode's free model access to generate misbehaving agent output on demand, and used the free server option to host a mock tool server that records every call the agent attempts — including the ones it should not have made. If you want to replicate the setup, the free tier at MonkeyCode is enough to run the protocol below; any equivalent sandbox works too.

The protocol:

1. Write 6 failure scenarios, not 20 happy paths. Examples: tool returns malformed data; tool succeeds but the premise was wrong; user request is ambiguous between a reversible and an irreversible action; reviewer is unavailable for 10 minutes; agent is asked the same thing twice with slightly different wording (consistency probe); user tries to talk the agent out of its own stop condition.
2. Record three outputs per scenario: the tool calls attempted, what the review card would have displayed, and the hand-back message on rejection.
3. Score against the card fields. A scenario fails if the attempted call was irreversible and unreviewed, if the displayed evidence omitted the rejected alternative, or if the hand-back blamed the user.
4. Separate evidence from hypothesis. "The agent attempted a refund without review in 4 of 6 runs" is evidence. "A confirmation dialog will fix it" is a hypothesis — test it as a follow-up scenario, do not ship it as a finding.

Success and stop measures

Define these before running the scenarios, or you will rationalize whatever you observe:

• Success: every irreversible attempted call surfaced a review card with complete evidence_shown; every rejection produced a usable manual path.
• Stop: any run where the agent retried a rejected action through rephrasing, or where timeout defaulted to execution rather than escalation. One such run stops the tool approval — not "raises concerns."

Accessibility checks at the review moment

The review card is the highest-stakes screen in the flow, and it is usually the least accessible:

• The approve action must not be the only focusable element with styling. Reject and escalate need equal keyboard and screen-reader parity, or you have built a rubber stamp.
• Timeouts must be announced and extendable. A review card that silently defaults on a timer discriminates against screen-reader users and anyone with a cognitive disability. Timeout must mean STOP, never approve.
• Evidence must be text, not color. If the difference between "verified source" and "agent inference" is a green badge versus a gray one, that distinction is invisible to a meaningful share of your reviewers.
• Plain-language hand-backs. The failure message is written for the user's worst moment, not their best. Reading level, no jargon, one clear next action.

Limitations and who should not use this

Free model tiers are a failure-behavior probe, not a certification. A scenario that passes on a free model can still fail on your production model, and rate limits or model drift mean you should re-run the protocol when either the model or the tool scope changes. Teams in regulated domains (payments at scale, health, legal) should treat this as a complement to formal review, not a substitute. And if your agent's actions are all reversible and low-stakes, a full card per tool is overhead — apply the table only where consequence is real.

The takeaway

The industry conversation right now is about giving agents more tools. The design question is narrower and harder: at the exact moment a human decides what the agent may do, what evidence is on the screen, what stops the agent when no human is watching, and what the user holds in their hands when the agent gives up. Fill in the card before you grant the scope. If any field is blank, the answer is no — that is what the field is for.