Completion Claim ≠ Accepted State: A Verification Contract for Agents

Original research note: English · 中文 · Research Center

An Agent finishes its last action and says the task is complete. Should the runtime believe it?

In long-running Agent work, that is where a surprising number of failures begin. The final message may sound convincing even when a required action was skipped, the result was never read back, an external blocker stopped the workflow, or an unwanted side effect occurred.

The worker's statement is useful, but it is still only a completion claim.

Microsoft Research's Universal Verifier work makes this distinction concrete for computer-use Agents. Its architecture separates process quality from outcome attainment, distinguishes controllable from uncontrollable failures, and retrieves screenshot evidence against individual rubric criteria instead of trusting the last screen or the Agent's own summary.

The reusable engineering pattern is not “add one more judge model.” It is a separation of responsibilities:

worker action
→ completion claim
→ deterministic readback + claim-specific evidence
→ independent verification
→ acceptance authority
→ completion receipt

Three checks that should not be collapsed

1. Deterministic checks

These answer questions that should not require interpretation:

• Does the database record exist?
• Did the expected file change?
• Did the test suite pass?
• Was the commit created and pushed?
• Is the external system actually in the requested state?

2. Learned verification

Some outcomes need interpretation. A verifier may inspect a trajectory, screenshots, intent alignment, failure responsibility, and side effects. This is useful, but it should produce a versioned judgment with evidence references—not an unquestionable truth value.

3. Human or policy acceptance

Irreversible, ambiguous, regulated, or high-impact work still needs an acceptance authority. That authority may accept, reject, escalate, or leave the claim undetermined.

The three checks can disagree. A worker may follow the correct process but be blocked by a CAPTCHA. A visible outcome may be achieved through an unexpected valid route. A task may look complete while producing an unrequested side effect. Preserving those disagreements is safer than forcing every run into completed or failed.

A minimal runtime state model

running
→ completion_claimed
→ verifying
→ accepted | rejected | escalated | undetermined

The Microsoft paper reports strong results, including a 1% false-positive rate on its internal split and 8% on the external Browserbase split. That is promising, but not a universal production guarantee. The benchmark is relatively small, the work focuses on web computer-use trajectories, and human labels are still judgments rather than an objective completion oracle.

The main lesson is architectural: the same component should not be both claimant and final acceptor.

How are you handling this in Agent systems?

• Can the worker mark its own task complete?
• Which evidence must exist before done is accepted?
• Do you preserve undetermined and escalated as real states?

Sources: Microsoft Research paper · Universal Verifier implementation

More research notes: JoinWell52 Research Center