Human control should not mean approving every step.
It should mean that people retain the power to stop, reserve, revoke, and accept the actions that carry real consequences.
An agent that waits for approval before every tool call is not very autonomous. An agent that can do anything because someone once clicked "Allow" is not responsibly autonomous either.
This is the design tension. We want agents to handle more work, but we do not want capability to turn into a blank check.
In Part 1 of this series, I argued that agent autonomy is missing a layer of verifiable human authority. This article asks a narrower question: what does the human's final say look like when an agent is expected to operate without constant supervision?
"Human roles and responsibilities in decision making and overseeing AI systems need to be clearly defined and differentiated."
NIST also makes a useful distinction. Some systems may not need human oversight. Others specifically do. That sounds obvious, but many agent products still use one generic approval pattern for everything.
The wrong model of human control
When teams hear "human final say," they often picture a queue of confirmation dialogs. The agent proposes an action, a person clicks a button, and the same cycle repeats until everyone is tired of paying attention.
This does not produce meaningful control. It produces approval fatigue.
After enough prompts, people stop evaluating the action and start clearing the queue. The human remains technically present, but the system has trained that person to behave like an automatic approval service.
| Weak control | Useful control |
|---|---|
| Ask before every action | Reserve actions that cross a defined risk boundary |
| Use "important" as an undefined prompt word | Specify tools, data, limits, deadlines, and effects |
| Record that a button was clicked | Bind the decision to an identity, task, and proposed action |
| Let the same system act, verify, and accept | Separate execution from final acceptance |
The goal is not to maximize human involvement. The goal is to preserve human decision rights where the outcome becomes hard to reverse, externally visible, financially meaningful, or capable of harming someone else.
Final say is a boundary, not a meeting
A usable authority model needs four properties.
The decision maker is identifiable
The system knows which person or organizational role may authorize this class of action.
The intervention happens before execution
A reserved action pauses before the side effect, not after a retrospective alert.
The scope is explicit
The decision covers a named task, tool, target, limit, and time window.
The decision survives the interface
A third party can verify what was authorized without trusting the original product's dashboard.
None of these properties requires a committee. A person may approve an action in seconds. The important part is that the approval is precise enough for a machine to enforce and durable enough for another party to verify later.
This changes the role of the human. The person is no longer supervising the agent's every move. The person is defining the limits of autonomous work and deciding the few transitions that the system is not allowed to make alone.
A risk ladder for agent actions
OpenWorkProof resolves an action into three states before execution: allowed, reserved, or denied. The names are simple because the dispatcher needs an unambiguous answer.
| State | Meaning | Example |
|---|---|---|
| Allowed | The current signed authority permits autonomous execution | Read an approved repository inside the active work order |
| Reserved | A designated human must make a fresh decision first | Apply a patch that changes the candidate work product |
| Denied | The action is outside the available authority | Deploy to production when the work order grants no production access |
The local Human Agency example in OpenWorkProof currently demonstrates this distinction:
profile verified : True
resolved status : active
owp.repo_read : delegated -> allowed
owp.apply_patch : reserved -> AGENCY_HUMAN_DECISION_REQUIRED
The reserved patch does not execute and then ask for forgiveness. The dispatcher returns AGENCY_HUMAN_DECISION_REQUIRED before the handler starts.
This is how an agent can remain autonomous for routine work without receiving unlimited authority. Reading approved material may continue without interruption. Changing the deliverable crosses a boundary and pauses. Production deployment remains unavailable unless a new authority profile explicitly permits it.
Risk tiering is contextual. A repository read may be harmless in one work order and sensitive in another. The protocol records the decision boundary; it does not pretend that one global list can classify every organization.
Why a prompt cannot carry final authority
A prompt can tell an agent, "Ask me before doing anything important." The sentence is understandable to a person. It is weak as an authorization mechanism.
The model must decide what "important" means. A framework update can change how the instruction is interpreted. Another agent may never see the same conversation. The approval may remain inside a vendor log that the customer, verifier, or downstream partner cannot independently inspect.
A signed authority object behaves differently. It can bind the human decision to the work order, define the affected tools and targets, expire at a known time, and produce the same authorization result during offline verification.
What the WorkOrder permits
∩ What the CapabilityGrant delegates
∩ What the active HumanAgencyProfile allows
OpenWorkProof calls the third object a Human Agency Profile. It describes which part of the available capability a person is willing to let the agent use autonomously for this work.
Only the designated Acceptor can sign a profile transition. An appeal is a signed request for review. It does not restore permission, expand the scope, or turn a denied action into an allowed one by itself.
That last rule matters. If an agent could increase its own authority by filing an appeal, the safeguard would be decorative.
Keep the human out of the low-risk loop
Good oversight is selective. The human should see decisions that require judgment, not a transcript of everything the agent already did correctly.
A useful reserved-action request should answer concrete questions:
- Which work order is active?
- What exact action is waiting?
- Which resource, tool, or external party will be affected?
- What evidence and limits are already attached?
- What happens if the human does nothing?
The final question is easy to miss. A reserved action needs a safe timeout. Silence should not become implicit permission.
The system should also make denial cheap. If approval takes one click but rejection requires a long explanation, the interface is pushing the human toward the outcome the agent prefers. A signed denial can be brief. The record only needs to make the authority decision clear.
NIST's AI RMF Core calls for documented roles, responsibilities, and lines of communication. For agent execution, those organizational rules need a machine-readable form. Otherwise the written policy and the runtime can drift apart.
What businesses gain from selective authority
An enterprise does not want to choose between full manual review and unlimited autonomy. Both are expensive. The first consumes staff time. The second creates a risk that can surface after the action is irreversible.
Selective authority lets a team expand the autonomous surface while keeping a smaller set of consequential transitions under human control.
| Business requirement | Protocol behavior |
|---|---|
| Routine work should stay fast | Allowed actions execute without repeated approval |
| High-impact work needs accountable judgment | Reserved actions require a designated human signature |
| Agents must not negotiate their own permissions | Denied actions remain denied until an authorized profile changes |
| Delivery disputes need independent evidence | Authority and execution records can be verified offline |
This does not prove that an agent's output is correct. It proves a narrower set of facts: which authority existed, which action was proposed, whether the runtime allowed it, and who made the decision at the reserved boundary.
Verification and acceptance remain separate. A Verifier may confirm that the evidence supports the result. The Acceptor still decides whether the work meets the agreed delivery standard.
What exists in OpenWorkProof today
OpenWorkProof is open source under Apache-2.0. The repository's 1.3.0 local candidate includes the Human Agency Profile, Acceptor-signed transitions, pre-execution resolution, signed appeals, and offline verification.
The recorded repository snapshot reports 4,265 required-live tests and 183 candidate tests with zero failures or skips. These are local engineering results for that code and environment. They do not prove customer adoption, paid use, production reliability in another organization, legal compliance, or external acceptance.
The publicly released package and MCP Registry version remain 1.2.0. The 1.3.0 candidate has not been publicly released at the time of writing.
Evidence boundary: the implementation and local verification exist. Commercial adoption and production deployment require separate evidence.
Autonomy needs a stopping rule
Agent autonomy should remove unnecessary supervision. It should not remove the human right to decide what the system may do on that person's or organization's behalf.
The practical design is not "human in every loop." It is a defined stopping rule: the agent proceeds within signed limits, pauses at reserved boundaries, and cannot cross a denied boundary by persuasion or convenience.
That gives the agent room to work. It also gives the human a final say that the runtime can enforce and an independent party can verify.
If you are building an agent runtime, which actions would you allow, reserve, or deny by default? I am especially interested in where developers place the boundary for code changes, external communications, production access, and financial actions.
Explore OpenWorkProof on GitHub
Top comments (0)