Human in the Loop Is Not a Checkbox

Why approval steps fail when they are added at the end instead of designed into the workflow

Every serious enterprise AI design eventually adds the same box to the diagram.

It usually sits just before execution, and it says: Human Approval.

It makes everyone feel better. Security is satisfied. Compliance is satisfied. The business is satisfied. The model can make mistakes, but a person will check before anything important happens.

Then the system goes live.

In the first week, reviewers read every recommendation carefully. In the second week, they skim. By the end of the month, they approve forty requests in ten minutes, because the model is right most of the time and the queue keeps growing.

The box is still on the diagram.

The control is gone.

This is a hypothetical example, but anyone who has worked with approval workflows in ERP, finance, or sales operations has seen the pattern long before AI arrived.

AI makes it more important, not less.

In earlier articles, I wrote that a recommendation is not an authorization, and that high-impact actions should require human approval. Both statements are still true. But they leave an important question open:

What makes a human approval actually work?

A Human Who Approves Everything Is Not a Control

We often treat human approval as a safety property of the system. If a person approves the action, the action is safe.

But approval is not a property. It is a behavior.

And human behavior adapts to the workload and the feedback it receives.

When the model is right most of the time, reviewers learn that approving is almost always correct. When the queue is long, they learn that reading carefully is expensive. When nobody ever checks the quality of their reviews, they learn that speed is what matters.

None of this is laziness. It is a rational response to the way the workflow was designed.

The result has a familiar name: automation bias. People tend to trust the output of an automated system, especially when it is usually correct and when disagreeing requires extra effort.

This creates an uncomfortable situation:

Model is usually right
  ↓
Reviewers approve faster
  ↓
Reviews become shallow
  ↓
Rare model errors pass through
  ↓
The errors that matter most are the ones nobody catches

The human is still in the loop.

But the loop no longer contains judgment.

What remains is a name in the audit record: someone who formally approved the action, but no longer had the time, context, or incentive to say no.

Decide Where the Human Belongs

The first design mistake is putting a human approval step in front of everything.

It feels safe. In practice, it spreads human attention so thin that the important decisions receive the same three seconds as the trivial ones.

Human attention is a limited resource. It should be spent where it changes the outcome.

Three questions help decide where that is:

• How much damage can a wrong action cause? A wrong tag on a CRM note is not the same as a wrong credit limit.
• Can the action be reversed? Updating a draft can be undone. Sending an email to a customer, posting an invoice, or deleting data often cannot.
• How uncertain is the model, and how unusual is the case? A familiar request with complete data is different from an edge case the system has never seen.

Combining the first two gives a simple map:

	Easy to reverse	Hard to reverse
Low impact	Automate. Log it and sample it later.	Automate with strict validation and clear limits.
High impact	Automate with notification and a fast undo path.	Human approval, with real context and clear authority.

Uncertainty then moves individual cases between these zones. A low-impact action can still be routed to a person if the input is incomplete or the model's output fails a validation rule. A high-impact action with very predictable inputs may eventually need only sampling, once the system has earned that trust with data.

The goal is not to remove humans.

The goal is to make sure that when a person is asked to decide, the decision actually deserves a person.

Design the Review, Not the Button

Most approval screens are designed around a button.

The model produces a recommendation. The screen shows it. There is a green button and a red button.

That is not a review. It is a confirmation dialog.

A reviewer can only make a good decision if the screen gives them what they need to disagree. In practice, that usually means:

• The proposed action, in business terms. Not "call update_credit_limit with payload X", but "increase Acme's credit limit from 50,000 to 80,000".
• The evidence. Which records, documents, or messages the recommendation is based on, with links to the originals.
• What is unusual. A larger amount than usual, a customer with overdue invoices, missing data, a rule that was close to its limit.
• The consequence. What will happen if this is approved, and whether it can be undone.
• The alternatives. Approve, reject, edit, or send back with a question. A binary choice pushes people toward the default.

The third item is the most important and the most often missing.

A reviewer looking at forty similar requests cannot find the one that matters by reading forty summaries. The system has to help them see it. Highlight the differences, not the similarities.

There is also a subtle design question: should the reviewer see the model's recommendation before forming their own view?

For routine decisions, showing it saves time. For high-impact decisions, it can anchor the reviewer. Some teams ask the reviewer to look at the key facts first and only then show the recommendation. It is slower, but it keeps the human judgment independent.

The principle is simple:

If the approval screen makes approving easier than thinking, people will approve without thinking.

Approval Is Authority, Not Just a UI Step

An approval step is not only a screen. It is a transfer of responsibility.

When someone approves an AI-recommended action, the organization is saying: this person had the authority to make this decision, and they made it.

That statement only holds if the system enforces it.

This means the approval has to be part of the application's authorization model, not a flag the model or the frontend can set:

• The approver must have the authority. A sales representative should not approve a discount that requires a regional manager, just because the AI routed the request to the wrong queue.
• The approval must match what is executed. If the action changes after approval, for example a different amount or a different customer, the approval no longer applies.
• The approval must be fresh. An approval from last week may not be valid if the underlying data has changed since then.
• The requester should not approve their own request. Separation of duties does not disappear because an AI is involved.

The audit record should capture the whole decision, not only the click:

Request received
  → Model recommendation (with model and prompt version)
  → Evidence shown to reviewer
  → Reviewer identity and authority
  → Decision: approved, rejected, or edited
  → Reason, if rejected or edited
  → Action executed, and its result

This is what makes it possible to answer the question that eventually comes from an auditor, a customer, or a manager:

Who decided this, based on what?

"The AI recommended it and someone clicked approve" is not a good answer.

Measure the Loop

If human review is a control, it should be measured like one.

Most teams measure the model. Very few measure the reviewers. But the combined system is what produces the outcome.

A few signals tell you whether the loop is still working:

• Approval rate. If it is 99.8%, the reviewers are either looking at a perfect model or not really reviewing.
• Time per decision. Two seconds for a high-impact approval is a warning sign.
• Edit rate. How often reviewers change the recommendation instead of simply accepting or rejecting it. Edits are often the most valuable feedback the system gets.
• Errors found later. Approved actions that were corrected, reversed, or complained about afterward.
• Queue size and waiting time. A loop that delays every decision by two days will eventually be bypassed.

One useful technique is to occasionally insert known cases into the review queue: requests where the correct answer is already known, including some that should be rejected. It tells you whether reviewers catch real problems, not just whether they click.

And every rejection or edit should flow back into the evaluation set. A reviewer who says no is giving the system a lesson for free.

The strongest argument for removing the human

There is a fair objection to all of this.

Human review is slow and expensive. People are inconsistent. They get tired. In some workflows, a well-tested model with strong validation may make fewer mistakes than a hurried reviewer.

That can be true.

If the data shows that reviewers almost never change the outcome, keeping them in the loop may be costing time without adding safety. In that case, the honest answer is not to pretend the approval step is a control. It is to redesign it: automate the action, strengthen validation, add sampling, and keep a fast way to undo mistakes.

Removing a human from a step should be a decision based on evidence, just like adding one.

What is not acceptable is the middle state: a human who is formally responsible but practically absent.

A Practical Checklist

Before adding a human approval step to an AI workflow, it is worth asking:

• Which actions actually need a human, based on impact and reversibility?
• Which cases are routed to a person because they are unusual or uncertain?
• Does the reviewer see the proposed action in business terms, not as a tool call?
• Does the screen show the evidence and highlight what is unusual?
• Can the reviewer edit, reject, or ask a question, not just approve?
• Is the approver's authority enforced by the application, not by the routing?
• Is the approval tied to the exact action that will be executed?
• Does the audit record show who decided, based on what, with which model version?
• Do we measure approval rates, decision times, edits, and errors found later?
• Do rejections and edits flow back into the evaluation set?

If the answer to most of these is no, the approval step is probably a checkbox.

And a checkbox is not a control.

Conclusion

Adding a human to an AI workflow is easy. It takes one box on a diagram and one button on a screen.

Keeping that human meaningfully involved is much harder.

It requires deciding where human judgment actually changes the outcome. It requires review screens that make disagreement possible. It requires authorization rules that the application enforces, audit records that explain the decision, and metrics that show whether the loop still works.

This is not a new problem. Enterprise software has always had approvals that nobody really reads. AI simply increases the number of decisions and makes the recommendations more convincing.

The goal is not to put a human in front of every AI action.

The goal is to make sure that when a human decides, it is a real decision.

A human in the loop should be a source of judgment, not a signature on the model's work.