Human as the Last Line of Defense — or Just a “Meatproxy”?

Why Bill Gates’ “Human Reserved” Concept Makes Sense in Theory — but Could Fail in Practice

Artificial intelligence and robotics are fundamentally changing the world of work. Bill Gates has warned that numerous activities across fields such as law, customer service, healthcare, software development, and industry could be automated within the next decade.

His proposed counterpoint is what he calls “Human Reserved”: certain tasks should remain the responsibility of humans even when machines are technically capable of performing them.

The idea is compelling — and fundamentally problematic at the same time.

The central challenge lies in the gap between requiring human oversight in AI systems and actually creating the conditions necessary for meaningful human control.

Companies primarily adopt AI to make work faster, cheaper, and more scalable. Genuine human oversight, however, requires time, expertise, and money. If these resources are not deliberately built into the process, the human can be reduced to little more than the final approval click.

The person may remain legally or organizationally responsible while no longer understanding the recommendation well enough to challenge it.

At that point, the human is no longer the final authority.

They become a “Meatproxy”: a human proxy who gives a decision formal legitimacy even though the decision has effectively already been made by a machine.

This raises the more important question:

Does a human actually still have control — or is a human merely standing at the end of the workflow?

The decisive test is not whether a human is involved somewhere in the process.

What matters is whether that person still has the knowledge, time, authority, and practical ability to make a different decision.

What Does Bill Gates Mean by “Human Reserved”?

Gates compares his proposal to a nature preserve.

A particular area might technically be available for development. Society can nevertheless consciously decide to protect it because losing that area would be undesirable.

In the same way, certain activities could be technically automatable while still being deliberately reserved for humans.

One example Gates gives is caring for his father, who had Alzheimer’s disease. In his view, the human connection created by caregivers cannot simply be replaced by a robot.

Similarly, he asks whether a machine should ever be the one to tell a patient that their illness is likely to be fatal.

Technically, a machine could deliver that message.

From a social and ethical perspective, however, Gates argues that this responsibility should remain with humans.

It is important to note that Gates’ original essay, “The Turbulent AI Era Is Here. The Choices We Make Now Are Critical,” is more nuanced than some headlines suggest.

He is not simply arguing that entire professions should be protected from AI.

In education and psychological support, for example, he describes hybrid models in which humans retain responsibility while AI expands their capabilities.

He also acknowledges that the boundaries may differ from country to country.

A country facing a severe shortage of caregivers might evaluate care robots differently from a country that consciously chooses to preserve human caregiving as a social value.

Gates also makes an economic argument.

Certain activities could be temporarily protected if rapid automation were to eliminate large numbers of jobs while leaving affected workers with few realistic opportunities for retraining.

He has also proposed taxes or fees on robots and AI use to help fund retraining and social safety nets.

Gates therefore does not present a complete regulatory framework.

Instead, he raises a broader societal design problem.

Who decides which activities should be protected?

According to what criteria?

How do we prevent companies from circumventing such rules?

And how could such protections work in an international economy?

These questions remain open.

The Advantages of “Human Reserved”

1. Protecting Human Relationships

For some services, the human relationship is not merely the means by which the service is delivered.

The relationship itself is part of the service.

Caregiving, grief counseling, education, therapy, and delivering devastating news involve more than simply communicating accurate information.

Eye contact, compassion, situational awareness, and the willingness to take responsibility for another person have intrinsic value.

An AI system can generate language that sounds empathetic.

That does not automatically mean that a reciprocal human relationship exists between the person and the system.

If human connection is evaluated solely by how convincing a sentence sounds, we reduce something profoundly human to a user interface.

2. Protecting Normative Decisions

Many decisions cannot be reduced to calculating a single objectively correct answer.

They involve value judgments:

• What level of risk is acceptable?
• Which interests should take priority?
• When is an exception justified?
• Which long-term strategy best reflects an organization’s goals?
• What outcome is socially desirable?

AI can structure arguments, data, and scenarios.

But the underlying values must not disappear silently into training data, system instructions, or optimization objectives.

“Human Reserved” can therefore mean something more fundamental:

The machine may provide the analysis — but humans determine which goals are worth pursuing in the first place.

3. Resilience in Exceptional Situations

Automated systems are particularly effective when handling frequent and structured cases.

Rare exceptions are different.

Changing circumstances, unusual combinations of factors, or faulty input data can create systematic errors.

Humans can identify contradictions, add missing context, and stop a process before an error spreads further.

This becomes especially important when a single error could have serious consequences — or when automation can reproduce the same error thousands of times.

4. Social Stability

Work provides more than income.

It can create:

• Social belonging
• Recognition
• Daily structure
• Professional identity
• Personal meaning

The rapid displacement of large groups of workers can therefore create social costs that never appear in a conventional cost-benefit calculation.

A company can calculate savings in personnel costs without accounting for the broader social consequences of mass unemployment.

Temporarily protecting certain activities could slow the transition and create time for retraining, new business models, and policy adaptation.

5. Trust and Social Acceptance

People are more likely to accept AI when they do not feel completely powerless in relation to it.

An accessible human contact, an effective complaints process, and the ability to have an automated decision reviewed can help create trust.

But there is one crucial prerequisite:

The human must actually be able to intervene.

A human who can only confirm what the system recommends does not provide meaningful human oversight.

The Central Contradiction: Human Oversight Consumes the Time AI Was Supposed to Save

The biggest practical problem with Gates’ proposal lies in the economics of AI adoption.

AI is intended to accelerate research, analysis, writing, and decision preparation.

If a human subsequently has to reconstruct and verify every relevant step, a large portion of the saved working time disappears again.

Organizations therefore face a fundamental trade-off:

• The more thoroughly a human reviews the result, the smaller the time advantage becomes.
• The greater the pressure to realize the time savings, the more superficial the review becomes.
• The more reliable AI appears in everyday use, the less likely people are to question it.
• The fewer tasks humans perform themselves, the faster they can lose the ability to recognize rare errors.

A rule such as “The final decision must always be made by a human” does not resolve this conflict.

It can actually conceal it.

On the organizational chart, the human remains responsible.

In practice, however, the system may control most of the information, reasoning, and decision-making process.

From Decision-Maker to “Meatproxy”

A person becomes a Meatproxy when they remain formally involved but no longer exercise an independent decision-making function.

Warning signs include:

• AI recommendations are almost always accepted.
• The human has only a fraction of the time previously required to evaluate a case.
• Users see only a summary instead of the underlying sources.
• The system produces convincing explanations without clearly presenting uncertainty or counterarguments.
• The human can technically reject the recommendation but must provide special justification for doing so.
• Speed and case volume are included in performance evaluations, while careful review is not.
• Expertise is no longer regularly applied and gradually deteriorates.
• When something goes wrong, responsibility is assigned to the human approver even though that person had little control over the system, data, or model behavior.

Researcher Madeleine Clare Elish describes a related phenomenon as the “moral crumple zone.”

The term describes situations in which humans absorb responsibility for the failure of an automated system even though their actual control over that system was limited.

The human is then not necessarily protecting society from the machine.

Instead, they may be protecting the technical and organizational system from responsibility and accountability.

Automation Bias and the Loss of Human Expertise

Excessive reliance on automated recommendations is not a problem that began with generative AI.

Earlier research on automation bias showed that decision-support systems can improve overall performance while simultaneously causing users to incorrectly accept automated recommendations.

Generative AI can intensify this problem because it does not merely output a number, score, or warning.

It produces a coherent explanation in natural language.

And that explanation can sound convincing.

But plausibility is not proof of correctness.

A convincing explanation can actually increase trust even when it does not accurately represent how the system arrived at its conclusion.

The Generative Artificial Intelligence Profile published by the National Institute of Standards and Technology (NIST) explicitly warns against excessive reliance on increasingly capable AI systems. Automation bias can contribute to fabricated information, bias, and the homogenization of results.

There is another related problem: the out-of-the-loop effect.

When people stop actively performing a task themselves, they can lose situational awareness and practical skills.

When a rare system failure eventually requires them to take over, they may be least prepared precisely when their expertise is needed most.

Lisanne Bainbridge described this problem as one of the “Ironies of Automation” as early as 1983. Later research by Mica Endsley and Esin Kiris experimentally examined the out-of-the-loop problem.

The irony is simple:

The more reliably automation performs under normal conditions, the less practice humans receive for exceptional situations.

A human fallback is therefore not automatically reliable simply because a human is available.

What Does a Human Actually Need to Understand?

It would be unrealistic to expect every AI user to understand the mathematical architecture of a Large Language Model.

Users of conventional software do not understand every single line of code either.

The more important question is:

At what level does human understanding need to exist for meaningful control to be possible?

Several levels of human competence can be distinguished:

1. Awareness

• Human capability: The user sees the result.
• Is this enough for meaningful control? No.
• Simply seeing an AI-generated result does not mean that the person can meaningfully evaluate or challenge it.

1. Plausibility Check

• Human capability: Based on experience and context, the result appears reasonable.
• Is this enough for meaningful control? Only for low-risk and easily reversible tasks.
• This level of review can be sufficient when errors have limited consequences and can be corrected easily.

1. Verification

• Human capability: Sources, assumptions, and critical intermediate steps can be checked.
• Is this enough for meaningful control? Suitable for many medium-risk decisions.
• The human must be able to verify the essential foundations of the AI-generated result.

1. Independent Judgment

• Human capability: The person could make a reasoned decision without AI and recognize when they should disagree with the AI.
• Is this enough for meaningful control? Required for consequential individual decisions.
• This is where human control becomes substantive: the person is not dependent on the AI to determine the correct course of action.

1. System Competence

• Human capability: The organization understands system limitations, failure patterns, data dependencies, and monitoring procedures.
• Is this enough for meaningful control? Required for responsible overall operation.
• This competence does not need to exist at the level of every individual user. It does, however, need to be clearly established somewhere within the organization.

Not every employee needs to possess every one of these levels of competence.

However, the organization must clearly define who possesses which capabilities and where responsibility lies.

For example, a caseworker might be responsible for the substantive decision, while another team monitors model quality, data protection, security, and systemic bias.

The person who ultimately clicks “Approve” should not automatically become the person who bears responsibility for the entire decision-making process.

Responsibility should follow actual control.

If a person does not have the necessary expertise, information, time, or authority to challenge an AI-generated recommendation, assigning formal responsibility to that person does not create meaningful human oversight.

The Risks of a Permanent “Human Reserve”

Gates’ proposal protects important values, but it could also create new problems.

Higher Costs and Lower Availability

Human services are limited and expensive.

In regions facing a shortage of doctors, a high-quality AI-assisted initial consultation may be better than no consultation at all.

The same applies to education, psychological support, and caregiving.

If a service is reserved exclusively for humans, access to that service could actually become worse.

The humane solution is therefore not automatically the one with the highest proportion of human labor.

Sometimes the human benefit comes precisely from AI making a service affordable and available.

Protecting Existing Professions Instead of Human Values

Professional groups could use “Human Reserved” to protect themselves from competition and technological change.

In that case, the rule would protect neither dignity, relationships, nor safety.

It would protect existing market positions.

Inefficient processes could remain in place even when automation would clearly benefit customers and society.

We therefore should not necessarily protect entire professions.

We should protect specific relationships, decisions, and areas of responsibility.

Humans Are Not Automatically Better

Humans make mistakes too.

They overlook information, make inconsistent decisions, and are influenced by bias, fatigue, incentives, and personal interests.

A decision is not automatically better simply because a human made it.

For some tasks, a properly tested automated system may be more consistent, fair, and reliable than an unstructured human decision.

The goal should therefore not be:

Human versus machine.

Instead:

The right capability for the appropriate level of risk.

International Competitive Disadvantages

If one country requires human involvement while another fully automates the same process, significant cost differences can emerge.

Companies may move activities abroad or purchase automated services from other countries.

Gates himself identifies this as an unresolved problem.

Without international coordination, reserving certain activities for humans could either become a competitive disadvantage or a largely symbolic rule that is difficult to enforce.

New Forms of Social Inequality

Two opposing developments are possible.

Wealthier customers could receive personal human services while everyone else is directed toward cheaper AI alternatives.

Conversely, mandatory human involvement could make services so expensive that lower-income people lose access.

A meaningful framework therefore needs to ask more than:

“Is a human involved?”

It must also ask:

“Who has access to human support — and who can afford it?”

Four Possible Operating Models

1. Full Automation

How it works:

AI performs the task from beginning to end with little or no human involvement.

Main advantages:

• Maximum speed
• High scalability
• Consistent results

Main risks:

• Errors can spread quickly.
• Humans have little control over individual decisions.

Suitable for:

Routine tasks that are standardized, measurable, and easy to reverse.

2. AI Recommendation + Human Approval

How it works:

AI provides a recommendation that is then reviewed and approved by a human.

Main advantages:

• Easy to integrate into existing workflows
• A human remains formally involved

Main risks:

• Humans may simply confirm what the AI recommends.
• Responsibility may remain with the human even though the AI has effectively made the decision.

Suitable for:

Tasks where the human has sufficient time, expertise, and authority to genuinely review the recommendation.

3. Human Judgment First, AI Second

How it works:

The human makes an initial assessment before seeing the AI recommendation. The AI then provides a second opinion.

Main advantages:

• Preserves independent human judgment
• AI can challenge or supplement the human assessment

Main risks:

• Requires more time
• Can result in duplicated work

Suitable for:

Rare or particularly consequential decisions where independent human judgment is critical.

4. Risk-Based Human Oversight

How it works:

The level of human involvement is determined by the potential consequences of an AI error.

Main advantages:

• Maximum control where it matters most
• Avoids unnecessary human effort for low-risk tasks

Main risks:

• The risk classification itself may be incorrect or incomplete.

Suitable for:

Most real-world AI applications within organizations.

The Best Approach?

In practice, organizations probably should not require a human to approve every single AI decision.

Instead, it makes more sense to match the level of human oversight to the level of risk.

Low-risk tasks can be largely automated.

Medium-risk tasks can combine AI recommendations with meaningful human review.

For high-risk decisions, independent human judgment should be preserved.

The goal is not to involve humans everywhere. The goal is to keep humans genuinely in control where it matters.

A Practical Model for Human Oversight

Green: Automation with Monitoring

Green-category tasks have limited impact, easily recognizable errors, and simple ways to correct mistakes.

Examples:

• Formatting
• Preliminary tagging
• Duplicate detection
• Internal drafts
• Routine classification

AI can operate largely autonomously here.

Organizations should nevertheless maintain logging, quality metrics, regular sampling, and a simple mechanism for reversing incorrect results.

Yellow: Verifiable Decision Support

Yellow-category tasks can have meaningful but generally reversible consequences.

AI generates a recommendation.

The human checks the critical facts and sources rather than reconstructing every technical processing step.

The system should provide at least:

• Original sources
• Key assumptions
• Known uncertainties
• Potential counterarguments
• Missing or contradictory information
• The potential consequences of an incorrect decision

A merely convincing explanation is not enough.

Red: Independent Human Judgment

Red-category decisions have irreversible or difficult-to-reverse consequences, involve loss of rights, significant financial impact, health risks, or particularly important ethical or personal questions.

Here, the human should ideally form an initial judgment before seeing the AI recommendation.

AI can then serve as:

• A research assistant
• A second opinion
• A source of counterarguments
• A consistency check

Depending on the consequences, additional safeguards may be necessary, such as the four-eyes principle, documented reasoning, and separation between recommendation and approval.

The lower time savings at this level are not a design flaw.

They are the price of genuine human oversight.

Seven Requirements for Preventing the Meatproxy Effect

1. A Dedicated and Funded Oversight Budget

Review requires working time.

If that time is not explicitly planned and funded, human oversight exists only on paper.

Organizations should define for each risk category what level of review is realistically required and who is responsible for providing it.

2. Access to Primary Sources

Users must be able to move from an AI-generated answer to the underlying documents and data relevant to the decision.

Another AI-generated summary is not an independent source.

3. The Right and Ability to Disagree

Anyone expected to reject an AI recommendation needs both the professional expertise and organizational authority to do so.

Disagreement should not automatically be treated as a performance problem.

There must also be a clear escalation path for difficult cases.

4. Preserve Human Expertise

Regular manual work, blind tests without AI recommendations, training, and collaborative analysis of errors can help preserve expertise.

A fallback system that is never practiced is not a reliable fallback system.

5. Independent Quality Measurement

Organizations should not measure AI success solely by how many cases are completed faster.

They should also monitor:

• Error types
• Corrections
• Complaints
• Near misses
• Differences between human and AI decisions
• Human performance without AI assistance

A consistently near-100% approval rate can be a warning sign.

It may indicate excellent model performance.

But it may also mean that humans are no longer genuinely reviewing the results.

6. Visible Limitations Instead of Artificial Certainty

AI systems should clearly communicate uncertainty, missing information, and known limitations.

Explanations should help users evaluate an answer — not merely convince them to trust it.

7. Responsibility Must Follow Actual Control

Responsibility should be distributed according to actual influence across:

• System operators
• Developers
• Data providers
• Executives
• Subject-matter experts
• Decision-makers

The person at the end of the workflow should not automatically become the liability sink for failures that originated elsewhere in the system.

Regulation Alone Is Not Enough

The European Union AI Act follows a similar principle for high-risk AI systems.

Human oversight is intended to enable people to understand a system’s capabilities and limitations, account for automation bias, correctly interpret outputs, override or disregard outputs, and stop the system when necessary.

The voluntary AI Risk Management Framework developed by the National Institute of Standards and Technology (NIST) also emphasizes clearly defined roles, competency requirements, training, risk-based resources, and continuous monitoring.

These requirements are important.

But they do not resolve the underlying economic conflict.

No law or guideline automatically creates:

• Expertise
• Review time
• Organizational authority
• The willingness to challenge an AI system

An organization can formally assign a human a role in a process while simultaneously designing that process so that the person has virtually no realistic option other than confirming the result.

Effective human oversight is not a function of an approval button.

It is a property of the entire operating model.

Example: AI in IP Management Software

Consider software for managing patents and trademarks.

A sensible division of responsibilities could look like this:

1. Identifying Document Types and Suggesting Keywords

• AI role: Largely automated processing
• Human role: Conduct spot checks and make simple corrections

2. Summarizing File Contents

• AI role: Generate a draft with source references
• Human role: Review the content before external or strategic use

3. Structuring Research Results

• AI role: Organize results and identify similarities and gaps
• Human role: Assess relevance and legal significance

4. Assessing Deadlines or Legal Status

• AI role: Prepare information and flag inconsistencies
• Human role: Perform a binding review using reliable official and rule-based sources

5. Recommending Whether to Maintain or Abandon an IP Right

• AI role: Present costs, portfolio context, and different scenarios
• Human role: Make an independent business and legal decision

6. Communicating Negative Procedural Decisions

• AI role: Prepare the wording
• Human role: Take responsibility for the content and personally handle sensitive communication

Why Software Design Matters

Software design is critical.

A system that saves time by hiding context weakens human oversight.

A system that saves time by structuring sources, contradictions, and decision-relevant information, on the other hand, can strengthen it.

That distinction is fundamental.

Conclusion: Human Oversight Is Not a Free Feature

Bill Gates is right to raise a question that is often overlooked in the AI debate:

Not everything that can technically be automated should automatically be automated.

Human relationships, normative judgment, responsibility, and social stability have a value that cannot be fully expressed through processing time and cost.

However, the “Human Reserved” concept remains incomplete unless it is connected to the economic reality of organizations.

Companies adopt AI because they want to save time.

Meaningful human oversight costs time.

If organizations are unwilling to pay that price, the human becomes a Meatproxy: approving decisions they can no longer independently evaluate while assuming responsibility for systems they did not design.

The solution is neither unrestricted automation nor the universal requirement for a human to click “Approve.”

What we need is a deliberate distribution of responsibility:

• Automation for reversible, measurable routine tasks.
• Verifiable AI assistance for medium-risk decisions.
• Independent human judgment for irreversible and normative decisions.
• Human relationships where the relationship itself is part of the service.

“Human Reserved” should therefore not become a protected zone for entire professions.

It should become a protected space for human judgment, responsibility, and relationships.

Because meaningful human oversight requires time, expertise, authority, and money.

If organizations promise meaningful human control, they must provide all four.

Otherwise, the human is not the last line of defense.

They are merely the last signature.

Further Reading

• Bill Gates — The Turbulent AI Era Is Here. The Choices We Make Now Are Critical (2026)
• European Union — Regulation (EU) 2024/1689, the EU AI Act, particularly Article 14 on human oversight
• National Institute of Standards and Technology (NIST) — Artificial Intelligence Risk Management Framework
• National Institute of Standards and Technology (NIST) — Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile (2024)
• Lisanne Bainbridge — Ironies of Automation (Automatica, 1983)
• Mica R. Endsley & Esin O. Kiris — The Out-of-the-Loop Performance Problem and Level of Control in Automation (Human Factors, 1995)
• Madeleine Clare Elish — Moral Crumple Zones: Cautionary Tales in Human-Robot Interaction (2019)
• Kate Goddard, Abdul Roudsari & Jeremy C. Wyatt — Automation Bias: A Systematic Review of Frequency, Effect Mediators, and Mitigators (2012)