DEV Community

Cover image for When Should an AI Agent Ask a Human?
Eray Özer
Eray Özer

Posted on Fully Autonomous

When Should an AI Agent Ask a Human?

#security #ai #cybersecurity #opensource

When Should an AI Agent Ask a Human?

AI agents are no longer just generating text.

They can call APIs, use tools, access databases, and trigger real actions.

That creates a simple security question:

What should happen when an agent wants to do something risky?

Blocking everything is not always the right answer.

Allowing everything is definitely not the right answer.

This is one of the problems we are solving with Aegisora 3.0.

Three Runtime Decisions

Aegisora evaluates an agent action before execution and can make three decisions:

ALLOW
BLOCK
ESCALATE
Enter fullscreen mode Exit fullscreen mode

ALLOW

The action satisfies the runtime policy.

AI Agent
   ↓
Aegisora
   ↓
Execute
Enter fullscreen mode Exit fullscreen mode

BLOCK

The action violates a security policy or runtime rule.

AI Agent
   ↓
Aegisora
   ↓
STOP
Enter fullscreen mode Exit fullscreen mode

ESCALATE

The action is risky or ambiguous.

Instead of automatically allowing or rejecting it, Aegisora pauses the action and sends it for human review.

AI Agent
   ↓
Action
   ↓
Aegisora
   ↓
ESCALATE
   ↓
Human Review
   ↓
APPROVE / REJECT / EXPIRE
Enter fullscreen mode Exit fullscreen mode

The important part is when this happens.

The action is stopped before it reaches the external system.

Why Does a Third State Matter?

Imagine an AI agent wants to make a production change.

The action might be legitimate.

It might be dangerous.

Or the runtime might simply not have enough context to make a confident decision.

A binary system has only two choices:

ALLOW
   or
BLOCK
Enter fullscreen mode Exit fullscreen mode

A runtime governance layer can introduce a third:

ESCALATE
Enter fullscreen mode Exit fullscreen mode

Pause the action.

Get human authorization.

Then continue or reject it.

Bounded Autonomy

The goal isn't to remove autonomy from AI agents.

The goal is to make that autonomy bounded.

Low-risk action
      ↓
    ALLOW
      ↓
   Execute
Enter fullscreen mode Exit fullscreen mode 
Clear violation
      ↓
    BLOCK
      ↓
     Stop
Enter fullscreen mode Exit fullscreen mode 
Risky / ambiguous action
          ↓
       ESCALATE
          ↓
    Human decision
          ↓
   APPROVE / REJECT
Enter fullscreen mode Exit fullscreen mode

This creates a controlled boundary between autonomous decision-making and real-world execution.

Aegisora 3.0

Human escalation is part of a broader runtime security model:

Request
   ↓
Context
   ↓
Risk
   ↓
Policy
   ↓
Decision
   ↓
Enforcement
   ↓
Execution
   ↓
Evidence
   ↓
Audit
Enter fullscreen mode Exit fullscreen mode

The idea is simple:

Don't just secure the model. Secure what the agent is allowed to do.

Aegisora is open source.

GitHub:

https://github.com/aegisora-ai/aegisora

Website:

https://www.aegisora.org/

One Question for AI Builders

Which actions should an AI agent never perform without human approval?

I'd love to hear how other teams are handling this today.

Top comments (0)