DEV Community

Cover image for Controlled Autonomy: Designing Safe AI Agents for Software Engineering
bayu priatno
bayu priatno

Posted on

Controlled Autonomy: Designing Safe AI Agents for Software Engineering

Why the Future of AI Engineering Is Not Full Autonomy

The conversation around AI agents often focuses on one question:

How autonomous can an AI agent become?

Can it write an entire application?

Can it deploy infrastructure?

Can it fix production incidents?

Can it manage a software project without human intervention?

These are interesting questions.

But from an engineering perspective, there is another question that may be more important:

How should autonomy be controlled?

Because autonomy is not binary.

An agent does not need to be either:

fully autonomous

or

fully supervised.

There is a much more useful model:

Controlled autonomy.

Give an AI agent as much freedom as possible inside clearly defined boundaries, while progressively increasing controls as the potential impact, uncertainty, and blast radius increase.

This is an important direction for NAEOS — Nusantara AI Engineering Operating System.


1. Autonomy Changes the Engineering Model

Traditional software automation follows predictable workflows.

For example:

Commit
  ↓
Build
  ↓
Test
  ↓
Deploy
Enter fullscreen mode Exit fullscreen mode

The system follows predefined instructions.

AI agents introduce something different.

An agent can:

Observe
  ↓
Reason
  ↓
Plan
  ↓
Choose an action
  ↓
Execute
  ↓
Observe the result
  ↓
Adapt
Enter fullscreen mode Exit fullscreen mode

The system is no longer purely deterministic.

There is an intelligent decision-maker inside the workflow.

This creates enormous potential.

But it also introduces a new engineering requirement:

the decision-making process needs boundaries.


2. Capability Is Not Authority

This is one of the most important principles in AI-native engineering.

An agent may have the technical capability to perform an action.

That does not mean it has the authority to perform that action.

Consider:

Agent
  ↓
Has deployment tool
Enter fullscreen mode Exit fullscreen mode

This means:

The agent is technically capable of deploying.

It does not mean:

The agent is authorized to deploy to production.

The authorization decision should be separate.

A better architecture is:

Agent
  ↓
Intent
  ↓
Policy
  ↓
Capability
  ↓
Resource Scope
  ↓
Risk
  ↓
Validation
  ↓
Decision
Enter fullscreen mode Exit fullscreen mode

The resulting decision can be:

ALLOW
DENY
ESCALATE
Enter fullscreen mode Exit fullscreen mode

This separation is fundamental.


3. The Agent Should Not Be Its Own Security Boundary

An AI model can interpret instructions.

But interpretation is not enforcement.

Consider a system prompt:

Never access production credentials.
Never deploy directly to production.
Always ask for approval before modifying infrastructure.
Enter fullscreen mode Exit fullscreen mode

This is useful.

But it should not be the only protection.

The actual engineering system should enforce the boundary.

For example:

AI Agent
   │
   │ request:
   │ production.deploy
   ▼
Policy Engine
   │
   ├── identity
   ├── capability
   ├── resource
   ├── environment
   ├── risk
   └── validation
   │
   ▼
Decision
   │
   ├── ALLOW
   ├── DENY
   └── ESCALATE
Enter fullscreen mode Exit fullscreen mode

This creates an important architectural principle:

Prompts express intent and context. External controls enforce authority.


4. Why Binary Autonomy Is the Wrong Model

It is tempting to classify agents as:

Autonomous
Enter fullscreen mode Exit fullscreen mode

or:

Not Autonomous
Enter fullscreen mode Exit fullscreen mode

But real engineering systems are more nuanced.

Consider these tasks:

Task Potential Impact
Read source code Low
Generate documentation Low
Create unit tests Low
Modify application code Medium
Change API contract Medium
Modify architecture High
Change security policy High
Modify production infrastructure Critical
Delete production data Critical

Clearly, these actions should not have identical autonomy levels.

Therefore, autonomy should be evaluated at the action level, not merely at the agent level.


5. Risk-Based Autonomy

A better model is:

Autonomy ∝ Trust × Validation × Context
Autonomy ↓ as Impact × Uncertainty ↑
Enter fullscreen mode Exit fullscreen mode

In practical terms:

Low Risk
    ↓
High Autonomy

Medium Risk
    ↓
Agent + Validation

High Risk
    ↓
Agent + Validation + Review

Critical Risk
    ↓
Agent + Validation + Human Authorization
Enter fullscreen mode Exit fullscreen mode

This allows AI agents to move quickly without giving them unrestricted authority.


6. The Risk Context

Every consequential action should be evaluated against a specific context.

A useful model is:

Identity
+
Intent
+
Capability
+
Resource
+
Environment
+
Risk
+
Policy
+
Validation
Enter fullscreen mode Exit fullscreen mode

For example:

Agent:
deployment-agent

Intent:
Deploy payment-service

Capability:
production.deploy

Resource:
payment-service

Environment:
production

Risk:
high

Policy:
production-deployment-v4

Validation:
passed
Enter fullscreen mode Exit fullscreen mode

The policy engine can then determine:

Decision:
ESCALATE

Reason:
Human approval required for high-risk production deployment.
Enter fullscreen mode Exit fullscreen mode

The decision is explicit.

It is also auditable.


7. Blast Radius Matters

Risk should not be determined only by the type of action.

The potential blast radius matters.

Changing one development configuration file might have almost no impact.

Changing a shared production service could affect thousands of users.

Therefore:

Risk = f(
  Action,
  Resource,
  Environment,
  Scope,
  Reversibility,
  Dependency Impact
)
Enter fullscreen mode Exit fullscreen mode

A useful risk model considers:

Action

What is the agent trying to do?

Resource

What system or data will be affected?

Environment

Development, staging, or production?

Scope

One file or an entire infrastructure cluster?

Reversibility

Can the change easily be rolled back?

Dependency Impact

How many systems depend on the affected resource?


8. Reversibility Should Influence Autonomy

A useful engineering principle is:

The more reversible an action is, the more autonomy can reasonably be granted.

For example:

Create temporary test file
        ↓
Highly reversible
        ↓
High autonomy
Enter fullscreen mode Exit fullscreen mode

Compared with:

Delete production database
        ↓
Potentially irreversible
        ↓
Minimal autonomy
Enter fullscreen mode Exit fullscreen mode

This provides another dimension for the policy engine.

A risk engine could consider:

Impact
+
Reversibility
+
Confidence
+
Scope
Enter fullscreen mode Exit fullscreen mode

before determining the required approval level.


9. Confidence Is Not Enough

AI agents often provide confidence signals.

But model confidence should not be treated as authorization.

An agent might say:

"I am 98% confident this migration is safe."

That does not mean the migration should automatically execute.

Why?

Because:

confidence is a model property.

Authorization is an organizational policy decision.

These are fundamentally different concepts.

A better model is:

Model Confidence
       +
Validation Evidence
       +
Policy
       +
Risk
       +
Resource Scope
       ↓
Authorization Decision
Enter fullscreen mode Exit fullscreen mode

Confidence can contribute to the decision.

It should not define the decision by itself.


10. Evidence-Based Autonomy

The stronger an agent's evidence, the more autonomy can potentially be granted.

For example:

Agent proposes change
        ↓
Unit Tests
        ↓
Integration Tests
        ↓
Security Scan
        ↓
Static Analysis
        ↓
Architecture Validation
        ↓
Deployment Simulation
        ↓
Policy Evaluation
Enter fullscreen mode Exit fullscreen mode

Each successful validation reduces uncertainty.

This suggests a useful principle:

Autonomy should increase as validated evidence increases.

The agent earns autonomy through verification.


11. The Validation Pipeline

Consider a production database migration.

The agent generates the migration.

Instead of immediately executing it:

Agent
  ↓
Migration
  ↓
Schema Validation
  ↓
Compatibility Check
  ↓
Data Safety Analysis
  ↓
Test Environment
  ↓
Integration Tests
  ↓
Rollback Verification
  ↓
Policy Evaluation
  ↓
Human Approval
  ↓
Production
Enter fullscreen mode Exit fullscreen mode

This creates multiple safety boundaries.

The agent remains productive.

The organization remains in control.


12. Separation of Reasoning and Authority

This principle deserves special attention.

An agent can reason:

"The service requires a database migration."

It can propose:

"Add column X."

It can implement:

"Create migration Y."

It can validate:

"All tests passed."

But none of these automatically grant authority to execute the change.

The architecture should remain:

Reasoning
   ↓
Proposal
   ↓
Validation
   ↓
Authorization
   ↓
Execution
Enter fullscreen mode Exit fullscreen mode

not:

Reasoning
   ↓
Execution
Enter fullscreen mode Exit fullscreen mode

This separation creates a critical security boundary.


13. Human Approval Should Be Risk-Based

Human approval is valuable.

But requiring approval for every action would make autonomous systems inefficient.

Imagine requiring approval for:

Run unit tests
Enter fullscreen mode Exit fullscreen mode

or:

Format source code
Enter fullscreen mode Exit fullscreen mode

This would add unnecessary friction.

Instead, approval should be triggered by risk.

For example:

Low Risk
→ Automatic

Medium Risk
→ Automatic after validation

High Risk
→ Review required

Critical Risk
→ Explicit authorization required
Enter fullscreen mode Exit fullscreen mode

This allows humans to focus on consequential decisions.


14. Approval Is Also a Policy Decision

Human approval should not simply mean:

Someone clicked a button.

The approval event should contain context.

For example:

Change:
Payment service migration

Agent:
migration-agent-03

Risk:
High

Affected Resources:
payment-db

Validation:
Passed

Rollback:
Verified

Policy:
database-production-v3

Approver:
Authorized engineering role

Decision:
Approved
Enter fullscreen mode Exit fullscreen mode

This creates meaningful governance.


15. Policy Drift

Controlled autonomy introduces another challenge:

policy drift.

A policy can become outdated.

For example:

Policy v1
  ↓
Policy v2
  ↓
Policy v3
Enter fullscreen mode Exit fullscreen mode

Over time, the organization changes.

New infrastructure appears.

New security threats emerge.

New regulatory requirements arrive.

New AI capabilities become available.

If policies do not evolve, the control plane may enforce obsolete rules.

But uncontrolled policy changes are also dangerous.

Therefore, policy itself must be engineered.


16. Policies Need a Lifecycle

A mature policy lifecycle could look like:

Draft
  ↓
Validation
  ↓
Review
  ↓
Approval
  ↓
Published
  ↓
Active
  ↓
Deprecated
  ↓
Archived
Enter fullscreen mode Exit fullscreen mode

Policies should also have:

Version
Owner
Scope
Effective Date
Expiration
Dependencies
Impact
Audit History
Enter fullscreen mode Exit fullscreen mode

This allows organizations to understand not only:

Which policy exists?

but:

Which policy was active when this action occurred?


17. Policy Testing

Policies should be tested like software.

Consider:

Policy:
Production deployment requires approval.
Enter fullscreen mode Exit fullscreen mode

The organization should test scenarios such as:

Scenario 1:
Staging deployment
→ ALLOW

Scenario 2:
Production deployment without approval
→ DENY

Scenario 3:
Production deployment with approval
→ ALLOW

Scenario 4:
Untrusted agent
→ DENY
Enter fullscreen mode Exit fullscreen mode

This creates policy-as-code testing.

A policy change should not be considered safe simply because the YAML or JSON is syntactically valid.

It should be behaviorally validated.


18. Policy Observability

A policy engine should expose its decisions.

For example:

Policy Decision

Agent:
coding-agent-17

Action:
infrastructure.modify

Resource:
production-cluster

Result:
DENY

Policy:
production-infrastructure-v5

Reason:
Agent lacks production infrastructure capability.
Enter fullscreen mode Exit fullscreen mode

This is extremely useful for:

  • debugging
  • security investigations
  • compliance
  • engineering analytics
  • policy optimization

It also allows teams to discover unexpected patterns.

For example:

Why are 40% of deployment requests being denied?

That might indicate:

  • agents lack required permissions
  • workflows are incorrectly designed
  • policies are too restrictive
  • tasks are incorrectly classified
  • agents are misunderstanding requirements

Observability makes these problems visible.


19. The Policy Feedback Loop

Policy decisions themselves become engineering data.

Consider:

Policy Decision
      ↓
Outcome
      ↓
Observation
      ↓
Analysis
      ↓
Policy Improvement
      ↓
New Policy Version
Enter fullscreen mode Exit fullscreen mode

This creates a controlled feedback loop.

But again:

the system should not silently rewrite its own governance.

Policy changes should remain subject to governance.


20. Multi-Agent Systems Make This Harder

Now consider a multi-agent environment.

Planner Agent
Coder Agent
Testing Agent
Security Agent
Deployment Agent
Enter fullscreen mode Exit fullscreen mode

Each agent may have different capabilities.

For example:

Planner
→ read-only

Coder
→ repository.write

Testing
→ tests.execute

Security
→ security.scan

Deployment
→ staging.deploy
Enter fullscreen mode Exit fullscreen mode

The system needs to coordinate these capabilities.

The control plane becomes the shared authority.

                 Control Plane
                      │
        ┌─────────────┼─────────────┐
        ▼             ▼             ▼
     Planner        Coder       Deployment
      Agent         Agent          Agent
        │             │             │
        └─────────────┼─────────────┘
                      ▼
                 Shared Policy
Enter fullscreen mode Exit fullscreen mode

This prevents each agent from becoming its own security island.


21. Agent Identity Matters

Every agent should have an explicit identity.

Not merely:

AI Agent
Enter fullscreen mode Exit fullscreen mode

but:

agent:
  id: deployment-agent-03
  role: deployment
  owner: platform-engineering
  version: 2.4.1
Enter fullscreen mode Exit fullscreen mode

Identity can then be used in authorization.

For example:

deployment-agent-03
Enter fullscreen mode Exit fullscreen mode

may be authorized to:

staging.deploy
Enter fullscreen mode Exit fullscreen mode

but not:

production.database.write
Enter fullscreen mode Exit fullscreen mode

This provides accountability.


22. Resource Scope Matters

Permissions should also be scoped to resources.

Instead of:

database.write
Enter fullscreen mode Exit fullscreen mode

use:

database.write
  scope:
    service: payment
    environment: staging
Enter fullscreen mode Exit fullscreen mode

This follows the principle of least privilege.

The agent receives only the authority necessary for the task.


23. Environment Is Part of the Policy

The same action can have completely different risk depending on the environment.

For example:

repository.write
Enter fullscreen mode Exit fullscreen mode

in development:

Low Risk
Enter fullscreen mode Exit fullscreen mode

while:

repository.write
Enter fullscreen mode Exit fullscreen mode

against a production configuration repository:

High Risk
Enter fullscreen mode Exit fullscreen mode

Therefore, environment must be part of policy evaluation.

Action
+
Resource
+
Environment
Enter fullscreen mode Exit fullscreen mode

is more meaningful than action alone.


24. The AI Engineering Control Loop

Putting these ideas together:

┌─────────────────────────────┐
│         HUMAN INTENT        │
└──────────────┬──────────────┘
               ↓
┌─────────────────────────────┐
│       CONTEXT / KNOWLEDGE   │
└──────────────┬──────────────┘
               ↓
┌─────────────────────────────┐
│        AI REASONING         │
└──────────────┬──────────────┘
               ↓
┌─────────────────────────────┐
│           PLAN              │
└──────────────┬──────────────┘
               ↓
┌─────────────────────────────┐
│       POLICY EVALUATION     │
│                             │
│ Identity                    │
│ Capability                  │
│ Resource                    │
│ Environment                 │
│ Risk                        │
└──────────────┬──────────────┘
               ↓
        ┌──────┼───────┐
        ▼      ▼       ▼
      ALLOW   DENY   ESCALATE
        │              │
        │              ▼
        │       HUMAN APPROVAL
        │              │
        └──────┬───────┘
               ▼
┌─────────────────────────────┐
│          EXECUTION          │
└──────────────┬──────────────┘
               ↓
┌─────────────────────────────┐
│ VALIDATION / OBSERVABILITY  │
└──────────────┬──────────────┘
               ↓
┌─────────────────────────────┐
│       AUDIT / MEMORY        │
└──────────────┬──────────────┘
               ↓
           FEEDBACK
               │
               └──────────────► KNOWLEDGE
Enter fullscreen mode Exit fullscreen mode

This is the architecture of controlled autonomy.


25. NAEOS and Controlled Autonomy

This model aligns naturally with the broader architecture of NAEOS.

NAEOS can be understood as the engineering layer connecting:

Governance
      ↓
Constitution
      ↓
Knowledge
      ↓
Policy
      ↓
Risk
      ↓
Workflow
      ↓
AI Agents
      ↓
Execution
      ↓
Validation
      ↓
Audit
      ↓
Memory
Enter fullscreen mode Exit fullscreen mode

Each layer has a different responsibility.

The AI agent provides intelligence.

The policy system provides boundaries.

The knowledge system provides context.

The workflow system provides structure.

The validation system provides evidence.

The governance system provides authority.


26. The Agent Should Earn Trust

One interesting consequence of this architecture is that agent trust can become measurable.

Instead of saying:

"This agent is trusted."

we can define evidence:

Successful Tasks
Validation Pass Rate
Policy Violations
Rollback Rate
Security Findings
Human Overrides
Incident History
Change Risk
Enter fullscreen mode Exit fullscreen mode

This could contribute to an agent trust profile.

For example:

Agent Reliability Profile

Validation Success: 99.4%
Policy Violations: 0
Rollback Rate: 0.2%
Security Findings: 0
Human Override: 1.1%
Enter fullscreen mode Exit fullscreen mode

Trust should not automatically grant unrestricted authority.

But it can inform controlled autonomy.


27. Autonomy Levels Can Evolve

An agent could begin with:

Level 1
Read + Analyze
Enter fullscreen mode Exit fullscreen mode

After sufficient validation:

Level 2
Propose Changes
Enter fullscreen mode Exit fullscreen mode

Then:

Level 3
Execute Low-Risk Changes
Enter fullscreen mode Exit fullscreen mode

And potentially:

Level 4
Execute Controlled Production Changes
Enter fullscreen mode Exit fullscreen mode

Each transition should require explicit policy.

This creates:

Progressive autonomy.

The agent does not receive unlimited authority on day one.

It earns broader operational scope through evidence and governance.


28. The Principle of Least Autonomy

Security engineering traditionally emphasizes:

Least privilege.

AI-native engineering may need a complementary principle:

Least necessary autonomy.

Give an agent enough autonomy to complete the task efficiently.

Not more.

For example:

Task:
Generate API documentation.

Required:
repository.read

Not required:
repository.write
production.deploy
database.write
infrastructure.modify
Enter fullscreen mode Exit fullscreen mode

This reduces unnecessary attack surface.


29. Why This Matters Beyond Security

Controlled autonomy is not only a security problem.

It also improves:

Reliability

Agents operate within validated workflows.

Consistency

Different agents follow the same policies.

Compliance

Actions become auditable.

Maintainability

Policies are centralized.

Portability

Agents can change without rewriting governance.

Scalability

Organizations can operate more agents without losing control.

Trust

Humans can understand why actions were allowed or denied.


30. The Future of AI Engineering

The future may not look like:

Human
  ↓
AI
  ↓
Everything is automated
Enter fullscreen mode Exit fullscreen mode

A more realistic model is:

Human Intent
      ↓
Engineering System
      ↓
AI Agents
      ↓
Controlled Execution
      ↓
Validation
      ↓
Human Governance
Enter fullscreen mode Exit fullscreen mode

The goal is not to remove humans from engineering.

It is to move humans toward the decisions where human judgment provides the greatest value.

AI handles more execution.

Engineering systems handle more control.

Humans retain authority over consequential decisions.


31. The Bigger Principle

We started with a simple question:

How autonomous should an AI agent be?

The better answer is:

As autonomous as its context, evidence, capabilities, and policy boundaries allow.

This leads to a broader principle:

More Capability
      ↓
More Potential Impact
      ↓
More Need for Governance
Enter fullscreen mode Exit fullscreen mode

Therefore:

Autonomy
+
Policy
+
Validation
+
Observability
+
Human Authority
Enter fullscreen mode Exit fullscreen mode

must evolve together.


Conclusion

AI agents will become more capable.

They will write more code.

They will operate more tools.

They will manage increasingly complex workflows.

Eventually, some agents may perform tasks that today require entire engineering teams.

But the answer is not unrestricted autonomy.

The answer is controlled autonomy.

Agents should be able to reason freely within defined boundaries.

They should have explicit identities.

They should receive only the capabilities they need.

Their actions should be evaluated against resource scope, environment, risk, policy, and validation.

High-impact actions should be escalated.

Consequential authority should remain outside the model.

And every important decision should be observable and auditable.

The architecture can be summarized in one line:

Agent → Intent → Policy → Capability → Resource → Risk → Validation → Decision.

This is the direction NAEOS is exploring.

Not a world where humans simply hand control to AI.

But a world where humans and AI agents operate together inside an engineering system designed for controlled autonomy.

Because the future of AI engineering should not be:

maximum autonomy.

It should be:

maximum useful autonomy within minimum necessary risk.

Top comments (0)