Your AI Coding Agent Is Not Your Engineer — And That Should Terrify You

AI coding agents can write code.

They can read repositories.

They can modify dozens of files.

They can run tests.

They can open pull requests.

Some can even deploy to production.

So we started calling them software engineers.

I think that is a dangerous abstraction.

Not because AI agents are stupid.

But because they are becoming capable enough to create consequences that our current engineering controls were never designed to handle.

And this is where I think the AI engineering community is getting something fundamentally wrong.

The uncomfortable question

Imagine an AI agent receives this task:

"Upgrade the authentication system and deploy the changes."

The agent analyzes the repository.

It creates a plan.

It modifies the authentication middleware.

It updates dependencies.

It runs tests.

Everything passes.

Then it deploys.

Looks impressive.

But here is the question:

Who authorized the deployment?

Not:

Did the agent have permission to use the deployment tool?

A much harder question:

Was this specific external action actually authorized under the policy that was valid at the moment it happened?

Those are not the same thing.

---

Capability Is Not Authorization

This distinction sounds obvious in traditional software engineering.

A service account may have access to production.

That doesn't mean every operation performed using that credential is authorized.

A database user may have UPDATE permission.

That doesn't mean every update is legitimate.

Yet with AI agents, we often collapse these concepts.

We give an agent:

• repository access
• shell access
• Git access
• package managers
• cloud credentials
• deployment tools
• APIs
• browser access

And then we say:

"The agent has the tools it needs."

But having a capability is not the same as being authorized to exercise it in a particular context.

The distinction becomes critical when agents operate autonomously.

---

The Agent Can Be Correct and Still Be Wrong

This is the part I find most interesting.

Suppose the agent's reasoning is perfect.

It correctly understands the task.

It writes technically correct code.

All tests pass.

The implementation is architecturally sound.

And yet...

The action can still be unauthorized.

Why?

Because authorization is not a property of the model's reasoning.

Authorization is a property of the system.

This means we need to separate at least three things:

What the agent thinks
        ↓
What the policy allows
        ↓
What the runtime actually permits

These should not be the same component.

And the agent should not be allowed to decide all three.

---

"But We Have Human Approval"

Okay.

Let's make the scenario harder.

The agent creates a pull request.

A human reviews it.

The human clicks:

Approve

The system records:

approved = true

Is that enough?

I don't think so.

What exactly did the human approve?

Was it:

• the diff?
• the plan?
• the deployment?
• the dependency changes?
• the external API calls?
• the generated artifacts?
• the intended side effects?

And what happens if the policy changes after approval?

Consider:

10:00
Agent proposes dependency X

10:02
Human approves

10:03
Policy changes
Dependency X is now prohibited

10:04
Agent executes installation

Does the approval at 10:02 magically override the policy at 10:03?

If yes, your policy is weaker than your approval workflow.

If no, then the system needs to re-evaluate authorization at execution time.

This is why I believe:

Approval is not authorization.

Approval can be an input into authorization.

It should not automatically become authorization.

---

Audit Logs Are Not Enough Either

Another common answer is:

"Don't worry. We have audit logs."

Great.

What do they contain?

Maybe:

agent started
agent modified files
agent opened PR
human approved
deployment completed

That sounds good.

Until you ask:

Can we prove what was actually authorized at the exact moment of execution?

A useful audit system needs to answer questions like:

Who initiated the action?

Which agent identity was involved?

Which capability was requested?

Which capability was granted?

Which policy version was evaluated?

Which approval was referenced?

What payload was authorized?

What external side effect was dispatched?

Which runtime actually executed it?

What evidence proves that the runtime enforced the decision?

That is a very different thing from:

"We have logs."

An audit trail should not merely describe what happened.

It should help establish why the system was allowed to let it happen.

---

Agent Memory Is Not Evidence

This creates another uncomfortable distinction.

An AI agent might say:

"I already checked that."

Maybe it did.

Maybe it didn't.

Maybe the context was truncated.

Maybe the model forgot.

Maybe another agent performed the check.

Maybe the policy changed afterward.

Maybe the tool returned a manipulated result.

The model's memory is not a durable source of truth.

This gives us a fundamental principle:

What an agent remembers is not the same as what a system can prove.

And I think this distinction is going to become increasingly important as we move from single-agent coding assistants toward multi-agent engineering systems.

---

What Happens When Agents Disagree?

Now imagine two independent AI systems inspect the same change.

Agent A:

"This change is safe."

Agent B:

"This introduces a security risk."

Which one wins?

Today, the typical answer is:

"Ask a better model."

I'm not convinced.

Disagreement itself can be useful evidence.

Instead of hiding disagreement, an engineering system could surface:

Agent A: APPROVE
Agent B: REJECT

Confidence divergence: HIGH

Reason:
Agent B identified an authorization-path modification
that Agent A did not evaluate.

The goal isn't to blindly vote between models.

The goal is to make uncertainty observable.

That is a fundamentally different engineering philosophy.

---

The Bigger Problem: AI Agents Cross Trust Boundaries

An AI coding agent isn't just generating text.

It can cross boundaries.

Human
  ↓
AI Agent
  ↓
Repository
  ↓
Tool
  ↓
Cloud
  ↓
Production
  ↓
External users

Every transition introduces a trust question.

What did the upstream component request?

What was actually authorized?

What information was trusted?

What information was untrusted?

Can the downstream component distinguish data from instructions?

Can the request be replayed?

Can permissions be widened during the handoff?

Can a malicious tool response influence the next agent?

These aren't traditional prompt-engineering problems.

They are systems engineering problems.

---

This Is Why I Don't Think "Better Prompts" Solve AI Engineering

Prompt engineering is useful.

Better models are useful.

Better coding agents are useful.

But eventually you hit a boundary where:

The problem is no longer what the model can reason about.

The problem becomes:

What the system is allowed to do.

That requires things like:

• policy
• authorization
• capability control
• identity
• provenance
• auditability
• verification
• deterministic enforcement
• runtime boundaries
• explicit approvals
• replay protection
• versioned contracts

In other words:

engineering infrastructure for AI agents.

---

The Controversial Idea

This is where NAEOS comes in.

NAEOS — Nusantara AI Engineering Operating System — is based on a simple thesis:

AI coding agents should not be treated as autonomous engineers. They should operate inside an engineering system that constrains, verifies, observes, and governs their actions.

That sounds restrictive.

It is.

That's the point.

We don't need to make agents less capable.

We need to make the system around them more capable.

The agent should be able to reason.

But the system should decide what is authorized.

The agent can propose.

The system can verify.

The human can approve.

The runtime can enforce.

The audit system can preserve evidence.

Those responsibilities should not collapse into a single AI model.

---

NAEOS Is Not Another Coding Agent

This is another distinction worth making.

NAEOS isn't trying to compete with:

• Claude Code
• OpenAI Codex
• GitHub Copilot
• Cursor
• Gemini CLI
• Cline
• Roo Code
• Windsurf
• other coding agents

The premise is almost the opposite.

Use whichever agent you want.

The engineering system should sit around the agent.

Think of it as an architectural layer:

┌─────────────────────────────┐
│          Human              │
├─────────────────────────────┤
│ Governance & Approval       │
├─────────────────────────────┤
│ Policy & Authorization      │
├─────────────────────────────┤
│ Verification & Audit        │
├─────────────────────────────┤
│ NAEOS Runtime / Control     │
├─────────────────────────────┤
│ AI Coding Agent             │
├─────────────────────────────┤
│ Tools / APIs / Infrastructure│
└─────────────────────────────┘

The agent is powerful.

But it isn't sovereign.

---

"Isn't This Just Enterprise Overengineering?"

Maybe.

If your AI agent only generates a Python function on your laptop, you probably don't need this.

If an agent can:

• modify production infrastructure
• change authentication
• access customer data
• trigger financial operations
• send external communications
• deploy software
• modify security policies
• operate other agents

then the equation changes.

The larger the change surface and blast radius, the stronger the verification and authorization requirements should become.

Not simply:

small diff = low risk
large diff = high risk

A two-line change to an authentication policy can be more dangerous than a 500-line refactor.

Risk should depend on:

change surface
×
blast radius
×
change characteristics

Not just lines of code.

---

The Future Problem Nobody Wants to Talk About

We're going to build systems where one AI agent delegates to another.

Something like:

Planner Agent
     ↓
Coding Agent
     ↓
Testing Agent
     ↓
Security Agent
     ↓
Deployment Agent
     ↓
Production

Now ask:

What exactly is transferred between them?

A message?

A plan?

A capability?

A permission?

A claim?

A tool result?

An instruction?

This is where protocol-neutral handoffs become important.

A handoff should have explicit semantics around things like:

initiator
requested capability
authorized capability set
payload contract
untrusted inputs
allowed downstream actions
policy version
contract version
canonicalization version
provenance
expiry
replay state

Otherwise we're effectively saying:

"Here is some text another AI generated. Please trust it."

That doesn't sound like engineering.

It sounds like distributed prompt passing.

---

Maybe We Are Building the Wrong Layer First

The AI industry is obsessed with making agents more autonomous.

More tools.

More memory.

More context.

More actions.

More autonomy.

But autonomy without control is not maturity.

It's simply a larger blast radius.

I think the next generation of AI engineering infrastructure will care less about:

"How do we make the agent do more?"

and more about:

"How do we prove what the agent was allowed to do?"

That is a much harder problem.

And probably a much more important one.

---

The Question I Want Engineers to Argue About

I don't want everyone to agree with NAEOS.

Actually, I would prefer the opposite.

Tell me where this model is wrong.

Should an AI agent ever be allowed to authorize its own actions?

Is human approval sufficient?

Should authorization be evaluated only once, or again immediately before execution?

Can an audit trail prove anything meaningful without binding policy, identity, payload, and execution evidence together?

Should disagreement between independent AI systems be treated as a failure — or as an observability signal?

And perhaps the biggest question:

At what point does an AI coding agent stop being a coding assistant and become a production system that requires the same engineering controls as any other distributed system?

I don't think we've answered that yet.

But we're already deploying the agents.

That's what worries me.

---

NAEOS

NAEOS is an open-source attempt to explore this problem as an AI Engineering Operating System rather than another AI coding tool.

The goal isn't to make AI less autonomous.

The goal is to make autonomy governable, verifiable, auditable, and enforceable.

Because eventually, the most important question about an AI system won't be:

"How intelligent is the agent?"

It will be:

"Can we prove why it was allowed to do what it did?"

That is the conversation I think AI engineering needs to have next.