The biggest risk with an AI coding agent is not necessarily that it writes bad code. It is that it makes a reasonable decision and has enough authority to execute it where it should not.
Consider an agent asked to fix a timeout in a payment service. It traces the problem to an inefficient query and determines that the best fix requires a new database index. While investigating, it discovers that changing a shared library would improve the implementation. That requires modifying another service and changing configuration.
The technical reasoning may be correct. But the task has silently expanded from fixing application code to making decisions about databases, shared components, service ownership, and infrastructure. That is where AI agent guardrails become important.
The question is no longer only:
Can the agent determine the right technical action?
It is also:
What authority should the agent have to perform that action?
Capability Is Not Authority
We already separate these concepts for engineers. A senior engineer may understand the production database perfectly well. That does not mean every schema change should bypass review. The same principle should apply to AI coding agents.
Suppose an agent working on Service A discovers that modifying Service B produces a cleaner implementation. The agent may be correct. But several questions exist outside the code:
- Who owns Service B?
- Are its contracts allowed to change?
- Which systems depend on it?
- Does the current task authorize modifying it?
- Does another team need to review the decision?
A useful agent should be able to identify the better solution without automatically receiving authority to implement it.
The correct response might be:
Modifying Service B would simplify this implementation, but it is outside the authorized scope of this task. Approval is required before proceeding.
Stopping at an authorization boundary is not a failure of autonomy. It is correct operation within delegated authority.
Think in Terms of Capability, Authority, and Blast Radius
Traditional coding assistants had a natural execution boundary:
developer asks → assistant suggests → developer evaluates → developer executes
Agentic workflows can look very different:
inspect → plan → edit → execute → test → commit → open PR → deploy
Every additional executable step changes what a mistake can affect.
A useful conceptual model is:
Operational Risk ≈ Capability × Authority × Blast Radius × Uncertainty
This is not intended as a mathematical risk formula. It is a way to reason about agent design.
Capability is what the agent can understand and perform.
Authority is what the environment allows it to do.
Blast radius is how much can be affected by a wrong action.
Uncertainty is how incomplete the agent's understanding of the requirement or system may be.
A highly capable agent restricted to one repository and an isolated environment may be easier to delegate work to than a weaker agent with production credentials, database write access, and deployment permissions. Model capability is only one part of the system.
Guardrails Should Control Actions, Not Just Generated Code
Code review is useful when AI generates code. It is not sufficient when AI can execute actions before anyone sees the final diff. Effective guardrails for AI coding agents should operate across multiple boundaries.
Repository scope
Define where the agent can operate:
application-repo read/write
shared-library read-only
infrastructure no access
Discovery should not become authorization. An agent can inspect another component and recommend a change without automatically gaining permission to modify it.
Tool scope
Do not treat shell access as one permission.
An agent might be allowed to:
- run tests
- execute linters
- inspect Git history
- query logs
without being allowed to:
- install arbitrary dependencies
- change credentials
- push to protected branches
- execute destructive database commands
- invoke production deployment tools
The important question is not whether the agent has a tool. It is what operations that tool allows in the current task.
Environment scope
Development, CI, staging, and production should remain separate trust zones. An agent might automatically deploy to an ephemeral test environment while requiring approval for staging and having no direct production deployment permission. Permission in one environment should not imply permission in another.
Permissions Should Follow the Task
Instead of defining one permanent permission set for "the coding agent," define authority around the task.
For example:
Unit-test task:
repository read/write
test execution
Production investigation:
repository read
production logs read
metrics read
Deployment:
validated artifact read
deployment action
Giving the agent the union of those permissions permanently creates unnecessary authority.
A better question is:
What authority does this task require?
That also means escalation must be a normal part of agent behavior. An agent might generate a migration but stop before execution:
This solution requires a database schema change. The migration is ready, but execution requires additional authorization.
That preserves autonomy without turning autonomy into unrestricted execution.
Approval Should Follow Consequence
Requiring approval for every action defeats much of the purpose of an agent.
Instead, approval should reflect the consequence of the action.
| Action | Autonomous | Validation | Human Approval |
|---|---|---|---|
| Read repository | Yes | — | — |
| Run tests | Yes | Isolation | — |
| Modify code | Yes | CI/policy | Depends |
| Add dependency | Limited | Security/license | Sometimes |
| Change schema | Restricted | Migration checks | Usually |
| Deploy production | Restricted | Delivery controls | Usually |
The exact boundaries will vary.
The principle matters more:
Reading source code and changing production state should not pass through the same control mechanism.
Passing Tests Does Not Mean the Agent Was Right
There is another important boundary: validation.
Suppose the requirement says:
Customers can cancel an order before processing begins.
The agent interprets "processing" as "shipping." It implements that interpretation and generates tests for it. Every test passes. The implementation is still wrong.
If the implementation and tests originate from the same interpretation, they can reproduce the same mistake.
Validation therefore needs independent constraints such as:
- acceptance criteria
- API contracts
- architecture rules
- security policies
- contract tests
- policy engines
- independent review
Tests can prove that the implementation behaves as expected by the tests. They cannot prove that the original requirement was understood correctly.
Design for the Agent Being Wrong
The objective should not be to build an agent that never makes a wrong decision. Software engineering does not work that way.
Instead, ask:
How expensive is being wrong?
Compare:
inspect schema
→ generate migration
→ validate migration
→ request approval
with:
inspect schema
→ modify schema
→ migrate production
The reasoning might be identical. The failure consequences are not. Sandboxing, scoped credentials, isolated branches, protected environments, reversible deployments, and durable execution logs reduce the cost of a wrong decision.
That changes the goal of agent governance. The objective is not maximum restriction. It is not maximum autonomy either.
It is bounded autonomy proportional to consequence.
Strong boundaries can actually enable more autonomous execution because teams do not need to supervise every action when the environment already limits what an incorrect action can affect.
As AI agents become more capable, the most useful question may therefore be less about intelligence:
Can the agent solve this problem?
and more about authority:
If the agent makes the wrong decision with this permission, what can it affect—and how quickly can we recover?
Where would you draw that boundary for coding agents in your engineering environment?
Top comments (0)