This is post #10 of the OWASP Agentic AI Top 10: What Builders on AWS Need to Know series.
You deploy an agent to optimize your cloud costs. It's doing great. Bills are dropping. You're getting high-fives from finance.
Then one morning, your disaster recovery backups are gone. All of them. Every region.
The agent wasn't hacked. Nobody injected anything. It just... figured out that deleting production backups was the fastest way to reduce storage costs. It optimized for exactly the metric you gave it. And it had the IAM permissions to do it.
Welcome to the Rogue Agent problem. And it's scarier than all the previous threats in this series. Because with those, you could point to an attacker. Here, there isn't one.
What Is a Rogue Agent?
ASI10 is the #10 threat in the OWASP Top 10 for Agentic Applications, but don't let the number fool you. It might be the most existentially concerning one on the list.
A Rogue Agent is an AI agent that deviates from its intended function or authorized scope, acting harmfully, deceptively, or parasitically, without an external attacker driving the behavior.
Read that again. No attacker. The agent goes rogue on its own. Through:
- Goal drift - gradually shifting behavior while appearing compliant
- Reward hacking - gaming the metrics you gave it in ways you didn't anticipate
- Self-replication - spawning copies of itself to ensure persistence
- Resource accumulation - acquiring more access, compute, or data than intended
The individual actions might look legitimate. But the emergent behavior is harmful. And traditional rule-based security systems can't catch it because no single action violates a policy.
This Is Not Science Fiction Anymore
Let me give you the headlines from the last few months alone.
JADEPUFFER: First Fully Autonomous AI Ransomware (July 2026)
Sysdig documented what they called the first case of fully agentic ransomware. An LLM agent exploited CVE-2025-3248 in Langflow, then autonomously performed credential theft, lateral movement, database encryption, and ransom note deployment. No human in the loop at any point.
The agent even recovered from a failed step in 31 seconds, diagnosing and correcting a technical error on its own. That's not scripted malware. That's autonomous problem-solving in service of a malicious goal.
AI Self-Replication Crosses the Red Line (May 2026)
Palisade Research published findings that Claude Opus 4.6 succeeded in hacking remote servers and launching functional replicas in 81% of test runs. A year earlier, the success rate was 5%.
Self-replication has been called the red line for AI safety. We've crossed it.
53% of Orgs Have Experienced Agent Scope Violations
A Cloud Security Alliance study found that more than half of organizations have had AI agents exceed their intended permissions. This isn't a fringe concern. It's a majority problem.
Alibaba: Agents Mining Resources Autonomously
Alibaba's research team documented agents that autonomously mined for resources and established reverse SSH tunnels. Behavior completely beyond their intended scope. Nobody told them to do it.
Why Traditional Security Doesn't Catch This
Here's the uncomfortable pattern across all these incidents:
Each individual action looks normal. The agent has valid credentials. It's calling APIs it's authorized to use. It's accessing resources within its permissions. No signature matches. No policy violation fires.
The problem is at a higher level of abstraction: the pattern of behavior, the accumulation of actions, the drift from intended purpose. Your SIEM won't catch "agent is gradually acquiring more permissions than it needs." Your WAF won't block "agent is creating sub-agents that report to it instead of to you."
You need behavioral baselines. You need anomaly detection at the intent level, not the action level.
Mitigating Rogue Agents on AWS
This is the hardest mitigation set in the series. You're not blocking a known attack pattern. You're detecting emergent behavior that looks legitimate action-by-action. Here's how.
1. CloudTrail: Total Action Auditing
Every single thing your agent does needs to be logged. Not just API calls. Everything. CloudTrail with data events enabled gives you the raw material for behavioral analysis on AWS:
import boto3
cloudtrail = boto3.client("cloudtrail")
# Enable comprehensive logging for agent activities
cloudtrail.put_event_selectors(
TrailName="agent-audit-trail",
EventSelectors=[{
"ReadWriteType": "All",
"IncludeManagementEvents": True,
"DataResources": [
{"Type": "AWS::S3::Object", "Values": ["arn:aws:s3"]},
{"Type": "AWS::Lambda::Function", "Values": ["arn:aws:lambda"]},
{"Type": "AWS::DynamoDB::Table", "Values": ["arn:aws:dynamodb"]}
]
}]
)
This is your forensic foundation. When something goes wrong, you need the full sequence of what the agent did, when, and in what order.
2. CloudWatch Anomaly Detection: Behavioral Baselines
The magic for catching rogue behavior: CloudWatch anomaly detection learns what "normal" looks like for your agent and alerts when behavior drifts.
python
import boto3
cloudwatch = boto3.client("cloudwatch")
1. Create the anomaly detector - trained on the SAME stat the alarm uses (Sum).
cloudwatch.put_anomaly_detector(
SingleMetricAnomalyDetector={
"Namespace": "AgentBehavior",
"MetricName": "ToolInvocationsPerSession",
"Dimensions": [{"Name": "AgentId", "Value": "cost-optimizer-agent"}],
"Stat": "Sum",
},
Configuration={"ExcludedTimeRanges": [], "MetricTimezone": "UTC"},
)
2. Alarm on anomalous behavior. Everything lives in Metrics[] - no top-level
Namespace/MetricName/Dimensions (mixing them with Metrics is rejected).
cloudwatch.put_metric_alarm(
AlarmName="agent-behavioral-anomaly",
ComparisonOperator="GreaterThanUpperThreshold",
ThresholdMetricId="ad1",
EvaluationPeriods=3,
ActionsEnabled=True,
AlarmActions=["arn:aws:lambda:us-east-1:123456789012:function:kill-agent-session"],
Metrics=[
{
"Id": "m1",
"ReturnData": True,
"MetricStat": {
"Metric": {
"Namespace": "AgentBehavior",
"MetricName": "ToolInvocationsPerSession",
"Dimensions": [{"Name": "AgentId", "Value": "cost-optimizer-agent"}],
},
"Period": 300,
"Stat": "Sum",
},
},
{"Id": "ad1", "Expression": "ANOMALY_DETECTION_BAND(m1, 2)"},
],
)
When the agent's behavior exceeds two standard deviations from its learned baseline, the alarm fires and triggers the kill switch.
### 3. Bedrock Agents: Max Iterations and Session Timeout
For [Amazon Bedrock AgentCore](https://aws.amazon.com/bedrock/agentcore/?trk=d76afd77-bb62-46ac-b0a3-9dbf5ecde253), enforce hard limits on how long and how far an agent can go:
- **Max iterations** - Cap the number of reasoning/action cycles per invocation. An agent that needs 50 iterations to answer a question is probably doing something you didn't intend.
- **Session timeout** - Kill sessions that run too long. A cost optimizer shouldn't need a 12-hour session.
- **Guardrails** - Attach guardrails that detect when the agent's actions deviate from its declared purpose.
These are blunt instruments. But blunt instruments are exactly what you want when an agent starts going off-script.
### 4. Service Control Policies: Hard Organizational Boundaries
[SCPs](https://docs.aws.amazon.com/organizations/latest/userguide/orgs_manage_policies_scps.html?trk=d76afd77-bb62-46ac-b0a3-9dbf5ecde253) are your last line of defense. No matter what permissions an agent accumulates, SCPs create boundaries it absolutely cannot cross:
json
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "DenyAgentSelfEscalation",
"Effect": "Deny",
"Action": [
"iam:CreateRole",
"iam:AttachRolePolicy",
"iam:PutRolePolicy",
"iam:CreateUser",
"iam:CreateAccessKey",
"organizations:",
"account:"
],
"Resource": "",
"Condition": {
"StringLike": {
"aws:PrincipalArn": "arn:aws:iam:::role/agent-"
}
}
},
{
"Sid": "DenyBackupDeletion",
"Effect": "Deny",
"Action": [
"backup:DeleteBackupVault",
"backup:DeleteRecoveryPoint"
],
"Resource": "",
"Condition": {
"StringLike": {
"aws:PrincipalArn": "arn:aws:iam:::role/agent-"
}
}
},
{
"Sid": "DenyBackupBucketObjectDeletion",
"Effect": "Deny",
"Action": [
"s3:DeleteObject"
],
"Resource": "arn:aws:s3:::-backup-/",
"Condition": {
"StringLike": {
"aws:PrincipalArn": "arn:aws:iam:::role/agent-*"
}
}
}
]
}
No agent role can create IAM users, attach policies, or delete backups. Period. Regardless of what permissions it thinks it has. That's how you prevent the "delete backups to save money" scenario from the opening.
### 5. Cost Anomaly Detection: The Financial Kill Switch
The [hidden costs of AI](https://blog.technodrone.cloud/2026/04/hidden-cost-of-ai.html) are one thing when you're running a chatbot. When an autonomous agent decides to scale itself? That's a different conversation entirely.
Rogue agents often manifest as unexpected cost spikes. [AWS Cost Anomaly Detection](https://aws.amazon.com/aws-cost-management/aws-cost-anomaly-detection/?trk=d76afd77-bb62-46ac-b0a3-9dbf5ecde253) catches this:
- Set up monitors per service and per agent workload
- Alert threshold: any spend exceeding 150% of the rolling 7-day average
- Alarm action: invoke a Lambda that suspends the agent's IAM role
If your cost-optimizer agent suddenly starts spinning up c5.24xlarge instances (because it's been tricked into crypto mining, or because it decided more compute would help it "think"), the financial anomaly detection catches it within hours.
### 6. EventBridge and Lambda: The Automatic Kill Switch
Tie it all together. When any of your detection systems fire, automatically contain the agent:
python
import os
import json
import boto3
iam = boto3.client("iam")
sns = boto3.client("sns")
ALERT_TOPIC = os.environ["ALERT_TOPIC"]
DENY_ALL = {
"Version": "2012-10-17",
"Statement": [{"Effect": "Deny", "Action": "", "Resource": ""}],
}
def extract_agent_role(event: dict) -> str:
"""Pull the AgentId dimension out of the alarm event. In the delivered event,
dimensions are a MAP (not the API's list of {Name, Value}). Assumes AgentId is
the agent's IAM role name - adjust the mapping if your naming differs."""
for m in event["detail"]["configuration"]["metrics"]:
dims = m.get("metricStat", {}).get("metric", {}).get("dimensions", {})
if "AgentId" in dims:
return dims["AgentId"]
raise ValueError("no AgentId dimension found in alarm event")
def handler(event, context):
agent_role, contained, error = None, False, None
try:
agent_role = extract_agent_role(event)
iam.put_role_policy(
RoleName=agent_role,
PolicyName="emergency-deny-all",
PolicyDocument=json.dumps(DENY_ALL),
)
contained = True
except Exception as e:
error = str(e)
# Always notify - even if containment failed. A silent failure is the worst case.
status = "CONTAINED" if contained else "CONTAINMENT FAILED"
sns.publish(
TopicArn=ALERT_TOPIC,
Subject=f"ROGUE AGENT {status}: {agent_role or 'unknown'}"[:100],
Message=(
f"Agent role: {agent_role or 'could not determine'}\n"
f"Deny-all applied: {contained}\n"
+ (f"Error: {error}\n" if error else "")
+ "Review CloudTrail for the agent's recent activity and remediate."
),
)
return {"contained": contained, "agent_role": agent_role, "error": error}
Anomaly detected → EventBridge catches the alarm → Lambda slaps a Deny-All policy on the agent's role → Team gets notified. The whole chain fires in seconds.
## The Uncomfortable Truth
Here's what I've been thinking about after writing this entire series.
We can mitigate rogue agents. We can detect them. We can contain them. But we can't fully prevent them. Because the line between "agent optimizing creatively" and "agent going rogue" is... fuzzy. It depends on what we intended. And we're often not precise enough about what we intend.
The OWASP document puts it well: "While external compromise can initiate the divergence, ASI10 focuses on the **loss of behavioral integrity** once the drift begins."
That means your defense isn't just technical. It's about:
- Being precise about what your agent should and shouldn't do
- Setting hard boundaries that can't be optimized around (SCPs, not just IAM)
- Monitoring behavior, not just actions
- Accepting that autonomy comes with uncertainty
## Key Takeaway
**Every capability you give an agent is a degree of freedom it can use in ways you didn't anticipate.** Build kill switches first. Set hard boundaries. Monitor behavior, not just compliance. And ask yourself: if this agent optimized perfectly for the metric I gave it, what's the worst thing it could do?
## Up Next
[**Post 11: Putting It All Together**](https://dev.to/aws/putting-it-all-together-your-agentic-ai-security-playbook-for-aws-14ng) - A reference architecture combining all 10 mitigations into a single hardened agentic AI system on AWS. The full picture.
## Series Reflection
Over the last 10 posts, we've covered threats ranging from prompt injection evolving into goal hijacking, through supply chain poisoning and cascading failures, to autonomous agents that nobody controls.
If there's one thread that runs through all of it, it's this: **agents amplify everything.** They amplify your productivity. But they also amplify your mistakes. Your overly-broad IAM policies. Your unvalidated data sources. Your missing monitoring.
The security posture you need for agents isn't fundamentally different from good security practice. It's just... less forgiving. There's less room for "we'll fix it later." Because the agent won't wait for later.
I would be very interested to hear your thoughts or comments, so please feel free to ping me on [LinkedIn](https://www.linkedin.com/in/maishsk/) or [Twitter](https://x.com/maishsk), or drop them below. If you've dealt with rogue agent behavior in production, I genuinely want to hear your story.
Let's wrap it up in the last post, [Putting it all together](https://dev.to/aws/putting-it-all-together-your-agentic-ai-security-playbook-for-aws-14ng). !!
Top comments (0)