It’s 2:00 AM. A CloudWatch alert triggers. The on-call engineer wakes up, opens five console tabs, and begins reconstructing the timeline of events from scratch.
The AWS DevOps Agent is designed to handle that initial hour of painstaking analysis. AWS refers to it as a "frontier agent"- one that not only resolves incidents but prevents them before they occur. It analyzes CloudWatch metrics, CloudTrail data, VPC Flow Logs, RDS Performance Insights, and IaC stacks, subsequently providing a root-cause analysis and an action plan for remediation.
I was curious to see how this works in practice, so I attended an AWS DevOps Agent workshop focused on incident investigation. The workshop consisted of four labs: one on system setup and three involving specially prepared environments with pre-injected faults. I attended this engaging event - organized by the AWS User Group 3City community in collaboration with AWS-some time ago.
The AWS DevOps Agent console: "Resolve. Prevent. Improve."
| Scenario | What I broke | What the agent concluded | Time to root cause |
|---|---|---|---|
| EC2 / Auto Scaling | Ran a CPU-burn script on the ASG through SSM | The CPU alarm had no actions attached and the ASG had no scaling policy, so the group couldn't scale out | ~10 min |
| Network security | Port scan + HTTP flood from my laptop | A security group open to 0.0.0.0/0 on 22/80/443/8080 since stack creation. It also found that 3 of 4 earlier alarm firings were false positives caused by S3 return traffic |
~20–25 min |
| RDS MySQL | 140 connections + a terrible CROSS JOIN … ORDER BY RAND()
|
A runaway workload from the bastion host using the admin user. It also spotted broken CloudWatch log export
|
~16 min |
In all three labs the agent did more than restate the alarm. It found the configuration gap underneath it, tied the fix back to the CloudFormation stack so the fix wouldn't become drift, and wrote a mitigation plan in five phases: prepare → pre-validate → apply → post-validate → rollback.
Part 1: Setting it up takes about 5 minutes
You create an Agent Space. This is the boundary for what the agent can see and do. In the setup wizard you:
- name the space and optionally choose a response language (Russian, Japanese, Portuguese and others are available)
- let it auto-create a read role for your primary AWS account
- optionally turn on Agent Actions, a separate role for write actions. Every change needs your explicit approval.
- turn on the web app, which is the operator UI your on-call team works in
Once the space exists, you can add more capabilities: secondary AWS accounts, Azure, GitHub/GitLab/Azure DevOps pipelines, Datadog/Dynatrace/New Relic/Splunk/Grafana telemetry, Slack/ServiceNow/PagerDuty, any MCP server, and even remote agents over the A2A protocol.
👉 Full walkthrough: Part 1: Setting up AWS DevOps Agent
Part 2: "Why didn't my Auto Scaling Group scale?"
A Lambda used SSM Run Command to push one t3.micro instance to 100% CPU. The CloudWatch alarm went red, and the group stayed at one instance.
About 1m48s in, the agent had its first finding: ScalingPolicies: [] and AlarmActions: []. The alarm worked. It just wasn't wired to anything.
It also:
- ruled out CPU credit throttling, since
CPUCreditBalancestayed at the t3.micro maximum of 288 - showed the spike wasn't user traffic, because the ALB stayed at 0–1 req/min
- found the exact SSM Run Command ID that started the CPU burn
- said plainly what it couldn't do: CloudTrail was blocked in the workshop account, so it couldn't attribute the IAM principal
👉 Part 2: EC2 CPU spike investigation
Part 3: Investigating a network attack using VPC Flow Logs
I ran a script from my laptop that port-scanned the web server, enumerated admin pages, tried SSH logins and flooded it with HTTP requests.
The agent rated it Severity: high. Using VPC Flow Logs, it identified the source IP, listed the scanned ports, and confirmed that 2,764 accepted packets on port 80 were the real attack.
The part that stood out was historical context I never asked for. Looking at earlier firings of the same alarm, it worked out that 3 of the 4 were benign S3 return traffic, which means the alarm itself needs tuning.
👉 Part 3: Network security investigation
Part 4: RDS connection exhaustion, and where the agent contradicted itself
A script on the bastion host opened about 140 connections against a max_connections=150 limit and pinned CPU with Cartesian joins.
The investigation was excellent. It split the load using Performance Insights into 108 sessions sitting in SLEEP() and 38 AAS of the CROSS JOIN query, and traced all of it to a single host and DB user.
Then I asked follow-up questions in chat and found two contradictions between the chat and the mitigation plan. The two disagreed on whether 150 is above or below the default max_connections, and on whether wait_timeout would help. Part 4 covers which one was right.
👉 Part 4: RDS performance investigation
What I liked
- It answers why, not only what. None of the root causes was simply "CPU was high". Each was a missing link: no scaling policy, a world-open security group, a host with no connection limit.
- It knows about IaC. Every mitigation plan warned that an API-only fix would drift from the CloudFormation stack, and every one included a code change specification to update the template as well.
- Mitigation plans include rollback. Every plan had prepare, pre-validate, apply, post-validate and rollback steps with copy-paste CLI commands.
- It reports its gaps. Each investigation had an "Investigation gaps" section listing what it couldn't access (CloudTrail, GuardDuty, RDS log files).
- It respects guardrails. Its safe-deployment policy refused to auto-run destructive security group rule removals and marked them for a human to run.
- Agent-ready spec. You can copy the remediation as a spec for a coding agent to implement.
What to watch out for
- Verify the chat answers against the investigation. The structured findings were more careful than the conversational answers (see Part 4).
- IAM scope determines what it can find. The agent can only investigate what its role and your SCPs allow. In the workshop account, CloudTrail lookups, GuardDuty and RDS log downloads were blocked, and those gaps showed up in the results.
- Investigations take time. Expect 8–16 minutes. That's quick compared with a human starting at 2 AM, but it isn't instant.
- Ignore the "Unable to describe support level" error. The workshop says it's expected.
- AWS DevOps Agent Pricing
Should you try it?
I'd put it in the on-call path as the first responder that hands a human a pre-built timeline, not as an autonomous fixer… yet
Screenshots are from my own run of the AWS DevOps Agent workshop on 3 October 2026. All resources were in a temporary workshop account that has since been deleted.
Top comments (0)