AWS DevOps Agent in Practice: Setup, Investigations, and Integrations

The worst part of an incident usually isn't fixing it — it's the scramble beforehand, piecing together what's even happening across a dozen tools. That's exactly the gap AWS DevOps Agent is built to close and you already understand the problem AWS DevOps Agent exists to solve. AWS DevOps Agent is AWS's autonomous agent for operations — it investigates incidents the moment they happen, learns your infrastructure well enough to reason about blast radius, and connects into the tools your team already lives in, like Slack and Jira.

In this post we go hands-on: standing one up, running an investigation both on demand and automatically.

Foundation of AWS DevOps Agent

Everything AWS DevOps Agent does happens inside a container called an Agent Space — the boundary that defines exactly which AWS accounts, integrations, and users the agent is allowed to touch. Understanding this boundary is the first thing to get right, because it's the security model the rest of the product builds on.

On top of that boundary sits a deliberately split interface:

• For Administrators : To do configuration for agents via console or IaC.
• For users : Web App to see investigation, generate reports, create custom agents etc.

Setting Up AWS DevOps Agent

Getting a first Agent Space running comes down to three steps :

1. Create an Agent Space in the AWS Management Console — Go to AWS Console → AWS DevOps Agent → Create Agent Space. We can also create Devops agent via CLI, IaC - Cloudformation, Terraform.

While creating agent space, you need to mention:
- Name: As per you naming convention
- IAM Role for Agent Space: IAM Role used by agent space access your AWS resources. I will be using default as part of this blog.
- IAM Role for DevOps Agent WebApp: This is IAM role used by WebApp to access Agent Space.
- Encryption Key: AWS managed or CMK. You can mention this only during creation, later cannot be configured.

For a demo or test environment, just use the default IAM role — the AWS-managed AIDevOpsAgentAccessPolicy gives read-only access out of the box, no extra configuration needed. For production, don't carry that forward as-is: watch which additional permissions your test environment's investigations actually reach for, then add only those as scoped inline policies on a custom role — resource-restricted, not account-wide.

Once all field done, hit create.

Accessing DevOps Agent WebApp

On creation of agent, you should able to access it under Access tab :

DevOps agent can integrate with

• Communication channel - Slack, PagerDuty, ServiceNow
• Telemetry tools - DataDog, NewRelic, Splunk, Grafana, Dynatrace
• Source code - Github, Gitlab, AzureDevOps
• Add 3rd party MCP servers, Remote agent with A2A support
• Webhooks

More on this integration in next blog.

On-Demand Investigation

You can start an investigation manually any time from the Incident Response tab of your Agent Space web app — either with free-form text describing what you want investigated, or by picking one of three pre-configured starting points:

• Latest alarm: investigates your most recently triggered alarm, analyzing underlying metrics and logs to determine root cause
• AWS Platform health: Investigate if there are any planned changes from AWS side.

I used prompts : Check for stale/unused resources, it returns nice summary of idle resources

DEMO #1 : Investigate EC2 Alarms

Part of this demo I have EC2 instance created and intentionally ran process to spike CPU (with stress-ng), let's check if DevOps agent able to investigate that.

As we can see above DevOps agent able to detect, it was an intentionally CPU spike. It does it but accessing metrics, cloudtrail logs and co-relating it. Under rootcause tab in incident investigation, you will able to find what are the gaps in investigation , in this example, since we don't have cloduwatch logs for ec2, gap was devops agent was not able detect exact process for CPU spike

Automated Investigation

Rather than waiting for someone to notice and manually start an investigation, AWS DevOps Agent can kick one off itself the moment something fires. So instead of reactive approach, let's go for pro-active approach.

Currently, we cannot target AWS DevOps agent as an action for CloudWatch alarm. So we will do trick here, we will be using lambda as target. Please find below flow we will be configuring.

Webhook Integration - DevOps Agent

To configure generic webhook, follow below steps

1. Navigate to Agent Space → Capabilities tab → Webhook section → Configure

1. Generic webhooks: choose Generate webhook, then pick an auth type — HMAC (signs each request, verified via the x-amzn-event-signature header) or API key (sent as Authorization: Bearer ). I will be using HMAC authentication.

1. Copy the endpoint URL and store the secret/key securely — it's shown only once and can't be retrieved again. We will be configuring this as part of lambda.

Configuring Lambda

Next step is read payload from Cloudwatch alarm and send it to AWS DevOps agent. Below part of lambda code show, forming payload and send payload to devops agent.

• I have configured 3 environment variables for tha lambda DEVOPS_AGENT_WEBHOOK_SECRET, DEVOPS_AGENT_WEBHOOK_URL & SERVICE_NAME.
• IAM Permission: Use AWS Managed permission AWSLambdaBasicExecutionRole.

Please note that, its not recommended hard code DEVOPS_AGENT_WEBHOOK_SECRET even as environment variable, you should use SSM Parameter Secure String or SecreteManager and configure path of SSM as environment variable.

def _build_payload(event):
    alarm_data = event.get("alarmData", {})
    alarm_name = alarm_data.get("alarmName", "Unknown Alarm")
    alarm_arn = event.get("alarmArn", "")
    account_id = event.get("accountId", "unknown")
    region = event.get("region", "")

    state_block = alarm_data.get("state", {})
    state = state_block.get("value", "ALARM")
    reason = state_block.get("reason", "")
    prev_state = alarm_data.get("previousState", {}).get("value", "")

    config = alarm_data.get("configuration", {})
    desc_text = config.get("description", "")

    namespace, metric_name = _extract_metric(alarm_data)
    dimensions = _extract_dimensions(alarm_data)

    title = f"[{state}] {alarm_name}"
    if region:
        title += f" ({region})"

    desc_parts = []
    if desc_text:
        desc_parts.append(desc_text)
    if reason:
        desc_parts.append(reason)
    description = "\n".join(desc_parts)

    return {
        "eventType": "incident",
        "incidentId": _incident_id(alarm_name, account_id),
        "action": _STATE_TO_ACTION.get(state, "updated"),
        "priority": _STATE_TO_PRIORITY.get(state, "MEDIUM"),
        "title": title,
        "description": description,
        "timestamp": _iso_ts(),
        "service": SERVICE_NAME,
        "data": {
            "source": "aws.cloudwatch",
            "alarmName": alarm_name,
            "alarmArn": alarm_arn,
            "state": state,
            "previousState": prev_state,
            "reason": reason,
            "accountId": account_id,
            "region": region,
            "namespace": namespace,
            "metricName": metric_name,
            "dimensions": dimensions,
        },
    }

def _post_webhook(payload, request_id):
    timestamp = _iso_ts()
    payload_str = json.dumps(payload, separators=(",", ":"), ensure_ascii=False)
    signature = _sign(WEBHOOK_SECRET, timestamp, payload_str)

    logger.info("[%s] Sending to DevOps Agent. preview=%s",
        request_id, payload_str[:1200])

    req = urllib.request.Request(
        url=WEBHOOK_URL,
        data=payload_str.encode("utf-8"),
        method="POST",
        headers={
            "Content-Type": "application/json",
            "x-amzn-event-timestamp": timestamp,
            "x-amzn-event-signature": signature,
        },
    )
    with urllib.request.urlopen(req, timeout=20) as resp:
        body = resp.read().decode("utf-8", errors="replace")
        logger.info("[%s] Webhook response status=%s body=%s",
            request_id, resp.status, body[:1000])
        return {"status": resp.status, "body": body[:1000]}

Configure CloudWatch Alarm Action

Once AWS lambda is in place, you can edit existing CloduWatch alarm with action.

DEMO 2: Automated Investigation

I have Webapp- URL-shortner behind CloudFront. Part of this webapp, lambda process requests, to intentional failure in this app, I have set lambda concurrency to 0 which disable lambda execution and clodufront throws 5xx errors.

I have created CloudFront5xx alarm which gets triggered when website throws 5xx errors.

From below screenshot, we see when alarm gets triggered respective lambda which sent payload to devops agent

On checking further in AWS DevOps Agent Space, we can see agent started with investigation. Open DevOps agent WebApp, under Incident you should able to see investigation has started.

We can also see how devops agent approached investigation in timeline e.g.

In few minutes, DevOps agent came up with root cause of this issue.

From the above screenshot we able to see accuracy of DevOps agent is ~100%.

For most of the scenarios I tried, accuracy for DevOps agent is always > 90% and this is quite impressive.

Key Takeaways

• Everything runs inside an Agent Space, isolated across account access, user access, data, and chat history — that's the foundation everything else builds on.
• Setting up AWS DevOps Agent
• You can investigate on demand from the web app, or let the agent trigger automatically from Cloudwatch alarm via Webhooks

Hopefully this gives you a solid picture of how to configure AWS DevOps Agent for CloudWatch alarms.

Next up in this series: integration with Slack, Jira, and GitHub.