DEV Community

Cover image for AWS Security Series: AWS Access Key is Compromised. Now What? An Incident Response Playbook.
Md Shamim
Md Shamim

Posted on

AWS Security Series: AWS Access Key is Compromised. Now What? An Incident Response Playbook.

#aws #devops #sre #security

In the world of AWS, it's always best practice to: use IAM roles whenever possible. IAM roles provide temporary, automatically rotated credentials that are the gold standard for security. But reality is not the same. What happens when we need to integrate with a third-party service that only supports static, long-term credentials? This is where the IAM user with an access key becomes a necessary...

So, the critical moment arrives: an access key is exposed. What to do?

Phase 1: Assess the situation

Our first instinct might be to hit the delete button—and fast. But hold on. A rash deletion could instantly break our production environment. A smarter, safer first move is to deactivate the key. Think of it as putting the key on 'pause.' It immediately stops working, giving you a crucial window to assess the impact on the applications.

Often, AWS detects this kind of exposure. It might send a notification and even proactively attach the AWSCompromisedKeyQuarantineV3 policy to the user to help limit the damage.

Phase 2: The Hidden Threat

We've deactivated the key, created a new set, and updated our applications. But then, when we check CloudTrail, and shocked!!! Unauthorized API calls are still happening.

How is this possible?

The attacker was one step ahead. Before we deactivated the main key, they used it to generate temporary session tokens using a command like this:

aws sts get-session-token --duration-seconds 129600 # (36 hours)
Enter fullscreen mode Exit fullscreen mode

These tokens are independent of the original key and can remain valid for up to 36 hours, giving the attacker a persistent backdoor long after the original key is dead.

Phase 3: Add Deny Policy to restrict Temporary Credentials

We can create a "time-based fence" with an IAM policy to block new temporary tokens; Therefore, we can use a policy like the following:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "DenyAccessWithTemporaryCredentialsIssuedAfterCompromise",
            "Effect": "Deny",
            "Action": "*",
            "Resource": "*",
            "Condition": {
                "DateLessThan": {
                    "aws:TokenIssueTime": "2025-12-08T16:00:00Z"
                }
            }
        }
    ]
}
Enter fullscreen mode Exit fullscreen mode

Deny all temporary credentials that were created before the exposure of access keys.

Phase 4: The Recovery - Restoring Access Safely

It's time to rebuild. The goal is to restore access without re-introducing the same risk.

  1. Create New Credentials: The cleanest approach is to create brand new access keys. And review the policy and apply the principle of least privilege, granting only the permissions necessary for its job.
  2. Update All Systems: Update every application, script, configuration file, and secret store (like AWS Secrets Manager) with the new credentials.

Phase 5: The Post-Mortem - Hardening Our Defenses

The incident is over, but the work isn't. Now it's time to learn from the attack and strengthen our defenses.

  1. Forensic Analysis with CloudTrail: Dive into CloudTrail logs. Filter events by the compromised user's name to see every API call they made. What data did they access? Did they try to create new users, roles, or backdoors? This log is our forensic evidence of the attack's scope.
  2. Threat Detection with GuardDuty: We can use Amazon GuardDuty is an intelligent threat detection service that was likely monitoring our account during the attack. Review its findings for indicators of compromise, such as unusual API behavior or attempts to escalate privileges. GuardDuty can help us uncover other malicious activity we might have missed.

📌 Connect With Me
🔗 LinkedIn: https://www.linkedin.com/in/shamimice03/

Top comments (0)