Rethinking What You Need to Do When Your Access Keys Are Compromised

"A Note from the Author"

I work in the Technical Support division of an AWS reseller operating under the AWS Solution Provider Program in Japan. This post is written from that perspective — sitting between our customers and AWS, handling incidents day-to-day.

• Japan's IT industry has traditionally relied heavily on outsourcing to external vendors for building and managing cloud infrastructure, rather than developing in-house capabilities. This means that when a security incident like an access key leak occurs, the response often involves coordination across multiple organizations, which can slow down decision-making at a critical moment. Some of the observations in this post reflect that reality.
• Also, my English writing skills are limited, so I used GenAI (Kiro CLI) to help translate this article from the original Japanese. I hope it reads well — any awkwardness is on me, not the AI.

---

Hello, everyone.

I'm Ichino from the Technical Support team, and I'm a big fan of the AWS CLI.

Unintended access key leaks have been around for a long time, and they show no signs of stopping — whether it's accidental commits to public repositories on GitHub or exposure through server-side vulnerabilities.

Today, I want to walk through what you need to do when an access key leak happens and what kind of response is required.

The Typical Notification

When unauthorized use of an access key is suspected, AWS Support notifies the account holder by opening a support case with a message like the one below.¹

Note that AWS does not publicly disclose exactly what behaviors trigger detection (that's internal information), nor can we give a definitive answer on how quickly notifications are sent.

That said, as a colleague of mine has written, my personal impression is that detection tends to be fast when sts:GetCallerIdentity — commonly used as the first move in an attack — is called from an unusual location.
(This is just my gut feeling and not backed by any official data.)

Japanese article / 攻撃の初動としても利用される GetCallerIdentity とは何か

Example Subject Line

[CASE 123456789012345] [Action Required] Unexpected Activity Detected on your AWS Account 123456789012

Example Body

We detected potentially unexpected activity in your AWS account. This activity is related to your AWS access key(s) belonging to User(s) on the account. Please review the detailed list of access key(s) and User(s) in the existing Support Case within your AWS Console.

Refer to the user guide [1] for detailed instructions.

As a security best practice, we recommend that you enable multi-factor authentication (MFA) [2].

Step 1: If your application uses the exposed access key, you need to replace the key. To replace the key, first create a second key (at that point, both keys will be active). Then, modify your application to use the new key.

Next, disable (do not delete) the exposed key by clicking on the "Make inactive" option in the console. If there are any problems with your application, you can reactivate the exposed key. When your application is fully functional using the new key, please delete the exposed access key(s).

To delete IAM user keys, go to [3].
To delete Root user keys, go to [4].

Please note, only rotating and deleting the exposed key may not be sufficient to protect your account, continue to Step 2.

Step 2: Check your CloudTrail log for unwanted activity.

Check your account for any unwanted activity, such as creation of unapproved IAM users and/or associated passwords (login profile), access keys, policies, roles or temporary security credentials by checking your CloudTrail log, and immediately delete them.

To delete IAM users, go to [5].
To delete policies, go to [6].
To delete roles, go to [7].

Please note, deleting IAM users may impact production workloads and should be done carefully.

Step 3: Review your AWS account for any unwanted usage.
Check your account for any unwanted usage, such as EC2 instances, Lambda functions, or EC2 Spot bids by logging into your AWS Management Console and reviewing each service page. You can also do this by checking the "Bills" page in the Billing console [8].

Please note, unwanted usage can occur in any region and your console only displays one region at a time. To switch regions, use the drop-down menu in the top-right corner of the console.

Step 4: Please work with your TAM/Account Manager and respond to the existing Support Case or create a new one [9] to confirm completion of steps 1-3 and apply for a billing adjustment, if applicable. We will work with you to evaluate billing adjustments after the account is secured.

We will provide a list of potentially unexpected resources related to this event through the associated Support Case within the next 2 hour(s). Please review the resources listed for each of your services, and take action to stop, delete, or terminate any that are unwanted. The deeplinks in the attached file should take you to the exact resource page, or you can manually navigate to the resource to investigate. In addition to the resources detailed in the Support Case, it's important that you review all your services and credentials to identify and address any other unwanted usage, as described.

If you need help securing your account, let us know through the Support Case where you can request a phone call or chat session for immediate assistance. Alternatively, if you believe that your account is secured and there is no unwanted access or usage, please contact us immediately via the Support Case to confirm this in writing.

Thank you for your immediate attention to this matter.

[1] https://aws.amazon.com/premiumsupport/knowledge-center/potential-account-compromise/
[2] https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_mfa_enable.html
[3] https://console.aws.amazon.com/iam/home#users
[4] https://console.aws.amazon.com/iam/home#security_credential
[5] https://console.aws.amazon.com/iamv2/home#/users
[6] https://console.aws.amazon.com/iam/home#/policies
[7] https://console.aws.amazon.com/iam/home#/roles
[8] https://console.aws.amazon.com/billing/home#/bill
[9] https://console.aws.amazon.com/support/home?#/

Breaking Down the Required Actions

Step 1

The notification says:

Step 1: If your application uses the exposed access key, you need to replace the key. To replace the key, first create a second key (at that point both keys will be active). Then, modify your application to use the new key.

Next, disable (do not delete) the exposed key by clicking on the "Make inactive" option in the console. If there are any problems with your application, you can reactivate the exposed key. When your application is fully functional using the new key, please delete the exposed access key(s).

The wording can be a bit confusing about whether you should or shouldn't delete the key, but here's the gist:

1. Ultimately, the compromised access key must be deleted.
   • However, your organization may be legitimately using that key in applications or systems, so deleting it immediately could cause a business outage.
2. Create a new access key and update any applications or systems that were using the compromised one.
   • Make sure your systems are configured to use the new key.
3. Once that's done, deactivate (not delete) the compromised key.
   • By deactivating instead of deleting, you leave yourself a rollback path in case something breaks.
   • Verify that your applications and systems work correctly with the new key.
4. After confirming there's no business impact, delete the compromised access key.

The key was originally issued for a purpose — it's being used in some application or system. The point is: keep your legitimate workloads running while discarding the key that fell into the wrong hands.

Step 2

The notification says:

Step 2: Check your CloudTrail log for unwanted activity.

Check your account for any unwanted activity, such as creation of unapproved IAM users and/or associated passwords (login profile), access keys, policies, roles or temporary security credentials by checking your CloudTrail log, and immediately delete them.

This step asks you to investigate whether any unauthorized changes or new resources were created, primarily focusing on IAM.

Here, you'll need to use AWS CloudTrail to investigate, keeping these important points in mind:

• By default, CloudTrail records 90 days of management events as "Event history."
• Event history is recorded per AWS Region, so you need to explicitly select the region you want to examine.²
• AWS has a concept called "global service events," and IAM falls into this category.
• Most global service events are logged as having occurred in the US East (N. Virginia) region.

In other words, to review CloudTrail event history for IAM resources, you need to check the US East (N. Virginia) region. ³

The compromised access key details are typically provided in a follow-up message within the support case opened by AWS, in a format like this:

Access Key: AKIAIOSFODNN7EXAMPLE
IAMUser: foouser
Event Name: GetCallerIdentity
Event Time: January 1, 2026 00:00:00 (UTC+00:00)
IP: 203.0.113.39
IP Country/Region: US

Use the timestamp, IAM user name, and access key ID from this information as the primary filters when reviewing CloudTrail event history in the US East (N. Virginia) region.

In recent incidents, attackers commonly delete the login profile (i.e., the console password) of existing IAM users to delay discovery. Rather than continuing to use the leaked key directly, they often create new IAM users with fresh access keys, building out additional IAM resources to carry out their activities. Modifications to existing IAM roles and policies are also frequently observed.

You need to verify all of these actions and confirm that no IAM resources were created, modified, or deleted without the account owner's intent.

In short, review all actions performed using the compromised access key across IAM resources. If new IAM resources were created, follow the chain — examine the activity performed by each newly created resource, then check what those resources created in turn. Repeat this process to uncover every unauthorized IAM resource.

Investigation perspective for IAM resources
You need to follow the chain: actions taken with the leaked key AND actions taken with any newly created keys.

For a complete list of IAM actions, refer to the official documentation below. Focus your review on non-Read operations — particularly Create and Update actions:

Actions, resources, and condition keys for AWS Identity and Access Management (IAM)

AWS CloudTrail Event history page in the Management Console
IAM event history must be checked in the US East (N. Virginia) region.

For detailed instructions on using CloudTrail, see the official documentation:

Viewing recent management events with the console

Step 3

The notification says:

Step 3: Review your AWS account for any unwanted usage. Check your account for any unwanted usage, such as EC2 instances, Lambda functions, or EC2 Spot bids by logging into your AWS Management Console and reviewing each service page. You can also do this by checking the "Bills" page in the Billing console.

Please note, unwanted usage can occur in any region and your console only displays one region at a time.

Unlike Step 2, this step requires you to check a broad range of AWS services — not just IAM — for any unauthorized resource creation, and to delete them as needed.

What Should You Look For?

This is a genuinely difficult question. In a nutshell, it depends on the IAM policies attached to the compromised access key's IAM user.

For example, Amazon SES SMTP users are internally issued as IAM users with access keys (though the notification subject and handling differ when these are abused). If created following the recommended procedure, the key is issued with only the ses:SendRawEmail permission via the AmazonSesSendingAccess inline policy at the time of writing. ⁴ In that case, the attacker can only send emails via Amazon SES. While mass spam distribution is still a serious problem — it degrades your SES sender reputation — it wouldn't lead to a proliferation of resources across many AWS services.

On the other hand, if the compromised key had *FullAccess, PowerUserAccess, or AdministratorAccess attached, the scope of what an attacker can do expands dramatically.

Therefore, you need to consider what the compromised and any newly created access keys are authorized to do. Based on that, you must check all enabled regions for unauthorized resource creation by the malicious actor.

Investigation perspective for Step 3
Here too, you need to check actions from both the leaked key and any newly created keys, across all enabled regions.

Manually reviewing event history across all regions with various filter conditions through the console is extremely tedious.

If you have a CloudTrail trail configured with events stored in an S3 bucket, querying with Athena is very useful. CloudTrail Lake (though new onboarding has been announced as ending) is also helpful. However, if these services weren't enabled when the leak occurred, you'll only have the default 90 days of management event history.

Reviewing default management events through the Management Console is very difficult, so I recommend using the AWS CLI to extract the data and convert it to CSV or another format for easier analysis.

I've put together a sample shell script that makes it easier to extract event history. It works in any environment with aws cli and jq installed — feel free to use it as a reference:

GitHub Repository : research-via-cloudtrail

Step 4

Step 4: Please work with your TAM/Account Manager and respond to the existing Support Case or create a new one to confirm completion of steps 1-3 and apply for a billing adjustment, if applicable.

After completing the verification and remediation described in Steps 1–3, you can apply for a billing adjustment if needed.

If you're using our billing agency service (as is common with AWS resellers in Japan), you cannot reply to or create support cases directly. You'll need to go through us — but you can request a review for billing adjustments related to AWS charges incurred by unauthorized resource creation.

However, neither AWS nor we can guarantee that charges will be reduced. Every case is individually reviewed by the relevant AWS team, taking into account the full circumstances — what happened, the background, and what remediation was performed. The review criteria and process are not disclosed; you simply receive an approval or denial. That said, if the same type of incident has occurred before and a credit was previously granted, a second occurrence is almost always denied.

For high-value unauthorized charges, the review may take considerable time. Even in cases where a credit is ultimately approved, you may need to pay the charges upfront first.

Do not assume that AWS will bail you out if something goes wrong. Think of it as: there's a possibility of relief, but it's not guaranteed. The priority should always be preventing these incidents from happening in the first place.

Closing Thoughts

Working in the Technical Support division — positioned between our customers and AWS — we see a lot of access key compromise incidents.

When an access key is compromised, the notification generally follows the flow we've walked through here. By working through each step methodically, you can address the situation.

However, attacker techniques have become increasingly sophisticated and automated in recent years. If a compromised access key has broad permissions for resource creation, a wide variety of resources can be spun up automatically, racking up charges in the hundreds of thousands of yen (thousands of dollars) in no time.

Leak vectors include accidental commits to public repositories like GitHub, exposed .env files, and other forms of unintended disclosure. Proper key management is critical.

For use cases that traditionally required access keys — such as calling AWS APIs from outside AWS — alternatives like the aws login command and IAM Roles Anywhere are now available. With these options on the table, it's worth revisiting whether your current use of access keys is still necessary.

That said, there are still scenarios where access keys are the only option for external access. In those cases, access keys aren't inherently evil. But following best practices — regular rotation, avoiding overly broad permissions — remains essential.

I usually close these posts with "I hope this helps someone," but this time: I hope you never need this post.

Until next time.

---

1. Cases opened by AWS are sometimes referred to as "outbound cases." ↩

2. CloudTrail Event history provides a viewable, searchable, downloadable, and immutable record of the past 90 days of CloudTrail management events in an AWS Region. (Quoted from the AWS CloudTrail documentation) ↩

3. For most services, events are recorded in the Region where the action occurred. For global services such as AWS Identity and Access Management (IAM), AWS STS, and Amazon CloudFront, events are delivered to trails that include global services. Most global service events are logged as occurring in the US East (N. Virginia) Region, though some are logged in other regions such as US East (Ohio) or US West (Oregon). (Quoted from the AWS CloudTrail documentation) ↩

4. https://docs.aws.amazon.com/ses/latest/dg/smtp-credentials.html ↩