DEV Community: Atsushi Suzuki

Taking the KCSA Exam on My Washing Machine (And Passing!)

Atsushi Suzuki — Thu, 19 Mar 2026 01:47:58 +0000

After grabbing my CKA on Jan 11, KCNA on Jan 23, and CKAD on Jan 26, I kept the momentum going and took the KCSA on February 6th!

Briefly, the KCSA (Kubernetes and Cloud Native Security Associate) is a multiple-choice certification that tests your fundamental knowledge of Kubernetes security.

Kubernetes and Cloud Native Security Associate

Exam Results

I passed with a score of 81% (passing score is 75%).

Usually, I rent a private room in a coworking space to take these exams, but this time I wanted to save some money and decided to take it at home. After desperately searching for a completely empty space in my house that met the proctoring requirements, I ended up putting my MacBook on top of my washing machine!

Since the KCSA is a multiple-choice exam without practical labs, I figured, "It doesn't matter if the laptop wobbles a bit!"

Date: Feb 6, 2026, 10:00 AM~
Location: My laundry room
Device: MacBook Air (2022, M2, 13.6")

Impressions & Difficulty

By the time I finished my first pass through the questions, I had flagged about 40 out of 60 questions for review. Honestly, I thought, "I'm completely doomed..."

Moreover, since it's the only one of the five core Kubernetes certifications not offered in Japanese, I stumbled over English reading comprehension in a few places. However, there was plenty of time left on the clock, so I read the questions very carefully, tried to visualize the scenarios, and managed to work my way through.

Looking back, if I had taken the CKS first, I would have had a clearer image of the scenarios and found some questions much easier. I managed to pass this time, but if you want to solidify your knowledge and guarantee a pass, I honestly recommend taking the CKS before the KCSA.

Study Materials I Used

I used KodeKloud and a Kubernetes Security KCSA Mock Exam Simulator.
In total, I spent just under 20 hours studying.

KodeKloud

I watched all the video lectures on KodeKloud and looped through their 3 Mock Exams about four times.

Reflecting on it now, while concepts like STRIDE and the 4C's are specific to KCSA, topics like securityContext, Admission Controllers, Istio, and Falco heavily overlap with CKS. If I had done hands-on CKS prep beforehand, I could have studied much more efficiently.

Kubernetes and Cloud Native Security Associate (KCSA)

GitHub Mock Exams

From reading other people's review articles, I felt that KodeKloud alone wouldn't give me enough practice, so I additionally tackled a web-based mock exam repository containing about 300 questions.

You can launch an app to practice, but since it lacked a feature to flag questions I couldn't solve, I cloned the repository locally and opened it in my IDE (Cursor). My workflow was simple: I manually deleted the questions I solved correctly, and asked the Cursor AI chat for explanations on the ones I didn't understand.

I felt these questions were closer to the actual exam content than the KodeKloud mock exams, so I highly recommend doing them!

Kubernetes Security KCSA Mock Exam Simulator

What's Next?

Now, all that's left is to get my CKS and achieve the ultimate title of Kubestronaut! Wish me luck!

CKAD is Easier Than CKA: How I Passed with 20 Hours of Study

Atsushi Suzuki — Tue, 03 Feb 2026 02:40:15 +0000

In January 2026, I achieved the "Kubernetes Triple Crown" (CKA, KCNA, and CKAD) in a single month. This post is a report of my experience passing the CKAD exam.

Certified Kubernetes Application Developer (CKAD)

Exam Result

I passed with a score of 81% (passing score is 66%).

Exam Date: January 28, 2026
Environment: Coworking space private room
Kubernetes Version: v1.34
Device: MacBook Air (2022, M2, 13.6-inch)

Thoughts on Difficulty

Overall, the CKAD felt significantly easier than the CKA. I didn't encounter any questions that left me completely stumped. The main reason for this lower difficulty, in my opinion, was the lack of complex troubleshooting or tricky Helm-related questions.

While I found the actual CKA exam to be more difficult than the Killer Shell simulator, the CKAD was much easier than its Killer Shell counterpart. I never felt rushed for time and even had 30 minutes left to review my answers. Although I solved 10 mock exams as preparation, I feel that if you have already passed the CKA, you might not need to go that far.

Study Materials Used

I used KodeKloud and Killer Shell. Total study time was approximately 20 hours.

KodeKloud

Since the exam scope overlaps significantly with the CKA, I focused only on the CKAD-specific topics. I ran through the Lightning Labs three times and the Mock Exams once (as they were relatively straightforward).

Observability (Readiness Probes, Liveness Probes, Logging, Monitoring)
POD Design (Deployment Strategy)
Lightning Labs (1, 2)
Mock Exams (1, 2)

Certified Kubernetes Application Developer (CKAD) - KodeKloud

As the initial course only had two mock exams, which felt insufficient, I tackled an additional 8 mock exams from the series below, repeating any questions I missed three times.

Ultimate CKAD Mock Exam Series

Killer Shell

Just like the CKA, scheduling your exam gives you two sessions for the Killer Shell simulator. It is essential to use this because it replicates the actual remote desktop environment.

Since I was using a MacBook, the keyboard shortcuts for copying and pasting change within the remote desktop. Practicing these in the simulator was vital. Note that each session expires 36 hours after activation. If you want to review after the session expires, I recommend saving the answer pages as PDF or HTML. I only used one of the two sessions this time (saving the other in case I failed).

Killer Shell

Tips

1. Master the CKAD-specific domains

Make sure to read the documentation for areas not covered in CKA, such as Job and CronJob. You should also be familiar with specific options, such as creating a one-off Job from an existing CronJob:
kubectl create job test-job --from=cronjob/a-cronjob

2. Memorize the Documentation URL

The browser doesn't automatically open to the docs when the exam starts. Be ready to quickly navigate to https://kubernetes.io/docs.

3. Practice Copy-Pasting in Remote Desktop

Your speed in copying and pasting can literally determine whether you pass or fail. Practice these shortcuts in Killer Shell:

Copy (Terminal): Ctrl + Shift + C
Paste (Terminal): Ctrl + Shift + V

Closing Thoughts

The Linux Foundation was offering a 35% discount coupon, so I’m planning to take the KCSA in February. The CKS is still a bit expensive, so I'll save that for when I have a bit more budget!

How I Passed the KCNA in Just 3 Hours (After Getting CKA)

Atsushi Suzuki — Mon, 26 Jan 2026 05:00:57 +0000

After starting my Kubernetes journey in November and earning CKA (Certified Kubernetes Administrator) on January 11, 2026, I decided to keep the momentum going.

CKA (Certified Kubernetes Administrator) Exam Report 2026: Don’t Rely on Old Guides (Mastering the Post-2025 Revision)

Since I had some extra time in January after recovering from a cold, I challenged myself to take the KCNA (Kubernetes and Cloud Native Associate) exam.

Kubernetes and Cloud Native Associate (KCNA)

Exam Results

I passed with a score of 87 % (Passing score: 75 %).
The exam is 90 minutes long, but I finished everything, including a full review, in just 30 minutes.

Exam Date: Jan 21, 2026
Location: Private room in a co-working space
Device: MacBook Air (2022, M2, 13.6")

Reflections and Difficulty

Coming off a fresh CKA win, I thought this would be a breeze. However, after the first round of 60 questions, I had flagged 27 questions for review. For a moment, I actually felt I might fail!

During the second round, I carefully switched between English and Japanese to grasp the nuances. Eventually, I narrowed down the "completely unsure" questions to about 9.

While my CKA knowledge covered the core components perfectly, I struggled with broader CNCF ecosystem questions—such as "What is the official name of this specific CNCF event?" These caught me off guard.

Preparation Materials

I spent a total of about 3 hours studying using these resources.

1. KodeKloud

I completed three Mock Exams (2 rounds each).
The questions were quite straightforward—simpler than the actual exam—so it took less than 2 hours to fly through them.
Kubernetes and Cloud-Native Associate (KCNA)

2. Kindle Practice Exams

Feeling a bit under-prepared with just KodeKloud, I used a practice book (Japanese) available on Kindle unlimited. This felt closer to the actual exam difficulty, though the questions were still relatively simple.
2週間で合格！Kubernetes and Cloud Native Associate (KCNA)

Key Study Tips

Based on my experience, here is what you should focus on.

Master Pod Scheduling

There were many questions about NodeSelector, NodeAffinity, PodAffinity, and Taints/Tolerations. Make sure you can clearly distinguish their use cases.

Understand the CNCF Ecosystem

Unlike the CKA, the KCNA tests your knowledge of CNCF project names and event details. Don't skip this part!

GitOps Concepts

I knew of ArgoCD, but I wasn't familiar with its specific operational details. It’s worth reviewing GitOps workflows.

Monitoring & Observability Tools

Be clear on the boundaries between tools like Prometheus, Fluentd, and Jaeger. Know exactly which tool handles which part of the observability stack.

What's Next?

I’m riding this wave of momentum and will be taking the CKAD (Certified Kubernetes Application Developer) on January 26. Wish me luck!

CKA (Certified Kubernetes Administrator) Exam Report 2026: Don’t Rely on Old Guides (Mastering the Post-2025 Revision)

Atsushi Suzuki — Tue, 13 Jan 2026 04:58:27 +0000

Despite being active as an AWS Community Builder (Containers) and a Docker Captain, for some reason, I had never really touched Kubernetes before. So, starting in November 2025, I began studying for 1-2 hours on weekdays outside of work.

I am the type of person who finds it easier to solidify foundational knowledge when I have a specific goal like a certification, so I decided to aim for the Certified Kubernetes Administrator (CKA) first.

Certified Kubernetes Administrator (CKA)

Why I am writing this article

If you search for "CKA Pass" you will find many experience reports. However, there are still not many articles written after the exam scope revision in February 2025. Having actually taken the exam, I felt that the "sense of difficulty" and "essential techniques" mentioned in older reports are no longer entirely applicable. I wrote this article as a reference for those who are planning to take the exam in the future.

Exam Results

I passed with a score of 72% (passing score is 66%). It was my first time taking a hands-on practical exam, and compared to multiple-choice exams, I felt constantly rushed—I couldn't stop sweating. I didn't finish everything within the time limit and didn't have time to go back to the troubleshooting questions I had flagged.

Exam Date: 2026/01/11 13:30~
Location: Private room in a co-working space
Kubernetes Version: v1.34
Device: MacBook Air (2022, M2, 13.6-inch)

Impressions of the New Exam Scope (Post-Feb 2025) and Difficulty

To be honest, I felt it was significantly more difficult than the image I had from older experience reports. My personal impression was that it felt about as challenging as the "Killer Shell" simulator (mentioned later).

Below is the revised exam scope, and I felt that questions related to the newly added content made up about half of the exam.

CKA Program Changes - Feb 2025

While I cannot provide specific details about the questions, you will likely struggle if your understanding of the following newly added topics is weak:

Helm
Kustomize
Gateway API (Gateway, HttpRoute, etc.)
Network Policy
CRDs (Custom Resource Definitions)
Extension interfaces (CNI, CSI, CRI, etc.)

Learning Resources Used

I used Udemy, KodeKloud, and Killer Shell.
Many reports from before the revision said that Udemy and Killer Shell were enough, but my personal feeling is that I was only able to put up a fight because I went as far as doing the additional labs on KodeKloud.

Udemy

This is the standard Udemy course recommended in almost every report. I bought it during one of their frequent sales.

Certified Kubernetes Administrator (CKA) Practice Exam Tests

The course consists of video lectures and practice tests (hosted on a separate service called KodeKloud). Since my Kubernetes knowledge was literally zero, I studied each component through the videos before attempting the practice tests.

I went through the lectures once and repeated the practice questions I didn't understand multiple times (up to 4 times). For my weak areas (Helm, Kustomize, Gateway API, CRD), I re-watched the lectures several times.

Once I could solve over 80% of the practice questions, I tackled the three Mock Exams in the final section until I could solve them perfectly (up to 4 rounds for some).

For explanations I didn't quite understand, I asked AI for help.

KodeKloud

This is the KodeKloud platform included with the Udemy course:
Certified Kubernetes Administrator (CKA)

Since having only three Mock Exams felt a bit uncertain, I paid for the following course to solve five additional Mock Exams:
Ultimate Certified Kubernetes Administrator (CKA) Mock Exam Series

Killer Shell

When you register for the exam, you get two sessions of the "Killer Shell" exam simulator.
It uses the same Remote Desktop environment as the actual exam, so you should absolutely use it to get used to the interface. In my case, since I took the exam on a MacBook, the shortcut keys for commands change in the Remote Desktop, so I was able to practice copy-pasting in the simulator.

Note that the simulator access is provided twice, and each session expires after 36 hours. If you want to review after it expires, it’s better to save the answer pages as PDF or HTML.

Killer Shell

Tips

No need to add settings to `.bashrc`

Common tips in older reports, such as "aliases for switching namespaces," are no longer necessary. This is because the current exam format involves using ssh to switch between different environments for each task.

Memorize the Documentation URL

The browser doesn't automatically open to the documentation page when the exam starts, so make sure you can quickly navigate to https://kubernetes.io/docs.

Learn Copy-Paste Shortcuts for Remote Desktop

Whether or not you can copy-paste quickly can make or break your pass/fail result. Practice the operations on Killer Shell.

Copy (Terminal): Ctrl + Shift + C
Paste (Terminal): Ctrl + Shift + V

Prepare for Applied Questions

Don't just memorize each component; imagine how they combine with others. Simply "memorizing Helm commands" might not be enough for some of the harder questions.

Helm + CRD
Migration from Ingress → Gateway + HttpRoute, etc.

Exam Environment

The conditions for the exam environment are strict: "a private space without noise," "no objects on the desk," etc. Unfortunately, I didn't have such an environment at home, so I rented a private room in a co-working space.

Interaction with the proctor is via chat, so there is no verbal conversation. However, it seems that even slight noise will be pointed out, so choosing a private room is the safe bet.

Instructions from the proctor come in your native language (Japanese in my case), but they seem to be using machine translation, as some instructions were a bit confusing. I assumed the proctor was non-Japanese, so I replied in English (just basic things like "OK," "thanks," etc.).

Following the proctor's instructions, I used the webcam to show the room (ceiling, floor, desk, walls), both ears, both wrists, and my powered-off smartphone. It took about 15 minutes to actually start the exam.

Also, I didn't use the external monitor at the co-working space, but in retrospect, having a larger workspace would have made things easier, so I probably should have used it.

During the exam, I didn't really feel the presence of the proctor, but when I was stuck on a problem and touched my chin, a chat message arrived saying, "Please refrain from gestures that cover your face."

Final Thoughts

Obtaining the CKA was quite a challenge, but I plan to take the CKAD while I still have this momentum. The second half of 2025 was quite rough at my company and I felt zero personal growth, so I want to make up for that in 2026.

I Automated My Air Conditioner with Kubernetes (kind + CronJob + SwitchBot)

Atsushi Suzuki — Fri, 26 Dec 2025 08:06:13 +0000

Even though I’m an AWS Community Builder in the Containers category, I had honestly never touched Kubernetes before. I felt a bit guilty about that, so I recently decided to properly start learning it.

After studying for about a month for the CKA (Certified Kubernetes Administrator) exam, I became familiar with the basic Kubernetes resources and kubectl commands. At that point, I wanted to actually try using Kubernetes resources end-to-end in a real project.

Around the same time, I realized that the SwitchBot Hub Mini I had bought during Black Friday might be perfect for a small hands-on experiment, so I decided to play around with it.

SwitchBot Hub Mini

As a first step, I built a Kubernetes cluster locally on my Mac using kind, and created a CronJob that turns on the air conditioner in my bedroom every day at 7:00 PM(JST).

The total implementation time was about 30 minutes.

SwitchBot Hub Mini Setup

Prerequisites

The SwitchBot app is installed
A SwitchBot account has been created
The air conditioner remote has already been registered in the app

Getting the Token / Secret Key / Device ID

Open Profile > Preferences > About in the SwitchBot app.

Tap App Version repeatedly, and Developer Options will appear.

Make a note of your Token and Secret Key.

Using the obtained Token, call the SwitchBot API to retrieve the list of registered devices, and make a note of the Device ID (deviceId) for your air conditioner.

curl "https://api.switch-bot.com/v1.0/devices" -H "Authorization: ${token}" | jq .

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   350  100   350    0     0   1523      0 --:--:-- --:--:-- --:--:--  1528
{
  "statusCode": 100,
  "body": {
    "deviceList": [
      {
        "deviceId": "xxxxxxxxxx",
        "deviceName": "Hub Mini E7",
        "deviceType": "Hub Mini",
        "enableCloudService": true,
        "hubDeviceId": "000000000000"
      }
    ],
    "infraredRemoteList": [
      {
        "deviceId": "xx-xxxxxxxx-xxxx",
        "deviceName": "Air Conditioner",
        "remoteType": "Air Conditioner",
        "hubDeviceId": "xxxxxxxxxx"
      }
    ]
  },
  "message": "success"
}

Setting Up the Kubernetes Environment (kind)

This time, I used kind to create a local Kubernetes cluster on my Mac.
As long as you have Docker Desktop installed, you’re good to go.

kind

The cluster can be created with just two commands.

brew install kind
kind create cluster

If the node status is Ready, everything is set.

kubectl get nodes

NAME                 STATUS   ROLES           AGE    VERSION
kind-control-plane   Ready    control-plane   3m9s   v1.35.0

Application Structure

Repository

Since the codebase is quite small, it’s probably fastest to just look at the repository directly.

GitHub

Directory Structure

switchbot-k8s-cronjob/
├── README.md
├── Makefile
├── docker/
│   └── Dockerfile
├── app/
│   ├── switchbot.py # Simply calls the SwitchBot API
│   └── requirements.txt
├── k8s/
│   ├── cronjob.yaml
│   ├── secret.example.yaml # Tokens are managed via Kubernetes Secrets
│   └── namespace.yaml
└── .gitignore

What I Got Stuck On

At first, the CronJob kept failing with ImagePullBackOff.

In a kind cluster, Docker images built locally cannot be pulled automatically.
Because of this, it was necessary to set imagePullPolicy: Never.

apiVersion: batch/v1
kind: CronJob
metadata:
  name: switchbot-cron
  namespace: switchbot
spec:
  schedule: '0 19 * * *' # every day at 19:00
  timeZone: 'Asia/Tokyo' # JST
  jobTemplate:
    spec:
      template:
        spec:
          containers:
            - name: switchbot
              image: switchbot-job
              imagePullPolicy: Never # NOTE: use local image
              envFrom:
                - secretRef:
                    name: switchbot-secret
          restartPolicy: Never

Final Thoughts

Even though CronJob itself is outside the CKA exam scope, it was great to see that with just the knowledge gained from CKA study, I could build something that actually works in about 30 minutes.

There are plenty of other things I could try with this setup, so I’m planning to experiment a bit more during the New Year holidays.

How to Re-Encrypt Aurora Snapshots with a CMK for Cross-Account Migration

Atsushi Suzuki — Fri, 14 Nov 2025 07:33:15 +0000

When we rebuilt our AWS Organizations account structure, we needed to migrate the Aurora data from the old account to the new one as-is.

However, the Aurora cluster in the old account was encrypted using the AWS-managed key (aws/rds). After some investigation, we learned that aws/rds cannot be shared across accounts, meaning we couldn’t simply copy the snapshot to the new account...

To address this, we created customer-managed keys (CMKs) in both the old and new accounts, re-encrypted the snapshot using the CMK in the old account, and then copied/restored it to the new account.

We ran into several subtle issues with KMS key policies, so here is a complete step-by-step record for future reference.

Overall Flow

Step	Account	Description
1	Old	Create CMK for snapshot encryption
2	Old	Create Aurora snapshot
3	Old	Copy snapshot with re-encryption using old CMK
4	Old	Share CMK-encrypted snapshot with new account
5	New	Create CMK for re-encryption in new account
6	New	Re-encrypt copied snapshot with new CMK
7	New	Restore Aurora from CMK-encrypted snapshot

Architecture

Steps

1. Create a CMK for snapshot encryption in the old account

Create a CMK in the same region as the Aurora cluster.
Here is the key policy required for cross-account snapshot sharing.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowOldAccountAdmin",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::<OLD_ACCOUNT_ID>:root"
      },
      "Action": "kms:*",
      "Resource": "*"
    },
    {
      "Sid": "AllowUseFromOldAccountViaRDS",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::<OLD_ACCOUNT_ID>:root"
      },
      "Action": [
        "kms:Encrypt",
        "kms:Decrypt",
        "kms:ReEncryptFrom",
        "kms:ReEncryptTo",
        "kms:GenerateDataKey*",
        "kms:DescribeKey"
      ],
      "Resource": "*",
      "Condition": {
        "StringEquals": {
          "kms:ViaService": "rds.<REGION>.amazonaws.com"
        }
      }
    },
    {
      "Sid": "AllowUseFromNewAccountViaRDS",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::<NEW_ACCOUNT_ID>:root"
      },
      "Action": [
        "kms:Decrypt",
        "kms:ReEncryptFrom",
        "kms:DescribeKey",
        "kms:CreateGrant"
      ],
      "Resource": "*",
      "Condition": {
        "StringEquals": {
          "kms:ViaService": "rds.<REGION>.amazonaws.com"
        }
      }
    }
  ]
}

Note 1: Why `ReEncryptFrom` / `ReEncryptTo` are required

To share Aurora snapshots across accounts, encryption must transition through multiple layers.

aws/rds → old CMK → new CMK

ReEncryptFrom is needed to decrypt the existing key, and is used during.

AllowUseFromOldAccountViaRDS → decrypting snapshot encrypted with aws/rds
AllowUseFromNewAccountViaRDS → decrypting snapshot encrypted with old CMK

ReEncryptTo is needed to encrypt data with the new key.

Used during Step 3 to re-encrypt with old CMK

Note 2: Creating a CMK via the AWS Console

Refer to the following screenshots.

2. Create an Aurora snapshot

From console or CLI.

aws rds create-db-cluster-snapshot \
  --db-cluster-snapshot-identifier <snapshot> \
  --db-cluster-identifier <cluster>

3. Copy and re-encrypt snapshot using the old account’s CMK

aws rds copy-db-cluster-snapshot \
  --source-db-cluster-snapshot-identifier arn:aws:rds:REGION:OLD_ACCOUNT:cluster-snapshot:my-snap \
  --target-db-cluster-snapshot-identifier my-snap-cmk \
  --kms-key-id arn:aws:kms:REGION:OLD_ACCOUNT:key/<CMK-old>

4. Share the CMK-encrypted snapshot with the new account

aws rds modify-db-cluster-snapshot-attribute \
  --db-cluster-snapshot-identifier my-snap-cmk \
  --attribute-name restore \
  --values-to-add NEW_ACCOUNT_ID

Once shared successfully, the new account should see the snapshot.

5. Create a new CMK in the new account

This CMK will be used to re-encrypt the snapshot again.
To re-encrypt the snapshot using the new CMK, you need to grant the ReEncryptTo permission in the key policy.

Key policy example.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Enable IAM User Permissions",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::<NEW_ACCOUNT_ID>:root"
      },
      "Action": "kms:*",
      "Resource": "*"
    },
    {
      "Sid": "Allow rds.amazonaws.com to use the key",
      "Effect": "Allow",
      "Principal": {
        "Service": "rds.amazonaws.com"
      },
      "Action": [
        "kms:ReEncryptTo",
        "kms:GenerateDataKey*",
        "kms:Encrypt",
        "kms:DescribeKey",
        "kms:Decrypt",
        "kms:CreateGrant"
      ],
      "Resource": "*"
    }
  ]
}

6. Copy and re-encrypt the shared snapshot using the new CMK

You then copy the snapshot again using the new CMK.
Internally, the encrypted data is extracted using the old CMK’s ReEncryptFrom permission, and then re-encrypted using the new CMK’s ReEncryptTo permission.

aws rds copy-db-cluster-snapshot \
  --source-db-cluster-snapshot-identifier arn:aws:rds:REGION:OLD_ACCOUNT:cluster-snapshot:my-snap-cmk \
  --target-db-cluster-snapshot-identifier my-snap-new \
  --kms-key-id arn:aws:kms:REGION:NEW_ACCOUNT:key/<CMK-new>

7. Restore Aurora from the re-encrypted snapshot in the new account

After that, restore the Aurora cluster from the snapshot you created.

If you are restoring it using Terraform, the configuration will look like this.

resource "aws_rds_cluster" "example" {
  cluster_identifier  = "example-cluster"
  snapshot_identifier = "arn:aws:rds:REGION:NEW_ACCOUNT:cluster-snapshot:my-snap-new"
  kms_key_id          = aws_kms_key.db.arn
  engine              = "aurora-mysql"
}

resource "aws_rds_cluster_instance" "example_instances" {
  count              = 2
  identifier         = "example-cluster-${count.index}"
  cluster_identifier = aws_rds_cluster.example.id
  instance_class     = "db.r8g.large"
  engine             = aws_rds_cluster.example.engine
}

Conclusion

If you foresee migrating from a single-account setup to a multi-account structure, it’s much easier to use a CMK from the beginning. While it does cost more, it can save a huge amount of migration and operational effort later — making it well worth the investment.

Practical Terraform Tips for Secure and Reliable AWS Environments

Atsushi Suzuki — Fri, 10 Oct 2025 13:12:12 +0000

Recently, I had the opportunity to migrate our AWS setup from a single account to a multi-account architecture, which led me to completely redesign my Terraform repository from scratch.

Although both HashiCorp and AWS now officially provide MCP-based Terraform integrations, I wanted to document some practical tips and lessons I’ve picked up through the redesign process. Before implementing them, I studied this excellent book:

https://www.oreilly.co.jp/books/9784814400133/

I initially thought I already knew most of the content, but as I flipped through it (well, “swiped” through on Kindle), I realized how many important things I had overlooked.
It was honestly worth every yen, and I highly recommend it to anyone who’s been using Terraform “somewhat casually.”

In this post, I’ll introduce several beginner-friendly tips that I actually adopted in my AWS environment and found helpful.

Automatically Generate Module READMEs with `terraform-docs`

When I was managing the infrastructure alone, I didn’t really think about documentation for other team members. But as the team grew, I realized how bad tribal knowledge can be.

To address this, I started using terraform-docs to automatically generate documentation for each module’s Inputs, Outputs, and Resources.

Here’s my .terraform-docs.yml configuration:

formatter: "markdown table"
sections:
  show:
    - header
    - requirements
    - providers
    - modules
    - resources
    - inputs
    - outputs
output:
  file: "README.md"
  mode: inject
  template: |-
    <!-- BEGIN_TF_DOCS -->
    {{ .Content }}
    <!-- END_TF_DOCS -->
sort:
  enabled: true
  by: name
settings:
  hide-empty: true
  required: true
  sensitive: true
  type: true
  anchor: true

Then, I created a helper script to generate docs for all modules:

#!/usr/bin/env bash
set -euo pipefail
if ! command -v terraform-docs >/dev/null 2>&1; then
  echo "ERROR: terraform-docs not found. Please install it." >&2
  exit 1
fi

PROJECT_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
CONFIG="${PROJECT_ROOT}/.terraform-docs.yml"
cd "${PROJECT_ROOT}/modules"

for module_dir in */ ; do
  [ -d "$module_dir" ] || continue
  echo "📝 Generating docs for ${module_dir}"
  pushd "$module_dir" >/dev/null

  if [ ! -f "README.md" ]; then
    cat > README.md <<'EOF'
# Module
<!-- BEGIN_TF_DOCS -->
<!-- END_TF_DOCS -->
EOF
  fi

  terraform-docs --config "$CONFIG" . >/dev/null
  popd >/dev/null
done

Now, CI automatically updates each module’s README when changes are made — preventing missing documentation in pull requests.

S3 Backend Locking with `use_lockfile`

When configuring the S3 backend in versions.tf, I added the use_lockfile = true option to prevent state inconsistencies during simultaneous terraform plan or apply executions.

This feature became available starting with Terraform v1.11 — previously, you needed a separate DynamoDB table for state locking.

terraform {
  required_version = "~> 1.13.0"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 6.11.0"
    }
  }

  backend "s3" {
    bucket       = "terraform-state"
    key          = "environments/dev/terraform.tfstate"
    region       = "ap-northeast-1"
    encrypt      = true
    use_lockfile = true
  }
}

Enable Versioning on the S3 State Bucket

Always enable versioning on your state bucket.
I learned this the hard way — in a previous setup, we forgot to do this, and I shudder to think what could have happened if someone deleted the state file by mistake.

Simplify Subnet Calculation with `cidrsubnet`

I used to manually calculate subnet ranges from the VPC CIDR and hardcode them, but cidrsubnet() makes this much simpler:

locals {
  vpc_cidr = "192.168.0.0/20"
}

resource "aws_subnet" "a" {
  cidr_block = cidrsubnet(local.vpc_cidr, 4, 0)
  # 192.168.0.0/24
}

resource "aws_subnet" "b" {
  cidr_block = cidrsubnet(local.vpc_cidr, 4, 1)
  # 192.168.1.0/24
}

A small but very handy function.

Use `default_tags` to Avoid Tag Omissions

Previously, I manually added common tags like Project and Environment to each resource. Naturally, some were missing.
With provider-level default_tags, all resources now include them automatically.

I also added ManagedBy = "terraform" for quick visual identification in the AWS Console.

provider "aws" {
  region = "ap-northeast-1"
  default_tags {
    tags = {
      Project     = "MyApp"
      Environment = "prod"
      ManagedBy   = "terraform"
    }
  }
}

Protect Critical Resources with `prevent_destroy`

I once accidentally deleted an ElastiCache (Valkey) cluster because I didn’t review the plan carefully enough before applying.
To avoid repeating that mistake, I now set prevent_destroy = true for critical resources like databases and KMS keys.

resource "aws_rds_cluster" "aurora_cluster" {
  ...
  lifecycle {
    prevent_destroy = true
  }
}

You can also use ignore_changes to skip diff detection for specific attributes — useful when Blue/Green deployments cause dynamic changes like target group switches.

resource "aws_lb_listener_rule" "rules" {
  for_each = var.listener_rules

  listener_arn = aws_lb_listener.https[tostring(each.value.listener_port)].arn
  priority     = each.value.priority

  action {
    type             = each.value.action_type
    target_group_arn = aws_lb_target_group.tg[each.value.target_group].arn
  }

  condition {
    host_header {
      values = [each.value.host]
    }
  }

  dynamic "condition" {
    for_each = each.value.paths != null ? [each.value.paths] : []
    content {
      path_pattern {
        values = condition.value
      }
    }
  }

  lifecycle {
    ignore_changes = [action[0].target_group_arn]
  }
}

Import Existing Resources with the `import` Block

For manually created resources (e.g., S3, Amplify, AppConfig), I used to run terraform import via CLI.
Since Terraform 1.5.0, the import block allows for a more declarative and readable approach.

resource "aws_s3_bucket" "example" {
  bucket = "my-bucket"
}

import {
  to = aws_s3_bucket.example
  id = "my-bucket"
}

When you run terraform plan, Terraform will display the differences between the existing resources and your code.
Update your configuration to match the actual state, and then run terraform apply — the resources will be imported into your state file.

You can also auto-generate configurations using:

terraform plan -generate-config-out=generated.tf

Rename Resources Without Recreating Them via `terraform state mv`

If you mistakenly name a resource (e.g., aws_s3_bucket.exampl), renaming it directly in code can trigger a destroy/create.
Instead, use terraform state mv to safely rename without recreating:

terraform state mv aws_s3_bucket.exampl aws_s3_bucket.example

Always back up your state before doing this — versioning helps!

Granularity of Environment File Splitting

The part I struggled with the most during the repository redesign was how to structure the files under environments/{env}/.

At first, I dumped everything into a single main.tf file.
But as the number of resources grew, the file quickly became bloated — and scrolling through it turned into pure hell.
So I decided to split the configuration by functional domains.

applications.tf      # Cognito, Amplify, SES
compute.tf           # ECS, ECR, ALB, Auto Scaling
db.tf                # RDS, ElastiCache
monitoring.tf        # CloudWatch Logs, SNS, Chatbot
networking.tf        # VPC, Subnet, Route Table
security.tf          # IAM, Security Group, KMS
s3.tf
lambda.tf
locals.tf
providers.tf
versions.tf
...

When deciding how to split files, my guiding principle was simple:

“When I need to modify this resource, which file do I want to open?”

For example, when adding a new ALB listener rule, I usually need to check the ECS service and target groups at the same time — so I grouped them together in compute.tf.

# compute.tf
module "ecs" { ... }
module "ecr" { ... }
module "alb" { ... }
module "ecs_autoscaling_visitor" { ... }
module "ecs_autoscaling_company" { ... }

On the other hand, for networking, I often need to see the VPC, subnets, and route tables as a set, so I put them all together in networking.tf.

# networking.tf
module "vpc" { ... }
module "subnet" { ... }
module "route_table" { ... }
module "internet_gateway" { ... }
module "nat_gateway" { ... }
module "vpc_endpoint" { ... }
module "network_associations" { ... }

However, security.tf has now grown to over 1,000 lines…
That’s what happens when you pack IAM roles, policies, security groups, and KMS keys all in one file.

In hindsight, I probably should have split it up like this (and plan to refactor soon):

iam.tf – IAM roles and policies
security_groups.tf – Security groups
kms.tf – KMS keys

On the flip side, grouping ElastiCache and Aurora together in db.tf turned out to be a great decision.
Database-related changes often involve both cache and Aurora, so this granularity feels just right.

I also really like applications.tf:

# applications.tf
module "cognito" { ... }
module "amplify" { ... }
module "ses" { ... }

This file collects application-layer services such as authentication, frontend hosting, and email delivery.
Whenever I add new user-facing functionality, this is usually the file I open first.

In the end, there’s no single “correct” answer for file granularity.
The right structure depends on your team’s development style and the project’s scale.
In my case, I settled on a simple rule of thumb:

“Group resources that are closely related, and separate those with different update frequencies.”

That balance has worked very well for us.

Conclusion

The reference book above contains far more useful knowledge beyond these tips.
If you’re using Terraform somewhat intuitively, I strongly recommend revisiting your setup with these practices in mind.

References

Configure default tags for AWS resources

How I Combined Strands Agents, Bedrock AgentCore Runtime, and AgentCore Browser to Automate AWS Docs

Atsushi Suzuki — Mon, 15 Sep 2025 02:20:51 +0000

This summer, I barely had any time or energy to explore new technologies due to internal troubles at work and a family member being hospitalized. But by mid-September, I finally got some breathing room—so I decided to dive into Amazon Bedrock AgentCore, which had been generating buzz since July.

AgentCore is a suite of services designed for production-grade AI agent operations. Its core execution platform, AgentCore Runtime, quickly caught attention for being extremely easy to deploy.

Introducing Amazon Bedrock AgentCore: Securely deploy and operate AI agents at any scale (preview)

Since I had the chance, I also tried combining the AgentCore Browser, a managed browser for agents, to experiment with browser automation.

Amazon Bedrock AgentCore Browser Tool

What I Built

I created an agent with Strands Agents, deployed it on AgentCore Runtime, and connected it to the AgentCore Browser through Browser Use.

The goal: when asked something like

“How can I create a lifecycle policy for an S3 bucket?”,
the agent should navigate through official Amazon S3 documentation pages, extract key information, and generate an answer.

Prerequisites

Python 3.11+
AWS account with configured credentials
Region: us-west-2
- Unfortunately, AgentCore is not yet available in ap-northeast-1
Access to the Claude Sonnet 4 base model
- I used us.anthropic.claude-sonnet-4-20250514-v1:0

Code Overview

Here are the main points of the implementation.
The full source code is available on GitHub:

strands-agentcore-browseruse

Project Structure

Since there’s only one browser tool, I kept it simple:

src/
├── agents/
│   └── main_agent.py      # Main agent
├── runtime/
│   └── agentcore_app.py   # Integration with AgentCore Runtime
└── tools/
    └── browser_tool.py    # Browser automation logic

Implementing the Main Agent

The agent is built using Strands Agents.
It uses Claude Sonnet 4 as the model and the browse_url_tool for browser interaction.

def create_agent():
    """Create AWS Docs search agent"""
    model = BedrockModel(
        model_id='us.anthropic.claude-sonnet-4-20250514-v1:0',
        params={
            "max_tokens": 2048,
            "temperature": 0.3,
            "top_p": 0.8
        },
        region='us-west-2',
        read_timeout=600,
    )

    agent = Agent(
        name="aws_docs_agent",
        model=model,
        tools=[browse_url_tool],
        system_prompt=SYSTEM_PROMPT
    )

    return agent

Connecting to AgentCore Runtime

Integrating with AgentCore Runtime is surprisingly simple.
Just use the @app.entrypoint decorator to expose an endpoint that accepts requests.

app = BedrockAgentCoreApp()

@app.entrypoint
async def invoke(payload: Dict[str, Any], context) -> Dict[str, Any]:
    try:
        prompt = payload.get("prompt", "")
        agent = create_agent()
        result = agent(prompt)

        return {
            "result": str(result),
            "status": "success"
        }
    except Exception as e:
        return {
            "error": f"Error: {str(e)}",
            "status": "error"
        }

Browser Automation Logic

This is the key part.
It launches AgentCore Browser and controls it from Browser Use via Chrome DevTools Protocol (CDP).

Start an AgentCore Browser session
Get the CDP WebSocket URL
Pass it to Browser Use
Provide proper authentication headers

async def execute_browser_task(instruction: str, starting_url: str = "https://docs.aws.amazon.com") -> str:
      client = BrowserClient(region=region)

      try:
          client.start()
          ws_url, headers = client.generate_ws_headers()

          browser_profile = BrowserProfile(
              headers=headers,
              timeout=180000,
          )

          browser_session = BrowserSession(
              cdp_url=ws_url,
              browser_profile=browser_profile,
              keep_alive=True
          )

          await browser_session.start()

          browser_use_agent = BrowserUseAgent(
              task=instruction,
              llm=bedrock_chat,
              browser_session=browser_session,
          )

          result = await browser_use_agent.run()
          return result

      finally:
          if browser_session:
              await browser_session.close()
          client.stop()

Important
Without pinning the version of browser-use, I ran into WebSocket 403 errors.
Be sure to specify it in your requirements.txt:
browser-use<0.3.3

Deploying to AgentCore Runtime

Create a virtual environment:

python -m venv venv
source venv/bin/activate

Run the initial setup (this creates IAM roles, an Amazon Elastic Container Registry repo, etc.):

agentcore configure --entrypoint src/runtime/agentcore_app.py --region us-west-2

You’ll need to manually attach this policy to the AmazonBedrockAgentCoreSDKRuntime-us-west-2-~ IAM role created by agentcore configure so it can use AgentCore Browser:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "BrowserAccess",
      "Effect": "Allow",
      "Action": [
        "bedrock-agentcore:StartBrowserSession",
        "bedrock-agentcore:StopBrowserSession",
        "bedrock-agentcore:ConnectBrowserAutomationStream",
        "bedrock-agentcore:ConnectBrowserLiveViewStream",
        "bedrock-agentcore:ListBrowserSessions",
        "bedrock-agentcore:GetBrowserSession",
        "bedrock-agentcore:ListBrowsers",
        "bedrock-agentcore:GetBrowser"
      ],
      "Resource": "*"
    }
  ]
}

Then deploy:

agentcore launch

For redeployments:

agentcore launch --auto-update-on-conflict

Execution Result

After deployment, I tested it locally:

agentcore invoke '{"prompt": "How can I create a lifecycle policy for an S3 bucket?"}'

It took about 10 minutes (long), but the response was a fully detailed explanation of how to set up Amazon S3 lifecycle policies.

Response:
{"result": "Based on my research of the official AWS documentation, here's a comprehensive guide on how to create an S3 bucket lifecycle policy:\n\n## How to Create an S3 Bucket
Lifecycle Policy\n\nS3 lifecycle policies help you automatically manage your objects throughout their lifecycle, transitioning them between different storage classes and eventually
expiring them to optimize costs. Here are the different methods to create them:\n\n### **1. Using AWS CLI**\n\nThe most direct programmatic approach:\n\n**Basic
Command:**\n```

bash\naws s3api put-bucket-lifecycle-configuration --bucket YOUR-BUCKET-NAME --lifecycle-configuration file://lifecycle.json\n

```\n\n**Example JSON Configuration
File (lifecycle.json):**\n```

json\n{\n  \"Rules\": [\n    {\n      \"ID\": \"ExampleRule\",\n      \"Status\": \"Enabled\",\n      \"Filter\": {\n        \"Prefix\":
\"documents/\"\n      },\n      \"Transitions\": [\n        {\n          \"Days\": 30,\n          \"StorageClass\": \"STANDARD_IA\"\n        },\n        {\n          \"Days\":
90,\n          \"StorageClass\": \"GLACIER\"\n        }\n      ],\n      \"Expiration\": {\n        \"Days\": 365\n      }\n    }\n  ]\n}\n

```\n\n**Additional CLI Commands:**\n-
**View existing policy:** `aws s3api get-bucket-lifecycle-configuration --bucket YOUR-BUCKET-NAME`\n- **Delete policy:** `aws s3api delete-bucket-lifecycle --bucket
YOUR-BUCKET-NAME`\n\n### **2. Using AWS Management Console**\n1. Navigate to the S3 console\n2. Select your bucket\n3. Go to the \"Management\" tab\n4. Click \"Create lifecycle
rule\"\n5. Configure your rule settings (filters, transitions, expiration)\n6. Review and create the rule\n\n### **3. Using REST API**\nThe API operations available:\n-
**PutBucketLifecycleConfiguration**\n- **GetBucketLifecycleConfiguration** \n- **DeleteBucketLifecycle**\n\n### **4. Using AWS SDKs**\nAvailable for Java, .NET, Ruby, Python, and
other languages with complete lifecycle management capabilities.\n\n## **Key Configuration Elements**\n\n### **Basic XML Structure:**\n```

xml\n<LifecycleConfiguration>\n  <Rule>\n
<ID>Unique-Rule-Name</ID>\n    <Filter>\n      <Prefix>folder/</Prefix>\n    </Filter>\n    <Status>Enabled</Status>\n    <Transition>\n      <Days>365</Days>\n
<StorageClass>GLACIER</StorageClass>\n    </Transition>\n    <Expiration>\n      <Days>3650</Days>\n    </Expiration>\n  </Rule>\n</LifecycleConfiguration>\n

```\n\n### **Available
Storage Classes (in transition order):**\n- **STANDARD** → **STANDARD_IA** → **GLACIER** → **DEEP_ARCHIVE**\n\n### **Filter Options:**\n- **Prefix**: Target objects with specific
key prefixes\n- **Tags**: Apply rules based on object tags\n- **Size**: Filter by object size (ObjectSizeGreaterThan/LessThan)\n- **Combinations**: Use `<And>` to combine multiple
filters\n\n## **Important Best Practices**\n\n1. **Configuration Replacement**: S3 replaces the entire lifecycle configuration when you add a new one. To update, retrieve existing
config first, modify it, then reapply.\n\n2. **Propagation Delay**: Expect a few minutes delay before new configurations take full effect.\n\n3. **Small Objects**: Objects smaller
than 128KB are prevented from transitioning by default (use size filters to override).\n\n4. **Testing**: Always verify your configuration after applying by checking rule counts
and testing with sample objects.\n\n5. **Cost Optimization**: Design rules to tier down storage classes over object lifetime to minimize costs.\n\n6. **Versioning Support**: Use
`NoncurrentVersionTransition` and `NoncurrentVersionExpiration` for versioning-enabled buckets.\n\n## **Common Use Cases**\n\n- **Log Archival**: Move application logs to cheaper
storage after 30 days, archive after 90 days\n- **Backup Management**: Transition backups to Glacier, expire after retention period\n- **Document Management**: Move old documents
through storage tiers based on access patterns\n- **Cleanup**: Remove incomplete multipart uploads and expired delete markers\n\nFor detailed examples, configuration samples, and
troubleshooting guidance, refer to the AWS S3 User Guide on lifecycle configuration.\n\n**Documentation Links:**\n- [S3 Lifecycle Configuration
Examples](https://docs.aws.amazon.com/AmazonS3/latest/userguide/lifecycle-configuration-examples.html)\n- [Setting S3 Lifecycle
Configuration](https://docs.aws.amazon.com/AmazonS3/latest/userguide/how-to-set-lifecycle-configuration-intro.html)\n", "status": "success"}

I could also see the Live View of browser operations in the AgentCore Browser console:

References

Fixing the “Invalid Parameter” Error When Registering an SNS Topic for SES Feedback Notifications

Atsushi Suzuki — Thu, 11 Sep 2025 01:11:44 +0000

While migrating an existing service to a new AWS account, I ran into a strange error when trying to set up an SNS topic for SES feedback notifications (Bounce, Complaint, Delivery):

An invalid or out-of-range value was supplied for the input parameter.

I had created the SNS topic as a "Standard" type, in the same region as SES, and configured the access policy to allow ses.amazonaws.com with sns:Publish. Everything seemed correct, so I couldn’t figure out what was wrong.

Root Cause

The problem turned out to be insufficient KMS key policy permissions on the encryption key used for the SNS topic.

When publishing to an encrypted SNS topic, the publishing service (in this case, SES) needs permissions for both kms:GenerateDataKey and kms:Decrypt. The actual encryption/decryption is handled by SNS, but SES must be able to trigger the KMS API calls required for that process.

However, in this case I had used the AWS managed key alias/aws/sns for SNS topic encryption—which cannot be edited to adjust the key policy.

Solution

The workaround was to create a customer managed key (CMK) named sns-ses-dev-1, attach the following key policy, and configure it for the SNS topic:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowSESToUseKMSKey",
            "Effect": "Allow",
            "Principal": {
                "Service": "ses.amazonaws.com"
            },
            "Action": [
                "kms:GenerateDataKey",
                "kms:Decrypt"
            ],
            "Resource": "*"
        }
    ]
}

After applying this, I was finally able to configure the SNS topic for SES feedback notifications successfully.

Final Thoughts

Since CMKs incur additional cost, it might not be worth enabling SNS topic encryption in development environments at all. Using encryption only in production could be a more balanced approach.

References

Configuring Amazon SNS notifications for Amazon SES

AWS Control Tower Landing Zone Setup: Troubleshooting Account Limits and KMS Policies

Atsushi Suzuki — Tue, 02 Sep 2025 00:41:04 +0000

While setting up a multi-account environment and trying to enable AWS Control Tower, I encountered two mysterious errors. Here's how I resolved them.

"Account limit reached" error (even though I hadn't reached the limit)
CloudFormation deployment failure (KMS key policy related)

Account Limit Error

When enabling Control Tower, it creates a Log Archive account and an Audit account. However, only the Audit account was created, and I got an error saying "Account limit reached" for the Log Archive account.

AWS Control Tower failed to set up your landing zone completely: AWS Control Tower cannot create an account because you have reached the limit on the number of accounts in your organization.

When I checked Service Quotas, the Default maximum number of accounts was set to 10, but my actual account count was clearly below that (only 3 including the management account).

I couldn't figure out the root cause (which is frustrating), but I resolved it by increasing the Service Quota to 15. Perhaps Control Tower internally reserves some account slots...?

CloudFormation Deployment Failure (KMS Key Policy Related)

Just when I thought I'd resolved the account limit error, I ran into a CloudFormation deployment error.

AWS Control Tower failed to set up your landing zone completely: AWS Control Tower failed to deploy stack(s): arn:aws:cloudformation:ap-northeast-1:XXXXXXXX:stack/AWSControlTowerBP-BASELINE-CLOUDTRAIL-MASTER/4787d990-8144-11f0-ad99-0625194a54fd. To continue, review the failed stack(s) and try again.

When I opened the CloudFormation console, I saw this:

I had configured CloudTrail and Config to use KMS for encryption, but apparently these services lacked the necessary permissions to access the KMS key.

Required Permissions:

Config: kms:Decrypt, kms:GenerateDataKey
CloudTrail: kms:GenerateDataKey*, kms:Decrypt

So I added the following to my KMS policy:

{
  "Sid": "AllowConfigToUseKms",
  "Effect": "Allow",
  "Principal": { "Service": "config.amazonaws.com" },
  "Action": ["kms:Decrypt","kms:GenerateDataKey"],
  "Resource": "arn:aws:kms:ap-northeast-1:xxxxxxxxxxxx:key/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
},
{
  "Sid": "AllowCloudTrailToUseKms",
  "Effect": "Allow",
  "Principal": { "Service": "cloudtrail.amazonaws.com" },
  "Action": ["kms:GenerateDataKey*","kms:Decrypt"],
  "Resource": "arn:aws:kms:ap-northeast-1:xxxxxxxxxxxx:key/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
  "Condition": {
    "StringEquals": {
      "aws:SourceArn": "arn:aws:cloudtrail:ap-northeast-1:xxxxxxxxxxxx:trail/aws-controltower-BaselineCloudTrail"
    },
    "StringLike": {
      "kms:EncryptionContext:aws:cloudtrail:arn": "arn:aws:cloudtrail:*:xxxxxxxxxxxx:trail/*"
    }
  }
}

After that, I deleted the failed CloudFormation stack AWSControlTowerBP-BASELINE-CLOUDTRAIL-MASTER and retried the Control Tower setup. All errors were resolved!

Conclusion

I'm still puzzled about the root cause of the account limit error. If anyone knows what might have caused it, please let me know!

References

https://dev.classmethod.jp/articles/aws-control-tower-setup-failed-kms/

How I Reduced ELB Access Log Analysis Time by 80% Using AWS Data Processing MCP Server

Atsushi Suzuki — Sat, 09 Aug 2025 06:08:12 +0000

The other day, we had a sudden spike in requests to one of our services, which triggered a flood of ELB latency alerts.

Fortunately, I had already built a foundation for analyzing ELB access logs via Athena.
The problem? In an incident scenario, running SQL-based analysis is slow. Writing a query → fixing it → waiting for aggregation → fixing again… it all takes time.

I eventually tracked down a suspicious IP hammering us with requests and blocked it, but I was left thinking "That could have gone faster."

While browsing AWS’s official MCP servers again, I noticed one I’d previously ignored the AWS Data Processing MCP Server.
The name made me think it wasn’t relevant, so I had skipped it before—but it turns out it lets you query Athena using natural language.

This post is a quick walkthrough of how I set it up, the snags I hit along the way, and how it worked in practice.

Setup

I connected it from Claude Desktop by adding the following to claude_desktop_config.json.
work-mcp is an AWS_PROFILE I created just for this purpose.

{
  "mcpServers": {
    "aws.dp-mcp": {
      "command": "uvx",
      "args": ["awslabs.aws-dataprocessing-mcp-server@latest"],
      "env": {
        "FASTMCP_LOG_LEVEL": "ERROR",
        "AWS_PROFILE": "work-mcp",
        "AWS_REGION": "ap-northeast-1"
      },
      "autoApprove": [],
      "disabled": false
    }
  }
}

IAM Policy (Where I Got Stuck)

The permissions listed in the official docs weren’t enough for my environment—I ran into missing permissions like athena:StartQueryExecution.
Here’s the policy I ended up using. You’ll need to replace the S3 bucket names with your own.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AthenaQueryAccess",
            "Effect": "Allow",
            "Action": [
                "athena:StartQueryExecution",
                "athena:GetQueryExecution", 
                "athena:GetQueryResults",
                "athena:StopQueryExecution"
            ],
            "Resource": "*"
        },
        {
            "Sid": "AthenaMetadataAccess", 
            "Effect": "Allow",
            "Action": [
                "athena:GetDatabase",
                "athena:GetTableMetadata",
                "athena:ListDatabases",
                "athena:ListTableMetadata"
            ],
            "Resource": "*"
        },
        {
            "Sid": "GlueDataCatalogAccess",
            "Effect": "Allow", 
            "Action": [
                "glue:GetDatabase",
                "glue:GetDatabases",
                "glue:GetTable",
                "glue:GetTables"
            ],
            "Resource": "*"
        },
        {
            "Sid": "S3LogsAccess",
            "Effect": "Allow",
            "Action": [
                "s3:GetObject",
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::your-alb-logs-bucket",
                "arn:aws:s3:::your-alb-logs-bucket/*"
            ]
        },
        {
            "Sid": "AthenaResultsS3Access",
            "Effect": "Allow",
            "Action": [
                "s3:GetBucketLocation",
                "s3:GetObject", 
                "s3:ListBucket",
                "s3:PutObject",
                "s3:DeleteObject"
            ],
            "Resource": [
                "arn:aws:s3:::your-athena-results-bucket",
                "arn:aws:s3:::your-athena-results-bucket/*"
            ]
        }
    ]
}

Example Usage

Question 1

I want to analyze `alb_access_logs` in Athena.
Between 2025-08-07 00:00:00 and 23:59:59 JST, are there any bot-like requests?

Answer 1 (Sensitive data masked)

# Bot Analysis of ALB Access Logs (Aug 7, 2025 JST / Some details masked)

Analyzed ALB access logs in Athena to extract confirmed and suspected bot traffic.
* External IPs are anonymized as `[IP-X]`
* Internal API paths are parameterized
* Counts are shown in approximate ranges

## 🤖 Bots Detected via User-Agent

1. **VirusTotalBot** — ~160 requests
   - Representative IPs: `[IP-V1]`, `[IP-V2]`, `[IP-V3]`

2. **Googlebot** — a few requests
   - Representative IP: `[IP-G1]`

3. **Google-Safety Bot** — a few dozen requests
   - Representative IPs: `[IP-GS1]`, `[IP-GS2]`

4. **Plesk screenshot bot** — ~50 requests
   - Representative IP: `[IP-P1]`

5. **Others**:
   - **AmazonProductDiscovery bot** (few)
   - **Brainfeedbot** (few)
   - **meta-externalads** (few, Facebook-related)
   - **MicroAdBot** (single instance)
   - **curl** (few)

6. **User-Agent not set (“-”)** — ~1.1k requests (most from `[IP-U1]`)

## 🚨 High-Volume Suspicious Activity (Top Examples)

1. **[IP-A]** — ~280k requests (most for the day)
   - Main UA: Generic desktop Chrome
   - Unique URLs: ~2.0k
   - Pattern: Very frequent polling for a single session (peak ~3k–5k/hour)

2. **[IP-B]** — ~180k requests
   - 2 distinct UAs
   - Unique URLs: ~18k

3. **[IP-C]** — ~120k requests
   - 3 distinct UAs
   - Unique URLs: ~10k

4. **[IP-D]** — ~56k requests
   - Concentrated over ~3 hours (peak ~4k/hour)

## 🔍 Notable Patterns (Excerpt)

**[IP-A]** strongly suspected to be a bot or misconfigured client due to:
- Continuous high-frequency polling of similar URLs
- Targeting endpoints like `/v1/log/{sessionId}` for session monitoring

## 📊 Daily Totals

- **Total requests**: ~15.2M
- **Clear bots**: ~1.4k
- **High-volume sources (≥1k/day)**: ~20 IPs totaling ~2.0M requests

**Conclusion:**  
`[IP-A]` is likely a bot or a misconfigured client, given the excessively short polling interval.  
Recommend rate-limiting, access control, or switching to a push/caching approach.

Question 2

If we enable AWS WAF Bot Control, will the block count be high?
And would `[IP-A]` or `[IP-B]` be caught?

Answer 2

# AWS WAF Bot Control Assessment (Details masked)

## 🛡️ Bots Likely to be Blocked (Clearly identifiable by UA)

1. **VirusTotalBot**: ~160 requests
2. **Googlebot**: Few (usually whitelisted)
3. **Google-Safety Bot**: Few dozen (usually whitelisted)
4. **MicroAdBot**: ~10
5. **Brainfeedbot**: ~10
6. **Plesk screenshot bot**: Few–dozen
7. **curl**: Few

**Total:** ~230–290 requests/day

## User-Agent Not Set (“-”): ~0.5k–1.1k/day
- Some would be detected by Bot Control
- Could include legitimate proxy traffic

### 📊 Block Volume Prediction
- Clear bots: ~300–400 requests/day
- Share of total: ~0.002–0.003% (on 15–18M daily requests)

## 🎯 Would `[IP-A]` and `[IP-B]` Be Blocked?

### `[IP-A]` (Global IP)
- Likely bypassed
- UA: Generic desktop Chrome (Bot Control focuses on UA-based detection)
- Could be caught only via behavioral analysis (e.g., consistent ~25k/day mechanical access)

### `[IP-B]` (Corporate network)
- Almost certainly bypassed
- UA: Common browser
- Many legitimate authenticated API requests
- Access patterns suggest possible misconfiguration rather than malicious intent

## 🔧 Suggested WAF Enhancements

1. **Rate Limiting Rules**
   - Per IP: ~100 req/min, ~1,000 req/hour

2. **Endpoint-Specific Limits**
   - `/v1/companies/*`: ~10 req/min per IP
   - `/v1/log/*`: ~5 req/min per IP

3. **Behavioral Rules**
   - Burst access to same URL
   - Unusual high-frequency overnight traffic
   - Evenly timed polling

4. **Whitelist Exceptions**
   - Fixed IP ranges for trusted companies
   - Certain UAs for internal monitoring tools

### Effectiveness Forecast
- Bot Control alone: ~300–500 blocked requests/day
- Problematic `[IP-A]/[IP-B]` traffic remains
- Combining with rate-limiting significantly improves impact with minimal false positives

How This Influenced WAF Design

Manually writing SQL for “How many bot-like requests?” is tedious.
This tool delivered a detailed breakdown instantly—helping me confirm that Bot Control wouldn’t address our main offenders.
That was valuable not only for effectiveness but also for cost analysis, since Bot Control is billed separately from other WAF rules.

The MCP server doesn’t make SQL obsolete, but it dramatically shortens the time from alert to first hypothesis.

Run Multi-Agent AI in the Cloud Without a Local GPU Using Docker Offload and Compose

Atsushi Suzuki — Mon, 21 Jul 2025 04:45:38 +0000

With the release of Docker Desktop 4.43 (July 3, 2025), you can now declare AI models, agents, and MCP tools in a single compose.yaml file and launch them all with a single docker compose up.

On top of that, the beta release of Docker Offload allows you to run Compose projects directly in the cloud with NVIDIA L4 GPUs. This opens the door to running large-scale models from even a modest laptop.

In this post, I’ll walk through how to use docker/compose-for-agents' official A2A Multi‑Agent Fact Checker sample entirely with Compose. I'll also demonstrate how to offload the workload to the cloud using Docker Offload.

Some images in this post are sourced from the official Docker Offload Content Kit provided to Docker Captains.

Sample Overview

The A2A Multi‑Agent Fact Checker is a multi-agent system built with Google’s ADK (Agent Development Kit) and the A2A protocol. It features three agents—Auditor, Critic, and Reviser—that work together to research, verify, and revise a given claim, then return a final conclusion.

Auditor: Breaks down the user’s claim into subtasks and delegates them to Critic and Reviser. Collects the final answer and returns it via the UI.
Critic: Performs external web searches using the DuckDuckGo MCP tool to gather supporting evidence.
Reviser: Refines and verifies the output using the evidence gathered by Critic and the initial draft from Auditor.

The Critic communicates with the outside world via the MCP Gateway, and the inference model (Gemma 3 4B‑Q4) is hosted via Docker Model Runner.

Key Highlights of `compose.yaml`

Here’s the full compose.yaml used to define the multi-agent system:

services:
  # Auditor Agent coordinates the entire fact-checking workflow
  auditor-agent-a2a:
    build:
      target: auditor-agent
    ports:
      - "8080:8080"
    environment:
      - CRITIC_AGENT_URL=http://critic-agent-a2a:8001
      - REVISER_AGENT_URL=http://reviser-agent-a2a:8001
    depends_on:
      - critic-agent-a2a
      - reviser-agent-a2a
    models:
       agents:
         endpoint_var: MODEL_RUNNER_URL
         model_var: MODEL_RUNNER_MODEL

  critic-agent-a2a:
    build:
      target: critic-agent
    environment:
      - MCPGATEWAY_ENDPOINT=http://mcp-gateway:8811/sse
    depends_on:
      - mcp-gateway
    models:
       gemma3:
         # specify which environment variables to inject into the container
         endpoint_var: MODEL_RUNNER_URL
         model_var: MODEL_RUNNER_MODEL

  reviser-agent-a2a:
    build:
      target: reviser-agent
    environment:
      - MCPGATEWAY_ENDPOINT=http://mcp-gateway:8811/sse
    depends_on:
      - mcp-gateway
    models:
       gemma3:
         endpoint_var: MODEL_RUNNER_URL
         model_var: MODEL_RUNNER_MODEL

  mcp-gateway:
    # mcp-gateway secures your MCP servers
    image: docker/mcp-gateway:latest
    use_api_socket: true
    command:
      - --transport=sse
      - --servers=duckduckgo
      # add an MCP interceptor to log the responses
      - --interceptor
      - after:exec:echo RESPONSE=$(cat) >&2

models:
  # declare LLM models to pull and use
  gemma3:
    model: ai/gemma3:4B-Q4_0
    context_size: 10000 # 3.5 GB VRAM
    #context_size: 131000 # 7.6 GB VRAM

Top-Level `models`

As of Compose v2.38, you can declare LLM images as OCI Artifacts under the top-level models field. Docker Model Runner will automatically pull the image and expose it as an API endpoint.

Per-Service `models`

Each service defines which model to use and how to inject its URL and model name via environment variables:

export MODEL_RUNNER_URL=http://model-runner:12434
export MODEL_RUNNER_MODEL=gemma3

This means your app can simply read environment variables without hardcoding the model path.

MCP Gateway

The docker/mcp-gateway image acts as a secure relay for MCP servers like DuckDuckGo. It communicates with the Critic agent using Server-Sent Events (SSE). The --interceptor flag logs the raw responses directly to stderr.

Dependency Management

Like traditional Compose setups, depends_on is used to manage startup order: MCP Gateway → Critic/Reviser → Auditor. This eliminates the need for retry logic.

Running Locally

You can launch the stack locally with:

docker compose up --build

[+] Running 8/9
 ✔ reviser-agent-a2a                  Built       0.0s
 ✔ critic-agent-a2a                   Built       0.0s
 ✔ auditor-agent-a2a                  Built       0.0s
 ⠴ gemma3 Configuring                            76.5s
 ...

Since Gemma 3 4B‑Q4 is quantized, it even runs on my MacBook Air M2.

Open your browser to http://localhost:8080, type in a claim like:

How far is the Moon from the Earth?

The Critic performs a DuckDuckGo search, the Reviser polishes the output, and the Auditor returns the final answer.

Using Docker Offload

If you want to use a larger model like Gemma 27B Q4, local GPUs might not cut it. That’s where Docker Offload comes in—just enable the feature and override your model config to run in the cloud.

Enabling Docker Offload

First, sign up for beta access on Docker's official site. (As a Docker Captain, I received early access.)

Then, go to Settings > Beta Features and enable both:

"Enable Docker Offload"
"Enable Docker Offload GPU Support"

Switch the Docker Desktop toggle to the cloud icon to activate Offload (or run docker offload start).

`compose.offload.yaml`

Prepare a separate file to override the model definition:

models:
  gemma3:
    model: ai/gemma3-qat:27B-Q4_K_M 
    context_size: 10000 # 18.6 GB VRAM
    # context_size: 80000 # 28.37 GB VRAM
    # context_size: 131000 # 35.5 GB VRAM

Running with Offload

To launch in the cloud, combine the two Compose files:

docker compose -f compose.yaml -f compose.offload.yaml up --build

This overrides the top-level models field with the Offload-specific config.

Docker Offload gives you 300 GPU credits for free, and any additional usage is billed at \$0.015 per GPU second. Don’t forget to stop the service afterward:

docker offload stop

Final Thoughts

Trying out Compose and Offload together really shows the power of unified agent, model, and tool orchestration. It’s incredibly convenient to use the same docker compose up command for both local and cloud environments.

The agent space is evolving rapidly, so if you have a better workflow or tips, I’d love to hear about them.

If you’re curious about Docker Offload, start experimenting within the free credit limit—you might be surprised how far you can go.

DEV Community: Atsushi Suzuki

Taking the KCSA Exam on My Washing Machine (And Passing!)

Exam Results

Impressions & Difficulty

Study Materials I Used

KodeKloud

GitHub Mock Exams

What's Next?

CKAD is Easier Than CKA: How I Passed with 20 Hours of Study

Exam Result

Thoughts on Difficulty

Study Materials Used

KodeKloud

Killer Shell

Tips

1. Master the CKAD-specific domains

2. Memorize the Documentation URL

3. Practice Copy-Pasting in Remote Desktop

Closing Thoughts

How I Passed the KCNA in Just 3 Hours (After Getting CKA)

Exam Results

Reflections and Difficulty

Preparation Materials

1. KodeKloud

2. Kindle Practice Exams

Key Study Tips

Master Pod Scheduling

Understand the CNCF Ecosystem

GitOps Concepts

Monitoring & Observability Tools

What's Next?

CKA (Certified Kubernetes Administrator) Exam Report 2026: Don’t Rely on Old Guides (Mastering the Post-2025 Revision)

Why I am writing this article

Exam Results

Impressions of the New Exam Scope (Post-Feb 2025) and Difficulty

Learning Resources Used

Udemy

KodeKloud

Killer Shell

Tips

No need to add settings to .bashrc

Memorize the Documentation URL

Learn Copy-Paste Shortcuts for Remote Desktop

Prepare for Applied Questions

Exam Environment

Final Thoughts

I Automated My Air Conditioner with Kubernetes (kind + CronJob + SwitchBot)

SwitchBot Hub Mini Setup

Prerequisites

Getting the Token / Secret Key / Device ID

Setting Up the Kubernetes Environment (kind)

Application Structure

Repository

Directory Structure

What I Got Stuck On

Final Thoughts

How to Re-Encrypt Aurora Snapshots with a CMK for Cross-Account Migration

Overall Flow

Architecture

Steps

1. Create a CMK for snapshot encryption in the old account

Note 1: Why ReEncryptFrom / ReEncryptTo are required

Note 2: Creating a CMK via the AWS Console

2. Create an Aurora snapshot

3. Copy and re-encrypt snapshot using the old account’s CMK

4. Share the CMK-encrypted snapshot with the new account

5. Create a new CMK in the new account

6. Copy and re-encrypt the shared snapshot using the new CMK

7. Restore Aurora from the re-encrypted snapshot in the new account

Conclusion

Practical Terraform Tips for Secure and Reliable AWS Environments

Automatically Generate Module READMEs with terraform-docs

S3 Backend Locking with use_lockfile

Enable Versioning on the S3 State Bucket

Simplify Subnet Calculation with cidrsubnet

Use default_tags to Avoid Tag Omissions

Protect Critical Resources with prevent_destroy

Import Existing Resources with the import Block

Rename Resources Without Recreating Them via terraform state mv

Granularity of Environment File Splitting

No need to add settings to `.bashrc`

Note 1: Why `ReEncryptFrom` / `ReEncryptTo` are required

Automatically Generate Module READMEs with `terraform-docs`

S3 Backend Locking with `use_lockfile`

Simplify Subnet Calculation with `cidrsubnet`

Use `default_tags` to Avoid Tag Omissions

Protect Critical Resources with `prevent_destroy`

Import Existing Resources with the `import` Block

Rename Resources Without Recreating Them via `terraform state mv`

Key Highlights of `compose.yaml`

Top-Level `models`

Per-Service `models`

`compose.offload.yaml`