The Setup
Google's Online Boutique is an open-source 11-service e-commerce demo — Go, Python, Node.js, C#, and Java, all communicating over gRPC. It ships designed for GKE. The task here: build a complete production AWS platform around it — Terraform, EKS, ECR, IAM, Helm, GitHub Actions — without changing a single line of the application source.
The application layer stays exactly as Google shipped it. Everything else gets built from scratch.
The Build
Seven Terraform files, each with one job:
terraform-aws/
├── vpc.tf → VPC, 3 AZs, public + private subnets, NAT Gateway
├── eks.tf → EKS 1.30, cluster "online-boutique-production", 2× t3.small nodes
├── ecr.tf → 12 ECR repos, one per service, image scan on push
├── iam.tf → IRSA roles, least-privilege, no node-wide IAM
├── backend.tf → S3 remote state, DynamoDB locking
├── variables.tf → region, cluster name, instance types
└── outputs.tf → cluster endpoint, kubeconfig command, ECR URLs
VPC applied in 13 seconds. IAM roles, under a second each. EKS control plane, 9m37s — normal for EKS. Total: 110 AWS resources, one terraform apply.
Node instance type is t3.small — 2 vCPU, 2GB RAM. Not an arbitrary choice: this AWS account is restricted to free-tier-eligible instance types, and t3.small is the smallest one that comfortably runs all 11 lightweight services with room to spare.
The IAM Design
enable_irsa = true in eks.tf turns on the cluster's OIDC provider — the mechanism that lets a Kubernetes ServiceAccount assume an AWS IAM role directly, without any node-wide credentials.
# node role — attached to every node, every pod inherits this
# AmazonEC2ContainerRegistryReadOnly — every pod needs to pull images
# ebs_csi_irsa_role — scoped to exactly one ServiceAccount:
# kube-system:ebs-csi-controller-sa — nothing else can assume it
The distinction matters: the node role carries only what every pod needs regardless of which service it is (image pull). IRSA carries what only one specific pod needs. Getting that split right is most of what production IAM design actually is.
The CI/CD Pipeline
.github/workflows/aws-eks-deploy.yaml triggers on push to main, on version tags, or manually via workflow_dispatch. The permissions block:
permissions:
id-token: write
That one line is what lets the workflow request an OIDC token from GitHub at all. The credentials step that follows:
- uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: arn:aws:iam::<account-id>:role/github-actions-deploy
aws-region: us-east-1
No aws-access-key-id. No aws-secret-access-key. GitHub issues a short-lived token proving exactly which workflow, in which repo, is running. AWS's IAM trust policy — scoped to this specific repository — accepts it and issues temporary credentials for that job only. When the job ends, the credentials are gone. Nothing to rotate, nothing to leak.
The build job matrix runs all 12 services in parallel (fail-fast: false), pushing each to its own ECR repo, then helm upgrade --install deploys the release to EKS.
The Helm Layer
helm-chart/values-aws-production.yaml layers on top of the base values.yaml:
helm upgrade --install online-boutique helm-chart/ \
-f values.yaml \
-f values-aws-production.yaml \
--namespace online-boutique
images.repository points at the ECR registry, not Docker Hub. Every service carries its own resource request/limit block — recommendationservice asks for 220Mi, checkoutservice only 64Mi — because the scheduler uses the request value to decide which node a pod can fit on. The serviceAccounts block carries an IRSA annotation wired to the ECR-pull role, applied to every service.
The Result
kubectl get nodes
NAME STATUS ROLES AGE
ip-10-0-1-23.ec2.internal Ready <none> 4m
ip-10-0-2-45.ec2.internal Ready <none> 4m
kubectl get pods -n online-boutique
NAME READY STATUS RESTARTS AGE
adservice-... 1/1 Running 0 2m
cartservice-... 1/1 Running 0 2m
checkoutservice-... 1/1 Running 0 2m
...
11/11 pods Running. One Helm release manages all of them.
kubectl get svc frontend-external -n online-boutique
# EXTERNAL-IP: <real-elb-hostname>.us-east-1.elb.amazonaws.com
Live storefront, real AWS LoadBalancer, real cart flow — add a product to cart, and that request hits frontend, which calls cartservice, which writes to Redis.
What I Learned
1. IAM should be scoped per pod, not per node.
Node-wide IAM means one compromised pod can reach everything every other pod on that node can reach. IRSA fixes that — a ServiceAccount can assume exactly one role, scoped to exactly what it needs, nothing shared.
2. Free-tier account restrictions shape real infra decisions.
t3.small wasn't picked for cost alone — this account is restricted to free-tier-eligible instance types at all. Checking aws ec2 describe-instance-types --filters Name=free-tier-eligible,Values=true before writing terraform.tfvars avoids surprises later.
3. Keyless CI/CD removes an entire class of leaked-credential incidents.
No static AWS key ever exists in GitHub Secrets. OIDC + a scoped trust policy means there's nothing long-lived to rotate or leak.
4. Small instance types are enough for lightweight microservices.
2 vCPU / 2GB per node, 2 nodes, ran all 11 services comfortably. The workload never needed anything bigger.
Try It
GitHub: https://github.com/ThinkWithOps/thinkwithops-online-boutique-production
Video walkthrough: https://youtu.be/qjnJab8mqcI
cd terraform-aws/bootstrap && terraform init && terraform apply
cd ../terraform-aws
cp terraform.tfvars.example terraform.tfvars
# fill in your AWS account ID, then:
terraform init && terraform plan -out=tfplan && terraform apply "tfplan"
What's your default IAM boundary in Kubernetes — per node, or per pod? Curious how other teams are actually doing this.
Top comments (0)