DEV Community: Tanseer

Build and Deploy a 3 Tier Serverless App on AWS with Terraform Modules

Tanseer — Tue, 14 Jul 2026 04:59:11 +0000

Introduction

Most tutorials show you how to create a Lambda function by clicking around the AWS console. That works once. Then you need a staging environment, and you click everything again. Then production. Then someone asks "wait, did staging have the 30 second timeout or was that only dev?" and nobody knows, because the infrastructure lives in memory and screenshots.

This post walks through a project where the entire 3 tier stack comes from a single terraform apply. Tier one is the frontend on Amplify, tier two is the API on Lambda and API Gateway, tier three is a MySQL database on RDS. Run the apply in the dev folder, you get a dev environment. Run it in prod, you get prod. Same code, zero clicking.

The app itself is a small todo application. That is on purpose. The app is just a passenger. The real work is the Terraform: reusable modules, three isolated environments, remote state stored in S3, and module outputs wired into module inputs so everything builds in the correct order.

The full source is here: github.com/TanseerS/serverless-web-application

Quick note on the word serverless before we start. It does not mean there are no servers. It means AWS manages the servers and you only bring code. You pay when the code runs, not while it sits idle.

What we are building

The flow is simple to trace. A user opens a React app hosted on AWS Amplify (a service that builds and hosts frontends straight from a GitHub repo). The app calls a REST API on API Gateway (the AWS service that gives your code an HTTPS URL). Every route on that API points to a single Lambda function (code that runs on demand without a server you manage). Inside the Lambda, an Express app handles the routing and talks to a MySQL database on RDS (managed relational databases on AWS).

Deployments are split in two. Push to GitHub and Amplify rebuilds the frontend on its own, since CI/CD comes free with Amplify. Change the backend code or any infrastructure, and terraform apply ships it. Terraform even re-zips the backend folder automatically whenever the source changes.

The repository layout

serverless-web-application/
├── frontend/            # React 18 + Vite app
├── backend/             # Express + Sequelize API, packaged for Lambda
├── assets/              # diagrams and screenshots
└── infrastructure/
    ├── bootstrap/       # run once, creates the S3 state bucket
    ├── modules/
    │   ├── database/    # RDS MySQL, security group, subnet group
    │   ├── backend/     # Lambda, API Gateway, IAM, code packaging
    │   └── frontend/    # Amplify app, branch, IAM
    └── environments/
        ├── dev/
        ├── staging/
        └── prod/

Two ideas make this layout work.

First, modules are reusable building blocks. The database module does not know or care whether it is building dev or prod. It takes a project and an environment variable and names everything ${var.project}-${var.environment}-..., so the same code stamps out todo-app-dev-rds, todo-app-staging-rds and todo-app-prod-rds.

Second, each environment folder is tiny. It looks up the default VPC with data sources, calls the three modules, and passes in values from its own terraform.tfvars file. Promoting a change from dev to prod means changing directory and applying again. You never edit module code to deploy an environment.

Step 1: solve the chicken and egg problem

Terraform keeps a state file, which is its record of everything it has created. By default that file sits on your laptop, which is fine until your laptop is not around or a teammate runs an apply and now there are two versions of the truth. The fix is remote state: store the file in an S3 bucket instead.

But here is the catch. The S3 bucket has to exist before Terraform can store state in it, and we want Terraform to create the bucket. Who creates the creator?

The answer is a small run once setup called the bootstrap pattern. The infrastructure/bootstrap/ folder is a tiny Terraform project with plain local state. Its only job is to create the state bucket. You run it once per AWS account and forget about it.

data "aws_caller_identity" "current" {}

resource "aws_s3_bucket" "this" {
  bucket = "${var.project}-statefile-${data.aws_caller_identity.current.account_id}"
}

Notice the bucket name. S3 bucket names are globally unique across every AWS account on the planet, so todo-app-statefile alone would probably be taken. The aws_caller_identity data source reads your AWS account ID at plan time and appends it, so the name is unique without hardcoding anything.

The bootstrap also turns on three protections for the bucket: versioning (every old copy of your state is kept, which is what saves you when an apply goes wrong), full public access blocking, and AES 256 encryption at rest.

Step 2: remote state with S3 native locking

With the bucket in place, every environment points at it through a backend.tf file. Same bucket, different key per environment:

terraform {
  backend "s3" {
    bucket       = "todo-app-statefile-<account-id>"
    key          = "dev/terraform.tfstate"
    region       = "ap-south-1"
    encrypt      = true
    use_lockfile = true
  }
}

Because dev, staging and prod each write to their own key, an apply in dev can never touch prod. They are fully isolated.

The use_lockfile = true line deserves attention. State locking stops two people from running apply at the same time and corrupting the state. For years the standard answer was a DynamoDB table just for the lock. Terraform 1.10 added native S3 locking: Terraform drops a lock object into the bucket itself while an apply runs. One less resource to create, pay for and explain.

Here is the bucket after all three environments have been initialized, one prefix per state file:

Step 3: the database module

The database module creates three things. A DB subnet group that tells RDS which subnets it can live in. A security group (a virtual firewall) allowing inbound MySQL on port 3306. And the RDS instance itself:

resource "aws_db_instance" "this" {
  identifier        = "${var.project}-${var.environment}-rds"
  db_name           = "TodoAppDb"
  engine            = "mysql"
  engine_version    = "8.0"
  allocated_storage = 20
  storage_type      = "gp3"
  instance_class    = "db.t3.micro"

  username = var.username
  password = var.password

  db_subnet_group_name   = aws_db_subnet_group.this.name
  vpc_security_group_ids = [aws_security_group.this.id]

  publicly_accessible = true
  skip_final_snapshot = true
}

MySQL 8.0 on a db.t3.micro with 20 GB of gp3 storage, which keeps it inside the free tier.

One honest admission. The security group currently allows 3306 from 0.0.0.0/0, meaning anywhere on the internet, and the instance is publicly accessible. That is a demo shortcut so the Lambda can reach the database without living inside the VPC. Production would put RDS in private subnets, place the Lambda inside the VPC, and scope the security group to the Lambda alone. The repo says this out loud in its trade offs section rather than pretending it is fine.

The module ends by outputting db_endpoint, db_name and db_port. Remember those. They are about to become someone else's inputs.

Step 4: the backend module, where most of the Terraform lives

This module packages the code, creates the Lambda, and builds the entire REST API. A few pieces are worth slowing down for.

Packaging code without a build script

data "archive_file" "this" {
  type        = "zip"
  source_dir  = "${path.module}/../../../backend"
  output_path = "${path.module}/../../build/function.zip"
}

Terraform zips the backend/ folder at plan time. The Lambda resource then sets source_code_hash = data.archive_file.this.output_base64sha256, which means the function only gets updated when the code actually changed. Edit a Terraform file and apply, the Lambda is untouched. Edit routes/tasks.js and apply, Terraform notices the hash changed and ships the new zip. No separate deploy pipeline for the backend at all.

The Lambda function

resource "aws_lambda_function" "this" {
  function_name    = "${var.project}-${var.environment}-lambda-function"
  handler          = "server.handler"
  runtime          = "nodejs22.x"
  timeout          = 30
  memory_size      = 512

  environment {
    variables = {
      DB_HOST     = var.db_endpoint
      DB_NAME     = var.db_name
      DB_PORT     = var.db_port
      DB_USER     = var.username
      DB_PASSWORD = var.password
    }
  }
}

Node.js 22, 512 MB of memory, 30 second timeout. The database connection details arrive as environment variables, and look where they come from: var.db_endpoint is the output of the database module. The environment folder wires them together like this:

module "backend" {
  source      = "../../modules/backend"
  db_endpoint = module.database.db_endpoint
  db_name     = module.database.db_name
  db_port     = module.database.db_port
  ...
}

This is the trick that makes the whole project build in the right order. Terraform sees that the backend needs an output from the database, so it builds the database first. Nobody wrote depends_on for that. The dependency graph falls out of the references themselves.

Eight API routes from one map

The API has four paths (/api, /api/health, /api/tasks, /api/tasks/{id}) and eight method and path combinations once you count the OPTIONS methods needed for CORS preflight. Writing eight method blocks and eight integration blocks by hand would be sixteen nearly identical resources. Instead, the routes live in one locals map:

locals {
  routes = {
    health_get    = { resource_id = aws_api_gateway_resource.health.id,     http_method = "GET" }
    tasks_get     = { resource_id = aws_api_gateway_resource.tasks.id,      http_method = "GET" }
    tasks_post    = { resource_id = aws_api_gateway_resource.tasks.id,      http_method = "POST" }
    task_put      = { resource_id = aws_api_gateway_resource.task_by_id.id, http_method = "PUT" }
    task_delete   = { resource_id = aws_api_gateway_resource.task_by_id.id, http_method = "DELETE" }
    # ...plus three OPTIONS entries
  }
}

resource "aws_api_gateway_method" "this" {
  for_each      = local.routes
  rest_api_id   = aws_api_gateway_rest_api.this.id
  resource_id   = each.value.resource_id
  http_method   = each.value.http_method
  authorization = "NONE"
}

The for_each loops over the map and stamps out one resource per entry. The integrations do the same, every one an AWS_PROXY integration pointing at the Lambda, which means API Gateway forwards the raw request and lets Express figure out the routing inside the function. Adding a new route later means adding one line to the map.

The silent API Gateway gotcha

Here is a bug that costs beginners hours. API Gateway keeps a deployed snapshot of your API separate from its configuration. You can change methods and integrations in Terraform, apply successfully, and API Gateway will keep serving the old snapshot without a single complaint. Everything looks green. Your change is just not live.

The fix is to force a redeploy whenever the routing changes:

resource "aws_api_gateway_deployment" "this" {
  rest_api_id = aws_api_gateway_rest_api.this.id

  triggers = {
    redeploy = sha1(jsonencode([
      aws_api_gateway_method.this,
      aws_api_gateway_integration.this,
    ]))
  }

  lifecycle {
    create_before_destroy = true
  }
}

The trigger is a hash of the routing config. Config changes, hash changes, Terraform creates a fresh deployment. The create_before_destroy lifecycle rule makes sure the new deployment exists before the old one goes away, so the stage never has a gap where the API is down.

Step 5: the frontend module

The Amplify app connects directly to the GitHub repo using a personal access token, with the build instructions written inline:

resource "aws_amplify_app" "this" {
  name         = "${var.project}-${var.environment}-amplify-app"
  repository   = var.repository
  access_token = var.github_access_token

  environment_variables = {
    VITE_API_URL = var.api_uri
  }
}

That VITE_API_URL line is my favorite detail in the project. The value comes from module.backend.api_invoke_url, the output of the backend module. So Terraform creates the API, learns its URL, and hands it to Amplify as a build time environment variable. The React app bakes it in during npm run build. The frontend learns its backend address from Terraform and nothing is ever hardcoded.

The module also adds a rewrite rule so deep links like /tasks/5 fall back to index.html instead of returning 404, which every single page app on Amplify needs, and a branch resource with auto build turned on. The branch stage is set with a conditional: var.environment == "prod" ? "PRODUCTION" : "DEVELOPMENT".

Step 6: the backend code that makes Express work in Lambda

A normal Express app ends with app.listen(3000). There is no port to listen on inside Lambda, so the app is wrapped instead:

const serverlessExpress = require('@vendia/serverless-express');
const app = express();
// routes...
exports.handler = serverlessExpress({ app });

The @vendia/serverless-express package translates the event API Gateway sends into the request object Express expects, and translates the response back. Your Express code does not change at all.

One more pattern worth stealing. The database table has to exist before the first query, but you do not want to create it on every request. The code caches the promise:

let dbReady = null;
app.use('/api/tasks', async (req, res, next) => {
  try {
    dbReady = dbReady || sequelize.sync();
    await dbReady;
    next();
  } catch (err) {
    dbReady = null; // don't cache a failed attempt
    res.status(500).json({ error: err.message });
  }
}, tasksRouter);

The first request in each Lambda container runs sequelize.sync() which creates the tasks table if needed. Every later request awaits the already resolved promise, which costs nothing. And if the sync fails, the cache resets so the next request tries again instead of being stuck with a failed promise forever. There is no manual schema step anywhere in the deployment.

Deploying it yourself

You need an AWS account with credentials configured, Terraform 1.15 or newer, and a GitHub personal access token that can read your fork and create webhooks.

First, bootstrap the state bucket, once per account:

cd infrastructure/bootstrap
terraform init
terraform apply

Second, edit backend.tf in each environment folder and set the bucket name to what the bootstrap printed. Backend blocks cannot take variables, so this one value is a literal.

Third, deploy an environment:

cd infrastructure/environments/dev
cp terraform.tfvars.example terraform.tfvars   # fill in DB password, repo URL, token, branch
terraform init
terraform plan
terraform apply

RDS takes a few minutes to come up. The result is 28 resources from one command:

Fourth, trigger the first frontend build. Amplify only builds on new commits, so push something to the configured branch after the apply. An empty commit does the job.

Finally, check it is alive:

curl https://<api-id>.execute-api.ap-south-1.amazonaws.com/dev/api/health
# {"status":"ok","database":"connected"}

What actually gets created

Everything below came from the same modules, applied once per environment folder. Three Lambda functions on Node.js 22:

Three regional REST APIs, each with a stage named after its environment:

A MySQL instance per environment:

And the Amplify apps, connected to GitHub and building on every push:

Notice what did not happen. Nobody opened the RDS console to click Create database three times. The staging environment is not "dev, but with some settings someone changed by hand last month." All three are provably identical because they came from identical code.

Honest trade offs

A few shortcuts were taken deliberately, and it is worth knowing what production would change. The open security group and public RDS would become private subnets with the Lambda inside the VPC. Database credentials would move from Lambda environment variables into Secrets Manager. The single AZ db.t3.micro with skip_final_snapshot would become Multi AZ with deletion protection. And the hardcoded bucket name in backend.tf would be handled with partial backend config through terraform init -backend-config.

The roadmap includes a GitHub Actions pipeline that runs terraform fmt, validate and plan on pull requests and applies on merge, plus tflint and checkov for static checks.

Conclusion

The todo app is forgettable. The patterns are not. The bootstrap trick for remote state, S3 native locking without a DynamoDB table, modules whose outputs feed the next module's inputs so the build order takes care of itself, for_each over a map instead of sixteen copy pasted resources, and a hash trigger that stops API Gateway from silently serving stale config. Every one of these shows up in real production Terraform, and this repo is a small enough place to learn them without drowning.

Clone it, deploy dev, then run terraform destroy when you are done exploring. RDS is the only always on cost here, and tearing it down is one command, the same way bringing it up was.

Repo: github.com/TanseerS/serverless-web-application

Contact

Questions, feedback, or stuck on a step? Reach me at khantanseer43@gmail.com. Happy to help.

Go Live in Minutes: Building Your First Live Stream with Amazon IVS

Tanseer — Tue, 07 Jul 2026 05:08:24 +0000

The same tech that powers Twitch, minus the servers, minus the headache

A few weeks ago I wanted to test a live streaming idea. My first thought was the usual one. I will need a media server, something like Wowza or a self hosted RTMP setup on EC2, then a CDN in front of it, then hours of tuning latency. Before I even opened the console I was already tired.

Then I found Amazon IVS, and honestly, the whole thing took less time than making tea.

If you are a developer who has never touched live streaming on AWS before, this post is for you. By the end of it you will know exactly what IVS is, why it is different from rolling your own setup, and how to go live in four steps.

What exactly is Amazon IVS

Amazon Interactive Video Service, or IVS for short, is a fully managed live streaming service from AWS. Fully managed means AWS runs everything behind the scenes. You do not create servers, you do not install streaming software on the cloud, and you do not worry about scaling.

Here is the part that made me sit up. IVS runs on the exact same infrastructure that powers Twitch. Twitch is one of the biggest live streaming platforms in the world, handling millions of concurrent viewers every single day. When you create an IVS channel, your tiny test stream rides on that same battle tested backbone.

So instead of you building a smaller, buggier version of Twitch infrastructure, AWS just hands you Twitch infrastructure as an API.

The killer feature: ultra low latency

If you have ever streamed with a typical setup, you know the pain. You wave at the camera, and viewers see the wave 15 to 30 seconds later. That delay is called latency, the time it takes for video to travel from your camera to the viewer's screen.

IVS brings that down to under 3 seconds.

Why does that matter? Because under 3 seconds, a stream stops feeling like a broadcast and starts feeling like a conversation. Viewers can react in chat and you can respond almost instantly.

IVS also lets you sync interactive elements with the video. Think live chat, polls, quizzes, or even shopping buttons that appear exactly when the host says "buy this now." This is done through something called timed metadata, small pieces of data you inject into the stream that arrive on the viewer's screen in sync with the video frame. For a marketplace or live commerce product, this is gold.

How is it serverless

The word serverless gets thrown around a lot, so let me be specific about what it means here.

There are no servers for you to manage. No EC2 instances, no patching, no capacity planning. You create a channel and AWS handles the rest.

It scales automatically. Whether 5 people show up to your stream or 5 million, you do nothing. The same channel handles both. There is no "upgrade your instance" moment at 2 AM when your stream suddenly goes viral.

You pay only for what you use. IVS pricing is based on two things: hours of video you send into the service (input) and hours of video delivered to viewers (output). No stream running means no bill running. This is the same pay as you go model that makes Lambda and other serverless tools so friendly for side projects.

Going live in 4 steps

This is the part that genuinely surprised me. Here is the entire setup.

Step 1: Create a channel

Log in to the AWS Console and search for Amazon IVS. Click Create channel, give it a name, and keep the default configuration for now. That is it. AWS provisions the channel in seconds.

Step 2: Copy your Ingest Server and Stream Key

Open your new channel and you will see two important values. The Ingest Server is the address your video gets sent to, something like rtmps://xxxxxx.global-contribute.live-video.net. The Stream Key is a secret token that proves the stream belongs to you. Treat the stream key like a password. Anyone who has it can broadcast on your channel.

Step 3: Paste them into OBS Studio

OBS Studio is free, open source broadcasting software and the de facto standard for streaming. Download it, open Settings, go to Stream, and choose Custom as the service. Paste the Ingest Server into the Server field and the Stream Key into the Stream Key field. Save.

Step 4: Click Start Streaming

Back on the main OBS screen, add a source (your webcam or screen), then hit Start Streaming. Switch to the AWS Console, open your channel, and you will see yourself live in the built in preview player within seconds.

Four steps. No infrastructure. You are now running on Twitch grade streaming systems.

Where to go from here

The console preview is just the beginning. IVS gives you a playback URL for every channel, and AWS provides a player SDK for web, iOS, and Android. That means you can drop your live stream into your own React app or mobile app with a few lines of code. Add the IVS chat feature and timed metadata, and you have a real interactive streaming product without touching a single server.

Conclusion

Live streaming used to be a project measured in weeks. With Amazon IVS it is a project measured in minutes. You get the infrastructure behind Twitch, latency under 3 seconds, automatic scaling from a handful of viewers to millions, and a bill that only grows when your stream does. If you have been putting off a live video feature because the plumbing felt scary, this is your sign to create a channel and press Start Streaming.

Let's connect

If you tried this and got stuck somewhere, or if you built something cool with IVS, I would love to hear about it. Reach out at khantanseer43@gmail.com.

Terraform guide to IAM Policies: The Two Cases You Need to Know

Tanseer — Wed, 01 Jul 2026 11:21:21 +0000

When to attach a ready made policy and when to write your own

When you first start writing IAM setup in Terraform, the examples online can feel like they contradict each other. One tutorial writes a long block called aws_iam_policy_document. Another just pastes a single line with a long string inside it. Both of them work, so which one is correct?

The answer is that both are correct. They simply solve two different problems. Once you can tell which situation you are in, IAM in Terraform stops feeling random and starts feeling obvious.

Before we get into the two cases, here are the words you will keep seeing:

IAM stands for Identity and Access Management. It is the AWS service that decides who is allowed to do what. An IAM role is an identity that an AWS service (like a Lambda function) takes on so it can act with certain permissions. An IAM policy is the document that lists which actions are allowed or denied. An ARN, short for Amazon Resource Name, is the unique address of a resource in AWS.

With that out of the way, let us look at the two cases.

Case 1: AWS Managed Policy (attach directly)

A managed policy is a policy that AWS already wrote and maintains for you. If AWS already provides the exact permissions you need, you do not create your own aws_iam_policy_document. You just point your role at the policy AWS already made.

For example, giving a Lambda function permission to write logs is a very common need, and AWS has a ready made policy for it:

resource "aws_iam_role_policy_attachment" "logs" {
  role       = aws_iam_role.this.name
  policy_arn = "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
}

Another common one is letting a Lambda function run inside a VPC (a Virtual Private Cloud, which is your own private network in AWS):

resource "aws_iam_role_policy_attachment" "vpc" {
  role       = aws_iam_role.this.name
  policy_arn = "arn:aws:iam::aws:policy/service-role/AWSLambdaVPCAccessExecutionRole"
}

Notice what is happening here. You are not writing any permissions yourself. You are taking an existing AWS managed policy, referring to it by its ARN, and attaching it to your role. That is the whole job.

Case 2: Your own custom permissions

Sometimes AWS does not have a managed policy that matches exactly what you need. Maybe you want a Lambda function to read and write files in one specific S3 bucket and nothing else. In that case you build the policy yourself, and it takes three small steps.

Step 1: Create the policy document

First you describe the permissions. This aws_iam_policy_document is a data source, which means it does not create anything in AWS on its own. It is a way to write your permission rules in clean Terraform instead of raw JSON.

data "aws_iam_policy_document" "s3_access" {
  statement {
    effect = "Allow"

    actions = [
      "s3:GetObject",
      "s3:PutObject"
    ]

    resources = [
      aws_s3_bucket.uploads.arn,
      "${aws_s3_bucket.uploads.arn}/*"
    ]
  }
}

Reading this in plain English: allow the actions of getting an object and putting an object, and only on the uploads bucket and everything inside it.

Step 2: Create the IAM policy

Now you turn that document into a real IAM policy that exists in AWS. The policy field pulls in the JSON produced by the document from Step 1.

resource "aws_iam_policy" "s3_access" {
  name   = "${var.project}-${var.environment}-s3-access"
  policy = data.aws_iam_policy_document.s3_access.json
}

Step 3: Attach it

Finally you attach the policy you just created to your role. This looks almost the same as Case 1, but the ARN now comes from your own policy instead of an AWS managed one.

resource "aws_iam_role_policy_attachment" "s3_access" {
  role       = aws_iam_role.this.name
  policy_arn = aws_iam_policy.s3_access.arn
}

The mental model

The easiest way to remember the difference is this. In Case 1 the policy already exists, so you skip straight to attaching. In Case 2 you have to build the policy first, then attach it.

The custom flow always moves in the same direction:

Policy Document
      │
      ▼
IAM Policy
      │
      ▼
Attach to Role

You write the document, the document becomes a policy, and the policy gets attached to the role. If you ever feel lost, just ask yourself one question: does AWS already have a policy for this? If yes, attach it directly. If no, build your own with the three steps above.

Conclusion

IAM in Terraform is not as confusing as it first looks. There are really only two situations. When AWS already provides the exact permissions, you attach a managed policy by its ARN in a single resource. When you need something specific that AWS does not offer, you create a policy document, turn it into an IAM policy, and attach that.

Keep that fork in mind and you will always know which approach the moment fits, instead of copying an example and hoping it is the right one.

Let us connect

If you have questions or want to share how you handle IAM in your own projects, reach out at khantanseer43@gmail.com. I am always happy to talk Terraform and AWS.

Lambda MicroVMs Are Kind of Insane: Tiny Virtual Machines That Boot in Milliseconds and Sleep When You Are Not Looking

Tanseer — Tue, 30 Jun 2026 04:53:28 +0000

A beginner friendly guide to the Firecracker powered microVMs behind AWS Lambda, the new suspend and resume superpower, and the big question everyone keeps asking: is this still serverless?

Introduction

Imagine a virtual computer so small and so fast that it starts up in less time than it takes you to blink. Now imagine thousands of them running on a single physical server, each one completely walled off from its neighbours, and each one quietly going to sleep the moment it has nothing to do, so that you stop paying for it.

That is not a science fiction pitch. That is roughly what is happening inside AWS Lambda today, and the newer microVM capabilities push the idea even further.

In this post we will start from zero. We will explain what a microVM even is, meet the tiny engine that makes the whole thing possible, walk through what happens when a single request comes in, and then look at the features that make people say this is a little bit crazy. By the end you will be able to answer the question that is dividing the cloud community: after all of this, is Lambda even serverless anymore?

Let us get into it.

So What Exactly Is a Lambda MicroVM?

A microVM is a tiny, lightweight virtual machine. It behaves like its own small computer with a strong security wall around it, but it throws away almost all the heavy baggage of a normal machine, so it starts in milliseconds and uses very little memory. Picture a tiny, fast, single purpose box that appears, does one job, and disappears.

Here is the part that surprises most people. Lambda has always run your code inside these microVMs. Every time your function runs, AWS quietly spins up a tiny isolated machine just for you. The recent excitement is about giving those microVMs new powers, such as the ability to pause when idle and to be reached directly over the internet. So when people say "Lambda MicroVMs," they are really talking about this next chapter of an idea that has been humming away under the hood for years.

Meet Firecracker, the Tiny Engine Doing All the Work

Powering all of this is Firecracker, a technology that AWS built and then released to the public as open source. AWS created it specifically to run microVMs safely at massive scale, and it is the reason a tiny machine can boot in around 125 milliseconds while still keeping the strong security wall of a real virtual machine. This same AWS made engine quietly runs underneath both Lambda and AWS Fargate.

How It Works: The Journey of a Single Request

Let us follow one request from start to finish so the architecture clicks.

You upload your code. This can be a simple zip file or a container image. You also choose how much memory the function gets, which in turn decides how much CPU power it receives.
Something triggers it. A user hits an endpoint, a file lands in storage, a message arrives on a queue, and Lambda decides your function needs to run.
A microVM appears. AWS spins up a Firecracker microVM with your chosen runtime and your code inside. This little box is called the execution environment. If a fresh one has to be created from scratch, that small delay is known as a cold start.
Your code runs. The environment initialises, your handler runs, and a response goes back.
The environment is kept around. Instead of throwing the microVM away immediately, Lambda often keeps it ready for a little while. If another request arrives soon, it reuses the same warm environment and skips the startup cost entirely. This is called a warm start, and it is why the second request to a function usually feels much faster than the first. So the architecture is really a fleet of these tiny machines being created, reused, and recycled constantly, all managed for you so you never touch a server.

Each One Gets Its Own Front Door: Dedicated HTTP Endpoints

An endpoint is just a web address that something can send a request to. HTTP is the everyday language of the web that browsers and apps use to talk to servers.

For a long time, if you wanted your Lambda function to be reachable over the web, you usually had to put another service in front of it to handle the incoming traffic. The newer approach gives a function its own dedicated HTTPS endpoint, a clean and direct web address that points straight at it. No extra plumbing in between.

The address looks something like this:

https://your-unique-id.lambda-url.your-region.on.aws

You can keep it open or lock it down so only authorised callers get through. For simple use cases, such as a webhook that some other service calls, or a small backend for a single page app, this is wonderfully direct. Your request lands on a private little microVM that exists just to answer you.

The Real Magic: Suspend and Resume

This is the feature that earns the word "crazy."

To understand it, you need one more idea: a snapshot. A snapshot is a complete saved picture of a running machine at a single moment in time, including everything sitting in its memory. If you take a snapshot and then restore it later, the machine wakes up exactly where it left off, as if no time had passed at all.

Firecracker can do this with microVMs. It can freeze a running microVM, save its full state, and bring it back to life almost instantly.

Now think about what that unlocks. Traditionally you had two bad choices. Either you kept a server running all the time so it was ready to respond instantly, and you paid for every idle second. Or you shut it down to save money and then suffered a slow start whenever traffic came back.

Suspend and resume gives you a third option. The microVM can stay paused while nothing is happening, and the headline benefit is simple: you are not paying for a fully running machine while it sits idle. When a request finally arrives, it wakes up quickly and carries on. You get the responsiveness of an always on server with a cost profile much closer to pay only for what you use.

That is the holy grail people have chased for years. Be instantly ready, but only pay when you are actually doing work.

Lambda MicroVM vs EC2 vs Classic Lambda

It helps to see where this sits between the two options most people already know.

EC2 is a rented virtual server that you control and manage yourself. Classic Lambda is short lived, event driven functions with a hard time limit. The microVM approach lands in the middle, borrowing the best of each.

Feature	EC2	Classic Lambda	Lambda MicroVM
What you manage	The whole server	Just your code	Just your code
How long it can run	As long as you like	Up to 15 minutes per run	Designed for longer and more stateful work than the 15 minute limit
Startup speed	Slow, minutes	Fast, with occasional cold starts	Fast, with quick resume from a paused state
Cost when idle	You keep paying while it runs	Nothing, it is not running	The big win is not paying for a fully running machine while idle
Scaling	You set it up	Automatic, scales to zero	Automatic, scales to zero
Best feeling	Total control	Pure simplicity	Control and simplicity together

The short version is this. EC2 gives you full control but you babysit it and pay for idle time. Classic Lambda is beautifully simple but tightly constrained. The microVM model tries to give you longer running, more flexible compute that still scales down to nothing when no one is using it.

What Is This Actually Good For?

A new tool is only exciting if it solves real problems. Here is where this approach shines.

It is great for long running jobs that do not fit neatly into a short window, such as heavier data processing or video work that would bump into the classic 15 minute ceiling.

It suits stateful sessions, meaning work where the machine needs to remember something between requests rather than starting fresh every single time.

It is a natural fit for AI and machine learning inference, where a large model can be expensive to load. You load it once, let the machine pause when traffic is quiet, and resume instantly when the next request comes, instead of paying to keep a big GPU style box running all day.

It works beautifully for interactive sandboxes and code execution, the kind of thing where each user needs their own safe, isolated little environment that spins up fast and tears down cleanly.

And it is useful for bursty traffic, where requests come in unpredictable spikes. You get instant readiness during the spike and near zero cost during the long quiet stretches in between.

When would you not bother? If your workload is genuinely tiny and event driven, classic Lambda is still the simplest possible answer. And if you need deep control over the operating system, custom networking, or hardware that must run nonstop, a traditional server like EC2 is still the right call.

What About the Price?

Cost is usually the deciding factor, so let us compare the three billing styles in plain terms. To follow along, you need one bit of vocabulary: a GB second is just one gigabyte of memory used for one second. It is the unit Lambda uses to measure how much work you actually consumed.

EC2 charges you for the time the server is switched on, whether or not anyone is using it. A box running quietly overnight with zero visitors still appears on your bill. You pay for capacity, not for usage.

Classic Lambda charges you in two parts: a tiny amount for each request, plus an amount for compute measured in those GB seconds. If nobody calls your function, it costs you nothing. You pay for usage, not for capacity.

The microVM model aims for the best of both. You get a more capable, longer lived machine, but the key promise is that you are not charged for a fully running VM while it is paused and idle. You pay mainly for the active time when it is actually doing something.

Important: cloud pricing changes often, and the exact rates for these options vary by region and configuration. Rather than trust any single number you read in a blog, including this one, always check the current AWS pricing pages before you make a decision.

So Is It Still Serverless?

Here is the fun part, and it is genuinely a matter of debate.

The word serverless never meant there were no servers. It meant you, the developer, never had to think about them. The classic definition came with a few expectations: your code is event driven and short lived, it scales automatically all the way down to zero, you never manage infrastructure, and you only pay for what you use.

Now look at what these microVMs offer. Long running. Able to hold state. Reachable through their own dedicated web address. Sitting paused and ready rather than vanishing completely. To some people that starts to look an awful lot like a small, managed server wearing a serverless costume.

The case for "yes, it is still serverless" is strong. You still never patch an operating system. It still scales to zero. You still pay for value rather than for idle capacity. By the spirit of the original promise, nothing has been broken.

The case for "no, the purity is gone" is also fair. The original idea of a tiny, stateless, event triggered function that lives for a moment and dies has clearly been stretched into something bigger and more persistent.

The honest answer is probably this. Serverless is becoming a spectrum rather than a strict box. On one end you have pure functions that flash in and out of existence. On the other you have long lived managed machines. These microVMs live somewhere in the comfortable middle, and they are quietly redefining what the word can mean. Maybe the better question is not "is this serverless," but "does this let me ship without babysitting infrastructure." If the answer is yes, the label matters a lot less than the freedom.

Wrapping Up

Lambda has always been powered by tiny, fast, isolated microVMs running on Firecracker. What is changing is how clever those microVMs are getting. They can now boot in a blink, answer the web directly, and most remarkably of all, doze off when idle so you are not paying for a machine that is doing nothing.

Whether you call it serverless, managed compute, or something brand new, one thing is clear. The line between a server and a function is getting blurrier, and that is a great problem for developers to have. The more the cloud handles for us, the more time we get to spend building the things we actually care about.

If you are new to AWS, the best next step is to try spinning up a simple function, give it a dedicated endpoint, and call it from your browser. Seeing your own tiny microVM answer back is the moment the whole idea finally clicks.

Let Us Connect

If you found this helpful, have a question, or spot something you want to discuss, I would love to hear from you. Reach out at khantanseer43@gmail.com.

Who Wins the Variable Fight in Terraform?

Tanseer — Tue, 30 Jun 2026 04:39:15 +0000

A beginner friendly guide to understanding which value Terraform actually picks when the same variable is set in many places

Introduction

When you start writing Terraform, you quickly learn that you can set the same variable in many different ways. You can give it a default value, you can put it in a file, you can pass it on the command line, and you can even set it through your computer's environment.

This is great for flexibility, but it also raises a confusing question. If the same variable gets a value from five different places, which value does Terraform actually use?

This is called variable precedence. Precedence simply means the order of priority. It is the set of rules Terraform follows to decide who wins when there is a conflict.

In this post we will walk through every way you can set a variable, then look at the exact order Terraform uses to pick a winner. We will use small, repeatable examples so you can try each one yourself.

Quick note on the words we will use. A variable is just a named value you can reuse, like a name tag on a box. A value is what is inside that box. Precedence is the priority order Terraform uses when two boxes have the same name tag but different contents.

First, the ways you can set a variable

Before we talk about who wins, let us list all the players in the game. There are six common ways to give a variable a value in Terraform.

The first is a default value. You write this directly inside the variable block in your code.

variable "region" {
  type    = string
  default = "us-east-1"
}

The second is an environment variable. An environment variable is a value stored in your computer's shell, outside of Terraform. Terraform reads any environment variable that starts with TF_VAR_ followed by the variable name.

export TF_VAR_region="us-west-2"

The third is a file called terraform.tfvars. A tfvars file is a plain file where you list variable names and the values you want to give them. Terraform loads this file automatically without you doing anything.

region = "ap-south-1"

The fourth is a file called terraform.tfvars.json. This is the same idea as above, but written in JSON format instead. JSON is just another way of writing structured data using curly braces and quotes.

{
  "region": "eu-west-1"
}

The fifth is any file ending in .auto.tfvars or .auto.tfvars.json. The word auto here means automatic. Terraform loads these files for you automatically too, just like the standard tfvars file. A common example name is prod.auto.tfvars.

region = "ca-central-1"

The sixth is a command line flag. A flag is an extra instruction you type when running a command. You can pass a value directly with -var, or point to a specific file with -var-file.

terraform apply -var="region=sa-east-1"

The order that decides the winner

Here is the part everyone wants to know. Terraform checks these sources in a fixed order, from lowest priority to highest priority. A source that comes later in the list overrides anything set earlier.

Here is the order from weakest to strongest.

The default value inside the variable block. This is the fallback that is used only when nothing else provides a value.
Environment variables that start with TF_VAR_.
The terraform.tfvars file.
The terraform.tfvars.json file.
Any files ending in .auto.tfvars or .auto.tfvars.json, loaded in alphabetical order of their file names.
The -var and -var-file flags on the command line, applied in the exact order you type them. The simplest way to remember this is that the command line always wins, and the default value always loses. Everything else sits in the middle.

A full example you can run

Let us prove the rules with a tiny project. Create a folder and add the following files.

First the variable declaration in a file called variables.tf.

variable "region" {
  type    = string
  default = "us-east-1"
}

Next a file called main.tf that just prints the value back to us using an output. An output is simply a way for Terraform to show you a value after it runs.

output "selected_region" {
  value = var.region
}

Now run this command to see the result.

terraform apply

At this point only the default exists, so Terraform will show us-east-1. The default wins because nobody else is competing.

Now add a file called terraform.tfvars with this line.

region = "ap-south-1"

Run apply again. This time Terraform shows ap-south-1. The tfvars file beat the default because it has higher priority.

Now add a file called prod.auto.tfvars with this line.

region = "ca-central-1"

Run apply once more. Now the answer becomes ca-central-1. The auto tfvars file sits higher than the standard tfvars file, so it takes over.

Finally, run apply with a command line flag.

terraform apply -var="region=sa-east-1"

The result is now sa-east-1. The command line beats every file, so it has the final say.

What happens with two auto tfvars files

A common question is what happens when you have more than one auto tfvars file and both set the same variable. The answer is alphabetical order.

Imagine you have two files named a.auto.tfvars and b.auto.tfvars, and both set the region. Terraform loads a.auto.tfvars first and b.auto.tfvars second. Because the later one overrides the earlier one, the value inside b.auto.tfvars wins.

So the file whose name comes later in the alphabet has the higher priority. This is easy to forget, so name your files carefully if order matters to you.

Command line order also matters

There is one more detail worth knowing. When you pass several flags on the command line, Terraform applies them left to right, and the rightmost one wins.

terraform apply -var="region=us-east-1" -var="region=us-west-2"

In this case the final value is us-west-2, because it appears last. The same rule applies when you mix -var and -var-file. Whatever comes last on the line takes priority.

A quick warning about missing values

If a variable has no default and you never give it a value through any source, Terraform will stop and ask you to type the value in. This is a safety feature so you never accidentally run with a blank value.

If you want a variable to be optional, give it a sensible default. If you want it to be required on purpose, leave the default out so Terraform forces someone to provide it.

Conclusion

Variable precedence in Terraform looks confusing at first, but it follows one calm and predictable rule. Lower priority sources set the value, and higher priority sources override it.

The order, from weakest to strongest, is the default value, then environment variables, then the standard tfvars file, then the JSON tfvars file, then the auto tfvars files in alphabetical order, and finally the command line flags which always win.

Once this clicks, you gain real control over your setup. You can keep safe defaults in your code, store normal settings in tfvars files, and override anything on the fly from the command line when you need to. That is the whole point of precedence, giving you layers you can trust.

Try the small example above and watch the value change with each step. Seeing it happen is the fastest way to make this stick.

Get in touch

If you have questions or want me to cover another Terraform topic, reach out at khantanseer43@gmail.com. I am always happy to help fellow builders.

Who Wins the Variable Fight in Terraform?

Tanseer — Wed, 24 Jun 2026 11:02:27 +0000

A beginner friendly guide to understanding which value Terraform actually picks when the same variable is set in many places

Introduction

This is great for flexibility, but it also raises a confusing question. If the same variable gets a value from five different places, which value does Terraform actually use?

This is called variable precedence. Precedence simply means the order of priority. It is the set of rules Terraform follows to decide who wins when there is a conflict.

Quick note on the words we will use. A variable is just a named value you can reuse, like a name tag on a box. A value is what is inside that box. Precedence is the priority order Terraform uses when two boxes have the same name tag but different contents.

First, the ways you can set a variable

Before we talk about who wins, let us list all the players in the game. There are six common ways to give a variable a value in Terraform.

The first is a default value. You write this directly inside the variable block in your code.

variable "region" {
  type    = string
  default = "us-east-1"
}

export TF_VAR_region="us-west-2"

region = "ap-south-1"

{
  "region": "eu-west-1"
}

region = "ca-central-1"

The sixth is a command line flag. A flag is an extra instruction you type when running a command. You can pass a value directly with -var, or point to a specific file with -var-file.

terraform apply -var="region=sa-east-1"

The order that decides the winner

Here is the order from weakest to strongest.

The default value inside the variable block. This is the fallback that is used only when nothing else provides a value.
Environment variables that start with TF_VAR_.
The terraform.tfvars file.
The terraform.tfvars.json file.
Any files ending in .auto.tfvars or .auto.tfvars.json, loaded in alphabetical order of their file names.
The -var and -var-file flags on the command line, applied in the exact order you type them. The simplest way to remember this is that the command line always wins, and the default value always loses. Everything else sits in the middle.

A full example you can run

Let us prove the rules with a tiny project. Create a folder and add the following files.

First the variable declaration in a file called variables.tf.

variable "region" {
  type    = string
  default = "us-east-1"
}

Next a file called main.tf that just prints the value back to us using an output. An output is simply a way for Terraform to show you a value after it runs.

output "selected_region" {
  value = var.region
}

Now run this command to see the result.

terraform apply

At this point only the default exists, so Terraform will show us-east-1. The default wins because nobody else is competing.

Now add a file called terraform.tfvars with this line.

region = "ap-south-1"

Run apply again. This time Terraform shows ap-south-1. The tfvars file beat the default because it has higher priority.

Now add a file called prod.auto.tfvars with this line.

region = "ca-central-1"

Run apply once more. Now the answer becomes ca-central-1. The auto tfvars file sits higher than the standard tfvars file, so it takes over.

Finally, run apply with a command line flag.

terraform apply -var="region=sa-east-1"

The result is now sa-east-1. The command line beats every file, so it has the final say.

What happens with two auto tfvars files

A common question is what happens when you have more than one auto tfvars file and both set the same variable. The answer is alphabetical order.

So the file whose name comes later in the alphabet has the higher priority. This is easy to forget, so name your files carefully if order matters to you.

Command line order also matters

There is one more detail worth knowing. When you pass several flags on the command line, Terraform applies them left to right, and the rightmost one wins.

terraform apply -var="region=us-east-1" -var="region=us-west-2"

In this case the final value is us-west-2, because it appears last. The same rule applies when you mix -var and -var-file. Whatever comes last on the line takes priority.

A quick warning about missing values

If you want a variable to be optional, give it a sensible default. If you want it to be required on purpose, leave the default out so Terraform forces someone to provide it.

Conclusion

Variable precedence in Terraform looks confusing at first, but it follows one calm and predictable rule. Lower priority sources set the value, and higher priority sources override it.

Try the small example above and watch the value change with each step. Seeing it happen is the fastest way to make this stick.

Get in touch

If you have questions or want me to cover another Terraform topic, reach out at khantanseer43@gmail.com. I am always happy to help fellow builders.

The Bucket That Must Exist Before Everything Else

Tanseer — Tue, 23 Jun 2026 10:31:31 +0000

How I created a dedicated S3 bucket to hold my Terraform remote backend, and every small lesson I learned along the way

Introduction

When you start working with Terraform, one of the first real puzzles you hit is a classic chicken and egg situation.

Terraform needs a safe place to store its state file. The state file is a record of everything Terraform has created for you, like a memory of your infrastructure. The most common safe place to keep this file is an S3 bucket on AWS. S3 stands for Simple Storage Service, and a bucket is just a container where you store files in the cloud.

Here is the puzzle. Terraform wants to store its state in an S3 bucket, but that bucket does not exist yet. So how do you create the very bucket that Terraform itself needs in order to remember what it created?

The answer is a small, separate Terraform setup called a bootstrap. You run it once, it creates the bucket, and then you never touch it again. In this post I will walk you through exactly how I built mine for my project, and I will explain every piece of jargon the first time it shows up. This is written for developers who are new to AWS, so nothing here assumes prior cloud knowledge.

What a Bootstrap Actually Is

A bootstrap is a tiny Terraform project whose only job is to create the foundation that the main project depends on. In our case the foundation is a single S3 bucket that will store the remote backend.

A backend is simply where Terraform keeps its state. A local backend keeps the state on your own computer. A remote backend keeps the state in the cloud, which is much safer and lets a whole team share it. We want a remote backend, and to have a remote backend we first need the bucket to hold it.

The golden rule of a bootstrap is this. Run it once, before anything else, then leave it alone forever.

The Folder Structure

I kept the bootstrap deliberately small. Here is the entire layout.

bootstrap/
├── main.tf        # everything in one file
├── variables.tf
└── README.md

That is it. Three files. A lot of beginners expect more files because larger Terraform projects have many. But a bootstrap is special, and the missing files are missing on purpose. Let me explain why, because the reasoning here taught me a lot.

Why there is no backend.tf

A file called backend.tf is where you would normally tell Terraform to use a remote backend. We do not have one here, and that is intentional. The bucket does not exist yet, so we cannot point Terraform at it. Instead the bootstrap uses local state, meaning the state file sits on our machine for now. This is the one time local state is the correct choice.

Why there is no outputs.tf

An output in Terraform is a value you ask Terraform to print or pass along after it runs, like the name of something it just created. You might think we need to output the bucket name so the main project can read it. We do not. The bucket name is already known from our variables, and the backend.tf file in the main project is static. Static means it is fixed text that cannot read or react to outputs. So an output here would serve no purpose.

Why there is no providers.tf

A provider is the plugin that lets Terraform talk to a specific cloud, in this case AWS. In big projects people split the provider setup into its own providers.tf file for tidiness. Our bootstrap is so small that everything fits comfortably in main.tf, so there is no need to split it.

The Order of Blocks Inside main.tf

Inside main.tf the order of things matters for readability. I followed this order from top to bottom.

terraform block → provider block → data source → resources

The terraform block holds settings about Terraform itself, such as which version to use. The provider block configures AWS. A data source reads information that already exists rather than creating anything new. Resources are the things Terraform actually creates for you.

The Four Resources, and Nothing More

The bootstrap creates exactly four resources. Each one has a clear reason to exist.

The first is aws_s3_bucket. This is the bucket itself. I named it this in the code, which is a common Terraform convention when there is only one of something.

The second is aws_s3_bucket_versioning. Versioning means S3 keeps old copies of a file whenever it changes. This matters a lot for a state file. If the state ever gets corrupted, versioning lets you roll back to a healthy earlier copy. It is your safety net.

The third is aws_s3_bucket_public_access_block. This shuts the bucket off from the public internet. It has four separate arguments, and I set all four to true. The reason I did this explicitly, rather than trusting AWS to do it for me, leads to one of the biggest lessons in this whole project, which I will come back to shortly.

The fourth is aws_s3_bucket_server_side_encryption_configuration. Encryption scrambles the stored data so that only authorized access can read it. I chose AES256, which is a strong and standard encryption method. Again I set this explicitly rather than relying on any default.

Here is the complete main.tf so you can see how all four resources fit together with the terraform block, the provider, and the data source.

# Add AWS Provider
terraform {
  required_version = "~> 1.15.0"
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 6.0"
    }
  }
}

# Configure the AWS Provider
provider "aws" {
  region = var.region
}

# AWS account id
data "aws_caller_identity" "current" {}

# Create s3 bucket
resource "aws_s3_bucket" "this" {
  bucket = "${var.project}-statefile-${data.aws_caller_identity.current.account_id}"
}

# Versioning bucket
resource "aws_s3_bucket_versioning" "this" {
  bucket = aws_s3_bucket.this.id
  versioning_configuration {
    status = "Enabled"
  }
}

# Block public access
resource "aws_s3_bucket_public_access_block" "this" {
  bucket = aws_s3_bucket.this.id

  block_public_acls       = true
  block_public_policy     = true
  ignore_public_acls      = true
  restrict_public_buckets = true
}

# Encryption
resource "aws_s3_bucket_server_side_encryption_configuration" "this" {
  bucket = aws_s3_bucket.this.id

  rule {
    apply_server_side_encryption_by_default {
      sse_algorithm = "AES256"
    }
  }
}

Notice the small detail inside each resource. The line bucket = aws_s3_bucket.this.id is how Terraform links resources together. It tells the versioning, public access, and encryption resources to attach themselves to the exact bucket we just created, rather than to some other bucket. This linking is how Terraform understands the order to build things in.

Reference image placeholder: a screenshot of the four resources listed in main.tf.

The Variables File

A variable is a named value you can reuse and change in one place. Here is the full variables.tf.

variable "region" {
  type        = string
  description = "AWS region where the statefile must exists"
  default     = "ap-south-1"
}

variable "project" {
  type        = string
  description = "Name of the project"
  default     = "todo-app"
}

The first variable is region. A region is the geographic location of your AWS data center. Mine is a string with a default of ap-south-1, which is the Mumbai region. A string simply means text.

The second variable is project. This is the name of the project, with a default of todo-app. A nice improvement you can add later is validation, which is a rule Terraform checks before running. A validation rule could force the name to use lowercase letters and hyphens only, which keeps bucket names clean and valid. I left it simple here, but it is worth knowing the option exists.

Notice what is missing. There is no account_id variable. Your account ID is the unique number that identifies your AWS account. Instead of typing it in by hand as a variable, where you could easily make a mistake, I read it automatically using a data source called data "aws_caller_identity". This always returns the correct account, so it can never be wrong.

How the Bucket Gets Its Name

S3 bucket names must be unique across all of AWS, not just within your account. So I built the name out of pieces that are guaranteed to be unique together.

${var.project}-statefile-${data.aws_caller_identity.current.account_id}

This combines the project name, the word statefile, and the account ID. Because the account ID is unique to me, the resulting name will not clash with anyone else.

One choice worth pointing out. I did not put the environment, such as dev or prod, into the name. One bucket serves all environments. This keeps things simple, and the different environments are separated inside the bucket by their file paths instead.

Version Constraints, and Why the Tiny Symbols Matter

In the terraform block I pinned the Terraform version like this.

required_version = "~> 1.15.0"

The little ~> symbol is called the pessimistic constraint operator. It controls how much Terraform is allowed to upgrade itself automatically. The exact form you write changes the meaning in a way that is easy to miss.

Writing ~> 1.15.0 allows only patch releases, meaning small bug fix updates like 1.15.1 or 1.15.2, but not 1.16.

Writing ~> 1.15 would also allow minor versions, meaning larger feature updates like 1.16 or 1.17.

These are very different. Patch only is cautious and safe. Minor allowed is more relaxed. The point is to pick one consciously rather than copying a symbol without understanding it.

A Small Rule About String Interpolation

Interpolation is the way Terraform inserts a variable value into a piece of text, using the ${...} syntax. There is a simple rule I learned to keep code clean.

Use the wrapped form "${var.region}" only when you are combining the variable with other text, like inside the bucket name above.

Use the plain form var.region when you are referring to a single variable on its own, with no surrounding text.

Wrapping a lone variable in ${...} for no reason just adds clutter, so I stopped doing it.

Problems I Hit, and What They Taught Me

This is the part I find most valuable, because every one of these came from getting something wrong first.

I tried to use a data source as the default value for a variable. This does not work. A default must be a static literal, meaning a fixed value typed directly in, not something that has to be looked up while Terraform runs.

I learned the account_id must be treated as a string, not a number. Account IDs can begin with a zero, and numbers drop leading zeros, which would silently break the value. Storing it as text keeps it intact.

I confirmed that backend.tf is static and is read during the init phase, which is the very first setup step when you run Terraform. Because it runs so early, it cannot use variables or outputs at all. That is exactly why the bootstrap uses local state instead.

I learned that providers.tf is also static and read at the same early init phase, for the same reason.

I learned never to rely on AWS defaults for security. AWS settings can change over time, and assuming a default protects you is risky. I now set encryption and public access blocking explicitly every single time, so the protection is written down and guaranteed.

And one more lesson that applies to everything, including advice from me. Always verify information. At one point I had a wrong belief about which Terraform version existed, and checking it directly set me straight. Trust, but confirm.

Git Rules

Git is the tool that tracks changes to your code. When using it, some files should be saved and shared, while others must never leave your machine because they contain secrets or local state. Here is how I split them.

# commit
*.tf
.terraform.lock.hcl
*.tfvars.example

# ignore
.terraform/
*.tfstate
*.tfstate.backup
*.tfvars

The files under commit are safe to share. The lock file pins the exact provider versions so everyone uses the same ones. The example file shows the shape of variable values without revealing real ones.

The files under ignore must stay private. The state files describe your live infrastructure and can hold sensitive details. The .tfvars files often contain real secret values. The .terraform folder is just local cache that does not belong in version control.

The Run Order

Finally, here is the exact order to run everything. The bootstrap goes first, the main project follows.

cd bootstrap → terraform init → terraform apply
cd environments/dev → terraform init → terraform apply

The command terraform init prepares the working folder and downloads what Terraform needs. The command terraform apply actually builds the resources. You run them in the bootstrap folder once to create the bucket, then move into your real environment and run them again to build the rest of your infrastructure, now safely backed by the remote state bucket you just made.

Reference image placeholder: a terminal screenshot showing terraform apply finishing successfully in the bootstrap folder.

Conclusion

The bootstrap is small, but it solves a real problem that confuses almost everyone at the start. You cannot store Terraform state in a bucket that does not exist, so you build that bucket once with a tiny standalone setup that uses local state, and then you never look back.

Along the way I learned that the missing files in a bootstrap are missing on purpose, that security should always be explicit rather than assumed, that tiny version symbols carry real meaning, and that even confident advice should be checked. None of these are hard once you see the reasoning, and together they make your foundation solid.

If you are new to AWS and Terraform, I hope walking through my journey makes your own first bootstrap far less mysterious than mine was.

Contact

If you have questions or want to share your own setup, feel free to reach out at khantanseer43@gmail.com.

Stop Hardcoding Passwords: A Beginner's Guide to AWS Secrets Manager

Tanseer — Tue, 23 Jun 2026 05:26:44 +0000

How to keep your database passwords and API keys safe, and why rotating them doesn't break your app the way you think it will

If you are new to AWS, there is a good chance you have done something like this at least once. You needed your app to connect to a database, so you took the database password and pasted it straight into your code, or into a Lambda environment variable, and moved on. It worked, so why worry?

The problem is that a password sitting in plain view is a password waiting to leak. In this tutorial we will walk through AWS Secrets Manager, a service built to store exactly these kinds of sensitive values safely. We will cover what it is, why environment variables are a weak place to keep secrets, what a KMS key actually does, and the one topic that confuses almost every beginner: automatic rotation. By the end you will understand why rotation does not break your application, even though it changes your password while the app is running.

Let us start from the beginning.

Jargon check before we go further: a secret just means any sensitive string your app needs but should never be public. Think database passwords, API keys, and access tokens. An API key is a long secret string that proves your app is allowed to use some service. A token is similar, usually a temporary one.

What Is AWS Secrets Manager?

AWS Secrets Manager is an encrypted vault for your secrets. Instead of writing your database password directly into your code, you store it inside Secrets Manager, and your app asks for it at runtime (meaning while the app is actually running, not while you are writing it).

A simple way to picture it is a hotel room safe. You do not tape your passport to the wall. You lock it in the safe, and only someone with the right key can open it. The hotel also keeps a log of every time the safe is opened. Secrets Manager works the same way.

It gives you three main benefits.

The first is encryption at rest. "At rest" means while the data is sitting in storage, as opposed to while it is travelling over the network. Your secret is scrambled so that even someone who somehow gets to the raw storage cannot read it.

The second is IAM controlled access. IAM stands for Identity and Access Management, which is the AWS system that decides who is allowed to do what. With Secrets Manager you can say "only my application is allowed to read this database password" and nothing else can touch it.

The third is automatic rotation, which means AWS can change your password on a schedule for you. This is the feature we will spend the most time on, because it is the most useful and the most misunderstood.

Why Not Just Use a Lambda Environment Variable?

An environment variable is a setting you attach to your app from the outside, so the value lives next to the app rather than inside your code. In AWS Lambda (the service that runs your backend code without you managing a server) you can set these in the function configuration. It works, and plenty of people do it. So what is wrong with it for secrets?

There are four real problems.

First, the value sits in plain text inside your function configuration. Anyone who can open the AWS console and view your Lambda can simply read the password. There is no lock on it.

Second, the value tends to leak into places you did not expect. If you manage your infrastructure with a tool like Terraform, the password ends up written into your Terraform state file, which is the record Terraform keeps of everything it created. It can also show up in deployment logs. So one password quietly becomes several copies scattered around.

Third, changing the value means redeploying or updating the function. There is no clean way to swap it on the fly.

Fourth, there is no audit trail. You cannot answer the question "who read this secret and when," because nothing was recorded.

Environment variables are perfectly fine for values that are not sensitive, like which region your app runs in or a feature toggle. They are just a weak place to keep real secrets.

What Is a KMS Key?

When we said Secrets Manager encrypts your data, something has to do the actual encrypting. That something is a KMS key.

KMS stands for Key Management Service. A KMS key is the cryptographic key used to scramble (encrypt) and unscramble (decrypt) your secret. Encryption is just the process of turning readable text into unreadable text so that only someone with the key can read it again.

Here is the clever part. You never see or hold the raw key yourself. It lives inside tamper resistant hardware, meaning special equipment built so the key cannot be copied out or stolen. You simply ask KMS "please decrypt this for me," and KMS checks your IAM permission before doing it.

Think of KMS as the master key to a building, locked inside a vault that you personally cannot open. You do not get to carry the master key around. Instead you hand your locked box to the security desk and say "open this please," and they check your badge first. Because the key never leaves the vault, nobody can walk off with it.

When Secrets Manager stores your password, it quietly uses a KMS key to encrypt it. AWS gives you a default key for free, so as a beginner you usually do not need to create your own.

Automatic Rotation: The Part Everyone Misunderstands

Rotation means automatically changing a password to a new one on a schedule. Here is the worry almost every beginner has, and it is a reasonable one:

"If Secrets Manager changes my password, but my database still has the old one, won't my app stop being able to connect?"

The answer is no, and understanding why is the heart of this whole topic.

Rotation does not change the password in only one place. It changes it in both the database and the secret, as a single coordinated operation. Here is the exact sequence the rotation process runs.

It generates a brand new random password.
It connects to the database using the current password (which still works at this point) and runs the command ALTER USER, which is the SQL instruction that sets a new password on the database account. Now the database expects the new password.
It stores that new password back into Secrets Manager as the current value.
It connects once more using the new value to confirm everything works. Because both sides are updated together, they are never out of sync in a way that locks you out.

But there is a second condition you must meet for this to stay smooth. Your app has to fetch the secret fresh from Secrets Manager at runtime, rather than holding its own permanent copy. If your code reads the current value when it needs it, then after rotation it simply reads the new value next time and carries on.

There is one trap here worth naming. To avoid asking Secrets Manager for the password on every single request (which costs a little money and adds delay), developers often cache it, meaning they keep a copy in memory for reuse. That is good practice, but if you cache the password forever and never refresh it, then after rotation your app is holding a stale password and will fail to connect. The fix is a cache with a TTL, which stands for Time To Live, a setting that tells the cache to throw the value away and fetch a fresh one after a few minutes.

So the rule is simple. Rotation does not break your app, as long as your app reads the secret fresh and does not cling to an old cached copy.

Versioning: Why the Old Password Sticks Around for a Moment

A secret in Secrets Manager does not hold just one value. It holds versions, and each version carries a label that tells you what it is for. Three labels matter.

AWSCURRENT is the value you get by default when you ask for the secret. This is the live, in use password.

AWSPREVIOUS is the value from just before the last change. Secrets Manager keeps it valid for a short window on purpose. Picture rotation happening at the exact moment a request is already in flight, meaning a request that started a split second before the change. That request may have grabbed the old value already. Keeping AWSPREVIOUS alive briefly lets that in flight request finish successfully instead of erroring out. It smooths over the handover.

AWSPENDING is the new value during rotation, after it has been created but before it has been promoted to current. It is the password in waiting.

You rarely need to think about these directly, because asking for the secret gives you AWSCURRENT automatically. But knowing the labels exist helps you understand why rotation is so smooth and why the brief overlap does not cause failures.

Cross Region Replication: A Feature for Later

By default, a secret lives in a single AWS region. A region is a geographic location where AWS runs its data centers, such as Mumbai (named ap-south-1) or Singapore.

Cross region replication keeps an automatically synced copy of your secret in other regions you choose. If you store a secret in Mumbai and replicate it to Singapore, then any change in Mumbai is copied across to the Singapore version on its own.

You would want this in two situations. One is disaster recovery, so that if an entire region has an outage, an app running in your backup region can still read the secret locally. The other is when your app runs in several regions at once and you want each one to read the secret nearby instead of reaching across the world every time.

As a beginner running a single region project, you almost certainly do not need this yet. File it away as a "when I grow into multiple regions" feature.

Conclusion

If you take away one idea from this tutorial, let it be this. Secrets stay in sync because rotation changes both sides at once, the database and the stored secret, in a single coordinated operation. And your application stays connected because it reads the current value fresh from Secrets Manager whenever it needs it, rather than holding a hardcoded copy that goes stale.

Everything else builds on that. Environment variables are weak for secrets because they sit in plain text and leak into your state files and logs. A KMS key does the encrypting behind the scenes without ever letting you touch the raw key. Versioning with AWSCURRENT and AWSPREVIOUS keeps a brief safety net so in flight requests survive a rotation. And cross region replication is there for the day you outgrow a single region.

So the next time you are tempted to paste a password into an environment variable, reach for Secrets Manager instead. Your future self, and anyone who inherits your code, will thank you.

Get in Touch

Found this helpful, or stuck on something? I would love to hear from you. Reach out at khantanseer43@gmail.com and I will do my best to help.

Terraform outputs.tf Explained: What It Is, When to Use It, and When to Skip It

Tanseer — Tue, 23 Jun 2026 04:53:47 +0000

Who This Is For

If you are learning Terraform and have come across outputs.tf in project structures or tutorials but are not fully clear on what it actually does or why it exists, this blog is for you.

We will start from the basics and build up to how outputs work across modules, what they look like in practice, and the rules around where they can and cannot be used. Every term will be explained along the way.

What Is an Output in Terraform?

When Terraform creates infrastructure, it stores everything it knows about that infrastructure in a file called the state file. The state file is Terraform's database. It tracks every resource, every attribute, every ID — everything.

But the state file is not something you read directly. It is large, it is internal, and most of the values in it are not things you care about day to day.

Outputs are a way to surface specific values from that state file — the ones you actually need. You decide what gets exposed. Everything else stays internal.

A good way to think about it:

State file = the entire database.

Outputs = a view on top of that database that exposes only what you chose to make visible.

Outputs do not add new information. They do not change what Terraform tracks. They simply choose what to surface from what already exists.

The Three Contexts Where Outputs Are Used

Outputs are not just for one purpose. They serve three distinct purposes depending on where you use them. Understanding the difference between these three will make the entire concept click.

Context 1: Passing Values Between Modules in the Same Project

This is the most common use case, and the one you will use in almost every real Terraform project.

Imagine your project has two modules: a database module and a backend module. The database module creates an RDS instance and stores credentials in AWS Secrets Manager. The backend module is a Lambda function that needs to connect to that database. To do that, it needs the ARN (unique identifier) of the secret.

But here is the problem. Module internals are private. A resource defined inside modules/database/main.tf is not visible anywhere outside that module. It is intentionally hidden. You cannot just reference it from another module directly.

This is where outputs.tf comes in. The database module uses outputs.tf to explicitly expose the values it wants to share. The backend module then receives those values as input variables.

The wiring between them happens in your environment file, for example environments/dev/main.tf. That file is the connection layer. It takes what one module exposes and passes it as input to another.

The flow looks like this:

modules/database/outputs.tf
        |
        v
environments/dev/main.tf   (the wiring layer)
        |
        v
modules/backend/variables.tf

Here is what that looks like in code:

# modules/database/outputs.tf
output "secret_arn" {
  value = aws_secretsmanager_secret.db_credentials.arn
}

output "secret_name" {
  value = aws_secretsmanager_secret.db_credentials.name
}

# environments/dev/main.tf
module "database" {
  source = "../../modules/database"
}

module "backend" {
  source     = "../../modules/backend"
  secret_arn  = module.database.secret_arn
  secret_name = module.database.secret_name
}

# modules/backend/variables.tf
variable "secret_arn" {}
variable "secret_name" {}

The database module is a black box. variables.tf is what it accepts as input. main.tf is its internal implementation. outputs.tf is what it hands back out. Nothing inside the module leaks out unless you explicitly put it in outputs.tf.

Context 2: Displaying Values in the Terminal After Apply

Outputs defined in the root module, meaning the environment folder like environments/dev, are printed to the terminal after terraform apply completes.

This is purely for you as the person running the deployment. It is a convenience feature so you do not have to go hunting through the AWS console for values you commonly need.

Good candidates for terminal outputs are things like:

The API Gateway URL your backend is reachable at.

The Amplify URL where your frontend is deployed.

The RDS endpoint for your database.

# environments/dev/outputs.tf
output "api_gateway_url" {
  value = module.backend.api_url
}

output "amplify_url" {
  value = module.frontend.app_url
}

After apply, these print cleanly to your terminal:

Outputs:

api_gateway_url = "https://abc123.execute-api.ap-south-1.amazonaws.com/dev"
amplify_url     = "https://main.abc123.amplifyapp.com"

No digging through the console. The values are right there.

Context 3: Sharing Values Across Completely Separate Terraform Projects

Sometimes infrastructure is split across multiple completely separate Terraform projects, each with their own state file. A networking team might manage VPCs in one project. An application team might manage Lambda functions in another. The application team needs the VPC ID from the networking project.

This is handled using something called terraform_remote_state. It is a data source (a way to read information from outside the current project) that reads another project's state file and exposes its outputs.

data "terraform_remote_state" "networking" {
  backend = "s3"
  config = {
    bucket = "company-tfstate"
    key    = "networking/terraform.tfstate"
    region = "ap-south-1"
  }
}

# Now you can use:
data.terraform_remote_state.networking.outputs.vpc_id

One important detail here: terraform_remote_state can only access values that were explicitly defined as outputs in the other project. It cannot reach into the raw state and pull arbitrary values. If the networking team did not put the VPC ID in their outputs.tf, you cannot get it from here.

This reinforces the same rule: outputs are the only gateway for values to leave a Terraform boundary, whether that boundary is a module or an entire separate project.

This pattern does not apply if your entire infrastructure lives in one root, which is the case for smaller projects. But it is a real and widely used pattern in team environments.

Where Outputs Cannot Be Used

This is important. Outputs work during the plan and apply phase, when Terraform is evaluating your configuration and building infrastructure. But not every file in your project is evaluated at that phase.

Two files are evaluated earlier, during the init phase, before any HCL evaluation happens:

backend.tf — This is where you configure where Terraform stores its state file, for example an S3 bucket. Terraform reads this file during terraform init, before it knows anything about your resources or variables. Everything here must be a hardcoded string. You cannot reference an output, a variable, or a local.

providers.tf — This is where you configure your cloud provider, for example the AWS region and profile. This is also read during init. Same restriction. Hardcode everything here.

Trying to use a variable or output in either of these files will cause Terraform to throw an error, and the reason is always the same: those values are not available yet at init time.

Here is a simple way to remember the two phases:

Init phase reads backend.tf and providers.tf statically. Downloads providers. Sets up the backend. No HCL logic allowed.

Plan and Apply phase evaluates everything else. Outputs, variables, locals, data sources, resource references — all of this works here.

Why You Cannot Just Skip outputs.tf and Read the State Directly

A common question when learning this is: the state file already has all the values, why do I need outputs at all?

There are two reasons.

Within a module, resources are private by design. Terraform intentionally encapsulates module internals so that modules are independent and reusable. You cannot reach into modules/database/main.tf from environments/dev/main.tf and reference a resource directly. The module must explicitly hand that value out through outputs.tf. This is the same reason functions in a programming language return values instead of letting callers read internal variables directly.

Across separate roots, terraform_remote_state only exposes outputs. There is no mechanism to read arbitrary resource attributes from another project's state file. Outputs are the only thing that crosses that boundary.

When outputs.tf Is Optional

Not every module needs an outputs.tf. The rule is straightforward.

If nothing downstream needs a value from your module, there is nothing to expose. The outputs file is optional.

A common example is a frontend module in a project. If the frontend module deploys an Amplify app and no other module needs the Amplify URL as an input, then outputs.tf in the frontend module is not needed. The URL might still be shown in the terminal via the root module's outputs, but the frontend module itself does not need to expose anything.

The question to ask before adding any output is:

Who is the consumer of this value, and how will they read it?

If the answer is nobody, the output does not belong. Adding outputs that nothing consumes is just noise in your codebase.

A Quick Reference

Here is everything in one place:

Outputs between modules: use outputs.tf in the source module, wire it in the environment's main.tf, receive it in the destination module's variables.tf.

Outputs in the terminal: define them in the root module's outputs.tf. They print after terraform apply.

Outputs across separate roots: use terraform_remote_state to read another project's outputs from its remote state file.

Outputs do not work in backend.tf or providers.tf because those are read during init before HCL evaluation.

You cannot skip outputs.tf because module internals are private and remote state only exposes outputs.

outputs.tf is optional when nothing downstream needs the module's values.

Conclusion

Outputs are one of those concepts in Terraform that seem small but are actually the backbone of how information flows through your infrastructure code. Once you understand that modules are black boxes and outputs are the only thing that leaves them, the entire system starts to make sense.

Before adding any output, always ask who the consumer is. That one question will keep your outputs intentional, your modules clean, and your project easy to follow.

Need Help?

If you are learning Terraform and have questions about project structure, modules, state management, or anything else, feel free to reach out.

Email me at khantanseer43@gmail.com

How to Add a Custom Domain to AWS Cognito Google Login (And the Errors Nobody Warns You About)

Tanseer — Tue, 02 Jun 2026 04:39:24 +0000

Who This Is For

If you are using AWS Cognito to handle user authentication in your app and you have added Google as a social login option, you may have noticed something uncomfortable. When a user clicks "Sign in with Google", the URL in the browser does not say your domain. It says something like:

https://yourapp.auth.us-east-1.amazoncognito.com

That is the default Cognito hosted UI URL. It works, but it looks like your users are leaving your app to sign in somewhere else. It does not look professional, and for security conscious users, it can feel suspicious.

The fix is to set up a custom domain so the login page shows your own URL, something like:

https://auth.yourdomain.com

This guide walks through the full setup step by step, and more importantly, it covers the three errors you will likely hit along the way.

Before You Start

Here is what you need to have in place before following this guide:

A Cognito User Pool already created with Google set up as a social identity provider. If you have not done that yet, set that up first and come back here.

A domain you own, managed in Route 53 (AWS's domain and DNS service). The subdomain you will be using for the login page, something like auth.yourdomain.com, should not be pointing anywhere yet.

Access to the Google Cloud Console where your OAuth app is configured.

Step 1: Create an ACM Certificate — But Only in us-east-1

This is the first place most developers go wrong, and it is the most confusing one because nothing in the Cognito console tells you about it until you are already stuck.

ACM stands for AWS Certificate Manager. It is the service AWS uses to create and manage SSL certificates. An SSL certificate is what makes your domain load over HTTPS (the secure version of HTTP) and shows the padlock icon in the browser.

You need to create a certificate for your custom domain, like auth.yourdomain.com. That part is straightforward. The gotcha is the region.

Even if your Cognito User Pool is in a different region, say ap-south-1 or eu-west-1, the ACM certificate must be created in us-east-1. No exceptions.

The reason is that Cognito uses CloudFront behind the scenes to serve the hosted UI. CloudFront is a global content delivery service, and it only accepts SSL certificates from the us-east-1 region. Your Cognito User Pool can be anywhere, but the certificate must always be in us-east-1.

Here is how to create it:

Go to the AWS Console and switch your region to US East (N. Virginia) which is us-east-1. You can change the region from the top right dropdown in the console.
Search for "Certificate Manager" and open ACM.
Click "Request a certificate".
Choose "Request a public certificate" and click Next.
Enter your domain name. Use the subdomain you want for login, for example auth.yourdomain.com.
Choose "DNS validation" as the validation method. This is the recommended option.
Click "Request". After requesting, ACM will give you a CNAME record that you need to add to your DNS to prove you own the domain. Go to Route 53, open your hosted zone, and add that CNAME record exactly as ACM shows it.

Once the record is in place, ACM will automatically verify it and mark the certificate as "Issued". This usually takes a few minutes but can take up to 30 minutes.

Do not move to the next step until the certificate status shows "Issued".

Step 2: Add the Custom Domain in Cognito

Now that the certificate is ready, go back to your Cognito User Pool. Make sure you are in the correct region where your User Pool lives, not us-east-1 unless that is where your pool is.

Open your User Pool in the Cognito console.
Go to the "App integration" tab.
Scroll down to "Domain" and click "Actions", then "Create custom domain".
Enter your custom domain, for example auth.yourdomain.com.
In the SSL certificate dropdown, select the ACM certificate you just created. It will only appear if the certificate is in us-east-1 and has a status of Issued.
Click "Create". After saving, Cognito will provision a CloudFront distribution for your domain. This can take anywhere from 15 to 40 minutes. You will see a CloudFront domain name appear once it is ready, something like d1234abcde.cloudfront.net. Copy this value. You will need it in the next step.

Step 3: Add the DNS Record in Route 53

This is the second place where things can go wrong.

Once Cognito gives you the CloudFront domain, you need to create a DNS record so that when someone visits auth.yourdomain.com, they are pointed to that CloudFront distribution.

The record type to use here is an Alias record, not a CNAME. Route 53 has a special Alias record type that is designed to point to AWS services like CloudFront. Using a regular CNAME at the root level of a subdomain can cause issues, so always use Alias when pointing to a CloudFront domain in Route 53.

Here is how to add it:

Open Route 53 in the AWS Console.
Go to "Hosted zones" and open your domain.
Click "Create record".
Set the record name to auth (Route 53 will append your domain automatically).
Set the record type to "A".
Toggle on "Alias".
In the "Route traffic to" dropdown, choose "Alias to CloudFront distribution".
Paste the CloudFront domain that Cognito gave you.
Click "Create records". DNS changes can take a few minutes to a few hours to fully propagate. You can check the status using a tool like MXToolbox or WhatsMyDNS by looking up your subdomain.

Step 4: Update the Redirect URI in Google Cloud Console

This is the third gotcha, and it is easy to miss because everything might look like it is working until a user actually tries to sign in with Google and gets an error.

When Google handles the OAuth login flow (OAuth is the protocol that lets users sign in with their Google account), it redirects the user back to a specific URL after they authenticate. That URL is called the redirect URI.

Before you set up the custom domain, Cognito's redirect URI was something like:

https://yourapp.auth.us-east-1.amazoncognito.com/oauth2/idpresponse

Now that you have a custom domain, the redirect URI has changed to:

https://auth.yourdomain.com/oauth2/idpresponse

Google will reject any redirect to a URI that is not explicitly listed as an authorized redirect URI in your OAuth app settings. If you do not update this, users will see a Google error page saying the redirect URI is not authorized.

Here is how to update it:

Go to the Google Cloud Console at console.cloud.google.com.
Open your project and navigate to "APIs and Services", then "Credentials".
Click on the OAuth 2.0 Client ID you created for Cognito.
Under "Authorized redirect URIs", click "Add URI".
Add your new redirect URI: https://auth.yourdomain.com/oauth2/idpresponse
Also update "Authorized JavaScript origins" if it was set to the old Cognito domain. Add https://auth.yourdomain.com.
Click "Save". You do not need to remove the old Cognito URI right away. Keep both for now while you test the new setup, and remove the old one once everything is confirmed working.

Step 5: Test the Full Flow

Once all the above steps are done and DNS has propagated, test the complete sign in flow from your app.

Open your app and trigger the Google login. The URL in the browser should now show auth.yourdomain.com instead of the Cognito domain. The user should be able to complete the Google sign in and land back in your app without any errors.

If you see a Google error about redirect URI mismatch, double check Step 4. If you see an SSL error, double check that the ACM certificate is in us-east-1 and is fully issued. If the DNS is not resolving, give it more time or check your Route 53 record.

Quick Summary of the Three Errors to Avoid

The ACM certificate must be in us-east-1 no matter what region your Cognito User Pool is in. Creating it in the wrong region means it will not appear in the Cognito domain setup and you will waste time wondering why.

The DNS record in Route 53 should be an Alias A record pointing to the CloudFront domain, not a plain CNAME. Using the wrong record type can cause resolution issues.

The redirect URI in Google Cloud Console must be updated to the new custom domain. Forgetting this means Google will block the login flow and your users will see an error.

Conclusion

Adding a custom domain to your Cognito login page is a small change that makes a big difference in how professional and trustworthy your app feels. The steps themselves are not complicated, but the three gotchas around the ACM region, DNS record type, and Google redirect URI are not obvious and not well documented in one place.

If you follow this guide in order and do not skip steps, you should have your custom domain up and running without the frustration of debugging silent failures.

Need Help?

If you are stuck at any of these steps or running into an error not covered here, feel free to reach out. Happy to help.

Email me at khantanseer43@gmail.com

Lambda Execution Roles Are Quietly Breaking Your Least Privilege Policy

Tanseer — Thu, 21 May 2026 05:26:38 +0000

Who This Is For

If you are using AWS Lambda to build serverless applications and you have never stopped to look closely at the IAM roles attached to your functions, this blog is for you.

We are going to talk about what a Lambda execution role is, why the way most people set them up creates a security problem, and exactly what you should do instead. Every term will be explained along the way.

A Quick Refresher: What Is an Execution Role?

When a Lambda function runs, it needs permission to interact with other AWS services. For example, if your function reads data from a database or writes a file to S3 (AWS's file storage service), AWS needs to know whether your function is allowed to do that.

This permission is controlled by something called an IAM execution role. IAM stands for Identity and Access Management. It is the system AWS uses to control who or what can access which resources.

Every Lambda function has an execution role attached to it. When the function runs, it automatically assumes that role and gets whatever permissions the role has. Think of it like a staff ID card. The card determines which rooms in the building you are allowed to enter.

What Is the Least Privilege Principle?

The principle of least privilege is a security concept that says: every user, system, or service should have only the minimum permissions it needs to do its job, and nothing more.

If a Lambda function only needs to read from one specific DynamoDB table (a type of AWS database), it should have permission to read from that one table. It should not have permission to read from all tables, write to tables, delete tables, or access any other AWS service.

This sounds obvious. But in practice, it is almost never what happens.

How Execution Roles Are Usually Set Up

When you create a Lambda function for the first time, AWS creates a basic execution role for you automatically. This default role usually includes permission to write logs to CloudWatch (AWS's logging service), which is fine.

But here is where the problem starts.

As you build your function and it needs to access more services, the easiest thing to do is go to the role and add more permissions. Need to read from S3? Add s3:GetObject. Actually, to save time, just add s3:* which gives the function permission to do everything in S3. Need to write to DynamoDB? Add dynamodb:*. Need to send messages? Add sns:*.

Within a few iterations, your Lambda function has a role that can read, write, and delete across half your AWS account.

And then there is the even worse pattern: one shared role for all Lambda functions. Instead of creating a separate role for each function, many developers just reuse the same role across every function in the project. It is faster to set up. But now every single function has every permission that any function ever needed.

Why This Is a Real Problem

You might be thinking: my app is not a high value target, nobody is going to attack my Lambda function.

That thinking is what attackers rely on.

Lambda functions are often triggered by external events. An API Gateway request, an S3 file upload, an SNS message. Any of these entry points can potentially be exploited if there is a vulnerability in your code, a dependency with a known security issue, or even a misconfigured trigger.

If an attacker gains control of one Lambda function that has an overly permissive role, they do not just have access to what that function was supposed to do. They have access to everything that role allows. That could mean reading sensitive data from your database, writing files to your storage, sending messages to your users, or even modifying other parts of your infrastructure.

One compromised function becomes a key to your entire account.

The Three Most Common Mistakes

Mistake 1: Using wildcard permissions

A wildcard in IAM policy looks like this:

{
  "Effect": "Allow",
  "Action": "dynamodb:*",
  "Resource": "*"
}

The dynamodb:* means allow all DynamoDB actions. The "Resource": "*" means on all resources. So this one policy line gives your Lambda function full DynamoDB access across your entire account. If your function only needed to read from one table, this is massively over-permissioned.

Mistake 2: Sharing one role across multiple Lambda functions

Every function has different responsibilities. A function that sends a password reset email should not have the same permissions as a function that processes payments. When you share a role, you are taking the maximum permissions needed by any one function and giving them to all functions.

Mistake 3: Never revisiting the role after initial setup

During development, it is common to grant broad permissions just to get things working. The problem is that most developers never go back to tighten those permissions before going live. The overly permissive development role becomes the production role.

How to Fix It

Fix 1: One role per Lambda function

Each Lambda function should have its own dedicated execution role. Yes, this means more roles to manage. But it means a compromised function can only access what it specifically needs, not what any other function needs.

Fix 2: Scope permissions to specific actions and resources

Instead of dynamodb:* on *, write out exactly what the function needs:

{
  "Effect": "Allow",
  "Action": [
    "dynamodb:GetItem",
    "dynamodb:PutItem"
  ],
  "Resource": "arn:aws:dynamodb:us-east-1:123456789012:table/MySpecificTable"
}

This gives permission only to read and write items, and only on one specific table. Nothing more.

The arn in the resource field is a unique identifier for a specific AWS resource. Every resource in AWS has one. Using it in your policy means you are granting access to that one resource, not everything in the service.

Fix 3: Use IAM Access Analyzer to generate policies from actual usage

IAM Access Analyzer is a free AWS tool that can look at your CloudTrail logs (the record of every action taken in your AWS account) and tell you exactly which permissions your Lambda function actually used over a given time period.

Instead of guessing what permissions a function needs, you can run it for a while, then use Access Analyzer to generate a policy based on real usage. This is one of the most accurate ways to arrive at a least privilege policy.

To use it, go to IAM in the AWS console, find the role attached to your Lambda function, and look for the "Generate policy" option under Access Analyzer.

Fix 4: Use permission boundaries as a safety net

A permission boundary is a policy that sets a ceiling on what permissions a role can ever have. Even if someone attaches a broad policy to the role later, the boundary limits the effective permissions.

This is especially useful in team environments where multiple developers are creating and modifying Lambda functions. It acts as a guardrail so that even if someone makes a mistake, the damage is contained.

A Simple Checklist Before You Deploy

Before your Lambda function goes live, go through these:

Does this function have its own dedicated execution role, not shared with any other function?

Does the role use specific actions like dynamodb:GetItem instead of dynamodb:*?

Does the role point to specific resource ARNs instead of *?

Have you removed any permissions that were added during development and are no longer needed?

If you are working in a team, is there a permission boundary in place?

Conclusion

Least privilege is one of the most talked about security principles in cloud development. But it is also one of the most commonly ignored in practice, not because developers do not care, but because the default path leads away from it.

AWS makes it easy to create broad roles. It is faster to write dynamodb:* than to list out every specific action. It is more convenient to reuse one role across all your functions. But each of those shortcuts quietly widens the blast radius if something goes wrong.

Fixing this does not require rebuilding anything. It just requires going back to your Lambda functions one by one, looking at their execution roles, and asking: does this function actually need all of this?

Most of the time, the answer will be no.

Need Help?

If you want help auditing your Lambda execution roles or figuring out what permissions your functions actually need, feel free to reach out.

Email me at khantanseer43@gmail.com

AWS Amplify Looks Simple, Until It Is Not

Tanseer — Tue, 12 May 2026 05:22:47 +0000

Who This Is For

If you are new to AWS and thinking of using Amplify to deploy your app, this blog is for you.

Amplify is marketed as an easy, all-in-one deployment platform. And for simple use cases, it is. But once you start pushing it a little : different package managers, larger projects, custom configurations : you start running into walls that are not mentioned anywhere in the getting started guides.

I have spent a lot of time deploying different kinds of applications on Amplify. I have hit these walls firsthand. This blog is a collection of the things I discovered the hard way, explained simply, so you do not have to go through the same debugging sessions.

A Quick Intro to AWS Amplify for Beginners

Before we get into the issues, here is a quick explanation of what Amplify actually is.

AWS Amplify is a hosting and deployment platform from Amazon Web Services. You connect your GitHub repository to Amplify, and every time you push code, Amplify automatically builds and deploys your app. It handles the servers, the CDN (Content Delivery Network : the system that delivers your app quickly to users around the world), and the deployment pipeline for you.

It sounds perfect. And for many projects, it works well. But here are the things you need to know before you go all in.

Issue 1: The Cache Feature Does Not Actually Help

The Problem

Amplify has a built-in caching feature. In theory, it saves folders like node_modules (the folder where all your app's dependencies or packages live) between builds. The idea is that on the next build, those packages are already there and do not need to be downloaded again, making the build faster.

In practice, the opposite happens.

Amplify's cache works by zipping up the folder and uploading it to storage after a build, then downloading and unzipping it before the next build. This zip and unzip process itself takes significant time — in my experience, around 3 minutes to restore and 3 minutes to save.

Now here is the deeper problem. The two most common commands used to install packages are:

npm ci : This is the recommended command for automated deployments. It deletes node_modules completely and reinstalls everything from scratch every single time. It does not matter that Amplify just spent 3 minutes restoring that folder. npm ci will delete it immediately.

npm install: This one does not always delete node_modules, but it re-evaluates your packages and may reinstall parts of it anyway.

In both cases, the cache Amplify restored is either deleted immediately or partially ignored.

I ran experiments on both large projects and minimal ones. The result was consistent — removing the cache made builds faster, not slower.

The Solution

Remove the cache section from your amplify.yml file entirely. Here is what a clean build config looks like without cache:

version: 1
frontend:
  phases:
    preBuild:
      commands:
        - npm ci
    build:
      commands:
        - npm run build
  artifacts:
    baseDirectory: build
    files:
      - '**/*'

No cache block. Cleaner, faster, simpler.

Issue 2: The 220MB Artifact Size Limit

The Problem

Amplify has a hard limit on the size of your build artifacts. Artifacts are the files produced after your build, things like your compiled JavaScript, CSS, images, and in some cases the node_modules folder if your app needs it at runtime.

The limit is 220MB.

This sounds like a lot until you are building a Next.js app (a popular React framework) with server-side features. Next.js SSR (Server-Side Rendering — where pages are generated on the server instead of the browser) requires parts of node_modules to be bundled with the output. A medium-sized project can easily exceed 220MB once all those dependencies are included.

When you hit this limit, the build fails with a vague error. If you are not aware of this constraint, you will spend a long time looking in the wrong places.

The Solution

There are a few ways to handle this:

The first option is to use output: 'standalone' in your next.config.js file if you are using Next.js. Standalone mode bundles only the exact files needed to run the app, which significantly reduces the output size.

// next.config.js
const nextConfig = {
  output: 'standalone',
};

module.exports = nextConfig;

The second option is to audit your dependencies. Tools like npm-analyze or the --why flag in pnpm can help you find packages that are large and may not be needed at runtime.

The third option, if you are consistently hitting this limit, is to consider moving to a different deployment setup such as running your app in a container on ECS (Elastic Container Service) or using a different hosting provider that does not have this constraint.

Issue 3: pnpm's Symlink Structure Breaks at Runtime

The Problem

pnpm is a modern, faster alternative to npm for managing packages. It is popular because it saves disk space and installs faster. However, pnpm uses a unique approach to storing packages : it uses symlinks (think of them as shortcuts that point to where a file actually lives) instead of copying files directly into your node_modules folder.

Amplify's build and runtime environment does not handle these symlinks well.

What typically happens is this: your build completes successfully. Everything looks fine. But when your app actually runs, it throws a Cannot find module error : meaning it cannot locate a package that is clearly installed. The app is broken at runtime even though the build passed.

This is a known issue that has been reported by multiple developers and is documented in Amplify's GitHub issues.

The Solution

The fix is to force pnpm to use a "hoisted" install mode, which makes it behave more like npm by copying packages directly instead of using symlinks. Add this to your amplify.yml in the preBuild section:

preBuild:
  commands:
    - echo "node-linker=hoisted" >> .npmrc
    - corepack enable
    - pnpm install --shamefully-hoist

The --shamefully-hoist flag tells pnpm to put all packages directly in node_modules the traditional way. The name sounds alarming but it is a well-known workaround specifically for environments that do not support symlinks.

Alternatively, if you are not tied to pnpm, switching to npm for your Amplify builds is the simplest fix. You can keep pnpm locally for development and use npm in the Amplify build config.

A Note on These Constraints

None of these limitations are front and center in Amplify's documentation. Most of them are buried in GitHub issue threads, Stack Overflow answers, or AWS re:Post forums. As a beginner, you would have no reason to look for them until something breaks.

That is the pattern with Amplify, it works smoothly for straightforward use cases, but the moment you step slightly outside the default path, you hit constraints that are hard to debug without prior knowledge.

This is not to say Amplify is bad. For small to medium projects, it is genuinely convenient. But going in with eyes open about these limitations will save you real time.

Quick Reference

Here is a summary of everything covered:

Cache, remove it entirely. It adds time instead of saving it because npm ci and npm install both ignore or delete the cached node_modules anyway.

220MB artifact limit plan for it. Use Next.js standalone output mode and audit your dependencies if you are close to the limit.

pnpm symlinks, use hoisted mode. Add node-linker=hoisted to your .npmrc and use --shamefully-hoist when installing, or switch to npm for Amplify builds.

Have You Hit Other Amplify Limitations?

This list is based on my own experience. Amplify has more quirks than these three, and I am sure other developers have hit walls I have not encountered yet.

If you have run into something that is not covered here, I would love to hear about it. Drop a comment or reach out directly, let us build a more complete picture of what Amplify can and cannot do, so the next developer does not have to figure it out the hard way.

Need Help?

If you are stuck on any of these issues or running into something else with your Amplify setup, feel free to reach out. I am happy to help.

Email me at khantanseer43@gmail.com