5 Steps to Deploying an AWS Lambda with EventBridge using Terraform

#aws #serverless #terraform #tutorial

I used to think serverless meant "upload code, magic happens." Then today I tried to deploy a Python Lambda that needed custom dependencies, a schedule trigger, and enough memory to actually stay alive, and quietly questioning my career choices.

Nevermind, Here's how I finally got my atlas-worker Lambda running, containerized, scheduled, and only slightly over-provisioned. (You can jump to the last part to get the whole code)

Step 1: Accept That ZIP Files Are for Cowards (Use Containers)

Look, you could package your Lambda as a ZIP. You could also commute to work on a unicycle. Both are technically possible, but why suffer?

I needed psycopg2, some geospatial libraries, and enough room to breathe, also zip has a size limit. So I went with a container image. The first hurdle? Terraform needs to log into ECR before it can push the image, but it needs to know the registry URL to log in... which requires knowing your account ID.

That's why Use aws_caller_identity and aws_ecr_authorization_token data sources. It feels like Terraform inception, but it works:

data "aws_caller_identity" "current" {}
data "aws_ecr_authorization_token" "token" {
  registry_id = data.aws_caller_identity.current.account_id
}

provider "docker" {
  registry_auth {
    address  = "${data.aws_caller_identity.current.account_id}.dkr.ecr.${var.aws_region}.amazonaws.com"
    username = data.aws_ecr_authorization_token.token.user_name
    password = data.aws_ecr_authorization_token.token.password
  }
}

I hope your aws cli is already setup.

Step 2: Let Terraform Build the Image (Yes, Really)

I used to build the Docker image separately, tag it manually, push it, then update Terraform. That's three steps too many. The terraform-aws-modules/lambda module has a docker-build submodule that handles this in one go.

module "docker_image" {
  source = "terraform-aws-modules/lambda/aws//modules/docker-build"

  create_ecr_repo = true
  ecr_repo        = "atlas-worker"
  use_image_tag   = true
  image_tag       = var.image_tag # Keep a track on it
  source_path     = "../apps/workers" # your docker file location
}

IMP: The image will only build again if you change the image_tag not if you change the code, so keep a track on it.
Set create_ecr_repo = true and Terraform provisions the registry, builds the image, and pushes it. It's terrifyingly convenient. I kept waiting for the catch.

(There is a catch. We'll get to it in step 4½.)

Step 3: Configure the Lambda

AWS defaults are... optimistic. A 3-second timeout and 128MB of RAM works for "Hello World." My worker connects to Postgres, queries an API, and uploads to S3. So I maxed out the timeout to 15 minutes and gave it 1GB of memory.

module "lambda_function" {
  source = "terraform-aws-modules/lambda/aws"

  function_name  = "atlas-worker"
  create_package = false
  image_uri      = module.docker_image.image_uri
  package_type   = "Image"

  timeout     = var.lambda_timeout  # 900 seconds = "Please just finish"
  memory_size = var.lambda_memory   # 1024 MB = "I believe in you"
}

Yes, I'm paying for a Lambo to do grocery runs. But it works.

Step 4: EventBridge (Or "CloudWatch Events" for Us Old Timers)

I wanted this thing to run weekly. My brain said "CloudWatch cron," but AWS renamed CloudWatch Events to EventBridge five years ago and my muscle memory hasn't caught up.

The EventBridge module connects your Lambda to a schedule:

module "eventbridge" {
  source  = "terraform-aws-modules/eventbridge/aws"
  version = "4.2.2"

  create_bus = false  # Don't need a custom event bus

  rules = {
    weekly_sync = {
      description         = "Trigger ARGO float weekly sync"
      schedule_expression = var.schedule_expression  # cron(0 12 ? * SUN *)
    }
  }

  targets = {
    weekly_sync = [
      {
        name  = "atlas-worker-lambda"
        arn   = module.lambda_function.lambda_function_arn
        input = jsonencode({ operation = "update" })
      }
    ]
  }
}

That input field? It sends a custom JSON payload to your Lambda. My function checks event["operation"] to know whether it's a scheduled run or a manual invocation.

Step 4½: The Permission That Haunts My Dreams ☠️

Here's the "½ step"—the thing that isn't in the main tutorial but will break everything if you skip it.

EventBridge can say it will trigger your Lambda, but unless you explicitly give it permission, AWS will silently drop those invocations. No error logs. No failed invocations metric. Just... nothing happening at 2 AM when your cron fires.

You need the allowed_triggers block in your Lambda module:

allowed_triggers = {
  EventBridgeRule = {
    principal  = "events.amazonaws.com"
    source_arn = module.eventbridge.eventbridge_rule_arns["weekly_sync"]
  }
}

I spent a morning thinking my cron expression was wrong. Nope. The Lambda just... rejected the trigger. Politely. Without telling me.

This is the ½ step because it's invisible until it isn't.

Step 5: Environment Variables (AKA "How Many Secrets Can I Fit In Here")

Last the config. Database URLs, S3 credentials, the whole messy reality of "my app needs to talk to things":

environment_variables = {
  PG_WRITE_URL   = var.pg_write_url
  S3_ACCESS_KEY  = var.s3_access_key
  S3_SECRET_KEY  = var.s3_secret_key
  S3_ENDPOINT    = var.s3_endpoint
  S3_BUCKET_NAME = var.s3_bucket_name
  ARGO_DAC       = var.argo_dac
}

Are these in AWS Secrets Manager? No. Should they be? Probably. But this is a dev blog, not a security audit. We'll pretend I used Terraform Cloud workspaces with encrypted variables and move on.

I assume you have already setup your main.tf so here's the full worker.tf. Now just setup the env vars and Thanks me later :)

data "aws_caller_identity" "current" {}

data "aws_ecr_authorization_token" "token" {
  registry_id = data.aws_caller_identity.current.account_id
}

provider "docker" {
  registry_auth {
    address  = "${data.aws_caller_identity.current.account_id}.dkr.ecr.${var.aws_region}.amazonaws.com"
    username = data.aws_ecr_authorization_token.token.user_name
    password = data.aws_ecr_authorization_token.token.password
  }
}

module "lambda_function" {
  source = "terraform-aws-modules/lambda/aws"

  function_name  = "atlas-worker"
  create_package = false

  image_uri    = module.docker_image.image_uri
  package_type = "Image"

  timeout     = var.lambda_timeout # 15 minutes
  memory_size = var.lambda_memory  # 1GB

  environment_variables = {
    PG_WRITE_URL   = var.pg_write_url
    S3_ACCESS_KEY  = var.s3_access_key
    S3_SECRET_KEY  = var.s3_secret_key
    S3_ENDPOINT    = var.s3_endpoint
    S3_BUCKET_NAME = var.s3_bucket_name
    ARGO_DAC       = var.argo_dac
  }

  # Allow EventBridge to invoke this Lambda
  create_current_version_allowed_triggers = false
  allowed_triggers = {
    EventBridgeRule = {
      principal  = "events.amazonaws.com"
      source_arn = module.eventbridge.eventbridge_rule_arns["weekly_sync"]
    }
  }
}

module "docker_image" {
  source = "terraform-aws-modules/lambda/aws//modules/docker-build"

  create_ecr_repo = true
  ecr_repo        = "atlas-worker"

  use_image_tag = true
  image_tag     = var.image_tag

  source_path = "../apps/workers"
}

module "eventbridge" {
  source  = "terraform-aws-modules/eventbridge/aws"
  version = "4.2.2"

  create_bus = false

  rules = {
    weekly_sync = {
      description         = "Trigger ARGO float weekly sync"
      schedule_expression = var.schedule_expression
    }
  }

  # Connect Lambda as target
  targets = {
    weekly_sync = [
      {
        name  = "atlas-worker-lambda"
        arn   = module.lambda_function.lambda_function_arn
        input = jsonencode({ operation = "update" })
      }
    ]
  }

  tags = {
    Name = "atlas-worker-scheduler"
  }
}