DEV Community

Cover image for How to Fix CloudFormation Circular Dependencies in AWS SAM
Mathabo Motaung
Mathabo Motaung

Posted on

How to Fix CloudFormation Circular Dependencies in AWS SAM

#aws #java #backend #architecture

The "Chicken and Egg" problem when connecting S3 Buckets to Lambda Triggers.

If you have ever tried to build an Event-Driven architecture with AWS SAM, you have likely seen this error message. It usually appears 5 minutes into a deployment, just when you think everything is working:

Error: Failed to create changeset for the stack: s3entinel-stack
Status: FAILED. Reason: Circular dependency between resources:
[ImageProcessorFunctionRole, ImageProcessorFunction, S3entinelBucket, ImageProcessorFunctionFileUploadPermission]

I hit this wall while building S3entinel, a Spring Boot + Serverless hybrid application. Here is exactly why this happens and the simple Sub-trick I used to fix it.

The Architecture

The goal was simple:

  1. S3 Bucket (S3entinelBucket) receives a file.

  2. Lambda Function (ImageProcessorFunction) gets triggered automatically.

  3. Lambda reads the file metadata and logs it.

To make this work, two things must happen in the infrastructure:

  1. The Bucket needs to know the Lambda's ARN to send the notification.

  2. The Lambda needs permission (IAM Policy) to read from The Bucket.

The Problem: Chicken vs. Egg

In CloudFormation (the engine underneath SAM), resources are created in a specific dependency order.

  • To create the Bucket Notification, AWS needs the Lambda Function to exist first.
  • To create the Lambda Function, AWS needs to create its IAM Role.
  • If you reference the Bucket inside that IAM Role (to give it Read permissions), the Role now waits for the Bucket.

The Cycle:

Bucket waits for Lambda → Lambda waits for Role → Role waits for Bucket.
Enter fullscreen mode Exit fullscreen mode

The "Bad" Code (What Causes the Loop)

Here is the intuitive (but wrong) way to write the template.yaml. Notice how I tried to rely on !Ref S3entinelBucket inside the function's policy.

In YAML file:

Resources:
  S3entinelBucket:
    Type: AWS::S3::Bucket

  ImageProcessorFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: .
      Handler: com.codebymathabo.s3entinel.lambda.ImageProcessor::handleRequest
      Events:
        FileUpload:
          Type: S3
          Properties:
            Bucket: !Ref S3entinelBucket # Lambda needs Bucket for Event
            Events: s3:ObjectCreated:*
      Policies:
        - S3ReadPolicy:
            BucketName: !Ref S3entinelBucket # Role needs Bucket for Permissions
Enter fullscreen mode Exit fullscreen mode

This creates the deadlock. CloudFormation cannot resolve the dependency graph.

The Fix: Break the Chain with Strings

To solve this, we must remove one of the hard dependencies. We can’t change the Event trigger (that requires a real resource reference), but we can cheat on the IAM Policy.

Instead of asking CloudFormation to "Wait for the Bucket to exist and then give me its name" (!Ref), we can construct the bucket name manually using a deterministic naming pattern.

In AWS, if you know the Region and Account ID, you can predict the bucket name before it is even created.

The "Good" Code
Here is the fix I implemented in S3entinel. I replaced the !Ref in the Policy section with a !Sub string interpolation.

In YAML file:

Resources:
  # Define the Bucket Name explicitly using ID and Region
  S3entinelBucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketName: !Sub 's3entinel-iac-${AWS::Region}-${AWS::AccountId}'

  ImageProcessorFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: .
      Handler: com.codebymathabo.s3entinel.lambda.ImageProcessor::handleRequest

      # Reconstruct the name string manually.
      # This grants permission to "what the bucket WILL be named"
      # without forcing the IAM Role to wait for the Bucket resource.
      Policies:
        - S3ReadPolicy:
            BucketName: !Sub 's3entinel-iac-${AWS::Region}-${AWS::AccountId}'

      Events:
        FileUpload:
          Type: S3
          Properties:
           # Keep the hard link for the trigger
            Bucket: !Ref S3entinelBucket 
            Events: s3:ObjectCreated:*
Enter fullscreen mode Exit fullscreen mode

Why This Works

By using !Sub 's3entinel-iac-${AWS::Region}-${AWS::AccountId}', we are giving the IAM Role a string of text, not a resource dependency.

  1. CloudFormation creates the IAM Role immediately (it only needs the string, not the bucket).

  2. It creates the Lambda Function using that Role.

  3. Finally, it creates the S3 Bucket (which depends on the Lambda for the notification trigger).

The loop is broken, the stack deploys, and the Circular Dependency error disappears.

Code Reference:

Top comments (0)