Dynamic number of SNS subscribers using AWS CloudFormation ForEach

CloudFormation is an AWSome service that empowers users to define and provision AWS resources using simple JSON or YAML templates. It allows for the automated deployment and management of complex cloud infrastructures, ensuring consistency and reducing manual errors. With CloudFormation, you can describe cloud-based applications and services in a declarative way, making it easier to update, delete, and recreate resources as needed. By providing infrastructure as code capabilities, CloudFormation streamlines the process of building and maintaining AWS environments, promoting scalability and reliability in cloud deployments.

The problem arises when you start using it at scale. One of the most common use-cases for CloudFormation is to simply deploy some basic templates/resources. While Terraform is well-suited for complex projects, CloudFormation, in my opinion, is superior when you only need to accomplish a simple task. For instance, I often need to set up some basic alerting in customer environments, such as budget alerts or basic CloudWatch alarms.

One of the most frustrating issues I have encountered is that it became increasingly difficult to create an SNS topic with a dynamic number of subscribers without modifying the actual CloudFormation template (thus avoiding the addition of extra Parameters, for example). The only way to achieve this was via Custom Resources. While this approach works fine, it necessitates the creation of additional Lambda functions as part of your template, which requires IAM roles and code maintenance over time, even when all you needed was to add another email address as a subscriber to your SNS topic.

The solution below is not flawless, and it introduces a different bottleneck that can complicate your template, but it simplifies setups like these. So, let's explore the full solution first, and then we can delve into the details of why certain parts are as they are.

AWSTemplateFormatVersion: 2010-09-09
Transform: 
  - 'AWS::LanguageExtensions'
  - 'AWS::Serverless-2016-10-31'
Description: Sample Cloudformation file for SNS subscriptions
Parameters:
  Subscribers:
    Type: String
    Description: Comma separated list of subscribers
    Default: ''
  SubscriberIndexes:
    Type: String
    Description: |
      Comma separated list of indexes.
      Should start from 0 and should be in sequence upto the number of subscribers.

      It is used as a helper to create unique logical ids for subscriptions.
    Default: ''
Resources:
  SampleTopic:
    Type: AWS::SNS::Topic
    Properties:
      DisplayName: Sample Topic
      TopicName: sample-topic
  Fn::ForEach::Subscriptions:
      - EmailIndex
      - !Split [',', !Ref SubscriberIndexes]
      - SampleTopicSub${EmailIndex}:
          Type: AWS::SNS::Subscription
          Properties:
            Endpoint: !Select [!Sub "${EmailIndex}", !Split [',', !Ref Subscribers]]
            Protocol: email
            TopicArn: !Ref SampleTopic

The first thing you notice, is that there are 2 parameters.

1. Subscribers
2. SubscriberIndexes

The param Subscribers speaks for itself. It is what we need after all. A comma (,) separated list of email addresses that should be subscribed to our new SNS topic.

So let's look at what SubscriberIndexes is for. It should be a list of "indexes" that we will need to use unfortunately, to get around some of the limitations of the Fn::ForEach function.

If you look at the documentation for it, you may notice that we need to set the key dynamically. Unfortunately, this key has some limitations as it can only contain alphanumeric characters. As you probably correctly recall, an at sign (@) or the dot (.) in the email does not count as "alphanumeric". So we need a way to generate a new index.

To get around this problem, we will actually build our loop on a new parameter called SubscriberIndexes instead of the original list of emails.

...
  Fn::ForEach::Subscriptions:
      - EmailIndex
      - !Split [',', !Ref SubscriberIndexes]
      - SampleTopicSub${EmailIndex}:
...

Now, this is all good, but we still need to get the actual emails into the resource definition. If you think about the comma separated list of emails, you will notice that they can be converted into an array using the Fn::Split function the following way:

!Split [',', !Ref Subscribers]

Once that's done, we now just need to reference the right items. This is where we will use the original EmailIndex variable. Users need to make sure that it's been set correctly, as it will fail otherwise. I don't cover it here, but it would be possible to setup some validation Rules by following the documentation.

After that, we jut need to access the right items by using the Fn::Select function. For example in this case:

!Select [!Sub "${EmailIndex}", !Split [',', !Ref Subscribers]]

Putting al this together, we will get the following template. (just make sure you set the index params right during deployment)

TLDR:

AWSTemplateFormatVersion: 2010-09-09
Transform: 
  - 'AWS::LanguageExtensions'
  - 'AWS::Serverless-2016-10-31'
Description: Sample Cloudformation file for SNS subscriptions
Parameters:
  Subscribers:
    Type: String
    Description: Comma separated list of subscribers
    Default: ''
  SubscriberIndexes:
    Type: String
    Description: |
      Comma separated list of indexes.
      Should start from 0 and should be in sequence upto the number of subscribers.

      It is used as a helper to create unique logical ids for subscriptions.
    Default: ''
Resources:
  SampleTopic:
    Type: AWS::SNS::Topic
    Properties:
      DisplayName: Sample Topic
      TopicName: sample-topic
  Fn::ForEach::Subscriptions:
      - EmailIndex
      - !Split [',', !Ref SubscriberIndexes]
      - SampleTopicSub${EmailIndex}:
          Type: AWS::SNS::Subscription
          Properties:
            Endpoint: !Select [!Sub "${EmailIndex}", !Split [',', !Ref Subscribers]]
            Protocol: email
            TopicArn: !Ref SampleTopic