Template for creating a 3-layer subnet VPC using CloudFormation's intrinsic function Fn::ForEach

In the beginning

Hello everyone,

In Japan, where I live, and abroad, I have seen articles presenting examples of templates that use Fn::ForEach to deploy as many resources of a single resource type as exist in an array (Collection).

On the other hand, we cannot judge whether it is practical if we cannot set up a reference relationship (!Ref) or dependency relationship (DependsOn) between resource A created with Fn::ForEach and resource B created with Fn::ForEach without problems.

I cannot completely deny the possibility that I may have missed something, but I could not find any articles or documents, either in Japanese or English, that strongly mention this or include an anti-pattern.

Therefore, I decided to verify what happens when I actually use Fn::ForEach on multiple resources and refer to them by their properties.

What I'm creating this time

The following is a configuration diagram. Although not shown in the diagram, RouteTable, Route, ElasticIP, etc. are also created.

First, the template form I arrived at

AWSTemplateFormatVersion: 2010-09-09
Transform: AWS::LanguageExtensions
# ------------------------------------------------------------#
# Parameters:
# ------------------------------------------------------------#
Parameters:
  ZoneList:
    Type: List<String>
    Default: a,c

  SubnetList:
    Type: List<String>
    Default: PublicSubnet,ProtectedSubnet,PrivateSubnet
# ------------------------------------------------------------#
# Mappings:
# ------------------------------------------------------------#
Mappings: 
  # ------------------------------------------------------------#
  # ZoneMappings
  # ------------------------------------------------------------#
  ZoneMappings: 
    a: 
      AvailabilityZone: a

    c: 
      AvailabilityZone: c
  # ------------------------------------------------------------#
  # SubnetMappings
  # ------------------------------------------------------------#
  SubnetMappings: 
    PublicSubnet: 
      Mapping: PublicSubnetMappings

    ProtectedSubnet: 
      Mapping: ProtectedSubnetMappings

    PrivateSubnet: 
      Mapping: PrivateSubnetMappings
  # ------------------------------------------------------------#
  # PublicSubnetMappings
  # ------------------------------------------------------------#
  PublicSubnetMappings: 
    a: 
      CidrBlock: 10.0.0.0/24

    c: 
      CidrBlock: 10.0.1.0/24
  # ------------------------------------------------------------#
  # ProtectedSubnetMappings
  # ------------------------------------------------------------#
  ProtectedSubnetMappings: 
    a: 
      CidrBlock: 10.0.2.0/24

    c: 
      CidrBlock: 10.0.3.0/24
  # ------------------------------------------------------------#
  # PrivateSubnetMappings
  # ------------------------------------------------------------#
  PrivateSubnetMappings:
    a: 
      CidrBlock: 10.0.4.0/24

    c: 
      CidrBlock: 10.0.5.0/24
# ------------------------------------------------------------#
# Resources
# ------------------------------------------------------------#
Resources:
  # ------------------------------------------------------------#
  # VPC
  # ------------------------------------------------------------#
  VPC:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: 10.0.0.0/16
      Tags:
        - Key: Name
          Value: VPC
  # ------------------------------------------------------------#
  # InternetGateway
  # ------------------------------------------------------------#
  InternetGateway:
    Type: AWS::EC2::InternetGateway
    Properties: 
      Tags:
        - Key: Name
          Value: InternetGateway
  # ------------------------------------------------------------#
  # VPCGatewayAttachment
  # ------------------------------------------------------------#
  VPCGatewayAttachment:
    Type: AWS::EC2::VPCGatewayAttachment
    Properties: 
      InternetGatewayId: !Ref InternetGateway
      VpcId: !Ref VPC
  # ------------------------------------------------------------#
  # ZoneLoop
  # ------------------------------------------------------------#
  Fn::ForEach::ZoneLoop:
    - ZoneItem
    - !Ref ZoneList
    # ------------------------------------------------------------#
    # NatGateway
    # ------------------------------------------------------------#
    - PublicSubnet1${ZoneItem}NatGateway:
        Type: AWS::EC2::NatGateway
        Properties:
          AllocationId: !GetAtt [!Sub 'PublicSubnet1${ZoneItem}EIPforNatGateway', AllocationId]
          SubnetId: !GetAtt [!Sub 'PublicSubnet1${ZoneItem}', SubnetId]
          Tags: 
            - Key: Name
              Value: !Sub PublicSubnet1${ZoneItem}NatGateway
    # ------------------------------------------------------------#
    # EIP
    # ------------------------------------------------------------#
      PublicSubnet1${ZoneItem}EIPforNatGateway: 
        Type: AWS::EC2::EIP
        Properties:
          Domain: vpc
          Tags: 
            - Key: Name
              Value: !Sub PublicSubnet1${ZoneItem}EIPforNatGateway
    # ------------------------------------------------------------#
    # Route
    # ------------------------------------------------------------#
      PublicSubnet1${ZoneItem}Route: 
        Type: AWS::EC2::Route
        Properties:
          DestinationCidrBlock: 0.0.0.0/0
          RouteTableId: !GetAtt [!Sub 'PublicSubnet1${ZoneItem}RouteTable', RouteTableId]
          GatewayId: !Ref InternetGateway

      ProtectedSubnet1${ZoneItem}Route: 
        Type: AWS::EC2::Route
        Properties:
          DestinationCidrBlock: 0.0.0.0/0
          RouteTableId: !GetAtt [!Sub 'ProtectedSubnet1${ZoneItem}RouteTable', RouteTableId]
          NatGatewayId: !GetAtt [!Sub 'PublicSubnet1${ZoneItem}NatGateway', NatGatewayId]
      # ------------------------------------------------------------#
      # SubnetLoop
      # ------------------------------------------------------------#
      Fn::ForEach::SubnetLoop:
        - SubnetItem
        - !Ref SubnetList
        # ------------------------------------------------------------#
        # Subnet
        # ------------------------------------------------------------#
        - ${SubnetItem}1${ZoneItem}:
            Type: AWS::EC2::Subnet
            Properties:
              VpcId: !Ref VPC
              CidrBlock: !FindInMap [!FindInMap [SubnetMappings,!Ref SubnetItem,Mapping],!Ref ZoneItem,CidrBlock]
              AvailabilityZone: !Sub ap-northeast-1${ZoneItem}
              Tags:
              - Key: Name
                Value: !Sub ${SubnetItem}1${ZoneItem}
        # ------------------------------------------------------------#
        # RouteTable
        # ------------------------------------------------------------#
          ${SubnetItem}1${ZoneItem}RouteTable:
            Type: AWS::EC2::RouteTable
            Properties:
              VpcId: !Ref VPC
              Tags:
                - Key: Name
                  Value: !Sub ${SubnetItem}1${ZoneItem}RouteTable
        # ------------------------------------------------------------#
        # SubnetRouteTableAssociation
        # ------------------------------------------------------------#
          ${SubnetItem}1${ZoneItem}RouteTableAssociation: 
            Type: AWS::EC2::SubnetRouteTableAssociation
            Properties:
              SubnetId: !GetAtt [!Sub '${SubnetItem}1${ZoneItem}', SubnetId]
              RouteTableId: !GetAtt [!Sub '${SubnetItem}1${ZoneItem}RouteTable', RouteTableId]

I want to express my gratitude for archiving this Youtube video

I found the video "AWS On Air ft. Accelerate your CloudFormation authoring experience with Fn::ForEach looping function" on the "AWS Events" channel.
First of all, I would like to thank the person who archived this video.

The part I want to focus on is the screen switching (scrolling) at 22:54.

https://www.youtube.com/watch?v=YSsWbHmLGTs

In this video he declares multiple Types (DynamoDB::Table and IAM::Policy) in a single Fn::ForEach.

I was initially under the mistaken impression that Fn::ForEach could only be used for each resource type (*1:1).

In fact, it was possible to create (1:n) resources for Fn::ForEach.
(This may be an oversight in the documentation text due to my assumption.)

Because of the above assumption, I spent several days setting up ForEach for NatGateway and EIP respectively,
string references.

So, I sometimes wished that "LanguageExtention" could solve the problem of placing only strings or "!Ref" in "DependsOn",

Sometimes I encountered too many errors and wondered if it was an Issue.
I thought about submitting it to "cfn-language-discussion" several times.

I even searched for blogs, both national and international, that verified how many layers of built-in function nesting "LanguageExtention" actually allows.

This video solved all those things too.
Thank you so much.

This may be a bit complicated to explain, but please bear with me.

Detailed Description

Parameters

# ------------------------------------------------------------#
# Parameters:
# ------------------------------------------------------------#
Parameters:
  ZoneList:
    Type: List<String>
    Default: a,c

  SubnetList:
    Type: List<String>
    Default: PublicSubnet,ProtectedSubnet,PrivateSubnet

I have two parameters of type List.
One is to separate Zones. The other is to separate Subnet.

By using (nesting) these later in Fn::ForEach, I assume that I can create up to 6 resources, 2 x 3 for each resource type.

Mappings

# ------------------------------------------------------------#
# Mappings:
# ------------------------------------------------------------#
Mappings: 
  # ------------------------------------------------------------#
  # ZoneMappings
  # ------------------------------------------------------------#
  ZoneMappings: 
    a: 
      AvailabilityZone: a

    c: 
      AvailabilityZone: c
  # ------------------------------------------------------------#
  # SubnetMappings
  # ------------------------------------------------------------#
  SubnetMappings: 
    PublicSubnet: 
      Mapping: PublicSubnetMappings

    ProtectedSubnet: 
      Mapping: ProtectedSubnetMappings

    PrivateSubnet: 
      Mapping: PrivateSubnetMappings
  # ------------------------------------------------------------#
  # PublicSubnetMappings
  # ------------------------------------------------------------#
  PublicSubnetMappings: 
    a: 
      CidrBlock: 10.0.0.0/24

    c: 
      CidrBlock: 10.0.1.0/24
  # ------------------------------------------------------------#
  # ProtectedSubnetMappings
  # ------------------------------------------------------------#
  ProtectedSubnetMappings: 
    a: 
      CidrBlock: 10.0.2.0/24

    c: 
      CidrBlock: 10.0.3.0/24
  # ------------------------------------------------------------#
  # PrivateSubnetMappings
  # ------------------------------------------------------------#
  PrivateSubnetMappings:
    a: 
      CidrBlock: 10.0.4.0/24

    c: 
      CidrBlock: 10.0.5.0/24

You may be wondering a bit about Mappings.
This is where you will catch a glimpse of my struggles when I look back later.

I will explain this in more detail later.

Resources that do not require Fn::ForEach

# ------------------------------------------------------------#
# Resources
# ------------------------------------------------------------#
Resources:
  # ------------------------------------------------------------#
  # VPC
  # ------------------------------------------------------------#
  VPC:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: 10.0.0.0/16
      Tags:
        - Key: Name
          Value: VPC
  # ------------------------------------------------------------#
  # InternetGateway
  # ------------------------------------------------------------#
  InternetGateway:
    Type: AWS::EC2::InternetGateway
    Properties: 
      Tags:
        - Key: Name
          Value: InternetGateway
  # ------------------------------------------------------------#
  # VPCGatewayAttachment
  # ------------------------------------------------------------#
  VPCGatewayAttachment:
    Type: AWS::EC2::VPCGatewayAttachment
    Properties: 
      InternetGatewayId: !Ref InternetGateway
      VpcId: !Ref VPC

These are determined to be created on their own, so there is no change from the usual.
They are located further outside of Fn::ForEach at the top level.

Resources required, one per Zone

  # ------------------------------------------------------------#
  # ZoneLoop
  # ------------------------------------------------------------#
  Fn::ForEach::ZoneLoop:
    - ZoneItem
    - !Ref ZoneList
    # ------------------------------------------------------------#
    # NatGateway
    # ------------------------------------------------------------#
    - PublicSubnet1${ZoneItem}NatGateway:
        Type: AWS::EC2::NatGateway
        Properties:
          AllocationId: !GetAtt [!Sub 'PublicSubnet1${ZoneItem}EIPforNatGateway', AllocationId]
          SubnetId: !GetAtt [!Sub 'PublicSubnet1${ZoneItem}', SubnetId]
          Tags: 
            - Key: Name
              Value: !Sub PublicSubnet1${ZoneItem}NatGateway
    # ------------------------------------------------------------#
    # EIP
    # ------------------------------------------------------------#
      PublicSubnet1${ZoneItem}EIPforNatGateway: 
        Type: AWS::EC2::EIP
        Properties:
          Domain: vpc
          Tags: 
            - Key: Name
              Value: !Sub PublicSubnet1${ZoneItem}EIPforNatGateway
    # ------------------------------------------------------------#
    # Route
    # ------------------------------------------------------------#
      PublicSubnet1${ZoneItem}Route: 
        Type: AWS::EC2::Route
        Properties:
          DestinationCidrBlock: 0.0.0.0/0
          RouteTableId: !GetAtt [!Sub 'PublicSubnet1${ZoneItem}RouteTable', RouteTableId]
          GatewayId: !Ref InternetGateway

      ProtectedSubnet1${ZoneItem}Route: 
        Type: AWS::EC2::Route
        Properties:
          DestinationCidrBlock: 0.0.0.0/0
          RouteTableId: !GetAtt [!Sub 'ProtectedSubnet1${ZoneItem}RouteTable', RouteTableId]
          NatGatewayId: !GetAtt [!Sub 'PublicSubnet1${ZoneItem}NatGateway', NatGatewayId]

Here are the resources directly under the first Fn::ForEach.
I hope you can get an idea of how the resources are created one by one for the two zones a and c.

Parameters:
  ZoneList:
    Type: List<String>
    Default: a,c

For example, here NatGateway is a resource needed only for PublicSubnet.

It is important not to think of them as two out of six subnets, but as two for two zones.

They can be considered as needed for a Zone within a VPC.

Resources required, one per Subnet

      # ------------------------------------------------------------#
      # SubnetLoop
      # ------------------------------------------------------------#
      Fn::ForEach::SubnetLoop:
        - SubnetItem
        - !Ref SubnetList
        # ------------------------------------------------------------#
        # Subnet
        # ------------------------------------------------------------#
        - ${SubnetItem}1${ZoneItem}:
            Type: AWS::EC2::Subnet
            Properties:
              VpcId: !Ref VPC
              CidrBlock: !FindInMap [!FindInMap [SubnetMappings,!Ref SubnetItem,Mapping],!Ref ZoneItem,CidrBlock]
              AvailabilityZone: !Sub ap-northeast-1${ZoneItem}
              Tags:
              - Key: Name
                Value: !Sub ${SubnetItem}1${ZoneItem}
        # ------------------------------------------------------------#
        # RouteTable
        # ------------------------------------------------------------#
          ${SubnetItem}1${ZoneItem}RouteTable:
            Type: AWS::EC2::RouteTable
            Properties:
              VpcId: !Ref VPC
              Tags:
                - Key: Name
                  Value: !Sub ${SubnetItem}1${ZoneItem}RouteTable
        # ------------------------------------------------------------#
        # SubnetRouteTableAssociation
        # ------------------------------------------------------------#
          ${SubnetItem}1${ZoneItem}RouteTableAssociation: 
            Type: AWS::EC2::SubnetRouteTableAssociation
            Properties:
              SubnetId: !GetAtt [!Sub '${SubnetItem}1${ZoneItem}', SubnetId]
              RouteTableId: !GetAtt [!Sub '${SubnetItem}1${ZoneItem}RouteTable', RouteTableId]

I have a further nested Fn::ForEach within the previous Fn::ForEach.
In total, six resources should be created.

As I mentioned at the beginning of this article, this is realized by ZoneList × SubnetList.

Further areas worth mentioning

EC2::Route is split in two.

    # ------------------------------------------------------------#
    # Route
    # ------------------------------------------------------------#
      PublicSubnet1${ZoneItem}Route: 
        Type: AWS::EC2::Route
        Properties:
          DestinationCidrBlock: 0.0.0.0/0
          RouteTableId: !GetAtt [!Sub 'PublicSubnet1${ZoneItem}RouteTable', RouteTableId]
          GatewayId: !Ref InternetGateway

      ProtectedSubnet1${ZoneItem}Route: 
        Type: AWS::EC2::Route
        Properties:
          DestinationCidrBlock: 0.0.0.0/0
          RouteTableId: !GetAtt [!Sub 'ProtectedSubnet1${ZoneItem}RouteTable', RouteTableId]
          NatGatewayId: !GetAtt [!Sub 'PublicSubnet1${ZoneItem}NatGateway', NatGatewayId]

The reason these are divided into two is in the properties.

GatewayId:" is required for Routes that are needed for public subnets, and "NatGatewayId:" is required for Routes that are needed for protected subnets.

Can we find a way to use "Condition" in resources in combination with "Conditions" and "If" as a way of branching these?

For example, as a value to be referenced in Conditions
Can we connect the logical ID of AWS::EC2::Route and RouteTableId with a dot to refer to the value?
I think the answer is no.

AWS::EC2::Route only returns CidrBlock in GetAtt and Ref.

This does not achieve branching.

In the first place, we often see things in Conditions that judging what the Value of a parameter is, but it is also impossible to ask CloudFormation to judging "what is being written to be referenced" for the Value of a Resource property!

And, Condition does not branch property units.

Even if all of these were fine, two Type: AWS::EC2::Route resource types would still need to be created. Compared to the previous example, the number of lines involved in Condition would just increase meaninglessly.

That is why this is a ZonalLoop.

Another reason is that private subnets do not require a route.

RouteTable for each public subnet exists

There is a RouteTable for each public subnet with a 0.0.0.0/0 route to the same InternetGateway.

Although this should only be one RouteTable, it is unavoidable when using Fn::ForEach to create a RouteTable.

It is possible to put the public one outside the top-level Fn::ForEach, with only the protected and private ones, or the protected subnets, in the scope of the ZonalLoop, but this is not my preference. I feel this is not consistent with the policy of applying Fn::ForEach as much as possible, which was decided when the template was first created.

Fn::FindInMap is nested inside Fn::FindInMap

        # ------------------------------------------------------------#
        # Subnet
        # ------------------------------------------------------------#
        - ${SubnetItem}1${ZoneItem}:
            Type: AWS::EC2::Subnet
            Properties:
              VpcId: !Ref VPC
              CidrBlock: !FindInMap [!FindInMap [SubnetMappings,!Ref SubnetItem,Mapping],!Ref ZoneItem,CidrBlock]

I expressed earlier about the Mappings part that there was "distress",

I would not have done this if I could have used Sub within the!FindInMap.
SubnetItem and ZoneItem because they can be string concatenated.

FindInMap inside to see if it is public, protected or private.
Then you can find the Key in the following part.

The corresponding Value has the name of the following Mapping.

Mappings: 
  # ------------------------------------------------------------#
  # SubnetMappings
  # ------------------------------------------------------------#
  SubnetMappings: 
    PublicSubnet: 
      Mapping: PublicSubnetMappings

    ProtectedSubnet: 
      Mapping: ProtectedSubnetMappings

    PrivateSubnet: 
      Mapping: PrivateSubnetMappings

This Value is converted to a Key in the first argument of the outer FindInMap.

From the Key of each of the following Mappings, a single letter (a or c) of the Zone specified in the second argument is referenced to Cidr as the Key.

# ------------------------------------------------------------#

  # PublicSubnetMappings

  # ------------------------------------------------------------#

  PublicSubnetMappings: 

    a: 

      CidrBlock: 10.0.0.0/24

<span class="na">c</span><span class="pi">:</span> 
  <span class="na">CidrBlock</span><span class="pi">:</span> <span class="s">10.0.1.0/24</span>



    

    




# ------------------------------------------------------------#

  # ProtectedSubnetMappings

  # ------------------------------------------------------------#

  ProtectedSubnetMappings: 

    a: 

      CidrBlock: 10.0.2.0/24

<span class="na">c</span><span class="pi">:</span> 
  <span class="na">CidrBlock</span><span class="pi">:</span> <span class="s">10.0.3.0/24</span>



    

    




# ------------------------------------------------------------#

  # PrivateSubnetMappings

  # ------------------------------------------------------------#

  PrivateSubnetMappings:

    a: 

      CidrBlock: 10.0.4.0/24

<span class="na">c</span><span class="pi">:</span> 
  <span class="na">CidrBlock</span><span class="pi">:</span> <span class="s">10.0.5.0/24</span>



    

    

How many rows have been reduced?

Below is a comparison of the results with all extra lines removed.
Without Fn::ForEach: 190 lines
Created with Fn::ForEach: 110 lines

It can be said that the number of lines has been reduced by about one-half.

How these are displayed in the CloudFormation designer

When read without using Fn::ForEach

When Fn::ForEach is used to read

The CloudFormation designer expresses the presence of CustomResouce about Fn::ForEach.
I am curious about this difference and wanted to consider the reason at a later date.

That is all

Some of the difficulties involved in this endeavor may seem quite comical to those who use the CDK.

I myself would like to leave aside for the moment whether the introduction of Fn::ForEach is great or not, or whether this template is beautiful in its own right.

I feel that the appearance of this built-in function is not just an addition of functionality, but rather an attempt to destroy some of the concepts that CFn had retained and to find a way to change its form.

Since this is too long an article, I have decided to summarize my pattern of failure in another blog.
I will be happy if there is something useful for anyone in the world.

Finally, not everything is certain even in my mind. If there is any lack of understanding or mistakes in the article, I would like to correct them immediately. I want to let you know that I am always grateful for the words generated by the experience of experts from all over the world.

Thank you for reading.