SIMPLE AWS DEPLOYMENTS ACROSS REGIONS AND ACCOUNTS WITH CDK

Infrastructure as Code has made it possible to consistently repeat deployments and keep track of the details in version control. AWS CDK, the Cloud Development Kit, is a native Infrastructure as Code solution by AWS that allows you to write you infrastructure definitions in normal programming languages. Deploying a collection of cloud resources with it is as simple as running cdk deploy.

Well, simple... It is simple, as long as your solution spans a single AWS Account and Region. Things tend to become more complicated when multiple regions or other accounts are involved. Regions and accounts are isolated in AWS. Coordinating multi-region or multi-account deployments has often required us to write custom deployment scripts. However, CDK offers us the right set of tools to perform these kinds of deployments in a clear and concise manner.

In this post, we will show how we can use CDK to pick up this tangle of cross-environment deployments and make them simple! By defining our environments either with files in our version control, or with environment variables, we will show a variety of methods. No matter how you want to deploy your code, this post will show you how to do it.

A quick primer on definitions

Two core concepts in CDK are Stacks and Apps. AWS describes a Stack as a "unit of deployment", and an App as "a container for one or more stacks".

A CDK Stack is roughly equivalent to a CloudFormation stack.
It is a template you deploy to a region on an AWS account. A stack can contain many types of AWS resources, but they all exist in the same account and region (with some exceptions for special region-bound resources). So a Stack is the smallest, indivisible unit of deployment.

When we want to deploy a single project across regions or even accounts, we have to deploy multiple Stacks. Using CDK, we have a logical abstraction of a project that consists of multiple stacks: an App. After all, it is a collection of stacks - nothing says it has to be in one region or even in one account.

In CDK lingo, a combination of an account and a region is an Environment.

A cross-environment Application

We will first show how to define an Application that spans multiple Environments.

For the examples in this post, we will be using Python, but the concepts can easily be applied to the other supported languages. The CDK documentation that we will link to shows an example of each individual concept in every supported language.

The AWS Documentation describes the way you can pass arguments to Stack Construct to target an account or region in various programming languages. For demonstration purposes, we will assume we have two different stacks, each of which needs to go to a different account and region. Maybe we are demonstrating a cross-account notification system, with one stack being the sender, and the other being the subscriber in the demo!

The simplest example uses static accounts and regions for its environments:

import aws_cdk as cdk

from demo.event_publisher_stack import EventPublisherStack
from demo.event_receiver_stack import EventReceiverStack

app = cdk.App()
EventPublisherStack(app, "EventPublisherStack",
                    env=cdk.Environment(
                        account='111111111111',
                        region='eu-west-1',
                    ))
EventReceiverStack(app, "EventReceiverStack",
                   env=cdk.Environment(
                       account='222222222222',
                       region='eu-west-1',
                   ))

app.synth()

You can deploy all stacks in this application in one command:

# Option A: deploy each stack in one command
cdk deploy --all

Or with multiple separate commands, which can be beneficial if you need to assume another role for each stack:

# Option B: deploy each stack individually
cdk deploy EventPublisherStack
cdk deploy EventReceiverStack

For a simple demo, using static account ids and regions in your Environments might be good enough. But if this is an Application that will end up in production, you should not hardcode these values.

The CDK documentation has good details on the concepts we will show, though it helps to have a starting point. Because CDK is written in a full programming language, you can use any method you normally would to pass the configuration to it.

To get you started, we have prepared a few short examples showing two approaches you can take.

Method 1: Environment variables

Environment variables provide a lot of flexibility.

import os

import aws_cdk as cdk

from demo.event_publisher_stack import EventPublisherStack
from demo.event_receiver_stack import EventReceiverStack

app = cdk.App()

EventPublisherStack(app, "EventPublisherStack",
                    env=cdk.Environment(
                        account=os.environ['TARGET_ACCOUNT_PUBLISHER'],
                        region=os.environ['TARGET_REGION_PUBLISHER'],
                    ))
EventReceiverStack(app, "EventReceiverStack",
                   env=cdk.Environment(
                       account=os.environ['TARGET_ACCOUNT_RECEIVER'],
                       region=os.environ['TARGET_REGION_RECEIVER'],
                   ))

app.synth()

Where they truly shine is in deployment pipelines. You can define these variables as properties of the target environment.

When deploying from your local command line, a deployment command will likely look like this:

TARGET_ACCOUNT_PUBLISHER=111111111111 TARGET_REGION_PUBLISHER=eu-west-1 TARGET_ACCOUNT_RECEIVER=222222222222 TARGET_REGION_RECEIVER=eu-west-1 pipenv run cdk deploy --all.

Very concise, we know. That's one disadvantage of using environment variables: when run from your local development machine, the deployment commands can become a bit verbose.

Method 2: Configuration files

By putting your environments in configuration files, you make it simpler to see the intended result in the code itself. This does mean putting your account id's in your code. The debate whether this is a security risk has been responded to by AWS (and commented on by others in the community), and this should pose no security risk.

#!/usr/bin/env python3
import json
import logging
import os
import sys
from pathlib import Path

import aws_cdk as cdk

from demo.event_publisher_stack import EventPublisherStack
from demo.event_receiver_stack import EventReceiverStack

log = logging.getLogger("app")

environment = os.environ.get('ENVIRONMENT', None)
if environment is None:
    log.error("Make sure a valid environment is specified in the ENVIRONMENT env var.")
    sys.exit(1)

config_file = Path(f"configs/{environment}.json")
if not config_file.is_file():
    log.error(f"Make sure the config file for the given ENVIRONMENT exists in ./configs/{environment}.json and is readable.")
    sys.exit(1)

with config_file.open() as f:
    config = json.load(f)

app = cdk.App()
EventPublisherStack(app, "EventPublisherStack",
                    env=cdk.Environment(
                        account=config['publisher']['account_id'],
                        region=config['publisher']['region'],
                    ))
EventReceiverStack(app, "EventReceiverStack",
                   env=cdk.Environment(
                       account=config['receiver']['account_id'],
                       region=config['receiver']['region'],
                   ))

app.synth()

We can define our configurations for our deployment stages by placing json files in ./configs. Our developers can define their own configuration file in there, and keep it out of version control if they do not want to clutter the project with their account ids.

> ls ./configs
acceptance.json
dev.json
production.json
test.json

> cat ./configs/test.json
{
  "publisher": {
    "account_id": "111111111111",
    "region": "eu-west-1"
  },
  "receiver": {
    "account_id": "222222222222",
    "region": "eu-west-1"
  }
}

If we then run CONFIG=test cdk deploy --all, our publisher stack will be deployed to account 111111111111, and our receiver stack to account 222222222222, both in region eu-west-1.

An added benefit of this approach is that you can store other configuration in these json files as well. Examples might be the names of artifact buckets or external services that differ depending on the target environment. These can then be passed to the Stacks as extra configuration.

Bootstrapping all your environments for deployment

To deploy to any Environment, that environment must first be bootstrapped by CDK. During bootstrapping, CDK creates the resources it needs to deploy to an environment, such as S3 buckets for artifacts, but also IAM roles and policies that are assumed during deployment. When you run cdk deploy, the CDK command line will attempt to assume one of these roles in the target environment to perform the deployment.

If all your stacks live in the same account, or if you deploy each stack individually, a simple cdk bootstrap in every environment is enough. But if your application spans account, we need a bit more.

Deployments to multiple accounts

If your Application spans accounts, and not just regions, being able to deploy with a cdk deploy --all is a little more involved. You will need to extend trust to assume the CDK IAM resources from one account to another. To do so, modify your cdk bootstrap as follows:

# This will deploy to account 222222222222, so make sure your CLI has a valid login session for that account.
cdk bootstrap aws://222222222222/eu-west-1 --trust 111111111111 --cloudformation-execution-policies "arn:aws:iam::aws:policy/YOUR_CDK_POLICY"

By specifying the trusted account with --trust, the IAM role in account 222222222222 will be modified to be assumed by roles in account 111111111111. If the role or user you assume in account 111111111111 are then allowed to assume this role, you can deploy to the environment of account 1111111111 and account 2222222222 at the same time.

Do note the explicit --cloudformation-execution-policies in this command. When deploying with CDK, the deployment role will assume this policy to perform the deployment. With an unchanged cdk bootstrap, this will default to AdministratorAccess. When bootstrapping an account with a --trust of another account, CDK chooses to make you explicitly choose this role. Using the default of AdministratorAccess carries potential security risks, especially when you allow other accounts to assume this role as well. Create a role with the minimal permissions required to deploy your applications and pass that role to --cloudformation-execution-policies when bootstrapping your environments.

Looking back

CDK offers us powerful abstractions and tools to manage complex applications. These help us truly unlock the promise and power of the cloud. Infrastructure that spans multiple regions is one of these promises, that have been a hard concept to do encapsulate in your Infrastructure as Code for too long a time. By correctly using the capabilities of CDK, we truly unlock this promise.

In this post, we have given you some handholds to get started with these types of deployments. We have shown two approaches to defining your multi-account or -region environments: By using configuration files, we define our environments in a way that's easy to include in our codebase. Alternatively, by using environment variables, we can easily deploy our apps to varying environments.

Of course, there's nothing stopping using another method entirely - that's what having a real programming language to back your Infrastructure as Code definitions enables you to do!