DEV Community: Dakota Lewallen

Configuring CI/CD For Your CDK Application, With CDK

Dakota Lewallen — Tue, 01 Apr 2025 12:53:04 +0000

As your project gains traction, it will likely exceed what is reasonable for manual CLI deployments. At this point, you’ll need to decide how to automate your deployments. Since CDK is a CloudFormation (or Terraform) synthesizer, setting up a process to accomplish this is fairly open-ended. All you really need is a compute space capable of running CLI commands.

CDK Pipelines Construct

CDK Pipelines is an opinionated construct library. - the docs

I’m starting with this because it is what I personally have the most experience with, and is built by the CDK team. Bias aside, it has saved me from bad deployments a number of times and I do recommend it. But admittedly, it can be finicky sometimes.

Pros

It’s included in the CDK library.
There is no need to install additional dependencies.
It’s maintained by the same team that maintains the rest of the library.
- Typical third-party library concerns aren’t relevant due to this being internal to the library.
It is moderately easy to set up.
- The most basic usage involves simply wrapping the necessary constructs around the stack you wish to deploy.
It is native to AWS.
- In the end, it deploys CodeSuite services to perform the process.
- There is no need to cross lines between companies like you might with Github, CirceCI, or other CI/CD platforms.

Cons

It’s slow to run.
- Compared to other processes, the time it takes for a commit to happen, the process to run, and the changes to get propagated is long.
The Self Mutate step is a lynchpin and a black box.
- It is very frustrating to debug if something does go wrong on this step. There are common reasons that can be found via traditional things like “googling”. But when this step breaks, it’s usually not pleasant.
- Redeeming quality, you can disable it. I have not attempted a setup with it disabled. As 99.999% of the time, the step works no problem. But oh boy, that .001% of the time can be a doozy.
It can be overkill for some uses.
- If you’re deploying a lambda or two, the setup for the Pipelines construct is small. But its output is significant by comparison.
- As I mentioned at the start, this is a tradeoff. If the project is destined to be much more than a lambda or two, this might only matter temporarily. But if the purpose is small, you may want to try an alternative process.

Example

Github Actions

Going down this route can be an easy next step after outgrowing manual deployments. You can take whatever you were already doing, stick it in a YAML file, and call it a day. But you’re essentially stepping back into older forms of IaC. If you’re versed in those tradeoffs, you’ll probably manage fine. If not, this might not be the route for your project.

Pros

Easy-as-pie setup
- Getting GitHub Actions running is as simple as adding a YAML file to your repo. In development, it’s hard to get much simpler than that.
Coexists with your code in Github
- An underrated point. Dramatically simplifies the reasoning process through what’s happening, when you can see the changes directly next to what’s happening.
Template files
- For most use cases, these are simpler to reason through than a coding language. And nowadays, they have quite a bit of extensibility, meaning they can grow as the project grows.

Cons

Template files
- But not nearly as much as a coding language. Supporting the process with an interpreter or compiler vastly outperforms template files in terms of extensibility.
External System
- It’s more challenging to integrate with other AWS services.
- Incur networking charges for moving information in and out of the AWS cloud. Along with other networking challenges you may face when dealing with VPCs.
Security and Compliance are made more difficult.
- Credentials to be managed
- External attack surface
- Less control over the build environment

Example

Roll your own CodePipeline

For a smooth, refreshing blend of the previous two approaches. You can develop an in-house pipeline using the CodeSuite L3 constructs directly. This approach allows you to maintain everything in your AWS accounts while providing a smooth transition. The caveat with this one is, “With great power comes great responsibility”.

Pros

Contained within AWS
Allows you to use template files if you’d like. But does not require them.
- As you can see in the example below, you can build the buildspec in your language of choice. Or you can use the fromAsset or fromSourceFile APIs to pull a templated Buildspec from within the repo.
Can produce simple setups that run quickly.
- The keyword here is can. Akin to Github Actions, you can start out by wrapping your manual process with a bash script and having CodeBuild run that. But whether or not it stays simple and fast is up to you.
Can handle performing tasks that the Pipeline constructs cannot

… very specifically it is not built to use CodeDeploy to deploy applications to instances, or deploy your custom-built ECR images to an ECS cluster directly

Cons

You are responsible for a large amount of the process.
- AWS provides quite a few things, like build images, visualizations, and such. But most of the “important” details you will have to sort yourself. If you have strict compliance or regulations you have to adhere to, this might be a boon. If you don’t, this might be a burden.
Can hide complexity
It is possible to have a Source and a Build step and just stick everything in a bash file that the build step runs. Essentially allowing you to tuck away the meat of what’s going on.

Example

Conclusion

One of the best reasons for going with CDK is its open-endedness and that doesn’t stop once you deploy. You can take your CDK code and deploy it where you like and how you like it. Adapting it to fit your needs.

Managing My AWS Certified Status

Dakota Lewallen — Wed, 26 Feb 2025 17:00:00 +0000

Assessing where I stand

As of writing this, I’ve managed to obtain the Cloud Practitioner and Certified Developer - Associate. Unfortunately, the last time I sat an exam was in November of 2022. Meaning if I would like to maintain my certified status, I’ll need to sit at least one exam before November of this year.

Why stay certified in the first place?

There are simple things like the shiny badge, certified swag, and the certified lounge at Re:Invent. But personally I believe going through the process leads me to creating better solutions for my users. Getting certified through AWS goes beyond memorizing pricing models and feature lists. To score well on the exams, you have to spend time getting to know the services to their core. Understanding what they seek to accomplish for you and your users. Having this knowledge while developing solutions for people can be a massive difference maker.

What certs do I want/need to get?

AWS maintains this very helpful page that lists what you would need to accomplish in order to maintain a given certification. While it looks very nice, I’ll save you the click and put the information relevant to my situation below.

Based on this, at a minimum, I would need to sit the Developer Associate exam again in order to maintain my two current certifications going into the future.

But I’m not here to do the minimum.

Therefore, my only option is to sit the DevOps - Professional exam 😁.

“But what about Gen AI” - you (probably)

So if you haven’t been living under a rock, you’ve probably heard about Gen AI and tools like ChatGPT. Recently, AWS launched a foundational-level certification aptly named “AWS Certified AI Practitioner”. As someone who would like to consider themselves a “Full-Stack” developer. I imagine Gen AI will soon become a regular member of said stack. While I did get a crash course in the fundamentals this year while attending Re:Invent, I still lack the knowledge to honestly say, “I could build an app or feature from front to back that centers around Gen AI”.

So, even before sitting the DevOps - Pro exam this year, I will obtain the AI Practitioner certification.

What’s a good goal, without a stretch goal?

Who says the sky’s the limit? Not me! Because I’m also going to set a stretch goal of obtaining a third certification in 2025. The “AWS Certified Solutions Architect - Associate”. For some (looking at you golden jackets 😅) three certifications in a year might not be a stretch. But for me, it will be!

Even beyond trying to set lofty goals, I do believe having a certification more focused on higher-level concepts of using AWS services would be beneficial, especially given that I’m already branching out with the AI practitioner certification. So, regardless of whether I can make it happen within the 2025 calendar year, I will be trying to sit this exam—just not before the other two.

So we’ve got goals. What’s the plan?

Study
Practice
Build

For all three of these certifications, there’s a whole lot of information that is currently not in my head and will need to be there before I sit the exams. The solution for that is going through a series of courses made by a fellow member of the AWS community, Stephane Maarek. I’ve taken his courses for both of my existing certifications, so I trust his courses for these certifications will hold true.

Additionally, I’ll be taking Jon Bonso's practice exams. Don’t get me wrong; you could likely pass the exam solely by taking a prep course. But the best way I’ve found to have the confidence to do something is by having already done it. So, pairing a course with taking a few well-made mock exams is how I’ve prepared in the past. If it’s not broke, don’t fix it.

One final preparation step I’m taking is to build a ChatGPT-like product over the top of Bedrock, Amazon’s Gen AI wrapper service. This is an OSS project titled “My-SDC” (My Self Deployed Chat). The aim is to deliver the best of both worlds. Having the conversational interface of ChatGPT but adding enhanced capabilities that Bedrock provides, like the flexibility of model-switching.

This is where we depart. For now.

It is the beginning of a new year, and these are some lofty aspirations. I’ve put pen to paper and set a course forward. Check back here as the year goes on, and hopefully, come January of 2026, my chart will look a bit more like this.

Save Time and Money by Shifting HIPAA Compliance Checks Left with CDK-Nag

Dakota Lewallen — Sat, 15 Feb 2025 14:07:19 +0000

Originally published to: makingitup.substack.com

With cdk-nag you can check your infrastructure before it’s deployed. Potentially preventing issues before they happen. Saving you time and money.

Credit

TL;DR: Here’s a working repo with a sample configuration.

Getting Started

The steps in this tutorial will assume you already have a CDK project made. If you don’t you can follow this guide from the docs or clone the sample repo from above.

1. Install cdk-nag

npm i cdk-nag

Simple as.

2. Import and setup Rules and Packs

In your root application file, you’ll need to import the packs and rules you’d like to have applied to your infrastructure. Once available, you hook the rules into your application by adding stack aspects.

3. Output issues in flatfile formats (Optional)

It’s great that you can now check for issues. But unless you take the time to develop a system to bring these issues up and out, they will likely just be disregarded.

To remedy this, you should either import the NagReportLogger provided as part of the cdk-nag project or develop your own. The logger they supply can output JSON or CSV, making it ideal for integrating with existing CI/CD systems. If this isn’t enough, they have a great guide on how to implement your own custom logger.

Recap

Fancy buzzwords aside. By including cdk-nag as part of your application, you can check for potential HIPAA violations as early as the code is written. Preventing the lengthy debug cycle of

Write
Build
Deploy
Validate

And reducing it to

Write
Build

Conclusion

The savings only begin there. Out of the box cdk-nag supplies rule packs for

NIST 800-53 rev 4
NIST 800-53 rev 5
PCI DSS 3.2.1

As well as the ability to build your own.

Don’t want developers deploying g6e.24xlarge instances to run the latest LLM? You can write a rule for that. The abilities provided can have tremendous ROI with very little upfront investment. However, the ROI you could see by investing in engineering power to shape what your cloud can and cannot look like is vast.

Re:Invent Reflections

Dakota Lewallen — Fri, 13 Dec 2024 19:13:13 +0000

Last year, I wrote

Up until last week (as of 12/3/23) I would say I’ve been something of an AI pessimist. However, after getting hands-on with services like SageMaker, and Q, and seeing how products are starting to leverage it as part of their solutions, I’m starting to see where it could come into play.

Twelve months (and another Re:Invent) later, it's pretty easy to say Gen AI is at the forefront of software. I believe we were entering the frontier last year, and now we're beginning to see the first semblances of towns start to set up shop. The trailblazers are now beginning to bring products to the market that greatly simplify the deployment process of Gen AI applications. I expect this coming year to bring an onslaught of Gen AI products into the B2B and B2C spaces.

Traditionally speaking, software systems are (for the most part) deterministic. You build a product that houses tools containing predefined, deterministic outcomes. Take reporting, for example. You probably collected data from sources on your customer's behalf, housed it in some persistence layer, and served tools to chop and slice the data in various manners back to them. Someone with that express purpose in mind is building all of that. Additionally, the user has to understand both the system you provide and the underlying concepts to truly be able to leverage it to its fullest extent.

With Gen AI in the equation, we can now build systems that can take in the nearly infinite set of natural language communication possibilities and (hopefully) deliver the same breadth back to them. This removes the current bottleneck, which is the very narrow predetermined confines of the applications that we build today. I predict that in the wave of Gen AI products to come, the ones that carry this as a core tenant will be here for the long haul.

The example of this I routinely saw given was

If you ask an LLM, “What is 1+1?” you might get five back. But ask it to help you plan your week in Cabo, and you’ll be amazed.

“1+1” has a very narrow predeterminable solution. Your week in Cabo does not.

Continuing the line of thought from last year, you need to build your systems so they can fail gracefully. You have the pieces available; you just need to wire them together. As part of building an evolutionary product, you need to trust that you can step backward if you make a misstep. Doing so makes taking a misstep much less punishing and widens the range of possibilities you can explore.

Gunnar Grosch shared a concept during his wonderful presentation: continuous configuration. My understanding is that within your application, you should have it routinely polling for configuration values and adapting to them as time goes on rather than the standard practice of redeploying. All this is made possible by managing your application, infrastructure, and configuration through CDK.

Conclusion

If I had to draw a through line for all these ideas, Gen AI is broadening the possibilities regarding what software can deliver. “What points on the horizon are worth going to?” remains a question to be answered. However, the solutions that believe “Everything fails all the time” will most likely have the depth necessary to make it to the horizon.

Looking forward to meeting you all out on the horizon.

How to Migrate From CodeCommit

Dakota Lewallen — Fri, 09 Aug 2024 00:37:11 +0000

ICYMI

AWS recently announced that they will no longer be onboarding new customers for a number of services. Potentially one of the most painful being CodeCommit. While shuttering projects that aren't up to snuff isn't out of the ordinary, the suddenness with which they delivered this might be.

Luckily for you, you found this article. Where I'm going to show you how to quickly and easily migrate your code away from CodeCommit and to another source control service provider.

Prerequisites

Ensure you're setup with a source control provider. The most common choices on the market are Github, GitLab, and BitBucket. At minimum you'll need to make an account with the provider of you're choice.

Once you've completed that you'll need to make a repository with your new provider to house your code. Whichever provider you've chosen will have guides made that you can follow for this step. If you had any special repository configuration (e.g. branch protections) you may want to consider setting them up now.

The only caveat I have to add here is to ensure you don't commit to the new repo. We will be migrating our branches and their history to the new provider. Committing to your new provider before migrating will almost certainly cause conflicts (particularly to your main/master branch).

Getting Started

Before we get into the meat and potatoes, I want to outline our problems.

Moving all the contents of our repository from one remote repository to another

"moving house".

Updating all local repositories to use the new remote

Make sure all our friends know we've moved.

Updating automated systems to use the new remote

Make sure the government knows we've moved.

As promised, I've made a repo with some shell scripts to help make this easier.

Moving House

Now let's get down to business.

To start ensure:

your CLI is in the cloned helper repo
you have the absolute path(s) of the repo(s) being moved
you have the url(s) of the new remote repo(s)

From there you can plug all the above into either


$: bash main.sh /full/path/to/repo ${newRemoteUrl} ${newRemoteName}

Or if you have many repositories that need moved:


$: bash many.sh ${newRemoteUrl} ${newRemoteName}

Ensure you answer yes to both of the questions (pushing to the new remote and replacing origin in your local repository.)

This initial run of the scripts will add the new remote to your local repository and push your version of the history. It runs a git fetch so it should be up-to-date as long as you have an active internet connection. You then will have the option to update your local repository's origin remote to the new remote.

Updating Your Friends

Now ensure everyone working on the repo(s) also updates their local. Make sure everyone pushes any branches that they may have worked stored on locally. Then have them copy the previous step with one change.

Answering no to the push question


$: bash main.sh /full/path/to/repo ${newRemoteUrl} ${newRemoteName}


$: bash many.sh ${newRemoteUrl} ${newRemoteName}

Now on these later runs the script will add the new remote, but will not push to the new remote. Again as part of the script they can follow precedent and decide if they want the new remote to replace CodeCommit as origin.

Updating the Authorities

For this last step, you may be able to use the scripts I've provided, depending on your CI/CD architecture. But this bit likely will be out of scope and up to you to make the changes. If you run into issues, there are tons of amazing people who would love to help on

re:Post
AWS StackOverflow Collective
The Support Center within your AWS account
Many other platform and tool specific forums

Conclusion

Having to migrate Git providers can be an absolute nightmare with long tail consequences. Hopefully, with these scripts and the framework I've outlined you can make short work of what can be a daunting task.

Best of luck and hope this helps!

Find me on Mastodon | LinkedIn | Github | Substack

To Build or to Reuse? A CDK Question

Dakota Lewallen — Thu, 11 Apr 2024 11:04:10 +0000

Unless you are starting on a fresh application, the project you are working on will likely have resources already in use. One of the benefits of using CDK over other Infrastructure-as-Code tools is conditionals! Along with the ability to import resources via their ARN, we can create solutions that can use existing resources, or create new ones. This means we can deploy into both fresh and existing environments from the same codebase!

Getting Started

Let’s assume you have a CDK application made and have bootstrapped the target Region & Account. Starting with a new stack, we can begin to scaffold what we’ll need.

import { Stack , StackProps } from 'aws-cdk-lib';
import { Construct } from 'constructs';

export interface NewStackProps extends StackProps {};

export default class NewStack extends Stack {
  constructor(scope: Construct, id: string, props: NewStackProps) {
    super(scope, id, props);
  }
}

Working backward, we’ll say our example use case is to add a new lambda that processes data once it’s dropped in a bucket. For deploying into production, the bucket is already in use and managed by another project. But for the testing environment, there is no bucket. So we will need to make one.

To begin let’s add the lambda, as it’s a constant regardless of where it’s deployed. So the implementation is simple.

import { Stack , StackProps } from 'aws-cdk-lib';
import * as node from 'aws-cdk-lib/aws-lambda-nodejs';
import { Construct } from 'constructs';

export interface NewStackProps extends StackProps {};

export default class NewStack extends Stack {
  public readonly bucketProcessor: node.NodeJsFunction;
  constructor(scope: Construct, id: string, props: NewStackProps) {
    super(scope, id, props);

    this.bucketProcessor = new node.NodeJsFunction();
  }
}

Easy enough!

Now onto the juicy stuff. We’ll add a property to the NewStackProps interface to have the bucket ARN supplied externally.

import { Stack , StackProps } from 'aws-cdk-lib';
import * as node from 'aws-cdk-lib/aws-lambda-nodejs';
import { Construct } from 'constructs';

export interface NewStackProps extends StackProps {
  // Mark the property as optional, this way we can _optionall_ build the bucket if it's not supplied
  bucketArn?: string;
};

export default class NewStack extends Stack {
  public readonly bucketProcessor: node.NodeJsFunction;
  constructor(scope: Construct, id: string, props: NewStackProps) {
    super(scope, id, props);

    // pull the arn out into a construct-scoped variable for later use.
    const { bucketArn } = props;

    // Tip: For ultra clean code, you would want to add validation against this `bucketArn` string to confirm it is in fact an ARN. 

    this.bucketProcessor = new node.NodeJsFunction();
  }
}

Now armed with the knowledge of “Do we have an existing bucket”, we can either make a new one or reuse the old one

import { Stack , StackProps } from 'aws-cdk-lib';
import * as node from 'aws-cdk-lib/aws-lambda-nodejs';
import * as s3 from 'aws-cdk-lib/aws-s3';
import { Construct } from 'constructs';

export interface NewStackProps extends StackProps {
  // Mark the property as optional, this way we can _optionall_ build the bucket if it's not supplied
  bucketArn?: string;
};

export default class NewStack extends Stack {
  public readonly bucketProcessor: node.NodeJsFunction;
  constructor(scope: Construct, id: string, props: NewStackProps) {
    super(scope, id, props);

    const { bucketArn } = props;

    this.bucketProcessor = new node.NodeJsFunction();

    let bucket;
    if (bucketArn) {
      bucket = s3.Bucket.fromBucketArn(this, 'importedBucket', bucketArn);
    } else {
      bucket = s3.Bucket(this, 'createdBucket')
    }
  }
}

To wrap up, we’ll add a property to the stack so that the bucket can be referenced elsewhere in the CDK application and hook the lambda up to the bucket so it can process events.

import { Stack , StackProps } from 'aws-cdk-lib';
import * as node from 'aws-cdk-lib/aws-lambda-nodejs';
import * as s3 from 'aws-cdk-lib/aws-s3';
import * as s3n from 'aws-cdk-lib/aws-s3-notifications';
import { Construct } from 'constructs';

export interface NewStackProps extends StackProps {
  // Mark the property as optional, this way we can _optionall_ build the bucket if it's not supplied
  bucketArn?: string;
};

export default class NewStack extends Stack {
  public readonly bucketProcessor: node.NodeJsFunction;
  public readonly bucket: s3.IBucket;
/**
Use of the interface here is important. The `fromBucketArn` method returns an instance of the interface. But the new Bucket returns `s3.Bucket` which _implements_ the `IBucket` interface. So specificying the property as `IBucket` ensures that we can house _either_ the imported or created bucket. The tradeoff is that we lose access to some of the created buckets "fancier" features.
**/

  constructor(scope: Construct, id: string, props: NewStackProps) {
    super(scope, id, props);

    const { bucketArn } = props;

    this.bucketProcessor = new node.NodeJsFunction();

    if (bucketArn) {
      this.bucket = s3.Bucket.fromBucketArn(this, 'importedBucket', bucketArn);
    } else {
      this.bucket = s3.Bucket(this, 'createdBucket')
    }

    this.bucket.addEventNotification(s3.EventType.OBJECT_CREATED, new s3n.LambdaDestination(this.bucketProcessor), {prefix: 'some/object/prefix/*'});
  }
}

Conclusion

With CDK, it’s possible now more than ever to reuse infrastructure patterns across environments. No matter the state the target environment is in. I would even consider this the tip of the iceberg! To go even further, you could make a custom L3 construct containing the reuse-or-build logic to simplify the process further! The possibilities are endless!

Find me on LinkedIn | Github | Mastodon

Multi-region Deployments with CDK

Dakota Lewallen — Thu, 28 Mar 2024 11:32:51 +0000

IamFlowZ / multi-region-cdk-example

Companion repository for the Multi Region Deployments blog post

Multi Region Stack Example

This is an example of one way to deploy multiple Cloudformation stacks to different regions, without having forks within the code.

After deployment has completed, you should find function urls listed in your CLI corresponding to the regions you deployed to. Opening them should respond with Hello from ${region}

The cdk.json file tells the CDK Toolkit how to execute your app.

Credentials

By default this assumes you have an AWS profile configured, and will use the default account and region associated with it. The default is the account the credentials belong to and (typically) us-east-1.

If you would like to target a specific account and environment, you can modify bin/multi-region-example.ts to use either of the commented env lines. One is for hard coding the other is for environment variables. You can learn more about CDK environments here.

Useful commands

npm run build compile typescript to…

View on GitHub

From the second you deploy your resources into the AWS Cloud, you have access to a global computing network. Something unimaginable in the past and grossly underutilized even by today’s standards. In this article, I’ll show you two simple ways you can begin leveraging the power of global deployments with CDK.

Prerequisites

If you plan on following along, make sure you have completed these tasks.

Installed the AWS CLI
Installed the CDK CLI
Configured IAM credentials
Bootstrapped at least two regions in an AWS account using the cdk bootstrap command

Root Stack with Substacks

This design implements a “Root” stack that then deploys the subsequent region-specific stacks. The root stack will be deployed into the default region associated with the profile (us-east-1 unless specified). There are several things achieved with this.

We develop a clear hierarchy
- The main benefit of this is dependency management since we create a clear route by which we can pass information both up and back down the tree
We create a point of centralization that we can use to build dependencies that are used by many dependents
- For example, if we have an SQS queue that is used globally within the application, we can build it in the root stack and pass it through to the props of the application stacks
We have a “non-regional” stack to house global resources
- Building on the last point, we have a stack that handles deploying for things that aren’t regional in AWS (IAM, S3, etc.).

The stack itself is tied to a region, but there’s no reason that the resources the stack deploys also have to be tied to that region.

Here is an example of the root stack taken from here in the repository.

import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import { AppStack } from './app-stack/app-stack';

export class MultiRegionExampleStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    const regions = ['us-east-1', 'eu-west-1'];

    // store the regional stacks in an object for debugging
    const regionalStacks = regions.reduce((accu, region) => {
      const regionalStack = new AppStack(this, region, {
        env: { region },
      });
      return { ...accu, [region]: regionalStack };
    }, {});
  }
}

Some caveats:

If you’re familiar with the NestedStack construct, you are probably thinking “What a perfect use case!”. Sadly at the time of writing this, it seems that NestedStacks have to be deployed in the same region that the source stack is deployed in. I don’t believe this is a CDK limitation, but instead a CloudFormation one. There is an open issue with some discussion around implementing this in CDK, however it does seem to be rather complicated. But hey if this is a blocker for you, get active and let them know!
Another Cloudformation feature that is commonly used to solve this problem is Stack Sets. At the time of writing this, there is no support out of the box in CDK. But there is a CDK Labs construct in the experimental phase you can try out. But per their readme, I would avoid getting overly tied to its implementation until it is generally released.
In theory, you shouldn’t need to reference values from an application stack deployed in Region A in an application stack deployed in Region B. As they are siblings, so dependencies should be managed via prop drilling. However reality is often finicky, so if you do end up in this situation, I would recommend storing the dependency value in ParameterStore and dynamically referencing it in the dependent stack. If you’d like to learn more, [I wrote a deep-dive here] that you may find useful.

Deploy the Stacks Directly

For a more traditional IaC experience, we can remove the root stack. Opting to instead deploy the stacks directly. We do lose some of the “superpowers” the first option provides, with the benefit of having something simpler to reason about.

There are two main options I’ve observed

Instantiate your region-specific stack however many times you need in the /bin/app.ts file. You can find a full example here in the CDK docs. As well as this branch in the repository.
Instantiate your region-specific stack once in the /bin/app.ts file, and use the cdk deploy command to deploy it to the regions you desire. This branch has one approach you might take.

While this does supply you with a more traditional experience there are still benefits to using CDK, that you wouldn’t have access to otherwise.

Higher-level constructs
- If like many others you are on a Serverless journey, you may be looking to migrate to Fargate. One of my personal favorites in the CDK library is the ApplicationLoadBalancedFargateService. It’s a mouthful because it does a lot for very little. In ~30 lines (or less) of code, it will deploy an ECS cluster, service, and task definition. As well as provide one-line APIs for wiring in the cluster, load balancer, and any VPC configuration you may need to do. All of this would be hundreds of lines of Cloudformation templating.
Component libraries
- If the standard library can’t meet your needs, constructs.dev is an open-source marketplace that houses thousands of constructs. Ranging from monitoring tools, linters, and everything else you can imagine codifying in the cloud.
- Not only are there public libraries, but you can also create centralization within your organization through private ones. Making it possible to standardize higher-level patterns in the cloud.

Conclusion

With CDK, it’s never been easier to duplicate resources into multiple AWS regions. You can bring your code closer to your customers and increase your fault tolerance with just a few clicks.

Find me on LinkedIn | Github | Mastodon

Three Tips for Writing CDK Tests

Dakota Lewallen — Mon, 18 Mar 2024 12:25:22 +0000

1. Use the assertions module

Something of a no-brainer, but use the tools you’re given! One of the best parts of the CDK library is the assertions module. It contains two sections. One for “fine-grained” assertions, like what you would write during unit testing. The other is for performing “snapshot” tests, in which you compare the new template against one that is already deployed. It’s a section of the library that can be easily missed but provides so much value when used correctly.

2. Make any Nested Stacks easily accessible

One of the benefits of using CDK is the ability to create declarative dependency trees. But, testing the implemented Nested Stacks can be tricky. One of the best ways I’ve found to simplify this is to add Nested Stacks as properties of the Stack class that handles deploying them. For example…

import SubStack from 'substack-stack.ts';

export class TopStack extends Stack {   
  constructor(scope, id, props) {     
    const subStack = new SubStack(this, 'substack', props);   
  } 
}

Is pretty standard. While it is possible to test this, you can simplify the process by “stashing” a reference.

import SubStack from 'substack-stack.ts';

export class TopStack extends Stack {   
  subStack: SubStack;    
  constructor(scope, id, props) {     
    this.subStack = new SubStack(this, 'substack', props);   
  } 
}

Now accessing this for fine-grained assertions can look like…

import { Capture, Match, Template } from "aws-cdk-lib/assertions";
import * as cdk from "aws-cdk-lib"; 
import { TopStack } from "top-stack";  

describe("TopStack", () => {   
  test("has the substack", () => {     
    const app = new cdk.App();      
    const topStack = new TopStack(app, "topStack", {});     
    const { subStack } = topStack; // we pull the substack through the property      

    // Pass the subStack to template.     
    const template = Template.fromStack(subStack);   
  } 
}

3. Write your template to a file (sometimes)

While writing assertions, you can find situations where the tests are failing for reasons that may not make sense. When this happens, one of the first things I will do is write the CloudFormation template out into a file. This way you can get a cold hard copy of what it is you are building in your hand. Rather than trying to parse through complex logs or mysterious error messages. Not a guarantee but often getting to see the complete output can solve many common testing pitfalls.

Conclusion

Testing is great! Testing your infrastructure… Even better! If you’re just getting started with CDK or if you’ve been around since the early days, I hope these tips can help you along your journey! If you have any tips you'd like to add, share them in the comments. Always looking to learn more!

Find me on LinkedIn | Github

Organize Your CDK lib Folder by Function Not Service

Dakota Lewallen — Fri, 08 Mar 2024 14:40:53 +0000

Don’t: Organize Your CDK lib Folder by Service

When working with IaC tools, it's standard to organize your work based on the service. You specify a service and then describe how you would like it configured. Carrying that pattern into CDK might look something like so.

At first, this will feel great! Things are going great until you start to work on functionality across many services. At this point, changes will span many directories resulting in many CloudFormation stacks needing to be deployed.

Do: Organize Your CDK lib Folder by Function

If we follow the principle of working backward, we’ll end up with a structure that more aligns with how we’re going to work in the future.

Starting with our business use case. Let’s say you are going to receive flat files through an SFTP server from clients. You will need to perform some basic ETL processes to house it within your system. Instead of having a lambda folder, a step function folder, and an event bridge folder, you build a single FlatFileHandler folder. With a single stack, that deploys the resources necessary to perform this function.

Now whenever work is needed on the flat file handling system, you can find everything you need in a singular place.

Conclusion

One of the greatest strengths of CDK, is the ability to perform more complex organizational patterns for your resources. This is one example of a pattern that can be useful. Like anything that's in code, you're now free to organize as you see fit! If you find a more useful pattern, let the world know (me in particular 😉)!

Find me on LinkedIn | Github

CDK Dependency Strategies

Dakota Lewallen — Tue, 12 Dec 2023 14:53:29 +0000

Originally posted to dakotalewallen.substack.com

This topic has been in my backlog to write about for a long time. I’ve had to sit with it for so long since I was “missing a piece of the puzzle.” But after attending Re:Invent this year I feel like I’ve got enough to share my thoughts. The topic is also surprisingly nuanced, so I figured laying some groundwork by describing the most common patterns would be a place to start.

Expanding the props object

This is most commonly used as it’s one of the most natural dependency mechanisms in JavaScript. For a given stack or construct, it declares its need for something external. Then at runtime, you have to satisfy that need during the call to the constructor. You define this dependency by expanding the props object which inherits from the StackProps CDK class. There are two variations here.

The first is simpler declaring a dependency for a specific L2 or L3 construct. This is where most applications start, and if you’re building a construct library as far as you need to go. However in the long run I’ve found this method to be taxing on the synth command, leading to circular dependency issues that can obstruct the project’s ability to build successfully.

// myConstruct.ts
export type MyConstructProps = {
  someInstance: ec2.Instance;
}

export default class MyConstruct extends Construct {
  constructor(scope: Construct, id: string, props: MyConstructProps) {
     const { someInstance } = props;
     const securityGroup = ec2.SecurityGroup.fromSecurityGroupId(this,' SG', 'sg-12345', {...});
     someInstance.addSecurityGroup(securityGroup);
  }
}

// somewhereElse.ts

import MyConstruct from './myConstruct';

const someInstance = new ec2.Instance(this, 'targetInstance', {...});

const myConstruct = new MyConstruct(this, 'mine', { someInstance })

The second variation of this dependency style involves building explicit dependency paths between the stacks that deploy resources. This does remove some of the “generic” functionality, meaning that we can’t receive any old EC2 instance anymore, instead, we have to receive one from somewhere specific within the application. Additionally, this will rely on there being at least one stack that orchestrates between the dependent and provider stacks.

// orchestratorStack.ts
import ProviderStack from './providerStack'
import DependentStack from './dependentStack'

export default class Orchestrator extends Stack {
  constructor(scope: Construct, id: string, props: StackProps) {
    const provider = new ProviderStack(this, 'provider');

    const dependent = new DependentStack(this, 'dependent', {provider});
    // This next line is _vitally_ important. As it informs the build system that the provider stack has to be fully built prior to building the dependent. Omitting this line will lead to them being built in parallel which can lead to either circular dependencies or race conditions. Neither of which are fun to deal with.
    // See the docs below for more information (strongly suggest reading at least this section)
    // https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib-readme.html#construct-dependencies
    dependent.node.addDependency(provider);
  }
}

// providerStack.ts

export default class ProviderStack extends Stack {
  someInstance: ec2.instance;

  constructor(scope: Construct, id: string) {
    this.someInstance = new ec2.Instance(this, 'instance', {...});
  }
}

// dependentStack.ts

export type DependentStackProps = StackProps & {
  providerStack: ProviderStack;
}

export default class DependentStack extends Stack {
  constructor(scope: Construct, id: string, props: DependentStackProps) {
    const { providerStack } = props;
    const { someInstance } = providerStack;
    const securityGroup = ec2.SecurityGroup.fromSecurityGroupId(this,          'SG', 'sg-12345', {...});
    someInstance.addSecurityGroup(securityGroup);
  }
}

This does have the drawback of not being as reusable. So for libraries, this approach would be a non-starter. However, working within the boundaries of a contained application can save lots of headaches and potentially even some build time.

Using dynamic references via SecretsManager and ParamaterStore

Going further down the rabbit hole, we move away from passing values around in memory during the synth and build steps, instead opting toward external services. Namely, SecretsManager and ParamterStore.

To remove the need to pass the construct, let’s store the ARN in Parameter Store.

// providerStack.ts

export default class ProviderStack extends Stack {
  constructor(scope: Construct, id: string, props: StackProps) {
    const ourQueue = new sqs.Queue(this, 'queue', {...});
    new ssm.StringParameter(this, 'QueueArn', {
      allowedPattern: '.*',
      description: 'The queue ARN',
      parameterName: 'QueueArnParameter',
      stringValue: ourQueue.queueArn,
      tier: ssm.ParameterTier.ADVANCED,
    });
  }
}

Now using CloudFormation dynamic references, let’s update the dependent code.

// dependentStack.ts

export default class DependentStack extends Stack {
  constructor(scope: Construct, id: string, props: StackProps) {
    const queueArn = new CfnDynamicReference(
      CfnDynamicReferenceService.SSM,
      'ssm:QueueArnParameter',
    );

    const targetQueue = sqs.fromQueueArn(queueArn);

    const ourRole = new iam.Role(...);
    targetQueue.grantReadWrite(ourRole);
  }
}

We have successfully decoupled the two resources within the codebase, by abstracting away the connection via ParameterStore and a dynamic reference. This would hopefully allow us to have more freedom to add and remove dependents without having to worry about maintaining a complex dependency tree within our typing system and instead opting to rely on an external system.

Referencing secrets in Secrets Manager can be done in the same manner as Parameter Store, so rather than rehash that, I’ll leave you with some general advice. You can create your secrets in CDK. But you should never fill your secrets from within your CDK codebase. The docs for the Secret construct mention this outright. So just like with anything else, you should not be checking secret anything into your code base. Ever. Build it however, fill it manually and only load the value when you need it.

// dependentStack.ts

export default class DependentStack extends Stack {
  constructor(scope: Construct, id: string, props: StackProps) {
    const apiSecret = new CfnDynamicReference(
      CfnDynamicReferenceService.SECRETS_MANAGER,
      'ApiToken:SecretString',
    );
    const lambda = lambda.NodeJsFuntion(this, 'myFunc', {
      ...
      environment: {
        API_TOKEN: apiSecret,
      },
    }); 
  }
}

Conclusion

Likely over the life of your application, you’ll find your needs evolving from using the simpler methods at the beginning to the more complex ones as you progress. I am always an advocate for KISS (keep it stupid simple) for as long as possible. Only introducing complexity once it’s needed. So don’t feel you need to migrate everything as soon as possible. If what you have has been satisfactory up until now, it will likely be just as satisfactory tomorrow. Wait until you start to notice hot spots before starting to move things up in terms of complexity.

Find me on LinkedIn | Github

Split Horizon DNS in AWS Made Easy!

Dakota Lewallen — Mon, 30 Jan 2023 12:13:19 +0000

Originally published to dakotalewallen.substack.com

A few months ago, I deployed my first private hosted zone to Route 53. Unbeknownst to me, I had set up a split horizon DNS scenario. My public zone was routing calls to my load balancer correctly. But every time my resources deployed in the VPC called into the same domain, it would result in errors like...
getaddrinfo ENOTFOUND _domain_...
I spun and thrashed and tried everything I could think of. Then after hours of debugging, while doing some "pair debugging" they made the suggestion that we duplicate the A record in the public zone into the private zone. Lo and behold problem is solved. So, if you have resources deployed to a VPC and a private zone attached to said VPC, and a public zone sharing the domain name. When your resources are all in the domain, it will use the private zone and the private zone only for resolution.
I spent some time learning and digging deeper into Route53 and DNS in general, to better understand the situation and why the solution was what it was. I found this article in the Route53 docs that explained almost the exact situation I was in. Considering how much I enjoy working with the CDK, and how much I struggled with this DNS configuration. I figured what better way to improve my understanding than to build a solution, that hopefully will help others avoid the headache I went through.

Introducing...

The aws-cdk-split-horizon-dns CDK construct!

import { SplitHorizonDns, AliasTarget } from 'aws-cdk-split-horizon-dns'

const firstTarget: AliasTarget = {
  target: ['8.8.8.8'],
  private: true,
};

const bucketWebsite = new s3.Bucket(stack, 'BucketWebsite', {
  bucketName: exampleDomain, // www.example.com
  publicReadAccess: true,
  websiteIndexDocument: 'index.html',
});

const constructTarget: AliasTarget = {
  target: new targets.BucketWebsiteTarget(bucketWebsite),
  public: true,
};

new SplitHorizonDns(scope, 'MyDNS', {
  zoneName: 'example.com',
  privateZoneVpcs: [vpc],
  targets: [constructTarget, firstTarget],
})

Pass it your domain name, any vpc's you want to be attached to the private zone, and a list of either resources or IPs you would like A records for, and boom. Pack it up and send it. For the resources, you can flag them as public, private, or both. And the rest is taken care of.
This project is still early stages. So if you try it and have success. Let me know! Or if this manages to bring the whole house down, please also let me know! Future goals will depend pretty highly on what's demanded of the construct. I would imagine there will be some want for being able to specify the record type that gets created. As with this initial release it is limited to A records. Additionally, a point of configuration that will likely be added soon is the ability to set the TTL of the record.
As of now, support for TypeScript is being tested. Python is included, but untested. If you want your stack included in the project, feel free to open an issue or PR here in the repository.

Hope this helps!

Find me on Mastodon | LinkedIn | Github | Substack

TIL: How to CDK Diff Your Entire App

Dakota Lewallen — Fri, 13 Jan 2023 20:58:51 +0000

If you use the AWS CDK to deploy resources, you may also likely use the Pipelines construct to deploy. Let's say you do, and you run a cdk diff against your app prior to committing. It's awfully disappointing as it will only show you changes to the encasing pipeline.
However thanks to this fantastic answer from @fedonev on StackOverflow, I learned that there is special syntax within the CDK CLI that allows you to wildcard the stack you would like a command to run against.

You may also use wildcards to specify IDs that match a pattern.
? matches any single character
* matches any number of characters (* alone matches all stacks)
** matches everything in a hierarchy

So running

  cdk diff '**'

Will diff not only diff the pipeline stack, but all the stack's within your application.

Find me on Mastodon | LinkedIn | Github | Substack