Using Serverless Framework and CDK together

While helping folks work through our Serverless Stack guide, we noticed that most folks have a really hard time using CloudFormation templates to define their AWS infrastructure. And so it's no surprise that we are really excited about AWS CDK! It allows you to define any AWS infrastructure using modern programming languages. Making CDK, truly infrastructure as code.

Serverless Framework on the other hand is great for building Serverless applications on AWS. While you can build Serverless apps purely with CDK, it doesn't have a great developer experience. On the flip side, Serverless Framework allows you to deploy any non-Serverless AWS infrastructure. But it requires you to use CloudFormation templates written in YAML or JSON.

The overlap between these two frameworks have led folks to write about why you should use one versus the other. However, we think they are meant to be used together, in tandem. And we've created a tool to do so — Serverless Stack Toolkit (SST). Allowing you to get the best of both worlds!

Let's look at this in detail.

Background

Before we dive in, let's quickly get some background on these frameworks.

What is Serverless Framework

Serverless Framework was released back in October 2015. It allows you to develop Serverless applications locally and deploy them with ease. It's driven by a serverless.yml config file that looks something like this:

service: serverless-example

provider:
  name: aws
  runtime: nodejs12.x

functions:
  hello:
    handler: handler.hello

You can invoke your Lambda functions locally using:

$ serverless invoke local -f hello

And deploy them by:

$ serverless deploy

Serverless Framework will automatically package your Lambda functions and upload them to AWS.

You can also add other AWS infrastructure using CloudFormation templates in YAML. For example, you can add a DynamoDB table using:

Resources:
  NotesTable:
    Type: AWS::DynamoDB::Table
    Properties:
      TableName: ${self:custom.tableName}
      AttributeDefinitions:
        - AttributeName: userId
          AttributeType: S
        - AttributeName: noteId
          AttributeType: S
      KeySchema:
        - AttributeName: userId
          KeyType: HASH
        - AttributeName: noteId
          KeyType: RANGE
      BillingMode: PAY_PER_REQUEST

What is AWS CDK

AWS CDK (Cloud Development Kit) was released in Developer Preview back in August 2018. It allows you to use TypeScript, JavaScript, Java, .NET, and Python to create AWS infrastructure.

So for example, here is how a CloudFormation template for a DynamoDB table would compare to the CDK version:

- Resources:
-   NotesTable:
-     Type: AWS::DynamoDB::Table
-     Properties:
-       TableName: ${self:custom.tableName}
-       AttributeDefinitions:
-         - AttributeName: userId
-           AttributeType: S
-         - AttributeName: noteId
-           AttributeType: S
-       KeySchema:
-         - AttributeName: userId
-           KeyType: HASH
-         - AttributeName: noteId
-           KeyType: RANGE
-       BillingMode: PAY_PER_REQUEST


+ const table = new dynamodb.Table(this, "notes", {
+   partitionKey: {
+     name: 'userId', type: dynamodb.AttributeType.STRING,
+   },
+   sortKey: {
+     name: 'noteId', type: dynamodb.AttributeType.STRING,
+   },
+   billingMode: dynamodb.BillingMode.PAY_PER_REQUEST,
+ });

It also allows you to create higher order constructs. This means that you can combine and compose lower level resources to create new reusable constructs.

Similar to Serverless Framework, you can deploy a CDK app using:

$ cdk deploy

It also allows you to define Lambda functions as well.

const helloSrc = fs.readFileSync('handler.js').toString();

const helloLambda = new lambda.Function(this, 'HelloLambda', {
  runtime: lambda.Runtime.NODEJS_12_X,
  handler: 'handler.hello',
  code: new lambda.InlineCode(helloSrc, {
    encoding: 'utf-8',
  })
});

However, it doesn't offer a great local development experience for your Lambda functions.

Using Serverless Framework and CDK together

To recap, here's how Serverless Framework and CDK compare.

	Serverless Framework	AWS CDK
Perfect for Lambda functions	✅	〰️
Perfect for other infrastructure	〰️	✅

So if you are building a Serverless application that includes other AWS infrastructure, it would make a ton of sense to use Serverless Framework for your Lambda functions and CDK for the rest.

As an aside, a similar design pattern of splitting up your Lambda functions from the other infrastructure is common in the Serverless community and we recommend it over on Serverless Stack as well.

However, there are a couple of design differences between the two that you need to be wary of.

Design differences between Serverless Framework and CDK

Serverless apps are made up of multiple services and the app as a whole is deployed to the same environment.

Each service is deployed as a CloudFormation stack to the target AWS account. You can specify a stage, region, and AWS profile to customize this.

 $ AWS_PROFILE=development serverless deploy --stage dev --region us-east-1

The --stage option here prefixes your stack names with the stage name. So if you are deploying multiple stages to the same AWS account, the resource names will not thrash.

This allows you to easily deploy your Serverless app to multiple environments. Even if they are in the same AWS account.

AWS CDK apps on the other hand are made up of multiple stacks. And each stack is deployed to the target AWS account as a CloudFormation stack. However, unlike Serverless apps, each stack can be deployed to a different AWS account or region.

This means that each time you deploy your CDK app, it could potentially create a stack in multiple environments. This critical design difference prevents us from using CDK apps alongside our Serverless services.

You can fix this issue by following a certain convention in your CDK app, as outlined in this post — Deploy aws-cdk app to different environments. However, this is only effective if these conventions are enforced.

Enter, Serverless Stack Toolkit

To fix this, we created the Serverless Stack Toolkit (SST).

SST allows you to follow the same conventions as Serverless Framework. This means that you can deploy your Lambda functions using:

$ AWS_PROFILE=production serverless deploy --stage prod --region us-east-1

And use CDK for the rest of your AWS infrastructure:

$ AWS_PROFILE=production npx sst deploy --stage prod --region us-east-1

Just like Serverless Framework, the stacks in your CDK app are prefixed with the stage name. Now you can use Serverless Framework and CDK together! Allowing you to do something like this:

Note that, if you define the AWS account or region as a part of your CDK stack, like so:

new MyStack(app, "my-stack", { env: { account: "1234", region: "us-east-1" } });

You'll get an error.

Error: Do not directly set the environment for a stack

And there's more!

SST also has a couple of other key advantages.

• Concurrent deployments

  AWS CDK deployments are currently very slow. CDK deploys your CloudFormation stacks in sequence. It'll submit a CloudFormation template for deployment and wait till it completes, before starting the next one. This means that CDK deployments for large apps can easily take at least half an hour. SST fixes this by deploying your CloudFormation stacks concurrently.

• Asynchronous deployments

  SST also supports deploying your CloudFormation stacks asynchronously. So you don't have to waste CI build minutes waiting for CloudFormation to complete. Seed natively supports concurrent asynchronous deployments for your SST apps. Making it nearly 5 times faster and virtually free to deploy!

• Supports ES6 (and TypeScript) out-of-the-box

• Automatically lints your CDK code using ESLint

Differences between SST and CDK

SST is a simple extension of CDK with a couple of minor differences.

1. There is no cdk.json

   Instead there is a sst.json with the defaults for your stage and region.
   

   {
     "name": "my-sst-app",
     "type": "@serverless-stack/resources",
     "stage": "dev",
     "region": "us-east-1"
   }
   

2. There is no bin/*.js

   Instead there is a lib/index.js that has a default export function where you can add your stacks. Note that, SST creates the App object for you.
   

   import MyStack from "./MyStack";
   
   export default function main(app) {
     new MyStack(app, "my-stack");
   
     // Add more stacks
   }
   

3. Stacks extend sst.Stack

   Your stack classes extend sst.Stack instead of cdk.Stack. This allows SST to prefix your stack names with the stage you are deploying to.
   

   import * as sst from "@serverless-stack/resources";
   
   export default class MyStack extends sst.Stack {
     constructor(scope, id, props) {}
   }
   

You can read more about moving a CDK app to SST here. So if you'd like to use SST but have an existing CDK app, you can do so by following a couple of easy steps.

Summary

We think AWS CDK is a vast improvement over CloudFormation templates that are written in YAML or JSON. And the combination of Serverless Framework and CDK allows you to leverage the best of both worlds. Develop and deploy Lambda functions using Serverless Framework and define the rest of your AWS infrastructure with CDK. To make sure that you can use the two together, we created the Serverless Stack Toolkit (SST). A simple extension of CDK that allows you to deploy CDK apps alongside your Serverless Framework services!

Comments

[Simon Bracegirdle]

Could you please elaborate on this?

While you can build Serverless apps purely with CDK, it doesn't have a great developer experience.

What problems do you face with serverless on native CDK that are solved by this toolkit? I'm struggling to understand what the value add is.

And speeds up your deployments by deploying all your stacks concurrently!

I'm assuming this is for manual deploys via cdk deploy? This can be avoided if you're using CodePipeline, especially via the new CDK pipelines module.

[Frank Wang]

Hi Simon,

What problems do you face with serverless on native CDK that are solved by this toolkit?

Serverless Framework has a great local developer experience for Lambda and API Gateway. It also has a great plugin ecosystem that covers most use cases. It makes it much easier to test and deploy your Serverless apps. CDK on the other hand only does the packaging and deployment. You can use it for one off Lambdas but I would not recommend building entire Serverless applications with it.

I'm assuming this is for manual deploys via cdk deploy?

You are right. But for folks that are using CI services like Circle, Travis and GH Actions, SST allows them to deploy concurrently without having to complicate their pipelines.

[Greg Clark]

Excellent article describing the differences and how your tool works to get the best of both worlds. I can't wait to try it.