AWS CDK and DynamoDB: This One Configuration Line Is Costing You Hundreds of Dollars

Elias Tazartes — Tue, 30 Aug 2022 14:16:36 +0000

Hey there, serverless aficionadas and aficionados 😶‍🌫️! Have you ever made an unfortunate mistake 🙊 that cost your team hundreds of dollars? More than that? No way. Thousands!? First, I'm sorry you went through this. Second, I know how you feel.

TL;DR

When using the AWS Cloud Development Kit to provision a DynamoDB table, CDK sets by default the billing mode as "Provisioned" instead of pay-per-request. Therefore, Dynamodb essentially ceases to be truly serverless - remember scale-to-zero?

Quick fix: when instantiating your AWS Dynamodb table, make sure to explicitly set the billing mode to "pay-per-request". If you're playing around with a toy stack, you can also set the removal policy to "destroy". This means that if you remove your stack, the table will be deleted as a result (probably a bad idea to use this setting in production).

// code/dynamodb-cdk.ts

import { App, RemovalPolicy, Stack } from '@aws-cdk/core';
import { AttributeType, Table, BillingMode } from '@aws-cdk/aws-dynamodb';

const app = new App();
const stack = new Stack(app);

const table = new Table(stack, 'MyTable', {
  partitionKey: { name: 'PK', type: AttributeType.STRING },
  sortKey: { name: 'SK', type: AttributeType.STRING },
  billingMode: BillingMode.PAY_PER_REQUEST,
  removalPolicy: RemovalPolicy.DESTROY,
});

Getting acquainted with serverless as a software engineer

One of my first missions as a junior dev was to help design a "serverless 101 course". We aimed to help the future rookies in my team feel familiar with cloud-native concepts quickly. We created a concise tutorial that goes through the basics of serverless: lambdas, API gateway, IAM roles and dynamodb tables - with AWS as our cloud provider. I was pretty happy about the result! Newcomers could finally play in a sandbox environment with complex - and scary at first - concepts such as functions-as-a-service (FaaS) and cloud-native managed databases!

I felt proud. We had a small protocol where every new developer would deploy a toy stack. It served its purpose well and serverless newbies liked it.

I used to take serverless tech's scale-to-zero property for granted 🥺

Surely, if you read the TL;DR and the small background story, you're starting to see where I'm going with this whole rant.

Each of our team's new engineer deployed a toy stack to learn about serverless and AWS. The serverless tutorial uses AWS CDK. To keep it light, we mostly went with default settings. Amongst these settings hid the "provisioned" billing mode and the "retain" removal policy.

To simplify, with the DynamoDB "provisioned" billing mode, AWS provisions a floor storage space, and minimum Read capacity units (RCU) & Write capacity units (WCU). Then, AWS charges you for it, regardless of your usage.

For DynamoDB, I found that the "provision" billing mode costs about $11/month per table (in US regions) even if you're not using the table at all. Moreover, when a stack is removed, the "retain" removal policy prevents the DynamoDB table from being deleted with the stack.

Each rookie deploys a sandbox stack. Storage resources are provisioned, i.e. the table costs a floor amount even if we don't use it. Tables are prevented from being deleted. Yep, a potentially bottomless pit was created.

Wrapping it up: check your serverless expenses each month and don't take serverless' low prices for granted.

Serverless is inexpensive compared to non-serverless tech stacks. An order of magnitude cheaper. My manager seemed a little surprised when the stacks deployed as part of our serverless tutorial were costing us hundreds each month!

Thankfully ✨, my teammates caught the error after a couple of months, thus limiting my oopsie from wasting thousands of dollars (and lots of electricity 💚).

The ultimate Serverless DevX: Serverless Framework, AWS CDK and Typescript

Elias Tazartes — Wed, 24 Aug 2022 12:36:34 +0000

Serverless Framework and AWS CDK are two important players in the serverless development ecosystem. At first glance, you would think they are competitors, but I've been combining Serverless Framework, AWS CDK and Typescript for half a year to develop cloud-native applications. In my experience, it's mostly been a breeze - mostly 🥺.

TL;DR

Combining Serverless Framework and the AWS Cloud Development Kit forms one of the best stacks to reliably and quickly build robust cloud-native applications.

To achieve that, I recommend using the @swarmion/serverless-cdk-plugin plugin to seamlessly integrate your AWS CDK resources in Serverless Framework.

Most compelling pros:

🏗 Leverage the no. 1 framework's (Serverless framework) awesome community and developer experience to create and deploy lambdas,
🍭 Easily create other AWS resources thanks to the AWS CDK's higher levels abstractions,
👷 Build a state-of-the-art serverless app by writing your application's logic and configuration in the same repository with Typescript (strongly typed infrastructure as code 🧞‍♂️!).

How does it work?

Build your cloud-native application using Serverless Framework.
Create non-lambda AWS resources (DynamoDB tables, SQS queues, EventBridge's event buses, etc.) using the aws-cdk-lib (the aws-cdk v2).
Leverage Serverless Framework's powerful plugin ecosystem and install the @swarmion/serverless-cdk-plugin. Check out our github open source repository!
Reference your aws-cdk construct in your Serverless framework configuration file.
That's it! Serverless Framework seamlessly provisions these resources on deployment.

// code/serverless-config.ts

// serverless.ts

import { AWS } from '@serverless/typescript';

import type { ServerlessCdkPluginConfig } from '@swarmion/serverless-cdk-plugin';

import { MyCdkConstruct } from 'resources/dynamodb';

const serverlessConfiguration: AWS & ServerlessCdkPluginConfig = {
  service: `my-service`,
  frameworkVersion: '*',
  provider: {
    name: 'aws',
    runtime: 'nodejs14.x',
  },

  // Import the plugin in your serverless configuration.
  plugins: ['@swarmion/serverless-cdk-plugin'],

  // Reference your custom aws-cdk construct at the "construct" key. That's it!
  construct: MyCdkConstruct,
  // ...More configuration props
};

module.exports = serverlessConfiguration;

Building high quality Serverless apps

Writing your application's logic and infrastructure as code in the same repository and language feels really good. Nevertheless, have you ever tried to provision a DynamoDB table, a SQS queue or an EventBus with Serverless Framework? Being lambda-centric by design, this framework does not provision other AWS resources out-of-the-box for you.

From there, you have two options:

Handling the deployment of your other resources separately, for instance using the AWS CDK. Sebastian Bille wrote a cool article about this subject. The main consequence of this approach is having two separate stacks that deploy independently. Nevertheless, one of the deployments might fail, for instance, due to circular dependencies between the two stacks - e.g. SQS => Lambda => SQS flow.
Have Serverless Framework provision your resources as part of its deployment cycle. The trade-off is having to write vanilla Cloud Formation. If you've never done it before, believe me, it's a DevX faux-pas 🙅. Thankfully, Fred Barthelet wrote an awesome article on how to replace vanilla Cloud Formation by AWS CDK constructs.

Essentially, aws-cdk stack classes have a transpilation method that allows the dev-friendly Typescript code you wrote to be transpiled to Cloud formation. The downside to this process is the necessity to write a lot of boilerplate code every time you need a new resource (SQS queue, etc.).

These two solutions solve many interesting use-cases that developers face when building cloud-native apps:

What if one could build an awesome Serverless application, use the AWS CDK for specific resources while having only one deployment process and no boilerplate?

Enter the @swarmion/serverless-cdk-plugin: a bridge between Serverless Framework and the AWS CDK.

At Kumo, we have worked on developing an awesome tool to achieve just that: a complete and stable typescript stack to build serverless applications. Did you know? Our plugin integrates seamlessly with Swarmion, a framework for Serverless Typescript microservices.

How does the plugin work?

Upon deploying (or packaging) your Serverless application (e.g. sls deploy), the @swarmion/serverless-cdk-plugin converts your aws-cdk construct code to Cloud Formation and integrates it in the Serverless deployment cycle under the hood. This is essentially some kind of transpilation (typescript => cloud formation). Then, Serverless Framework provisions it on deploy.

🚨 Caveats: Using AWS CDK with Serverless Framework does not allow you to deploy resources that require a so-called bootstrap stack.

For instance, using the AWS CDK to create a lambda function. There, the bootstrap stack consists in provisioning an S3 bucket with the lambda's code. This S3 bucket would then be referenced and used at a later stage of the deployment cycle to provision the lambda function.

Trying to create constructs that require a boostrap stack will thus result in a strange error at build time. The plugin helps in that it throws a very explicit error. The only way to deploy those type of constructs is to use a separate deployment process.

How can I use the plugin in my Serverless Framework app?

🎙 Disclaimer: This plugin supposes you are using Typescript as your Serverless app's main language. The bridge made between AWS CDK and Serverless Framework supposes users have a serverless.ts config file (sorry to all you yamlers out there 💅).

Import and use the @swarmion/serverless-cdk-plugin in your service's serverless.ts configuration file (as shown in the above code snippet).

Write aws-cdk constructs - if you want to create more than one construct, use the AWS CDK composition to create a higher level construct - reference it in the serverless.ts file, and ta-da 🥳 🤩! Upon running your deploy script, the Serverless Framework will recognize that you want to provision these resources and deploy them.

For instance, if you want to provision a DynamoDB table:

// code/serverless-construct-testcase.ts

import { AttributeType, BillingMode, Table } from 'aws-cdk-lib/aws-dynamodb';
import { Construct } from 'constructs';

export class MyCdkConstruct extends Construct {
  public dynamodbArn: string;
  public dynamodbName: string;

  constructor(scope: Construct, id: string) {
    super(scope, id, props);

    const { tableArn, tableName } = new Table(this, 'MyDynamoDBTable', {
      partitionKey: { name: 'PK', type: AttributeType.STRING },
      sortKey: { name: 'SK', type: AttributeType.STRING },
      billingMode: BillingMode.PAY_PER_REQUEST,
    });

    this.dynamodbArn = tableArn;
    this.dynamodbName = tableName;
  }
}

What you're getting:

An all-in-one reliable stack to build serverless application!
Helper functions to reference your resources inside your lambdas shipped with the plugin! For instance, your lambda functions can now access attributes of your DynamoDB in a type-safe manner (ARN, table name, and more)!
Moreover, the plugin exposes the Serverless context inside your construct! This allows for a better DevX: one no longer needs to hardcode ARNs or resources information coming from Serverless' context inside their constructs' code. When using the serverless <> cdk plugin, you can now type your construct as a ServerlessCdkPlugin.ServerlessConstruct. This will result in your construct having access to the Serverless Framework context and configuration.

// code/serverless-construct-testcase-with-serverless.ts

// MyCdkConstruct.ts

import { AttributeType, BillingMode, Table } from 'aws-cdk-lib/aws-dynamodb';
import { Construct } from 'constructs';

import { ServerlessConstruct, ServerlessProps } from 'types';

export class MyCdkConstruct extends ServerlessConstruct {
  public dynamodbArn: string;
  public dynamodbName: string;
  public testServerlessConfigValue: string | undefined;

  constructor(scope: Construct, id: string, props: ServerlessProps) {
    super(scope, id, props);

    const { tableArn, tableName } = new Table(this, 'MyDynamoDBTable', {
      partitionKey: { name: 'PK', type: AttributeType.STRING },
      sortKey: { name: 'SK', type: AttributeType.STRING },
      billingMode: BillingMode.PAY_PER_REQUEST,
    });

    this.dynamodbArn = tableArn;
    this.dynamodbName = tableName;

    /*
      Serverless context variables is now accessible inside the CDK code!
      There are numerous other use-cases, e.g. accessing your lambda's names.
     */
    this.testServerlessConfigValue = props.serverless.resources.Outputs?.TestOutput.Description;
  }
}

Hey 👷, thank you so much for reading this article. Have a good one!

DEV Community: Elias Tazartes

AWS CDK and DynamoDB: This One Configuration Line Is Costing You Hundreds of Dollars

TL;DR

Getting acquainted with serverless as a software engineer

I used to take serverless tech's scale-to-zero property for granted 🥺

Wrapping it up: check your serverless expenses each month and don't take serverless' low prices for granted.

The ultimate Serverless DevX: Serverless Framework, AWS CDK and Typescript

TL;DR

Building high quality Serverless apps

Enter the @swarmion/serverless-cdk-plugin: a bridge between Serverless Framework and the AWS CDK.

How does the plugin work?

How can I use the plugin in my Serverless Framework app?

What you're getting: