loading...
Cover image for CDK Patterns at 20! Let's Walk Through all 20 Serverless Patterns for AWS

CDK Patterns at 20! Let's Walk Through all 20 Serverless Patterns for AWS

nideveloper profile image Matt Coulter ・12 min read

There are now 20 fully deployable serverless architecture patterns at cdkpatterns.com, all built with AWS CDK TypeScript/Python and all including the vanilla CloudFormation Template.

This seemed like a good time to stop and reflect on what has been built showcasing the use cases for each pattern in one article. I will simply walk through them alphabetically but if you want to view them by AWS Component used you can use the component filter or if you want to view them by the AWS Well Architected pillar they relate to you can use the well architected pattern matcher

For the patterns that exist in Jeremy Dalys new Serverless Reference Architecture collection I will also include a link to there for users of other frameworks than AWS CDK (SAM, Serverless, Pulumi etc).

 Polly

polly arch

cdk patterns link

This is a pattern that integrates the Amazon Polly service into an AWS Lambda Function so that you can synthesise text into speech using a serverless stack. It also integrates with Amazon Translate to allow you to choose the language for the spoken text.

When To Use

If you are publishing content you could use this to support international readers/listeners in their native language

Alternatives

I am sure external vendors exist, this is the only serverless AWS native solution I know.


S3 Deploy

s3 deploy arch

This is a simple pattern for taking a website built in Angular or React and deploying it to an S3 bucket for a super cheap and scalable hosting solution. The construct bundled with the pattern is flexible and allows you to expand on the bundled pattern by adding CloudFront, a custom domain name and SSL.

Deconstructing S3 Deploy

When To Use

You need to deploy a website on AWS through CDK

Alternatives

AWS Amplify


The Big Fan

the big fan architecture

cdk patterns link

This is a pattern from Heitor Lessa that allows you to fan out your messages to multiple different consumers. The beauty is that you are filtering the messages based on properties in the message so consumers only receive messages relevant to them saving compute time / cost.

Heitor Lessa at re:Invent

re:Invent slides

When To Use

If you are building an asynchronous event driven flow that needs to fan out to multiple consumers based on different message types

 Alternatives

Amazon EventBridge - look at The EventBridge ATM


The CloudWatch Dashboard

cloudwatch dashboard arch

cdk patterns link

When you start building serverless applications the big piece that feels missing compared to a traditional application is observability. You have all these small pieces running asynchronously and when something goes wrong it can feel like logs are your only friend. This pattern shows you have to generate the metrics needed to build a custom dashboard and add Alerts publishing to SNS for a serverless stack containing API Gateway HTTP API, Lambda and DynamoDB

When To Use

Anyone building production serverless applications should be using CloudWatch dashboards and alerts to understand their application state

 Alternatives

You can use other third party tools but this is the only AWS option


The Destined Lambda

destined lambda arch

cdk patterns link

This pattern combines Lambda Destinations with Amazon EventBridge to show you that with EventBridge rules you can decouple your components in an event driven architecture and by combining it with lambda destinations you can strip out EventBridge specific code from your lambda functions themselves and decouple further.

Deconstructing The Destined Lambda

When To Use

Lambda destinations are a brilliant way of removing boilerplate from your orchestration logic but they only work when executed asynchronously so if your flow is synchronous then this is not for you yet.

They also forward the full context of an event which means that when you send an error to a Dead Letter Queue through destinations you get everything including the event that caused the error so you have everything you need to implement automatic retry or error hospital functionality.

 Alternatives

If you don't want to use destinations, you can still use Amazon EventBridge just with more logic in your Lambda Function or you could use AWS Step Functions.


The Dynamo Streamer

dynamo streamer arch

cdk patterns link

This is a variation on The Simple Webservice pattern that has been created by Eric Johnson only instead of a lambda being connected to the apigateway, the dynamodb is connected directly and the api gateway uses templates to transform the incoming message to insert the data. A lambda then listens for events coming from dynamodb streams and can be used to do data transforms after insertion meaning if an error occurs you lose no data.

"When thinking about #Serverless architectures, consider how much of your processing can happen AFTER the data is saved. Thinking asynchronously can lead to greater resiliency and often, less code."

When To Use

I would use a direct API Gateway integration with DynamoDB as much as you can. Unfortunately you cannot use VTL yet with the new HTTP API so you need to use the more expensive REST API but removing a Lambda Function from the flow reduces latency and execution costs. If cold starts are an issue for your use case this eliminates them from the flow.

 Alternatives

The simple webservice pattern


The EFS Lambda

efs lambda arch

cdk patterns link

This is a pattern from Danilo Poccia that attaches an EFS file system to your lambda function to give it expandable, persistent storage. Having this level of storage in a Lambda Function opens the door to many new possibilities (multiple functions can even use the same file system).

When To Use

If you need to share state between multiple Lambda Functions or if you need the Lambda to be able to access something bigger than the max in memory allows

 Alternatives

Convert from using a Lambda Function to using an AWS Fargate container of you could go all the way to EC2


The EventBridge ATM

eventbridge atm arch

cdk patterns link

This is an example CDK stack to deploy the code from this blogpost by James Beswick

In this example, a banking application for automated teller machine (ATM) produces events about transactions. It sends the events to EventBridge, which then uses rules defined by the application to route accordingly. There are three downstream services consuming a subset of these events.

When To Use

If you need to build an asynchronous flow where the contents of the message dictate which consumer(s) should receive the message

 Alternatives

You could use The Big Fan pattern with SNS


The EventBridge Circuit Breaker

the eventbridge circuit breaker arch

cdk patterns link

This is an example CDK stack to deploy my own interpretation of The Circuit Breaker stack described by Jeremy Daly in this article and by Martin Fowler all the way back in 2014 here

In this example, we have a lambda behind an API gateway that is supposed to integrate with an external webservice (www.google.com). The problem is that Google is down and it takes 10 seconds for your lambda to return that error. You pay for every ms of execution with Lambda so this is bad if lots of consumers hit your service.

Don't worry, we have integrated a circuit breaker into this system. When a call to google fails an error event is pushed to EventBridge where it is routed to a lambda that inserts a record into DynamoDB with a 60 second lifespan.

When a consumer calls our lambda we check if there have been 3 failure events in the last 60 seconds and if so we fail immediately, this saves over 9 seconds of execution costs. As the error events expire after 60 seconds our failure events should gradually drop below 3 where we call the service again and check status.

When To Use

If you integrate with external webservices you should probably integrate a circuit breaker so that your system is more resilient in the event of that service going down

 Alternatives

AWS ElastiCache could be used to cache successful calls/responses but this won't help if you haven't called before and the service is down.


The EventBridge ETL

EventBridge ETL Arch

cdk patterns link

This is an example stack showing how you can use EventBridge to orchestrate events through an ETL process. This pattern was insired by Vyas Sarangapani and Hervé Nivon. The CDK Patterns README details their implementations.

Note - This is a learning pattern, if I was implementing this in a production system I would make a couple of changes:

  • Use batching of events where possible to reduce costs
  • KMS Encryption of sensitive data in events
  • SQS between EventBridge and the Lambdas for resiliency
  • Add in error events to EventBridge and error event rules

Deconstructing The EventBridge ETL

When To Use

This pattern is more a demonstration of what is possible with EventBridge in terms of building complex decoupled flows. I would use this to learn EventBridge.

 Alternatives

AWS Glue for complex flows or you could use a Lambda Function streaming larger files from S3.


The Lambda Power Tuner

Lambda Power Tuner Arch

cdk patterns link

This is an AWS CDK project that deploys the awesome AWS Lambda Power Tuning project by Alex Casalboni.

AWS Lambda Power Tuning is an AWS Step Functions state machine that helps you optimize your Lambda functions in a data-driven way.

When To Use

If you are using Lambda Functions in your application you will want to use the Lambda Power Tuning step function to optimize performance vs cost.

 Alternatives

You could tune it by hand with enough effort


The Lambda Trilogy

Lambda Trilogy Arch

cdk patterns link

This is three patterns in one representing the three states of AWS Lambda.

The three states of AWS Lambda are something that has been discussed by many serverless heroes since their invention. This is probably the most controversial subject in all of serverless so I am not going to tell you which of the three is the best because like everything you need to adapt the right implementation to fit your context!

This pattern includes examples from Paul Swail, Yan Cui, Jeremy Daly and others

Deconstructing The Lambda Trilogy

When To Use

If you are using Lambda Functions in your application you will want to use one of the three states


The RDS Proxy

rds proxy arch

cdk patterns link

This is a project that has been configured with a MySQL RDS DB, an RDS Proxy, a Lambda Function to run queries and an API Gateway HTTP API to trigger the lambda function. The RDS Proxy is designed to bridge the gap between the massively scalable, serverless Lambda function and the less scalable RDS Instance

When To Use

If you need to have a synchronous flow where a serverless component communicates with a non-serverless RDS Instance

 Alternatives

If you can go asynchronous you can use the scalable webhook


The SAGA Step Function

saga step function arch

cdk patterns link

This is a pattern that demonstrates how use handle distributed transactions within a serverless system. It also demonstrates single table DynamoDB design inspired by Alex Debrie

Deconstructing The Saga Step Function

When To Use

If you need to make multiple smaller transactions pass or fail in one atomic unit like in the example where you are booking a holiday and need flights and hotel booked together.

 Alternatives

This is a SAGA design pattern implemented with Step Functions, you could try implementing it with other components but it will probably be less efficient.


The Scalable Webhook

scalable webhook arch

This is a pattern from Jeremy Daly for integrating serverless resources like Lambda with non serverless ones like RDS in an asynchronous flow.

Deconstructing The Scalable Webhook

When To Use

If you cannot predict your volume of traffic and you can use an asynchronous process to process the events

 Alternatives

The RDS Proxy but the scalable webhook can be applied to any resource, not just RDS.


The Simple GraphQL Service

simple graphql arch

cdk patterns link

This is a simple MVP implementation of AWS AppSync to use GraphQL to query a Lambda resolver and a DynamoDB table.

When To Use

GraphQL allows your consumer to specify what pieces of information they need from your API which is really powerful. It also is helpful as an extra unifying wrapper on top of disparate APIs

 Alternatives

You could deploy Apollo in a Lambda Function


 The Simple Webservice

simple webservice arch

The most basic pattern on cdkpatterns, the start of most peoples serverless journey. This is an API Gateway HTTP API, connected to a Lambda Function that queries DynamoDB

When To Use

If you need to expose your DynamoDB table (or other serverless component) with a RESTful endpoint.

 Alternatives

The Simple GraphQL service and The Dynamo Streamer


The State Machine

state machine arch

cdk patterns link

This is a simple State Machine implementation using Step Functions to implement the pattern of the same name described by Jeremy Daly

When To Use

If you have complex orchestration logic between multiple Lambda Functions

 Alternatives

It is possible to also build event driven flows with EventBridge and Rules. You get a lot for free with Step Functions though like complete insights into every step executed


 The XRay Tracer

XRay Tracer Arch

cdk patterns link

Learn about using AWS X-Ray for tracing events through your system. This pattern has X-Ray enabled on API Gateway, Lambda, DynamoDB, External HTTP calls, SNS and SQS

When To Use

If you use serverless components in your system you should be using a tracing system to understand its behaviour

 Alternatives

You can instrument some third party alternatives like Thundra


Posted on by:

nideveloper profile

Matt Coulter

@nideveloper

Software Architect working on enabling engineers to rapidly deliver serverless-first solutions in a Fortune 100 organisation - passionate about Serverless, AWS, dev.to/cdkpatterns, TCO, and CI/CD

Discussion

markdown guide
 

I love that!
Thanks for sharing :)

 

Very nice article!
Congratulations.