DEV Community: CDK Patterns

Add Voice To Your Serverless Apps with Alexa Through AWS CDK

Matt Coulter — Thu, 22 Oct 2020 13:23:55 +0000

Introducing "The Alexa Skill" pattern at CDK Patterns

This is a community contribution by Chris Plankey which allows you to deploy an Alexa skill that is backed by a Lambda Function and a DynamoDB table. Out of the box the pattern will hold a short conversation where it will tell you a random selection of pattern names from cdk patterns.

Available Versions

//TypeScript
npx cdkp init the-alexa-skill

//Python
npx cdkp init the-alexa-skill --lang=python

Prerequisites

Unfortunately these steps cannot be skipped or shortened as you need to configure an Amazon developer account and acquire an OAuth 2.0 Refresh token that allows you to deploy your skill using CloudFormation.

1. Create an Amazon Developer account

Alexa is an Amazon product even though it can be deployed through AWS you still need to have a separate Amazon Developer account

2. Create a Developer Account Security Profile.

Open the Developer Account Security Profile page and feel free to use whatever values you want for the Security Profile Name and Description. The Privacy Notice URL must be a valid URL format but does not need to be a valid URL. Once you create your security profile, navigate to the Web Settings tab and add the following as Allowed Return URLs:

http://127.0.0.1:9090/cb
https://s3.amazonaws.com/ask-cli/response_parser.html

3. Copy Your `Client Id` and `Client Secret` from the security profile

Keep these values safe as we will use them in a second.

4. Make Sure You Have An AWS Account with CLI Access

You will need this to deploy the CDK stack anyway but for the next step we need to associate Alexa with our AWS Account so this is crucial

5. Setup ASK CLI on your local machine

Alexa needs to associate your Amazon Developer Account with your AWS Account. The easiest way to do this is to run ask configure after you have installed the Alexa Skills Kit CLI

6. Generate a Refresh Token

We are going to use Postman to fetch a new OAuth 2.0 token

Set the following key/values in the request:

Field	Value
Grant Type	Authorization Code
Callback URL	http://127.0.0.1:9090/cb
Auth URL	https://www.amazon.com/ap/oa
Access Token URL	https://api.amazon.com/auth/o2/token
Client ID	{YOUR_CLIENT_ID}
Client Secret	{YOUR_CLIENT_SECRET}
Scope	alexa::ask:skills:readwrite alexa::ask:models:readwrite alexa::ask:skills:test alexa::ask:catalogs:read alexa::ask:catalogs:readwrite

A Pop-Up should show up prompting you to log into your Developer account. Log in and you will be redirected to Postman where you should have a refresh_token to use in the next steps

7. Copy Your Vendor ID

Visit the Customer Details Page for your Amazon Developer Account and make a note of your "vendor ID"

Before You Deploy

You need to add your ClientID, ClientSecret, Refresh Token and VendorID to the skill resource which can be found in the-alexa-skill/typescript/lib/the-alexa-skill-stack.ts

      vendorId: 'foo',
      authenticationConfiguration: {
        clientId: 'foo',
        clientSecret: 'bar',
        refreshToken: 'foobar'
      },

Deployment

Since this is a CDK project you can deploy it with the CDK Deploy command

How Do I Test This After Deployment?

Navigate to the Alexa Developer Console, or follow this link - https://developer.amazon.com/alexa/console/ask
If you see a skill named "CDK Patterns Sample" in your Alexa Skills list then it has successfully been uploaded! Now we just need to test the skill itself.
Select the CDK Patterns Sample skill by clicking on the name.
On the next page, select "Test" at the top of the screen.
Amazon will ask if you'd to use your microphone or not. This is entirely optional as you may test Alexa using either voice commands or the text box provided.
Change the "skill testing is enabled in:" option from "Off" to "Development" if needed.
Either type or say "CDK Patterns" (Case sensitive if typing) and wait for a response.
The response should be "Hey, it\'s Pancakes the CDK Otter here, what would you like to know?"
For further testing, either type or say "What patterns do you have?"
The response should be "I have many patterns for you to see! For example, there is" followed by three pattern names randomly picked from CDK Patterns.
Now we just need to confirm that it is interacting with DynamoDB correctly.
Go to the AWS Console and navigate to DynamoDB. Open your tables and find the one corresponding to TheAlexaSkillStack.
Confirm that one item is in the table (It should have 2 attributes and a UserID). If it does then congratulations! Everything works!

Add Resiliency To Your Lambda Function with a Circuit Breaker

Matt Coulter — Thu, 15 Oct 2020 09:00:15 +0000

This pattern takes advantage of the awesome circuitbreaker-lambda library from Gunnar Grosch

View Codebase:
https://github.com/cdk-patterns/serverless/tree/master/the-lambda-circuit-breaker

To Clone:

//TypeScript
npx cdkp init the-lambda-circuit-breaker
//Python
npx cdkp init the-lambda-circuit-breaker --lang=python

The AWS Well-Architected Framework helps you understand the pros and cons of
decisions you make while building systems on AWS. By using the Framework, you will learn architectural best practices for designing and operating reliable, secure, efficient, and cost-effective systems in the cloud. It provides a way for you to consistently measure your architectures against best practices and identify areas for improvement.

We believe that having well-architected systems greatly increases the likelihood of business success.

The Reliability Pillar

Note - The content for this section is a subset of the Serverless Lens Whitepaper with some minor tweaks.

The reliability pillar includes the ability of a system to recover from infrastructure or service disruptions, dynamically acquire computing resources to meet demand, and mitigate disruptions such as misconfigurations or transient network issues.

REL 2: How are you building resiliency into your serverless application?

Evaluate scaling mechanisms for Serverless and non-Serverless resources to meet customer demand, and build resiliency to withstand partial and intermittent failures across dependencies.

Best Practices:

1 / Manage transaction, partial, and intermittent failures: Transaction failures might occur when components are under high load. Partial failures can occur during batch processing, while intermittent failures might occur due to network or other transient issues.

What's Included In This Pattern?

This is an implementation of the simple webservice pattern only instead of our Lambda Function using DynamoDB to store and retrieve data for the user it is being used to tell our Lambda Function if the webservice it wants to call is reliable right now or if it should use a fallback function.

To demonstrate this behaviour the Lambda function has some logic in it to simulate failure. The below logic will randomly fail:

function unreliableFunction () {
  return new Promise((resolve, reject) => {
    if (Math.random() < 0.6) {
      resolve({ data: 'Success' })
      message = 'Success'
    } else {
      reject({ data: 'Failed' })
      message = 'Failed'
    }
  })
}

Then we have a circuitbreaker configured with a fallback mechanism for when it has failed too many times recently:

function fallbackFunction () {
  return new Promise((resolve, reject) => {
    resolve({ data: 'Expensive Fallback Successful' })
    message = 'Fallback'
  })
}

const options = {
  fallback: fallbackFunction,
  failureThreshold: 3,
  successThreshold: 2,
  timeout: 10000
}

exports.handler = async (event:any) => {
  const circuitBreaker = new CircuitBreaker(unreliableFunction, options)
  await circuitBreaker.fire()
  const response = {
    statusCode: 200,
    body: JSON.stringify({
      message: message
    })
  }
  return response
}

When You Would Use This Pattern

When integrating with an external service and you want to provide a cost effective, resilient service for your consumers. If you don't do this and the external service is down you will be paying lambda invocation costs for the full request timeout plus your consumers will be waiting for an ultimately frustrating experience.

How To Test This Pattern

After you deploy this pattern you will have a url for an API Gateway where if you open it in a browser you will get a JSON payload back with one of two messages:

// when the circuit is closed and the unreliable function worked
{
   "message": "Success"
}
// when the circuit is open
{
   "message": "Fallback"
}

So refresh the browser a few times and you will see the message switch to Fallback then if you open the CloudWatch logs for your Lambda Function as you hit it again you should be able to watch the circuit breaker change state

You will see messages like:

INFO CircuitBreaker state: OPEN
INFO CircuitBreaker state: HALF
INFO CircuitBreaker state: CLOSED

Open means that no requests are going through to the unreliable function, half means that some requests are let through to test the stability of the unreliable function and closed means operating as normal.

CDK Patterns on the AWS Architecture Blog

Matt Coulter — Sun, 26 Jul 2020 07:41:17 +0000

I recently worked with AWS on an article talking about how we use AWS CDK, CDK Patterns and AWS Well Architected within Liberty IT.

AWS posted it on their Architecture blog:

To view the article visit this link

CDK Patterns at 20! Let's Walk Through all 20 Serverless Patterns for AWS

Matt Coulter — Mon, 20 Jul 2020 15:28:54 +0000

This is a link to the original post under my nideveloper account but I cannot move it yet because multiple sources still actively deep link to that post. I will swap over the content in a couple of weeks but I want visitors to cdkpatterns to find the article:

CDK Patterns at 20! Let's Walk Through all 20 Serverless Patterns for AWS

Matt Coulter ・ Jul 14 '20 ・ 12 min read

#aws #cdk #serverless #tutorial

Open your content to the world by combining Amazon Translate and Polly - Natural language translations and voice synthesis

Matt Coulter — Sat, 11 Jul 2020 09:03:19 +0000

This is a pattern that integrates the Amazon Polly service into an AWS Lambda Function so that you can synthesize text into speech using a serverless stack. It also integrates with Amazon Translate to allow you to choose the language for the spoken text.

Some Useful References:

Author	Link
Amazon Polly	Amazon Polly Site
Polly Pricing	Polly Pricing
Polly Permissions	Polly IAM Permissions
Amazon Translate	What Is Amazon Translate?
Translate Pricing	Translate Pricing
AWS Blogs	Giving your content a voice with the Newscaster speaking style from Amazon Polly
AWS Docs	Using Amazon Polly with Amazon Translate
Timothy Mugayi	Text-to-Speech: Build Apps That Talk With AWS Polly and Node.js
Philip Kiely	Text-To-Speech With AWS (Part 1)

Available Versions

What is Included In This Pattern?

After deployment you will have an API Gateway HTTP API configured where all traffic points to a Lambda Function that calls the Polly / Translate service.

API Gateway HTTP API

This is setup with basic settings where all traffic is routed to our Lambda Function

Lambda Function

Takes in whatever voice you want and whatever text you want, translates it to whatever language you want then sends it to the Polly service and returns an Audio stream

Testing The Pattern

After deployment in the deploy logs you will see the url for the API Gateway.

If you open that URL in chrome it will play an audio recording saying "To hear your own script, you need to include text in the message body of your restful request to the API Gateway"

For examples of the voice clips produced checkout the mp3 files in the recordings folder

You can customise this message based on how you call the url:

Changing the voice

You can pick from 3 voices "Matthew" (the default), "Joanna" or "Lupe". This is using the newsreader style of voice which AWS recently launched so it currently only supports these 3.

To change voices just add a query param onto your url like:

https://{api-url}/?voice=Lupe
https://{api-url}/?voice=Joanna
https://{api-url}/?voice=Matthew

Changing the language spoken

This Lambda Function is integrated with Amazon Translate so you can have Polly speak a variety of languages

To have Lupe speak Spanish just add the translateTo query param

https://{api-url}/?voice=Lupe&translateTo=es

If the text you are translating is in a language other than english you can use the translateFrom parameter

To understand what languages are possible please refer to the documentation

Changing the text

If you use a tool like Postman to send text in the body of a POST request to the url it will use Polly to synthesize your text

Calling Translate / Polly

Integrating our Lambda Function with these services was relatively straightforward.

First I needed to make sure the function had IAM permissions

// https://docs.aws.amazon.com/polly/latest/dg/api-permissions-reference.html
    // https://docs.aws.amazon.com/translate/latest/dg/translate-api-permissions-ref.html
    const pollyStatement = new iam.PolicyStatement({
      effect: iam.Effect.ALLOW,
      resources: ['*'],
      actions: [
        "translate:TranslateText",
        "polly:SynthesizeSpeech"
      ],
    });
    pollyLambda.addToRolePolicy(pollyStatement);

Then we can use translate and polly from the AWS SDK.

For Translate:

// If we passed in a translation language, use translate to do the translation
  if(translateTo !== translateFrom){
    const translate = new Translate();

    var translateParams = {
      Text: text,
      SourceLanguageCode: translateFrom,
      TargetLanguageCode: translateTo
    };

    let rawTranslation = await translate.translateText(translateParams).promise();
    text = rawTranslation.TranslatedText;
  }

For Polly:

// Use Polly to translate text into speech

  const polly = new Polly();

  const params = {
    OutputFormat: 'mp3',
    Engine:'neural',
    TextType:'ssml',
    Text: `<speak><amazon:domain name="news">${text}></amazon:domain></speak>`,
    VoiceId: voice,
  };

  let synthesis = await polly.synthesizeSpeech(params).promise();
  let audioStreamBuffer = Buffer.from(synthesis.AudioStream);

  return sendVoiceRes(200, audioStreamBuffer.toString('base64'));

Attach a FileSystem to your AWS Lambda Function

Matt Coulter — Thu, 09 Jul 2020 12:29:27 +0000

This is a pattern that attaches an EFS file system to your lambda function to give it expandable, persistent storage. Having this level of storage in a Lambda Function opens the door to many new possibilities (multiple functions can even use the same file system).

Some Useful References:

Author	Link
AWS Docs	Using Amazon EFS with Lambda
AWS Docs	Configuring file system access for Lambda functions
AWS Samples	EFS for Lambda - Example SAM applications
James Beswick	Using Amazon EFS for AWS Lambda in your serverless applications
Danilo Poccia	A Shared File System for Your Lambda Functions
Yan Cui	Unlocking New Serverless Use Caes With EFS and Lambda

Available Versions

What's Included In This Pattern?

This pattern covers the first half of Danilo Poccia's awesome blog post. After deployment you will have an API Gateway HTTP API where any url you hit gets directed to a Lambda Function that is integrated with EFS.

VPC

A VPC is bundled in this pattern because EFS requires it, this is using the default settings from CDK so if you want to put this in production you will have to review / refine this

EFS FileSystem

A FileSystem is included in the above VPC with a removal policy of destroy. In a production system you probably would want to retain your storage on stack deletion.

POSIX permissions are also setup for this File System

Lambda Function

A simple Python lambda function that interacts with the file system - storing, retrieving and deleting messages

API Gateway HTTP API

This is configured with the Lambda Function as the default handler for any url you hit.

How Do I Test This Pattern?

Our deployed Lambda Function is acting as a shared message broker. It allows you to send messages to it which it stores in EFS, then you can retrieve all messages to read them or delete all messages after you have finished reading.

The Lambda Function will behave differently based on the RESTful verb you use:

GET - Retrieve messages
POST - Send a message (whatever you send in the body is the message)
DELETE - Deletes all stored messages

The URL for the HTTP API to use these commands will be printed in the CloudFormation stack output after you deploy.

Note - After deployment you may need to wait 60-90 seconds before the implementation works as expected. There are a lot of network configurations happening so you need to wait on propagation

Protect your RDS MySQL DB from AWS Lambda Scalability

Matt Coulter — Mon, 06 Jul 2020 07:13:41 +0000

RDS MySQL is a resource that cannot scale at the same rate as fully serverless components like AWS Lambda. That is why you should always implement some kind of buffer between the two. Otherwise if your application gets a spike in traffic your RDS could fall over.

AWS have built that buffer for you in AWS RDS Proxy, it is a like for like replacement for your RDS integration and just sits in front of your existing DB.

I have built a reference architecture for you in AWS CDK that you can deploy in one command to understand how this works:

This is a project that has been configured with a MySQL RDS DB, an RDS Proxy, a Lambda Function to run queries and an API Gateway HTTP API to trigger the lambda function.

A VPC is included in this project that has the RDS Subnets configured and custom security groups for allowing communication between Lambda -> Proxy -> MySQL.

Some Useful References:

Author	Link
AWS RDS Proxy	RDS Proxy Site
AWS User Guide	Managing Connections with Amazon RDS Proxy
Ben Smith	Introducing the serverless LAMP stack - part 2
George Mao	Using Amazon RDS Proxy with AWS Lambda
SSL Cert for RDS MySQL	AmazonRootCA1.pem
AWS User Guide	Manual Steps for RDS Creation
AWS User Guide	MySQL in Amazon RDS
Node MySQL Lib	MySQL
AWS Docs	Secrets Manager JS SDK Docs
AWS User Guide	Creating and Retrieving a Secret

Available Versions

We believe that having well-architected systems greatly increases the likelihood of business success.

Serverless Lens Whitepaper

Well Architected Whitepaper

The Reliability Pillar

Note - The content for this section is a subset of the Serverless Lens Whitepaper with some minor tweaks.

REL 1: How are you regulating inbound request rates?

Best Practices:

Use mechanisms to protect non-scalable resources: Functions can scale faster than traditional resources, such as relational databases and cache systems. Protect non-scalable resources by adapting fast scaling components to downstream systems throughput.

For relational databases such as Amazon RDS, you can limit the number of connections per user in addition to the global maximum number of connections.

What is Included In This Pattern?

This pattern is a version of the scalable webhook built using AWS RDS Proxy.

You get a MySQL Database setup inside a VPC with appropriate subnets and security groups to connect with an RDS Proxy. That RDS Proxy is then communicated with via a Lambda Function / API Gateway HTTP API.

The big benefit here is that you are using fully managed infrastructure to protect the RDS DB, you have not needed to spin up your own queue and mechanism for how rapidly to pull from it.

For the AWS official benefits see this page

VPC

The VPC bundled with this pattern is the default one setup by the CDK L2 construct. In a production system you would want to tailor this to your needs

Security Groups

I bundled 2 security groups lambdaToRDSProxyGroup and dbConnectionGroup.

dbConnectionGroup allows TCP traffic on port 3306 from other peers within this group. It also allows TCP traffic on port 3306 for peers within the lambdaToRDSProxyGroup group.

I added the second group because I saw no need for peers to be allowed to hit the Lambda Function with TCP traffic on 3306. This separated the capability.

Auto Generated Password and Username in Secrets Manager

The username is a static value but we use secrets manager to generate the password to be used for our DB. We then give our Lambda Function permissions to read this secret so that it can connect to the DB. That means the only value that needs to be shared as an environment variable is the secret name,

MySQL RDS Instance

This is just a small, burstable instance using MySQL 5.7.22. I have removed the deletion protection and told Cloudformation to delete it on stack deletion because this is a learning stack. In a production stack, never use these two properties.

RDS Proxy

This is what we are using to protect the MySQL DB which is a small instance from the massively scalable Lambda Function that will be querying it. The proxy makes sure we do not overload it and shares connections between queries.

Lambda Function

This reads our username and password for our proxy from Secrets Manager then uses the MySQL library to create a database and table if they do not exist then insert a record for the url you hit on the API Gateway. Finally it queries the database for all records stored and returns them.

API Gateway HTTP API

Any url you hit on this gateway will integrate with the Lambda Function

Testing The Pattern

After you deploy this pattern you will have an API Gateway HTTP API where any url you hit gets routed to a Lambda function that inserts the URL path you hit into our MySQL table.

Simply open the url printed out in the deployment logs for our HTTP API in a browser and you should see a table containing all the urls you have hit. Try hitting a couple of different urls and watch the table grow.

Tune your AWS Lambdas for best cost vs performance with "The Lambda Power Tuner" serverless pattern

Matt Coulter — Sat, 09 May 2020 09:21:03 +0000

The Lambda Power Tuner

Available CDK Versions

Alternatively use the cdkp command line tool to download the code:

TypeScript

npx cdkp init the-lambda-power-tuner

Python

npx cdkp init the-lambda-power-tuner --lang=python

This is an AWS CDK project that deploys the awesome AWS Lambda Power Tuning project.

AWS Lambda Power Tuning is an AWS Step Functions state machine that helps you optimize your Lambda functions in a data-driven way.

The state machine is designed to be quick and language agnostic. You can provide any Lambda function as input and the state machine will run it with multiple power configurations (from 128MB to 3GB), analyze execution logs and suggest you the best configuration to minimize cost or maximize performance.

The input function will be executed in your AWS account - performing real HTTP calls, SDK calls, cold starts, etc. The state machine also supports cross-region invocations and you can enable parallel execution to generate results in just a few seconds. Optionally, you can configure the state machine to automatically optimize the function and the end of its execution.

The reason for doing this is that it helps with two of the Serverless Well Architected pillars:

Performance Efficiency Pillar
Cost Optimization Pillar

We believe that having well-architected systems greatly increases the likelihood of business success.

Serverless Lens Whitepaper

Well Architected Whitepaper

Performance Efficiency Pillar

The performance efficiency pillar focuses on the efficient use of computing resources to meet requirements and the maintenance of that efficiency as demand changes and technologies evolve.

Performance efficiency in the cloud is composed of four areas:

Selection
Review
Monitoring
Tradeoffs

Take a data-driven approach to selecting a high-performance architecture. Gather data on all aspects of the architecture, from the high-level design to the selection and configuration of resource types. By reviewing your choices on a cyclical basis, you will ensure that you are taking advantage of the continually evolving AWS Cloud.

Monitoring will ensure that you are aware of any deviance from expected performance and can take action on it. Finally, you can make tradeoffs in your architecture to improve performance, such as using compression or caching, or by relaxing consistency requirements.

PER 1: How have you optimized the performance of your serverless application?

Selection

Run performance tests on your serverless application using steady and burst rates. Using the result, try tuning capacity units and load test after changes to help you select the best configuration:

Lambda: Test different memory settings as CPU, network, and storage IOPS are allocated proportionally.

Cost Optimization Pillar

The cost optimization pillar includes the continual process of refinement and improvement of a system over its entire lifecycle. From the initial design of your first proof of concept to the ongoing operation of production workloads, adopting the practices in this document will enable you to build and operate cost-aware systems that achieve business outcomes and minimize costs, thus allowing your business to maximize its return on investment.

There are four best practice areas for cost optimization in the cloud:

Cost-effective resources
Matching supply and demand
Expenditure awareness
Optimizing over time

COST 1: How do you optimize your costs?

Cost-Effective Resources

Serverless architectures are easier to manage in terms of correct resource allocation. Due to its pay-per-value pricing model and scale based on demand, serverless effectively reduces the capacity planning effort.

As covered in the operational excellence and performance pillars, optimizing your serverless application has a direct impact on the value it produces and its cost.

As Lambda proportionally allocates CPU, network, and storage IOPS based on
memory, the faster the execution the cheaper and more value your function produces due to 100-ms billing incremental dimension.

Default Configuration Settings Provided

There are some variables that you can pass into the SAR app to manipulate the power tuning step function. You can find two that I have set for you at the top of the cdk stack

let powerValues = '128,256,512,1024,1536,3008';
let lambdaResource = "*";

the powerValues lets you pick exactly what AWS Lambda memory settings you want to tune against. The full list of allowed values is:

['128','192','256','320','384','448','512','576','640','704','768','832','896','960','1024','1088','1152','1216','1280','1344','1408','1472','1536','3008']

lambdaResource is about what IAM permissions do you want to give the state machine? In general, you want to give your components the least privileges they require to reduce their blast radius.

By default the power tuner uses * permissions which means that it has wide scope and can tune any function. If you can scope this down to something more specific that is advisable.

Alex gave me this advice

I can see 3 common patterns :
1) use * (easy default, not always ideal)
2) restrict to region or name prefix (better)
3) restrict to only 1 ARN (not very flexible but ideal for CI/CD scenarios where you’ll delete the stack immediately after tuning)

An example of option 3 is included in the stack but currently commented out, so all you have to do is uncomment it.

// Uncomment to only allow this power tuner to manipulate this defined function
//lambdaResource = exampleLambda.functionArn;

How To Test This Pattern

After deployment, navigate to the step functions section of the AWS Console.

from the list of availabe state machines, pick the power tuner state machine.

Now click "start execution" in the top right

In the input field enter the following JSON and add in the ARN to the lambda you want to test.

You can either use the example lambda we bundled by getting the ARN from the cdk deploy logs or any another function in your account if you know the ARN.

{
  "lambdaARN": "your lambda arn to test",
  "powerValues": [
    128,
    256,
    512,
    1024,
    2048,
    3008
  ],
  "num": 10,
  "payload": {},
  "parallelInvocation": true,
  "strategy": "cost"
}

Click "Start Execution" in the bottom right.

When the tuner has finished your visual workflow should look like:

Then you can scroll down to the very last event and expand it to get the URL for your results graph:

Learn Serverless Tracability with "The X-Ray Tracer" CDK Pattern - Lambda, SNS, SQS, DynamoDB

Matt Coulter — Mon, 04 May 2020 18:57:28 +0000

The X-Ray Tracer

This is a pattern not defined by the components used but how they send information back to the AWS X-Ray service to help you make your application perform better when viewed through the Serverless Well-Architected lens. A fully well architected solution would use embedded metric format for the logs like in the Julian Wood reference below but I am saving that for another pattern so as not to confuse the concepts.

Some useful references:

Author	Link
AWS X-Ray Developer Guide	X-Ray Developer Guide
AWS X-Ray Concepts	X-Ray Concepts
AWS Serverless Lens Whitepaper	Serverless Lens Whitepaper
AWS Well Architected Whitepaper	Well Architected Whitepaper
Julian Wood	Building Well Architected Applications
AWS Developer Blog	Category: AWS X-Ray
AWS Training	Introduction to AWS X-Ray

Available CDK Versions

Alternatively use the cdkp command line tool to download the code:

TypeScript

npx cdkp init the-xray-tracer

Python

npx cdkp init the-xray-tracer --lang=python

We believe that having well-architected systems greatly increases the likelihood of business success.

Serverless Lens Whitepaper

Well Architected Whitepaper

The Operational Excellence Pillar

Note - The content for this section is a subset of the Serverless Lens Whitepaper with some minor tweaks.

The operational excellence pillar includes the ability to run and monitor systems to deliver business value and to continually improve supporting processes and procedures.

OPS 1: How do you understand the health of your Serverless application?

Distributed Tracing

Similar to non-serverless applications, anomalies can occur at larger scale in distributed systems. Due to the nature of serverless architectures, it’s fundamental to have distributed tracing.

Making changes to your serverless application entails many of the same principles of deployment, change, and release management used in traditional workloads. However, there are subtle changes in how you use existing tools to accomplish these principles.

Active tracing with AWS X-Ray should be enabled to provide distributed tracing capabilities as well as to enable visual service maps for faster troubleshooting.

X-Ray helps you identify performance degradation and quickly understand anomalies, including latency distributions.

Service Maps are helpful to understand integration points that need attention and resiliency practices. For integration calls, retries, backoffs, and possibly circuit breakers are necessary to prevent faults from propagating to downstream services.

Another example is networking anomalies. You should not rely on default timeouts and retry settings. Instead, tune them to fail fast if a socket read/write timeout happens where the default can be seconds if not minutes in certain clients.

X-Ray also provides two powerful features that can improve the efficiency on identifying anomalies within applications: Annotations and Subsegments.
Subsegments are helpful to understand how application logic is constructed and what external dependencies it has to talk to. Annotations are key-value pairs with string, number, or Boolean values that are automatically indexed by AWS X-Ray.

Combined, they can help you quickly identify performance statistics on specific
operations and business transactions, for example, how long it takes to query a
database, or how long it takes to process pictures with large crowds.

What is Included In This Pattern?

I wanted to make this pattern as "real" as possible for people so I included most of the serverless components you will use everyday. I have included:

API Gateway -> SNS -> Lambda (not SQS for reasons documented later)
Lambda -> DynamoDB
Lambda -> SQS -> Lambda
Lambda -> External Http Endpoint
Lambda -> SNS -> Lambda

When I map the flow in a high level conceptual image it looks like this:

After deployment, the X-Ray service map looks something like (You get two circles per Lambda):

Or if you look at CloudWatch Service Lens:

You can see that these two diagrams aren't a massive distance away from my high level conceptual flow, the difference is the X-Ray generated diagram is 100% accurate because it is created from real traces based on user flow. When viewed through the AWS Console, it cannot become the out of date diagram you found on a wiki last updated 12 months ago - it is always accurate. If a developer checks in a piece of code that changes the flow, you will see it immediately.

Testing This Pattern

After you deploy this pattern you will have an API Gateway with the URL being output in the CDK Deploy logs.

Any URL you hit on that gateway will trigger this flow, it uses your URL as the message sent to SNS.

This URL is inserted into DynamoDB with a counter of how many times it was hit, the SNS and the SQS consumer lambdas both log the message to CloudWatch.

To see the random service errors try to hit the URL at least 10 times then you can either navigate to the X-Ray section of the AWS Console followed by clicking on "Service Map" in the sidebar. Please be aware that sometimes there can be a 30 second or so delay before your calls show up in service map.

Alternatively you can go to CloudWatch in the AWS Console and click "Service Lens" in the sidebar. Both offer views onto the system.

Deliberate Random Errors

I introduced a random SSL Cert error into the Lambda that connects to the External Http Endpoint to let you experiment with using X-Ray to source an error

Service map showing something isn't healthy:

Trace details showing error:

Enabling X-Ray

Depending on the component you are using and what it is integrating with you need to enable X-Ray in a different way.

API Gateway

This is done by simply setting a property of tracingEnabled: true on deployOptions:

let gateway = new apigw.RestApi(this, 'xrayTracerAPI', {
    deployOptions: {
    metricsEnabled: true,
    loggingLevel: apigw.MethodLoggingLevel.INFO,
    dataTraceEnabled: true,
    tracingEnabled: true,
    stageName: 'prod'
    }
});

Lambda

You set a tracing property to lambda.Tracing.ACTIVE

this.httpLambda = new lambda.Function(this, 'httpLambdaHandler', {
    runtime: lambda.Runtime.NODEJS_12_X,
    code: lambda.Code.asset('lambdas'),
    handler: 'http.handler',
    tracing: lambda.Tracing.ACTIVE
});

SNS/SQS/DynamoDB

These just pick up when being called from a component with tracing enabled, they do not need a specific setting to enable it.

AWS SDK Calls

You need to make sure your AWS SDK code is wrapped with X-Ray during invocation. This is true of any SDK calls e.g. Lambda to Lambda direct invoke, DynamoDB queries, Publishing to SNS etc

const AWSXRay = require('aws-xray-sdk');
// Wrap AWS SDK with X-Ray
const AWS = AWSXRay.captureAWS(require('aws-sdk'));

exports.handler = async function(event:any) {

  // Create an SQS service object as normal
  var sqs = new AWS.SQS({apiVersion: '2012-11-05'});

External HTTP Requests

You need to wrap the https module with X-Ray:

const AWSXRay = require('aws-xray-sdk');
// Wrap HTTPS module with X-Ray
var https = AWSXRay.captureHTTPs(require('https'));

exports.handler = async function(event:any) {

    // Make a call to a webservice as normal
    const req = https.get("https://url.com", (res:any) => {

Subsegments, Metadata and Annotations

Subsegments

A segment can break down the data about the work done into subsegments. Subsegments provide more granular timing information and details about downstream calls that your application made to fulfill the original request. A subsegment can contain additional details about a call to an AWS service, an external HTTP API, or an SQL database. You can even define arbitrary subsegments to instrument specific functions or lines of code in your application.

I have included some custom subsegments in this pattern, like "external HTTP Request" below:

These are easy to create inside the Lambda Functions:

const subsegment = segment.addNewSubsegment('external HTTP Request');

let response = await new Promise((resolve:any, reject:any) => {
    // Make a call to a webservice
    const req = https.get("https://url.com", (res:any) => {
        ... //resolve promise
    });
    ... //reject promise
});

subsegment.addMetadata("response", response)
subsegment.close();

Metadata

You are allowed to put whole objects inside metadata, this is brilliant for showing things like the response from a webservice.

Metadata are key-value pairs that can have values of any type, including objects and lists, but are not indexed for use with filter expressions. Use metadata to record additional data that you want stored in the trace but don't need to use with search.

Annotations

Annotations are key-value pairs with string, number, or Boolean values. Annotations are indexed for use with filter expressions. Use annotations to record data that you want to use to group traces in the console, or when calling the GetTraceSummaries API.

X-Ray Groups

X-Ray groups enable customers to slice and dice their X-Ray service graph and focus on certain workflows, applications, or routes.

Customers can create a group by setting a filter expression. All the traces that match the set filter expression will be part of that group. Customers can then view service graphs for the selected group, and understand performance bottlenecks, errors, or faults in services belonging to that service graph.

Deep dive into AWS X-Ray groups and use cases

A useful tip is that Annotations can be used in filter expressions so you can easily create groups for smaller bounded contexts within a larger service map and then create CloudWatch alerts per group.

Known X-Ray "Quirks"

There are a couple of X-Ray quirks that I need to document, I thought it better to show them than refactor the pattern to hide them then you hit one later. If these are a deal breaker for you there are other tools out there for tracing that I have been promised will integrate with no extra code changes like Epsagon

SQS -> Lambda Traces

There is a known bug where this doesn't connect and you end up with two paths on your service map.

I have included some logic inside the SQS subscriber lambda to move an X-Ray custom subsegment trace circle from the new second flow to where it should be but this is a workaround and hopefully that bug gets closed sooner than later.

API Gateway -> SNS through direct integration

X-Ray does work as expected with SNS when using the AWS SDK but for some reason when I do a direct integration with API Gateway through VTL the service map shows the subscribers of the SNS topic as being connected to API GW rather than SNS which is fine because I am not missing information but it's not correct. If I workout a fix for this I will update the pattern.

Service Map Changes Design Randomly

Unlike other tracing solutions I have used that let you position all of the circles where they give a feeling of inner calmness based on personal OCD; X-Ray randomly positions the circles on every refresh which can lead to some interesting map layouts. The important thing is being able to spot anomalies which you can definitely still do so this is purely aesthetic.

Watch me deconstruct "The Scalable Webhook" AWS Serverless Pattern - Lambda, API GW, SQS

Matt Coulter — Wed, 29 Apr 2020 07:49:24 +0000

The Scalable Webhook

This is an example CDK stack from CDK Patterns to deploy The Scalable Webhook stack described by Jeremy Daly here - https://www.jeremydaly.com/serverless-microservice-patterns-for-aws/#scalablewebhook

An advanced version of this pattern was talked about by Heitor Lessa at re:Invent 2019 as Call me, “Maybe” (Webhook)

Desconstructing The Scalable Webhook

If you want a walkthrough of the theory, the code and finally a demo of the deployed implementation check out:

High Level Description

You would use this pattern when you have a non serverless resource like an RDS DB in direct contact with a serverless resource like a lambda. You need to make
sure that your serverless resource doesn't scale up to an amount that it DOS attacks your non serverless resource.

This is done by putting a queue between them and having a lambda with a throttled concurrency policy pull items off the queue and communicate with your
serverless resource at a rate it can handle.

NOTE: For this pattern in the cdk deployable construct I have swapped RDS for DynamoDB.

Why? Because it is significantly cheaper/faster for developers to deploy and maintain, I also don't think we lose the essence of the pattern with this swap given we still do the pub/sub deduplication via SQS/Lambda and throttle the subscription lambda. RDS also introduces extra complexity in that it needs to be deployed in a VPC. I am slightly worried developers would get distracted by the extra RDS logic when the main point is the pattern. A real life implementation of this pattern could use RDS MySQL or it could be a call to an on-prem mainframe, the main purpose of the pattern is the throttling to not overload the scale-limited resource.

Pattern Background

When people move to the cloud (especially serverless) they tend to think that this means their applications are now infinitely scalable:

For the right reasons this just isn't true. If any one person's resources were infinitely scalable then any one person could consume the whole of AWS no matter how scalable the platform.

If we weren't using DynamoDB, we would need to know the max connections limit configured for our instance size:

We need to slow down the amount of direct requests to our DB somehow, that is where the scalable webhook comes in:

We can use SQS to hold all requests in a queue as soon as they come in. Again, SQS will have limits:

120,000 in flight messages with an unlimited backlog I think will be effective enough as a buffer.

View SQS quotas

Now we have our messages in a queue but we need to subscribe to the queue and insert the records into the DB. To do this we create a throttled lambda where we set the max number of concurrent executions to whatever scale we are happy with. This should be less than the max connections on our DB and should take into account any other Lambdas running in this account.

One final improvement that we could make if implementing this in a production system is to delete the Lambda between the API Gateway and SQS. You can do a direct integration which will reduce costs and latency:

If you want an AWS managed service to try and help with this scalability problem you can check out AWS RDS Proxy which is in preview

Learn "the saga stepfunction" pattern today - Single Table DynamoDB, Lambdas, Step Function and API Gateway

Matt Coulter — Fri, 17 Apr 2020 08:10:24 +0000

Youtube Walkthrough

Getting The Code

This is available on cdkpatterns.com or you can run the following commands:

// Typescript version
npx cdkp init the-saga-stepfunction

// Python version
npx cdkp init the-saga-stepfunction --lang=python

The Saga Step Function

This is a pattern that I found via Yan Cui and his 2017 Blog Post.

After doing research I found some other references:

Author	Link
Hector Garcia-Molina	Research Paper
AWS Well Architected WhitePaper	Serverless Application Lens
Yan Cui	Blog Post
Caitie McCaffrey
Heitor Lessa	AWS Serverless Airline Booking Example and YouTube Recording
Jeremy Daly	EventBridge Saga Tweet
Chris Richardson	Blog Post and Book
Chaos Gears	Blog Post
Yos Riady	Blog Post

What Is The Saga Pattern?

Hector Garcia-Molina described it in his paper as follows:

Long lived transactions (LLTs) hold on to database resources for relatively long periods of
time, signficantly delaying the termination of shorter and more common transactions To alleviate these problems we
propose the notion of a saga.

A LLT is a saga if it can be written as a sequence of transactions that can be interleaved
with other transactions. The database management system guarantees that either all the transactions in a saga are successfully completed or compensating transactions are run to amend a partial execution.

You can think of this as a complete transaction is made up of a series of smaller tasks. We need all of these tasks to
be successful for us to call the transaction a success.

Caitie uses a holiday booking example to demonstrate this which Yan elaborated on so let's continue the trend. If you are booking a holiday let's say you need at a minimum:

To Book Flights
To Book A hotel
To Pay

You wouldn't be very happy if you booked a holiday then found out when you landed that you had a reservation at the hotel but an error occured with payment so they gave it away. The saga pattern forces you to have a compensating action for that payment error, either you have some other payment selection process or you roll back the whole booking and ask the customer to try again.

Every action must have a corresponding reaction for error. Note the reaction cannot always be equal as Caitie points out, if one of the actions was to send an email you cannot undo that send but you can send a follow up to say it was an error.

If we assume from this point we will roll back when an error hits then the flow might look something like:

Success

This flows as you might expect - we reserve a room in the hotel, a spot on the plane, take the payment, then confirm the booking with the airline and hotel. Finally we notify the customer that it was a successful booking.

Failure

If after reserving the flight and hotel our payment fails then we need to release that reservation and notify the customer it failed.

Notice how it peels back the layers, it doesn't do one massive compensation step. It runs the cancel steps in reverse order until the system should be the way it was before we started.

If the first ReserveHotel task had failed the only difference is the number of Cancel tasks that run:

What Does The Saga Step Function Look Like?

We have an API Gateway connected to a Lambda through a {proxy+} setup. This lambda starts a stepfunction workflow representing the flows above. 8 lambdas inside that workflow communicate with 1 DynamoDB table to complete a travel booking transaction:

Saga Lambda and Step Fuction Exection

The Saga Lambda is a function that takes in input from the query parameters in the url and passes them to a step function execution. The data passed to the step function looks like:

let input = {
        "trip_id": tripID, //taken from queryParams
        "depart": "London",
        "depart_at": "2021-07-10T06:00:00.000Z",
        "arrive": "Dublin",
        "arrive_at": "2021-07-12T08:00:00.000Z",
        "hotel": "holiday inn",
        "check_in": "2021-07-10T12:00:00.000Z",
        "check_out": "2021-07-12T14:00:00.000Z",
        "rental": "Volvo",
        "rental_from": "2021-07-10T00:00:00.000Z",
        "rental_to": "2021-07-12T00:00:00.000Z",
        "run_type": runType //taken from queryParams
    };

Lambdas Inside Our Step Function

Author	Description
Reserve Hotel	Inserts a record into DynamoDB for our hotel booking with a transaction_status of pending
Reserve Flight	Inserts a record into DynamoDB for our flight booking with a transaction_status of pending
Cancel Hotel Reservation	Deletes the record from DynamoDB for our pending hotel booking
Cancel Flight Reservation	Deletes the record from DynamoDB for our pending Flight booking
Take Payment	Inserts a record into DynamoDB for the payment
Cancel Payment	Deletes the record from DynamoDB for the payment
Confirm Hotel	Updates the record in DynamoDB for transaction_status to confirmed
Confirm Flight	Updates the record in DynamoDB for transaction_status to confirmed

Error Handling and Retry Logic

If an error occurs in any of the reserve tasks, confirm tasks or the take payment task (either by you manually passing the trigger or a real error) we have step function catch logic to route to the appropriate cancel event.

You also need to account for errors in the cancel functions. That is why there is a random fail trigger in each cancel function.

if (Math.random() < 0.4) {
    throw new Error("Internal Server Error");
}

To handle this each cancel function has a built in retry policy of 3 attempts as part of the step function definition.

DynamoDB Table

We have 3 separate entities inside the one DynamoDB table, this was inspired by Alex Debrie and his brilliant book. If you want to learn more about advanced single table DynamoDB patterns it is worth a purchase.

You can see that the sort key on our table is overloaded to allow us to effectively filter results:

More columns exist than is shown above. The data inserted for each record is as follows:

// Hotel Data Model
var params = {
    TableName: process.env.TABLE_NAME,
    Item: {
      'pk' : {S: event.trip_id},
      'sk' : {S: 'HOTEL#'+hotelBookingID},
      'trip_id' : {S: event.trip_id},
      'type': {S: 'Hotel'},
      'id': {S: hotelBookingID},
      'hotel' : {S: event.hotel},
      'check_in': {S: event.check_in},
      'check_out': {S: event.check_out},
      'transaction_status': {S: 'pending'}
    }
  };

// Flights Data Model
var params = {
      TableName: process.env.TABLE_NAME,
      Item: {
        'pk' : {S: event.trip_id},
        'sk' : {S: 'FLIGHT#'+flightBookingID},
        'type': {S: 'Flight'},
        'trip_id' : {S: event.trip_id},
        'id': {S: flightBookingID},
        'depart' : {S: event.depart},
        'depart_at': {S: event.depart_at},
        'arrive': {S: event.arrive},
        'arrive_at': {S: event.arrive_at},
        'transaction_status': {S: 'pending'}
      }
    };

// Payments Data Model
var params = {
      TableName: process.env.TABLE_NAME,
      Item: {
        'pk' : {S: event.trip_id},
        'sk' : {S: 'PAYMENT#'+paymentID},
        'type': {S: 'Payment'},
        'trip_id' : {S: event.trip_id},
        'id': {S: paymentID},
        'amount': {S: "450.00"},
        'currency': {S: "USD"},
        'transaction_status': {S: "confirmed"}
      }
    };

How Do I Test This After Deployment?

After deployment you should have an API Gateway where any url you hit triggers the step function to start.

You can manipulate the flow of the step function with a couple of url parameters:

Successful Execution - https://{api gateway url}
Reserve Hotel Fail - https://{api gateway url}?runType=failHotelReservation
Confirm Hotel Fail - https://{api gateway url}?runType=failHotelConfirmation
Reserve Flight Fail - https://{api gateway url}?runType=failFlightsReservation
Confirm Flight Fail - https://{api gateway url}?runType=failFlightsConfirmation
Take Payment Fail - https://{api gateway url}?runType=failPayment

Inserting Muliple trips into DynamoDB, by default it will use the same ID on every execution
https://{api gateway url}?tripID={whatever you want}

It is important to note that the Cancel Lambdas all have a random failure built in and retry logic up to a max of 3. So when you look at the execution of your stepfunction in the aws console if you see failures in the cancel lambdas this is intentional. The reason why is to teach you that the cancel logic should attempt to self recover in the event of an error. Given that they only retry 3 times it is still possible for the cancel process to fail 3 times and the step function to terminate early.

To actually view what happened you will need to log into the AWS console and navigate to the step functions section where you can see every execution of your saga step function. You can also look inside the DynamoDB table at the records inserted. If you are fast enough with refresh you can watch them go from pending to confirmed status.

Watch me deconstruct "The EventBridge ETL" Serverless Architecture Pattern

Matt Coulter — Sun, 12 Apr 2020 08:11:44 +0000

The EventBridge ETL

This is an example stack showing how you can use EventBridge to orchestrate events through an ETL process. This pattern was insired by Vyas Sarangapani and Hervé Nivon as you can see at the bottom of this page.

Note - This is a learning pattern, if I was implementing this in a production system I would make a couple of changes:

KMS Encryption of sensitive data in events
SQS between EventBridge and the Lambdas for resiliency
Add in error events to EventBridge and error event rules

Architecture:

Architecture Notes

Fargate ECS Task

I chose to use a Fargate container to download the file from s3 rather than using Lambda. For the small bundled test data csv Lambda would have worked but I felt it would be misleading and suggestive that you could pull larger files down onto a Lambda function. Lambda functions have a few limitations around memory, storage and runtime. You can do things like partially stream files from s3 to Lambda (if they happen to be in the right format) and then store state somewhere between timeouts but I felt that having an ECS Task that you can define CPU, RAM and Disk Space was the much more flexible way to go and being Fargate you are still on the serverless spectrum. You can see how cheap Fargate is if you go into the cost breakdown in Hervé's GitHub repo.

Throttling The Lambda Functions

Without throttling, if you put every row in a huge csv file onto EventBridge with a subscriber lambda; That lambda can scale up until it uses all the concurrency on your account. This may be what you want (probably not though). That is why I limited all the concurrency of the lambdas, you can remove this limit or tweak as much as you want but you always need to think about what else is running in that account. Isolate your stack into its own account if possible.

Observer Lambda

In the current format this is more of a technical demo to show what is possible with event driven architectures. Everything that it logs is already in the logs of all the individual components. You could probably use this to keep a tally for every record that gets pulled from the csv to make sure it gets inserted into DynamoDB by pulling the ids from the extraction and load events.

When You Would Use This Pattern

If you need to create a process where a user uploads a csv and it gets transformed and inserted into DynamoDB

How to test pattern

After deployment you will have an s3 bucket where if you go into the aws console for that bucket and upload the file test_data.csv found in the data-to-upload folder you will kick everything off.

After you upload that file everything should automatically start working. You should be able to watch the process by looking in the cloudwatch logs for your observer lambda. Note this is a multi-threaded, concurrent process so the order of the events in the observer logs may not be what you were expecting!

Finally all of the data ends up in your DynamoDB table so you should be able to open it in the console and view the data after transform.

DEV Community: CDK Patterns

Add Voice To Your Serverless Apps with Alexa Through AWS CDK

Introducing "The Alexa Skill" pattern at CDK Patterns

Available Versions

Prerequisites

1. Create an Amazon Developer account

2. Create a Developer Account Security Profile.

3. Copy Your Client Id and Client Secret from the security profile

4. Make Sure You Have An AWS Account with CLI Access

5. Setup ASK CLI on your local machine

6. Generate a Refresh Token

7. Copy Your Vendor ID

Before You Deploy

Deployment

How Do I Test This After Deployment?

Add Resiliency To Your Lambda Function with a Circuit Breaker

The Reliability Pillar

What's Included In This Pattern?

When You Would Use This Pattern

How To Test This Pattern

CDK Patterns on the AWS Architecture Blog

CDK Patterns at 20! Let's Walk Through all 20 Serverless Patterns for AWS

CDK Patterns at 20! Let's Walk Through all 20 Serverless Patterns for AWS

Matt Coulter ・ Jul 14 '20 ・ 12 min read

Open your content to the world by combining Amazon Translate and Polly - Natural language translations and voice synthesis

Available Versions

What is Included In This Pattern?

API Gateway HTTP API

Lambda Function

Testing The Pattern

Changing the voice

Changing the language spoken

Changing the text

Calling Translate / Polly

Attach a FileSystem to your AWS Lambda Function

Available Versions

What's Included In This Pattern?

VPC

EFS FileSystem

Lambda Function

API Gateway HTTP API

How Do I Test This Pattern?

Protect your RDS MySQL DB from AWS Lambda Scalability

Available Versions

The Reliability Pillar

What is Included In This Pattern?

VPC

Security Groups

Auto Generated Password and Username in Secrets Manager

MySQL RDS Instance

RDS Proxy

Lambda Function

API Gateway HTTP API

Testing The Pattern

Tune your AWS Lambdas for best cost vs performance with "The Lambda Power Tuner" serverless pattern

The Lambda Power Tuner

Available CDK Versions

Performance Efficiency Pillar

Selection

Cost Optimization Pillar

Cost-Effective Resources

Default Configuration Settings Provided

How To Test This Pattern

Learn Serverless Tracability with "The X-Ray Tracer" CDK Pattern - Lambda, SNS, SQS, DynamoDB

The X-Ray Tracer

Available CDK Versions

The Operational Excellence Pillar

Distributed Tracing

What is Included In This Pattern?

Testing This Pattern

Deliberate Random Errors

Enabling X-Ray

API Gateway

Lambda

SNS/SQS/DynamoDB

AWS SDK Calls

External HTTP Requests

Subsegments, Metadata and Annotations

Subsegments

Metadata

3. Copy Your `Client Id` and `Client Secret` from the security profile