DEV Community: Marko Djakovic

Add Chat AI Summary Using Amazon Bedrock and HTTP Response Streaming

Marko Djakovic — Mon, 06 Apr 2026 18:24:23 +0000

Ever since I started writing what turned out to be a series of articles on how to build real-time chat apps natively on AWS, I had a huge backlog of ideas for extending the basic chat, which included adding authentication & authorization, full-text search, threads, reactions, and many other things that make any chat system production grade and up to par with what users expect these days. Given how AI has entered every pore of the industry, extending my small chat project had to go in this direction sooner or later. It all started with creating a solution that (ab)uses IoT Core, which worked exceptionally well - maybe even too well, as AWS ended up designing AppSync Events in a similar fashion. Read more on what I mean in my blog post about Serverless Chat on AWS with AppSync Events. Shortly after the initial release, AWS pushed out the promised improvements for AppSync Events, which I explore in the following blog post about leveraging WebSockets with it.

To build on top of existing chat solution, this time we won't touch the main infrastructure that powers the chat but rather extend the group chat capabilities by adding a dedicated AI summary endpoint. This is one of the most common AI use cases, and apps like Viber or Slack already have it. Most of us can relate to the pain of having to go through numerous Slack threads with dozens of messages each - so I'd say this feature is more than only nice to have at this point. Let's see how we can do it using Amazon Bedrock's ConverseStreamCommand, and leveraging HTTP response streaming capability of API Gateway.

Architecture

As I mentioned, we won't be touching the core infrastructure of the chat solution which is AppSync and the whole WebSockets part, but rather extend the system with a dedicated endpoint to create the AI summary of chat messages. So, essentially what we're talking about here is shown in the following diagram:

New, and the most interesting addition here is Amazon Bedrock. Also, since API Gateway supports HTTP response streaming, we'll make sure to leverage that to get the fastest possible response when invoking AI summary endpoint.

If we zoom back out, the whole solution looks something like:

So just to be clear - this is what will get deployed if you decide to run this in your own account. Let's break down how the two new key pieces fit into the mix.

Amazon Bedrock

I won't waste words on this since it is quite familiar, but in short - Bedrock is a fully managed, serverless AWS service for building generative AI applications. It provides a single API access for wide variety of foundational models from leading AI companies such as Anthropic, Meta, and Amazon itself. For this particular example I will rely on one of the smallest Amazon models - amazon.nova-micro-v1:0. It is more than enough for showcasing the idea, but feel free to play with others according to your liking.

Small caveat: when specifying the model, it is not enough to put only the name of the model, but you need to include also the region prefix of where your app will run. For example in my case it was eu.amazon.nova-micro-v1:0. Worth knowing just so you don't get confused when you see it in the code.

A dedicated Lambda function for AI summary will first fetch group chat messages from the database, and then create a prompt for Bedrock to create a short summary. It uses ConverseStreamCommand by BedrockRuntimeClient to send the prompt to Bedrock and get the stream response. To make the response smoothly stream into Lambda function's response, there is a small caveat compared to how you would normally define the handler in TypeScript. One is defining the handler as:

awslambda.streamifyResponse(async (event: APIGatewayProxyEvent, responseStream: awslambda.HttpResponseStream)

and handling the streaming response:

const httpStream = awslambda.HttpResponseStream.from(responseStream, {
    statusCode: 200,
    headers: {
        'Content-Type': 'text/plain; charset=utf-8',
        'Access-Control-Allow-Origin': '*',
    },
});

...and then the httpStream will be used to write the stream response from Bedrock into it.

Note: for simplicity I have set a limit of 50 messages to fetch for the summary, however it is configurable.

HTTP Response Streaming with API Gateway

Finally, the last piece to get the streaming response to the client is required to be configured on API Gateway. This capability was introduced in November 2025 and has CDK support from version 2.227.0 via responseTransferMode property. Setting it to ResponseTransferMode.STREAM on the Lambda integration resource will enable streaming:

getSummaryResource.addMethod('GET', new LambdaIntegration(getSummaryLambda, {responseTransferMode: ResponseTransferMode.STREAM}));

Demo

To showcase how the finished feature works, I've created a short video. I prefilled the group chat with test messages to simulate a conversation between team of engineers working on a production deployment. The AI summary will summarize their agreements and key points they discussed about the deployment.

Watch the demo video

Conclusions

In the end, building an AI summary today isn’t something groundbreaking, and to be honest that might be the point. What used to feel like cutting-edge quickly became the standard what users expect in these types of applications. Amazon Bedrock doing the heavy lifting on the model side makes the overall path from idea to working solution really smooth. I encourage you to try it yourself, check out the code and follow the instructions to deploy to your own AWS account:

https://github.com/imflamboyant/serverless-aws-chat/tree/main/chat-appsync-events-websocket

💸 Note: as always, be aware that usage might incur real costs, though relatively small for this example. If you have free tier or credits, just make sure that Bedrock is covered.

Thanks for reading, and stay tuned, because next I’ll take this further into agentic chat and explore what’s possible with Strands.

How To: Better Serverless Chat on AWS over WebSockets

Marko Djakovic — Mon, 03 Nov 2025 19:48:26 +0000

When I first wrote about building a serverless chat on AWS using AppSync Events, it was lacking two key capabilities:

two-way communication over WebSockets, and
message persistence on publish

Since then, AppSync Events API received some improvements, including these two capabilities that it initially lacked. Having tested this first hand on a simple example, my aim with this post is to share impressions of what now seems to be a way more complete service offering. Spoiler alert - the impressions are rather good, and make sure to stick around to the end as code example is included!

Note: this post is also published on my personal website: marko.dj

Before

There were some workarounds proposed at the time by AWS themselves, but mine was maybe a bit more robust, coming with the complexity trade-off. You can read the previous article here, but this is how the architecture looks like:

I made a clear separation - use AppSync Events only for subscribing, while I had a separate API for publishing. Messages were stored and streamed to AppSync, making their way to all subscribers. This isn't ideal, but ensures consistency.

After

Given the new features AWS introduced to the service, now it's possible to achieve even better performance with even simpler architecture. The new solution looks like in the below diagram:

With these improvements we remove several elements of the previous solution:

No DynamoDB Streams
No EventBridge Pipes
No separate API endpoint for publishing messages

The new improvements allow sending and receiving messages completely over WebSockets natively using AppSync Events realtime endpoint, so there is no need for workarounds anymore. We still keep the GET API endpoint for fetching existing messages, so no change regarding that. Note that this can be simplified even further by using only Lambda Function URL, but we'll be keeping the approach with API Gateway. Now, let's address what's needed to make the new solution work.

Making it work

As with both previous articles we will be using AWS CDK for the project. Perhaps the most interesting part is creating the AppSync Events API, configuring the channel namespace, and the publishing handler function. All of that is set up in this piece of CDK code:

// AppSync Event API, channel namespace, and handler
const eventApi = new EventApi(this, 'ChatAppSyncEventsApi', {
    apiName: 'ChatAppSyncEventApiWs',
});
// Add AppSync DynamoDB data source & add needed permission
const appSyncDynamoDbDataSource = eventApi.addDynamoDbDataSource('ddbsource', table);
table.grantReadWriteData(appSyncDynamoDbDataSource);
// create namespace with datasource and handler for onPublish
eventApi.addChannelNamespace('serverlesschat', {
    code: AppSyncCode.fromAsset(path.join(__dirname, '../src/appsyncjs/serverlesschat-handler.js')),
    publishHandlerConfig: {
        dataSource: appSyncDynamoDbDataSource,
    },
});

Simple enough, right? Let's break it down a bit.

Channel namespaces

Channel namespaces are something that's different than when we built this solution with IoT Core Topics. Topics are pretty loose and you can define your own topics without prior declaration or creation. While AppSync requires explicit declaration of the namespace first, the channels you can create within it are still very flexible.

Event handlers & data sources

This is the real breakthrough feature addition - more complex handlers and the ability to interact with data sources. This is what enables us to ditch the whole workaround from the previous solution that was using a separate REST API for storing messages to DynamoDB, and then streaming them to EventBridge Pipes to get them to subscribers. In the new solution, messages are published directly via WebSockets, and stored into DynamoDB using the onPublish event handler - no need for any additional streaming as subscribers get messages directly.

Two types of handlers exist - onSubscribe and onPublish, and can be used for various things, such as transformations, persistence to a data source, or even authorization. They are written in JS, using the APPSYNC_JS runtime, which might be familiar to users of AppSync already. The scope of the article is not to go deep dive into this topic, even though it very well could - for those interested in more details feel free to check out official AWS CDK docs on this.

Seeing it in action

To test the solution yourself, checkout the code and follow the README at:

https://github.com/imflamboyant/serverless-aws-chat/tree/main/chat-appsync-events-websocket

Thanks for reading! Have some ideas how you would extend the solution? Do let me know in the comments!

Serverless Chat on AWS with AppSync Events

Marko Djakovic — Mon, 20 Jan 2025 22:26:03 +0000

In my previous article I've shown how easy it can be to create a real-time chat application using AWS IoT Core. As I mention there, I'd like to explore implementing the same solution using a recently announced AppSync Event API. In this article, I will do just that. Along the way, we will discuss the similarities and the differences between the two approaches, while covering the basics of AppSync Events, architecture choices, caveats and more. Same like last time, I will include a CDK project so you can play with it on your own using a simple client to test it live.

Side note: this post is also published on my personal website - marko.dj.

AppSync Events

AppSync Events was announced in October 2024 and it really sounds exciting. I won't bother you much with repeating what is already said about the service itself, so in a nutshell, citing the AWS guide:

AWS AppSync Events lets you create secure and performant serverless WebSocket APIs that can broadcast real-time event data to millions of subscribers, without you having to manage connections or resource scaling.

I immediately wanted to try it out, but I quickly realized it is still "young" and lacks a lot of features that would be necessary for a lot of use cases. For example, currently there is no direct way to store the messages from channels anywhere. I expected that you'd be able to at least trigger a Lambda function and do something custom with the messages, like storing them to DynamoDB. Message processing is possible via namespace handlers by using onPublish and onSubscribe handlers, however since they run on AppSync's JavaScript runtime there is no possibility to do much more than on the fly processing or filtering of the payload.

In this article, I'll mainly focus on working around the lack of persistence capabilities. A great re:Invent 2024 session contains a detailed presentation of the service, addressing the drawbacks and suggesting alternative solutions. The session is very informative and also addresses the roadmap, according to which all the missing features (and more) are planned to be delivered. Now, I would like to dig in the event persistence possibilities with AppSync Events.

Events Persistence Issue

In the re:Invent session shared above, the presenter addresses the event persistence issue at around 44:35, presenting workarounds as "Advanced patterns". The workaround is basically having an API next to the AppSync Event API that will be used to persist events separately. This can look something like:

Since the side API would be there anyway for fetching previous messages, adding one more endpoint for storing messages is not a big deal. Relatively straightforward, it will work, but there's a catch. This way the client is burdened with making two requests when publishing messages - one towards the AppSync Event API to publish it to subscribers, and one to the side API that will actually persist the event to the database. If there's an issue with the API persisting the events, there will be inconsistencies between what is persisted and what the subscribers get. Since the client has to send events to two destinations and both need to succeed, handling retries in case of failures is adding complexity as well. I would rather call this an anti-pattern.

A Look at an Alternative Solution

I would like to present an alternative approach to this challenge. Besides the WebSockets endpoint, AppSync Event API has an HTTP endpoint which was highlighted as a neat feature to integrate not only frontend clients, but also backends that could publish events to it. To achieve consistency and a more robust solution, leveraging DynamoDB Streams and EventBridge Pipes can be the way to go. This way the client makes only one request when sending messages, and the subscribers always get the messages that are actually persisted. Effectively, this means that the client does not use the HTTP endpoint of AppSync Events at all, but relies on the side API for sending messages, and only listens to incoming messages via the realtime endpoint. The diagram below shows this architecture more clearly.

This is possible by leveraging some of many EventBridge capabilities. Namely, DynamoDB Streams are activated on the messages table, streaming each new message through an EventBridge Pipe to an API Destination, connected to the actual HTTP endpoint of AppSync Events.

Since I'm a big fan of CDK, and like to be on the bleeding-edge, I am relying on a few alpha modules from CDK: @aws-cdk/aws-pipes-alpha, @aws-cdk/aws-pipes-sources-alpha, @aws-cdk/aws-pipes-targets-alpha and this is what it takes to connect with DynamoDB Streams:

// EventBridge Pipe DynamoDB Stream Source
const dynamoDbStreamSource = new DynamoDBSource(table, {
    startingPosition: DynamoDBStartingPosition.LATEST,
});

// EventBridge API Destination
const appSyncEventsApiDestination = new ApiDestination(this, 'AppSyncEventsApiDestination', {
    connection: appSyncEventsApiConnection,
    endpoint: appSyncEventsApiEndpoint,
    httpMethod: HttpMethod.POST,
});

// EventBridge Pipe with API Destination Target & Input Transformation
const pipe = new Pipe(this, 'EBPipe', {
    source: dynamoDbStreamSource,
    target: new ApiDestinationTarget(appSyncEventsApiDestination, {
        inputTransformation: InputTransformation.fromObject({
            channel: 'serverlesschat/channels/' + '<$.dynamodb.NewImage.channel.S>',
            events: [
                JSON.stringify({
                    channel: '<$.dynamodb.NewImage.channel.S>',
                    timestamp: '<$.dynamodb.NewImage.timestamp.S>',
                    username: '<$.dynamodb.NewImage.username.S>',
                    message: '<$.dynamodb.NewImage.message.S>',
                })
            ],
        }),
        // api key must be sent with each message
        headerParameters: {
            'X-Api-Key': appSyncEventsApiKey,
        }
    }),
});

Input transformation is performed to accommodate DynamoDB Streams to a format required by AppSync Events endpoint:

{
  "channel": "string",
  "events": ["..."]
}

Note: events is an array of strings, hence json messages are stringified during transformation.

Pro tip: activating CloudWatch logs for the Pipe with Log execution data turned on can save you a bunch of time troubleshooting.

Trade-offs

As with any architecture decision, there are trade-offs, so let's just address them with regard to adding DynamoDB Streams and an EventBridge Pipe. To improve consistency and simplify client communication, the speed of message delivery is reduced. What does this mean? There is a slight latency noticed in message delivery to subscribed clients. This is not surprising since the message goes
through the API Gateway and a Lambda function, to a DynamoDB table and from there it is streamed to the AppSync Event API via an EventBridge Pipe, which is basically another HTTP call. So, a lot of stuff happening in a few dozen lines of code. I just can't help but notice that the IoT Core solution is slightly snappier overall. This will of course be solved when AppSync Events gets two-way WebSocket communication.

Authorization

A few words on authorization. The easiest way to get started with AppSync Events is to use API Key auth, and that's exactly how this project is configured. Other modes are supported as well, pretty standard for AWS: Cognito User Pool, IAM, OIDC, and custom AWS Lambda. These choices offer great flexibility for various authorization needs, especially given the fact that it can be configured per namespace.

Authorization with EventBridge API Destination

As seen in the above CDK code snipped, an API Destination is configured with a Connection to support authorization when sending messages towards the AppSync Event HTTP API endpoint. Additionally, the destination target configured in EventBridge Pipe is required to have X-Api-Key header sent with each request for the messages to get through. What I noticed is that once this header is missing, or the value is incorrect - the Connection becomes Deauthorized automatically and the whole Pipe stops working as a result. I haven't found a way to fix that, but just to destroy and deploy the stack again. I am not sure what it takes to authorize the API Destination again, but it was rather inconvenient from developer experience standpoint.

Demo & Code

Similar as in the last article, I created a simple HTML client powered by some JavaScript to test & demo this solution. Generally, examples on configuring the client use Amplify, which is a neat library when building with SPA frameworks. I opted for a plain HTML page which uses native browser fetch and websocket APIs, without any external dependencies. You don't have to worry about the technicalities if you just want to run the example yourself, however if you'd like to dive deeper on how to connect to AppSync Events without Amplify, feel free to consult Understanding the Event API WebSocket protocol AWS guide and my code.

https://github.com/imflamboyant/serverless-aws-chat

AWS Deployment and usage instructions are provided in the repository's README file.

Conclusions

In this article I have shared my experience in building a familiar real-time chat solution, this time with a brand-new AWS offering - AppSync Events. It does feel similar to IoT Core, mostly due to the way subscriptions are organized with namespaces/channels, but I must admit it gives a more "native" developer experience.

The roadmap shared at re:Invent 2024 sounds promising, stating that we can expect features like two-way websocket communication, data persistence options, and Lambda handlers, among others. When all that gets released, I truly believe this will be one of the most important AWS services for building modern applications.

Building a Cloud Native Serverless Chat On AWS

Marko Djakovic — Mon, 23 Dec 2024 11:43:56 +0000

I've wanted to write about this for over a year probably, but I kept putting it off for various reasons. I built a real-time chat on AWS by (ab)using the IoT Core service, and I've been eager to share my experience. If you'd prefer, you can read this post also on my personal website - marko.dj. Now, back to the topic. Recently, AWS announced AppSync Events and my immediate thought was, "wow, I can replace the IoT Core solution with this!"

...a new solution for building secure and performant serverless WebSocket APIs to power real-time web and mobile experiences at any scale.

Not so fast... It turns out the initial version of AppSync Events has some significant limitations. For starters, there's no way to persist data by integrating with other AWS services. Persistence, along with many other features, is said to be on the roadmap for next year. I'm eagerly awaiting these updates, but until then, let me walk you through how to build real-time apps on AWS using this IoT Core based approach. It's a surprisingly straightforward method that leverages AWS serverless capabilities effectively.

I'll present a cloud architecture for a real-time chat application, accompanied by useful code snippets and infrastructure written in CDK, ready to be deployed and tested in your AWS environment. Along the way, I'll explain the design choices, discuss the options available, and also highlight some caveats of this approach.

Architecture

We are building a simple group chat that will have some pre-defined channels that users can subscribe to and send and receive messages. The application will use IoT Core Topics as the backbone for real-time messaging, with a few additional AWS serverless services to support things like data persistence and authorization. Apart from the real-time segment, the app will have a supporting REST API to fetch existing messages per channel.

This architecture allows us to publish messages and subscribe to topics directly on the IoT Core service. Published messages will automatically be stored in a DynamoDB table and subscribed clients will receive them in real time. As you can see on the diagram, there is also a Lambda authorizer which is needed to allow actions on IoT Core. For fetching previous messages from the database upon connecting, clients will use the get messages API that queries the DynamoDB table fetching messages from a requested channel. I won't go into technicalities of this API as it's not the main focus of the article, and it is a pretty standard serverless API.

Chat message payload looks quite simple as well:

{
  "channel": "general",
  "timestamp": "2018-11-07T00:25:00.073UTC",
  "username": "johndoe",
  "message": "hello world"
}

Before diving into the implementation, let's cover some basic concepts important for understanding the solution.

AWS IoT Core

AWS IoT Core is a pretty powerful and feature-rich service, and the depths of its possibilities are beyond the scope of this article. What's important for us at the moment is to understand that it's essentially an MQTT broker which enables real time communication between connected applications, using Topics, Rules and Actions. In all AWS accounts there is an endpoint in each region that can be used to connect to the broker, and subscribe and publish messages to topics.

Topics, Rules & Actions

Creating a Topic isn't necessary as it is just a logical resource specified as a plain string at runtime. Usually, topic names are divided by slashes into logical parts, for example chat/general. One of the standout features of IoT Core are Rules, which allows you to filter, transform, and route messages to other AWS services through Actions. For instance, you can set up a rule to listen for messages published to a topic, and then trigger an action to store the message in DynamoDB or invoke a Lambda function. Or do a plethora of other things listed under AWS IoT rule actions. I believe this service is often overlooked partly because of its name, and partly because of its apparent complexity, but it can definitely be effective outside of IoT realm. Let's suppose that our application publishes messages to chat/general topic. A rule in CDK can be creatred to automatically store them in a DynamoDB table:

new TopicRule(this, 'SendMessagesToDynamoDB', {
  topicRuleName: 'chat_ddb_rule',
  sql: IotSql.fromStringAsVer20160323("SELECT * FROM 'chat/general'"),
  actions: [
    new DynamoDBv2PutItemAction(tableName),
  ]
});

The code snippet above creates a topic rule, which uses an SQL-like syntax for filtering data from the topic, and defines a DynamoDB PutItem action. The result of the action is a message being stored in the given DynamoDB table. One of the conditions is to have the table key names match the field names in the message payload. If not, the SQL statement could also transform the field names, for example SELECT username AS userId, in case the table key is userId. To go in depths of this very powerful feature, full spec is available at AWS IoT SQL reference.

Authorization

The IoT Core regional endpoint is readily available, and after setting up a topic rule, you might expect to be ready to start publishing messages. Not quite yet. By default, no clients are permitted to connect to the broker. IoT Core provides robust and flexible authorization methods to manage access to its resources. In this example, we use a custom Lambda authorizer, allowing you to implement any custom logic to secure your application. The only condition is that authorizer Lambda's response must include a valid policy object. While there are numerous options for authorization, this article focuses on a simplified approach. For a deeper dive into the available options, refer to the IoT Core Authorization Guide.

To keep things straightforward, this example uses a highly permissive policy that grants full access to the client application by default. The goal is to demonstrate real-time communication and help you quickly test the setup. However, this permissive policy should never be used in a production environment. The Lambda authorizer's response is an IoTCustomAuthorizerResult object, which contains the following policy document:

[
  {
    Version: '2012-10-17',
    Statement: [
      {
        Action: 'iot:Connect',
        Effect: 'Allow',
        Resource: `arn:aws:iot:${region}:${accountId}:client/User*`
      },
      {
        Action: 'iot:Subscribe',
        Effect: 'Allow',
        Resource: `arn:aws:iot:${region}:${accountId}:topicfilter/serverlesschat/channels/*`
      },
      {
        Action: ['iot:Receive', 'iot:Publish'],
        Effect: 'Allow',
        Resource: `arn:aws:iot:${region}:${accountId}:topic/serverlesschat/channels/*`
      }
    ]
  }
]

The policy document consists of three statements:

allowing a clients with ids starting with User* to connect to the broker
allowing subscriptions to all topics under serverlesschat/channels/*
allowing publishing and receiving of messages on the above topics

Connecting by using an MQTT client with an appropriate client id is then quite simple:

mqtt.connect(brokerUrl, { clientId: 'User123' })

Caveats

There are some caveats regarding the authorizer that proved they could be tricky while developing this solution, so I'd like to highlight them specifically.

Caveat 1: Granting the IoT Core service permission to invoke the Lambda authorizer function is essential for it to work. In CDK it can be as easy as:
authLambda.grantInvoke(new ServicePrincipal('iot.amazonaws.com'))

This small detail can be a real pain to troubleshoot, because there are no visible errors or logs, the clients just get rejected. So make sure it's set in case you face connection issues.

Caveat 2: As mentioned earlier, the authorizer response should be of IoTCustomAuthorizerResult type. However, for an unknown reason it didn't work, the policy just wasn't applied. Only when I changed the response to a plain string, and stringified the result did the policy actually apply. Refer to this piece of code in src/authorizer-handler.ts.
return JSON.stringify(response);

I am still not quite sure why, but I remember having similar issues with this in Java as well.

If all this so far feels a bit blurry, don’t worry! The complete code for the authorizer, along with the CDK setup and an example client, is available in the GitHub repository linked in the next section.

Show Me the Code

To demonstrate the chat functionality, I’ve created a simple HTML client powered by some JavaScript. This basic client connects to the IoT Core endpoint, enables publishing and subscribing to channels, and retrieves any persisted messages from the REST API for the selected channel. Each client instance gets a random username in the format of Userxyz, where xyz is a random three-digit number. You can find it, along with the CDK code, in the GitHub repository below. Clone it, deploy it to your AWS account, and start testing right away.

https://github.com/imflamboyant/serverless-aws-chat

Deployment and usage instructions are provided in the repository's README file.

Of course, you're not limited to this example. You can create your own clients using any technology, such as SPA frameworks or mobile applications.

Conclusions

In this article I’ve demonstrated how easy it can be to create a real-time chat application on AWS. I have not covered pricing, but given the cheap combined costs of all the involved AWS services, you can process hundreds of thousands of messages before this solution costs you a few dollars.

What else I like about it is that the extension possibilities are huge. You could extend the API to also manage users and channels, integrate the authorizer with your preferred Identity Provider using JWKS and validate JWTs, or enhance security even more by adjusting the authorizer to check user permissions before allowing subscriptions to specific channels. The chat itself could also be enriched with features like message reactions, threads, and image sharing, or even GenAI capabilities by integrating with Bedrock.

This architecture lays a solid foundation not only for chat systems but for any real-time communication features you might envision.

In the next article, I’ll migrate this solution to AppSync Events, explore options to handle the lack of data persistence, and compare its features with those of IoT Core.

Thank you for reading, and stay tuned for the next article!

Power Chat Messages Search with DynamoDB & Amazon OpenSearch

Marko Djakovic — Mon, 29 Jan 2024 09:08:18 +0000

Intro

We have built a group chat application that stores chat messages in Amazon DynamoDB and it is running for a few months now. As the application grew, users wanted to be able to search through historical chat messages. To enable this, we will integrate DynamoDB with Amazon OpenSearch Service for full-text search capabilities. There are multiple ways of dealing with this, but I would like to experiment with something new. During the 2023 re:Invent AWS announced general availability of DynamoDB zero-ETL integration with Amazon OpenSearch service. In this article I will give it try on our group chat full-text search use case and share my insights and experience.

Side note: the post is also published on my personal website - marko.dj.

Current situation

Our application already leverages DynamoDB for fast and scalable storage and retrieval of chat messages via an API. The API consists of an API Gateway and a few Lambda functions that interact with a DynamoDB table. That looks something like in the diagram below.

This is the already standard, boring setup for a Serverless API on AWS 🙃 DynamoDB is not well suited for the full-text search use case. Let's see what needs to be added to achieve this.

Bridge the gap towards full-text search

To facilitate the full-text search over chat messages we will add Amazon OpenSearch to our architecture. Now, how do we approach syncing the messages data from DynamoDB to OpenSearch? The first thing that comes to mind it of course enabling DynamoDB Streams and having a Lambda function handle them and send the data to OpenSearch. However, we won't do that. We will leverage the zero-ETL integration between the two services. So, our architecture changes to something like:

Setting up Amazon OpenSearch

Amazon OpenSearch is a managed service and that means the underlying infrastructure is handled for us. We can set up an OpenSearch Domain, which means we specify the instance size and count, and it will be up and running after a short while. The second choice is to use the Serverless option, which is out of scope for this article. The important thing is that both options support zero-ETL sync from DynamoDB.

For this example, I provisioned a cluster with only 1 node of type t3.small.search. This is of course not production grade, but more than enough for this small PoC.

Data ingestion to OpenSearch

Having the OpenSearch cluster up and running is a great first step, but now we need to achieve two things:

Import all existing messages from DynamoDB
Set up constant sync of all future messages

Luckily, Amazon OpenSearch Ingestion pipeline can take care of both. How this works is the Pipeline initially exports all the table data to an existing S3 bucket of your choice, imports the data from there into OpenSearch, and then continues streaming all new changes in the table towards OpenSearch.

Before creating the Pipeline, some pre-conditions on the DynamoDB table need to be fulfilled:

Point-in-time-recovery (PITR) is enabled
DynamoDB Streams are enabled

Besides that, we need an S3 bucket that will be used for the data export and import. I created this bucket upfront.

And finally, we need an IAM Role that the Pipeline will assume to carry out all the work. Accordingly, the Role Policy must allow all actions that the Pipeline needs to perform on these resources.

IAM Role Policy for the Pipeline

For convenience, I am providing the Policy below:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "allowRunExportJob",
            "Effect": "Allow",
            "Action": [
                "dynamodb:DescribeTable",
                "dynamodb:DescribeContinuousBackups",
                "dynamodb:ExportTableToPointInTime"
            ],
            "Resource": [
                "arn:aws:dynamodb:{region}:{aws-account-no}:table/{table-name}"
            ]
        },
        {
            "Sid": "allowCheckExportjob",
            "Effect": "Allow",
            "Action": [
                "dynamodb:DescribeExport"
            ],
            "Resource": [
                "arn:aws:dynamodb:{region}:{aws-account-no}:table/{table-name}/export/*"
            ]
        },
        {
            "Sid": "allowReadFromStream",
            "Effect": "Allow",
            "Action": [
                "dynamodb:DescribeStream",
                "dynamodb:GetRecords",
                "dynamodb:GetShardIterator"
            ],
            "Resource": [
                "arn:aws:dynamodb:{region}:{aws-account-no}:table/{table-name}/stream/*"
            ]
        },
        {
            "Sid": "allowReadAndWriteToS3ForExport",
            "Effect": "Allow",
            "Action": [
                "s3:GetObject",
                "s3:AbortMultipartUpload",
                "s3:PutObject",
                "s3:PutObjectAcl"
            ],
            "Resource": [
                "arn:aws:s3:::{bucket-name}/*"
            ]
        },
        {
            "Sid": "allowDescribeDomain",
            "Effect": "Allow",
            "Action": "es:DescribeDomain",
            "Resource": "arn:aws:es:*:{aws-account-no}:domain/*"
        },
        {
            "Sid": "allowESHttp",
            "Effect": "Allow",
            "Action": "es:ESHttp*",
            "Resource": "{opensearch-domain-arn}"
        }
    ]
}

The role needs to have a Trust relationship with osis-pipelines.amazonaws.com. Once we make sure all of the above is set, we can proceed with creating the Pipeline.

Creating the Ingestion Pipeline

There are several options you need to pick and choose when creating a Pipeline. I'll focus only on a sub-set which proved useful to me for this example. After choosing the name, there is the Pipeline capacity configuration. It is measured in units called Ingestion-OCU and it is billed hourly while the Pipeline is Active, regardless of whether it is actually processing data. The Ingestion Pipeline billing resmbles the Serverless option of OpenSearch Service. I just left the minimum and maximum values to 1 and 4, as it was provided by default.

The Pipeline configuration is a YAML file with a lot of options, but luckily there are ready made blueprints for common use cases. I chose the DynamoDbChangeDataCapturePipeline blueprint, which gives a nice template with some sensible defaults for this use case. The configuration ends up looking like the one below. I intentionally left the comments from the blueprint and added some of my own.

###
  # dynamodb-pipeline:
  # This pipeline is capable of exporting a DynamoDB table to OpenSearch, and processing DynamoDB streams to
  # keep the DynamoDB table and OpenSearch cluster in sync for create, update, and delete Events.
###

version: "2"
dynamodb-pipeline:
  source:
    dynamodb:
      acknowledgments: true
      tables:
        # REQUIRED: Supply the DynamoDB table ARN and whether export or stream processing is needed, or both
        - table_arn: "{table-arn}"
          # Remove the stream block if only export is needed
          stream:
            start_position: "LATEST"
          # Remove the export block if only stream is needed
          export:
            # REQUIRED for export: Specify the name of an existing S3 bucket for DynamoDB to write export data files to
            s3_bucket: "{bucket-name}"
            # Specify the region of the S3 bucket
            s3_region: "{region}"
            # Optionally set the name of a prefix that DynamoDB export data files are written to in the bucket.
            s3_prefix: "ddb-to-opensearch-export/"
      aws:
         # REQUIRED: Provide the role to assume that has the necessary permissions to DynamoDB, OpenSearch, and S3.
         sts_role_arn: "{role_arn}"
         # Provide the region to use for aws credentials
         region: "{region}"
  sink:
    - opensearch:
        # REQUIRED: Provide an AWS OpenSearch endpoint
        hosts: [ "{opensearch_endpoint}" ]
        index: "table-index" # Define the name of the index for this data
        index_type: custom
        document_id: "${getMetadata(\"primary_key\")}"
        action: "${getMetadata(\"opensearch_action\")}"
        document_version: "${getMetadata(\"document_version\")}"
        document_version_type: "external"
        aws:
          # REQUIRED: Provide a Role ARN with access to the domain. This role should have a trust relationship with osis-pipelines.amazonaws.com
          sts_role_arn: "{role_arn}" # This needs to be the same role as defined in the previous section
          # Provide the region of the domain.
          region: "{region}"

Apart from the network configuration, ensuring that the Pipeline logs are sent to CloudWatch proved to be very useful for me, especially during the setup, as I wasn't able to get this working immediately. This was mostly due to misconfiguring the Role Policy. While there are many more options for high-availability and durability, those are out of scope for this discussion.

So, that should be it. Once the Pipeline is created, it should be started, and in a few minutes, if everything is configured properly, you should see the data appear in the designated S3 bucket for export, as well as in the OpenSearch Index, of course. As I already mentioned, CloudWatch logs were a big time-saver when something went wrong.

Showcase

As I mentioned at the start, I already had a table full of chat messages with which I tested this setup. For the sake of testing, I am providing this gist which contains CSV representation of an AI generated group chat simulation about a trip to Hawaii 🌴 Feel free to use this for practicing and load it into a DynamoDB table.

After the pipeline runs the initial import, I am able to search all the data very quickly. Also, each new message ends up in OpenSearch within seconds. To test the search, let's say that I want to find out if the trip is confirmed and hopefully get more info on that. By issuing a very simple GET _search/q=confirmed request in OpenSearch Dashboards DevTools, the result can be seen in the screenshot below.

We see that there is one hit and the message contains just what we are looking for. Also, please notice that the _id field contains the primary key of our DynamoDB table, which consists of a group chat id and a message timestamp. These mappings are performed automatically for us.

Conclusions

This article shows that by leveraging the DynamoDB to OpenSearch zero-ETL integration, we can quickly and efficiently search for messages based on keywords without the need for complex infrastructure management. This goes for both the cluster and Serverless variants. The integration also allows us to scale our search solution as our application grows. Overall, Amazon DynamoDB and Amazon OpenSearch are a powerful combination that can help organizations build fast and efficient search solutions for their applications.

Nevertheless, I believe that the scale of data for both ingestion and search needs to be substantial to warrant the incorporation of OpenSearch into a solution of this nature. Considering that search is not the primary feature of our application, the associated costs do not justify its implementation. Given our architecture, operating the smallest OpenSearch instance along with ingestion for a weekend incurs expenses equivalent to running the entire application for an entire month. OpenSearch Serverless, despite being even more costly, is still unnecessary for us since our current workload is far from reaching the scale that would warrant such a solution. As mentioned earlier, the primary objective of this article is to delve into the new feature of zero-ETL integration. It is likely that we will opt for an alternative solution for search that aligns better with our smaller scale and budget.

Single Sign-On with Azure AD and Amazon Cognito using OIDC and AWS Amplify

Marko Djakovic — Wed, 17 May 2023 08:17:11 +0000

Single sign-on (SSO) is often the preferred way of accessing applications as it relieves users from the burden of having to remember yet another, probably insecure password. In my latest project, it was very convenient to have SSO as all our users are using Microsoft accounts. We implemented identity federation between Azure AD and our Cognito User Pool using OpenID Connect. This post will quickly show how to do it according to best practices using the amplify-js library, while also including CloudFormation templates for the User Pool and accompanying resources.

My assumptions about the readers of this post:

You have your own Azure Active Directory running and are able to configure an App Registration
You have an AWS account and aws-cli configured
You already have or can run a simple front-end project in any of the popular front-end frameworks

Side note: you can read this also on my personal website - marko.dj - Single Sign-On with Azure AD and Amazon Cognito using OIDC and AWS Amplify

Setting up an App Registration in Azure AD

In the Azure Portal, we need to create a new App Registration that represents our application and will be used as an OIDC provider with Cognito. After creating it, we will also need to generate a new secret under the "Certificates & secrets" section. Once generated, copy the secret and securely store it in the Systems Manager Parameter Store in your AWS account. Do the same with the client id, which is located in the Overview page of your App Registration under the label "Application (client) ID". We will reference it together with the client id in the CloudFormation template in the next section, so make sure to give these two parameters some meaningful names, for example my-azure-clientid and my-azure-clientsecret.

Setting up Amazon Cognito

All of the required AWS resources will be defined in a single CloudFormation stack, e.g. in a file named cognito.yml. First, we define a User Pool and a User Pool Domain:

Resources:
  MyUserPool:
    Type: AWS::Cognito::UserPool
    Properties:
      UserPoolName: my-user-pool
  MyUserPoolDomain:
    Type: AWS::Cognito::UserPoolDomain
    Properties:
      Domain: my-user-pool # domain name
      UserPoolId: !Ref MyUserPool

The User Pool Domain will be referenced by Azure AD during the authentication flow. When deployed, the domain will receive a value similar to https://my-user-pool.auth.{region}.amazoncognito.com.

After that, we add an OIDC User Pool Identity Provider and a corresponding User Pool Client in the cognito.yml:

MyUserPoolIdentityProvider:
  Type: AWS::Cognito::UserPoolIdentityProvider
  Properties:
    UserPoolId: !Ref MyUserPool
    ProviderName: "MyCompany" # your custom name
    ProviderType: "OIDC"
    ProviderDetails:
      client_id: ${ssm:my-azure-clientid}
      client_secret: ${ssm:my-azure-clientsecret}
      attributes_request_method: 'GET'
      oidc_issuer: 'https://login.microsoftonline.com/{tenant_id}/v2.0'
      authorize_scopes: 'email profile openid'
    AttributeMapping:
      name: 'name'
      preferred_username: 'preferred_username'
      given_name: 'unique_name'
      email: 'email'
      username: 'sub'

Pay attention to the oidc_issuer field below - the tenant_id should be swapped with the real value under the label "Directory (tenant) ID" on the App Registration's Overview page. The AttributeMapping section specifies how to map certain attributes from the authenticated user from AD to the Cognito User Pool. That way, we automatically capture the user's name and email address, for example.

The final component to make this come together is a User Pool Client, specified below:

MyUserPoolClient:
  Type: AWS::Cognito::UserPoolClient
  DependsOn: MyUserPoolIdentityProvider
  Properties:
    AllowedOAuthFlows:
      - code
    AllowedOAuthFlowsUserPoolClient: True
    AllowedOAuthScopes:
      - email
      - openid
      - profile
    CallbackURLs:
      - 'http://localhost:4200'
    LogoutURLs:
      - 'http://localhost:4200'
    ClientName: my-user-pool-client
    PreventUserExistenceErrors: "ENABLED"
    SupportedIdentityProviders:
      - MyCompany # Refer to the OIDC provider name
    UserPoolId: !Ref MyUserPool

This approach uses the authorization_code flow, which is defined under AllowedOAuthFlows. You probably notice that CallbackURLs and LogoutURLs point to localhost, and by the port value, you can guess that I have an Angular single-page application to test this locally. These two values contain the addresses where the user will be redirected after a successful authentication and logout, respectively. If your front-end app is deployed somewhere, these values should reflect that remote address.

Deploy this template using aws-cli using the command below:

aws cloudformation deploy --stack-name my-cognito --template-file cognito.yml

After successful deployment, there is still one more thing to configure in the Azure Portal before the whole flow works. Remember the Cognito User Pool Domain? That is the final piece that will make the AD "see" our User Pool.

That is configured in the App Registration under Authentication. When clicking the "Add platform" button, a side sheet will open with several options of web and mobile applications to choose from. Even though we are creating a single-page application, the correct choice here is "Web". That is because the AD is actually interacting with Cognito, not with our single-page app directly. After picking the Web platform, in the Redirect URIs input field we paste our User Pool Domain, appended with "/oauth2/idpresponse" so it looks like https://my-user-pool.auth.{region}.amazoncognito.com/oauth2/idpresponse.

Using @aws-amplify/auth library

Why do I think you should use this library? Firstly, it performs all the communication with the Cognito API. More importantly, it does so using security best current practices, like using authorization code with PKCE.

As I mentioned, I used Angular to test this, but any other framework should have it in a similar way. After adding @aws-amplify/auth and @aws-amplify/core dependencies to the project, Amplify Auth should be configured:

Amplify.configure({
  Auth: {
    region: '',
    userPoolId: '',
    userPoolWebClientId: '',
    oauth: {
      domain: '',
      scope: ['openid', 'email', 'profile'],
      redirectSignIn: 'http://localhost:4200',
      redirectSignOut: 'http://localhost:4200',
      responseType: 'code',
    },
  },
});

The properties userPoolId, userPoolWebClientId, and domain can be found in the AWS Console under the Cognito deployment, and region is the AWS region where the stack is deployed. On your app's login page, a button to initiate the login flow should call the following method:

Auth.federatedSignIn({ customProvider: 'MyProvider' });

The user will be redirected to log in with the provider (Microsoft), after which they will be redirected back to the app where Amplify Auth will automatically exchange the given authorization code for tokens from Cognito. All of this is done securely and according to the best practices.

To get the currently logged-in user, a call to the Auth.currentAuthenticatedUser() method should do the job.

Conclusion

By following the steps outlined in this post, you can set up the necessary AWS resources and Azure configurations to try out SSO using the amplify-js library. It's worth noting an interesting observation: Cognito pricing for users logging in with SAML or OIDC is almost 3 times higher compared to regular users. Also, the free tier is capped at only 50 OIDC monthly active users, compared to 50k regular monthly active users. For example, if you are out of the free tier, and have 1000 monthly active users using OIDC, that would cost $15.

I hope you found this guide helpful and informative. If you have any questions or would like to share your experiences with implementing SSO using Cognito, I invite you to leave your comments below. I look forward to engaging with you and addressing any inquiries you may have.

How AWS and Serverless Revived Our MVP Development Journey

Marko Djakovic — Sun, 23 Apr 2023 23:10:01 +0000

Introduction

A team of talented individuals embarks on a journey to build something cool for the rest of their colleagues to help them in day to day life at the office. This bright idea of a super powerful, social-network-like app for company employees emerged. User flows are quickly sketched out, dozens of features planned, technical diagrams are drawn, we had it all figured out and were super motivated to start. On top of that, we'll build it and host it ourselves on some VMs on premise!

You may have guessed - it didn't see the light of day.

In this blog post, I'll share why this approach failed and how, in a second attempt and by adopting AWS and Serverless, we successfully developed and deployed a functioning MVP on a limited time and budget. You can also read it on my personal website - marko.dj - How AWS and Serverless Revived Our MVP Development Journey

The First Attempt: A Lesson in Complexity

We have our architecture laid out and it's time to make those diagrams a reality. We're going to deploy Kubernetes to our existing servers with Apache Kafka as the backbone for the system. The app is going to be packed with features so it needs to be powerful, resilient and future-proof and what not.
Setting up the Kubernetes cluster required significant effort, even with the help of our seasoned ops colleagues, but eventually it was up and running on 5 nodes, managed in house. Next, configure Kafka and integrate it with several microservices that are being developed in the meantime - also not a walk in the park. Neither was setting up self-managed GitLab Runners for deploying the microservices.

I could go on with a few more things, but I hope you see where this is going. Not a single feature was really finished! And when I say finished, I don't mean works perfectly or has no bugs, I mean just deployed and working OK. And we had a team with plenty of expertise, so the reason definitely wasn't lack of talent.

Managing all the infrastructure components took a big bite of our time and diverted our attention from building core features of our product.

Soon enough, people had to focus on different priorities within the company, so the team disassembled and the project was abandoned.

A New Approach: Embracing Serverless and AWS

Some (a lot of) time passed, our company adopted the cloud, everybody gained more experience with it along the way, especially on AWS. Coincidentally, a part of the original team was working together again and, determined to learn from our past mistakes, our idea was about to get a fresh start. The goal now is to get a usable app in the hands of first users as soon as possible. After a few UI and cloud architecture sketches, we were ready to start. We took out one feature from the original, full-blown app idea, and worked out an MVP scope for it. It was basically a web application communicating with a REST API and a WebSocket to support real time group chat. Laser focused on our goal, relying heavily on AWS managed services, the first deployment happened within a week or so. To give you an idea of our starting landscape, take a look at the diagram below.

One piece is missing and that's IoT Core, but I just wanted to use this new set of AWS icons for Excalidraw so badly.

Trimming Down Features: The Importance of Scope Control

While cloud and Serverless enabled us to be productive already in the first week, they will not protect you from scope-creep! Everybody from the team had brilliant ideas on what to implement next. Arguing how "adding just this small thing would be great for this and that", or slipping into a trap of "this will take just 2 additional hours and bring more value". Fiercely cutting down on scope expansion was instrumental in keeping the team focused on the main goal - and that is serving the first group of users as soon as possible.

This was especially important as we knew our time to work on this project is quite limited - new assignments in the company were looming around the corner.

User Feedback Or Where Are The Emojis?

We are building something for our colleagues, for internal use, basically solving our own problems, and that gives us good insight in which features we need to build first, right? Surprisingly, the feedback from every single person from our first group of pilot users was that the chat needed to be richer, more precisely to support emojis and emoji reactions on messages. On the other hand, people are really used to rich chat features, since they have them in every actual chat app. Even if it wasn't considered crucial from our point of view, we realized that missing this could lower the chances of anyone using our app. It was the first thing on our backlog the following week.

So, we missed an important feature, but the fast feedback enabled by our cloud powered development setup makes it easy for us to quickly iterate and add the missing value.

Next Steps: Preparing for Production

We are still working on a few quick wins that our first group of users suggested, while aiming to tighten a few more knots before we open the app to all users. Currently, it's deployable and we believe it is functionally OK, so next step is validating that it's secure, reliable and performant.
Security will be checked by penetration testing and to make sure the app is reliable and that we spot any performance bottlenecks quickly, we've setup CloudWatch monitoring dashboards and alarms. Having an overview of metrics like number of API requests, latency, error count, and alarms on any server error enables us to react pro-actively. This would've been much harder if we were running this on-prem.

Conclusion

This story is an example of how AWS and its Serverless services can save time and resources when building under constraints. Apart from that, it touches the subject of scope-creep and the importance of keeping it under control when building an MVP. Also, discusses the steps to take after, like ramping up security and considering ways to measure the general quality and success of the product as it evolves.

Not to get ahead of ourselves - we still have some work to do before we can think about measuring if the product is successful. However, for things like basic monitoring, alarming and automation - AWS makes this ridiculously easy to do that it's a no-brainer.

Deploying a Spring Boot API to AWS with Serverless and Lambda SnapStart

Marko Djakovic — Tue, 31 Jan 2023 16:27:15 +0000

In my last two blog posts I wrote about NestJS Mono-Lambda and how we dealt with cold starts. Cold starts are not that big of an issue when running NodeJS Lambdas, but with Java the numbers are unacceptable if you're building synchronous, user-facing APIs. At re:Invent 2022 AWS announced SnapStart, a feature for Lambda using Java runtime. I always liked the simplicity of deployment to the cloud that Serverless Framework gives, combined with the developer experience of frameworks like NestJS and SpringBoot. After years of developing TypeScript applications on AWS Lambda, I wanted to go back to my first love, Spring Framework, to build an API and test SnapStart in the process.

This post (also available on my personal website marko.dj) will cover how to create an API that returns the closest Starbucks locations near the given point and radius. The API will be backed by MongoDB to utilize geospatial queries, with a Spring Boot app wrapped into a Mono-Lambda on top, deployed seamlessly to AWS with Serverless Framework. When it's all up in the cloud and working, we will take a closer look at the performance and try to optimize it further by utilizing AWS Lambda SnapStart.

Pre-requisites:

Your own AWS account (this exercise fits in the free-tier)
Serverless Framework installed and configured

Let's get to it.

Setup MongoDB Atlas

I signed up for a free MongoDB Atlas account and launched a free cluster in AWS Frankfurt region. To add the test data to the database, I found a collection of US Starbucks locations and forked it to support GeoJSON objects for position. This was needed because I want to index the items in order to be able to perform efficient geospatial queries that Mongo offers. Next, I created a 2dsphere index on the position field. To test if index works, we can execute the following query on MongoDB:

{
    position: {
        $nearSphere: {
            type: "Point",
            coordinates: [ -73.9655834, 40.7825547 ] // Central Park
        },
        $maxDistance: 1000 // 1km
    }
}

It should return some results near Central Park. To read more about geospatial queries in MongoDB, feel free to refer to the docs. Now our data is ready, let's proceed with creating our Spring Boot application that will query Starbucks locations.

Note: For the purpose of this exercise make sure that in the MongoDB Atlas Security section, Network Access IP Access List contains the 0.0.0.0/0 rule. That will make the database publicly accessible. None of this is a good idea to do in production, including usage of this free, shared-tier Mongo cluster. More secure way is the use VPC peering, however that is beyond the scope of this article, and it is also not included in the shared-tier MongoDB Atlas cluster. 💰

Spring Boot API

To quickly get up and running with development, let's use Spring Initializr. After selecting Java 11, Maven, Spring Boot 2.7.8 project properties, and picking Spring Web and Spring Data MongoDB dependencies, we get a project that is ready for further development. Now, let's create the usual three-layered Repository, Service, Controller setup. We'll start by creating a StarbucksDocument that represents one MongoDB document:

import org.springframework.data.annotation.Id;
import org.springframework.data.mongodb.core.geo.GeoJsonPoint;
import org.springframework.data.mongodb.core.mapping.Document;

@Document("starbucks")
public class StarbucksDocument {
    @Id
    private String id;
    private String name;
    private String address;
    private GeoJsonPoint position;

    // getters and setters...
}

Leveraging Spring Data MongoDB module, we can create a repository by just extending MongoRepository interface. We get the standard CRUD operations out of the box, so we just need to add a method for querying by position. This way we can fetch stores that are within a given distance from the passed in position, page by page.

import org.springframework.data.domain.Page;
import org.springframework.data.domain.Pageable;
import org.springframework.data.geo.Distance;
import org.springframework.data.mongodb.core.geo.GeoJsonPoint;
import org.springframework.data.mongodb.repository.MongoRepository;

public interface StarbucksRepository extends MongoRepository<StarbucksDocument, String> {
    Page<StarbucksDocument> findByPositionNear(GeoJsonPoint position, Distance distance, Pageable pageable);
}

The repository layer is set up, and we will create a StarbucksService and a StarbucksController so that our API looks like this:

GET /starbucks/{id}
GET /starbucks?lng=-73.9655834&lat=40.7825547&distance=2&page=x

If you build and test this, it wouldn't work of course, because of the missing database connection configuration in application.properties file:

spring.data.mongodb.uri=mongodb+srv://username:password@clusterxxx.xyz.mongodb.net/dbname?retryWrites=true&w=majority

Make sure to swap the values with your own, obtained from MongoDB Atlas.

🚀 Run it, test it

./mvnw spring-boot:run

curl 'http://localhost:8080/starbucks?lng=-73.9655834&lat=40.7825547&distance=1' | json_pp

{
  "data": [
    {
      "address": "86th & Columbus_540 Columbus Avenue_New York, New York 10024_(212) 496-4139",
      "id": "63d52640712420c4e81c9a20",
      "latitude": 40.78646447,
      "longitude": -73.97215027,
      "name": "Starbucks - NY - New York [W]  06186"
    },
    {
      "address": "81st & Columbus_444 Columbus Avenue_New York, New York 10024",
      "id": "63d52640712420c4e81c9a26",
      "latitude": 40.78335323,
      "longitude": -73.97441845,
      "name": "Starbucks - NY - New York [W]  06192"
    },
    {
      "address": "87th & Lexington_120 EAST 87TH ST_New York, New York 10128",
      "id": "63d52640712420c4e81c9a29",
      "latitude": 40.78052553,
      "longitude": -73.95603158,
      "name": "Starbucks - NY - New York [W]  06195"
    },
    {
      "address": "Lexington & 85th_1261 Lexington Avenue_New York, New York 10026",
      "id": "63d52640712420c4e81c9a41",
      "latitude": 40.778801,
      "longitude": -73.956099,
      "name": "Starbucks - NY - New York [W]  06219"
    }
  ],
  "total": 4
}

Apparently we have 4 coffee shops within 1 kilometre of Central Park. Nice. ☕️

How to deploy it to AWS?

A few more steps are needed for deploying our API to AWS with the Serverless Framework.

Maven dependency

Add the aws-serverless-java-container-springboot2 dependency to pom.xml:

<dependency>
    <groupId>com.amazonaws.serverless</groupId>
    <artifactId>aws-serverless-java-container-springboot2</artifactId>
    <version>1.9.1</version>
</dependency>

Maven profiles

For convenience, create two Maven profiles - local and shaded-jar. This makes it easier to separate running the app locally and packaging the jar for AWS Lambda. The shaded-jar profile contains the spring-boot-starter-web dependency but without spring-boot-starter-tomcat. To package the app for AWS Lambda, the maven-shade-plugin is used to package all the dependencies into one big jar.

<profiles>
    <profile>
        <id>local</id>
        <activation>
            <activeByDefault>true</activeByDefault>
        </activation>
        <dependencies>
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter-web</artifactId>
            </dependency>
        </dependencies>
        <build>
            <plugins>
                <plugin>
                    <groupId>org.springframework.boot</groupId>
                    <artifactId>spring-boot-maven-plugin</artifactId>
                </plugin>
            </plugins>
        </build>
    </profile>
    <profile>
        <id>shaded-jar</id>
        <dependencies>
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter-web</artifactId>
                <exclusions>
                    <exclusion>
                        <groupId>org.springframework.boot</groupId>
                        <artifactId>spring-boot-starter-tomcat</artifactId>
                    </exclusion>
                </exclusions>
            </dependency>
        </dependencies>
        <build>
            <plugins>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-shade-plugin</artifactId>
                    <version>3.2.4</version>
                    <configuration>
                        <createDependencyReducedPom>false</createDependencyReducedPom>
                    </configuration>
                    <executions>
                        <execution>
                            <phase>package</phase>
                            <goals>
                                <goal>shade</goal>
                            </goals>
                            <configuration>
                                <artifactSet>
                                    <excludes>
                                        <exclude>org.apache.tomcat.embed:*</exclude>
                                    </excludes>
                                </artifactSet>
                            </configuration>
                        </execution>
                    </executions>
                </plugin>
            </plugins>
        </build>
    </profile>
</profiles>

Lambda Handler

Implement RequestStreamHandler for running the app in Lambda. We leverage async initialization described in the aws-serverless-java-container docs.

public class StreamLambdaHandler implements RequestStreamHandler {
    private static final SpringBootLambdaContainerHandler<AwsProxyRequest, AwsProxyResponse> handler;

    static {
        try {
            handler = new SpringBootProxyHandlerBuilder<AwsProxyRequest>()
                    .defaultProxy()
                    .asyncInit()
                    .springBootApplication(StarbucksApiApplication.class)
                    .buildAndInitialize();
        } catch (ContainerInitializationException e) {
            // if we fail here, we re-throw the exception to force another cold start
            e.printStackTrace();
            throw new RuntimeException("Could not initialize Spring Boot application", e);
        }
    }

    @Override
    public void handleRequest(InputStream inputStream, OutputStream outputStream, Context context)
            throws IOException {
        handler.proxyStream(inputStream, outputStream, context);
    }
}

Serverless Framework

And the final touch, to configure our Serverless Framework deployment by creating serverless.yml file in the project root:

service: starbucks-api
frameworkVersion: '3'

provider:
  name: aws
  runtime: java11
  region: eu-central-1
  environment:
    DBPW: ${ssm:dbpw}

package:
  artifact: target/starbucks-api-0.0.1-SNAPSHOT.jar

functions:
  api:
    handler: com.starbucks.api.lambda.StreamLambdaHandler::handleRequest
    timeout: 15
    memorySize: 2048
    events:
      - http:
          path: /{proxy+}
          method: any

Notice the DBPW environment variable? I used AWS Systems Manager Parameter Store to securely store the database password. You should set the DBPW environment variable locally when building the project, if you have any tests that load the Spring's Context. Feel free to experiment with the memorySize and timeout values.

Finally, build it and deploy it with:

./mvnw clean package -P shaded-jar
serverless deploy

Maven creates the shaded jar and then the Serverless Framework deploys it to AWS Lambda.

The first invocation takes a whopping 11 seconds, even though I've put 2 gigs of memory for the Lambda. Cold start seems pretty bad. I haven't been able to get a response in less than 10 seconds. Warm invocations run in under 100ms on average.

SnapStart to the rescue

Let's try to optimize this immediately by enabling SnapStart. This is done in serverless.yml by supplying the snapStart: true option to the api function and run serverless deploy.

What we can immediately notice is that the deployment takes about 2 minutes longer. That is because Lambda creates a snapshot of the function and saves it for future invocations. If we test it again, the improvements are huge - cold start time is between 1 and 1.5 seconds. Note that this is observed only on the function level, end-to-end latency is still above 2 seconds for the cold-start invocation. For some reason, API Gateway adds some more latency than usual.

A closer look at duration and pricing

To dive a bit deeper let's take a look at CloudWatch logs for one particular invocation:

REPORT RequestId: c88a0706-70fa-455b-be3a-1138f981b7e7  Duration: 1124.43 ms    Billed Duration: 1423 ms    Memory Size: 2048 MB    Max Memory Used: 206 MB Restore Duration: 392.10 ms Billed Restore Duration: 298 ms

There are a few important numbers here, and the first one is obviously Duration. That is how long it took to execute the function. Billed Duration is also familiar, but as you can see it is slightly higher than the actual duration. How come? To answer that, we have to look at the two newly added fields - Restore Duration and Billed Restore Duration. The first one is how much it took for Lambda to restore the snapshot and the second one is how much of that time we are charged for. Why are we charged less?

Restore Duration includes the time it takes for Lambda to restore a snapshot, load the runtime (JVM) and run any afterRestore hooks. You are not billed for the time it takes to restore the snapshot. However, the time it takes for the runtime (JVM) to load, execute any afterRestore hook, and signal readiness to serve invocations is counted towards the billed Lambda function duration.

Source: Starting up faster with AWS Lambda SnapStart

Further optimization - Runtime Hooks and Priming

Turning SnapStart on improved the cold start significantly, we can safely say that for a lot of use cases with some real production load this would be good enough. But ever since it was introduced, there's been talk about Priming. Runtime hooks allow you to execute some code just before the snapshot is taken, as well as right after the snapshot is restored by Lambda. I encourage you to read a more detailed explanation about it on AWS blog.

In this case, priming comes down to trying to make dummy calls to your code, so it gets compiled and saved with the snapshot, with the goal to speed up the next cold start even more. I tried a few things, and I've got mixed results.

Database connections

The first thing I did was just made a request from the beforeCheckpoint hook to one of the endpoints that fetches some data from the database. After deploying to AWS and testing, my cold-start invocation just timed out. I quickly realized that the socket timeout configuration on MongoDB is set to infinity by default, and because the connection to MongoDB was established in the snapshot process, my function was just hanging until the timeout. I changed the socketTimeoutMS setting to something like 500ms and that seemed to work, as the application quickly "realized" that it needs to create a fresh connection to the database. However, it seems that it did add these 500ms to the cold start time, beating the purpose of priming in the first place.

Regardless, default config values probably shouldn't be used in most production cases anyway.

Priming with dummy calls instead

Not wanting to mess with database connections, I decided to create a dummy request and return a response even before the code tries to fetch something from Mongo. I immediately didn't like this, as it requires some caveats in parts of the application code. However, after first tests it did seem to reduce the cold start by a few hundred milliseconds.

Let's take a quick look at the results of testing the same API with and without priming. The load was 1000 requests with the concurrency of 10.

Percentage of the requests served within a certain time (ms) - no priming

percentage	time (ms)
50%	168
98%	378
99%	1317
100%	2341

Percentage of the requests served within a certain time (ms) - with priming

percentage	time (ms)
50%	155
98%	497
99%	1050
100%	1799

Conclusions

This blog post showed how easy it can be to build a MongoDB backed Spring Boot Mono-Lambda API, with the benefits of effortless deployment to the cloud with Serverless Framework. While the example is far from being production grade, I think it's still an interesting experiment. There's a lot of room to further explore this setup, for example:

try MongoDB Atlas Serverless
ramp up security by using VPC peering and Security Groups
tweak MongoDB connection configuration that would suit potential production use
test the behavior with slightly larger API

Useful links

On top of all the links throughout the post, I highly recommend checking out the following as well. These offer additional perspectives and dive deeper into the topic:

Thanks for reading and feel free to reach out to discuss any of the related topics.

AWS Lambda Cold Starts: The Case of a NestJS Mono-Lambda API

Marko Djakovic — Fri, 08 Jul 2022 16:17:00 +0000

Earlier this year I wrote about a NestJS API packed in an AWS Lambda. If you haven't read it, feel free to check it out if you'd like to learn how to deploy an API quickly using Serverless framework. This article is a follow-up on that one and it will explore the famous "cold start" issue with Lambda functions, particularly in this case of a so called mono-lambda or fat lambda. You can also read it in full on my personal website - marko.dj - AWS Lambda Cold Starts: The Case of a NestJS Mono-Lambda API.

To test and demonstrate the cold start statistics, we will be hitting the Songs API with a variable load of requests for a couple of minutes. Then, we'll briefly analyze the results by looking at the Latency metric for the API Gateway and other useful Lambda metrics like (Init) Duration and Concurrent Executions.

What is a cold start?

There are a lot of articles already explaining this, so I'll be super short and use an image below, referenced from AWS:

Bottom line is your Lambda will be a lot slower the first time it runs compared to the invocations that follow on an already "warm" instance. This means that even if the majority of requests are processed really fast, those that are impacted by the cold start will be noticeably slower.

How did we come to this?

We've already been running one project in production with more than a 100 Lambdas written in TypeScript. When the second project came, we wanted to try something different - develop an API using NestJS, a framework we haven't worked with before but heard nice things about.

Since we relied on CloudWatch to monitor our APIs, the usual approach we saw on tutorials with only one http event that has method any and /{proxy+} path wouldn't work for us. We wanted detailed API monitoring per endpoint, so first we did something like this:

...
functions:
  getSongs:
    handler: dist/lambda.handler
    events:
      - http: 
          method: get
          path: songs
  getOneSong:
    handler: dist/lambda.handler
    events:
      - http:
          method: get
          path: songs/{id}
...

As you can see, a function per endpoint, but each function actually pointing to a whole NestJS app. And very soon we had something to see on our monitoring dashboard - this was getting pretty slow because each request was spinning up a new Lambda most of the times. And the application wasn't even that big yet.

We started looking for solutions to the cold start problem. One pretty common was to use libraries or workarounds that are "warming up" the Lambdas using a scheduled pinging mechanism. On why this approach cannot guarantee a warm Lambda execution, you can read more on AWS: Lambda execution environments. However, if you're interested to experiment with this, there's a plugin for Serverless Framework called Serverless Plugin Warmup.

The second approach is sort of an official recommendation from AWS - provisioned concurrency. Worrying about scaling beats the purpose of going serverless, right? More importantly, the API we were building was relatively new and we didn't have usage patterns which would help define the provisioned concurrency. And of course the price was a big factor, as it would cost the same as some average EC2 instance 💸

So, we ended up deploying this mono-lambda, but with a slight change to our Serverless configuration:

...
functions:
  api:
    handler: dist/lambda.handler
    events:
      - http:
          method: get
          path: songs
      - http:
          method: get
          path: songs/{id}
      - http:
          method: post
          path: songs
...

This results in all the API Gateway resources getting created, enabling us to have detailed monitoring per endpoint while minimizing the number of cold starts, since it's just one Lambda with multiple http events.

Note: In order to have metrics per API Gateway resource (endpoint), Detailed CloudWatch Metrics option needs to be enabled. It won't be enabled for the purposes of this article, as it incurs some additional costs. More information can be found on API Gateway metrics and dimensions ℹ️

I just want to hit this API with some requests so we can see the results in the monitoring and observe available metrics. With the help of Apache Bench tool I can do just that with a few lines of bash code. The load is an average of 10 requests per second for about 2 minutes. Let's discuss the results real quick.

The diagram above shows maximum latency which is very high at the beginning, due to the cold start. It falls drastically and quickly for sub-sequent requests as the Lambdas are warm. So how many cold starts did we have during this small experiment? The metrics say there are 4 concurrent Lambda executions, which means we got 4 cold starts.

For about 1000 requests, that's less than half percent. For these slow requests, lasting around 1.5 seconds, it was Lambda's Init Duration that was taking up about 1 to 1.1 seconds. The rest was Lambda handler initialization + function execution. We can see the most expensive executions, lasting under 400ms each, on the image below (Init Duration not included):

Average request time was 70ms which means that those warm Lambdas were really hitting it off. While this API is probably not like anything you'd have in production (or is it?), it's still an interesting topic to discuss.

Conclusions

I would say that API in a "fat" Lambda does not have to be terrible. Lambda should definitely not be obese, but a chubby one can perform. This seems to be a good fit for an API up to a certain size and if you don't want to manage or worry about infrastructure. As it grows it can be split into micro-services behind an API Gateway. And if performance is critical, the sweet spot might be scheduling provisioned concurrency after you understand the usage patterns.

Thanks for reading and feel free to leave your thoughts and experiences in the comments! 👋

Deploy a NestJS API to AWS Lambda with Serverless Framework

Marko Djakovic — Wed, 20 Apr 2022 17:22:39 +0000

Have you ever wondered how easy it can be to deploy and host an API? Scalable, stable, a piece of cake to deploy and costs almost nothing. The goal of this article is to demonstrate just that. We will develop a simple API which will be deployed to AWS cloud as a single Lambda function behind an API Gateway - a so-called Mono-Lambda. Whether Lambda "should" be used in that way is a different topic which I'd gladly discuss over beer. 🙂🍺

By the way, this article is now published on my personal website as well: marko.dj - Deploy a NestJS API to AWS Lambda with Serverless Framework - feel free to continue reading there!

What to expect from this article?

We will just scratch the surface of NestJS framework and its neat development experience. Once we wire it with Serverless Framework, we'll learn how quickly our API can see the light of day, going from localhost to AWS cloud in just a few steps. To demonstrate this, we will create an API for managing a database of songs - Songs API, and we'll pretend it's not useless.

Requirements

Songs API will expose endpoints for listing all songs in the database, fetching a single song details, adding and removing songs. Given the requirements, the song model has properties id, name, artist, length in seconds, genre and album. API endpoints could look something like this:

GET songs
GET songs/:id
POST songs
DELETE songs/:id

Tech stack

NestJS - a powerful framework for creating sever-side applications
TypeORM - an ORM library for TypeScript, integrates nicely with Nest for database access
Serverless Framework - easy to use framework for developing and deploying serverless apps
Serverless Jetpack - a low-config plugin that packages our code to be deployed to AWS Lambda
Serverless Express - library that makes our "plain" NestJS API play nicely with Serverless
AWS managed services like Lambda, API Gateway and RDS

I hope it sounds fun and simple enough, so let's dig in.

Install Nest CLI and create a new project and module

npm i -g @nestjs/cli
nest new songs-api

At this point the API is already set up - run it using npm run start and open localhost:3000 to see the hello world response. This is made possible by the main.ts file that is generated in the project root:

import { NestFactory } from '@nestjs/core';
import { AppModule } from './app.module';

async function bootstrap() {
  const app = await NestFactory.create(AppModule);
  await app.listen(3000);
}
bootstrap();

Now we're going to create song module, which will contain the controller, service and entity definitions. Each of these can be created individually, but the Nest CLI provides a useful command to create the module and all the required files in it in one go. It comes in handy when creating REST APIs.

nest generate resource song

Skeleton of the song module is generated. Next, we have to install dependencies for accessing the database. Since the API will run on top of a MySQL database, the following libraries should be added to the project:

npm install --save @nestjs/typeorm typeorm mysql2

Implementation

Generating the module skeleton was convenient, but of course our business logic needs to be written. Perhaps we won't be needing all the generated DTOs, we might change or add some paths to the controller, and we need to implement our entity, of course.

Since we installed TypeORM dependency, let's use it to configure object-relational mapping for the Song entity according to the above specification:

import { Column, Entity, PrimaryGeneratedColumn } from 'typeorm';

@Entity()
export class Song {
  @PrimaryGeneratedColumn()
  id: number;

  @Column()
  name: string;

  @Column()
  artist: string;

  @Column()
  duration: number;

  @Column()
  genre: string;

  @Column()
  album: string;
}

To make it work now we just need to add import to the module definition:

import { Module } from '@nestjs/common';
import { SongService } from './song.service';
import { SongController } from './song.controller';
import { TypeOrmModule } from '@nestjs/typeorm';
import { Song } from './entities/song.entity';

@Module({
  imports: [TypeOrmModule.forFeature([Song])],
  controllers: [SongController],
  providers: [SongService],
})
export class SongModule {
}

Now, let's implement the service layer. SongService uses Repository provided by TypeORM to access the database:

import { Injectable } from '@nestjs/common';
import { InjectRepository } from '@nestjs/typeorm';
import { Repository } from 'typeorm';
import { Song } from './entities/song.entity';

@Injectable()
export class SongService {
  constructor(
    @InjectRepository(Song) private songRepository: Repository<Song>,
  ) {
  }

  async create(song: Song): Promise<Song> {
    return await this.songRepository.save(song);
  }

  async findAll(): Promise<Song[]> {
    return await this.songRepository.find();
  }

  async findOne(id: number): Promise<Song> {
    return await this.songRepository.findOne({ id });
  }

  async remove(id: number): Promise<void> {
    await this.songRepository.delete(id);
  }
}

For simplicity, I'll re-use the entity as a DTO, so we can remove the whole dto folder that was generated. Then our controller and service will be rewritten to look something like this:

import { Body, Controller, Delete, Get, Param, ParseIntPipe, Post } from '@nestjs/common';
import { SongService } from './song.service';
import { Song } from './entities/song.entity';

@Controller('songs')
export class SongController {
  constructor(private readonly songService: SongService) {
  }

  @Post()
  async create(@Body() song: Song): Promise<Song> {
    return await this.songService.create(song);
  }

  @Get()
  async findAll(): Promise<Song[]> {
    return await this.songService.findAll();
  }

  @Get(':id')
  async findOne(@Param('id', ParseIntPipe) id: number): Promise<Song> {
    return await this.songService.findOne(id);
  }

  @Delete(':id')
  async remove(@Param('id', ParseIntPipe) id: number): Promise<void> {
    await this.songService.remove(id);
  }
}

As a general rule, it's always better to de-couple DTO and entity classes and have some sort of object mapper.

Database

Database is mentioned quite a few times, but where is it? 🤔

Firstly, let's test our code against a local MySQL database. Once you connect to local server, execute the following init script:

CREATE DATABASE `songsapi`;

USE `songsapi`;

CREATE TABLE `song`
(
    `id`       int(11)      NOT NULL AUTO_INCREMENT,
    `name`     varchar(200) NOT NULL,
    `artist`   varchar(200) NOT NULL,
    `duration` int(11)      DEFAULT NULL,
    `genre`    varchar(45)  DEFAULT NULL,
    `album`    varchar(200) DEFAULT NULL,
    PRIMARY KEY (`id`)
) ENGINE = InnoDB
  DEFAULT CHARSET = utf8;

After that make sure the API can connect to it by adding the following configuration to app.module.ts:

import { Module } from '@nestjs/common';
import { AppController } from './app.controller';
import { AppService } from './app.service';
import { TypeOrmModule } from '@nestjs/typeorm';
import { SongModule } from './song/song.module';

@Module({
  imports: [
    TypeOrmModule.forRoot({
      type: 'mysql',
      host: 'localhost',
      port: 3306,
      username: 'root',
      password: 'xxx',
      database: 'songsapi',
      autoLoadEntities: true,
    }),
    SongModule,
  ],
  controllers: [AppController],
  providers: [AppService],
})
export class AppModule {
}

Feel free to hardcode the values above to those corresponding to your local database configuration.

Run it 🚀

Type npm run start in the terminal and in a few seconds it should be up and running. Test it by sending some requests:

curl -X POST 'localhost:3000/songs' \
--header 'Content-Type: application/json' \
--data-raw '{
    "name": "In corpore sano",
    "artist": "Konstrakta",
    "duration": 182,
    "album": "In corpore sano",
    "genre": "pop"
}'
# Response:
# {"name":"In corpore sano","artist":"Konstrakta","duration":182,"album":"In corpore sano","genre":"pop","id":1}

# Get a single song by id
curl 'localhost:3000/songs/1'
# Response:
# {"name":"In corpore sano","artist":"Konstrakta","duration":182,"album":"In corpore sano","genre":"pop","id":1}

It works! Now that we've tested our API locally, it's time to deploy it to the cloud and make it available to the world!

Moving to the cloud 🌥

NOTE 1: It is assumed that you already have an AWS account, so creating one will not be covered.

NOTE 2: Make sure you have enough privileges to follow the steps. In case of IAM user, the shortcut is to have arn:aws:iam::aws:policy/AdministratorAccess managed policy attached.

Configure AWS account credentials

Add a profile to your AWS credentials file (usually ~/.aws/credentials):

...
[profile-name]
region=your_region
aws_access_key_id=xxx
aws_secret_access_key=yyy
aws_session_token=... (if applicable)
...

After that an environment variable should be set to activate the profile:

export AWS_PROFILE=profile-name

You should be ready to interact with your AWS cloud, feel free to quickly test if it's setup correctly by listing all S3 buckets for example:

aws s3 ls

Spin up a free-tier RDS database

So far we have successfully tested the API with local MySQL database, but now we need one on AWS. It can be done manually through the AWS Console, or you can execute the CloudFormation template provided here.

*WARNING: Please be informed about the pricing and free-tier eligibility of your account. All new AWS customers should get 1 year of free tier for certain services. Otherwise you might incur some costs as described in the official AWS RDS pricing guide -> https://aws.amazon.com/rds/mysql/pricing

AWSTemplateFormatVersion: '2010-09-09'

Resources:
  SongsDatabase:
    Type: AWS::RDS::DBInstance
    Properties:
      AllocatedStorage: 20
      DBInstanceClass: db.t3.micro
      DBInstanceIdentifier: songs-database
      PubliclyAccessible: true
      StorageType: gp2
      MasterUsername: xxx # change
      MasterUserPassword: yyy # change
      Engine: mysql
      EngineVersion: 8.0.28

Save the file above as rds.yaml for example and run it using AWS CLI:

aws cloudformation deploy --stack-name songs-api-db --template-file rds.yaml

In a few minutes the database will be ready.

Obtain the database URL either through AWS Console by navigating to RDS, or by listing exports of CloudFormation using the following command aws cloudformation list-exports. Connect to it and execute the database init script as it was done for the local instance.

Now that our database is running in the cloud, it's time to reconfigure our app to work with the RDS database instead of local one - so don't forget to update the relevant details like url, password and the rest in app.module.ts file. After that it's ready to be deployed, which is covered in the next ste.

Install and configure Serverless Framework

Install Serverless Framework CLI:

npm install -g serverless

In the root of the project, we should create the serverless.yaml file which describes the deployment:

service: songs-api

frameworkVersion: '3'

plugins:
  - serverless-jetpack

provider:
  name: aws
  runtime: nodejs14.x
  region: eu-central-1 # or whatever your region is

functions:
  api:
    handler: dist/lambda.handler
    events:
      - http:
          method: any
          path: /{proxy+}

With this configuration, the API Gateway will just proxy every request to the Lambda function and our NestJS app will handle it. The handler value is a file that contains the entry point for our app and will be explained in a minute.

Notice the serverless-jetpack plugin - it takes care of packaging our app very efficiently for Serverless. There are other plugins for this, but I've discovered this one recently and it's a lot faster than others I've used so far. Read more about it on its official github page.

Install it as a dev dependency using npm:

npm i -D serverless-jetpack

Now there's one more step before we can deploy our API - Serverless Express library to make it work in Lambda environment and it concerns the function handler.

Serverless Express

Install serverless-express library that bootstraps Express based apps to work with Lambda:

npm i @vendia/serverless-express

Then, in the source folder create a lambda.ts file that contains the Lambda handler function, which is the entry point, as referenced in the above serverless.yaml.

import { configure as serverlessExpress } from '@vendia/serverless-express';
import { NestFactory } from '@nestjs/core';
import { AppModule } from './app.module';

let cachedServer;

export const handler = async (event, context) => {
  if (!cachedServer) {
    const nestApp = await NestFactory.create(AppModule);
    await nestApp.init();
    cachedServer = serverlessExpress({ app: nestApp.getHttpAdapter().getInstance() });
  }

  return cachedServer(event, context);
}

Build, deploy & test 🚀

Finally, we are going to deploy our API to the cloud. It's fairly simple, first it should be built:

npm run build

... and then deployed:

serverless deploy

Shortly, you'll get an auto-generated url which you can use to hit the API so feel free to test it by adding, listing and removing songs. You can see logs and monitor how your app performs in the built-in dashboards on Lambda & CloudWatch services on AWS Management Console.

Clean-up

After you've played around a bit with your API, it's time to clean-up all the resources you created on your AWS cloud. If you followed the steps exactly, you'll have two CloudFormation stacks deployed - one for the database and the other for the Serverless deployment. You can either remove them manually via the Console or by running the following CLI commands:

serverless remove
aws cloudformation delete-stack --stack-name songs-api-db

Conclusion & final thoughts

I hope you made it this far and that I didn't bore you too much. Even though the main focus was on Serverless deployment on AWS Lambda, this article covered a few things along the way like setting up a simple NestJS project with TypeORM and creating an RDS MySQL database instance on AWS via CloudFormation.

What would be great for this kind of API to scale better is configuring an RDS Proxy on top of the database. Also, adding user authentication by using AWS Cognito is something which would fit nicely into this setup. Very recently AWS announced Lambda function URL feature, which eliminates the need for API Gateway but has other trade-offs, which I plan to explore next.

There are definitely some security aspects worth discussing for this to become production-ready, but it is beyond the scope of this article.

Thanks for reading and if you have any questions or suggestions feel free to comment!

There is a follow-up post on this, check it out - AWS Lambda Cold Starts: The Case of a NestJS Mono-Lambda API 🙂