DEV Community: Sidath Munasinghe

Efficient Nested Resolvers in AWS AppSync with Lambda Batching

Sidath Munasinghe — Thu, 12 Jun 2025 13:36:38 +0000

GraphQL has emerged as a modern alternative to RESTful APIs, offering a more flexible and efficient way for clients to query data. Unlike REST, where clients often make multiple requests to different endpoints and receive fixed response structures, GraphQL allows clients to request exactly the data they need — and nothing more — in a single round trip. This reduces the issues of over-fetching and under-fetching common in REST, and gives frontend developers more control over the shape of the response.

AWS AppSync is a managed service that helps developers build scalable, real-time GraphQL APIs with minimal operational overhead. It integrates seamlessly with various AWS data sources, including DynamoDB, Lambda, RDS, and OpenSearch, and supports features such as offline access, subscriptions, and fine-grained authorization. AppSync takes care of scaling and security, allowing teams to focus on defining their data and resolvers.

In AppSync, resolvers are the core building blocks that connect GraphQL fields to data sources. Each field in a GraphQL query — including nested fields — can have its own resolver. When a query is executed, AppSync invokes these resolvers individually, mapping request and response data using Velocity templates (VTL) or direct Lambda functions. While this resolver-per-field model gives developers flexibility, it can introduce a performance challenge known as the N+1 problem when working with nested data.

In this post, we’ll explore what the N+1 problem looks like in AWS AppSync, why it becomes a bottleneck at scale, and how to architect efficient resolvers to solve it using batch resolvers and Lambda optimizations.

Understanding the N+1 Problem in AppSync

When working with GraphQL, it’s common to request nested data in a single query. AppSync supports this by allowing each field — including deeply nested ones — to have its own resolver that fetches data from a backend data source. While this design provides flexibility and modularity, it can lead to an inefficient execution pattern known as the N+1 problem.

What Is the N+1 Problem?

The N+1 problem usually occurs with list queries. When your GraphQL API ends up making one query to fetch the root items (1), and N additional queries for each nested field, where N is the number of root items returned in the list.

Let’s take an example to understand this clearly.

query {
  books {
    name
    title
    author {
      firstnName
      lastName
    }
  }
}

Here’s what typically happens behind the scenes in AppSync

books resolver fetches a list of books — let’s say 100 (N) items.
For each of those 100 books, the author resolver is called individually (resulting in 100 calls).

In total, that’s 1 + 100 = 101 (N+1) resolver invocations for a single client query.

If you have even more nested queries, this becomes worse. In the below query, there is an additional field (address) that requires more resolver invocations.

query {
  books {
    name
    title
    author {
      firstName
      lastName
      address {
        city
        state
      }
    }
  }
}

books resolver fetches a list of books — let’s say 100 items.
For each of those 100 books, the author resolver is called individually (resulting in 100 calls).
Then for each author, the address resolver is called again (another 100 calls).

In total, that’s 1 + 100 + 100 = 301 resolver invocations for a single client query.

Why Is This a Problem?

This approach scales poorly:

Performance degrades linearly with the number of parent items.
It results in high latency due to the number of sequential or parallel resolver invocations.
It increases backend load and pressure on data sources like Lambda, RDS, or DynamoDB.
It can quickly hit throttling limits or increase costs when using Lambda or other pay-per-request services.

While this might be acceptable for small datasets or low traffic, the N+1 pattern becomes a serious performance bottleneck at scale. Imagine serving thousands of queries per second — this inefficient pattern can overwhelm backend systems, increase response times, and degrade the user experience.

Solving N+1 with Batch Resolvers in AppSync

One of the most effective ways to overcome the N+1 problem in AWS AppSync is by using batch resolvers. The idea is simple: instead of resolving nested fields one-by-one (which results in many resolver calls), we batch them together into a single call , usually handled by a Lambda function.

Let’s explore how this works and why it’s such a powerful pattern.

How Batch Resolvers Work

In AppSync, each nested field (such as author or address) can have its own resolver, which is typically invoked for each parent object.

To convert this into a batch operation:

Instead of calling the author resolver N times (once for each book), you configure the author field to invoke a single Lambda function that accepts a list of book IDs (or author IDs).
This Lambda function fetches all the authors in one go and returns the results mapped back to their respective books.

Think of it as “fan-in” batching: one resolver invocation processes multiple parent objects.

Let’s apply this to our previous query and see how this works.

query {
  books {
    name
    title
    author {
      firstName
      lastName
    }
  }
}

If you use a batch resolver for the author field:

AppSync groups all books[*].author field resolvers into one Lambda call.
You receive an array of bookIds or authorIds within the lambda function.
The Lambda fetches and returns the authors in bulk.

With this optimization, you’ve reduced the number of records from N+1 to just 2, which is a substantial improvement. Moreover, it’s now independent of the number of records.

Here are some additional benefits of this solution:

Fewer Resolver Invocations: Reduces hundreds of resolver calls to just one. This saves you from the lambda concurrency limit and also reduces the pressure on downstream services.
Faster Performance: Lower network overhead and latency.
Clean Separation: Keeps resolver responsibilities modular while still optimizing performance.
Cost-Efficient: Fewer Lambda invocations result in reduced AWS costs.

Enabling Batch Resolvers

Batch resolvers are currently only compatible with Lambda data sources, and this feature is not enabled by default. However, enabling this is very straightforward.

Create a Lambda datasource as usual
Go to create a resolver and select the created Lambda datasource
Enable batching and set the batching size

4.Update the resolver request function to use the BatchInvoke operation

export function request(ctx) {
    return {
        operation: 'BatchInvoke',
        payload: {
            ctx: ctx
        },
    };
}

5.Now your lambda function will receive not a single context, but an array of contexts for each listed item. You can update the lambda function logic to do a batch get and return the results. You must ensure the returned items are in the same order as the received context order.

It’s as simple as that, but it offers significant performance gains to your GraphQL service.

Conclusion

Optimizing GraphQL queries in AWS AppSync is essential when building scalable and performant APIs — especially when dealing with nested data structures. The N+1 problem, while subtle, can lead to serious performance bottlenecks if left unaddressed.

By leveraging batch resolvers, you can drastically reduce the number of resolver calls, minimize round-trips to your data source, and deliver faster, more efficient responses to your clients. Whether you choose direct Lambda resolvers or pipeline resolvers, designing with batching in mind ensures your AppSync APIs are ready to perform at scale.

As your application grows, keeping an eye on resolver patterns and query performance becomes even more important. With the right strategy, tools, and architecture in place, you can build GraphQL services that are both elegant and efficient.

Efficient Nested Resolvers in AWS AppSync with Lambda Batching

Sidath Munasinghe — Thu, 12 Jun 2025 13:36:38 +0000

Understanding the N+1 Problem in AppSync

What Is the N+1 Problem?

Let’s take an example to understand this clearly.

query {
  books {
    name
    title
    author {
      firstnName
      lastName
    }
  }
}

Here’s what typically happens behind the scenes in AppSync

books resolver fetches a list of books — let’s say 100 (N) items.
For each of those 100 books, the author resolver is called individually (resulting in 100 calls).

In total, that’s 1 + 100 = 101 (N+1) resolver invocations for a single client query.

If you have even more nested queries, this becomes worse. In the below query, there is an additional field (address) that requires more resolver invocations.

query {
  books {
    name
    title
    author {
      firstName
      lastName
      address {
        city
        state
      }
    }
  }
}

books resolver fetches a list of books — let’s say 100 items.
For each of those 100 books, the author resolver is called individually (resulting in 100 calls).
Then for each author, the address resolver is called again (another 100 calls).

In total, that’s 1 + 100 + 100 = 301 resolver invocations for a single client query.

Why Is This a Problem?

This approach scales poorly:

Performance degrades linearly with the number of parent items.
It results in high latency due to the number of sequential or parallel resolver invocations.
It increases backend load and pressure on data sources like Lambda, RDS, or DynamoDB.
It can quickly hit throttling limits or increase costs when using Lambda or other pay-per-request services.

Solving N+1 with Batch Resolvers in AppSync

Let’s explore how this works and why it’s such a powerful pattern.

How Batch Resolvers Work

In AppSync, each nested field (such as author or address) can have its own resolver, which is typically invoked for each parent object.

To convert this into a batch operation:

Instead of calling the author resolver N times (once for each book), you configure the author field to invoke a single Lambda function that accepts a list of book IDs (or author IDs).
This Lambda function fetches all the authors in one go and returns the results mapped back to their respective books.

Think of it as “fan-in” batching: one resolver invocation processes multiple parent objects.

Let’s apply this to our previous query and see how this works.

query {
  books {
    name
    title
    author {
      firstName
      lastName
    }
  }
}

If you use a batch resolver for the author field:

AppSync groups all books[*].author field resolvers into one Lambda call.
You receive an array of bookIds or authorIds within the lambda function.
The Lambda fetches and returns the authors in bulk.

With this optimization, you’ve reduced the number of records from N+1 to just 2, which is a substantial improvement. Moreover, it’s now independent of the number of records.

Here are some additional benefits of this solution:

Fewer Resolver Invocations: Reduces hundreds of resolver calls to just one. This saves you from the lambda concurrency limit and also reduces the pressure on downstream services.
Faster Performance: Lower network overhead and latency.
Clean Separation: Keeps resolver responsibilities modular while still optimizing performance.
Cost-Efficient: Fewer Lambda invocations result in reduced AWS costs.

Enabling Batch Resolvers

Batch resolvers are currently only compatible with Lambda data sources, and this feature is not enabled by default. However, enabling this is very straightforward.

Create a Lambda datasource as usual
Go to create a resolver and select the created Lambda datasource
Enable batching and set the batching size

4.Update the resolver request function to use the BatchInvoke operation

export function request(ctx) {
    return {
        operation: 'BatchInvoke',
        payload: {
            ctx: ctx
        },
    };
}

It’s as simple as that, but it offers significant performance gains to your GraphQL service.

Conclusion

Creating Smart AI Agents with AWS Bedrock

Sidath Munasinghe — Mon, 23 Dec 2024 05:43:35 +0000

Introduction

Generative AI is transforming industries by automating tasks, improving user experiences, and driving efficiency. One of the most powerful uses of AI is the creation of AI agents — intelligent systems that can perform tasks autonomously based on user input or predefined rules. These agents are being used in everything from customer support to data analysis and even in complex decision-making systems. With the advent of AWS Bedrock, building these AI agents has become more accessible than ever.

In this post, we'll dive into what AI agents are, why AWS Bedrock is the perfect platform to build them and guide you through creating your first AI agent on AWS Bedrock.

Understanding AI Agents

AI agents are autonomous systems that perform tasks or make decisions without continuous human intervention. They can interact with users, learn from data, and even evolve.

Common types of AI agents include:

Chatbots: AI agents that simulate human conversations to assist customers.
Recommendation Systems: AI agents that suggest products or content based on user behavior.
Personal Assistants: Agents that help users by scheduling tasks, setting reminders, and more.
Automation Bots: Agents that automate repetitive business processes, such as data entry or transaction processing.

AI agents can either be reactive (responding to user inputs) or proactive (anticipating user needs based on patterns or previous interactions). In this guide, we will focus on building an AI agent that leverages AWS Bedrock for its powerful foundation models.

Why Use AWS Bedrock for AI Agents?

AWS Bedrock provides a suite of tools and services designed to simplify the creation, customization, and deployment of AI models.

Never heard of BedRock before? Read my previous post here.

It offers several benefits when building AI agents:

Access to State-of-the-Art Foundation Models: AWS Bedrock provides pre-trained models from top AI research labs like Anthropic, Stability AI, and Cohere. These models serve as the backbone for your AI agents, allowing you to focus on customization rather than training models from scratch.
Serverless Architecture: AWS Bedrock takes care of the infrastructure, enabling you to build AI agents without worrying about scaling or resource management. This makes it an ideal solution for businesses of all sizes.
Customizable Models: While you can use pre-trained models, AWS Bedrock also allows you to fine-tune these models on your own datasets. This means your AI agent can be trained for specific domains, such as customer support or medical inquiries, enhancing its accuracy and relevance.
Seamless Integration with AWS Services: AWS Bedrock integrates well with other AWS services like Lambda, S3, etc, enabling you to create a fully integrated solution that can interact with other applications or databases in your environment.
Security and Compliance: With AWS Bedrock, you benefit from the robust security and compliance features provided by AWS, ensuring your AI agents operate securely and meet industry standards.

Components of an AI Agent

An AI Agent has several components. The diagram below illustrates the high-level architecture of how Bedrock agents operate. AWS Bedrock simplifies these components, making it easier for developers to build and deploy AI agents. Let's explore each component. Since it is entirely serverless, we only need to configure each component to make the agent work as we require.

Now, let's take a moment to understand the purpose of each component.

Foundational model

The foundational model serves as the “backbone” of your AI agent. These pre-trained, large-scale machine learning models provide natural language understanding, text generation, and more capabilities. AWS Bedrock offers access to various foundational models from leading providers like Anthropic, HuggingFace, Cohere, Amazon, and many more.

The foundational model provides the base intelligence for the agent, saving you from the need to train models from scratch.

Instructions

Instructions act as the “brain” of the AI agent by defining how the foundational model should behave. These guidelines specify the scope, tone, and task of the agent. Instructions can include:

The agent's role (e.g., “You are a customer service assistant.”).
Behavior and restrictions (e.g., “Answer politely and avoid technical jargon.”).
Desired outputs (e.g., “Provide short, concise answers.”).

Providing clear instructions ensures the foundational model behaves according to your application's requirements.

Action Groups

Action groups enable the AI agent to perform specific tasks or trigger actions beyond text generation. These could include API calls, database queries, or interactions with other systems. For example:

Retrieving user details from a database.
Sending notifications via an email or messaging API.
Performing calculations or processing data inputs.

Action groups allow the AI agent to go beyond static responses and interact dynamically with external systems, making it more functional and capable of solving real-world problems.

In Bedrock, you can integrate a lambda function or OpenAPI schema to define the API operations to invoke an API.

Knowledge bases

A knowledge base provides the AI agent with domain-specific information, enabling it to answer questions or perform tasks that require contextual knowledge. AWS Bedrock allows you to integrate custom datasets (e.g., documents, product catalogs, FAQs) as a knowledge base.

The agent uses this knowledge base to generate more accurate and relevant responses tailored to your organization or use case. Due to this, it is optional to use a knowledge base.

Prompt Templates

Prompt templates provide a structured way to send input to the foundational model. They combine user input with predefined instructions or placeholders to ensure consistency in responses. Amazon Bedrock Agents exposes the default four base prompt templates that are used during the pre-processing, orchestration, knowledge base response generation, and post-processing. By optionally customizing these prompt templates, you can control the quality and format of the AI agent's behavior & outputs.

Creating Your First AI Agent

In this section, we'll walk through the steps to create an AI agent using AWS Bedrock and see how easy it would be to create one. Here, we will create a simple AI Agent that can handle medical appointments to demonstrate its capabilities.

There are several ways that we can create agents on AWS. The easiest way would be to use the conversational builder so that it will interact with us to get the requirements and create the agent according to our responses. However, we will manually configure the agent to gain a better understanding.

1) First, we must request access to a foundation model that can work with agents. Here, we are going to use the Nova Pro model. You can request access to the model via the model access section in the Bedrock configuration on the AWS Management Console.

2) Then, we can start creating an agent from the builder tools by entering the agent name and description. Since we will create only an agent and not integrate with multiple agents, we are not enabling multi-agent collaboration.

3) After that, we can further configure the agent via the agent builder. You can attach the foundation model to the agent and provide instructions. The instructor must be clear and precise to ensure the model works accurately.

4) Then, we can add action groups to provide additional capabilities to the agent. Here, we are going to add three lambda functions to,

check whether an appointment is available,
list available appointments
make an appointment

We can create an action group in the console like the one below, which will also create a Lambda function for us. Later, we can update the logic as needed.

Further, we need to define the required parameters for the lambda function. We must also provide a meaningful description so the AI model can infer the values from the user input and invoke the lambda function.

We can follow the same method to create all three action groups.

5) Once all three action groups have been created, there should also be three lambda functions corresponding to each. We need to update the logic in each lambda function to perform each action.

Below is a sample lambda handler implementation to check the availability of a given appointment. You can update this implementation to query a database or invoke an API to get the result. This implementation guarantees that appointments will be available except for December 31, 2024. As you can see, we can access the previously defined parameters from the lambda event to implement the logic based on the input values.

Additionally, there is an expected response structure from Lambda functions in Bedrock. The lambda handler should return the correct payload. You can find more information about this structure here.

import json

def lambda_handler(event, context):
    agent = event['agent']
    actionGroup = event['actionGroup']
    function = event['function']
    parameters = event.get('parameters', [])

    print(parameters)

    paramDict = {item['name']: item['value'] for item in parameters}

    print(paramDict['date'])
    print(paramDict['location'])
    print(paramDict['time'])
    print(paramDict['providerName'])

    # Execute your business logic here. For more information, refer to: https://docs.aws.amazon.com/bedrock/latest/userguide/agents-lambda.html
    if(paramDict['date'] == "31/12/2024"):
        print("Unavailable date")
        responseBody =  {
            "TEXT": {
                "body": "This slot is not available"
            }
        }

    else:
        print("Available date")
        responseBody =  {
            "TEXT": {
                "body": "This slot is available"
            }
        }

    action_response = {
        'actionGroup': actionGroup,
        'function': function,
        'functionResponse': {
            'responseBody': responseBody
        }

    }

    dummy_function_response = {'response': action_response, 'messageVersion': event['messageVersion']}
    print("Response: {}".format(dummy_function_response))

    return dummy_function_response

Similarly, we can implement the lambda handler to list appointments by hardcoding some available dates. Again, you can customize this to meet any specific business requirements.

import json

def lambda_handler(event, context):
    agent = event['agent']
    actionGroup = event['actionGroup']
    function = event['function']
    parameters = event.get('parameters', [])

    print("Looking for all available slots")

    # Execute your business logic here. For more information, refer to: https://docs.aws.amazon.com/bedrock/latest/userguide/agents-lambda.html
    responseBody =  {
        "TEXT": {
            "body": "These are the available slots: On 01/01/2025, an appointment is available in Colombo at 9am with Dr. Silva. Another slot is open on 15/01/2025 in Kandy at 11am with Dr. Fernando."
        }
    }

    action_response = {
        'actionGroup': actionGroup,
        'function': function,
        'functionResponse': {
            'responseBody': responseBody
        }

    }

    dummy_function_response = {'response': action_response, 'messageVersion': event['messageVersion']}
    print("Response: {}".format(dummy_function_response))

    return dummy_function_response

Finally, we can implement the lambda function to create the appointment. We will only add a print statement to confirm the behavior in the sample implementation.

import json

def lambda_handler(event, context):
    agent = event['agent']
    actionGroup = event['actionGroup']
    function = event['function']
    parameters = event.get('parameters', [])

    paramDict = {item['name']: item['value'] for item in parameters}

    print("Placed appointment for:", paramDict)

    # Execute your business logic here. For more information, refer to: https://docs.aws.amazon.com/bedrock/latest/userguide/agents-lambda.html
    responseBody =  {
        "TEXT": {
            "body": "Appointment placed successfully"
        }
    }

    action_response = {
        'actionGroup': actionGroup,
        'function': function,
        'functionResponse': {
            'responseBody': responseBody
        }

    }

    dummy_function_response = {'response': action_response, 'messageVersion': event['messageVersion']}
    print("Response: {}".format(dummy_function_response))

    return dummy_function_response

We need to deploy the lambda functions with the new changes we added. We must also save and update the configuration we added on Bedrock to test it.

Here, we will not use a knowledge base or prompt templates to keep it simple. But we can use them to customize the agent even more.

Demo

Here is the output of the agent we created. It asks for missing details if the user hasn't given enough information to proceed. Furthermore, as we defined in the lambda function, the agent says that there are no available appointments on 31/12/2024 and it uses the ListAppointments action to recommend available slots. Finally, it is using the CreateAppointment action to make the appointment.

Here is a video demo of the agent we created. In the CloudWatch logs, we can see all the parameters we entered are captured correctly as well.

Advanced Features

Once you’ve built a basic AI agent, AWS Bedrock offers a range of advanced features that can elevate your solution to the next level. These features enable better customization, optimization, scalability, and integration, making your AI agents smarter and more adaptable for real-world applications.

Custom Fine-Tuning for Domain-Specific Tasks: AWS Bedrock allows you to fine-tune foundational models to better align with specific use cases by providing domain-specific datasets (e.g., legal, healthcare, e-commerce). Using this fine-tuned model, you can enhance the agent's performance and accuracy for specialized tasks.
Multi-Agent Systems: AWS Bedrock supports the design of multi-agent systems, where multiple AI agents collaborate to perform complex tasks. Each agent can specialize in a specific function and work together by exchanging data and context.
Guardrails for Safe and Responsible AI: AWS Bedrock allows you to implement guardrails to ensure AI agents act responsibly and align with business and ethical guidelines. Guardrails can be customized to filter, block, or monitor specific behaviors, enhancing the trustworthiness of AI outputs.
Integration with other AWS services: By integrating with other AWS services, you can easily address countless use cases (e.g., integration with step functions to provide AI capabilities to complex workflows).

Conclusion

AI agents powered by AWS Bedrock open up endless possibilities for businesses looking to integrate intelligent automation, enhance user experiences, and drive innovation. By leveraging foundational models, dynamic knowledge bases, guardrails, and multi-agent systems, you can build scalable, secure, and highly customizable solutions.

AWS Bedrock's powerful features make it accessible to teams of all sizes, enabling them to develop sophisticated AI solutions without requiring deep machine-learning expertise. Whether you're automating workflows, building interactive customer support systems, or analyzing complex data, Bedrock provides the tools and flexibility needed to succeed.

As AI continues to evolve, adopting these technologies ensures your business remains competitive and future-ready. Start exploring AWS Bedrock today to unlock the full potential of AI agents and accelerate your journey towards smarter, automated solutions.

Unleashing the Power of CDK and Terraform in Cloud Deployments

Sidath Munasinghe — Sat, 22 Jun 2024 07:59:02 +0000

Introduction

Deploying applications to the cloud has become a cornerstone of modern software development. AWS offers CloudFormation as a service to facilitate cloud deployments and tools like the AWS Cloud Development Kit (CDK). At the same time, Terraform has emerged as a powerful solution for Infrastructure as Code (IaC), enabling faster deployments to multiple cloud providers. In this article, we’ll explore the benefits of using AWS CDK and Terraform together and walk through a practical example of creating a REST API with CDK in TypeScript.

What is Terraform and CDK?

Terraform and CDK are prominent tools that empower the definition of infrastructure as code. Each solution possesses its own set of advantages and disadvantages. Let's delve into a bit more information on both.

Terraform

Terraform is a tool created by HashiCorp that allows you to define your infrastructure in a high-level configuration language called HCL (HashiCorp Configuration Language). Terraform is cloud-agnostic and can manage infrastructure across various cloud providers, including AWS, Azure, and Google Cloud Platform. It also enables faster deployments when compared to CloudFormation, specifically for AWS.

AWS CDK

The AWS Cloud Development Kit (CDK) is an open-source software development framework for defining cloud infrastructure in code and provisioning it through AWS CloudFormation. CDK uses familiar programming languages, including TypeScript, to model your applications. Underneath, CDK generates plain CloudFormation templates to create the infrastructure using the code we implement with CDK. The advantage is due to this abstraction, we could generate very lengthy CloudFormation templates within a few lines using high-level CDK constructs. So, it helps developers implement and maintain infrastructure code conveniently with their favourite programming language.

Benefits of Using Terraform and CDK Together

Using both tools together, we can enjoy the benefits of both worlds. Although Terraform uses HCL, it may not be very convenient for developers. CDK solves this by providing high-level reusable CDK constructs to implement the infrastructure within a few lines. Also, since we use a very familiar programming language, it feels so close to the developers.

On the other hand, CDK uses CloudFormation behind the scenes, which is usually slower than Terraform. However, when we use CDK and Terraform together, we can make much faster cloud deployments since we use Terraform to perform the deployments.

We can achieve this through the use of CDK for Terraform, which is introduced as Cloud Development Kit for Terraform (CDKTF), allowing us to utilise familiar programming languages to define and provision infrastructure.

Setting up a Project

Let’s set up a Terraform project with CDK using Typescript as the language. We need to set up a few prerequisites for using CDK for Terraform.

Once the setup is complete, we can initiate a project. First, let’s create a folder to set up the initial code.

Then we can initiate a project with below CLI command. Here we are going to use TypeScript.

Once the project is initialized, we can update the main.ts file to define the infrastructure we need. Within the main.ts file, it has created a CDK app as well as a Stack. We can update the resources within the stack as needed to deploy. Let’s build a simple hello world REST API using API Gateway and a Lambda function.

Building a REST API

Before adding any AWS resources, let’s configure the AWS provider in Terraform since we will use AWS as the cloud provider. Further, we can use a S3 bucket to store the Terraform backend and track the deployment states.

We can simply configure this by adding the necessary CDK constructs (AwsProvider, S3Backend) with the required parameters like below.

Here, we have configured the AWS provider by providing the AWS account ID we need to deploy with the region. Similarly, we have configured the S3 backend by providing the bucket name and other configurations.

Now, let’s create an IAM role as the execution role for the Lambda function, including the permissions for a basic lambda execution role.

Now, it’s time to create the Lambda function. Let’s add the below code for the Lambda function code inside index.ts file within the src folder. Since we are building a simple hello-world application, the Lambda function is returning a simple hello-world response.

Once we have added the Lambda function handler implementation, we can add the CDK implementation to refer to that and create the Lambda function resource.

The above definition will create a S3 bucket to hold the function code and create the Lambda function. The role we defined earlier is also provided as the execution role for the function.

Once the Lambda function is ready, we can now create the API Gateway REST API and integrate the Lambda function with it.

Here, we are defining the constructs for the API Gateway, a resource for the /hello path and the GET method under that for the /hello GET endpoint. Finally, we have integrated it with the lambda function we created earlier as a proxy integration.

Since things are integrated correctly, we can create a stage in the API Gateway and create a deployment like the one below.

We have provided the stage name and API we want to create in the configurations.

Now, we have created all the resources we need. But there is one more thing we need to do. We need to ensure that the API Gateway service can invoke the provided lambda function. To do that, we must create and attach a resource-based policy to allow that action within the Lambda function. We can do it like below easily using the LambdaPermission construct.

This construct will add the required permissions to the Lambda function to be invoked by the API we created earlier. With this, we are complete with the implementation.

The full implementation of this project can be found on this GitHub repository.

Now everything is ready to deploy. Ensure you have configured your AWS credentials correctly so the Terraform can access AWS to provision the infrastructure. We can first build the code and then deploy it using the command below.

Once the above command is executed, CDK for Terraform will install if there are any missing packages and start the deployment. After the deployment is complete, you can verify the created resources and play with the API.

Moreover, we can discern that Terraform is executing the deployment significantly faster than CloudFormation, which is incredibly advantageous.

To delete the resources you created, you can run the cdktf destroy command. This will ensure that all the resources created by the project are properly cleaned up.

Conclusion

Using AWS CDK with Terraform offers several notable benefits for managing cloud infrastructure. CDK’s deep integration with AWS and support for familiar programming languages like TypeScript make defining AWS resources intuitive and maintainable. Terraform’s cloud-agnostic capabilities complement CDK by allowing for seamless management across multiple cloud providers. This combination provides flexibility, ease of use, and modularity, enhancing the overall infrastructure management workflow. By leveraging both tools, you can streamline deployments, improve efficiency, and achieve a more robust and versatile infrastructure management solution.

Generative AI on AWS with Amazon Bedrock

Sidath Munasinghe — Mon, 06 May 2024 04:14:32 +0000

Amazon Bedrock is a managed service that provides high-performing foundation models (FMs) from leading AI companies using a single interface. With Bedrock, we don’t need to worry about hosting and managing the infrastructure for the foundation models. We can directly jump into consuming these models with its APIs and start building apps. Further, we can customize these foundation models to fit our use cases and also integrate them with knowledge bases and agents to provide enhanced features.

Here are some key features of Amazon Bedrock.

Play with several foundation models and see which suites your use case mostly, and start building apps
Fine-tune or customize the foundation models with specific datasets and parameters to enhance its capabilities
Integrate knowledge bases and tailor and augment foundation models to specific tasks or domains
Integrate agents and enrich reasoning capabilities to trigger intelligent actions

Foundation Models

Foundation models are the basic building block of Bedrock. The following diagram shows a few foundation models provided by different AI companies on Bedrock. This list will continue to grow as AWS adds more models. Each model is specific for certain tasks, and depending on your use case, the most appropriate one needs to be selected. Further, each model has different pricing models.

AWS Bedrock provides a playground where you can experiment with different models by adjusting their parameters like temperature and observe how their behaviour change. The following diagram shows a scenario of using the Titan model created by Amazon to handle text inputs.

Additionally, to the playground, we can access these models programmatically with a single API using the AWS SDK. The implementation remains the same even if you want to change the model occasionally because of that. It’s simply a matter of updating the configurations to utilize the appropriate model.

Below is a script written in NodeJS where we can access these foundational models programmatically and get responses accordingly.

const client = new BedrockRuntimeClient({
  region: 'us-east-1',
  credentials: {
    accessKeyId: process.env.AWS_ACCESS_KEY_ID,
    secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
  },
});

async function ask(prompt) {
  const params = {
    modelId: 'amazon.titan-text-express-v1',
    body: JSON.stringify({
      inputText: prompt,
      textGenerationConfig: {
        temperature: 0.7,
        topP: 0.9,
        maxTokenCount: 800,
      },
    }),
    accept: 'application/json',
    contentType: 'application/json',
  };
  console.log('Prompt:', prompt);
  const command = new InvokeModelCommand(params);
  const response = await client.send(command);
  const decodedString = convertByteToString(response?.body);
  const data = convertToJSON(decodedString);
  return data?.results[0]?.outputText;
}

ask('Give me a short description about Sri Lanka').then((response) => console.log("Answer:", response));

Once the script is run, we can see that it is giving us responses.

The full implementation can be found on this GitHub repository.

This can be seamlessly integrated into any app and further expanded by customizing the prompt based on specific use cases. See how using Bedrock to fulfil the generative AI needs is effortless.

Custom Models

A common drawback with generative AI is that it’s too generic, meaning it’s trained with outdated data or doesn’t have specific knowledge of a given domain. We can enhance a foundation model’s performance for particular tasks by training it with more data and imparting it with more knowledge using the custom model capability.

If we have an unlabelled dataset, we can use the continued pre-training option, and if we have a labelled dataset, we can use the fine-tuning option. To perform this, we can follow the wizard in the AWS console by providing the dataset from an S3 location. We require a specific format for the training dataset, which is detailed here.

Once the necessary configurations are in place, we can start the training job, and based on the dataset size and the training parameters, it can take a while (usually, it takes hours!). AWS will manage all the infrastructure related to the training job. Once the training is complete, we can directly use the custom model and run queries against it like a regular foundation model.

Let’s create a very simple custom model with the below as the content in the dataset. We need to prepare a JSONL file containing the dataset to finetune the foundation models.

{"prompt": "who are you?", "completion": "I'm a customized Amazon Titan model"}

The above dataset should be able to customize the model name. As per the below screenshot, the original foundation model calls itself as a Titan build by Amazon. After training, we can see that for the same question, it gives a different output based on our training dataset.

Here is the response from the foundation model.

Here is the response from the custom model.

Further, it’s not just a rule-based training to provide the given answer for the given prompt. If you see the prompt in the given dataset and what I have asked are not exactly the same but they are similar. The model has been trained properly to answer similar types of queries as well, which is really great.

Knowledge Bases

Knowledge bases can be utilized to provide foundational AI models with additional contextual information, enabling them to generate customized or more accurate responses akin to custom models without the need for extensive retraining. So we don’t need to spend much time retraining the models with additional data.

We must employ a technique called Retrieval Augmented Generation (RAG) to accomplish this with LLMs. This technique helps to draw information from an external data store to augment the responses generated by Large Language Models (LLMs) without retraining the entire model. We can provide this additional information using a specialized database called a vector database, which generative AI models can understand.

With the knowledge base feature on Bedrock, we only need to provide a dataset, and it has the fully managed capability to fetch the documents, divide them into blocks of text, convert the text into embeddings, and store the embeddings in your vector database using RAG. You must first upload the dataset to a S3 bucket to create a knowledge base. Then, you can use the wizard in the AWS console to create the knowledge base by pointing to the uploaded dataset and integrating it with a foundation model for generating responses. By doing this, Bedrock will create an Amazon OpenSearch Serverless vector database to retrieve newly uploaded data.

Once the vector database is ready, we can use it directly to see retrieved information stored from the vector store. Otherwise, we can use it with a foundation model to generate more user-friendly responses that match our query. However, only the Anthropic Claude models are currently supported in generating responses.

The diagram below illustrates how the vector database can be utilized as an input to a foundation model to generate augmented responses.

I have created a knowledge base using the AWS documentation for bedrock using its PDF version. Once the knowledge base is ready, we can query it as shown below. Since I’m not utilizing a foundation model to create responses, I retrieve the row information from the vector database without performing any post-processing. Nonetheless, a pleasing feedback can be attained by employing a foundation model to generate a response.

Agents

Bedrock agents allow the triggering of actions based on specific inputs and the creation of autonomous agents. For instance, you could create an agent to accept hotel room reservations from customers by configuring an agent with a knowledge base of room availability and other relevant data and the respective backend to place reservations. When configuring the backend, we need to provide an OpenAPI specification of the backend services so that it knows which endpoints to call to satisfy the request.

To have this capability, we need to configure the below components in Bedrock.

Foundation model: This is needed to interpret the user input and continue the orchestration. Currently, only the Anthropic Claude models are supported.
Instructions: Instructions are prompts describing what the agent is supposed to do. Having a clear and detailed prompt for the instruction is crucial for getting accurate results from the agent.
Action groups: Here, we need to define the agent's actions. This consists of a lambda function and an OpenAPI specification. The lambda function has the implementation to act, and the OpenAPI specification provides the agent details on invoking the function. For example, we could implement a POST /reservation endpoint in the lambda function to create a reservation and provide the API specification on the details of the request, such as URL, request body, validation requirements, etc.
Knowledge base: Knowledge base is optional but is mandatory in most cases. This can be used to provide contextual information to the agent. For example, in this case, it would be some information about the room availability, pricing details, etc, so that the agent knows to perform the actions as intended.

Once the agent is correctly configured, it understands its responsibilities based on the provided instructions. The knowledge base contains comprehensive information about the specific domain. The action group provides details on initiating each action and achieving the desired outcomes. Then, the foundation model can do the magic by orchestrating the workflow to handle a given request and provide the output to the user.

You can see a cool demonstration of an agent working with a knowledge base here.

Besides these, Bedrock offers additional features for developing responsible AI policies, including guardrails and watermark detection. We can anticipate introducing more capabilities to Bedrock as the potential of generative AI continues to unfold.

In conclusion, Amazon Bedrock offers a powerful platform for leveraging generative AI capabilities on AWS. With its Foundation models and easy-to-use APIs, developers can quickly integrate AI-driven features into their applications. Additionally, the ability to create custom models, knowledge bases, and agents opens up endless possibilities for tailoring AI solutions to specific needs. By harnessing the power of Bedrock, developers can unlock new levels of innovation and create intelligent, personalized experiences for their users.

Deploy Kubernetes in Minutes: Effortless Infrastructure Creation and Application Deployment with Cluster.dev and Helm Charts

Sidath Munasinghe — Sat, 17 Feb 2024 17:48:50 +0000

Kubernetes has quickly become the leading orchestration tool for containerized applications, celebrated for its ability to scale applications robustly and resiliently. Its success lies in a powerful framework that adeptly handles the complex life cycles of distributed applications. Yet, deploying Kubernetes cluster infrastructure, essential for tapping into this platform’s capabilities, presents significant challenges. This complexity requires considerable effort and poses a major hurdle for many seeking to leverage Kubernetes for scalable application deployment.

This guide is designed to demystify the process, showing you how to set up a Kubernetes infrastructure swiftly and without hassle. We aim to transform the perceived daunting task of Kubernetes deployment into a streamlined and straightforward process so you don’t want to struggle with its complexities. By the end of this article, you’ll be equipped to utilize the full power of Kubernetes, making infrastructure deployment not just feasible but also simple, efficient, and repeatable. Let’s embark on this journey to simplify Kubernetes deployment, turning obstacles into opportunities for growth and innovation.

Complexities with Kubernetes

If you have tried to create your own Kubernetes cluster, you know the pain behind the complexities of that. The biggest problem is having a highly available and scalable control plane for cluster management. The easiest way to solve this challenge is to hand over that complexity to a managed service by a cloud provider so that you don’t need to worry about it.

However, that’s not the end. Still, you will need to do a lot of configurations to get it working. If you see the documentation of Elastic Kubernetes Service (EKS), the managed service for Kubernetes from AWS, it has a lot of things to do, like setting up IAM roles, creating certain subnets in the VPC, etc. Although we can use an IaC to provision infrastructure, implementing them would take significant effort. We can resolve this with Cluster.dev using its reusable templates so we don’t need to implement the same code repeatedly. With Cluster.dev, we can get a working Kubernetes cluster effortlessly within a few minutes. If you are not familiar with Cluster.dev, read my previous articles from here.

We now have an operational Kubernetes cluster ready to accept deployments. If you are familiar with Kubernetes concepts, to deploy an application, we need to create several Kubernetes resources such as deployments, services, ingress controllers, config maps, stateful sets, etc, as needed, which takes some effort to create the necessary YAML files including configurations. To simplify this, we can use Helm charts.

Helm is a package manager that automates Kubernetes applications' creation, packaging, configuration, and deployment by combining your configuration files into a single reusable package. This eliminates the requirement to create the mentioned Kubernetes resources by ourselves since they have been implemented within the Helm chart. All we need to do is configure it as needed to match our requirements. From the public Helm chart repository, we can get the charts for common software packages like Consul, Jenkins SonarQube, etc. We can also create our own Helm charts for our custom applications so that we don’t need to repeat ourselves and simplify deployments.

Setting up a Jenkins Service with Cluster.dev and Helm on Kubernetes

Now, let’s see how we can use these tools together to deploy and run a Jenkins service on AWS EKS. Since Cluster.dev natively supports Helm, we can make this even more effortless.

Let’s start by setting up the necessary prerequisites. We need to install the below CLIs to start the deployments. Although there are several tools to install, please note that this is a one-time setup. So we only need to spend time to install them only once.

Terraform: We use this to provision the AWS infrastructure to run the Kubernetes cluster
Cluster.dev: We use this to orchestrate the deployment by using Terraform to deploy the cluster and then use Helm to deploy Jenkins
AWS CLI: We use this to interact with our AWS account
kubectl: We use this to interact with the Kubernetes cluster
Helm: We use helm to simplify the Kubernetes deployments

Once the CLIs are ready, we can generate the Kubernetes infrastructure code and deploy it with Cluster.dev. We can use the below command to reuse a Cluster.dev template that I have created previously and do the generation instantly.

To do this, create a project folder and run the below Cluster.dev command inside it.

cdev project create https://github.com/sidathasiri/cdev-eks.git - interactive

This will give you an interactive guide like the one below to configure the template with custom configurations.

Now, we get a project generated with the necessary configurations to deploy a Kubernetes cluster. Let’s extend it to deploy Jenkins with Helm. To do that, we need to update the template.yaml file with additional units as follows. The first unit (kubeconfig) is a shell unit configuring the kubectl CLI to interact with the Kubernetes cluster we create. The second unit (jenkins) is a Helm unit configured with the Jenkins Helm chart from the public Helm repository. To access the Jenkins service from the public internet, we need to create a Kubernetes service as a load balancer. We can provide this configuration with a values.yaml file in template/helm/values.yaml path. The second snippet below shows the content of that file.

  - name: kubeconfig
      type: shell
      depends_on: this.eksCluster
      force_apply: true
      apply:
        commands:
          - aws eks update-kubeconfig --region {{ .variables.region }} --name {{ .variables.cluster_name }}
  - name: jenkins
    type: helm
    depends_on: this.kubeconfig
    kubeconfig: ~/.kube/config
    source:
      repository: 'https://charts.jenkins.io'
      chart: 'jenkins'
      version: '5.0.13'
    values:
        - file: ./helm/values.yaml

controller:
  serviceType: LoadBalancer

You can find the full implementation of this from this GitHub repo

That’s all we need to do. Now, we can run the command below, and Cluster.dev will create the Kubernetes cluster and then deploy the Jenkins service within it.

cdev apply - force

Once the deployment is complete, we can see that a load balancer has been created. We can access the deployed Jenkins service using the DNS address of the load balancer on the port 8080.

See how effortlessly we deployed an application on Kubernetes with minimal code, and even better, in just a few minutes! Say goodbye to the hassle of setting up Kubernetes — we’ve simplified it for you.

Conclusion

In conclusion, leveraging Cluster.dev alongside Helm charts offers a streamlined approach to Kubernetes infrastructure creation and application deployment. By utilizing Cluster.dev templates, we can efficiently generate infrastructure components, reducing setup time and complexity. Helm charts further simplify the deployment process, allowing for seamless application deployment within minutes. This integration enhances productivity and eliminates tedious manual configuration tasks, making Kubernetes more accessible to developers and teams. Embracing these tools can significantly accelerate the development and deployment lifecycle, enabling focus on core objectives and innovation within Kubernetes environments.

Check out my other articles on Cluster.dev series

Streamlining SonarQube on AWS ECS: Simplified Deployment Using Cluster.dev

Sidath Munasinghe — Sat, 03 Feb 2024 07:04:51 +0000

SonarQube, crafted by SonarSource, is an open-source platform designed to scrutinize code quality continuously. It proficiently identifies bugs and code smells across a spectrum of programming languages through automated reviews leveraging static code analysis.

However, if you are going to self-host SonarQube, it takes significant effort to provision both a resilient database infrastructure and a scalable compute layer capable of accommodating fluctuating traffic demands. Let's use AWS RDS for the resilient database and AWS ECS for the scalable compute layer. To simplify the deployment, let's use Cluster.dev. If you are new to Cluster.dev, I recommend you read my previous post for a comprehensive introduction and understand its benefits.

Below is the infrastructure setup we will build with Cluster.dev in this blog post.

Before jumping into the implementation, let's learn some basics about Cluster.dev first.

Cluster.dev Basics

Below are the fundamental building blocks of a Cluster.dev project.

Unit: A unit represents a unit of resource that we have in our infrastructure setup (ex. a load balancer). We can use a variety of technologies to implement a unit, such as Terraform modules, Helm charts, Kubernetes manifests, Terraform code, Bash scripts, etc. We need to provide specific inputs to a unit to configure it as we want, and it gives specific outputs to use as well as to refer to other units if required.
Stack Template: A stack template contains a set of units to implement an infrastructure pattern we need to provision. In this scenario, it's our SonarQube deployment on ECS. We can get the benefit of a variety of technologies by using different units and connecting them to lay out a complex infrastructure pattern in the stack template.
Stack: A stack is used to define different variables and configure the stack template as needed. This helps to tailor the defined infrastructure pattern in the stack template according to the use case.
Project: A project can be used to orchestrate one or more stacks depending on the complexity of the infrastructure. Any global variables that can be used across stacks can be defined at the project level.
Backend: This includes configuration of the location where Cluster.dev keeps track of its state of deployments.

The diagram below reveals how these building blocks are set up for SonarQube ECS deployment.

Implementing the Infrastructure

Before implementing any infrastructure pattern, we need to identify the resources we need to create for the infrastructure pattern as units and the technology we should use for each. For this setup, we will be using Terraform modules to create the AWS resources below.

ECS Cluster
ECS Task Definition
ECS Service
Load Balancer
Postgres RDS Database
Security groups for Database, Load balancer & ECS service
Necessary IAM roles

Let's start with the template.yaml file to define the resources we need. The below YAML file contains all the AWS resources we need to create for this setup. Note how we have connected different terraform modules to provision the infrastructure we need. Also, we have used several variables to make the infrastructure pattern repeatable for diverse use cases. The syntax for using a variable is {{ .variables.<variable_name> }}. Further, we can refer to the outputs of one unit in another using the {{ remoteState "this.<unit_name>.<attribute>" }} syntax.

Finally, we have a printer unit to output the DNS name of the load balancer to access the deployed SonarQube application.

_p: &provider_aws
  - aws:
      region: {{ .variables.region }}

name: cdev-sonarqube
kind: StackTemplate
units:
  - name: WebSecurityGroup
    type: tfmodule
    providers: *provider_aws
    source: terraform-aws-modules/security-group/aws//modules/http-80
    inputs:
      name: 'WebSecurityGroup'
      vpc_id: {{ .variables.vpc_id }}
      ingress_cidr_blocks: ["0.0.0.0/0"]

  - name: DBSecurityGroup
    type: tfmodule
    providers: *provider_aws
    source: terraform-aws-modules/security-group/aws
    inputs:
      name: 'DBSecurityGroup'
      vpc_id: {{ .variables.vpc_id }}
      ingress_with_source_security_group_id:
        - rule: "postgresql-tcp"
          source_security_group_id: {{ remoteState "this.ECSSVCSecurityGroup.security_group_id" }}

  - name: ECSSVCSecurityGroup
    type: tfmodule
    providers: *provider_aws
    source: terraform-aws-modules/security-group/aws
    inputs:
      name: 'ECSSVCSecurityGroup'
      vpc_id: {{ .variables.vpc_id }}
      ingress_with_cidr_blocks:
        - from_port: 9000
          to_port: 9000
          protocol: "tcp"
          cidr_blocks: "0.0.0.0/0"
      egress_with_cidr_blocks:
        - from_port: 0
          to_port: 0
          protocol: "-1"
          cidr_blocks: "0.0.0.0/0"

  - name: Database
    type: tfmodule
    providers: *provider_aws
    source: terraform-aws-modules/rds/aws
    inputs:
      engine: 'postgres'
      engine_version: '14'
      family: 'postgres14' # DB parameter group
      major_engine_version: '14' # DB option group
      instance_class: 'db.t4g.large'
      identifier: 'sonar-database'
      db_name: 'sonarqube'
      username: 'sonar_user'
      password: 'password'
      publicly_accessible: true
      allocated_storage: 5
      manage_master_user_password: false
      vpc_security_group_ids: [{{ remoteState "this.DBSecurityGroup.security_group_id" }}]
      subnet_ids: [{{ .variables.subnet_1 }}, {{ .variables.subnet_2 }}]

  - name: ECSCluster
    type: tfmodule
    providers: *provider_aws
    source: terraform-aws-modules/ecs/aws
    inputs:
      cluster_name: 'sonar-cluster'

  - name: ECSTaskDefinition
    type: tfmodule
    providers: *provider_aws
    source: github.com/mongodb/terraform-aws-ecs-task-definition
    inputs:
      image: 'sonarqube:lts-community'
      family: 'sonar'
      name: 'sonar'
      portMappings:
        - containerPort: 9000
          hostPort: 9000
          protocol: 'tcp'
          appProtocol: 'http'
      command:
        - '-Dsonar.search.javaAdditionalOpts=-Dnode.store.allow_mmap=false'
      environment:
        - name: SONAR_JDBC_URL
          value: jdbc:postgresql://{{ remoteState "this.Database.db_instance_endpoint" }}/postgres
        - name: SONAR_JDBC_USERNAME
          value: sonar_user
        - name: SONAR_JDBC_PASSWORD
          value: password
      requires_compatibilities:
        - 'FARGATE'
      cpu: 1024
      memory: 3072
      network_mode: awsvpc

  - name: LoadBalancer
    type: tfmodule
    providers: *provider_aws
    source: terraform-aws-modules/alb/aws
    inputs:
      name: 'sonarqube'
      vpc_id: {{ .variables.vpc_id }}
      subnets: [{{ .variables.subnet_1 }}, {{ .variables.subnet_2 }}]
      enable_deletion_protection: false
      create_security_group: false
      security_groups: [{{ remoteState "this.WebSecurityGroup.security_group_id" }}]
      target_groups:
        ecsTarget:
          name_prefix: 'SQ-'
          protocol: 'HTTP'
          port: 80
          target_type: 'ip'
          create_attachment: false
      listeners:
        ecs-foward:
          port: 80
          protocol: 'HTTP'
          forward:
            target_group_key: 'ecsTarget'

  - name: ECSService
    type: tfmodule
    providers: *provider_aws
    source: terraform-aws-modules/ecs/aws//modules/service
    inputs:
      name: 'sonarqube'
      cluster_arn: {{ remoteState "this.ECSCluster.cluster_arn" }}
      cpu: 1024
      memory: 4096
      create_task_definition: false
      task_definition_arn: {{ remoteState "this.ECSTaskDefinition.arn" }}
      create_security_group: false
      create_task_exec_iam_role: true
      assign_public_ip: true
      subnet_ids: [{{ .variables.subnet_1 }}, {{ .variables.subnet_2 }}]
      security_group_ids: [{{ remoteState "this.ECSSVCSecurityGroup.security_group_id" }}]
      load_balancer:
        service:
          target_group_arn: {{ remoteState "this.LoadBalancer.target_groups.ecsTarget.arn" }}
          container_name: sonar
          container_port: 9000

  - name: outputs
    type: printer
    depends_on: this.LoadBalancer
    outputs:
      sonar_url: http://{{ remoteState "this.LoadBalancer.dns_name" }}

With that, the complex part is done 🙂

Now, let's define the stack.yaml file, including the variables to configure the stack template. Here, we have defined the below configurations as variables so that we can change them and use the existing AWS networking infrastructure.

region: AWS region
vpc_id: ID of VPC we need to deploy
subnet_1: ID of subnet 1
subnet_2: ID of subnet 2

We can define more variables as needed to allow more flexibility to the stack template.

name: cdev-sonarqube
template: ./template/
kind: Stack
backend: aws-backend
variables:
  region: {{ .project.variables.region }}
  vpc_id: {{ .project.variables.vpc_id }}
  subnet_1: {{ .project.variables.subnet_1 }}
  subnet_2: {{ .project.variables.subnet_2 }}

Let's use a S3 bucket to store the backend state of Cluster.dev. We can define a backend.yaml to configure this.

name: aws-backend
kind: Backend
provider: s3
spec:
  bucket: {{ .project.variables.state_bucket_name }}
  region: {{ .project.variables.region }}

Now, we are ready to define the project.yaml file to use this stack. For this infrastructure pattern, we are only a single stack. Here, we can define the global variables for the project as well.

name: cdev-sonarqube
kind: Project
backend: aws-backend
variables:
  organization: <org-name>
  region: <aws-region>
  state_bucket_name: <state-bucket-name>
  vpc_id: <vpc-id>
  subnet_1: <subnet1-id>
  subnet_2: <subnet2-id>

The full implementation of this can be found on this GitHub repository.

Deploying the Infrastructure

Now, we can use the Cluster.dev CLI to deploy the infrastructure with the following command.

cdev apply

Once we run this command, it gives us a summary of the resources that it is going to deploy, like below.

Also, once the deployment is complete, the printer unit outputs the URL to access the deployed SonarQube application.

Also, you can notice that our deployment has auto-scaling enabled to scale out and scale in according to the incoming traffic.

As per the diagram above, we can see that it scales out when the CPU and memory reach certain thresholds to a max of 10 tasks. We can fine-tune these settings based on our requirements.

And there you have it — the culmination of our efforts. With the templates prepared, you can configure them to suit the specific use case, enabling seamless and repeatable deployments. This streamlined approach ensures adaptability and efficiency, allowing for quick and hassle-free setup whenever needed.

Conclusion

In conclusion, we've walked through the essential steps to deploy SonarQube on AWS ECS using Cluster.dev covering its key aspects. This guide provides a seamless and efficient approach, empowering users to set up SonarQube on AWS ECS effortlessly. By combining the capabilities of SonarQube with the simplicity of Cluster.dev, we've created a reliable and easily managed infrastructure for elevated code analysis and quality assurance practices.

Revolutionizing Infrastructure Management with Cluster.dev: A Journey into Effortless Orchestration

Sidath Munasinghe — Tue, 23 Jan 2024 09:01:02 +0000

In the ever-evolving landscape of cloud computing and DevOps, the quest for streamlined and efficient infrastructure management remains a perpetual challenge. Enter Cluster.dev, a cutting-edge tool poised to redefine the way we approach Infrastructure as Code (IaC). This innovative solution addresses the pain points that have long plagued existing IaC tools, offering a refreshing perspective and a promise of enhanced simplicity and effectiveness.

What is IaC, and Why is it Important?

Before we unravel the intricacies of Cluster.dev, let’s take a step back to understand the significance of Infrastructure as Code. IaC represents a paradigm shift in how we manage and provision infrastructure resources. Instead of relying on manual configuration and deployment processes, IaC allows developers and operations teams to define infrastructure components programmatically. This enhances repeatability and consistency, accelerates development cycles, and promotes team collaboration. In essence, IaC transforms infrastructure management into a code-centric, version-controlled, and automated process, aligning seamlessly with the principles of DevOps. Here are some popular IaC tools widely used in the industry,

Terraform
AWS CloudFormation
Ansible
AWS CDK
Pulumi

Challenges in the Current IaC Tools

Although there are well-adopted IaC tools, teams often find themselves caught in a web of diverse tools, each with its own set of advantages and limitations. Orchestrating different infrastructure tools under the same roof presents a formidable challenge, requiring teams to navigate a complex maze of configurations, syntaxes, and deployment strategies. This lack of cohesion can lead to inefficiencies and increased learning curves.
Moreover, maintaining consistent usage of these tools across an organization presents another challenge — ensuring adherence to best practices for establishing a resilient and reliable infrastructure.

What Cluster.dev Brings to the Current IaC Landscape?

Cluster.dev is a tool that uses Terraform, Helm alongside other infrastructure as code tools as building blocks to lay out complex infrastructures. It’s not competing with other IaC tools. Instead, it provides a higher abstraction, fixing the mentioned challenges while bringing different infrastructure tools under the same roof. Below are some key benefits introduced by Cluster.dev.

Orchestrating Diverse Tools

Cluster.dev acknowledges the diversity of tools in the IaC ecosystem and addresses the challenge of managing them cohesively. By providing a centralized platform that seamlessly integrates with various infrastructure tools, it enables teams to leverage the strengths of each tool without compromising on unity.

Templating for Consistency

One of the standout features of Cluster.dev is its powerful templating system. Templating allows organizations to implement common infrastructure patterns with best practices across the board. This reduces the implementation effort and ensures a consistent and standardized approach to infrastructure management.

Multi-Cloud Deployments

Cluster.dev breaks down the silos between cloud providers by allowing the deployment of the same infrastructure pattern across multiple platforms. This feature empowers organizations to embrace a multi-cloud strategy, mitigating vendor lock-in and optimizing resource utilization based on specific provider strengths.

Template Sharing and Customization

In the collaborative landscape of modern development, time is of the essence. Cluster.dev enables teams to share templates and tailor them to specific requirements swiftly. This fosters collaboration and saves valuable time by capitalizing on pre-defined templates that align with industry best practices.

Simplifying SaaS Deployments

For SaaS vendors, Cluster.dev opens new frontiers by allowing the seamless integration of infrastructure templates into their products. This means SaaS providers can ship infrastructure as part of their product, simplifying enterprise software deployment for end-users and ensuring a smoother onboarding experience.

Enforce Best Practices and Standards

One of the most significant challenges in infrastructure management is the consistent application of best practices and standards across various projects within an organization. Cluster.dev offers a solution to this predicament as well by allowing templates to be crafted and overseen by a dedicated platform/DevOps team possessing IaC expertise. This helps organizations to seamlessly implement and propagate best practices and patterns for infrastructure across all projects. Further, this ensures that development teams effortlessly adhere to standardized practices, fostering a more streamlined and efficient workflow.

Creating a Serverless API with Cluster.dev

Now, let’s see this in action to see how convenient it is to use Cluster.dev to provision infrastructure seamlessly. Suppose we are starting a new project, and we need to create a serverless API on the AWS cloud with an API Gateway backed by Lambda. Let’s use Cluster.dev to initialize the project with a template for this use case.

Create a S3 bucket to hold the Cluster.dev backend state to keep track the infrastructure changes. We can quickly do this with AWS CLI.

aws s3 mb s3://<bucket-name>

Crete a project folder and initialize it with the appropriate Cluster.dev template. When we run its interactive CLI command to create the project, it guides us in completing the needed configurations and preparing it for our new project.

Implementation of this template can be found on this GitHub repository.

mkdir cdev-serverless-api
cd cdev-serverless-api
cdev project create https://github.com/sidathasiri/cdev-serverless-api --interactive

Let’s see the Cluster.dev interactive CLI steps to understand how it guides us to complete the project setup.

You can see it’s generating all the necessary infrastructure code according to the configurations provided by us. With just a single command, the entire infrastructure code is at your disposal — a seamless and efficient process! 😎
Now, let’s deploy the infrastructure with the below command

cdev apply

Before starting the deployment, this will reveal the infrastructure resource changes that will be applied so that we can review them before deploying.

We can review them and hit continue to proceed with the deployment.

Once the deployment is complete, we can see it outputs the deployed API URL as well. We can verify that the deployment is complete by sending a request to the API using curl.

With Cluster.dev handling the complete infrastructure setup following the best practices, the development team can now direct their attention solely to implementing the core functionalities of the service. This seamless division of responsibilities streamlines the development process, allowing for a more focused and efficient approach to building the service functionalities.

When to Use Cluster.dev?

We saw how convenient it is to use Cluster.dev to manage our infrastructure effortlessly. Here are several scenarios where Cluster.dev truly excels, bringing numerous benefits to your organization.

When your development teams lack knowledge in IaC

Managing infrastructure can be a daunting task, especially when your development teams may not be well-versed in IaC. Cluster.dev comes to the rescue by providing a user-friendly interface for defining templates abstracting the complexities of infrastructure setup. This empowers development teams to work on their code efficiently without the need for extensive IaC expertise.

When you want to standardize your infrastructure

Achieving consistency and adherence to best practices across diverse projects becomes effortless with Cluster.dev. By enabling the creation and ownership of templates by a dedicated platform or DevOps team, organizations can standardize infrastructure configurations. This ensures a unified and standardized approach, facilitating easier maintenance and reducing the risk of inconsistencies across different projects.

When you are in a multi-cloud environment

Managing infrastructure across multiple cloud providers can be challenging. Cluster.dev simplifies the orchestration of resources in a multi-cloud environment, offering a unified solution for deploying and managing infrastructure across various cloud platforms. This enhances flexibility and minimizes vendor lock-in while allowing organizations to make the most of the advantages offered by different cloud providers.

When you are on a microservices architecture

In a microservices architecture, the complexity of managing numerous services and their associated infrastructure can be overwhelming. Cluster.dev aligns perfectly with this paradigm by allowing the definition of templates for each microservice, promoting scalability, flexibility, and efficient management of microservices-based infrastructures.

When you want to deploy repeatable complex infrastructure

Deploying complex infrastructure consistently is a challenge, but Cluster.dev excels in this scenario. By defining templates that encapsulate intricate configurations, organizations can deploy repeatable and complex infrastructures seamlessly. This ensures reliability and reduces the likelihood of errors in the deployment process.

When you want to benefit from several IaC tools

Cluster.dev acts as a bridge to various Infrastructure as Code tools, enabling organizations to leverage the strengths of different tools based on their requirements. Whether it’s Terraform, Helm, or other IaC tools, Cluster.dev provides a cohesive platform, allowing users to harness the benefits of multiple tools within a unified and streamlined workflow.

Conclusion

Infrastructure as Code (IaC) has become a cornerstone in modern development practices, and Cluster.dev is at the forefront of revolutionizing this space. We’ve explored the significance of IaC, identified challenges in existing tools, and witnessed how Cluster.dev brings a fresh perspective to infrastructure orchestration. Whether simplifying serverless API creation or addressing broader infrastructure challenges, Cluster.dev emerges as a dynamic solution. With its versatility and adaptability, Cluster.dev beckons organizations to embrace a more efficient, standardized, and future-ready approach to infrastructure management.

Harnessing Feature Flags on AWS AppConfig for Seamless Software Evolution

Sidath Munasinghe — Mon, 18 Dec 2023 04:26:37 +0000

In the fast-paced realm of software development, agility and adaptability are not just a virtue but a necessity. Feature flagging is a versatile technique that offers a strategic approach for releasing features to end users, enabling rapid software development while giving more control to teams to evolve products.

Feature flags decouple feature deployment from code deployment, allowing teams to release features incrementally and independently. Due to this, teams can do frequent code deployments darkly, although the entire feature development is incomplete. Once the entire feature development is complete and ready to enable users, it can be released on demand as needed. This promotes a continuous deployment approach, enabling faster time-to-market without disrupting the entire application.

Advantages of Feature Flags

Incorporating feature flags introduces a lot of advantages to the software development process as well as to the software releases.

Risk Mitigation: Feature flags can act as safety nets since they enable teams to turn off features in production if there are any issues without needing additional code deployments. This helps teams to troubleshoot issues and apply the fixes while keeping the production environment healthy.
Personalization and Experimentation: Feature flags provide the capability to enable features for specific users or user segments. This provides the canary feature enablement to roll out new features gradually. Further, this extends to conducting A/B testing and experimentation, where teams can gather valuable user feedback and data to make informed decisions about feature improvements.
Unblock Dependencies: It’s common in software development that teams get blocked due to dependencies with other teams. With feature flags, teams can work independently based on the design/contract and deploy even the partially completed implementation to production behind a feature flag. Once all teams have completed the implementation and everything is tested, it can be enabled for end users.
Graceful Degradation: There are situations where systems receive excessive traffic, putting them under stress and leading to performance issues. In those scenarios, teams can use feature flags to turn off non-critical features and let the core features perform as expected.

AppConfig Feature Flags

AWS AppConfig is a service that facilitates deploying and managing application configurations on the AWS cloud, including feature flags. It allows teams to create, manage, and deploy configurations seamlessly with several additional capabilities.

Rollout strategies: AWS AppConfig supports different rollout strategies to enable feature activation in a controlled manner. This helps to identify potential issues, gather user feedback in a controlled manner, and react accordingly to keep rolling out or revert.

Versioning: The service supports automatic versioning of configurations, allowing teams to track the entire history of the configuration changes.
Rollback: When there is a need to change the flag status, AppConfig provides a built-in capability to rollback configuration to previous versions.
Manage Environments: AppConfig has a concept called environment, which can be used to control feature flag values per each product environment. This helps to test the feature flag capabilities in non-prod environments first and then use them in production as needed.

Data Retrieval in AppConfig Feature Flags

The real magic behind the feature flags is we don’t need to redeploy applications to get the latest configuration changes. When we update the configuration settings, all the applications will get the latest changes automatically. Applications can get this capability by establishing a configuration session with the AppConfig server and polling for configuration changes. The sequence diagram below summarizes the overall communication flow.

Create a configuration session using the application name, configuration profile, and environment we must connect to. In response, it will return a token to get the latest configuration in the next poll request.
Once the application gets that token, it can be used to get the latest configuration by calling the GetLatestConfiguration API. In response, it will return the latest configuration settings and a new token for the next poll.
The application can periodically call the GetLatestConfiguration API using the token that was retrieved previously.

Here, the important thing is that the token can be used only once. So, the application needs to keep track of the latest token it received and use it in the subsequent request.

Integration Options

Since implementing this polling mechanism to get the flag status/configuration is not as straightforward, AWS has provided simplified integration options for common compute services.

AWS Lambda

AWS Lambda can be easily integrated with AppConfig feature flags with AWS AppConfig Agent Lambda extension as a layer to the lambda function. The lambda extension layer handles the polling implementation, and developers can benefit from feature flags. Further, the lambda layer will fetch and store the flag's statuses in a local cache, including the necessary tokens for subsequence API calls.

To access its configuration data, the function can call the AWS AppConfig extension at an HTTP endpoint running on localhost:2772. The following diagram shows how it works.

Amazon EC2

Like the lambda extension, an AWS AppConfig Agent can be installed on the instance to interact with AppConfig and fetch and cache the flag configuration data on your behalf. As we saw earlier, the retrieved configurations can be accessed from localhost:2772.

It is important to note that the agent is available for Linux operating systems running kernel version 4.15 or greater, and the agent can be installed via the yum command-line package-management utility.

Amazon ECS and Amazon EKS

AWS AppConfig can be integrated with AWS ECS and EKS using AWS AppConfig Agent. The agent functions as a sidecar container running alongside the main container application. The agent will manage all the interactions with AppConfig, and the fetched configurations can be accessed from localhost:2772.

Integrating into AWS Lambda

Now, let’s walk through how to implement this for AWS Lambda, a common serverless computing service. We will use AWS CDK for the infrastructure code to automate the infrastructure provisioning.

To implement this setup, we need to create the below resources on AWS.

AppConfig Application
AppConfig Environment
AppConfig Configuration Profile
AppConfig Configuration Version
Lambda Function

Let’s see each component and how to implement them with CDK constructs.

AppConfig Application

An AppConfig Application refers to a logical entity that utilizes AWS AppConfig for managing its configuration settings. This could be any software application or service that benefits from dynamic and centralized configuration management.

We can create an application with CDK using the code snippet below by providing a meaningful name for the use case.

const application = new CfnApplication(scope, `AppConfig Application`, 
  name: 'e-commerce-app',
});

AppConfig Environment

An AppConfig Environment is a deployment environment within an AWS AppConfig application where the configurations are managed independently from each other. Environments provide a way to separate configurations for different stages of development, testing, and production, allowing for controlled and efficient management of configurations across different deployment scenarios.

We can create an environment using the snippet below. Since we are creating an environment under an application, we need to provide a reference to the application we created earlier in addition to the environment name.

const environment = new CfnEnvironment(scope, `AppConfig Environment`, {
  applicationId: application.ref,
  name: 'DEV',
});

AppConfig Configuration Profile

A configuration profile helps to define what kind of configuration (feature flag/freeform) will be created and optionally defines any validators to ensure the configuration data is syntactically and semantically correct.

The code snippet below creates a feature flag type configuration profile called login. When we use the feature flag type, the location URI has to be set with hosted. Further, we need to specify the application for this configuration profile.

const configurationProfile = new CfnConfigurationProfile(
  scope,
  `AppConfig ConfigurationProfile`,
  {
    applicationId: application.ref,
    locationUri: 'hosted',
    name: 'login',
    type: 'AWS.AppConfig.FeatureFlags',
   }
);

AppConfig Configuration Version

An AppConfig Configuration Version represents a specific snapshot or version of a configuration. As configurations may evolve over time, different versions allow for tracking and managing changes. Each version is associated with a unique identifier.

Using the code snippet below, we create a configuration version for a feature flag called sso_enabled, which the value has set to true. Similar to the previous constructs, we need to connect this with the application and the configuration profile we need to use.

const configurationVersion = new CfnHostedConfigurationVersion(
  scope,
  `AppConfig ConfigurationProfileVersion`,
  {
    applicationId: application.ref,
    configurationProfileId: configurationProfile.ref,
    contentType: 'application/json',
    content: JSON.stringify({
      flags: {
        flagkey: {
          name: 'sso_enabled',
        },
      },
      values: {
        flagkey: {
          enabled: true,
        },
      },
       version: '1',
    })
  }
);

Lambda Function

Now, we can create the lambda function and attach the lambda layer to integrate it with AppConfig. The lambda function must be configured with proper environment variables so the agent can connect with the required AppConfig application, environment and configuration profile, as shown in the code snippet below. Further, we must grant permissions via IAM to the lambda function to access AppConfig and fetch the configurations.

Below is the CDK infrastructure code for the lambda function, including the lambda layer integration and permission granting to AppConfig.

const lambdaFunction = new NodejsFunction(this, 'my-lambda-fn', {
      entry: join(__dirname, '../src/lambdaHandler.ts'),
      runtime: Runtime.NODEJS_18_X,
      handler: 'handler',
      timeout: Duration.seconds(5),
      environment: {
        APPCONFIG_APPLICATION_ID: application.ref,
        APPCONFIG_ENVIRONMENT: environment.name,
        APPCONFIG_CONFIGURATION_ID: configurationProfile.ref
      },
});

lambdaFunction.addLayers(
  LayerVersion.fromLayerVersionArn(
    this,
    'AppConfigExtension',
    'arn:aws:lambda:us-east-1:027255383542:layer:AWS-AppConfig-Extension:128'
  )
);

lambdaFunction.role?.attachInlinePolicy(
  new Policy(this, 'PermissionsForAppConfig', {
    statements: [
      new PolicyStatement({
        actions: [
          'appconfig:StartConfigurationSession',
          'appconfig:GetLatestConfiguration',
        ],
        resources: ['*'],
      }),
    ],
  })
);

Now, we should be able to access localhost:2772 from the lambda handler function and get the configuration of the feature flag. The code snippet below shows a very basic implementation. In the request URL, we need to specify the AppConfig configurations we exposed as the environment variables to fetch the configuration from the correct application, configuration profile and the environment we want.

import { Handler } from 'aws-cdk-lib/aws-lambda';

export const handler: Handler = async (event: any) => {
  const applicationId = process.env.APPCONFIG_APPLICATION_ID;
  const environment = process.env.APPCONFIG_ENVIRONMENT;
  const configurationId = process.env.APPCONFIG_CONFIGURATION_ID;

  const url = `http://localhost:2772/applications/${applicationId}/environments/${environment}/configurations/${configurationId}`;

  try {
    const response = await fetch(url);
    const responseData = await response.json();

    console.log('data:', JSON.stringify(responseData));
  } catch (error) {
    console.log(console.error);
  }
};

Finally, when we test the lambda function from the AWS console, we can see that the AppConfig agent has started and the configuration values have fetched successfully from the CloudWatch logs.

The full implementation of this can be found on this GitHub repo.

Conclusion

In conclusion, adopting feature flags, particularly through AWS AppConfig, empowers software teams to achieve a smooth and adaptive software evolution. The advantages of feature flags, such as controlled rollout, risk mitigation, and dynamic configuration, are effectively harnessed using AppConfig, offering real-time control and seamless integration with AWS services.

Different integration options underscore AppConfig’s flexibility in catering to diverse development environments. Altogether, leveraging feature flags on AWS AppConfig enhances development agility, allowing teams to respond dynamically to evolving requirements and user feedback, ultimately fostering a resilient and user-centric software evolution.

Building WebSocket APIs with AWS API Gateway and CDK

Sidath Munasinghe — Sat, 21 Oct 2023 12:18:30 +0000

Introduction

In the continuously evolving landscape of web applications, real-time communication has become the gold standard for creating engaging and interactive user experiences. Whether you’re building a chat application, a collaborative online game, or a live dashboard, WebSocket technology has emerged as the enchanting solution that makes real-time magic happen. And when it comes to unleashing the full potential of WebSockets in a serverless and scalable manner, Amazon Web Services (AWS) has a spell of its own — AWS API Gateway.

In this article, we will go through the fundamental concepts of WebSocket API in Amazon API Gateway and create a small application to evaluate its capabilities using the AWS Cloud Development Kit (CDK).

Understanding WebSocket APIs

WebSocket is a protocol that allows full duplex communication between a client and server using a persistent connection for continuous communication. Since the WebSocket connection is persistent, it allows extremely fast data transmission.

Unlike the short-lived stateless connections in HTTP, WebSocket leverages long-lived opened connections to exchange messages with each other independently. This feature allows WebSocket connections to go beyond the traditional request and response model, making it the ideal solution for use cases like chat applications, live broadcasts, and extremely fast data synchronization.

However, when implementing a WebSockets API, some of the crucial areas that we need to pay extra attention to are,

connection management: manage incoming connections appropriately when clients connect and disconnect
security: ensure protection of data in transit
message routing: determine how WebSocket messages will be routed to the appropriate handlers on the server side
monitoring and logging: set up monitoring and logging to keep track of WebSocket connection status, performance, and potential issues

AWS API Gateway for WebSocket APIs

Amazon API Gateway is a fully managed service that makes it easy for developers to create, publish, maintain, monitor, and secure APIs at any scale. Apart from the HTTP-based restful APIs, it allows creating and managing WebSocket APIs as well by simplifying the areas we discussed above while allowing developers to focus on the core functionality of the real-time applications. Here are the key benefits of using AWS API Gateway for building your next WebSocket API.

Abstracts out the protocol understanding: AWS API Gateway abstracts much of the WebSocket protocol details, making it easier to use. It manages the handshaking process and connection handling, allowing you to focus on application logic.
Simplified infrastructure management: It takes care of the WebSocket server setup and configuration, eliminating the need for you to manage the WebSocket server infrastructure.
Integration options: API Gateway can be integrated with multiple backend sources such as lambda functions, DynamoDB tables, or HTTP endpoints to implement any logic based on the content of the messages it receives from the client.
Security: Access can be controlled by AWS IAM or lambda authorizers to implement your authorization logic. Further, it supports WSS (WebSocket Secure) to have encrypted connections for enhanced security, like protecting against man-in-the-middle attacks.
Easy message routing: AWS API Gateway permits you to define routes for WebSocket messages and map them to specific AWS Lambda functions, enabling you to handle them as required.
Monitoring and Logging: AWS CloudWatch Logs and Metrics monitors the WebSocket APIs created with AWS API Gateway, allowing us to monitor its statuses and get alerts when it needs our attention to investigate any critical issue.

Basics of AWS API Gateway for WebSocket APIs

AWS API Gateway service utilizes its fundamental concepts, such as models, authorizers, and stages for WebSocket APIs as well. Below are the additional concepts we need to know when using WebSocket APIs.

Routes

In a WebSocket API, incoming JSON messages are directed to backend integrations based on routes you configure, similar to the endpoints in RestFul APIs. API Gateway provides three predefined routes as below.

$connect: Triggers when a persistent connection between the client and a WebSocket API is initiated.
$disconnect: Triggers when the client or the server disconnects from the API.
$default: Triggers if the route selection expression cannot be evaluated against the message or if no matching route is found. This is useful for handling invalid messages gracefully.

We can also create custom routes to cater to the business requirements and functionalities. To handle incoming requests, these routes must be connected with available backend integrations, such as Lambda functions, DynamoDB tables, etc.

In WebSocket-based communications, connection IDs are very important since they need to be presented to exchange messages. The creation of connection IDs is handled by API Gateway for us, but implementing the logic to handle those events is up to us. For example, we can integrate a lambda function to the $connect route to implement some logic to store the newly created connection ID in a datastore for functionality like broadcasting an event. Also, we will need to delete those entries by implementing some logic behind the $disconnet route.

One-way/Two-way Communication

WebSockets are bidirectional by nature. However, AWS API Gateway allows us to configure one-way or two-way communication routes. If configured for one-way, there won’t be any response after completing the message processing. When two-way communication is enabled, we can implement some logic to receive acknowledgements after complete processing.

Route Selection Expression

API Gateway uses the route selection expression to determine which route to invoke when a client sends a message. When creating a WebSocket API, we must provide the route selection expression. Therefore, before creating the API, we must decide what the message structure looks like. For example, we can use an attribute called action in the message to specify the route like below.

{
   "action": "createTodo",
   "data": {
      "name": "Create WebSocket API",
      "completed": false
   }
}

In this scenario, we can use $request.body.action as the route selection expression. Here $request.body refers to the message payload. We can choose an appropriate attribute we want based on the message structure.

Building the WebSocket API with CDK
Now let’s create a very simple WebSocket API using AWS CDK with Typescript to define and deploy the infrastructure and apply the concept we just learned. We will also add a custom route to send a message to a connection so that we can play around and test it. Below are the main steps we have to follow.

Create three lambda functions to handle new WebSocket connections/disconnections and send a message to a connection.
Create a WebSocket API
Create a custom route for sending a message
Integrate lambda functions to routes
Grant any required permissions
Test the API

Lambda functions

Let’s first create the three lambda functions so we can directly integrate them when creating the WebSockets API routes. The below code snippet shows how to create the mentioned Lambda functions using the NodejsFunction construct in CDK.

// Lambda function to handle new connections
const wsConnectLambda = new NodejsFunction(this, 'ws-connect-lambda', {
      entry: join(__dirname, '../src/connect/index.ts'),
      handler: 'handler',
      functionName: 'connect-lambda',
      runtime: Runtime.NODEJS_18_X,
});

// Lambda function to handle disconnections
const wsDisconnectLambda = new NodejsFunction(
      this,
      'ws-disconnect-lambda',
      {
        entry: join(__dirname, '../src/disconnect/index.ts'),
        handler: 'handler',
        functionName: 'disconnect-lambda',
        runtime: Runtime.NODEJS_18_X,
      }
);

// Lambda function to handle sending messages
const sendMessageLambda = new NodejsFunction(this, 'send-message-lambda', {
      entry: join(__dirname, '../src/send-message/index.ts'),
      runtime: Runtime.NODEJS_18_X,
      functionName: 'send-message',
      handler: 'handler',
});

Now, let’s review the handler function implementations for each function. The below code snippet shows the implementation of the connection handler function.

Connection Handler Function

export const handler = async (event: APIGatewayProxyEvent) => {
  const connectionId = event.requestContext.connectionId;
  console.log('connection created:', connectionId);
  return { statusCode: 200, body: 'Connected.' };
};

We only log the generated connection ID when a new client connects with the API to keep this as simple as possible. If you have any sophisticated logic, we can implement them here.

Disconnection Handler Function

export const handler = async (event: APIGatewayProxyEvent) => {
  const connectionId = event.requestContext.connectionId;
  console.log('Disconnected:', connectionId);
  return { statusCode: 200, body: 'Disconnected.' };
};

We also only log the connection ID when a connection terminates for simplicity. We can extend this implementation to match the business requirements.

Send Message Handler Function

We use this function to send messages to another connection. Below is the structure of the message format we are going to use.

{
   "action":"sendMessage",
   "connectionId":"<receiver's connection id>",
   "message":"<message content>"
}

Below is the definition of these attributes.

action: Acts as the route selection attribute
connectionId: Contains the receiver’s connection ID
message: Message content

With this information, we can implement the below logic inside this lambda function to handle this requirement.

export const handler = async (event: APIGatewayProxyEvent) => {
  const apigwManagementApi = new ApiGatewayManagementApi({
    apiVersion: '2018-11-29',
    endpoint:
      event.requestContext.domainName + '/' + event.requestContext.stage,
  });

  const eventBody = JSON.parse(event.body ?? '');
  const connectionId = eventBody.connectionId;
  const message = eventBody.message;

  console.log('Payload:', { ConnectionId: connectionId, Data: message });

  try {
    await apigwManagementApi
      .postToConnection({ ConnectionId: connectionId, Data: message })
      .promise();
  } catch (error) {
    console.error('Error sending message:', error);
  }
  return { statusCode: 200, body: 'Message sent' };
};

Here, we get the connectionId and the message from the event body and then use AWS API Gateway Management API to post the received message to the received connection.

WebSocket API

Since the lambda functions are ready now, let’s create the WebSockets API. The below code snippet demonstrates how to create it.

// Create WebSocket API with connection/disconnection route integrations
const webSocketApi = new apigw2.WebSocketApi(
      this,
      'my-first-websocket-api',
      {
        connectRouteOptions: {
          integration: new WebSocketLambdaIntegration(
            'ws-connect-integration',
            wsConnectLambda
          ),
        },
        disconnectRouteOptions: {
          integration: new WebSocketLambdaIntegration(
            'ws-disconnect-integration',
            wsDisconnectLambda
          ),
        },
      }
);

// Create API stage
const apiStage = new apigw2.WebSocketStage(this, 'dev', {
      webSocketApi,
      stageName: 'dev',
      autoDeploy: true,
});

// Add the custom sendMessage route
webSocketApi.addRoute('sendMessage', {
      integration: new WebSocketLambdaIntegration(
        'send-message-integration',
        sendMessageLambda
      ),
});

We first create the WebSocket API using the WebSocketApi CDK construct in the above code. Since our connection/disconnection lambda functions are ready, we have also integrated those while creating the API. Then, we defined a dev stage for the API as usual in a typical API in AWS API Gateway. Finally, we have added our custom route for the message-sending purpose. Note that since we are using sendMessage as the action in the message payload, we need to mention the same here as well when adding the route. Further, we don’t need to mention the route selection expression here, since we can use the default value ($request.body.action) provided by CDK construct.

There is one last thing missing in this setup. Our send message lambda function needs permission to post messages to connections. So we need to add the below code segment as well to grant those permissions.

// Get the resource ARN of the created WebSocket API
const connectionsArns = this.formatArn({
      service: 'execute-api',
      resourceName: `${apiStage.stageName}/POST/*`,
      resource: webSocketApi.apiId,
});

// Attach the required policy to the relavant lambda function
sendMessageLambda.addToRolePolicy(
      new PolicyStatement({
        actions: ['execute-api:ManageConnections'],
        resources: [connectionsArns],
      })
);

Now everything is complete, and we can deploy the infrastructure to AWS using the cdk deploy command.

Testing

Once the deployment is complete, we can get the API’s WebSocket URL with wssprotocol from the API Gateway console. For testing purposes, we can use any WebSocket testing tool like piesocket to connect and send messages.

We can give our WebSocket API URL to this tool to test the connection creation. We should be able to see the connection ID in the Cloudwatch logs of connection handling lambda. We can connect with as many clients as we want and track their connection IDs to send messages.

Then, we can use these connection IDs to send messages to each other using the custom route we created.

Since the message sending is working as expected, now we can try the disconnection handler’s functionality. When a client disconnects, we should be able to see the appropriate log as well.

So everything is working as expected, and it takes very little time to implement using the capabilities of AWS API Gateway. We can enhance the security of the API by introducing authorizers and access control mechanisms. Further, you can use CloudWatch to monitor the service status using different metrics, such as the connection count.

The full source code for this example application can be found in this GitHub repo.

Conclusion

From the introduction to the basics of WebSocket APIs, we’ve navigated through the realm of real-time web development. AWS API Gateway stood as our gateway to real-time magic, simplifying the complexities and providing security and scalability. Armed with this knowledge, your journey into real-time web development begins, where you can create captivating applications with the help of AWS CDK. Embrace the magic, and let your applications come alive in the world of real-time experiences!

DEV Community: Sidath Munasinghe

Efficient Nested Resolvers in AWS AppSync with Lambda Batching

Understanding the N+1 Problem in AppSync

What Is the N+1 Problem?

Why Is This a Problem?

Solving N+1 with Batch Resolvers in AppSync

How Batch Resolvers Work

Enabling Batch Resolvers

Conclusion

Efficient Nested Resolvers in AWS AppSync with Lambda Batching

Understanding the N+1 Problem in AppSync

What Is the N+1 Problem?

Why Is This a Problem?

Solving N+1 with Batch Resolvers in AppSync

How Batch Resolvers Work

Enabling Batch Resolvers

Conclusion

Creating Smart AI Agents with AWS Bedrock

Introduction

Understanding AI Agents

Why Use AWS Bedrock for AI Agents?

Components of an AI Agent

Foundational model

Instructions

Action Groups

Knowledge bases

Prompt Templates

Creating Your First AI Agent

Demo

Advanced Features

Conclusion

Unleashing the Power of CDK and Terraform in Cloud Deployments

Introduction

What is Terraform and CDK?

Terraform

AWS CDK

Benefits of Using Terraform and CDK Together

Setting up a Project

Building a REST API

Conclusion

Generative AI on AWS with Amazon Bedrock

Foundation Models

Custom Models

Knowledge Bases

Agents

Read more

Deploy Kubernetes in Minutes: Effortless Infrastructure Creation and Application Deployment with Cluster.dev and Helm Charts

Complexities with Kubernetes

Setting up a Jenkins Service with Cluster.dev and Helm on Kubernetes

Conclusion

Further Reading

Streamlining SonarQube on AWS ECS: Simplified Deployment Using Cluster.dev

Cluster.dev Basics

Implementing the Infrastructure

Deploying the Infrastructure

Conclusion

Revolutionizing Infrastructure Management with Cluster.dev: A Journey into Effortless Orchestration

What is IaC, and Why is it Important?

Challenges in the Current IaC Tools

What Cluster.dev Brings to the Current IaC Landscape?

Orchestrating Diverse Tools

Templating for Consistency

Multi-Cloud Deployments

Template Sharing and Customization

Simplifying SaaS Deployments

Enforce Best Practices and Standards

Creating a Serverless API with Cluster.dev

When to Use Cluster.dev?

When your development teams lack knowledge in IaC

When you want to standardize your infrastructure

When you are in a multi-cloud environment

When you are on a microservices architecture

When you want to deploy repeatable complex infrastructure

When you want to benefit from several IaC tools

Conclusion

Harnessing Feature Flags on AWS AppConfig for Seamless Software Evolution

Advantages of Feature Flags

AppConfig Feature Flags

Data Retrieval in AppConfig Feature Flags

Integration Options