DEV Community: Rushi Patel

Mastering Dynamic Response Streaming Lambda with AWS Bedrock and TypeScript

Rushi Patel — Wed, 22 Nov 2023 20:33:46 +0000

In the dynamic landscape of serverless architecture, where efficiency and real-time responsiveness reign supreme, the concept of response streaming emerges as a pivotal player. Unlike traditional request-response models, response streaming allows for the incremental transmission of data, enabling a continuous flow of information from the server to the client.

Imagine a scenario where large datasets need to be processed and delivered to end-users in real-time, or where updates to a client application should be instantaneously reflected. Response streaming transforms the serverless paradigm by facilitating the progressive delivery of results, enhancing user experience, and optimizing resource utilization.

In this comprehensive guide, we’ll walk you through the intricacies of setting up AWS Bedrock with AWS CDK, crafting a TypeScript Lambda function for response streaming, and seamlessly integrating it all with AWS CDK. By the end of this journey, you’ll be well-equipped to enhance your serverless applications with unparalleled efficiency and responsiveness.

How to Setup Bedrock with AWS CDK

Check out my previous blog on “Unleashing the Power of Generative AI with AWS Bedrock, AWS CDK, TypeScript” for a comprehensive guide on setting up Bedrock with AWS CDK.

Example Lambda for Leveraging Response Streaming

Lambda response streaming allows your functions to send back a response in chunks, enabling real-time updates and more efficient handling of large datasets. This feature is especially powerful in scenarios where immediate and incremental processing of data is crucial.

Let’s dive into a TypeScript example that showcases the power of response streaming

import {
  BedrockRuntimeClient,
  InvokeModelWithResponseStreamCommand,
} from "@aws-sdk/client-bedrock-runtime";

import { streamifyResponse } from "lambda-stream";

import * as stream from 'stream';
import * as util from 'util';

const { Readable } = stream;
const pipeline = util.promisify(stream.pipeline);

function parseBase64(message: string) {
  return JSON.parse(Buffer.from(message, "base64").toString("utf-8"));
}

const client = new BedrockRuntimeClient({
  region: "us-west-2",
});

export const handler = streamifyResponse(
  async (event, responseStream, _context) => {

    const prompt = 'Can you please what is Pure Function in React?'

    const claudPrompt = `Human: Human:${prompt} Assistant:`;

    const params = {
      modelId: "anthropic.claude-v2",
      contentType: "application/json",
      accept: "*/*",
      body: `{"prompt":"${claudPrompt}","max_tokens_to_sample":2048,"temperature":0.5,"top_k":250,"top_p":0.5,"stop_sequences":[], "anthropic_version":"bedrock-2023-05-31"}`,
    };

    console.log(params);

    const command = new InvokeModelWithResponseStreamCommand(params);

    const response: any = await client.send(command);
    const chunks = [];

    for await (const chunk of response.body) {
      const parsed = parseBase64(chunk.chunk.bytes);
      chunks.push(parsed.completion);
      responseStream.write(parsed.completion);
    }

    console.log(chunks.join(""));
    responseStream.end();
  }
);
export default handler;

How to Configure Lambda Function URL as Response Stream in AWS CDK

Now that we have a TypeScript Lambda function capable of response streaming, the next step is to seamlessly integrate it with AWS CDK. This integration allows for better management and deployment of the serverless infrastructure.

const bedrockResponseStreamingLambda = new aws_lambda_nodejs.NodejsFunction(
      this,
      "BedrockResponseStreamingLambda",
      {
        runtime: aws_lambda.Runtime.NODEJS_16_X,
        handler: "handler",
        entry: path.join(__dirname, "../src/lambda/bedrock-repsone-streaming/index.ts"),
        bundling: {
          forceDockerBundling: false,
          nodeModules:['lambda-stream']
        },
        timeout: Duration.seconds(90),
      }
    );

const lambdaUrl = bedrockResponseStreamingLambda.addFunctionUrl({
      authType: aws_lambda.FunctionUrlAuthType.NONE,
      invokeMode: aws_lambda.InvokeMode.RESPONSE_STREAM
    });

new CfnOutput(this,'LambdaEndpoint',{
      value:lambdaUrl.url
    })

Testing the Configured Lambda Function

Before diving into testing, ensure you’ve deployed your infrastructure by running the cdk deploy command in your terminal.

To make testing even more straightforward, check out this video demonstration that walks you through the process of testing your Lambda function

For those who prefer the command line, you can use the AWS CLI along with CURL to test your Lambda function. Here’s an example CURL command:

curl — request GET “lambda_url" — user AWS_ACCESSS_KEY:AWS_SECRET_KEY — aws-sigv4 “aws:amz:us-west-2:execute-api”

Conclusion

As we wrap up our exploration of AWS Bedrock and Lambda Response Streaming, it’s evident that the synergy between these technologies can elevate your serverless architecture to new heights. By following the steps outlined in this blog, you’ve empowered your applications with enhanced efficiency and dynamic response capabilities.

Checkout my Github repository

Happy Coding :)

Secure Machine-to-Machine OAuth 2.0 Authentication Integration with AWS Cognito, CDK, and API Gateway

Rushi Patel — Fri, 28 Jul 2023 15:42:52 +0000

In today's interconnected world, machines often communicate with each other to exchange data. To protect sensitive information and maintain system integrity, it's crucial to have a secure authentication mechanism for these interactions. In this blog post, we'll explore how to achieve secure machine-to-machine authentication using AWS Cognito, AWS CDK, and API Gateway with a simple and easy-to-understand approach.

What is AWS Cognito?

AWS Cognito is a fully managed authentication service by Amazon Web Services. It allows us to add user sign-up, sign-in, and access control to our applications. With Cognito, we can easily create and manage user pools, which act as directories for authentication and authorization purposes. It's a powerful tool to manage user identities and permissions effectively.

Why Use this Mechanism Over IAM or Other M2M Authentication Mechanisms ?

While AWS provides several authentication mechanisms for machine-to-machine (M2M) communication, using AWS Cognito, CDK, and API Gateway offers distinct advantages that make it a preferable choice over other methods like IAM, API keys, mTLS, and EC2 instance profiles.

Fine-Grained Access Control: AWS Cognito, combined with custom scopes, allows for fine-grained access control. We can define specific permissions for different types of machines, granting them access only to the necessary resources. In contrast, IAM provides broader permissions at the user or role level, which might not be as granular as required for M2M communication.
Scalability and Flexibility: The integration of AWS Cognito, CDK, and API Gateway offers a scalable and flexible solution for M2M authentication. Managing client credentials within the Cognito User Pool allows us to easily add or revoke machine access, adapting to changing requirements. This dynamic control over access is not as straightforward with IAM or API keys.
OAuth 2.0 Integration: By incorporating OAuth 2.0 with AWS Cognito, we enable third-party applications to access our resources on behalf of users, enhancing the versatility of our M2M communication. This is particularly valuable when external services need secure access to specific resources while adhering to our authentication policies.
Centralized User Directory: AWS Cognito acts as a centralized user directory, facilitating efficient user and machine management. This becomes particularly beneficial when multiple applications and services need access to the same user pool. IAM, API keys, and instance profiles lack the user directory capabilities of AWS Cognito, making it less suitable for managing M2M communication across various services.
Reduced Risk with Client Credentials: The use of unique client credentials (client ID and client secret) in the OAuth 2.0 flow reduces the risk of unauthorized access. Unlike API keys, which can be more static and shared, client credentials are tied to specific machines, and they can be easily rotated to maintain a higher level of security.

In conclusion, the combination of AWS Cognito, CDK, and API Gateway provides a powerful and secure mechanism for machine-to-machine authentication. The solution offers fine-grained access control, scalability, flexibility, and built-in token validation, making it a preferred choice over IAM, API keys, mTLS, and instance profiles. By integrating OAuth 2.0, we enhance the versatility of our M2M communication while maintaining centralized user management and ensuring robust security for resources.

Setting up AWS CDK for Infrastructure as Code

To ensure a repeatable and scalable deployment process, we'll use AWS CDK as Infrastructure as Code. CDK allows you to define your cloud infrastructure using familiar programming languages like TypeScript, Python, or Java. Let's quickly go through the setup process:

Install AWS CDK CLI: Follow the official documentation to install the AWS CDK CLI on your local machine.
Initialize a New CDK Project: Create a new directory for your project and run below command to initialize a new CDK project. Here we're using typescript as a programming language.

cdk init app - language [csharp|fsharp|go|java|javascript|python|typescript]

Install Necessary Dependencies: Install the required AWS SDK and any other dependencies using npm or yarn.

Creating an AWS Cognito User Pool with Custom Scopes

To tailor the access permissions for our APIs, we'll create an AWS Cognito User Pool with custom scopes. Custom scopes allow us to define specific permissions, such as "read" or "write," for different types of users as per your requirement. By setting up custom scopes, we can easily manage access control for our machine-to-machine communication.

Here is sample code to create UserPool and custom scopes. We've created two scopes "user.read" and "user.write"

Now, we'll be attached this above scopes to resource server as below

Implementing Machine to Machine Authentication

Machine to machine communication often involves server-to-server interactions, where authentication is done using credentials specific to the machines rather than user credentials.

Generate Client Credentials: Within the Cognito User Pool, generate unique client credentials (client ID and client secret) for your machines.
Use Client Credentials: Securely store these credentials on your machines and use them to obtain access tokens from Cognito for subsequent API requests. You can get access token from this access point and read documentation here that how we can get the access token.

All together we configured while we're creating client for the UserPool and you can customise below options as per your need. Moreover, we defined all scopes here as oAuth.

Also, we configured domain for userpool, you have two option either Cognito Domain or Custom Domain. Here we set prefix text in the domain url.

Note: Remember the domain name should be unique all across AWS accounts.

Implementing API Gateway and Lambda

With AWS Cognito and OAuth 2.0 set up, we'll implement API Gateway to act as the entry point for our machine-to-machine communication. We'll create a Lambda function that returns a simple response to validate the authentication process.

Our Lambda is ready for rock and roll. We'll define our API Gateway (REST API)and resource(user).

We are going to create a new Authorizer within API Gateway, in this case CognitoUserPoolAuthorizer, with an attached UserPool that we generated before.

We have now added the GET method to the user resource in order to retrieve data from our lambda. Along with that, we added the appropriate scope as needed for this method and defined the Cognito authorizer and authorization type.

Deploying the Solution with AWS CDK

With all the components set up, it's time to deploy the complete solution using AWS CDK. CDK simplifies the deployment process by creating or updating the necessary resources based on the code we defined. This ensures consistency across deployments and reduces the chances of manual errors.

Deploy the Stack: Run cdk deploy to deploy the entire infrastructure, including AWS Cognito User Pool, API Gateway with Cognito authorizer, and any other necessary resources.

Testing

Now we're ready to test whole machine-to-machine authentication using AWS Cognito and API Gateway

You need to get your client id and client secret from aws console. It's very sensitive

To test our machine-to-machine authentication, we'll first get an access token using client credentials. This token represents the machine's identity and permissions.

curl - location 'https://sample-oauth.auth.eu-west-2.amazoncognito.com/oauth2/token' \ - header 'Content-Type: application/x-www-form-urlencoded' \ - header 'Authorization: Basic NuRnbjBjNWFycWYxMXFxbTNzOGtmJnYzZ2NjEzdWtxbGNuYmNqZTdgxoiqjBuc2hhNHRxdWJwJiiXVsdmw2NYQwcDHHmNhOWnmvbIFkcg==' \ - data-urlencode 'grant_type=client_credentials' \ - data-urlencode 'scope=OauthResourceServer/user.read'

In response, you get access token which is look like this

{ "access_token": "access_token_value", "expires_in": 3600, "token_type": "Bearer" }

We'll then use this access token to authenticate our API Gateway requests, ensuring that only authorized machines can access our APIs.

curl - location 'https://7scdlnkrc5.execute-api.eu-west-2.amazonaws.com/prod/user' \ - header 'Authorization: access_token_value'

If you get response as we defined in our lambda then your authorisation is granted if not then you received 401(Unauthorised) in the response.

How to Store the Credentials and How Often They Need to Be Rotated?

Storing machine-to-machine (M2M) credentials securely is crucial to prevent unauthorized access and ensure the integrity of our system. The client credentials, consisting of the client ID and client secret, are used for obtaining access tokens from AWS Cognito. Here’s how we can store and manage these credentials securely:

Use Secure Storage Service: Consider using AWS Secrets Manager or other secure credential management services provided by cloud platforms. These services offer encryption and access controls to safeguard sensitive information.
Credential Rotation Policy: Implement a credential rotation policy to enhance security. Regularly rotate the client secret at predefined intervals. The frequency of rotation depends on your organization’s security requirements and the sensitivity of the data being accessed. A common practice is to rotate credentials every 90 days or as mandated by your security policies.
Secure Communication: Ensure secure communication between machines and AWS Cognito during credential exchange. Use encrypted channels (e.g., HTTPS) when requesting access tokens to prevent eavesdropping or man-in-the-middle attacks.
Limited Access to Credentials: Limit access to the credentials to authorized personnel only. Follow the principle of least privilege, granting access only to those who need it for administrative or maintenance tasks.

Remember that the security of your machine-to-machine communication heavily relies on how well you protect and manage the client credentials. Regularly assess and update your security practices to align with industry best practices and evolving security threats. By employing a robust credential management strategy, you can ensure a secure and reliable M2M authentication mechanism.

Conclusion

In this blog post, we explored how to implement secure machine-to-machine authentication using AWS Cognito, AWS CDK, and API Gateway. By creating an AWS Cognito User Pool with custom scopes, and leveraging AWS CDK for infrastructure as code, we built a robust and scalable authentication mechanism for our machine-to-machine communication. Secure authentication is crucial for protecting sensitive data and ensuring the integrity of our systems in today's interconnected world.

You can check my Github repository for more information.

Happy Coding :)
Cheers

Jump into Datadog With AWS Serverless CDK App

Rushi Patel — Wed, 24 May 2023 09:41:15 +0000

This article will help you to start using DataDog with your AWS Serverless CDK app. Now, we will see some background knowledge of DataDog.

What is Datadog?

Through a SaaS-based data analytics platform, Datadog offers monitoring of servers, databases, tools, and services for cloud-scale applications.

You must first register with DataDog, which offers a free 14-day trial, by clicking this link

You might need to configure IAM Role Policy for DataDog after successfully signing up, both manually and automatically are options.

For Automatic (Recommend), use this link
For Manual, use this link
We’re about to start developing our AWS CDK app in typescript using typescript template. You can use the command below.

mkdir DatadogCDK

cd DatadogCDK

cdk init app — language typescript

The NodejsService function will now be created in the file lib/constructs/nodejs-service-function.ts and used for all of the Lamda functions in our app.
You can use the following code and modify it according to your needs.

Note: You must install the necessary dependencies such as aws-lambda, aws-cdk, middy, esbuild and winston.

We are now prepared to rock ’n’ roll in our primary stack by installing the necessary Datadog dependencies.

npm i datadog-cdk-constructs-v2 datadog-lambda-js

It’s time to write our first lambda, which should generate a single metric in DataDog and then just return an object. This is how your lambda should look.

Now, we’re defining our Datadog with the appropriate site and API key from your Datadog account. Then, for monitoring, we develop a single lambda function and tie it to Datadog. If you want logs of every resource in your AWS account, you can use DataDog Forwarder; more information can be found in the section below this article.

Your stack appears below:

Now that we’ve had DataDog Monitoring, we’re prepared to test our lambda. You must perform the commands listed below to bootstrap and deploy your CDK application:

npx cdk bootstrap

npx cdk deploy

Run your lambda function when it has successfully deployed, and feel free to tweak the code if you wish to see error logs.

Please wait at least 10 minutes after invoking lambda (to allow the data from lambda to Datadog to sync)

I repeatedly called my function using the happy and unhappy paths, and I could clearly see the logs and fascinating information about your lambda functions.

You can also see your metrics logs because our lambda includes coffee.order value. The logs panel allows you to view detailed logs. I’ve provided an example set from my DataDog Dashboard logs here.

If you have the associated services then the logs are also visible for your lambda function and may be seen by exploring other DataDog features. You need to refer to the documentation for that.

We will now have a look at a fascinating lambda function created by DataDog, called DataDog Forwarder.

What is DataDog Forwarder?
An AWS Lambda function called the Datadog Forwarder transports logs, custom metrics, and traces from your environment to Datadog. The Forwarder is able to forward logs of AWS Services like: Cloudwatch, ELB, S3, CloudTrail, SNS, AWS lambda, Kinesis data stream, and many more.

For installation, you need to follow the documentation by DataDog.

Here, You can find my [GitHub](https://github.com/rushi308/cdk-datadog) code if you want to have a look.

That’s it; you now have a foundational understanding of how DataDog fits into your Serverless architecture.

Happy Coding!! Cheers