DEV Community

Cover image for Build Serverless Generative AI API Service with AWS Lambda and Amazon Bedrock
Jimmy for AWS Community Builders

Posted on • Edited on

Build Serverless Generative AI API Service with AWS Lambda and Amazon Bedrock

In this article, we'll guide you through the process of constructing a Generative AI API Service using AWS Serverless Application Model (SAM). We'll then demonstrate how to deploy it to AWS Lambda, leveraging the robust capabilities of Amazon Bedrock for a powerful and scalable solution.

We will create an API endpoint like this

https://hwnfnespxzpw7pkz2ydjuwskpq0fokxv.lambda-url.ap-southeast-1.on.aws/
Enter fullscreen mode Exit fullscreen mode

and then send POST request to that endpoint with our prompt question explain to me about AWS Lambda? as payload

curl -k \
> -X POST \
> -H 'Content-Type: application/json' \
> -d '{"prompt":"explain to me about AWS Lambda"}' \
> https://hwnfnespxzpw7pkz2ydjuwskpq0fokxv.lambda-url.ap-southeast-1.on.aws/
Enter fullscreen mode Exit fullscreen mode

Then get a response like this generated by Amazon Bedrock

{"message":"\nAWS Lambda is a serverless computing service provided by Amazon Web Services (AWS). It allows users to run code in response to events, without the need for them to provision or manage servers. With Lambda, you can create and run code without thinking about servers.\n\nLambda is designed to run code in response to events, such as a file in an Amazon Simple Storage Service (Amazon S3) bucket being modified, or a table being updated in an Amazon DynamoDB table. Lambda automatically manages the computing resources required to execute the functions, and scales them as needed.\n\nTo use Lambda, you first create a function by writing code in Node.js, Java, Python, or Ruby. You can then configure triggers that invoke the function when an event occurs. Lambda also allows you to configure how much memory and time the function should use when it is invoked.\n\nLambda functions can run for a maximum of 15 minutes, and are charged based on a pricing model that calculates the amount of time and resources consumed by the function when it is invoked. This means that you only pay for the actual compute time used by your function, and not for idle time.\n\nOverall, Lambda is a powerful tool that allows you to run code in the cloud without having to worry about the underlying infrastructure."}
Enter fullscreen mode Exit fullscreen mode

Follow me for more

fullstacksaiyan; | Twitter, Instagram, Facebook, TikTok | Linktree

Share about Web, Mobile, Backend, Frontend, and Cloud Computing.

favicon linktr.ee

Amazon Bedrock

Amazon Bedrock is a fully managed service from Amazon Web Services that provides access to foundation models from Amazon and select third-party providers through a simple API. It allows customers to easily choose from a wide range of foundation models to find the one best suited for their use case. With Amazon Bedrock, customers can quickly get started with these models, privately customize them using their own data, and integrate them into applications. The service handles all infrastructure management so customers can focus on using the models without having to worry about deployment or maintenance. Amazon Bedrock also integrates with other Amazon Web Services like Amazon SageMaker for capabilities such as model testing and lifecycle management.

Table of Contents

  1. Prerequisites
  2. Enable Amazon Bedrock Model Access
  3. Prepare AWS SAM Project
  4. Prepare Lambda function code
  5. Deploy your SAM project
  6. Cleanup

Prerequisites Tools

Enable Amazon Bedrock Model Access

In order to utilize the Amazon Bedrock foundation model, you must activate model access, which is currently exclusively accessible in the us-east-1 (N. Virginia) region through the AWS Management Console.

amazon bedrock console

Check list the model that you want to use and then click Save changes
edit model access

Prepare AWS SAM Project

To prepare your AWS SAM project, start by creating a lambda-bedrock folder. Inside lambda-bedrock folder then create your AWS SAM template template.yaml.

mkdir lambda-bedrock
cd lambda-bedrock
touch template.yaml
Enter fullscreen mode Exit fullscreen mode

These commands create a new directory called lambda-bedrock, navigate into that directory, and create a new file called template.yaml. The mkdir command creates the directory, the cd command navigates into that directory, and the touch command creates the new file. This is a common set of commands used to create a new project directory and file.

Then define your resources inside template.yaml

AWSTemplateFormatVersion: "2010-09-09"
Transform: AWS::Serverless-2016-10-31
Description: >
  lambda-bedrock

  Sample SAM Template for lambda-bedrock


Resources:
  BedrockFunction:
    Type: AWS::Serverless::Function
    Region: us-east-1
    Properties:
      Timeout: 30
      CodeUri: src/
      Handler: app.handler
      Runtime: nodejs18.x
      FunctionUrlConfig:
        AuthType: NONE
        InvokeMode: BUFFERED
      Policies:
        - Statement:
            - Effect: Allow
              Action: bedrock:InvokeModel
              Resource: "arn:aws:bedrock:us-east-1::foundation-model/ai21.j2-ultra-v1"

Outputs:
  FunctionUrlEndpoint:
    Description: "Lambda Bedrock URL Endpoint"
    Value:
      Fn::GetAtt: BedrockFunctionUrl.FunctionUrl

Enter fullscreen mode Exit fullscreen mode

This is a CloudFormation template that defines an AWS Lambda function using the AWS::Serverless::Function resource type. The BedrockFunction resource is defined with the specified properties.

The Type field specifies the resource type, which is AWS::Serverless::Function. This resource type is used to define a serverless function that can be executed in response to an event.

The Region field specifies the AWS region where the function will be created. In this case, it is set to us-east-1.

The Properties field contains the properties that define the function. The Timeout property specifies the maximum amount of time that the function can run before it times out. In this case, it is set to 30 seconds.

The CodeUri property specifies the location of the function's code. In this case, it is set to src/, which indicates that the code is located in the src directory.

The Handler property specifies the name of the function's handler method. In this case, it is set to app.handler, which indicates that the handler method is located in the app.js file.

The Runtime property specifies the runtime environment that the function will use. In this case, it is set to nodejs18.x, which indicates that the function will run on Node.js version 18.x.

The FunctionUrlConfig property specifies the configuration for the function's URL. The AuthType property specifies the authentication type that will be used to access the function. In this case, it is set to NONE, which indicates that no authentication is required. The InvokeMode property specifies the mode in which the function will be invoked. In this case, it is set to BUFFERED, which indicates that the function will be invoked in buffered mode.

The BedrockInvokeModelPolicy resource defines an IAM policy that allows invoking a machine learning model. The policy is attached to an existing IAM role and applies to the specified machine learning model arn:aws:bedrock:us-east-1::foundation-model/ai21.j2-ultra-v1.

The Outputs field contains a list of output values that are exported by the CloudFormation stack. In this case, there is only one output value. The FunctionUrlEndpoint output value retrieves the URL of the BedrockFunctionUrl resource.

Prepare Lambda function code

mkdir src
touch src/app.js
cd src
npm init -y
npm install aws-sdk
Enter fullscreen mode Exit fullscreen mode

These commands create a new directory called src, navigate into that directory, create a new file called index.js, initialize a new Node.js project with default settings using npm init -y, and install the aws-sdk package using npm install aws-sdk.

The mkdir command creates the src directory, the touch command creates the app.js file inside the src directory, the cd command navigates into the src directory, the npm init -y command initializes a new Node.js project with default settings, and the npm install aws-sdk command installs the aws-sdk package as a dependency for the project.

Inside app.js

const AWS = require("aws-sdk");

const bedrockRuntime = new AWS.BedrockRuntime({
  apiVersion: "2023-09-30",
  region: "us-east-1",
});

async function invokeModel(prompt) {
  try {
    const response = await new Promise((resolve, reject) => {
      bedrockRuntime.invokeModel(
        {
          modelId: "ai21.j2-ultra-v1",
          contentType: "application/json",
          accept: "*/*",
          body: JSON.stringify({
            prompt: prompt,
            maxTokens: 1000,
            temperature: 0.9,
          }),
        },
        function (err, data) {
          if (err) {
            console.log(err, err.stack);
            reject(err);
          } else {
            const response = JSON.parse(data.body);
            resolve(response.completions[0].data.text);
          }
        }
      );
    });

    return response;
  } catch (err) {
    console.log(err);
  }
}

exports.handler = async (event, context) => {
  const { prompt } = JSON.parse(event.body);

  const response = await invokeModel(prompt);

  try {
    return {
      statusCode: 200,
      body: JSON.stringify({
        message: response,
      }),
    };
  } catch (err) {
    console.log(err);
  }
};
Enter fullscreen mode Exit fullscreen mode

This is a Node.js module that defines an AWS Lambda function that invokes a machine learning model using the BedrockRuntime class from the aws-sdk package. The handler function is exported as the entry point for the Lambda function.

The require function is used to import the aws-sdk package, which provides a JavaScript interface for interacting with AWS services. The BedrockRuntime class is used to invoke the machine learning model.

The invokeModel function is defined to invoke the machine learning model with the specified prompt. The function sends a request to the BedrockRuntime service with the specified parameters, including the model ID, content type, and request body. The function returns the response from the service, which includes the completed text generated by the machine learning model.

The handler function is defined as the entry point for the Lambda function. The function receives an event object and a context object as parameters. The event object contains the request data, including the prompt. The context object contains information about the execution environment.

The handler function parses the prompt from the request data and passes it to the invokeModel function. The function then returns a response object with a status code of 200 and a message containing the completed text generated by the machine learning model.

Deploy your SAM project

sam build
sam deploy --guided --region us-east-1
Enter fullscreen mode Exit fullscreen mode

These commands use the AWS Serverless Application Model (SAM) to build and deploy the Lambda function defined in the app.js file to the us-east-1 region.

The sam build command builds the application and prepares it for deployment. This command uses the template.yaml file to determine the build configuration.

The sam deploy command deploys the application to AWS. The --guided option prompts the user for configuration settings, such as the stack name and deployment bucket (you only need this for the first time deployment, for subsequent deploy just run sam deploy). The --region option specifies the AWS region where the application will be deployed.

You will be prompt with this input

        Setting default arguments for 'sam deploy'
        =========================================
        Stack Name [lambda-bedrock]:
        AWS Region [us-east-1]:
        #Shows you resources changes to be deployed and require a 'Y' to initiate deploy
        Confirm changes before deploy [Y/n]:
        #SAM needs permission to be able to create roles to connect to the resources in your template
        Allow SAM CLI IAM role creation [Y/n]:
        #Preserves the state of previously provisioned resources when an operation fails
        Disable rollback [y/N]:
        BedrockFunction Function Url has no authentication. Is this okay? [y/N]: y
        Save arguments to configuration file [Y/n]:
        SAM configuration file [samconfig.toml]:
        SAM configuration environment [default]: dev
Enter fullscreen mode Exit fullscreen mode

Then followed by this Initiating deployment confirmation

Initiating deployment
=====================

        Uploading to lambda-bedrock/5dd89a16d669f1bc7ef501c1a740d0fb.template  1185 / 1185  (100.00%)


Waiting for changeset to be created..

CloudFormation stack changeset
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------    
Operation                                                 LogicalResourceId                                         ResourceType                                              Replacement
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------    
+ Add                                                     BedrockFunctionRole                                       AWS::IAM::Role                                            N/A
+ Add                                                     BedrockFunctionUrlPublicPermissions                       AWS::Lambda::Permission                                   N/A
+ Add                                                     BedrockFunctionUrl                                        AWS::Lambda::Url                                          N/A
+ Add                                                     BedrockFunction                                           AWS::Lambda::Function                                     N/A
+ Add                                                     BedrockInvokeModelPolicy                                  AWS::IAM::Policy                                          N/A
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------    


Changeset created successfully. arn:aws:cloudformation:ap-southeast-1:498624870591:changeSet/samcli-deploy1696242138/3e3a06d0-33c5-46cb-ad69-56edea2d1321


Previewing CloudFormation changeset before deployment
======================================================
Deploy this changeset? [y/N]:
Enter fullscreen mode Exit fullscreen mode

After your deployment completed, you will get output like this

CloudFormation outputs from deployed stack
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------   
Outputs
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------   
Key                 FunctionUrlEndpoint
Description         Lambda Bedrock URL Endpoint
Value               https://hwnfnespxzpw7pkz2ydjuwskpq0fokxv.lambda-url.ap-southeast-1.on.aws/
Enter fullscreen mode Exit fullscreen mode

You can then take that Lambda endpoint and test sending your prompt like this

curl -k \
> -X POST \
> -H 'Content-Type: application/json' \
> -d '{"prompt":"explain to me about AWS Lambda"}' \
> https://hwnfnespxzpw7pkz2ydjuwskpq0fokxv.lambda-url.ap-southeast-1.on.aws/
Enter fullscreen mode Exit fullscreen mode

Then you will get response from your Lambda like this

{"message":"\nAWS Lambda is a serverless computing service provided by Amazon Web Services (AWS). It allows users to run code in response to events, without the need for them to provision or manage servers. With Lambda, you can create and run code without thinking about servers.\n\nLambda is designed to run code in response to events, such as a file in an Amazon Simple Storage Service (Amazon S3) bucket being modified, or a table being updated in an Amazon DynamoDB table. Lambda automatically manages the computing resources required to execute the functions, and scales them as needed.\n\nTo use Lambda, you first create a function by writing code in Node.js, Java, Python, or Ruby. You can then configure triggers that invoke the function when an event occurs. Lambda also allows you to configure how much memory and time the function should use when it is invoked.\n\nLambda functions can run for a maximum of 15 minutes, and are charged based on a pricing model that calculates the amount of time and resources consumed by the function when it is invoked. This means that you only pay for the actual compute time used by your function, and not for idle time.\n\nOverall, Lambda is a powerful tool that allows you to run code in the cloud without having to worry about the underlying infrastructure."}
Enter fullscreen mode Exit fullscreen mode

This response is generated by Amazon Bedrock from the prompt you send to your Lambda endpoint.

Cleanup

To delete all your resources that created using SAM Template, you can run this command

sam delete --region us-east-1
        Are you sure you want to delete the stack lambda-bedrock in the region us-east-1 ? [y/N]: y
        Do you want to delete the template file fbc4997de651419e130644091ddcceb5.template in S3? [y/N]: y
        - Deleting S3 object with key 3ba0b8c3b1f20b100cab919b0cf2291a
        - Deleting S3 object with key fbc4997de651419e130644091ddcceb5.template
        - Deleting Cloudformation stack lambda-bedrock
Enter fullscreen mode Exit fullscreen mode

This command will cleanup all the resources for you.

Check out my previous post

Top comments (1)

Collapse
 
sevimsoffice profile image
SevimsOffice

This is amazing work thanks for sharing Jimmy!