DEV Community: Andre Lopes

AWS AI - Getting Started With Bedrock

Andre Lopes — Thu, 03 Apr 2025 18:12:36 +0000

Let’s dive into the AWS LLM service and build a serverless AI agent with Lambda and API Gateway WebSockets

Hi People!

Bedrock is the AWS fully managed solution that offers access to AI foundation models from the most well-known companies, such as Anthropic, Cohere, DeepSeek, Meta, Mistral AI, and Amazon, in the cloud. Some of the models are also offered in the serverless mode, like Claud 3.5, which you can use directly without deploying any resources.

Instead of manually building and training the models, Bedrock offers this out of the box, making creating AI applications less complex.

You can find the code for this article under the folder bedrock in this repository.

What to expect?

We’ll be covering:

Bedrock — Cover the basic concepts, invoke and converse with foundation models, and add tool integrations.
Lambdas — We’ll use Go to write our lambdas, which will interact with Bedrock.
API Gateway — Use API Gateway with WebSockets to call our AI application and stream the results from our model.
Terraform — All our infrastructure will be provisioned using Terraform

Requirements

AWS Account
GitHub Account
Your favorite IDE/Text Editor

Basics

Let’s discuss some basic concepts regarding AWS Bedrock

Foundation Models

These AI models were already pre-trained with massive and diverse data, such as DeepSeek-R1, Claude 3.5, Claude 3.7, and others.

Each model has its specializations. For example:

Claude — Text generation, Code generation, Rich text formatting, Agentic computer use
Titan Text G1-Express — Text generation, Code generation, Instruction following
DeepSeek-R1 — Text generation, Code generation

Model catalog

A marketplace-like console where you can select the desired foundation model. It has a range of models, from serverless deployment to models that AWS automatically deploys for you. Each model has its own pricing.

You can also find the pricing for each serverless model here. For the deployed models, you can click on it and the pricing should be available there.

Note that model availability depends on the region. To see the model supported for each region, you can check the documentation here.

Also, please note that some models are only available through cross-region inference. So, when calling Bedrock, you’ll pass the Inference Profile ID instead of the Model ID. And, if you are giving granular access, you’ll need to give InvokeModel access to all regions of the Inference Profile.

Agents

AI applications can interpret an input and decide which actions to take, such as calling an API. Bedrock Agents allows you to create and assign agents to action groups, connect to Lambda functions, access knowledge bases, and even call other agents.

Knowledge Base

A collection of custom data with a defined context that can be given to a foundation model or agent. Internal documentation, manuals, and custom data types can help your agent or model provide better results.

Custom model fine-tuning

It allows you to fine-tune/train a foundation model using your knowledge base (documentation, manuals, processes, …).

Prompt

This is the input or instructions to the foundation model. A good prompt can help your model return a better result.

The process of crafting instructions (prompts) to elicit the best possible output from a generative AI model, ensuring the AI understands the task and generates accurate, relevant results, is called Prompt Engineering.

Bedrock SDK

AWS SDK provides two APIs that allow us to interact with Bedrock foundation models. The InvokeModel API and the Converse API. Both offer a synchronous and a stream variant.

InvokeModel API — This was the first API to interact with Bedrock models. Its purpose is to send a prompt to the FMs and get a response from them. If you’d like to create conversations, then you need to write the conversation prompts and parameters according to the FMs specifications. You can find information about each supported model's parameters here.
Converse API — It is built on top of the InvokeModel API and abstracts most of the model-specific parameters, so you can focus on quickly integrating with your desired FM.

It allows you to send documents for processing, which was not available in the InvokeModel API.

It also allows you to define and pass tools to the FM easily.

Please check here for the models that support the Converse API.

Requesting Access

In this article, we’ll focus on the serverless models, and to use them, you first need to request access through the Model Access console in Bedrock.

Then, you can enable access to all available or specific models.

Note that foundation model availability depends on the AWS region.

Here, I selected Titan Textg G1 — Express and Claude 3.5 Sonnet, then clicked on Next at the bottom of the page.

Titan Text G1 — Express selection

Claude 3.5 Sonnet selection

Then, you need to review the access and submit it.

Once the access is granted, you should see it as Access granted.

Now, we are ready to start building our application.

Getting started

Now that we've seen how to access Bedrock’s serverless foundation models, let’s start by building a Lambda that will call Bedrock for us. We’ll be building this function using Go 1.24.

In a iac folder, let’s initialize our Terraform code in a terraform.tf file:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.84"
    }
  }

  backend "s3" {
    bucket = "YOUR_BUCKET"
    key    = "state.tfstate"
  }
}

provider "aws" {

}

Don’t forget to change YOUR_BUCKET for your S3 bucket. Or remove the backend block if you don’t want to use a remote backend.

We’ll start with a simple text model from Amazon, the Titan Text G1 — Express model. If the model is not available in your region, you can pick any model of your choice, just make sure, for this first part, that it can be accessed without a cross-region inference profile. You can check here by seeing if your region has an asterisk(*) in front of it.

So, let’s get a data source for it. For that, we’ll need the Model ID, which you can find by clicking on the model in the Model Catalog:

In a bedrock.tf file, let’s add a data source pointing to the model:

data "aws_bedrock_foundation_model" "model" {
  model_id = "amazon.titan-text-express-v1"
}

This will return the model information, or nothing, if the model doesn’t exist. We could reference the model ID directly in our lambda, but this approach is more graceful.

Let’s then create our lambda that will interact with AWS Bedrock. First, we need the lambda’s policies. In a policies.tf, add:

data "aws_iam_policy_document" "assume_role" {
  statement {
    effect = "Allow"

    principals {
      type        = "Service"
      identifiers = ["lambda.amazonaws.com"]
    }

    actions = ["sts:AssumeRole"]
  }
}

data "aws_iam_policy_document" "policies" {
  statement {
    effect = "Allow"

    actions = [
      "logs:CreateLogGroup",
      "logs:CreateLogStream",
      "logs:PutLogEvents"
    ]

    resources = ["arn:aws:logs:*:*:*"]
  }
  statement {
    effect = "Allow"

    actions = [
      "bedrock:InvokeModel",
    ]

    resources = [data.aws_bedrock_foundation_model.model.model_arn]
  }
}

Now, in a lambda.tf, let’s create our lambda function:

resource "aws_lambda_function" "bedrock" {
  function_name    = "bedrock"
  runtime          = "provided.al2023"
  handler          = "bootstrap"
  architectures    = ["arm64"]
  filename         = data.archive_file.file.output_path
  source_code_hash = data.archive_file.file.output_base64sha256
  role             = aws_iam_role.role.arn
  timeout          = 30

  environment {
    variables = {
      MODEL_ID = "${data.aws_bedrock_foundation_model.model.model_id}"
    }
  }
}

resource "aws_iam_role" "role" {
  name               = "bedrock-lambda-role"
  assume_role_policy = data.aws_iam_policy_document.assume_role.json
}

resource "aws_iam_role_policy" "policies" {
  role   = aws_iam_role.role.name
  policy = data.aws_iam_policy_document.policies.json
}

data "archive_file" "file" {
  source_file = "${path.module}/init_code/bootstrap"
  output_path = "lambda_payload.zip"
  type        = "zip"
}

Since 2024, AWS has retired the old go1.x runtime. Now, we need to use the Amazon Linux runtime, here is the provided.al2023.

Because Bedrock responses can take a bit long, it is important to increase the timeout to a reasonable value. Here we are setting it to 30 seconds, as it is also the timeout for API Gateway.

The handler is the execution file. And it needs to be named bootstrap. You can get the bootstrap file, the compiled code here, or follow the README.md instructions in the base folder to compile the code by yourself. The base code is this one:

package main

import (
 "context"
 "encoding/json"
 "log"

 "github.com/aws/aws-lambda-go/lambda"
)

type response struct {
 Body       string `json:"body"`
 StatusCode int    `json:"statusCode"`
}

type message struct {
 Message string `json:"message"`
}

type handler struct{}

func (h *handler) handleRequest(ctx context.Context, event json.RawMessage) (*response, error) {
 log.Printf("Received event %s", event)
 body, _ := json.Marshal(message{Message: "Hello from Lambda!"})

 message := &response{
  Body:       string(body),
  StatusCode: 200,
 }

 return message, nil
}

func main() {
 h := handler{}
 lambda.Start(h.handleRequest)
}

Now, we can deploy this initial infrastructure with GitHub Actions. Create a .github/workflows/deploy-infra.yml file:

name: Deploy Infra
on:
  workflow_dispatch:
  push:
    branches:
      - main
    paths:
      - iac/**/*

defaults:
  run:
    working-directory: iac

jobs:
  deploy:
    name: 'Deploy'
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v3

      - name: Configure AWS Credentials Action For GitHub Actions
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: YOUR_REGION

      # Install the latest version of Terraform CLI and configure the Terraform CLI configuration file with a Terraform Cloud user API token
      - name: Setup Terraform
        uses: hashicorp/setup-terraform@v3

      # Initialize a new or existing Terraform working directory by creating initial files, loading any remote state, downloading modules, etc.
      - name: Terraform Init
        run: terraform init

      # Checks that all Terraform configuration files adhere to a canonical format
      - name: Terraform Format
        run: terraform fmt -check

      # Generates an execution plan for Terraform
      - name: Terraform Plan
        run: |
          terraform plan -out=plan -input=false

        # On push to "main", build or change infrastructure according to Terraform configuration files
        # Note: It is recommended to set up a required "strict" status check in your repository for "Terraform Cloud". See the documentation on "strict" required status checks for more information: https://help.github.com/en/github/administering-a-repository/types-of-required-status-checks
      - name: Terraform Apply
        run: terraform apply -auto-approve -input=false  plan

Don’t forget to change YOUR_REGION for your region.

Push this code to GitHub and wait for it to deploy. After it is complete, you should have your lambda in AWS. And you can test it in the Lambda console:

Lambda Code

Now that we have a functioning initial lambda, we can move to code the application. Let’s create an appfolder and run the following commands in it:

go mod init ai-app.com
go get github.com/aws/aws-lambda-go
go get github.com/aws/aws-sdk-go-v2
go get github.com/aws/aws-sdk-go-v2/config
go get github.com/aws/aws-sdk-go-v2/service/bedrockruntime

We need to create the request and response models, which will be mapped from the body of the input event. In a models.go:

package main

type Request struct {
 Prompt string `json:"prompt"`
}

type Response struct {
 Text string `json:"text"`
}

We will now create the Lambda handler. In a handler.go, we will a struct that will represent our

package main

import (
 "context"
 "encoding/json"
 "log"

 "github.com/aws/aws-lambda-go/events"
 "github.com/aws/aws-sdk-go-v2/aws"
 "github.com/aws/aws-sdk-go-v2/service/bedrockruntime"
 "github.com/aws/aws-sdk-go-v2/service/bedrockruntime/types"
)

type Handler struct {
 bedrockClient *bedrockruntime.Client
 modelID       string
}

In the same file, we will then create a method callBedrock, that will get the prompt and then call the Converse API:

func (h *Handler) callBedrock(prompt string, ctx *context.Context) (string, error) {
 input := &bedrockruntime.ConverseInput{
  ModelId: aws.String(h.modelID),
  Messages: []types.Message{
   {
    Content: []types.ContentBlock{
     &types.ContentBlockMemberText{
      Value: prompt,
     },
    },
    Role: types.ConversationRoleUser,
   },
  },
 }

 response, err := h.bedrockClient.Converse(*ctx, input)

 if err != nil {
  log.Printf("Failed to invoke model with error %s", err.Error())
  return "", err
 }

 outputMessage, _ := response.Output.(*types.ConverseOutputMemberMessage)

 text, _ := outputMessage.Value.Content[0].(*types.ContentBlockMemberText)

 return text.Value, nil
}

Now, we have the HandleRequest method, which gets the context and the event that was used to invoke that lambda function, and then calls the callBedrock method:

func (h *Handler) HandleRequest(ctx context.Context, event events.APIGatewayWebsocketProxyRequest) (*events.APIGatewayProxyResponse, error) {
 log.Printf("Received event %+v", event)

 var request Request
 err := json.Unmarshal([]byte(event.Body), &request)

 if err != nil {
  log.Fatal("Could not unmarshal request body")
 }

 response, err := h.callBedrock(request.Prompt, &ctx)

 if err != nil {
   log.Fatal("Failed to call bedrock")
 }

 responseBody, _ := json.Marshal(Response{
  Text: response,
 })

 apiResponse := &events.APIGatewayProxyResponse{
  StatusCode: 200,
  Body:       string(responseBody),
 }

 return apiResponse, nil
}

To finish the handler, we add a help method to initialize it:

func NewHandler(config aws.Config) *Handler {
 modelID := os.Getenv("MODEL_ID")

 return &Handler{
  bedrockClient: bedrockruntime.NewFromConfig(config),
  modelID:       modelID,
 }
}

And now, to start the lambda, in a main.go, we initialize our handler and pass it to the lambda.Start method:

package main

import (
 "context"
 "log"
 "os"

 "github.com/aws/aws-lambda-go/lambda"
 "github.com/aws/aws-sdk-go-v2/config"
)

func main() {
 cfg, err := config.LoadDefaultConfig(context.Background())
 if err != nil {
  log.Fatal(err)
 }

 h := NewHandler(cfg)

 lambda.Start(h.HandleRequest)
}

We need to deploy it. In the .github/workflows folder, create a file deploy-lambda.yml:

name: Deploy Lambda
on:
  workflow_dispatch:
  push:
    branches:
      - main
    paths:
      - app/**/*

defaults:
  run:
    working-directory: app

jobs:
  deploy:
    name: 'Deploy Lambda'
    runs-on: ubuntu-latest
    steps:
      # Checkout the repository to the GitHub Actions runner
      - name: Checkout
        uses: actions/checkout@v3

      - uses: actions/setup-go@v4.1.0
        with:
          go-version: '1.24.1'

      - name: Configure AWS Credentials Action For GitHub Actions
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: YOUR_REGION

      - name: Build Lambda
        run: GOOS=linux GOARCH=arm64 CGO_ENABLED=0 go build -o build/ .

        # The lambda requires that the executing file be named "bootstrap"
      - name: Rename file
        run: mv ./build/bedrock ./build/bootstrap

      - name: Zip build
        run: zip -r -j main.zip ./build

      - name: Update Lambda code
        run: aws lambda update-function-code --function-name=bedrock --zip-file=fileb://main.zip

Don’t forget to change YOUR_REGION for your region.

Now, push the code to GitHub and wait for the workflow to complete. Once it is completed, you can go to the Lambda console and send a test event with a similar structure:

{
  "body": "{\"prompt\": \"What is the size of the earth?\"}"
}

You should then get a response similar to:

{
 "statusCode": 200,
 "headers": null,
 "multiValueHeaders": null,
 "body": "{\"text\":\"\\nThe diameter of the Earth is 12,742 kilometers (7,917 miles).\"}"
}

Conversation Flow

Now, let’s dive into message roles. You might have noticed that we assigned the types.ConversationRoleUser to our message Role. This is converted to user and is how the LLM understands that this is a prompt from the user, and an answer must be provided. We have three types of roles:

user — The user prompt. Tells the LLM that this is an input prompt. If it is the last message in the conversation, the LLM will typically interpret it as a prompt or a request for further information and will generate a response as an assistant message. Example: “Give me the weather for Lisbon.”
assistant — The LLM’s response. Example: “Now it is sunny and the temperature is 20 degrees Celsius”
system — This is the system contextual message. It provides instructions or context to the model about the task it should perform or the persona it should adopt during the conversation. Example: “You are a weather assistant. You return the weather conditions (for example: sunny, cloudy, windy), the humidity, and the temperature in Celsius.”

System prompts play an important role in your AI application. A well-written and structured prompt will improve your application performance. Some reasons are:

Contextual responses — The AI will be focused on the desired topic and will avoid deviating from it
Accuracy— With clear instructions, the AI will know which topic to stay on top of and will avoid irrelevant answers
Structured responses — You can instruct your AI to only answer in a prompt with a defined response structure. For example, if you want to generate a list of the best movies, it should return only 10 movies in bullet points. Also, the list should be ordered from best to worst rated.

Let’s add the system role in providing a better context for our AI. For this, we need to use a model that supports it. You can find all the information in the documentation here. Let’s change our model in the bedrock.tf file to use Claude 3.5. You can find the model ID in the model catalog.

data "aws_bedrock_foundation_model" "model" {
  model_id = "anthropic.claude-3-5-sonnet-20240620-v1:0"
}

We are going to be adding the following system prompt:

You are a technology expert named Tony.
You answer technology related questions in a friendly and casual tone.
You break down complex topics into easy-to-understand explanations.
It's ok to not know the answer, but try your best to point to where the user might find more information about the topic.

In the callBedrock method, in the handler.go file, let’s add the system prompt :

func (h *Handler) callBedrock(prompt string, ctx *context.Context) (string, error) {
 input := &bedrockruntime.ConverseInput{
  ModelId: aws.String(h.modelID),
  Messages: []types.Message{
   {
    Content: []types.ContentBlock{
     &types.ContentBlockMemberText{
      Value: prompt,
     },
    },
    Role: types.ConversationRoleUser,
   },
  },
  System: []types.SystemContentBlock{
   &types.SystemContentBlockMemberText{
    Value: `You are a technology expert named Tony.
    You answer technology related questions in a friendly and casual tone.
    You break down complex topics into easy-to-understand explanations.
    It's ok to not know the answer, but try your best to point to where the user might find more information about the topic.`,
   },
  },
 }

 response, err := h.bedrockClient.Converse(*ctx, input)

 if err != nil {
  log.Printf("Failed to invoke model with error %s", err.Error())
  return "", err
 }

 outputMessage, _ := response.Output.(*types.ConverseOutputMemberMessage)

 text, _ := outputMessage.Value.Content[0].(*types.ContentBlockMemberText)

 return text.Value, nil
}

Now, once it is deployed, you can try with a technology-related prompt, like:

{
  "body": "{\"prompt\": \"What does RAM mean?\"}"
}

And you should get something similar to:

{
  "statusCode": 200,
  "headers": null,
  "multiValueHeaders": null,
  "body": "{\"text\":\"Hey there! Great question about RAM. \\n\\nRAM stands for Random Access Memory. It's basically your computer's short-term memory - like a digital workspace where your computer keeps all the stuff it's actively working on.\\n\\nThink of it like your desk. When you're working on a project, you spread out all the materials you need on your desk so you can access them quickly. That's what RAM does for your computer. It holds the data and programs that are currently in use so the computer can access them super fast.\\n\\nThe more RAM you have, the more \\\"desk space\\\" your computer has to work with multiple programs or handle bigger tasks without slowing down. That's why adding more RAM can often speed up an older computer.\\n\\nWhen you turn off your computer, everything in RAM is cleared - just like clearing off your desk at the end of the day. That's why RAM is called \\\"volatile\\\" memory.\\n\\nHope that helps explain it in a simple way! Let me know if you have any other tech questions - I'm always happy to chat about this stuff.\"}"
}

Notice how the response matches our system prompt. The assistant changed how the response was structured to match our system prompt.

Converse Stream API

In the previous code, we used the Converse API's synchronous version. It works well, but a better approach if you don’t necessarily need a synchronous flow is to use the API's stream version.

The ConverseStream, as well as the InvokeModelWithResponseStream, are implementations that return a stream instead of the whole result. This allows your application to have the data as soon as it is generated, unlocking many possibilities, like using the full potential of WebSockets, which is impossible with the standard API.

Let’s then update our callBedrock method in the handler.go file:

func (h *Handler) callBedrock(prompt string, ctx *context.Context) (*types.Message, error) {
 input := &bedrockruntime.ConverseStreamInput{
  ModelId: aws.String(h.modelID),
  Messages: []types.Message{
   {
    Content: []types.ContentBlock{
     &types.ContentBlockMemberText{
      Value: prompt,
     },
    },
    Role: types.ConversationRoleUser,
   },
  },
  System: []types.SystemContentBlock{
   &types.SystemContentBlockMemberText{
    Value: `You are a technology expert named Tony.
    You answer technology related questions in a friendly and casual tone.
    You break down complex topics into easy-to-understand explanations.
    It's ok to not know the answer, but try your best to point to where the user might find more information about the topic.`,
   },
  },
 }

 response, err := h.bedrockClient.ConverseStream(*ctx, input)

 if err != nil {
  log.Printf("Failed to invoke model with error %s", err.Error())
  return nil, err
 }

 outputMessage := response.GetStream().Events()

 result := h.handleOutput(outputMessage)

 return result, nil
}

Notice that our return changed from string to *types.Message. This is because we can receive multiple content blocks in a stream response. So we create this new struct to help us get a better result. In the models.go, let’s add this new struct and update the return from our lambda:

type Response struct {
 Messages []string `json:"messages"`
}

Now we need to implement the handleOutput method:

func (h *Handler) handleOutput(outputMessage <-chan types.ConverseStreamOutput) *types.Message{
 var result string

 var msg types.Message

 for event := range outputMessage {
  switch e := event.(type) {
  case *types.ConverseStreamOutputMemberMessageStart:
   log.Print("Message start")
   msg.Role = e.Value.Role

  case *types.ConverseStreamOutputMemberContentBlockStart:
   log.Print("Content block start")
   result = ""

  case *types.ConverseStreamOutputMemberContentBlockDelta:
   log.Print("Content block delta")
   textResponse := e.Value.Delta.(*types.ContentBlockDeltaMemberText)
   result = result + textResponse.Value

  case *types.ConverseStreamOutputMemberContentBlockStop:
   log.Print("Content block stop")
   msg.Content = append(msg.Content, &types.ContentBlockMemberText{
    Value: result,
   })

  case *types.ConverseStreamOutputMemberMetadata:
   log.Printf("Metadata %+v", e.Value)

  case *types.ConverseStreamOutputMemberMessageStop:
   log.Printf("Message stop. Reason: %s", e.Value.StopReason)

  case *types.UnknownUnionMember:
   log.Printf("unknown tag: %s", e.Tag)

  default:
   log.Printf("Received unexpected event type: %T", e)
  }
 }

 return &msg
}

Notice here that we receive the channel from the stream and loop through it to get the stream events as soon as they are available.

And then update the HandleRequest method to work with the new returns from callBedrock :

func (h *Handler) HandleRequest(ctx context.Context, event events.APIGatewayWebsocketProxyRequest) (*events.APIGatewayProxyResponse, error) {
 log.Printf("Received event %+v", event)

 var request Request
 err := json.Unmarshal([]byte(event.Body), &request)

 if err != nil {
  log.Fatal("Could not unmarshal request body")
 }

 if request.Prompt == "" {
  return &events.APIGatewayProxyResponse{
   StatusCode: 404,
   Body:       "",
  }, nil
 }

 log.Printf("Got user prompt message %s", request.Prompt)

 if err != nil {
  log.Fatal("Could not marshal model request body")
 }

 response, err := h.callBedrock(request.Prompt, &ctx, event.RequestContext.ConnectionID)

 if err != nil {
  log.Fatal("Failed to call bedrock")
 }

 var messages []string
 for _, content := range response.Content {
  if textContent, ok := content.(*types.ContentBlockMemberText); ok {
   messages = append(messages, textContent.Value)
  }
 }

 responseBody, _ := json.Marshal(Response{
  Messages: messages,
 })

 apiResponse := &events.APIGatewayProxyResponse{
  StatusCode: 200,
  Body:       string(responseBody),
 }

 return apiResponse, nil
}

Once you deploy, you can test it in the Lambda Console. Notice that you’ll get a similar behaviour, but if you go to CloudWatch, you’ll see multiple times the log Content block delta , indicating that the message was built when the LLM returns the content, which was ready.

Adding Tools

Tools enhance the LLM's ability to provide more information for a prompt.

They can increase the response accuracy to a given context, and the LLM can then decide whether to use the tool to provide a better response.

In Bedrock, tools are only supported in the Converse API. Please also check the documentation to see if the model you are using supports it.

To provide a tool to Bedrock, you need to give the tool schema, which is a JSON with:

Name — The tool’s name (it is the value Bedrock returns and you can use to link it back to the tool you want to use)
Description — A detailed description of what the tool does. To increase accuracy, please be thorough with the tool’s capabilities and provide examples.
Properties — A list of properties with their names and descriptions. Also, be detailed with the descriptions.
Required — A list of the properties required to run this tool.

Here is an example JSON Schema:


{
  "toolSpec": {
    "name": "getWeather",
    "description": "Get the current weather for a city or location. It returns the weather simplified with the city in the response. Examples: 'It is sunny in Berlin', 'It is raining in New York.",
    "inputSchema": {
      "json": {
        "type": "object",
        "properties": {
          "location": {
            "type": "string",
            "description": "The city or location to get the weather for. Example locations are Berlin, New York, Paris."
          }
        },
        "required": ["location"]
      }
    }
  }
}

Let’s implement a simple tool to get the weather for a location. It will just return a static It is sunny in with the location.

Under app, create a folder tools and then a file tools.go. Here we’ll define a simple GetWeather and the tool schema:

package tools

func GetWeather(location string) string {
 return "It is sunny in " + location
}

const (
 ToolGetWeather = "GetWeather"
)

func GetWeatherToolSchema() map[string]interface{} {
 return map[string]interface{}{
  "type": "object",
  "properties": map[string]interface{}{
   "location": map[string]interface{}{
    "type":        "string",
    "description": "The city or location to get the weather for. Example locations are Berlin, New York, Paris",
   },
  },
  "required": []string{"location"},
 }
}

The tool schema is how we inform the LLM about how our tool works, its parameters, and their meanings.

Now we update the system prompt to let the LLM know it can get weather if the user asks. And then we also need to add the ToolConfig let the LLM know that it has access to a tool if it needs.

input := &bedrockruntime.ConverseStreamInput{
  ModelId:  aws.String(h.modelID),
  Messages: []types.Message{
   {
    Content: []types.ContentBlock{
     &types.ContentBlockMemberText{
      Value: prompt,
     },
    },
    Role: types.ConversationRoleUser,
   },
  },
  System: []types.SystemContentBlock{
   &types.SystemContentBlockMemberText{
    Value: `You are a technology expert named Tony.
    You answer technology related questions in a friendly and casual tone.
    You break down complex topics into easy-to-understand explanations.
    It's ok to not know the answer, but try your best to point to where the user might find more information about the topic.
    You are also allowed to get the current weather.`,
   },
  },
  ToolConfig: &types.ToolConfiguration{
   Tools: []types.Tool{
    &types.ToolMemberToolSpec{
     Value: types.ToolSpecification{
      InputSchema: &types.ToolInputSchemaMemberJson{
       Value: document.NewLazyDocument(tools.GetWeatherToolSchema()),
      },
      Name:        aws.String(tools.ToolGetWeather),
      Description: aws.String("Get the current weather for a city or location. It returns the weather simplified with the city in the response. Examples: 'It is sunny in Berlin', 'It is raining in New York"),
     },
    },
   },
  },
 }

The description is a very important property because it gives the LLM context for how it should use the tool correctly. Try to be as detailed as possible.

The flow of how the LLM uses the tool is:

The prompt is sent to Bedrock with the tool
Bedrock processes the prompt and identifies that it needs to use the tool
Stops the current block content
Starts a new content block with the Tool Use ID and the Tool Name in the start property. For Golang, the block start is of type ContentBlockStartMemberToolUse .
Identifies the tool inputs and passes them down as a string in the Content Block Delta in the property input . In Golang, the delta block is of type ContentBlockDeltaMemberToolUse. In streaming mode, it passes chunks of the input.
Stops the content block
Stops the message with a stop reason tool_use
The application gets input from Bedrock's resulting message and uses the tool with the input provided by Bedrock.
The application gets the tool's result and then builds a new tool result message with the Tool Use ID and the tool result as content.

If the tool fails, the application also passes the status property error
The application appends the tool result message to the previous array of messages (the prompt message + the result messages from Bedrock's first conversation)
The application sends this new message array to Bedrock
Bedrock processes the tool result

Below you can find a simple diagram of the flow

With the flow in mind, we can update the handleOutput method to enable it to identify the tool use context and build the message:

func (h *Handler) handleOutput(outputMessage <-chan types.ConverseStreamOutput, connectionID string, ctx *context.Context) (*types.Message, types.StopReason) {
 var result string
 var toolInput string
 var stopReason types.StopReason

 var msg types.Message
 var toolUse types.ToolUseBlock

 for event := range outputMessage {
  switch e := event.(type) {
  case *types.ConverseStreamOutputMemberMessageStart:
   msg.Role = e.Value.Role

  case *types.ConverseStreamOutputMemberContentBlockStart:
   switch blockStart := e.Value.Start.(type) {
   case *types.ContentBlockStartMemberToolUse:
    toolUse.Name = blockStart.Value.Name
    toolUse.ToolUseId = blockStart.Value.ToolUseId
   }

  case *types.ConverseStreamOutputMemberContentBlockDelta:
   switch delta := e.Value.Delta.(type) {
   case *types.ContentBlockDeltaMemberText:
    result = result + delta.Value

   case *types.ContentBlockDeltaMemberToolUse:
    if delta.Value.Input != nil {
     toolInput = toolInput + *delta.Value.Input
    }
   }

  case *types.ConverseStreamOutputMemberContentBlockStop:
   if toolInput != "" {
    var toolInputMap map[string]string
    if err := json.Unmarshal([]byte(toolInput), &toolInputMap); err == nil {
     toolUse.Input = document.NewLazyDocument(toolInputMap)
     msg.Content = append(msg.Content, &types.ContentBlockMemberToolUse{
      Value: toolUse,
     })
    }
    toolUse = types.ToolUseBlock{}
    toolInput = ""
   } else {
    msg.Content = append(msg.Content, &types.ContentBlockMemberText{
     Value: result,
    })
    result = ""
   }

  case *types.ConverseStreamOutputMemberMetadata:
   log.Printf("Metadata %+v", e.Value)

  case *types.ConverseStreamOutputMemberMessageStop:
   stopReason = e.Value.StopReason

  case *types.UnknownUnionMember:
   log.Printf("unknown tag: %s", e.Tag)

  default:
   log.Printf("Received unexpected event type: %T", e)
  }
 }

 return &msg, stopReason
}

Now we can extract the code that builds the ConverseStreamInput, calls Bedrock, and handles the output into a streamMessages method so we can easily handle the tool calls:

func (h *Handler) streamMessages(ctx *context.Context, messages *[]types.Message, connectionID string) (*types.Message, error) {
 input := &bedrockruntime.ConverseStreamInput{
  ModelId:  aws.String(h.modelID),
  Messages: *messages,
  System: []types.SystemContentBlock{
   &types.SystemContentBlockMemberText{
    Value: `You are a technology expert named Tony.
    You answer technology related questions in a friendly and casual tone.
    You break down complex topics into easy-to-understand explanations.
    It's ok to not know the answer, but try your best to point to where the user might find more information about the topic.
    You are also allowed to get the current weather.`,
   },
  },
  ToolConfig: &types.ToolConfiguration{
   Tools: []types.Tool{
    &types.ToolMemberToolSpec{
     Value: types.ToolSpecification{
      InputSchema: &types.ToolInputSchemaMemberJson{
       Value: document.NewLazyDocument(tools.GetWeatherToolSchema()),
      },
      Name:        aws.String(tools.ToolGetWeather),
      Description: aws.String("Get the current weather for a city or location. It returns the weather simplified with the city in the response. Examples: 'It is sunny in Berlin', 'It is raining in Curitiba"),
     },
    },
   },
  },
 }

 response, err := h.bedrockClient.ConverseStream(*ctx, input)

 if err != nil {
  log.Printf("Failed to invoke model with error %s", err.Error())
  return nil, err
 }

 outputMessage := response.GetStream().Events()

 result, stopReason := h.handleOutput(outputMessage, connectionID, ctx)

 if stopReason == types.StopReasonToolUse {
  *messages = append(*messages, *result)

  for _, content := range result.Content {
   switch block := content.(type) {
   case *types.ContentBlockMemberToolUse:
    if *block.Value.Name == tools.ToolGetWeather {
     var params map[string]string
     _ = block.Value.Input.UnmarshalSmithyDocument(&params)

     weatherResult := tools.GetWeather(params["location"])

     *messages = append(*messages, types.Message{
      Role: types.ConversationRoleUser,
      Content: []types.ContentBlock{
       &types.ContentBlockMemberToolResult{
        Value: types.ToolResultBlock{
         ToolUseId: block.Value.ToolUseId,
         Content: []types.ToolResultContentBlock{
          &types.ToolResultContentBlockMemberText{
           Value: weatherResult,
          },
         },
         Status: types.ToolResultStatusSuccess,
        },
       },
      },
     })
    }
   }
  }

  result, err = h.StreamMessages(ctx, messages, connectionID)
 }

 return result, nil
}

And then, update the callBedrock method:

func (h *Handler) callBedrock(prompt string, ctx *context.Context, connectionID string) (*types.Message, error) {
 messages := []types.Message{
  {
   Content: []types.ContentBlock{
    &types.ContentBlockMemberText{
     Value: prompt,
    },
   },
   Role: types.ConversationRoleUser,
  },
 }

 msg, err := h.streamMessages(ctx, &messages, connectionID)
 if err != nil {
  return nil, err
 }

 return msg, nil
}

With this, you can test with a prompt that requires weather fetching, like How is the weather in New York?, and Bedrock should return the weather for you now. You can also add logging in the tool to see the flow in CloudWatch.

Cross-Region Inference

Cross-Region Inference is a method that increases the foundation model’s throughput by spreading and routing your requests to different regions during peak bursts.

The model needs to support it to be able to use it. You can find which models support it here.

Note that some models can only be accessed through cross-region inference, meaning that you cannot use the Model ID from the model catalog. Instead, use the model’s inference profile ID from the Cross-Region Inference console. An example is Claude 3.7 Sonnet, which can only be used through this method.

To use it, you can pass the Inference Profile ID, which is usually in the template {region}.{modelID} , for example eu.anthropic.claude-3–7-sonnet-20250219-v1:0. Also, you need to give bedrock:InvokeModel to the Inference Profile ARN and the ARN for all the regions this inference profile uses. For instance, Claude 3.7 Sonnet spams through Europe (Frankfurt), eu-north-1, Europe (Ireland), and Europe (Paris). So you need to permit like:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Stmt1743700361915",
      "Action": [
        "bedrock:InvokeModel"
      ],
      "Effect": "Allow",
      "Resource": [
        "arn:aws:bedrock:eu-central-1:011223344556:inference-profile/eu.anthropic.claude-3-7-sonnet-20250219-v1:0",
        "arn:aws:bedrock:eu-central-1::foundation-model/anthropic.claude-3-7-sonnet-20250219-v1:0",
        "arn:aws:bedrock:eu-north-1::foundation-model/anthropic.claude-3-7-sonnet-20250219-v1:0",
        "arn:aws:bedrock:eu-west-1::foundation-model/anthropic.claude-3-7-sonnet-20250219-v1:0",
        "arn:aws:bedrock:eu-west-3::foundation-model/anthropic.claude-3-7-sonnet-20250219-v1:0"
      ]
    }
  ]
}

Or you can give it access to all regions:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Stmt1743700361915",
      "Action": [
        "bedrock:InvokeModel"
      ],
      "Effect": "Allow",
      "Resource": [
        "arn:aws:bedrock:eu-central-1:011223344556:inference-profile/eu.anthropic.claude-3-7-sonnet-20250219-v1:0",
        "arn:aws:bedrock:*::foundation-model/anthropic.claude-3-7-sonnet-20250219-v1:0"
      ]
    }
  ]
}

You can find out about the supported models here. They are the ones marked with a *.

Integrating with a WebSocket API

Let’s integrate our app with API Gateway WebSockets.

API Gateway WebSockets have four types of routes:

$connect — When a new client connects to the websocket
$disconnect — When a client disconnects from the websockets
$default — When no routes match or we have no routes defined
custom_route — Custom routes that are defined in the API Gateway. Example: user, product,…

For this example, we don’t necessarily need the $connect and $disconnect routes. A use case for these routes would be if you needed to save the ConnectionID to the database to send a message to it through the websocket from another service. But, if the user sends the message through the WebSocket directly, as we’ll do, then API Gateway automatically sends the ConnectionID in the request context.

Let’s create a new API Gateway WebSockets with the $default route. In the iac folder, create a new file api_websocket.tf :

resource "aws_apigatewayv2_api" "api" {
  name                       = "bedrock"
  protocol_type              = "WEBSOCKET"
  route_selection_expression = "$request.body.action"
}

resource "aws_apigatewayv2_stage" "stage" {
  api_id      = aws_apigatewayv2_api.api.id
  name        = "live"
  auto_deploy = true
}

resource "aws_apigatewayv2_route" "default" {
  api_id    = aws_apigatewayv2_api.api.id
  route_key = "$default"
  target    = "integrations/${aws_apigatewayv2_integration.default.id}"
}

resource "aws_apigatewayv2_integration" "default" {
  api_id           = aws_apigatewayv2_api.api.id
  integration_type = "AWS_PROXY"

  connection_type           = "INTERNET"
  content_handling_strategy = "CONVERT_TO_TEXT"
  description               = "Default Websocket route"
  integration_method        = "POST"
  integration_uri           = aws_lambda_function.bedrock.invoke_arn
  passthrough_behavior      = "WHEN_NO_MATCH"
}

resource "aws_lambda_permission" "bedrock" {
  statement_id  = "AllowDefaultExecutionFromAPIGateway"
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.bedrock.function_name
  principal     = "apigateway.amazonaws.com"
  source_arn    = "${aws_apigatewayv2_api.api.execution_arn}/*/${aws_apigatewayv2_route.default.route_key}"
}

To send messages to the WebSocket from the Lambda function, we need to let it know where to send the messages. For that, we just need to add the API Gateway https invoke URL to our Lambda. In the lambda.tf file, let’s add it as an environment variable:

resource "aws_lambda_function" "bedrock" {
  function_name    = "bedrock"
  runtime          = "provided.al2023"
  handler          = "bootstrap"
  architectures    = ["arm64"]
  filename         = data.archive_file.file.output_path
  source_code_hash = data.archive_file.file.output_base64sha256
  role             = aws_iam_role.role.arn
  timeout          = 30

  environment {
    variables = {
      MODEL_ID             = "${data.aws_bedrock_foundation_model.model.model_id}"
      API_GATEWAY_ENDPOINT = "${replace(aws_apigatewayv2_stage.stage.invoke_url, "wss", "https")}"
    }
  }
}

Note that we are replacing wss for https . That is because the invoke_url is the WebSocket address to the active stage.

The lambda role now needs permission to access the /POST/@connections/{connectionId} endpoint in API Gateway. Let’s add this permission to the policies.tf file, in the aws_iam_policydocument polices resource, add the following statement:

statement {
  effect = "Allow"

  actions = [
    "execute-api:ManageConnections"
  ]

  resources = [
    "${aws_apigatewayv2_stage.stage.execution_arn}/POST/@connections/{connectionId}"
  ]
}

Great! The API Gateway WebSocket is ready. We now need to update our Lambda code to send messages to the connected client.

In the app folder, run the following command to get the ApiGatewayManagementAPI package:

go get "github.com/aws/aws-sdk-go-v2/service/apigatewaymanagementapi"

In the models.go , let’s add a new model for our message to the WebSocket and some helpful constants:

type WebSocketMessage struct {
 Event string `json:"event"`
 Data  string `json:"data"`
}

const (
 BedrockEventContent      string = "content"
 BedrockEventMessageStart string = "message_start"
 BedrockEventMessageStop  string = "message_stop"
 BedrockEventContentStart string = "content_start"
 BedrockEventContentStop  string = "content_stop"
 BedrockEventMetadata     string = "metadata"
)

These constants help the client understand when a new message starts and ends so it can react accordingly.

Now, in the handler.go file, let’s first update the Handler struct to add the API Gateway Management Client:

type Handler struct {
 bedrockClient              *bedrockruntime.Client
 modelID                    string
 apiGatewayManagementClient *apigatewaymanagementapi.Client
}

Let’s update the NewHandler function to get the API_GATEWAY_ENDPOINT environment variable and initialize an instance of apigatewaymanagementapi.Client :

func NewHandler(config aws.Config) *Handler {
 modelID := os.Getenv("MODEL_ID")
 apiGatewayEndpoint := os.Getenv("API_GATEWAY_ENDPOINT")

 return &Handler{
  bedrockClient: bedrockruntime.NewFromConfig(config),
  modelID:       modelID,
  apiGatewayManagementClient: apigatewaymanagementapi.NewFromConfig(config, func(o *apigatewaymanagementapi.Options) {
   o.BaseEndpoint = &apiGatewayEndpoint
  }),
 }
}

Let’s now create a new function to send messages to the WebSocket:

func (h *Handler) SendWebSocketMessageToConnection(ctx *context.Context, textResponse string, event string, connectionID string) {
 data, _ := json.Marshal(WebSocketMessage{Event: event, Data: textResponse})

 websocketInput := &apigatewaymanagementapi.PostToConnectionInput{
  ConnectionId: aws.String(connectionID),
  Data:         []byte(data),
 }

 response, err := h.apiGatewayManagementClient.PostToConnection(*ctx, websocketInput)

 if err != nil {
  log.Printf("ERROR %+v", err)
 }
}

We now need to use it in the handleOutput method:

func (h *Handler) handleOutput(outputMessage <-chan types.ConverseStreamOutput, connectionID string, ctx *context.Context) (*types.Message, types.StopReason) {
 var result string
 var toolInput string
 var stopReason types.StopReason

 var msg types.Message
 var toolUse types.ToolUseBlock

 for event := range outputMessage {
  switch e := event.(type) {
  case *types.ConverseStreamOutputMemberMessageStart:
   h.SendWebSocketMessageToConnection(ctx, "", BedrockEventMessageStart, connectionID)

   msg.Role = e.Value.Role

  case *types.ConverseStreamOutputMemberContentBlockStart:
   switch blockStart := e.Value.Start.(type) {
   case *types.ContentBlockStartMemberToolUse:
    toolUse.Name = blockStart.Value.Name
    toolUse.ToolUseId = blockStart.Value.ToolUseId
   default:
    h.SendWebSocketMessageToConnection(ctx, "", BedrockEventContentStart, connectionID)
   }

  case *types.ConverseStreamOutputMemberContentBlockDelta:
   switch delta := e.Value.Delta.(type) {
   case *types.ContentBlockDeltaMemberText:
    h.SendWebSocketMessageToConnection(ctx, delta.Value, BedrockEventContent, connectionID)
    result = result + delta.Value

   case *types.ContentBlockDeltaMemberToolUse:
    if delta.Value.Input != nil {
     toolInput = toolInput + *delta.Value.Input
    }
   }

  case *types.ConverseStreamOutputMemberContentBlockStop:
   if toolInput != "" {
    var toolInputMap map[string]string
    if err := json.Unmarshal([]byte(toolInput), &toolInputMap); err == nil {
     toolUse.Input = document.NewLazyDocument(toolInputMap)
     msg.Content = append(msg.Content, &types.ContentBlockMemberToolUse{
      Value: toolUse,
     })
    }
    toolUse = types.ToolUseBlock{}
    toolInput = ""
   } else {
    h.SendWebSocketMessageToConnection(ctx, "", BedrockEventContentStop, connectionID)

    msg.Content = append(msg.Content, &types.ContentBlockMemberText{
     Value: result,
    })
    result = ""
   }

  case *types.ConverseStreamOutputMemberMetadata:
   h.SendWebSocketMessageToConnection(ctx, fmt.Sprintf("%+v", e.Value), BedrockEventMetadata, connectionID)

  case *types.ConverseStreamOutputMemberMessageStop:
   stopReason = e.Value.StopReason
   h.SendWebSocketMessageToConnection(ctx, string(e.Value.StopReason), BedrockEventMessageStop, connectionID)

  case *types.UnknownUnionMember:
   log.Printf("unknown tag: %s", e.Tag)

  default:
   log.Printf("Received unexpected event type: %T", e)
  }
 }

 return &msg, stopReason
}

We need to then update the streamMessages method to receive and pass the connectionID :

func (h *Handler) StreamMessages(ctx *context.Context, messages *[]types.Message, connectionID string) (*types.Message, error) {
 input := &bedrockruntime.ConverseStreamInput{
  ModelId:  aws.String(h.modelID),
  Messages: *messages,
  System: []types.SystemContentBlock{
   &types.SystemContentBlockMemberText{
    Value: `You are a technology expert named Tony.
    You answer technology related questions in a friendly and casual tone.
    You break down complex topics into easy-to-understand explanations.
    It's ok to not know the answer, but try your best to point to where the user might find more information about the topic.
    You are also allowed to get the current weather.`,
   },
  },
  ToolConfig: &types.ToolConfiguration{
   Tools: []types.Tool{
    &types.ToolMemberToolSpec{
     Value: types.ToolSpecification{
      InputSchema: &types.ToolInputSchemaMemberJson{
       Value: document.NewLazyDocument(tools.GetWeatherToolSchema()),
      },
      Name:        aws.String(tools.ToolGetWeather),
      Description: aws.String("Get the current weather for a city or location. It returns the weather simplified with the city in the response. Examples: 'It is sunny in Berlin', 'It is raining in Curitiba"),
     },
    },
   },
  },
 }

 response, err := h.bedrockClient.ConverseStream(*ctx, input)

 if err != nil {
  log.Printf("Failed to invoke model with error %s", err.Error())
  return nil, err
 }

 outputMessage := response.GetStream().Events()

 result, stopReason := h.handleOutput(outputMessage, connectionID, ctx)

 if stopReason == types.StopReasonToolUse {
  *messages = append(*messages, *result)

  for _, content := range result.Content {
   switch block := content.(type) {
   case *types.ContentBlockMemberToolUse:
    if *block.Value.Name == tools.ToolGetWeather {
     var params map[string]string
     _ = block.Value.Input.UnmarshalSmithyDocument(&params)

     log.Printf("Got inputs %s from tool", params["location"])

     weatherResult := tools.GetWeather(params["location"])

     *messages = append(*messages, types.Message{
      Role: types.ConversationRoleUser,
      Content: []types.ContentBlock{
       &types.ContentBlockMemberToolResult{
        Value: types.ToolResultBlock{
         ToolUseId: block.Value.ToolUseId,
         Content: []types.ToolResultContentBlock{
          &types.ToolResultContentBlockMemberText{
           Value: weatherResult,
          },
         },
         Status: types.ToolResultStatusSuccess,
        },
       },
      },
     })
    }
   }
  }

  result, err = h.StreamMessages(ctx, messages, connectionID)
 }

 return result, nil
}

Update the callBedrock method:

func (h *Handler) callBedrock(prompt string, ctx *context.Context, connectionID string) (*types.Message, error) {
 messages := []types.Message{
  {
   Content: []types.ContentBlock{
    &types.ContentBlockMemberText{
     Value: prompt,
    },
   },
   Role: types.ConversationRoleUser,
  },
 }

 msg, err := h.streamMessages(ctx, &messages, connectionID)
 if err != nil {
  return nil, err
 }

 return msg, nil
}

And then the HandleRequest function to extract the ConnectionID from the request context:

func (h *Handler) HandleRequest(ctx context.Context, event events.APIGatewayWebsocketProxyRequest) (*events.APIGatewayProxyResponse, error) {
 var request Request
 err := json.Unmarshal([]byte(event.Body), &request)

 if err != nil {
  log.Fatal("Could not unmarshal request body")
 }

 response, err := h.callBedrock(request.Prompt, &ctx, event.RequestContext.ConnectionID)

 if err != nil {
   log.Fatal("Failed to call bedrock")
 }

 var messages []string
 for _, content := range response.Content {
  if textContent, ok := content.(*types.ContentBlockMemberText); ok {
   messages = append(messages, textContent.Value)
  }
 }

 responseBody, _ := json.Marshal(Response{
  Messages: messages,
 })

 apiResponse := &events.APIGatewayProxyResponse{
  StatusCode: 200,
  Body:       string(responseBody),
 }

 return apiResponse, nil
}

After you deploy it, we can test it using wscat tool in the terminal. Open the terminal and run the following command:

npm install -g wscat

You now need the WebSocket URL. You can find it in the API stage’s console:

You can copy it and use it in the following terminal command:

wscat -c WEBSOCKET_URL

Where WEBSOCKET_URL is the URL you got from the AWS Console. Once connected, you can send messages, and they should go to our Lambda in the $default route. So, try sending something like:

{ "prompt": "Hey!" }

You should see something similar to the example below:

And with the tool:

Conclusion

What a journey!

In this article, we learned and worked with the fantastic Bedrock, the AWS LLM service. It facilitates deploying and using the most well-known foundation models and provides serverless solutions for some models.

In addition to the basics and foundations of Bedrock, we also saw how we can easily integrate it with a serverless application using Lambda functions and API Gateway WebSockets.

We also had AI tools, which enriched the LLM’s power to give better and targeted results by providing functions that the model can use for the prompt context.

We wrote all the logic code using Golang, a simple, easy, and fast programming language.

Lastly, we used the power of Terraform to provide and manage our entire infrastructure.

Hope you enjoy it!

Happy coding 💻

AWS AI - Transcribe

Andre Lopes — Tue, 18 Mar 2025 14:27:46 +0000

Using AWS Transcribe serverless service to create text from speech

Hey people!

I want to start a new series covering the most used AWS AI services today. For the first service, we’ll discuss AWS Transcribe.

Transcribe is an automatic speech recognition service that uses machine learning models to convert audio to text. It is an excellent service for getting text from audio/videos or adding speech-to-text capabilities to an application. It has multiple language support, and you can:

Use the general model provided by AWS
Train your model with custom data
Create vocabularies to enrich the selected model
Create vocabulary filters to remove/redact selected data

Demo Architecture

For our demo architecture, we will build a simple app with an S3 bucket and two folders: media and transcription. We’ll then configure S3 notifications so that when a file is added to the media folder, it triggers a Lambda function to start a new job in Transcribe. The job will then generate and add a transcription to the transcription folder.

We'll be creating our infrastructure using Terraform as our Infrastructure as Code.

General Model

The quickest way to start with Transcribe is to use its General Model. It is suited for a broad audience but has no specialized capabilities.

Let’s start by creating our infrastructure. Create an iac folder and then add a terraform.tf file to initialize our Terraform configuration:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.84"
    }
  }

  backend "s3" {
    bucket = "YOUR_BUCKET"
    key    = "state.tfstate"
  }
}

provider "aws" {}

Don’t forget to change YOUR_BUCKET for the bucket you want to use for your backend configuration.

Let’s now create the S3 buckets and folders. Create a s3.tf file:

resource "aws_s3_bucket" "bucket" {
  bucket = "aws-ai-transcribe"
}

resource "aws_s3_object" "media" {
  bucket = aws_s3_bucket.bucket.id
  key    = "media/"
}

resource "aws_s3_object" "transcription" {
  bucket = aws_s3_bucket.bucket.id
  key    = "transcription/"
}

resource "aws_lambda_permission" "allow_bucket" {
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.transcribe.arn
  source_arn    = aws_s3_bucket.bucket.arn
  principal     = "s3.amazonaws.com"
}

resource "aws_s3_bucket_notification" "bucket" {
  bucket = aws_s3_bucket.bucket.id
  lambda_function {
    filter_prefix       = aws_s3_object.media.key
    events              = ["s3:ObjectCreated:*"]
    lambda_function_arn = aws_lambda_function.transcribe.arn
  }

  depends_on = [aws_lambda_permission.allow_bucket]
}

Here, we are creating our aws-ai-transcribe bucket, adding the two folders, media and transcription, and then creating an S3 notification to the lambda function for every file added to the media folder.

Note that we added an explicit depends_on = [aws_lambda_permission.allow_bucket]. Because we want this notification to be created after we have created the permission for our lambda function to be invoked from S3 notifications.

Now, let’s create our lambda function in a lambdas.tf file:

resource "aws_lambda_function" "transcribe" {
  function_name    = "transcribe"
  runtime          = "nodejs22.x"
  handler          = "index.handler"
  filename         = data.archive_file.file.output_path
  source_code_hash = data.archive_file.file.output_base64sha256
  role             = aws_iam_role.role.arn

  environment {
    variables = {
      JOB_ROLE_ARN = "${aws_iam_role.job_role.arn}"
      OUTPUT_KEY   = "${aws_s3_object.transcription.key}"
    }
  }
}

resource "aws_iam_role" "role" {
  name               = "transcribe-lambda-role"
  assume_role_policy = data.aws_iam_policy_document.assume_role.json
}

resource "aws_iam_role_policy" "policies" {
  role   = aws_iam_role.role.name
  policy = data.aws_iam_policy_document.policies.json
}

resource "aws_iam_role" "job_role" {
  name               = "transcribe-job-role"
  assume_role_policy = data.aws_iam_policy_document.assume_job_role.json
}

resource "aws_iam_role_policy" "job_policies" {
  role   = aws_iam_role.job_role.name
  policy = data.aws_iam_policy_document.job_policies.json
}

data "archive_file" "file" {
  source_dir  = "${path.root}/init_code"
  output_path = "lambda_payload.zip"
  type        = "zip"
}

Here, we are creating our lambda function and the role that our transcribe jobs will assume to be able to download the media file and upload the transcription to the S3 bucket.

This lambda requires an initialization code from the init_code folder. So, under iac, create a folder init_code with an index.js:

// Default handler generated in AWS
export const handler = async (event) => {
  const response = {
    statusCode: 200,
    body: JSON.stringify({ message: 'Hello from Lambda!' }),
  };

  return response;
};

And a package.json file:

{
  "name": "second-lambda",
  "version": "1.0.0",
  "main": "index.js",
  "type": "module",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC"
}

Last, let’s create the policies for our lambda and transcribe job IAM role in a policies.tf file:

data "aws_iam_policy_document" "assume_role" {
  statement {
    effect = "Allow"

    principals {
      type        = "Service"
      identifiers = ["lambda.amazonaws.com"]
    }

    actions = ["sts:AssumeRole"]
  }
}

data "aws_iam_policy_document" "assume_job_role" {
  statement {
    effect = "Allow"

    principals {
      type        = "Service"
      identifiers = ["transcribe.amazonaws.com"]
    }

    actions = ["sts:AssumeRole"]
  }
}

data "aws_iam_policy_document" "policies" {
  statement {
    effect = "Allow"

    actions = [
      "logs:CreateLogGroup",
      "logs:CreateLogStream",
      "logs:PutLogEvents"
    ]

    resources = ["arn:aws:logs:*:*:*"]
  }

  statement {
    effect = "Allow"

    actions = ["iam:PassRole"]

    resources = [aws_iam_role.job_role.arn]
  }

  statement {
    effect = "Allow"

    actions = ["transcribe:StartTranscriptionJob"]

    resources = ["arn:aws:transcribe:*:*:transcription-job/*"]
  }
}

data "aws_iam_policy_document" "job_policies" {
  statement {
    effect = "Allow"

    actions = ["s3:GetObject"]

    resources = ["${aws_s3_object.media.arn}*"]
  }
  statement {
    effect = "Allow"

    actions = ["s3:PutObject"]

    resources = ["${aws_s3_object.transcription.arn}*"]
  }
}

Our lambda role must have the transcribe:StartTranscriptionJob permission, or else it cannot start a job in Transcribe. The job role needs permission to get the media file from the S3 bucket folder and put the transcription into the S3 bucket folder.

Now that we have our infrastructure files let’s set up our GitHub workflow to create them. Create a .github/workflows folder and add a deploy-infrastructure.yml file:

name: Deploy Transcribe Infra
on:
  workflow_dispatch:
  push:
    branches:
      - main
    paths:
      - transcribe/iac/**/*

defaults:
  run:
    working-directory: transcribe/iac

jobs:
  deploy:
    name: 'Deploy'
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v3

      - name: Configure AWS Credentials Action For GitHub Actions
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: YOUR_REGION

      # Install the latest version of Terraform CLI and configure the Terraform CLI configuration file with a Terraform Cloud user API token
      - name: Setup Terraform
        uses: hashicorp/setup-terraform@v3

      # Initialize a new or existing Terraform working directory by creating initial files, loading any remote state, downloading modules, etc.
      - name: Terraform Init
        run: terraform init

      # Checks that all Terraform configuration files adhere to a canonical format
      - name: Terraform Format
        run: terraform fmt -check

      # Generates an execution plan for Terraform
      - name: Terraform Plan
        run: |
          terraform plan -out=plan -input=false

        # On push to "main", build or change infrastructure according to Terraform configuration files
        # Note: It is recommended to set up a required "strict" status check in your repository for "Terraform Cloud". See the documentation on "strict" required status checks for more information: https://help.github.com/en/github/administering-a-repository/types-of-required-status-checks
      - name: Terraform Apply
        run: terraform apply -auto-approve -input=false  plan

Make sure you have AWS_ACCESS_KEY and AWS_SECRET_ACCESS_KEY variables added to GitHub actions secrets and that these credentials have permission to create resources in your account. Also, don’t forget to replace YOUR_REGION with the region where you want these resources created.

Now, push the code to GitHub, and you should have the basic infrastructure created after it finishes.

Now, let’s create the lambda function that will start a transcription job whenever a file is added to S3 media bucket folder.

Create an app folder and run the following code to start our TypeScript project:

npm init -y
npm i -s @aws-sdk/client-transcribe
npm i -D typescript copyfiles @types/aws-sdk @types/aws-lambda

Now, we need to the a tsc and a build script to your package.json :

{
  "name": "app",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "type": "module",
  "scripts": {
    "tsc": "tsc",
    "build": "tsc && copyfiles package.json build/",
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "@aws-sdk/client-transcribe": "^3.760.0"
  },
  "devDependencies": {
    "@types/aws-lambda": "^8.10.147",
    "@types/aws-sdk": "^0.0.42",
    "copyfiles": "^2.4.1",
    "typescript": "^5.8.2"
  }
}

Now, let’s initialize our TypeScript project with:

npm run tsc -- --init --target esnext --module nodenext \
--moduleResolution nodenext --rootDir src \
--outDir build --noImplicitAny --noImplicitThis --newLine lf \
--resolveJsonModule

Or, for Windows:

npm run tsc -- --init --target esnext --module nodenext `
--moduleResolution nodenext --rootDir src `
--outDir build --noImplicitAny --noImplicitThis --newLine lf `
--resolveJsonModule

You should have a tsconfig.json file:

{
  "compilerOptions": {
    "target": "esnext",
    "module": "nodenext",
    "rootDir": "src",
    "moduleResolution": "nodenext",
    "resolveJsonModule": true,
    "outDir": "build",
    "newLine": "lf",
    "esModuleInterop": true,
    "forceConsistentCasingInFileNames": true,
    "strict": true,
    "noImplicitAny": true,
    "noImplicitThis": true,
    "skipLibCheck": true
  }
}

Now, to our actual lambda function code, create a src folder, and an index.ts file:

import {
  MediaFormat,
  StartTranscriptionJobCommand,
  TranscribeClient,
  type StartTranscriptionJobRequest,
} from '@aws-sdk/client-transcribe';
import type { S3Event } from 'aws-lambda';

const JOB_ROLE_ARN = process.env.JOB_ROLE_ARN;
const transcribeClient = new TranscribeClient({});

export const handler = async (event: S3Event) => {
  for (let record of event.Records) {
    const bucket = record.s3.bucket.name;
    const key = record.s3.object.key;

    const fileInput = `s3://${bucket}/${key}`;
    const mediaFormat = fileInput.split('.').at(-1);

    if (
      !mediaFormat ||
      !Object.values(MediaFormat).includes(mediaFormat as MediaFormat)
    ) {
      console.warn('No media format for this file');
      return;
    }

    const jobName = key.replace('/', '_') + Date.now();

    const jobRequest: StartTranscriptionJobRequest = {
      TranscriptionJobName: jobName,
      Media: { MediaFileUri: fileInput },
      MediaFormat: mediaFormat as MediaFormat,
      LanguageCode: 'en-US',
      OutputBucketName: bucket,
      OutputKey: `transcription/${jobName}.json`,
      JobExecutionSettings: {
        DataAccessRoleArn: JOB_ROLE_ARN,
      },
    };

    const job = new StartTranscriptionJobCommand(jobRequest);

    try {
      const response = await transcribeClient.send(job);
      console.log(
        'Finished job %s. Data %s',
        jobName,
        response.TranscriptionJob
      );
    } catch (error: any) {
      console.error(
        "Couldn't start transcription job %s. Error: %s",
        jobName,
        error
      );
      throw error;
    }
  }
};

Notice the LanguageCode we are passing to our StartTranscriptionJobRequest object. We know the language will be en-US and have set it up for better transcription (You can find all the supported languages here). If you do not know the languages spoken in your media, use either IdentifyLanguage or IdentifyMultipleLanguages and let Amazon Transcribe identify the languages for you. Just remember that one of these three language configurations is required.

Now, to deploy the lambda with GitHub Actions, create a new deploy-lambda.yml workflow under .github/workflows :

name: Deploy Transcribe Lambda
on:
  workflow_dispatch:
  push:
    branches:
      - main
    paths:
      - app/**/*

defaults:
  run:
    working-directory: app/

jobs:
  deploy:
    name: 'Deploy Lambda'
    runs-on: ubuntu-latest
    steps:
      # Checkout the repository to the GitHub Actions runner
      - name: Checkout
        uses: actions/checkout@v3

      - name: Setup NodeJS
        uses: actions/setup-node@v4
        with:
          node-version: 22

      - name: Configure AWS Credentials Action For GitHub Actions
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: YOUR_REGION

      - name: Install packages
        run: npm install

      - name: Build
        run: npm run build

      - name: Zip build
        run: cd build && zip -r ../main.zip .

      - name: Update Lambda code
        run: aws lambda update-function-code --function-name=transcribe --zip-file=fileb://main.zip

After the workflow finishes running, you can upload a media file to the media/ folder, and you should get a transcription in the transcription/ folder. For example, I uploaded a file of me saying, Hello, how are you?. The file you get in JSON format contains the whole transcript, broken down into multiple items:

{
  "jobName": "audio_hello.m4a17393009178780818",
  "accountId": "044256433832",
  "status": "COMPLETED",
  "results": {
    "transcripts": [{ "transcript": "Hello, how are you?" }],
    "items": [
      {
        "id": 0,
        "type": "pronunciation",
        "alternatives": [{ "confidence": "0.999", "content": "Hello" }],
        "start_time": "0.72",
        "end_time": "1.419"
      },
      {
        "id": 1,
        "type": "punctuation",
        "alternatives": [{ "confidence": "0.0", "content": "," }]
      },
      {
        "id": 2,
        "type": "pronunciation",
        "alternatives": [{ "confidence": "0.996", "content": "how" }],
        "start_time": "1.629",
        "end_time": "1.809"
      },
      {
        "id": 3,
        "type": "pronunciation",
        "alternatives": [{ "confidence": "0.999", "content": "are" }],
        "start_time": "1.809",
        "end_time": "1.99"
      },
      {
        "id": 4,
        "type": "pronunciation",
        "alternatives": [{ "confidence": "0.999", "content": "you" }],
        "start_time": "1.99",
        "end_time": "2.259"
      },
      {
        "id": 5,
        "type": "punctuation",
        "alternatives": [{ "confidence": "0.0", "content": "?" }]
      }
    ],
    "audio_segments": [
      {
        "id": 0,
        "transcript": "Hello, how are you?",
        "start_time": "0.709",
        "end_time": "2.349",
        "items": [0, 1, 2, 3, 4, 5]
      }
    ]
  }
}

Custom Vocabulary

Creating a Custom Vocabulary https://docs.aws.amazon.com/transcribe/latest/dg/custom-vocabulary.html

There are some considerations when using custom vocabularies:

Each AWS account can have up to 100 vocabularies.
The file size limit is 50 KB.
When using API to create, the file must be a text file (txt). For the AWS Console, it can also be a CSV file.
Each entry must have a maximum of 256 characters.
Vocabulary is region-locked, meaning you can only use it in the same region in which it was created.
Entries with multiple words must be separated with a hyphen (-). For example: Event-Driven-Architecture
Entries that represent acronyms must have a trailing period (.) after each letter. For example: A.W.S., A.I., C.L.I..
For entries with words and acronyms, words should be separated with a hyphen from the acronyms, and the acronyms should follow the rule above. Example: Dynamo-D.B.

To create a new vocabulary, we can do it in two ways:

Directly in the Terraform configuration
Through a configuration text file

To do it with the Terraform configuration, we need to add them to the phrases property:

resource "aws_transcribe_vocabulary" "vocabulary" {
  vocabulary_name = "example"
  phrases         = ["A.W.S.", "A.I.", "Serverless"]
  language_code   = "en-US"
}

For creating a vocabulary with a file, we need to upload it to S3 before referencing it to the vocabulary. This is my preferred approach, as you can better have control over the vocabulary entries, and it also allows you to set how the entry should be displayed in the transcription.

Let’s create a vocabulary file. It should be in the following structure:

Phrase[TAB]SoundsLike[TAB]IPA[TAB]DisplayAs
Entry[TAB][TAB][TAB]Entry
Another-Entry[TAB][TAB][TAB]Another Entry
X.Y.Z.[TAB][TAB][TAB]XYZ
Amazon-dot-com[TAB][TAB][TAB]Amazon.com
A.B.C.-s[TAB][TAB][TAB]ABCs
Dynamo-D.B.[TAB][TAB][TAB]DynamoDB
zero-one-two-A.B.[TAB][TAB][TAB]012AB

Where [TAB] is the tab character.

So, let’s create a vocabulary.txt file under the iac folder:

Phrase[TAB]SoundsLike IPA DisplayAs
A.W.S.   AWS
A.I.   AI
Serverless   serverless

Now, let’s create the object in S3. In the s3.tf file, add the following:

resource "aws_s3_object" "vocabulary_folder" {
  bucket = aws_s3_bucket.bucket.id
  key    = "vocabularies/"
}

resource "aws_s3_object" "vocabulary" {
  bucket      = aws_s3_bucket.bucket.id
  key         = "${aws_s3_object.vocabulary_folder.key}vocabulary.txt"
  source      = "${path.module}/transcribe/vocabulary.txt"
  source_hash = filemd5("${path.module}/transcribe/vocabulary.txt")
}

Then, let’s create a transcribe.tf file:

resource "aws_transcribe_vocabulary" "vocabulary" {
  vocabulary_name     = "example"
  vocabulary_file_uri = "s3://${aws_s3_object.vocabulary.bucket}/${aws_s3_object.vocabulary.key}"
  language_code       = "en-US"
}

We now add the vocabulary name to our lambda environment variables:

resource "aws_lambda_function" "transcribe" {
  function_name    = "transcribe"
  runtime          = "nodejs22.x"
  handler          = "index.handler"
  filename         = data.archive_file.file.output_path
  source_code_hash = data.archive_file.file.output_base64sha256
  role             = aws_iam_role.role.arn

  environment {
    variables = {
      JOB_ROLE_ARN    = "${aws_iam_role.job_role.arn}"
      OUTPUT_KEY      = "${aws_s3_object.transcription.key}"
      VOCABULARY_NAME = "${aws_transcribe_vocabulary.vocabulary.vocabulary_name}"
    }
  }
}

And lastly, you need to then add the Vocabulary ARN to the StartTranscriptionJob policy we created. It should be like:

statement {
  effect = "Allow"
  actions = ["transcribe:StartTranscriptionJob"]
  resources = [
    "arn:aws:transcribe:*:*:transcription-job/*",
    aws_transcribe_vocabulary.vocabulary.arn
  ]
}

You can now push the code to GitHub, wait for the action to finish, and then see the vocabulary in Transcribe.

Then, when you upload a media containing these words, Transcribe will give you a better transcription with a better match of words.

Vocabulary Filter

Sometimes, you might want to filter some words from your transcription, like removing offensive language or redacting/masking sensitive data.

In these cases, you can create and use a custom vocabulary filter, a list of words you want to filter from your transcription.

Things to note when creating your custom vocabulary filter:

Words are case-insensitive. So, “offensive” and “OFFENSIVE” are considered the same.
Word matching works with exact matches. You must include all variations of the words you want to filter from your transcription. Example: “hate”, “hating”, “hated”.
As mentioned previously, filters work with exact matches, so you don’t need to worry about variant words being filtered if you don’t want them. Example: You have the filter for “car”, so “scar” won’t be filtered.
Each entry can only contain one word (no spaces).
Text files must be in plain text with UTF-8 encoding.
100 custom vocabulary filters per account.
Vocabulary filter files can be up to 50 KB in size.
Only characters supported for the selected language can be used. (See documentation here for supported characters)

Just as with the custom vocabulary, we can create our vocabulary filter in two ways:

In line words through the words property in Terraform
Text file in S3

For text files, it needs to be in a txt file, and each word needs to be in a new line:

hate
offensive
masked

We can follow the same approach for the custom vocabulary. Add a file vocabulary_filter.txt with the content, then create the folder and object:

resource "aws_s3_object" "vocabulary_filters_folder" {
  bucket = aws_s3_bucket.bucket.id
  key    = "vocabulary_filters/"
}

resource "aws_s3_object" "vocabulary_filter" {
  bucket      = aws_s3_bucket.bucket.id
  key         = "${aws_s3_object.vocabulary_filters_folder.key}vocabulary_filter.txt"
  source      = "${path.module}/transcribe/vocabulary_filter.txt"
  source_hash = filemd5("${path.module}/transcribe/vocabulary_filter.txt")
}

Then, we can create a new vocabulary filter with:

resource "aws_transcribe_vocabulary_filter" "filter" {
  vocabulary_filter_name     = "example"
  vocabulary_filter_file_uri = "s3://${aws_s3_object.vocabulary_filter.bucket}/${aws_s3_object.vocabulary_filter.key}"
  language_code              = "en-US"
}

Now, if you want a vocabulary with inline words, you can easily add them through the words property:

After pushing to GitHub and the workflow completes, you can find the filters in the Transcribe vocabulary filtering console.

Now, to use it, let’s add the vocabulary filter name to our lambda environment variables:

resource "aws_lambda_function" "transcribe" {
  function_name    = "transcribe"
  runtime          = "nodejs22.x"
  handler          = "index.handler"
  filename         = data.archive_file.file.output_path
  source_code_hash = data.archive_file.file.output_base64sha256
  role             = aws_iam_role.role.arn

  environment {
    variables = {
      JOB_ROLE_ARN           = "${aws_iam_role.job_role.arn}"
      OUTPUT_KEY             = "${aws_s3_object.transcription.key}"
      VOCABULARY_NAME        = "${aws_transcribe_vocabulary.vocabulary.vocabulary_name}"
      VOCABULARY_FILTER_NAME = "${aws_transcribe_vocabulary_filter.filter.vocabulary_filter_name}"
    }
  }
}

Then, let’s give permission to our lambda role to use the filter when calling StartTranscriptionJob in the policies.tf file:

statement {
  effect = "Allow"
  actions = ["transcribe:StartTranscriptionJob"]
  resources = [
    "arn:aws:transcribe:*:*:transcription-job/*",
    aws_transcribe_vocabulary.vocabulary.arn,
    aws_transcribe_vocabulary_filter.filter.arn
  ]
}

And finally, let’s modify our lambda code to use the vocabulary filter when starting a transcription job:

import {
  MediaFormat,
  StartTranscriptionJobCommand,
  ToxicityCategory,
  TranscribeClient,
  VocabularyFilterMethod,
  type StartTranscriptionJobRequest,
} from '@aws-sdk/client-transcribe';
import type { S3Event } from 'aws-lambda';

const JOB_ROLE_ARN = process.env.JOB_ROLE_ARN;
const OUTPUT_KEY = process.env.OUTPUT_KEY; // transcription/
const vocabularyName = process.env.VOCABULARY_NAME;
const vocabularyFilterName = process.env.VOCABULARY_FILTER_NAME;

const transcribeClient = new TranscribeClient({});

export const handler = async (event: S3Event) => {
  for (let record of event.Records) {
    const bucket = record.s3.bucket.name;
    const key = record.s3.object.key;

    const fileInput = `s3://${bucket}/${key}`;
    const mediaFormat = fileInput.split('.').at(-1);

    if (
      !mediaFormat ||
      !Object.values(MediaFormat).includes(mediaFormat as MediaFormat)
    ) {
      console.warn('No media format for this file');
      return;
    }

    const jobName = key.replace('/', '_') + Date.now();

    const jobRequest: StartTranscriptionJobRequest = {
      TranscriptionJobName: jobName,
      Media: { MediaFileUri: fileInput },
      MediaFormat: mediaFormat as MediaFormat,
      LanguageCode: 'en-US',
      OutputBucketName: bucket,
      OutputKey: `${OUTPUT_KEY}${jobName}.json`,
      JobExecutionSettings: {
        DataAccessRoleArn: JOB_ROLE_ARN,
      },
      Settings: {
        VocabularyName: vocabularyName,
        VocabularyFilterMethod: VocabularyFilterMethod.MASK,
        VocabularyFilterName: vocabularyFilterName,
      },
    };

    const job = new StartTranscriptionJobCommand(jobRequest);

    try {
      const response = await transcribeClient.send(job);
      console.log(
        'Finished job %s. Data %s',
        jobName,
        response.TranscriptionJob
      );
    } catch (error: any) {
      console.error(
        "Couldn't start transcription job %s. Error: %s",
        jobName,
        error
      );
      throw error;
    }
  }
};

Notice that we have the VocabularyFilterMethod , it accepts one of the three. Let’s say we have the sentence This is an offensive content :

MASK — The filtered content will be masked. For example: { “transcript”: “This is an *** content.” }
REMOVE -The filtered content will be removed. Example: { “transcript”: “This is an content.” }
TAG — Adds a vocabularyFilterMatch: true tag to the filter content but doesn’t remove it from the transcript. Example:

{
  ...
  "results": {
    "transcripts": [
      { "transcript": "This is an offensive content." }
    ],
    "items": [
      ...
      {
        "id": 3,
        "type": "pronunciation",
        "alternatives": [{ "confidence": "0.758", "content": "offensive" }],
        "start_time": "1.549",
        "end_time": "2.589",
        "vocabularyFilterMatch": true
      }
    ]
  }
}

Content Redaction

Transcribe also has the capability of content redaction, where it can identify PII (personally identifiable information) data and automatically redact or flag it. To do so, we need to update our StartTranscriptionJobRequest to tell our job to do it for us:

import {
  MediaFormat,
  PiiEntityType,
  RedactionOutput,
  StartTranscriptionJobCommand,
  ToxicityCategory,
  TranscribeClient,
  VocabularyFilterMethod,
  type StartTranscriptionJobRequest,
} from '@aws-sdk/client-transcribe';
import type { S3Event } from 'aws-lambda';

const JOB_ROLE_ARN = process.env.JOB_ROLE_ARN;
const OUTPUT_KEY = process.env.OUTPUT_KEY; // transcription/
const vocabularyName = process.env.VOCABULARY_NAME;
const vocabularyFilterName = process.env.VOCABULARY_FILTER_NAME;

const transcribeClient = new TranscribeClient({});

export const handler = async (event: S3Event) => {
  for (let record of event.Records) {
    const bucket = record.s3.bucket.name;
    const key = record.s3.object.key;

    const fileInput = `s3://${bucket}/${key}`;
    const mediaFormat = fileInput.split('.').at(-1);

    if (
      !mediaFormat ||
      !Object.values(MediaFormat).includes(mediaFormat as MediaFormat)
    ) {
      console.warn('No media format for this file');
      return;
    }

    const jobName = key.replace('/', '_') + Date.now();

    const jobRequest: StartTranscriptionJobRequest = {
      TranscriptionJobName: jobName,
      Media: { MediaFileUri: fileInput },
      MediaFormat: mediaFormat as MediaFormat,
      LanguageCode: 'en-US',
      OutputBucketName: bucket,
      OutputKey: `${OUTPUT_KEY}${jobName}.json`,
      JobExecutionSettings: {
        DataAccessRoleArn: JOB_ROLE_ARN,
      },
      Settings: {
        VocabularyName: vocabularyName,
        VocabularyFilterMethod: VocabularyFilterMethod.MASK,
        VocabularyFilterName: vocabularyFilterName,
      },
      ContentRedaction: {
        RedactionOutput: RedactionOutput.REDACTED,
        RedactionType: 'PII', // Only value allowed
        // If PiiEntityTypes is not provided, all PII data is redacted
        PiiEntityTypes: [
          PiiEntityType.CREDIT_DEBIT_NUMBER,
          PiiEntityType.BANK_ACCOUNT_NUMBER,
        ],
      },
    };

    const job = new StartTranscriptionJobCommand(jobRequest);

    try {
      const response = await transcribeClient.send(job);
      console.log(
        'Finished job %s. Data %s',
        jobName,
        response.TranscriptionJob
      );
    } catch (error: any) {
      console.error(
        "Couldn't start transcription job %s. Error: %s",
        jobName,
        error
      );
      throw error;
    }
  }
};

Here, we added the ContentRedaction property to our job request, and we then added the following:

RedactionOutput — It tells our job how to handle the redaction. REDACTED or REDACTED_AND_UNREDACTED .
RedactionType — Right now, the valid value is PII
PiiEntityTypes — A list of the PII type you want to redact. CREDIT_DEBIT_NUMBER, ADDRESS, BANK_ACCOUNT_NUMBER , …

If this property is not provided, all PII data will be redacted.

Custom Model

If your application has extensive data that represents the media that will be transcribed, then you can use a custom model trained with your user data.

This type of model will increase the reliability of your transcriptions by using a model with the exact data your applications use. In many cases, you can dismiss the custom vocabulary because your data already represents all the words that can be transcribed.

We have two types of data that we can use. Training data and tuning data:

Training data — This is the primary dataset used to train a new model from scratch. It typically represents a broad and diverse set of language or domain-specific data. For example, it can be text from your website, training manuals, or application documentation.
Tuning data—This data is used to refine and optimize an already trained model. Tuning data is typically more specialized or narrower in focus than training data. It’s meant to improve the model’s accuracy on a particular subset of data or domain. For example, audio transcripts of phone calls or media content directly relevant to your use case can be used to tune data or data containing slang or specialized vocabulary.

Your training and tuning datasets should have the following requirements:

Plain text (.txt) file. (Word, CSV, and PDF are not accepted)
Single sentence per line.
Encoded in UTF-8.
Doesn’t contain any formatting characters, such as HTML tags.
Less than 2 GB for the total size of all training data.
Less than 200 MB for the total size of optional tuning data.

So, let’s start by uploading our training and tuning data to S3. You can use the data I have here. It is just a simple training dataset from Wikipedia to train the model to be more specialized in games. In the s3.tf file:

resource "aws_s3_object" "clm" {
  bucket = aws_s3_bucket.bucket.id
  key    = "clm/"
}

resource "aws_s3_object" "training_data" {
  bucket = aws_s3_bucket.bucket.id
  key    = "${aws_s3_object.clm.key}training_data/"
}
resource "aws_s3_object" "tune_data" {
  bucket = aws_s3_bucket.bucket.id
  key    = "${aws_s3_object.clm.key}tune_data/"
}

resource "aws_s3_object" "nintendo_switch" {
  bucket      = aws_s3_bucket.bucket.id
  key         = "${aws_s3_object.training_data.key}NintendoSwitch.txt"
  source      = "${path.module}/transcribe/training_data/NintendoSwitch.txt"
  source_hash = filemd5("${path.module}/transcribe/training_data/NintendoSwitch.txt")
}

resource "aws_s3_object" "ps5" {
  bucket      = aws_s3_bucket.bucket.id
  key         = "${aws_s3_object.training_data.key}PlayStation5.txt"
  source      = "${path.module}/transcribe/training_data/PlayStation5.txt"
  source_hash = filemd5("${path.module}/transcribe/training_data/PlayStation5.txt")
}

resource "aws_s3_object" "xbox" {
  bucket      = aws_s3_bucket.bucket.id
  key         = "${aws_s3_object.training_data.key}XboxSeries.txt"
  source      = "${path.module}/transcribe/training_data/XboxSeries.txt"
  source_hash = filemd5("${path.module}/transcribe/training_data/XboxSeries.txt")
}

resource "aws_s3_object" "tune_data_file" {
  bucket      = aws_s3_bucket.bucket.id
  key         = "${aws_s3_object.tune_data.key}tune_data.txt"
  source      = "${path.module}/transcribe/tune_data/tune_data.txt"
  source_hash = filemd5("${path.module}/transcribe/tune_data/tune_data.txt")
}

In the policies.tf , let’s create a new policy document with access to list our bucket and access our training data:

data "aws_iam_policy_document" "transcribe_s3" {
  statement {
    effect = "Allow"

    actions = [
      "s3:GetObject",
    ]

    resources = [
      "${aws_s3_object.clm.arn}*",
    ]
  }
  statement {
    effect = "Allow"

    actions = [
      "s3:ListBucket"
    ]

    resources = [
      "${aws_s3_bucket.bucket.arn}",
    ]
  }
}

Also modify the policies IAM Policy document to add the language model ARN to the resources:

data "aws_iam_policy_document" "policies" {
  statement {
    effect = "Allow"

    actions = [
      "logs:CreateLogGroup",
      "logs:CreateLogStream",
      "logs:PutLogEvents"
    ]

    resources = ["arn:aws:logs:*:*:*"]
  }

  statement {
    effect = "Allow"

    actions = ["iam:PassRole"]

    resources = [aws_iam_role.job_role.arn]
  }

  statement {
    effect = "Allow"

    actions = ["transcribe:StartTranscriptionJob"]

    resources = [
      "arn:aws:transcribe:*:*:transcription-job/*",
      aws_transcribe_vocabulary.vocabulary.arn,
      aws_transcribe_vocabulary_filter.filter.arn,
      aws_transcribe_language_model.model.arn
    ]
  }
}

In the transcribe.tf file, let’s create our custom language model. Here, we’ll provision the model and assign a role with access to our datasets so it can train:

resource "aws_transcribe_language_model" "model" {
  model_name = "example"

  // NarrowBand: Use this option for audio with a sample rate of less than 16,000 Hz. This model type is typically used for telephone conversations recorded at 8,000 Hz.
  // WideBand: Use this option for audio with a sample rate greater than or equal to 16,000 Hz.
  base_model_name = "WideBand"

  language_code = "en-US"
  input_data_config {
    s3_uri               = "s3://${aws_s3_object.training_data.bucket}/${aws_s3_object.training_data.key}"
    tuning_data_s3_uri   = "s3://${aws_s3_object.tune_data.bucket}/${aws_s3_object.tune_data.key}"
    data_access_role_arn = aws_iam_role.transcribe_clm.arn
  }

  depends_on = [aws_iam_role_policy.transcribe_clm_policy]
}

resource "aws_iam_role" "transcribe_clm" {
  name               = "transcribe_clm"
  assume_role_policy = data.aws_iam_policy_document.transcribe_assume_role.json
}

resource "aws_iam_role_policy" "transcribe_clm_policy" {
  name = "transcribe_clm"
  role = aws_iam_role.transcribe_clm.id

  policy = data.aws_iam_policy_document.transcribe_s3.json
}

Then, let’s add the model name to our lambda’s environment variables in the lambdas.tf file:

resource "aws_lambda_function" "transcribe" {
  function_name    = "transcribe"
  runtime          = "nodejs22.x"
  handler          = "index.handler"
  filename         = data.archive_file.file.output_path
  source_code_hash = data.archive_file.file.output_base64sha256
  role             = aws_iam_role.role.arn

  environment {
    variables = {
      JOB_ROLE_ARN           = "${aws_iam_role.job_role.arn}"
      OUTPUT_KEY             = "${aws_s3_object.transcription.key}"
      VOCABULARY_NAME        = "${aws_transcribe_vocabulary.vocabulary.vocabulary_name}"
      VOCABULARY_FILTER_NAME = "${aws_transcribe_vocabulary_filter.filter.vocabulary_filter_name}"
      CUSTOM_MODEL           = "${aws_transcribe_language_model.model.model_name}"
    }
  }
}

You can then push the code to GitHub Actions and wait for it to finish. According to AWS, training a Custom Language Model can take 6 to 10 hours, depending on the size of your training data. Mine took around 3 hours. Terraform will wait for it to finish training before completing the apply run.

If you are working on a distributed team, I recommend creating the IAM role and the training and tuning data in S3 through Terraform, and then you can make the Custom Language Model in the AWS console. Once finished, you can reference it in your Terraform configuration with a data source. This way, the model training won’t impact the delivery of other Terraform resources.

While it is being trained, you can see the status in the Transcribe Custom language model console:

After it finishes, you should see it as completed in the console:

We need to modify our Lambda code to pass the CLM name to the job request. Get the model name from the CUSTOM_MODEL environment variable and then pass in the ModelSettings.LanguageModelName property:

import {
  MediaFormat,
  PiiEntityType,
  RedactionOutput,
  StartTranscriptionJobCommand,
  ToxicityCategory,
  TranscribeClient,
  VocabularyFilterMethod,
  type StartTranscriptionJobRequest,
} from '@aws-sdk/client-transcribe';
import type { S3Event } from 'aws-lambda';

const JOB_ROLE_ARN = process.env.JOB_ROLE_ARN;
const OUTPUT_KEY = process.env.OUTPUT_KEY; // transcription/
const vocabularyName = process.env.VOCABULARY_NAME;
const vocabularyFilterName = process.env.VOCABULARY_FILTER_NAME;
const customModel = process.env.CUSTOM_MODEL;

const transcribeClient = new TranscribeClient({});

export const handler = async (event: S3Event) => {
  for (let record of event.Records) {
    const bucket = record.s3.bucket.name;
    const key = record.s3.object.key;

    const fileInput = `s3://${bucket}/${key}`;
    const mediaFormat = fileInput.split('.').at(-1);

    if (
      !mediaFormat ||
      !Object.values(MediaFormat).includes(mediaFormat as MediaFormat)
    ) {
      console.warn('No media format for this file');
      return;
    }

    const jobName = key.replace('/', '_') + Date.now();

    const jobRequest: StartTranscriptionJobRequest = {
      TranscriptionJobName: jobName,
      Media: { MediaFileUri: fileInput },
      MediaFormat: mediaFormat as MediaFormat,
      LanguageCode: 'en-US',
      OutputBucketName: bucket,
      OutputKey: `${OUTPUT_KEY}${jobName}.json`,
      JobExecutionSettings: {
        DataAccessRoleArn: JOB_ROLE_ARN,
      },
      Settings: {
        VocabularyName: vocabularyName,
        VocabularyFilterMethod: VocabularyFilterMethod.MASK,
        VocabularyFilterName: vocabularyFilterName,
      },
      ModelSettings: {
        LanguageModelName: customModel,
      },
      ContentRedaction: {
        RedactionOutput: RedactionOutput.REDACTED,
        RedactionType: 'PII', // Only value allowed
        // If PiiEntityTypes is not provided, all PII data is redacted
        PiiEntityTypes: [
          PiiEntityType.CREDIT_DEBIT_NUMBER,
          PiiEntityType.BANK_ACCOUNT_NUMBER,
        ],
      },
    };

    const job = new StartTranscriptionJobCommand(jobRequest);

    try {
      const response = await transcribeClient.send(job);
      console.log(
        'Finished job %s. Data %s',
        jobName,
        response.TranscriptionJob
      );
    } catch (error: any) {
      console.error(
        "Couldn't start transcription job %s. Error: %s",
        jobName,
        error
      );
      throw error;
    }
  }
};

Once you deploy your lambda code, you can test it by adding a media file with a more specific language for your domain, and you should start getting better transcriptions.

Testing in AWS Console

You can also test your Transcribe vocabularies, filters, and custom language models in the Transcribe Real-time transcription console:

Then, you can also specify the Vocabulary, Filter, and Custom Language Model in the Customizations options:

After configuring, you can click on Start streaming to test your Transcribe configurations.

Conclusion

In this story, we could learn more about AWS Transcribe, the serverless service to convert speech to text, as well as learn how you can quickly get started with a simple serverless architecture with S3, Lambda, and Transcribe to have your application to convert media files into text.

You could learn about the difference between using General and Custom language models and when you’d train your model.

Also, you can enrich the model you are using by adding vocabulary with words that are related to your domain.

We saw how we could filter and redact data by using custom vocabulary filters to match words we don’t want showing in our transcriptions and utilizing the content redaction native feature to redact or flag PII data automatically.

We saw how we can quickly train and tune a CLM with our data in the Custom Language Models domain. The only downside of doing that with Terraform is that the process will hang until the model is trained, which can take a few hours to complete, depending on the size of your data.

I hope you had fun with this story.

Happy coding! 💻

The code for this project can be found here.

Building a serverless API with email notifications in AWS with Terraform - Part 1

Andre Lopes — Tue, 09 Jan 2024 17:42:31 +0000

Cover Image by Fotis Fotopoulos on Unsplash

Hi earthlings!

In this journey, we'll build a system using Serverless architecture, Terraform, and GitHub Actions. Buckle up as we embark on an adventure.

Our main project will now be a serverless architecture featuring API Gateway, Lambdas, DynamoDB, SNS, and SQS. We'll work with Terraform, our infrastructure-as-code tool, and use GitHub Actions for our continuous integration and deployment, where we'll deploy our infrastructure and lambda apps with a simple push to the main branch.

We'll organize our HTTP methods, configure the AWS provider in Terraform, and even set up an S3 bucket for storing our precious Terraform state.

Let us begin!

Requirements

An AWS account
Any Code Editor of your choice - I use Visual Studio Code
NodeJS
GitHub account - We'll be using GitHub Actions to deploy our Terraform code

Regarding AWS Costs

Everything we'll be using here is free or has a very low cost that will not be charged unless you add very high usage. If you are afraid of unknown charges, you can set up a $0.01 budget to alert you if you are being charged for anything.

The Project

Here, we are going to be building a whole serverless architecture:

API Gateway: This is where the endpoints will be mapped and exposed.
Lambdas: They will handle API Gateway events and the SQS events.
DynamoDB: It will be our database.
SQS: Our message queue, where the email notification lambda will be notified whenever a movie is created, deleted, or updated.
SNS: Notification services to send events to SQS for a fanout pattern
SES: AWS Simple Email System to manage and send emails from AWS

We'll also be using:

Terraform: Our infrastructure as code that will create and manage our whole AWS infrastructure.
GitHub Actions: Our CI/CD, which will build and deploy our infrastructure and our lambdas

In this part 1, we'll be implementing the following components of our project.

Why Serverless?

Serverless computing is a cloud computing model where you don't have to provision or manage servers. Instead, the cloud provider automatically manages the infrastructure, allowing developers to focus solely on writing code and deploying applications.

The term "serverless" doesn't mean no servers are involved. It means you don't have to worry about the underlying server infrastructure.

Some of the benefits of serverless are:

Cost Savings: You only pay for the computing resources your code consumes.
Scalability: Serverless platforms automatically scale your applications based on demand without manual intervention.
Zero Idle Capacity: Your resources are only allocated when needed, so you won't have provisioned resources without being used.

Let's Start

Let's begin our project. We are adding our first lambda to get a movie by its ID.

Lambda Module

Create a folder for your project, and inside it, create a folder named iac. This is where we'll be adding all our infrastructure as a code. Now create a new folder inside it named modules. Here, we'll be adding our reusable Terraform modules. And now, add a folder lambda for our Lambda function module.

Inside the lambda folder, create three files: main.tf, datasources.tf, and variables.tf.

main.tf:

resource "aws_iam_role" "iam_for_lambda" {
  name               = "${var.name}-lambda-role"
  assume_role_policy = data.aws_iam_policy_document.assume_role.json
}

resource "aws_lambda_function" "lambda" {
  filename      = data.archive_file.lambda.output_path
  function_name = var.name
  role          = aws_iam_role.iam_for_lambda.arn
  handler       = var.handler
  runtime       = "nodejs20.x"
}

We also declare a role, which all lambda functions need, and the lambda code itself in aws_lambda_function.

Note the keywords data and var. The first is for data from the data sources, and the second is for anything passed to the module through variables.

Now for the variables.tf:

variable "name" {
  description = "The name of the Lambda function"
  type        = string
  nullable    = false
}

variable "handler" {
  description = "The handler function in your code for he Lambda function"
  type        = string
  default     = "index.handler"
}

And for the datasources.tf:

locals {
  filename = strcontains(var.runtime, "node") ? "index.mjs" : "main"
}

data "archive_file" "lambda" {
  type        = "zip"
  source_file = "./modules/lambda/init_code/${local.filename}"
  output_path = "${var.name}_lambda_function_payload.zip"
}

data "aws_iam_policy_document" "assume_role" {

  statement {
    effect = "Allow"

    principals {
      type        = "Service"
      identifiers = ["lambda.amazonaws.com"]
    }

    actions = ["sts:AssumeRole"]

  }
}

Here, we define the IAM policy for the lambda role and the file that will be added to the lambda.

This file is required for our Terraform code, even if you are doing deployment in a different flow, which you will be doing.

This is also seen in the filename local variable, where we assign the file depending on the lambda runtime. So, let's add our seed code.

data "aws_iam_policy_document" "lambda_logging" {
  statement {
    effect = "Allow"

    actions = [
      "logs:CreateLogGroup",
      "logs:CreateLogStream",
      "logs:PutLogEvents",
    ]

    resources = ["arn:aws:logs:*:*:*"]
  }
}

And then, in the main.tf, add the policy attachment:

resource "aws_iam_policy" "lambda_logging" {
  name        = "lambda_logging_${aws_lambda_function.lambda.function_name}"
  path        = "/"
  description = "IAM policy for logging from a lambda"
  policy      = data.aws_iam_policy_document.lambda_logging.json
}

resource "aws_iam_role_policy_attachment" "lambda_logs" {
  role       = aws_iam_role.iam_for_lambda.name
  policy_arn = aws_iam_policy.lambda_logging.arn
}

Create a folder named init_code under the lambda module folder.

For the Node.js seed code, you can create a new file index.mjs and add the following code:

// Default handler generated in AWS
export const handler = async (event) => {
  const response = {
    statusCode: 200,
    body: JSON.stringify('Hello from Lambda!'),
  };
  return response;
};

Note that it needs to be mjs file because we are not adding a project.json file to define the module. The file needs to be with this extension so Node.js will handle the code as ECMAScript modules.

Adding the main infra code

In the iac folder, create a lambdas.tf file with the following code:

module "get_movie_lambda" {
  source        = "./modules/lambda"
  name          = "get-movie"
  runtime       = "nodejs20.x"
  handler       = "index.handler"
}

We also need to configure Terraform to use AWS as its provider. Create a provider.tf file with the following code:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

# Configure the AWS Provider
provider "aws" {
  region = var.region
}

Now, create two files, variables.tf to declare the default variables of our IaC:

variable "region" {
  description = "Default region of your resources"
  type        = string
  default     = "eu-central-1"
}

And for variables.tfvars to pass variable values that are not secret, but we might want to change depending on the deployment configuration:

region="eu-central-1" // Chage here to your region here

If you'd like Terraform to keep track of the changes to update the components, you need to add where it will save and manage the state. Here, we'll be using an S3 bucket for that.

Create an S3 bucket with the name terraform-medium-api-notification and modify the provider.tf file with the following code:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }

  backend "s3" {
    bucket = "terraform-medium-api-notification"
    key    = "state"
    region = "eu-central-1" // Chage here to your region here
  }
}

# Configure the AWS Provider
provider "aws" {
  region = var.region
}

Note that you can choose the region that you are nearest to instead of eu-central-1. I just chose it because it is the closest to me. You could also provide this through the AWS_DEFAULT_REGION or AWS_REGION environment variables.

We are ready to build the workflow to deploy our infrastructure to AWS.

Deploying the infrastructure

To deploy our infrastructure, we'll be using Github Actions.
The CI solutions in GitHub allow us to run scripts in our code when we change it. If you'd like to know more about it, check the documentation here.

To perform this step, you'll need to generate an AWS Access Key and Secret for a user that has the rights to create the resources you define in AWS.

Add these secrets to your repository action secrets in your in Settings:

Now, in the root folder, let's create a .github folder and a workflows folder inside of it. Now, create a file named deploy-infrastructure.yml and add the following code:

name: Deploy Infrastructure
on:
  push:
    branches:
      - main
    paths:
      - iac/**/*
      - .github/workflows/deploy-infra.yml

defaults:
  run:
    working-directory: iac/

jobs:
  terraform:
    name: "Terraform"
    runs-on: ubuntu-latest
    steps:
      # Checkout the repository to the GitHub Actions runner
      - name: Checkout
        uses: actions/checkout@v3

      - name: Configure AWS Credentials Action For GitHub Actions
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: eu-central-1 # Use your preferred region

      # Install the latest version of Terraform CLI and configure the Terraform CLI configuration file with a Terraform Cloud user API token
      - name: Setup Terraform
        uses: hashicorp/setup-terraform@v3

      # Initialize a new or existing Terraform working directory by creating initial files, loading any remote state, downloading modules, etc.
      - name: Terraform Init
        run: terraform init

      # Checks that all Terraform configuration files adhere to a canonical format
      - name: Terraform Format
        run: terraform fmt -check

      # Generates an execution plan for Terraform
      - name: Terraform Plan
        run: terraform plan -out=plan -input=false -var-file="variables.tfvars"

        # On push to "main", build or change infrastructure according to Terraform configuration files
        # Note: It is recommended to set up a required "strict" status check in your repository for "Terraform Cloud". See the documentation on "strict" required status checks for more information: https://help.github.com/en/github/administering-a-repository/types-of-required-status-checks
      - name: Terraform Apply
        run: terraform apply -auto-approve -input=false plan

Every time you push a change to a file inside the iac folder, it will trigger this action, and it will automatically generate the resources in AWS for you:

Note that in the Terraform Plan step, terraform outputs all the changes it will perform in AWS based on the current state in the S3 bucket.

Now you can go to AWS on the Lambda page and see your recently created function:

To test it, you click on it and then on the Test tab.

There, you can give a name to the test event that will be sent to the Lambda and then click on the Test button.

You should see a success notification with the return from the lambda:

Adding GET endpoint

Now, let's add a GET endpoint through API Gateway so we can call our lambda through HTTP requests.

Let's first create a module for our HTTP methods. Under the folder modules, create a folder rest-api-method. Then, create three files: main.tf, variables.tf, and outputs.tf.

For the variables.tf, add the following code:

variable "http_method" {
  description = "The HTTP method"
  type        = string
}

variable "resource_id" {
  description = "The ID of the resource this method is attached to"
  type        = string
}

variable "api_id" {
  description = "The ID of the API this method is attached to"
  type        = string
}

variable "integration_uri" {
  description = "The URI of the integration this method will call"
  type        = string
}

variable "resource_path" {
  description = "The path of the resource"
  type        = string
}

variable "lambda_function_name" {
  description = "The name of the Lambda function that will be called"
  type        = string
}

variable "region" {
  description = "The region of the REST API resources"
  type        = string
}

variable "account_id" {
  description = "The ID of the AWS account"
  type        = string
}

For the outputs.tf:

output "id" {
  value = aws_api_gateway_method.method.id
}

output "integration_id" {
  value = aws_api_gateway_integration.integration.id
}

Now for the main.tf:

resource "aws_api_gateway_method" "method" {
  authorization = "NONE"
  http_method   = var.http_method
  resource_id   = var.resource_id
  rest_api_id   = var.api_id
}

resource "aws_api_gateway_integration" "integration" {
  http_method             = aws_api_gateway_method.method.http_method
  integration_http_method = "POST" # Lambda functions can only be invoked via POST
  resource_id             = var.resource_id
  rest_api_id             = var.api_id
  type                    = "AWS_PROXY"
  uri                     = var.integration_uri
}

resource "aws_lambda_permission" "apigw_lambda" {
  statement_id  = "AllowExecutionFromAPIGateway"
  action        = "lambda:InvokeFunction"
  function_name = var.lambda_function_name
  principal     = "apigateway.amazonaws.com"
  source_arn    = "arn:aws:execute-api:${var.region}:${var.account_if}:${var.api_id}/*/${aws_api_gateway_method.method.http_method}${var.resource_path}"
}

This will generate an HTTP method attached to your API and use lambda proxy integration. We want our Lambda to be responsible for the HTTP behavior of the request and response. So, only the API Gateway will pass it through.
Now, in the root iac folder, create a rest-api.tf file and add the following code:

# API Gateway
resource "aws_api_gateway_rest_api" "movies_api" {
  name = "movies-api"
}

resource "aws_api_gateway_deployment" "movies_api_deployment" {
  rest_api_id = aws_api_gateway_rest_api.movies_api.id
  stage_name  = aws_api_gateway_stage.live.stage_name

  triggers = {
    redeployment = sha1(jsonencode([
      aws_api_gateway_resource.movies_root_resource.id,
      module.get_movie_method.id,
      module.get_movie_method.integration_id,
    ]))
  }

  lifecycle {
    create_before_destroy = true
  }
}

resource "aws_api_gateway_stage" "live" {
  deployment_id = aws_api_gateway_deployment.movies_api_deployment.id
  rest_api_id   = aws_api_gateway_rest_api.movies_api.id
  stage_name    = "live"
}

resource "aws_api_gateway_resource" "movies_root_resource" {
  parent_id   = aws_api_gateway_rest_api.movies_api.root_resource_id
  path_part   = "movies"
  rest_api_id = aws_api_gateway_rest_api.movies_api.id
}

module "get_movie_method" {
  source               = "./modules/rest-api-method"
  api_id               = aws_api_gateway_rest_api.movies_api.id
  http_method          = "GET"
  resource_id          = aws_api_gateway_resource.movies_root_resource.id
  resource_path        = aws_api_gateway_resource.movies_root_resource.path
  integration_uri      = module.get_movie_lambda.invoke_arn
  lambda_function_name = module.get_movie_lambda.name
  region               = var.region
  account_id           = vat.account_id
}
In the variables.tf, add the variable for account_id:
variable "account_id" {
  description = "The ID of the default AWS account"
  type        = string
}

You can either add your account ID to the variables.tfvars files, or you can add it as an inline variable to the workflow Terraform Plan and the Terraform Apply steps:

# Generates an execution plan for Terraform
- name: Terraform Plan
  run: terraform plan -out=plan -input=false -var-file="variables.tfvars" -var account_id="YOUR_ACCOUNT_ID"

# On push to "main", build or change infrastructure according to Terraform configuration files
# Note: It is recommended to set up a required "strict" status check in your repository for "Terraform Cloud". See the documentation on "strict" required status checks for more information: https://help.github.com/en/github/administering-a-repository/types-of-required-status-checks
- name: Terraform Apply
  run: terraform apply -auto-approve -input=false plan

Just remember to replace the value of YOUR_ACCOUNT_ID with your account account ID value.

This will generate an API with the resource movies, which will be the path /movies of your API.

It will also create a stage named live. Stages are the equivalent of deployment environments. You need the API deployed to a stage to call it. So for our movies endpoint, it will be /live/movies.

Then, it will create a deployment that will configure a rule that it should deploy to the live stage whenever we make changes to the method, integration, or resource.

Now, push it to GitHub and wait for the workflow and your API to be created. After it is finished, you can go to the API Gateway page of AWS and see your API.

And when you click on it, you can see all the details about the resources:

To see the public URL, you can go to the Stages section:

Now, if you call the /movies in your browser, you should get the response from the Lambda

We must make one adjustment to ensure we use the correct path.

We created the resource /movies and added the method GET there, but our lambda will fetch a movie by ID in the future, so we need to create a new resource to attach our lambda to it correctly.

So, let's create a new resource by adding the following code to the root rest-api.tf file:

resource "aws_api_gateway_resource" "movie_resource" {
  parent_id   = aws_api_gateway_resource.movies_root_resource.id
  path_part   = "{movieID}"
  rest_api_id = aws_api_gateway_rest_api.movies_api.id
}

Add it to the redeployment trigger in the movies_api_deployment :

resource "aws_api_gateway_deployment" "movies_api_deployment" {
  rest_api_id = aws_api_gateway_rest_api.movies_api.id

  triggers = {
    redeployment = sha1(jsonencode([
      aws_api_gateway_resource.movies_root_resource.id,
      aws_api_gateway_resource.movie_resource.id,
      module.get_movie_method.id,
      module.get_movie_method.integration_id,
    ]))
  }

  lifecycle {
    create_before_destroy = true
  }
}

And then modifying the get_movie_method module to point to the new resource:

module "get_movie_method" {
  source               = "./modules/rest-api-method"
  api_id               = aws_api_gateway_rest_api.movies_api.id
  http_method          = "GET"
  resource_id          = aws_api_gateway_resource.movie_resource.id
  resource_path        = aws_api_gateway_resource.movie_resource.path
  integration_uri      = module.get_movie_lambda.invoke_arn
  lambda_function_name = module.get_movie_lambda.name
}

Push the code to GitHub, and Terraform will modify your infrastructure.
Your API should look like this:

Adding DynamoDB

Now that we have a functioning API let's add our database, DynamoDB, and hook it to our GET endpoint with some seed data.

Terraforming DynamoDB

So, let's start by adding a new file to our iac folder named dynamodb.tf with the following code:

resource "aws_dynamodb_table" "movies-table" {
  name           = "Movies"
  billing_mode   = "PROVISIONED"
  read_capacity  = 1
  write_capacity = 1
  hash_key       = "ID"
  range_key      = "Title"

  attribute {
    name = "ID"
    type = "S"
  }
}

This will generate a minimum capacity table named Movies, and with a partition key named ID of type string.

When you push the code to GitHub, and the action runs, you can go to the DynamoDB section of AWS and see the Movies table there.

Let's add a few seed items. In the dynamodb.tf file, add the following code for four table items:

resource "aws_dynamodb_table_item" "the_matrix" {
  table_name = aws_dynamodb_table.movies-table.name
  hash_key   = aws_dynamodb_table.movies-table.hash_key
  range_key  = aws_dynamodb_table.movies-table.range_key

  item = jsonencode(
    {
      ID    = { S = "1" },
      Title = { S = "The Matrix" },
      Genres = { SS = [
        "Action",
        "Sci-Fi",
        ]
      },
      Rating = { N = "8.7" }
    }
  )
}

resource "aws_dynamodb_table_item" "scott_pilgrim" {
  table_name = aws_dynamodb_table.movies-table.name
  hash_key   = aws_dynamodb_table.movies-table.hash_key
  range_key  = aws_dynamodb_table.movies-table.range_key

  item = jsonencode(
    {
      ID    = { S = "2" },
      Title = { S = "Scott Pilgrim vs. the World" },
      Genres = { SS = [
        "Action",
        "Comedy",
        ]
      },
      Rating = { N = "7.5" }
    }
  )
}

resource "aws_dynamodb_table_item" "star_wars" {
  table_name = aws_dynamodb_table.movies-table.name
  hash_key   = aws_dynamodb_table.movies-table.hash_key
  range_key  = aws_dynamodb_table.movies-table.range_key

  item = jsonencode(
    {
      ID    = { S = "3" },
      Title = { S = "Star Wars: Episode IV - A New Hope" },
      Genres = { SS = [
        "Action",
        "Adventure",
        "Fantasy",
        "Sci-Fi",
        ]
      },
      Rating = { N = "8.6" }
    }
  )
}

resource "aws_dynamodb_table_item" "star_wars_v" {
  table_name = aws_dynamodb_table.movies-table.name
  hash_key   = aws_dynamodb_table.movies-table.hash_key
  range_key  = aws_dynamodb_table.movies-table.range_key

  item = jsonencode(
    {
      ID    = { S = "4" },
      Title = { S = "Star Wars: Episode V - The Empire Strikes Back" },
      Genres = { SS = [
        "Action",
        "Adventure",
        "Fantasy",
        "Sci-Fi",
        ]
      },
      Rating = { N = "8.7" }
    }
  )
}

Now push to GitHub, wait for the workflow to run, and go to the DynamoDB Table in AWS to explore the table items and see the created records:

Updating the Lambda to fetch by ID

Now that we have our data, we need to modify our lambda to fetch our items.
First, we need to give our Lambda role rights to do GetItem actions in the Movies table.
Add the following code to the outputs.tf file in the lambda module folder:

output "role_name" {
  value = aws_iam_role.iam_for_lambda.name
}

Now, in the iac folder, add a file named iam-polices.tf with the following code:

data "aws_iam_policy_document" "get_movie_item" {
  statement {
    effect = "Allow"

    actions = [
      "dynamodb:GetItem",
    ]

    resources = [
      aws_dynamodb_table.movies-table.arn
    ]
  }
}

resource "aws_iam_policy" "get_movie_item" {
  name        = "get_movie_item"
  path        = "/"
  description = "IAM policy allowing GET Item on Movies DynamoDB table"
  policy      = data.aws_iam_policy_document.get_movie_item.json
}

resource "aws_iam_role_policy_attachment" "allow_getitem_get_movie_lambda" {
  role       = module.get_movie_lambda.role_name
  policy_arn = aws_iam_policy.get_movie_item.arn
}

This will generate a policy that allows GetItem in the Movies table and attach it to the current lambda IAM role.

Now, in the root folder, create a folder named apps, and then a folder get-movie.

Inside this folder, let's start a npm project with:

npm init -y

This will generate a new package.json file.

Most packages required for the lambda to work and connect with AWS are already packed in AWS and updated occasionally. We are creating this mostly to have the packages available in our local development environment and to set up our module types.

In the package.json file, add the following property:

"type": "module"

Your file should look similar to:

{
  "name": "get-movie",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "type": "module",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC"
}

Note that if the package is unavailable in the AWS environment, you must pack the node_modules folder with your lambda function code. Or create a Lambda layer that will hold the node_modules and can be shared between lambdas.

Let's install the packages we'll need with:

npm i --save aws-sdk @aws-sdk/client-dynamodb @aws-sdk/lib-dynamodb

Now, create a folder named src and add a file index.js in it.

We'll add the code to fetch a movie by its ID with:

import { DynamoDBClient } from "@aws-sdk/client-dynamodb";
import { DynamoDBDocumentClient, GetCommand } from "@aws-sdk/lib-dynamodb";

const tableName = "Movies";

export const handler = async (event) => {
  const movieID = event.pathParameters?.movieID;

  if (!movieID) {
    return {
      statusCode: 400,
      body: JSON.stringify({
        message: "Movie ID missing",
      }),
    };
  }

  console.log("Getting movie with ID ", movieID);

  const client = new DynamoDBClient({});
  const docClient = DynamoDBDocumentClient.from(client);

  const command = new GetCommand({
    TableName: tableName,
    Key: {
      ID: movieID.toString(),
    },
  });

  try {
    const dynamoResponse = await docClient.send(command);
    if (!dynamoResponse.Item) {
      return {
        statusCode: 404,
        body: JSON.stringify({
          message: "Movie not found",
        }),
      };
    }

    const body = {
      title: dynamoResponse.Item.Title,
      rating: dynamoResponse.Item.Rating,
      id: dynamoResponse.Item.ID,
    };

    body.genres = Array.from(dynamoResponse.Item.Genres);

    const response = {
      statusCode: 200,
      body: JSON.stringify(body),
    };

    return response;
  } catch (e) {
    console.log(e);

    return {
      statusCode: 500,
      body: JSON.stringify({
        message: e.message,
      }),
    };
  }
};

This lambda gets the event sent by API Gateway and extracts the movie ID. Then we do some simple validations and get the movie from DynamoDB, transform the data to an API resource so we don't expose our data model, and return it to the API Gateway to send to the client.

You can see the documentation here if you'd like to learn more about the event from API Gateway with Lambda proxy integration.

Remember to stringify the body, or you'll face 500 errors.

Building and deploying

Lastly, we must create a quick build script to organize our code.
First, install the following package:

npm i -D copyfiles

I'm using it because it makes commands to copy files independent from operating systems.

In the package.json file, add the following build script:

{
  "name": "get-movie",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "type": "module",
  "scripts": {
    "build": "copyfiles -u 1 src/**/* build/ && copyfiles package.json build/",
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "dependencies": {
    "@aws-sdk/client-dynamodb": "^3.468.0",
    "@aws-sdk/lib-dynamodb": "^3.468.0",
    "aws-sdk": "^2.1513.0"
  },
  "devDependencies": {
    "copyfiles": "^2.4.1"
  }
}

And now, let's add the workflow that will push our code to the get-movie lambda.

Create a deploy-get-movie-lambda.yml file in the .github/workflows folder and add the following code:

name: Deploy Get Movie Lambda
on:
  push:
    branches:
      - main
    paths:
      - apps/get-movie/**/*
      - .github/workflows/deploy-get-movie-lambda.yml

defaults:
  run:
    working-directory: apps/get-movie/

jobs:
  terraform:
    name: "Deploy GetMovie Lambda"
    runs-on: ubuntu-latest
    steps:
      # Checkout the repository to the GitHub Actions runner
      - name: Checkout
        uses: actions/checkout@v3

      - name: Setup NodeJS
        uses: actions/setup-node@v4
        with:
          node-version: 20

      - name: Configure AWS Credentials Action For GitHub Actions
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: eu-central-1

      - name: Install packages
        run: npm install

      - name: Build
        run: npm run build

      - name: Zip build
        run: zip -r -j main.zip ./build

      - name: Update Lambda code
        run: aws lambda update-function-code --function-name=get-movie --zip-file=fileb://main.zip

Remember to set the correct region.

Now push the code to GitHub, wait for it to run, and then call the API Gateway endpoint /movies/1. You should receive a response similar to:

{
  "id":"1",
  "title":"The Matrix",
  "rating":8.7,
  "genres":[
    "Action",
    "Sci-Fi"
  ]
}

Amazing! We have our first endpoint completed!

Conclusion

In this first part, we built the foundation for our project, a serverless CRUD API with email notification.

We saw how to write and deploy our infrastructure using Terraform, a powerful IaC tool that facilitates how we build and manage our infrastructure.

We wrote our first lambda using NodeJS and easily connected it to DynamoDB to fetch our data.

In the next part, we'll dive deeper and finish our CRUD API with more lambdas, but we'll build some of them using other languages, like Go.

Hope to see you there!

The code for this project can be found here.

Happy coding! 💻

The only .NET middleware guide you’ll need

Andre Lopes — Tue, 09 Jan 2024 17:35:03 +0000

Cover Image by Lukas from Pixabay

Olá humanos!

In a nutshell, middleware is any service that runs between the client and server. For example, an authorization server that will verify if the client has the right to access a resource.

In the .NET API domain, just like with NodeJS, it is any code built to run between when your API receives a request from the client until it returns a response.

A good example is the CORS middleware. If enabled, it verifies if the origin, method, headers, and credentials are allowed to process with your API. If the request is invalid, it prevents it from reaching the requested endpoint.

To use middleware, you need to do that in the Program.cs right after your web application is built with builder.Build() and a WebApplication instance is generated.

Now you can add middlewares, like:

app.UseAuthorization();
app.UseCors();
app.UseHttpsRedirection();

Custom middleware

Previously, we talked about how to use pre-built middlewares. But what if you want to write your own implementation?

A custom middleware will give you the power to read and manipulate the request and the response.

To do that, we have multiple ways of doing it.

Inline middleware

The simplest way of writing a custom middleware is to write it directly in the WebApplication through the method Use, in the Program.cs.

var app = builder.Build();

app.Use(async (context, next) =>
{
    var method = context.Request.Method;
    var path = context.Request.Path;

    var startTime = Stopwatch.GetTimestamp();

    await next(context);

    var responseCode = context.Response.StatusCode;

    Console.WriteLine("Request: {0} {1} response with Status {2} within {3} ms", method, path, responseCode, elapsedTime.TotalMilliseconds);
});

The middleware above prints to the console how long it took for a request to respond.

Middleware class

A better and more recommended approach to writing your custom middleware is to define it in a class.

To do that, you can create a class and implement the method InvokeAsync:

public class RequestLoggingMiddleware
{
    private readonly RequestDelegate _next;

    public RequestLoggingMiddleware(RequestDelegate next)
    {
        _next = next;
    }

    public async Task InvokeAsync(HttpContext context)
    {
      var method = context.Request.Method;
      var path = context.Request.Path;

      var startTime = Stopwatch.GetTimestamp();

      await _next(context);

      var elapsedTime = Stopwatch.GetElapsedTime(startTime);

      var responseCode = context.Response.StatusCode;

      Console.WriteLine("Request: {0} {1} response with Status {2} within {3} ms", method, path, responseCode, elapsedTime.TotalMilliseconds);
    }
}

Now you can register it in the Program.cs:

var app = builder.Build();

app.UseMiddleware<RequestLoggingMiddleware>();

With a middleware class, you get:

Better separation of responsibilities
Better code organization
Dependency injection usage

Factory-based activated

This is similar to how the middleware class works, but it adds a strong-typed layer to it.

Here, you make your middleware class implement the IMiddleware interface, which has the InvokeAsync method to be implemented.

Another difference is that instead of having the RequestDelegate in the constructor, it will always be passed in the InvokeAsync method.

public class RequestLoggingMiddleware : IMiddleware
{    
    public async Task InvokeAsync(HttpContext context, RequestDelegate next)
    {
      var method = context.Request.Method;
      var path = context.Request.Path;

      var startTime = Stopwatch.GetTimestamp();

      await next(context);

      var elapsedTime = Stopwatch.GetElapsedTime(startTime);

      var responseCode = context.Response.StatusCode;

      Console.WriteLine("Request: {0} {1} response with Status {2} within {3} ms", method, path, responseCode, elapsedTime.TotalMilliseconds);
    }
}

And then, you just need to register it in the DI container.

builder.Services.AddTransient<ExampleMiddleware>();

Middleware Class

As mentioned above, using middleware class you can get more out of designing a custom middleware.

Lifecycle and dependency injection

A middleware class is built during application startup, which means that the instance itself is a Singleton and the InvokeAsync method is called on every request.

We have two methods of dependency injection:

Constructor — The dependencies are injected during middleware construction
InvokeAsync / Invoke — The dependencies are injected each time the middleware is invoked

For Singleton and Transient, you can use both types of injection

For Scoped you must use InvokeAsync/Invoke method injection. It will throw a runtime error when invoking the middleware.

So the best practice for injecting are:

Singleton — Constructor injection
Scoped and Transient — InvokeAsync/Invoke injection.

The argument for the Transient instance scope is that you don’t want to extend its lifecycle to a singleton if not required.

An example of how to inject:

public class InjectionMiddleware
{
    private readonly RequestDelegate _next;
    private readonly ISingletonService _singletonService;

    public InjectionMiddleware(
      RequestDelegate next,
      ISingletonService singletonService)
    {
        _next = next;
        _singletonService = singletonService;
    }

    public async Task InvokeAsync(
      HttpContext context,
      IScopedService scopedService,
      ITransientService transientService)
    {
      _singletonService.DoSomething();
      scopedService.DoSomething();
      transientService.DoSomething();
    }
}

Middleware custom arguments

You can also pass custom arguments to your custom middleware during application startup, like a configuration, the deployment environment, etc.

To do that, you can add it to the middleware class constructor after the dependencies that the DI container will handle.

So, taking the previous example, we can do:

public class InjectionMiddleware
{
    private readonly RequestDelegate _next;
    private readonly ISingletonService _singletonService;
    private readonly string _environment;

    public InjectionMiddleware(
      RequestDelegate next,
      ISingletonService singletonService,
      string environment)
    {
        _next = next;
        _singletonService = singletonService;
        _environment = environment;
    }

    public async Task InvokeAsync(
      HttpContext context,
      IScopedService scopedService,
      ITransientService transientService)
    {
      Console.WriteLine("Doing something in environment {0}", _environment);

      _singletonService.DoSomething();
      scopedService.DoSomething();
      transientService.DoSomething();
    }
}

And then you can pass this argument during middleware registration:

app.UseMiddleware<InjectionMiddleware>("Production");

Factory-based middleware

As exemplified earlier, it is easy to define a strong-typed middleware with the factory-based approach. The advantages of using this over the conventional are:

Strong-typed class — It is much easier to get the middleware correctly built when you need to implement the IMiddleware interface.
Testability - enables you to test, inject, and mock your middleware in unit or integration tests to validate behaviors.

It is also important to note that injecting the middleware as a service removes the service lifecycle constraints from the conventional method. This means you can inject dependencies of all lifecycles in the constructor.

Drawbacks

One of the drawbacks is that you are not able to pass custom parameters to the middleware in the UseMiddleware method anymore.

Order matters! The request/response flow

When registering middlewares, you need to remember that the order of registration matters. The ones registered first will process the request first and the response last.

See the flow graph below:

Let’s take a look at the example middleware:

public class ExampleMiddleware : IMiddleware
{    
    public async Task InvokeAsync(HttpContext context, RequestDelegate next)
    {
        Console.WriteLine("I run before the request is processed.");

        await next(context);

        Console.WriteLine("I run after I receive a response.");
    }
}

Anything that runs before await _next(context) will process anything before it dispatches the request to the next middleware.

Anything after await _next(context) will process anything after the subsequent middlewares have finished processing.

Let’s say we have two middleware:

public class ExampleMiddleware
{    
    public async Task InvokeAsync(HttpContext context, RequestDelegate next)
    {
        Console.WriteLine("-> Example Middleware BEFORE Dispatching request");

        await next(context);

        Console.WriteLine("<- Example Middleware AFTER dispatching");
    }
}

and

public class AnotherMiddleware
{    
    public async Task InvokeAsync(HttpContext context, RequestDelegate next)
    {
        Console.WriteLine("-> Another Middleware BEFORE Dispatching request");

        await next(context);

        Console.WriteLine("<- Another Middleware AFTER dispatching");
    }
}

Now let’s register the services with:

builder.Services.AddTransient<ExampleMiddleware>();
builder.Services.AddTransient<AnotherMiddleware>();

And the middleware:

app.UseMiddleware<ExampleMiddleware>();
app.UseMiddleware<AnotherMiddleware>();

When you send a request to the API, you’ll see the following in the terminal:

-> Example Middleware BEFORE Dispatching request
-> Another Middleware BEFORE Dispatching request
...
<- Another Middleware AFTER dispatching
<- Example Middleware AFTER dispatching

This means that ExampleMiddleware is first to process the request but last to process the response. While AnotherMiddleware is last to process the request but first to process the response.

Knowing that, you can use this to your advantage in many ways, like creating the following middleware for global exception handling:

public class GlobalExceptionHandlingMiddleware : IMiddleware
{    
    public async Task InvokeAsync(HttpContext context, RequestDelegate next)
    {
        try
        {
            await next(context);
        }
        catch (Exception exception)
        {
            context.Response.StatusCode = 500;
            context.Response.ContentType = "application/json";
            await context.Response.WriteAsync("{\"message\": \"Something went wrong.\"}");
        }
    }
}

And then you register it before all other middlewares with:

builder.Services.AddTransient<GlobalExceptionHandlingMiddleware>();

var app = builder.Build();

app.UseMiddleware<GlobalExceptionHandlingMiddleware>();

With this, any unhandled exception thrown during the request processing will be caught through this middleware that will return a proper JSON response.

Note that this is just an example of global exception handling.

Conclusion

In the domain of .NET APIs, middleware acts as a critical intermediary between client requests and server responses. It plays a pivotal role in managing and altering these interactions. CORS middleware, for instance, ensures the validity of requests, preventing unauthorized access to API endpoints.

The implementation of middleware involves various approaches. While inline middleware offers simplicity by embedding code directly into the application, creating custom middleware through classes provides better code organization, and separation of concerns, and promotes the utilization of dependency injection.

Understanding the order of middleware registration is crucial, as it directly impacts the flow of requests and responses. The sequence determines which middleware processes requests first and responses last. This order can be strategically used, such as employing global exception handling middleware to capture and manage unhandled exceptions across the application.

Middleware can be tailored with custom arguments, allowing for flexibility in handling different environments or configurations. Proper dependency injection methods, aligned with the lifecycle of services (Singleton, Transient, Scoped), are essential for efficient middleware implementation.

By comprehending and harnessing the power of middleware, developers can streamline request-response processes, manage exceptions, and fine-tune their .NET API applications, enhancing overall performance and security.