DEV Community: Justin Coker

Image Summarization using AWS Bedrock

Justin Coker — Wed, 10 Jul 2024 23:47:13 +0000

In previous posts, I've explored various applications of using Amazon Rekognition for analyzing images and videos. Today, I wanted to take it a step further by integrating Rekognition’s powerful computer vision capabilities with the advanced summarization features of Amazon Bedrock’s large language models. Let's get started!

Use Cases

Search: By generating captions that describe the visual details and semantic information of images, image summarization allows for a more nuanced and accurate search experience. It bridges the gap between visual data and language, enabling users to find images based on textual descriptions that reflect the content of the images.
Accessibility: Image summarization can enhance accessibility by providing concise textual descriptions of visual content, which is crucial for individuals with visual impairments. It allows them to access information that would otherwise be inaccessible, fostering inclusivity and equal access to digital content.
Tagging: This solution could allow automatic tag generation based on content for metadata storage and refinement.

Services

The services we'll be using are pretty well known, but, in case they're new to you, here's a brief overview of each:

Amazon Bedrock is a fully managed service that offers a choice of high-performing foundation models (FMs) from leading AI companies like AI21 Labs, Anthropic, Cohere, Meta, Mistral AI, Stability AI, and Amazon through a single API, along with a broad set of capabilities you need to build generative AI applications with security, privacy, and responsible AI.
Amazon Rekognition offers pre-trained and customizable computer vision (CV) capabilities to extract information and insights from your images and videos.
AWS Step Functions is a visual workflow service that helps developers use AWS services to build distributed applications, automate processes, orchestrate microservices, and create data and machine learning (ML) pipelines.

Architecture

Prerequisites

This application will be deployed using Pulumi and JavaScript/TypeScript so you should have some familiarity with both in order to understand what's being deployed. You'll also need to make sure the following dependencies are installed:

Getting Started

After installing the prerequisites we can now start building our app. I'm going to outline each step, but if you prefer to simply see the finished product feel free to skip ahead and go directly to the Github repo.

Creating the Project

First, let's login to a Pulumi backend. By default, pulumi new attempts to use Pulumi Cloud for its backend, but for this simple example we'll use our local filesystem.



pulumi login --local

Next, we'll need to create a directory for our project and bootstrap Pulumi.



mkdir image-summarization && cd image-summarization
pulumi new aws-typescript

You will be prompted to enter the project name and project description



This command will walk you through creating a new Pulumi project.

Enter a value or leave blank to accept the (default), and press <ENTER>.
Press ^C at any time to quit.

project name: (image-summarization)
project description: (Image summary generation using AWS)
Created project 'image-summarization'

Next, you will be asked for a stack name. Hit ENTER to accept the default value of dev.



Please enter your desired stack name.
stack name: (dev)
Created stack 'dev'

Finally, you will be prompted for the region. For this example, I'll be using us-east-1.



aws:region: The AWS region to deploy into: (us-east-1)
Saved config

Authentication

Before we can deploy any services to AWS we have to set the credentials for Pulumi to use. I won't cover it here, but you can reference the Pulumi Docs which outlines your authentication options.

AWS Components

We could use some level of abstraction to make things more manageable, but, for this example, I'm simply going to put all components in the index.ts file at our project root.

Buckets

Like a lot of applications built in AWS, the first thing we need is some S3 buckets.



const inputBucket = new aws.s3.Bucket("input-bucket", {
  forceDestroy: true,
});

const outputBucket = new aws.s3.Bucket("output-bucket", {
  forceDestroy: true,
});

Now, let's go ahead and enable EventBridge notifications on the Input Bucket we just defined.



const inputBucketNotification = new aws.s3.BucketNotification(
  "input-bucket-notification",
  { eventbridge: true, bucket: inputBucket.id }
);

Lambdas

Detecting image labels using Rekognition produces a very verbose output—most of which is inconsequential to our LLM—so we'll create one lambda to filter the labels and another one simply to tidy up our final output.

Lambdas need an execution role, so let's go ahead and create that first. Our lambdas won't be calling any services, so they don't really need any permissions. Creating a role and a trust policy will suffice.



const lambdaTrustPolicy = aws.iam.getPolicyDocument({
  statements: [
    {
      effect: "Allow",
      principals: [
        {
          type: "Service",
          identifiers: ["lambda.amazonaws.com"],
        },
      ],
      actions: ["sts:AssumeRole"],
    },
  ],
});

const lambdaRole = new aws.iam.Role("ImageSummarizationLambdaRole", {
  name: "ImageSummarizationLambdaRole",
  assumeRolePolicy: lambdaTrustPolicy.then((policy) => policy.json),
});

Filter Labels

Next, let's create the lambda we'll use to filter the Rekognition labels. Let's put in a file called filterLabels.mjs under the lambdas/src directory in our project root. This function will filter out any labels below our set confidence level, count each object type, and format them into a comma-separated string for our LLM to consume.



export const handler = async (event) => {
  const confidenceLevel = parseInt(process.env.CONFIDENCE_LEVEL) || 90;

  const labels = event.Rekognition.Labels;

  const filteredLabels = labels
    .filter((label) => label.Confidence > confidenceLevel)
    .map((label) =>
      label.Instances.length > 0
        ? `${label.Instances.length} ${label.Name}`
        : label.Name
    )
    .join(", ");

  const response = {
    labels: filteredLabels,
  };
  return response;
};

We'll be deploying this as a zip package, so we'll go ahead and tell Pulumi to archive the file for us.



const filterLabelsArchive = archive.getFile({
  type: "zip",
  sourceFile: "lambdas/src/filterLabels.mjs",
  outputPath: "lambdas/dist/filterLabels.zip",
});

As you can see from the block above, Pulumi will place the output zip file in the lambdas/dist directory. Now we'll tell Pulumi to create the lambda using the zip file it just created.



const filterLabelsFunction = new aws.lambda.Function(
  "ImageSummarizationFilterLabels",
  {
    code: new pulumi.asset.FileArchive("lambdas/dist/filterLabels.zip"),
    name: "ImageSummarizationFilterLabels",
    role: lambdaRole.arn,
    sourceCodeHash: filterLabelsArchive.then(
      (archive) => archive.outputBase64sha256
    ),
    runtime: aws.lambda.Runtime.NodeJS20dX,
    handler: "filterLabels.handler",
    environment: {
      variables: {
        CONFIDENCE_LEVEL: "90",
      },
    },
  }
);

Build Output

Now, we're going to create a simple function to build the output we want from the results. The process will be the same as the filter labels function we just created so I'll only include the snippets.



export const handler = async (event) => {
  const response = {
    source: {
      bucket: event.detail.bucket.name,
      file: event.detail.object.key
    },
    labels: event.Rekognition.Labels,
    summary: event.Bedrock.Body.results[0].outputText
  }

  return response;
};



const buildOutputArchive = archive.getFile({
  type: "zip",
  sourceFile: "lambdas/src/buildOutput.mjs",
  outputPath: "lambdas/dist/buildOutput.zip",
});

const buildOutputFunction = new aws.lambda.Function(
  "ImageSummarizationBuildOutput",
  {
    code: new pulumi.asset.FileArchive("lambdas/dist/buildOutput.zip"),
    name: "ImageSummarizationBuildOutput",
    role: lambdaRole.arn,
    sourceCodeHash: buildOutputArchive.then(
      (archive) => archive.outputBase64sha256
    ),
    runtime: aws.lambda.Runtime.NodeJS20dX,
    handler: "buildOutput.handler",
  }
);

Step Function

Just like our lambdas, our step function needs an execution role, but the step function will actually need real permissions so we'll create those using references to components we've already defined.



const stateMachineTrustPolicy = aws.iam.getPolicyDocument({
  statements: [
    {
      effect: "Allow",
      principals: [
        {
          type: "Service",
          identifiers: ["states.amazonaws.com"],
        },
      ],
      actions: ["sts:AssumeRole"],
    },
  ],
});

const stateMachinePolicy = new aws.iam.Policy("ImageSummarizationSfn-Policy", {
  name: "ImageSummarizationSfn-Policy",
  path: "/",
  description: "Permission policy for Image Summarization state machine",
  policy: pulumi.jsonStringify({
    Version: "2012-10-17",
    Statement: [
      {
        Effect: "Allow",
        Action: ["lambda:InvokeFunction"],
        Resource: [
          filterLabelsFunction.arn,
          buildOutputFunction.arn,
        ],
      },
      {
        Action: ["s3:GetObject", "s3:DeleteObject", "s3:PutObject"],
        Effect: "Allow",
        Resource: [
          pulumi.interpolate`${inputBucket}/*`,
          pulumi.interpolate`${outputBucket}/*`,
        ],
      },
      {
        Action: "rekognition:DetectLabels",
        Effect: "Allow",
        Resource: "*",
      },
      {
        Action: ["bedrock:InvokeModel"],
        Effect: "Allow",
        Resource: "*",
      },
    ],
  }),
});

const stateMachineRole = new aws.iam.Role("ImageSummarizationSfn-Role", {
  name: "ImageSummarizationSfn-Role",
  assumeRolePolicy: stateMachineTrustPolicy.then((policy) => policy.json),
  managedPolicyArns: [stateMachinePolicy.arn],
});

Now that we have a role to use, we can create our state machine. We're going to define five step in our state machine Detect Labels, Filter Labels, Invoke Model, Build Output, and Save Output. Step function definitions are pretty verbose so I've stripped out everything but the most critical parameters.



const stateMachine = new aws.sfn.StateMachine(
  "ImageSummarizationStateMachine",
  {
    name: "ImageSummarizationStateMachine",
    roleArn: stateMachineRole.arn,
    definition: pulumi.jsonStringify({
      StartAt: "Detect Labels",
      States: {
        "Detect Labels": {
          Type: "Task",
          Parameters: {
            Image: {
              S3Object: {
                "Bucket.$": "$.detail.bucket.name",
                "Name.$": "$.detail.object.key",
              },
            },
          },
          Resource: "arn:aws:states:::aws-sdk:rekognition:detectLabels",
          Next: "Filter Labels",
          ResultPath: "$.Rekognition",
          ResultSelector: {
            "Labels.$": "$.Labels",
          },
        },
        "Filter Labels": {
          Type: "Task",
          Resource: "arn:aws:states:::lambda:invoke",
          Parameters: {
            "Payload.$": "$",
            FunctionName: filterLabelsFunction.arn,
          },
          ResultPath: "$.Lambda",
          ResultSelector: {
            "FilteredLabels.$": "$.Payload.labels",
          },
          Next: "Bedrock InvokeModel",
        },
        "Bedrock InvokeModel": {
          Type: "Task",
          Resource: "arn:aws:states:::bedrock:invokeModel",
          Parameters: {
            ModelId:
              "arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-text-premier-v1:0",
            Body: {
              "inputText.$":
                "States.Format('Human: Here is a comma seperated list of labels/objects seen in an image\n<labels>{}</labels>\n\n" +
                "Please provide a human readible and understandable summary based on these labels\n\nAssistant:', $.Lambda.FilteredLabels)",
              textGenerationConfig: {
                temperature: 0.7,
                topP: 0.9,
                maxTokenCount: 512,
              },
            },
          },
          ResultPath: "$.Bedrock",
          Next: "Build Output",
        },
        "Build Output": {
          Type: "Task",
          Resource: "arn:aws:states:::lambda:invoke",
          OutputPath: "$.Payload",
          Parameters: {
            "Payload.$": "$",
            FunctionName: buildOutputFunction.arn,
          },
          Next: "Save Output",
        },
        "Save Output": {
          Type: "Task",
          End: true,
          Parameters: {
            "Body.$": "$",
            Bucket: outputBucket.id,
            "Key.$": "States.Format('{}.json', $.source.file)",
          },
          Resource: "arn:aws:states:::aws-sdk:s3:putObject",
        },
      },
    }),
  }

Events

At this point we have buckets, lambdas, and a fully working step function capable of detecting labels and summarizing the results. The one thing missing is the event rule which routes and enables uploads to the Input Bucket to start the state machine.

First, let's create the rule for objects created in the Input Bucket.



const inputRule = new aws.cloudwatch.EventRule("input-bucket-rule", {
  name: "input-bucket-rule",
  eventPattern: pulumi.jsonStringify({
    source: ["aws.s3"],
    "detail-type": ["Object Created"],
    detail: {
      bucket: {
        name: [inputBucket.id],
      },
    },
  }),
  forceDestroy: true,
});

Now, we'll need a role capable of starting our state machine.



const inputRuleTrustPolicy = aws.iam.getPolicyDocument({
  statements: [
    {
      effect: "Allow",
      principals: [
        {
          type: "Service",
          identifiers: ["events.amazonaws.com"],
        },
      ],
      actions: ["sts:AssumeRole"],
    },
  ],
});

const inputRulePolicy = new aws.iam.Policy("ImageSummarizationRule-Policy", {
  name: "ImageSummarizationRule-Policy",
  policy: pulumi.jsonStringify({
    Version: "2012-10-17",
    Statement: [
      {
        Effect: "Allow",
        Action: ["states:StartExecution"],
        Resource: [stateMachine.arn],
      },
    ],
  }),
});

const inputRuleRole = new aws.iam.Role("ImageSummarizationRule-Role", {
  name: "ImageSummarizationRule-Role",
  assumeRolePolicy: inputRuleTrustPolicy.then((policy) => policy.json),
  managedPolicyArns: [inputRulePolicy.arn],
});

Finally, we'll tie the rule, role, and state machine together by defining an event target.



const inputRuleTarget = new aws.cloudwatch.EventTarget("input-rule-target", {
  targetId: "input-rule-target",
  rule: inputRule.name,
  arn: stateMachine.arn,
  roleArn: inputRuleRole.arn,
});

Time to play with our new application!

Wrapping Up

There are a couple of copyright free images contained in the assets folder of the repo I provided, but you're free to upload any images you like and test the results. For this I'm going to upload skateboard.jpeg from the repo and see what I get.

Let's see what the output looks like and compare it to the image. Here is what's contained in summary key of the output.

The image shows a city with a road and street in the neighborhood. There are 13 cars and 21 wheels. There is 2 buildings and 2 persons in the metropolis. The architecture is urban.

Not perfect, but really not too bad. The application clearly performs what we expected, so what's next?

Tinker with the confidence level: I have the confidence level set to 90 and changing this value can drastically alter the labels passed to our model.
Try different models: For this example, I used the Titan Text Large model, but Bedrock has many models to choose from that may produce better results.

And with that, we're done. Feel free to provide any comments or corrections, and I sincerely hope you enjoyed this post and thank you for reading!

Modernizing .NET Apps in AWS

Justin Coker — Fri, 18 Aug 2023 02:30:26 +0000

In this post we're going to discuss different AWS-based modernization strategies for .NET applications. Oftentimes, people assume modernization means completely re-architecting and rewriting an application to leverage new technologies such as microservices, purpose-built databases, serverless functions, etc. And, for some use cases, that is a perfectly valid approach. But most modernization efforts face constraints such as budget, timeline, resource availability, and overall complexity that make a full application overhaul impractical.

But there's good news! Modernization is not an all-or-nothing proposition. You can modernize as much or as little as is feasible and still see appreciable benefits along the way.

Planning Your Journey

Each journey will be different, but two determining factors are ever-present when it comes to modernizing .NET applications: (1) the version of .NET in use; and (2) the underlying operating system on which the application is hosted. The diagram above is very high-level, but should give you a good outline of the options at play.

Note about the diagram: I can assure you I haven't forgotten that applications typically have databases, but I've chosen to address modernizing MSSQL in a separate post. (Yes, I'm going out on a limb and assuming you're using SQL Server.)

.NET Version

This one is a bit obvious, but the first—and most important—determining factor for modernizing a .NET workload is the underlying version of .NET.

.NET Framework

If your application is stuck on .NET Framework you're starting from square one, and you're essentially down to two immediate options:

Using Windows containers: Lacks some of the benefits seen in the other destinations—particularly around cost.
Porting the application to .NET 6+: Can be a significant undertaking depending on the complexities and requirements of the existing environment.

Modern .NET

Conversely, if you're already utilizing a modern version of .NET; Congratulations, you're already well down the modernization path! Now—as we'll see in the destinations section below—if you also took the opportunity to host/rehost on Linux you're practically at the finish line.

Operating System

The second major determining factor when deciding your ultimate journey destination is the underlying operating system. Let's be clear, the vast majority of .NET workloads out there run on Windows. I can't find a reliable source, but, if I had to make a guess based on my personal experiences, I'd estimate the ratio to be at least 9:1 in Microsoft's favor.

As outlined in the next section, there are multiple benefits to taking the plunge and rehosting on Linux—but it's not without its risks. Existing workflows will be impacted, finding answers may be more difficult, and developer velocity will almost assuredly take a hit in the short term as personnel adapt to the new ways of working. Personally, I feel the rewards are great enough to warrant moving to Linux when possible—but that's just me. You'll ultimately have to decide what's best for your app, team, and organization.

Destinations

Windows-based Containers

Porting your legacy application to Windows containers is by far the easiest option for Windows-based workloads, but also delivers the least benefits. Both EKS and ECS (including Fargate) support Windows containers so you're covered from an AWS services perspective, but there are a few downsides when compared to Linux containers:

Higher cost
Larger containers
Slower start times
Less efficient usage of resources
Lower adoption

This can be a beneficial first step in modernizing .NET Framework applications, but—in my opinion—should not be the end of the journey.

Linux-based Containers

Although containers are now a multi-platform solution, they were initially developed with Linux in mind, and, because of this, Linux containers have a massive head start in terms of adoption, support, and maturity. While Windows containers have made strides in the last few years in terms of usage, Linux containers remain the de-facto standard for enterprise containerization. Unless you are forced to go with Windows for some reason, Linux should be the operating system of choice when containerizing workloads.

Serverless

Serverless is the gold-standard for cloud-native applications, and if we were talking about building a greenfield application, I would tell you that serverless is unequivocally the way to go. After all, the inherent benefits of serverless are well documented and numerous:

Built-in auto-scaling
Automatic high-availability
No server management
Quick deployments & updates
Extremely cost-efficient

But when it comes to modernizing an existing application—particularly a large and complicated application—the case for serverless becomes a bit less obvious. Converting an existing application to a serverless architecture can be challenging due to a variety of technical and architectural reasons, and many choose to end their modernization efforts with containerization.

Luckily—for those that choose to forge ahead—AWS Lambda supports container images as a deployment model, meaning the same containers in use today can be used to power your Lambda functions in the future.

How AWS Can Help?

Of course AWS has virtually any need covered from a services perspective, but I wanted to use this space to highlight some lesser-known tools AWS provides specific to .NET modernization:

AWS App2Container (A2C): Command line tool for migrating and modernizing .NET web applications running in IIS on Windows into container format.
Porting Assistant for .NET (PA): Analysis tool that scans .NET Framework applications and generates a modern .NET compatibility assessment, helping you port your applications to Linux faster.
AWS Microservice Extractor for .NET (MSE): Simplifies the process of refactoring older monolithic apps into small code projects to build a microservices-based architecture.
AWS Toolkit for .NET Refactoring (TR): Visual Studio extension that helps developers refactor legacy .NET applications to cloud-based alternatives on AWS.
AWS Modernization Calculator for Microsoft Workloads: Interactive tool that helps you estimate the cost of modernizing your Microsoft workloads to a new architecture, using open-source software and cloud-native services on AWS.

Wrapping Up

Hopefully, I've successfully illustrated that when it comes to modernization, there is no one-size-fits-all solution. Each modernization journey will have its own specific sets of external factors, requirements, and challenges that will ultimately determine how best to proceed. For those currently undertaking a .NET modernization effort, good luck! I'd love to hear about your experiences.

Finally, I sincerely hope you enjoyed this post and be on the lookout for my followup article on modernizing SQL Server!

Image Label Detection using AWS and Pulumi

Justin Coker — Fri, 28 Jul 2023 14:25:52 +0000

In this post, we are going to quickly build an automated image label detection application using S3, EventBridge, Rekognition, Step Functions, and Pulumi.

If you're anything like me, you're sick and tired of manually counting the dinosaurs in your pictures. When we're done, we should have a working application capable of accurately finding and counting the number of dinosaurs contained within an image. What a time saver!

Component Highlight

AWS Rekognition

Rekognition is Amazon's ML-powered image and video analysis service. Rekognition offers pre-trained and customizable computer vision capabilities to extract information and insights from images and videos, and, in this post, we'll be using Rekognition's label detection capabilities which can identify labels and scenes within images.

Pulumi

Similar to Terraform, Pulumi is an open-source universal infrastructure-as-code (IaC) tool that allows working with multiple cloud providers. But, whereas Terraform uses the proprietary Hashicorp configuration language (HCL), Pulumi works with a variety of programming languages (TypeScript, Go, .NET, Python, Java) or markup languages (YAML, CUE).

Great but why use it?

To be clear, I am not advocating adoption of Pulumi. In fact, this is my first time using it for a project, but the following factors convinced me to give it a shot.

While I've yet to encounter a client that demands usage of Pulumi, it does now come up in conversations alongside Terraform, CDK, SAM, etc. So, while adoption may not be there—yet, it is gaining in popularity.
Pulumi now has a native AWS provider which promises virtually instant API access to new AWS services. No more waiting on official or community-provided plugins.
It's a personal preference, but I would much rather work with TypeScript or Python over HCL.

Build

Now that we've covered the intros it's time to start building. Here is the reference architecture we'll be working from, and, as you can see, we won't need many services in our final application.

Let's start with the easiest components—buckets. Virtually every modernization effort in AWS requires buckets and we'll need two of them; one for ingestion and one to store the output produced by Rekognition.

// Creates source bucket
const sourceBucket = new aws.s3.BucketV2("source-bucket");

// Creates bucket for Rekognition output
const outputBucket = new aws.s3.BucketV2("output-bucket");

Next we need to trigger an event when objects are created in our source bucket. For the destination we could use SQS, SNS, or Lambda, but, for simplicity, we're going to use EventBridge which means we need to enable EventBridge notifications on the source bucket.

// Enables EventBridge notifications on the source bucket
const sourceBucketNotification = new aws.s3.BucketNotification(
  "sourceBucketNotification",
  {
    bucket: sourceBucket.id,
    eventbridge: true,
  }
);

Our logic will be contained within a state machine so we need to grant the following permissions to our state machine:

Access to call DetectLabels in Rekognition
Read permissions on the source bucket
Write permissions on the output bucket

Let's start by creating the role and allowing it to be assumed by the correct service principal (states.amazonaws.com).

// Role for the state machine
const stateMachineRole = new aws.iam.Role("stateMachineRole", {
  assumeRolePolicy: JSON.stringify({
    Version: "2012-10-17",
    Statement: [
      {
        Action: "sts:AssumeRole",
        Effect: "Allow",
        Sid: "",
        Principal: {
          Service: "states.amazonaws.com",
        },
      },
    ],
  }),
});

Next, we'll attach the policy with the necessary permissions to our state machine role

/*
Grants access to perform DetectLabels,
read from source bucket, and write to output bucket
*/
const stateMachineRolePolicy = new aws.iam.RolePolicy(
  "stateMachineRolePolicy",
  {
    role: stateMachineRole.id,
    policy: pulumi
      .all([sourceBucket.arn, outputBucket.arn])
      .apply(([sourceBucketArn, outputBucketArn]) =>
        pulumi.jsonStringify({
          Version: "2012-10-17",
          Statement: [
            {
              Action: ["rekognition:DetectLabels"],
              Effect: "Allow",
              Resource: "*",
            },
            {
              Action: ["s3:ListBucket", "s3:GetObject"],
              Effect: "Allow",
              Resource: [sourceBucketArn, `${sourceBucketArn}/*`],
            },
            {
              Action: ["s3:PutObject"],
              Effect: "Allow",
              Resource: `${outputBucketArn}/*`,
            },
          ],
        })
      ),
  }
);

This is our first time seeing some of the Pulumi-specific syntax with all and apply, so let me quickly explain what's happening in this snippet. All outputs from Pulumi are values of type Object<t>, which behave very much like a promise, so when referencing a value such as sourceBucket.arn it's returned as an Object<string> instead of a plain string as you might expect. Due to this, Pulumi includes methods to make dealing with outputs easier. I'll admit this tripped me up at first, but once you dig through the input/output doc it starts to make sense. Alright, back to work.

Time to build the heart of the application—the state machine. The workflow will perform the following steps:

Receive the S3 event details from EventBridge.
Call Rekognition DetectLabels with the bucket and file name contained within the event data.
Pass the output returned from Rekognition into a transform flow that filters the data to only include labels with a Name equal to "Dinosaur".
Write a new file to the output bucket. The name of the file will be the same as the original suffixed with .txt, and it will contain the count of dinosaurs in the original source image.

const stateMachine = new aws.sfn.StateMachine("stateMachine", {
  roleArn: stateMachineRole.arn,
  definition: pulumi.jsonStringify({
    StartAt: "DetectLabels",
    States: {
      DetectLabels: {
        Type: "Task",
        Parameters: {
          Image: {
            S3Object: {
              "Bucket.$": "$.bucket.name",
              "Name.$": "$.object.key",
            },
          },
        },
        Resource: "arn:aws:states:::aws-sdk:rekognition:detectLabels",
        InputPath: "$.detail",
        Next: "Transform",
        ResultPath: "$.Result",
      },
      Transform: {
        Type: "Pass",
        Next: "WriteDinosaurCount",
        InputPath: "$.Result.Labels[?(@.Name == 'Dinosaur')]",
        ResultPath: "$.Result",
      },
      WriteDinosaurCount: {
        Type: "Task",
        Parameters: {
          Body: {
            "Count.$": "States.ArrayLength($.Result[0]['Instances'])",
          },
          Bucket: outputBucket.id,
          "Key.$": "States.Format('{}.txt', $.detail.object.key)",
        },
        Resource: "arn:aws:states:::aws-sdk:s3:putObject",
        End: true,
      },
    },
  }),
});

After we've successfully defined our state machine, we need to create our EventBridge rule and allow it to kickoff the execution process.

const s3CreatedRule = new aws.cloudwatch.EventRule("s3CreatedRule", {
  description:
    "Launches the state machine when an object is uploaded to the source bucket",
  eventPattern: pulumi.jsonStringify({
    "detail-type": ["Object Created"],
    source: ["aws.s3"],
    detail: {
      bucket: {
        name: [sourceBucket.id],
      },
    },
  }),
});

Then we'll create the role for EventBridge to assume.

// Role for our EventBridge rule
const eventBridgeRole = new aws.iam.Role("eventBridgeRole", {
  assumeRolePolicy: JSON.stringify({
    Version: "2012-10-17",
    Statement: [
      {
        Action: "sts:AssumeRole",
        Effect: "Allow",
        Sid: "",
        Principal: {
          Service: "events.amazonaws.com",
        },
      },
    ],
  }),
});

Next, we'll grant the necessary permissions to the EventBridge role.

// Grants access to start the state machine
const eventBridgeRolePolicy = new aws.iam.RolePolicy("eventBridgeRolePolicy", {
  role: eventBridgeRole.id,
  policy: stateMachine.arn.apply((arn) =>
    pulumi.jsonStringify({
      Version: "2012-10-17",
      Statement: [
        {
          Action: ["states:StartExecution"],
          Effect: "Allow",
          Resource: arn,
        },
      ],
    })
  ),
});

Finally, we'll create the EventBridge rule target.

const s3CreatedTarget = new aws.cloudwatch.EventTarget("s3CreatedTarget", {
  rule: s3CreatedRule.name,
  arn: stateMachine.arn,
  roleArn: eventBridgeRole.arn,
});

Deploy

Assuming everything has gone according to plan, you should be able to run pulumi up to successfully deploy the application.

If you head over to your source bucket and upload an image file that contains dinosaurs you should be rewarded with a matching file in the outputs bucket that contain JSON content similar to the following. Woohoo!

{"Count":2}

Wrapping Up

Overall, I enjoyed working with Pulumi and will assuredly do so again in the future. (Though I'm not ready to dump the others just yet.)

Through all of my test runs, Rekognition was able to successfully count the dinosaurs in the image virtually every time. Really impressive results.

Finally, I sincerely hope you enjoyed this post and thank you for reading!