DEV Community: AWS Heroes

Understanding AWS Blocks as a CDK Developer

Kenta Goto — Wed, 17 Jun 2026 10:45:55 +0000

Understanding AWS Blocks as a CDK Developer

I tried out AWS Blocks, announced as a preview on June 17, 2026, from the perspective of someone who normally writes AWS CDK.

What Is AWS Blocks?

AWS Blocks is a toolkit for quickly building the backend of a full-stack app. It was announced as a preview on June 17, 2026, and its source code is published as open source at aws-devtools-labs/aws-blocks.

At its center is a component called a Building Block. It is a unit of functionality such as a data store or authentication, and a single Building Block plays the following three roles at the same time.

A CDK Construct (the infrastructure definition at deploy time)
A runtime implementation (AWS SDK calls running on Lambda)
A local mock (runs with npm run dev, no AWS environment needed)

In other words, the same Building Block description acts as a mock during local development, as a CDK Construct at deploy time, and as AWS SDK calls in production. As a result, a single piece of code can run locally on your machine without an AWS account, and can be deployed to AWS as is.

The Building Blocks provided include KVStore and DistributedTable (data), AuthBasic (authentication), Realtime (real-time communication), FileBucket (files), and Database (SQL).

Developers write both the infrastructure (such as data) and the API logic that uses it together in aws-blocks/index.ts (which can also be split into multiple files).

// aws-blocks/index.ts
import { Scope, ApiNamespace, KVStore } from '@aws-blocks/blocks';

const scope = new Scope('my-app');
const cache = new KVStore(scope, 'cache');   // ← at deploy time, this becomes a DynamoDB table (infrastructure)

export const api = new ApiNamespace(scope, 'api', () => ({
  getValue: (key: string) => cache.get(key), // ← API logic that uses that table
}));

The infrastructure definition new KVStore(...) and the API logic that calls it sit side by side in the same file. If CDK is about "declaring infrastructure as code (Infrastructure as Code)," then AWS Blocks is framed as "deriving infrastructure from application code (Infrastructure from Code, IfC)."

The frontend simply calls this api directly, like import { api } from 'aws-blocks', so that the frontend and backend share types without any code-generation step in between.

For type integration, type-safe clients are provided not only for web frameworks such as Next, Nuxt, React, and Vue, but also for native mobile targets such as Swift, Kotlin, and Dart/Flutter. In TypeScript you can load them directly via import, whereas for native targets the client code is generated at build time from a spec called blocks.spec.json, and the backend is called over JSON-RPC.

The Same import Resolves to Different Files Depending on Context

This is probably the most distinctive feature of AWS Blocks.

Both the import { api } from 'aws-blocks' you write on the frontend and the new DistributedTable(...) you call inside the backend's aws-blocks/index.ts look like ordinary TypeScript. Yet this 'aws-blocks' (and each Building Block) resolves to a completely different file depending on the context in which it runs.

This "resolves to something different depending on context" behavior is stated explicitly in the official README. It explains that the same new KVStore(scope, 'todos') becomes a local store during development, an Amazon DynamoDB table at deploy time, and an SDK call in production (without changing a single line of code). Below, I'll trace how this works from the implementation.

Context	Resolves to	Role
Local development (`npm run dev`)	Mock implementation	Stores to disk (`.bb-data/`). No AWS environment needed
CDK synth	Infrastructure (CDK Construct)	Defines DynamoDB tables, Lambdas, etc.
Lambda runtime	Application implementation	Calls the AWS SDK

Mechanism: conditional exports + a global variable

The mechanism consists of two elements.

The first is Node.js conditional exports (the exports field in package.json). Looking at the package.json of each Building Block, exports is defined as follows.

// package.json of @aws-blocks/bb-distributed-table (exports excerpt)
{
  "exports": {
    ".": {
      "browser": "./dist/index.browser.js",
      "cdk": {
        "types": "./dist/index.cdk.d.ts",
        "default": "./dist/index.cdk.js"
      },
      "aws-runtime": "./dist/index.aws.js",
      "types": "./dist/index.mock.d.ts",
      "default": "./dist/index.mock.js"
    }
  }
}

exports is a mechanism that assigns a different file per condition; when an import is resolved, the file matching an active condition is loaded. browser / types / default are standard Node.js conditions, whereas cdk and aws-runtime are conditions that AWS Blocks defines on its own, and they are not active unless explicitly specified at startup.

And the reason the resolution target changes per context is that each command passes "which condition to activate" at startup.

cdk synth runs with NODE_OPTIONS=--conditions=cdk, so it resolves to cdk (= Construct); the Lambda bundle passes --conditions: aws-runtime to esbuild, so it resolves to aws-runtime (= runtime). npm run dev specifies no condition, so it resolves to default (= mock), and for type checking TypeScript always looks at types.

The second is the hand-off at synth time. Before dynamically importing the aws-blocks/index.ts that the developer implemented, the CDK layer holds a reference to the stack instance currently being synthesized in a global variable. The Scope and each Building Block created inside index.ts are not given a parent stack explicitly, so they refer to this global variable to resolve which stack they belong to.

NOTE: packages/core/src/cdk/blocks-backend.ts

// packages/core/src/cdk/blocks-backend.ts (excerpt; the infrastructure-building parts are omitted)
private constructor(scope: Construct, id: string, props: BlocksBackendProps) {
  super(scope, id);
  // Expose self to Building Blocks at CDK time
  (globalThis as any).CURRENT_BLOCKS_STACK = this;   // ← expose the current stack globally
  // ...
}

static async create(scope: Construct, id: string, props: BlocksBackendProps) {
  assertCdkConditionActive();
  const backend = new BlocksBackend(scope, id, props);
  const mod = await import(`${props.backendCDKPath}?stack=${id}`);   // ← load index.ts
  // ...
  return backend;
}

In short, when new DistributedTable(...) runs inside the index.ts the developer implemented, that Construct gets the current stack from this global variable, creates a DynamoDB table, and grants the stack's Lambda read/write permissions on the table. This is how writing application code itself becomes the infrastructure definition.

Setting Up a Project

An AWS Blocks app starts from npm's create command (Node.js 22 or later / npm 10 or later is required).

npm create @aws-blocks/blocks-app@latest my-app
cd my-app
npm install
npm run dev

npm run dev starts a local server at http://localhost:3000, and all Building Blocks run as mock implementations (no AWS account or credentials needed). You can also choose a template, like --template react.

The generated directory looks roughly like this (the frontend contents vary by template, but the structure of aws-blocks/ is the same).

my-app/
├── aws-blocks/            # the entire backend
│   ├── index.ts           #   backend definition (API, data, auth) ← what developers mainly write
│   ├── index.cdk.ts       #   CDK entry (assembles the stack with BlocksStack.create)
│   ├── index.handler.ts   #   entry for the Lambda handler at deploy time
│   ├── client.js          #   typed client for the frontend (auto-generated)
│   └── scripts/           #   helper scripts such as the dev server
├── src/                   # frontend (React / Next, etc., depending on the template)
├── cdk.json
└── package.json

The aws-blocks/index.ts we've seen so far is the body of the backend definition, and aws-blocks/index.cdk.ts is the entry point as CDK.

Basically, what developers implement is aws-blocks/index.ts, and there's no need to touch aws-blocks/index.cdk.ts. However, as described later, you can also customize the CDK configuration by touching this file.

Implementing a Sample

Here's what it looks like in practice. Using AuthBasic, DistributedTable, and ApiNamespace, I've implemented a simple Todo app.

// aws-blocks/index.ts
import { ApiNamespace, Scope, AuthBasic, DistributedTable } from '@aws-blocks/blocks';
import { z } from 'zod';

const scope = new Scope('todo-app');

const auth = new AuthBasic(scope, 'auth', { passwordPolicy: { minLength: 8 } });
export const authApi = auth.createApi();

// The Zod schema serves as runtime validation + the TypeScript type + the DynamoDB table shape, all at once
const todoSchema = z.object({
  userId: z.string(),    // partition key (isolated per user)
  todoId: z.string(),    // sort key
  title: z.string(),
  completed: z.boolean(),
});

const todos = new DistributedTable(scope, 'todos', {
  schema: todoSchema,
  key: { partitionKey: 'userId', sortKey: 'todoId' },
});

export const api = new ApiNamespace(scope, 'api', (context) => ({
  async createTodo(title: string) {
    const user = await auth.requireAuth(context);   // ← this makes the method require authentication
    const todo = {
      userId: user.username,
      todoId: Date.now().toString(36),
      title,
      completed: false,
    };
    await todos.put(todo);
    return todo;
  },

  async listTodos() {
    const user = await auth.requireAuth(context);
    return Array.fromAsync(
      todos.query({ where: { userId: { equals: user.username } } })
    );
  },
}));

This sample implementation shows the following characteristics as well.

Authentication is specified explicitly per method: each method is a public RPC endpoint with no authentication by default. Call auth.requireAuth(context) at the beginning of any method you want to require authentication.
Reusing the Zod schema: a single Zod schema can cover runtime validation, the TS type, and the DynamoDB table definition.

Also, the frontend can call the API definition implemented here in a typed way, like import { api } from 'aws-blocks'.

Deploying

Once it works locally, you can deploy the same code to AWS as is.

npm run sandbox   # to a per-developer temporary environment (backend on AWS, frontend served locally)
npm run deploy    # full production deploy (including frontend hosting)

npm run sandbox is a temporary stack for quickly checking behavior, and npm run deploy is a production deploy that includes frontend hosting. Both run CDK internally (synth → deploy with --conditions=cdk, plus generating client code with --conditions=aws-runtime), and the first time, just like ordinary CDK, you need AWS credentials and cdk bootstrap.

Whereas npm run dev used local mocks to realize the app's behavior, here the app is deployed to a real environment without changing a single line of code. The earlier mechanism where "the same import resolves to different files depending on context" comes in handy here.

AWS Blocks from a CDK Perspective

A distinctive feature of AWS Blocks is that you can implement with it even without being familiar with AWS or CDK, but as a CDK user you may sometimes want to extend the CDK definition. AWS Blocks supports those use cases too.

You Can Extend the CDK Definition

aws-blocks/index.cdk.ts is the CDK layer, where BlocksStack is invoked. Here you can also define additional resources you want to create and pass them to blocksStack.handler (the Lambda's NodejsFunction).

// aws-blocks/index.cdk.ts
import * as sqs from 'aws-cdk-lib/aws-sqs';

// ...
// ...

export const blocksStack = await BlocksStack.create(app, 'my-app', {
  backendHandlerPath: join(__dirname, 'index.handler.ts'),
  backendCDKPath: join(__dirname, 'index.ts'),
});

// Create raw CDK resources in the same stack as Blocks, and pass permissions and environment variables to the Lambda
const jobsQueue = new sqs.Queue(blocksStack, 'JobsQueue');

jobsQueue.grantSendMessages(blocksStack.handler);
blocksStack.handler.addEnvironment('JOBS_QUEUE_URL', jobsQueue.queueUrl);

blocksStack.handler exposes methods like addToRolePolicy (adding IAM) and addEnvironment (injecting environment variables), and you can also grant the Lambda IAM permissions on your own CDK resources using grant~ and similar methods.

Checking the Generated CloudFormation with cdk synth

Since AWS Blocks is ultimately AWS CDK under the hood, you can of course run synth too.

npx cdk synth

Among the files generated when you set up the project is cdk.json, whose app command is npx tsx -C cdk aws-blocks/index.cdk.ts. The point is this -C cdk (= --conditions=cdk), which makes each Building Block resolve to a CDK Construct.

Conversely, if you run it without --conditions=cdk, the Building Blocks would resolve as mocks, so AWS Blocks stops you with an explicit Missing --conditions=cdk error.

Error: Missing --conditions=cdk: Building Blocks will silently load mock implementations instead of CDK constructs.

Fix: Set NODE_OPTIONS="--conditions=cdk" before running CDK synth:
  NODE_OPTIONS="--conditions=cdk" npx cdk synth

Integration Patterns

The documentation presents four patterns for integrating with existing infrastructure.

CDK-in-Blocks: connect raw CDK resources to the Blocks Lambda and use them (for resources that have no corresponding Building Block)
fromExisting: wrap already-deployed AWS resources with a Building Block and use them (for resources that have a corresponding Building Block)
Custom Block: build your own Building Block
Vendorize: pull the Block's code into your own repository

Let's look at a concrete example. SQS has no corresponding Building Block, so you connect it as raw CDK with CDK-in-Blocks. For instance, if you want to "send a message from the Blocks API to an existing SQS queue created by another stack," you reference the existing queue from index.cdk.ts and pass permissions and environment variables to the Lambda.

// aws-blocks/index.cdk.ts — reference an existing queue owned by another stack and wire it to the Lambda
import * as sqs from 'aws-cdk-lib/aws-sqs';

const externalQueue = sqs.Queue.fromQueueArn(
  blocksStack, 'ExternalQueue',
  'arn:aws:sqs:ap-northeast-1:123456789012:my-queue',
);
externalQueue.grantSendMessages(blocksStack.handler);
blocksStack.handler.addEnvironment('EXTERNAL_QUEUE_URL', externalQueue.queueUrl);

On the other hand, for resources that have a corresponding Building Block (existing DynamoDB tables, S3 buckets, RDS, Cognito, and so on), wrapping them with fromExisting is a better fit. Just by passing the existing resource name to something like table: when creating the Building Block, you can use the typed API and the local mock as is, and the Building Block grants IAM permissions automatically too. You write this on the index.ts (backend definition) side.

// aws-blocks/index.ts — wrap an existing DynamoDB table with DistributedTable
const todos = new DistributedTable(scope, 'todos', {
  schema: todoSchema,
  key: { partitionKey: 'userId', sortKey: 'todoId' },
  table: DistributedTable.fromExisting('my-existing-todos-table'), // ← pass the existing table name
});

IAM Is Automatic and Least-Privilege per Block

When you instantiate a Building Block, the Lambda is automatically given permissions limited to that resource. For example, with KVStore, read/write permissions on the generated DynamoDB table (and its indexes) are granted scoped to that table's ARN (internally equivalent to CDK's grantReadWriteData).

That is, it doesn't take the broad approach of granting dynamodb:* on all tables, and it can't touch other tables or other resources. The reason least privilege is applied without you hand-writing IAM policies is that the Building Block makes good use of CDK's grant~ behind the scenes.

Constraints You Should Know

The Architecture Is a Lambda-lith (All Methods in a Single Lambda)

When I actually ran cdk synth and checked the contents, all the API methods coexisted in a single Lambda function (memory 2048 MB / timeout 900 seconds). It's a configuration where API Gateway's proxy integration routes everything to a single Lambda (a so-called Lambda-lith).

This comes with the following constraints.

You can't separate scaling, permission isolation, or deployment units per method
Bundle size and cold starts grow in proportion to the size of the API

As of now, it's not suited to use cases where you want to split functions in a microservices style. By contrast, for a prototype or a small-to-medium API, a single function works perfectly well, so this won't be a problem as long as you understand it as a design trade-off. That said, this configuration reflects how it's built at this point in time, and there seems to be room for it to change in future updates.

Side Effects of `new`

new DistributedTable(...) and new KVStore(...) are not just object creation; behind the scenes they are converted into CDK resource definitions. AWS Blocks is built on a philosophy that blurs the boundary between app and infrastructure even more than AWS CDK does, so those accustomed to recent, modern app development may feel some discomfort here.

For example, suppose you new a resource you access in your business logic within that logic, and then, in order to access the same resource from another endpoint, you also new that resource definition in a different file. The same resource definition then ends up being new-ed in the CDK code, and an Already Exists error occurs at the CDK layer. It's also worth keeping in the back of your mind that casually increasing instances as if they were ordinary classes unintentionally increases your infrastructure, and casually deleting them deletes the actual resources too.

To avoid this, you need to declare the resource definition somewhere in a separate file (a class or a function) and have each piece of logic load it. With careful design and proper model separation, it's entirely possible to implement this without any awkwardness, but if you don't do it well, you end up with an implementation that has merely separated app and infrastructure, which can look like it goes against the philosophy of AWS Blocks.

Conversely, if you have concerns about this area, I think you may be better off simply using AWS CDK.

When to Use It

The first cases that come to mind are when you want to write code without being conscious of AWS or CDK, or when you don't have members who are very familiar with them. I also think it fits cases where the application you're building isn't very large—such as a prototype or internal tool you just want to launch quickly, or when you want to build something full-stack easily.

As a CDK user too, I think it's a good fit, since—as mentioned earlier—you can also extend CDK in index.cdk.ts. After all, CDK's flavor seeps through in how you write the code, with things like scope and id, so knowing CDK may let you implement smoothly.

AWS Blocks: Full-Stack Building Blocks That Run Locally Without an AWS Account

Vivek V. — Tue, 16 Jun 2026 21:08:56 +0000

Why I built a Custom Kiro Power to ship faster with AWS Blocks

Every developer building on AWS has hit the same wall. You need a sandbox account to test your idea. In most organizations that means a ticket, an approval workflow, a budget tag, and three days of waiting before you can find out if your DynamoDB schema even makes sense.

Once you get access, the iteration cycle starts: write code, deploy, wait two minutes, request pull request approvals from Cloud Engineering/Platform leads and discover your IAM policy is wrong, fix it, deploy again, wait again. For a single table and an API endpoint, you might burn half a day just getting to "hello world" on real infrastructure.

And when the sprint is over, somebody needs to tear it all down so the bill doesn't keep running.

AWS Blocks eliminates this entire loop. You write your application using self-contained building blocks that run locally on your laptop with zero AWS credentials, then flip a single switch to deploy real infrastructure when the feature is validated. No sandbox requests. No deploy-wait-fix cycles during development. No environment-specific configuration. The code you test locally IS the code that runs on AWS.

How AWS Blocks works

You import a building block and use it directly in your application logic. There is no separate infrastructure definition layer. AWS calls this "Infrastructure from Code" — your backend entry point (aws-blocks/index.ts) is both your runtime code and your infrastructure definition simultaneously.

import { Scope, ApiNamespace, DistributedTable } from '@aws-blocks/blocks';
import { z } from 'zod';

const scope = new Scope('my-app');

const tasks = new DistributedTable(scope, 'tasks', {
  schema: z.object({
    userId: z.string(),
    taskId: z.string(),
    title: z.string(),
    done: z.boolean(),
  }),
  key: { partitionKey: 'userId', sortKey: 'taskId' },
});

export const api = new ApiNamespace(scope, 'api', (context) => ({
  async addTask(userId: string, title: string) {
    await tasks.put({ userId, taskId: crypto.randomUUID(), title, done: false });
  }
}));

Run npm run dev and this works immediately. The table persists to .bb-data/ on disk (survives restarts). The API serves on localhost with hot reload. Your frontend imports the backend directly with full type safety — no codegen, no REST clients, no OpenAPI specs:

import { api } from 'aws-blocks';

const result = await api.addTask('user123', 'Write blog post');
// TypeScript knows the return type. Change the backend signature,
// and the frontend breaks at compile time. No contract drift.

Add a method to the API, and it shows up in frontend autocomplete instantly. Change a return type in the backend, and TypeScript breaks the frontend at compile time itself. No more developer excuses like "the API changed but nobody updated the client."

When the feature is ready, run npm run sandbox and the same code deploys as DynamoDB, API Gateway, and Lambda. You did not configure IAM policies, write CloudFormation, or think about capacity modes. The building block provisions itself.

And when you outgrow what a Block provides? Every AWS Blocks app is a CDK app. Drop into aws-blocks/index.cdk.ts and use any CDK construct directly. You're never stuck in an abstraction.

What changes for teams

A new developer clones the repository, runs npm run dev, and has a working application in thirty seconds without ever needing an AWS account. They build and test features for days or weeks against local mocks that behave identically to production services. The CI pipeline runs integration tests against the same typed API imports — no browser required, no sandbox credentials to manage, no access keys to rotate.

When a feature passes review, one deploy command creates real infrastructure. The team goes from "works on my machine" to "running on AWS" without changing a single line of application code, because there was never a separate infrastructure layer that could drift out of sync.

For organizations where AWS environment access is controlled, gated, or simply slow to provision, this is the difference between developers waiting and developers shipping.

What you can build

Twenty building blocks covering data, identity, communication, compute, storage, AI, and observability. Each runs locally with zero configuration and deploys to production AWS services:

You write	Runs locally as	Deploys to AWS as
`new DistributedTable(scope, 'orders', { schema, key, indexes })`	JSON file on disk	DynamoDB with GSIs
`new Database(scope, 'analytics', { migrationsPath })`	PGlite (WASM Postgres)	Aurora Serverless v2
`new DistributedDatabase(scope, 'global', { migrationsPath })`	PGlite (WASM Postgres)	Aurora DSQL (zero idle cost)
`new AuthCognito(scope, 'auth', { mfa, groups })`	Local JWT mock	Cognito User Pool
`new Realtime(scope, 'collab', { namespaces })`	Local WebSocket server	API Gateway WebSocket
`new AsyncJob(scope, 'process', { schema, handler })`	Runs in-process	SQS + Lambda
`new CronJob(scope, 'digest', { schedule, handler })`	Manual trigger	EventBridge + Lambda
`new FileBucket(scope, 'uploads')`	Local filesystem	S3 with presigned URLs
`new KnowledgeBase(scope, 'docs', { source })`	Local vector search	Bedrock Knowledge Bases
`new Agent(scope, 'assistant', { model, tools })`	Canned responses	Amazon Bedrock
`new EmailClient(scope, 'notify', { fromAddress })`	Console log	SES
`new Logger(scope, 'log')`	Console output	CloudWatch Logs
`new Metrics(scope, 'metrics')`	No-op	CloudWatch (EMF)
`new Tracer(scope, 'tracer')`	No-op	X-Ray

Plus KVStore for simple key-value, AuthBasic for JWT auth, AuthOIDC for social login, AppSetting for config/secrets, Dashboard for auto-generated observability views, and Hosting for frontend deployment (CloudFront + S3 with SSR support).

It works on every major web framework — Next.js, Nuxt, Astro, SolidStart, TanStack Start, React, Vue, Svelte, Angular — and generates typed native clients for iOS (Swift), Android (Kotlin), and Flutter (Dart) from the same backend definition.

Teaching your AI agent a new framework

AWS Blocks ships steering files inside the npm package that guide AI coding agents to build correct code. These work with any agent that reads node_modules documentation.

A Custom Kiro Power takes this further. It packages all twenty building block APIs, their constructor signatures, common pitfalls, deployment workflows, and scaffolding automation into a context bundle that activates dynamically when you mention "blocks" or "building blocks" in conversation. The agent doesn't need to dig through node_modules — it already knows the patterns.

Here is what that looks like in practice. You open Kiro, point it at an empty directory, and say:

"Create a bookmark manager with AWS Blocks where users can save links, tag them, and search by tag or date added"

The Power activates. The agent scaffolds the project with --yes (non-interactive), defines a DistributedTable with a schema for bookmarks, adds a byTag index and a byDate index, exports an ApiNamespace with CRUD methods plus tag-based queries, wires up AuthBasic for user isolation, and starts the dev server. Working app, running locally, in under a minute.

"Use building blocks to create a file sharing app with upload, download links, and automatic cleanup of expired files"

It uses FileBucket for presigned upload/download URLs, DistributedTable to track file metadata and expiry, CronJob to sweep expired files daily, and AuthOIDC with Google sign-in for user identity.

Without the Power, the agent would need to find and read each block's README from node_modules. With the Power, it builds complete applications from a single sentence because all the patterns, gotchas, and deployment knowledge are pre-loaded.

From local to production in two commands

AWS Blocks gives you two deployment paths:

Sandbox — fast, ephemeral, backend-only. Deploys in seconds using Lambda hot-swapping. Each developer gets an isolated environment. Use this during development to test against real AWS services.

npm run sandbox          # deploy backend to AWS (seconds)
npm run sandbox:destroy  # tear it down

Production — full CDK deployment via CloudFormation. Deploys your entire application including frontend hosting (CloudFront + S3), SSR if you use Next.js/Nuxt/Astro, custom domains, and WAF. Use this for staging and production environments.

npm run deploy   # full production deployment
npm run destroy  # tears down all deployed resources

The same code that ran on localhost:3000 is now running on AWS with DynamoDB tables, Lambda functions, API Gateway endpoints, and a CloudFront distribution. No infrastructure files were written. No IAM policies were hand-crafted. The building blocks provisioned themselves.

Install Custom Kiro Power for AWS Blocks

Kiro IDE: Powers panel > Add Custom Power > Import from GitHub > https://github.com/awsdataarchitect/aws-blocks

The Custom Kiro Power is at github.com/awsdataarchitect/aws-blocks for anyone who wants to go from plain English to a running AWS Blocks app without reading docs first.

Or Try it Manually

If you want to scaffold manually without a Power or any AI steering:

npm create @aws-blocks/blocks-app@latest my-app
cd my-app
npm run dev

Working application in seconds, deployable to AWS with npm run sandbox when you are ready. Full production deployment with hosting via npm run deploy. Tear it down via npm run destroy.

Links:

Now, what will you build with AWS Blocks?

Adding Memory to the Agent

Matt Lewis — Tue, 16 Jun 2026 08:54:17 +0000

This is the fourth in a series of posts documenting the architecture, implementation, and lessons learned from building the AWS Briefing Agent - a personalised AWS assistant deployed on Amazon Bedrock AgentCore Runtime.

Part 1: Building a Full-Stack AI Agent on Bedrock AgentCore
Part 2: Data Ingestion: RSS Feeds, Knowledge Base, S3 Vectors, and Metadata Filtering
Part 3: Strands Agents + AgentCore Runtime - a perfect match
Part 4: Adding Memory to the Agent
Part 5: Experimenting with API Gateway
Part 6: Observability and Evaluations
Part 7: Third Party Integrations - Identity, Gateway and Slack Notifications

As mentioned in the first blog post, each session on AgentCore Runtime is assigned a dedicated Firecracker microVM with isolated CPU, memory and filesystem resources. When the session finishes, the entire microVM is destroyed. There is no shared state between sessions, which prevents any cross-session data leakage.

When a user accesses our AWS Briefing Agent service for the first time, they are asked a number of questions.

This includes asking about the primary AWS services the user is interested in, their experience level in AWS, and if there are specific AWS areas they want to track closely.

Without any memory capability, the user will have to provide the same information each time they start a new session. This is where AgentCore Memory comes into play. This post walks through setting up AgentCore Memory using Strands Agents.

Configuring Memory in AgentCore

The agentcore.json file is the primary configuration file used in Amazon Bedrock AgentCore to define and manage AI agents, gateways, memory stores and datasets. It acts as the central orchestrator to package up the agents infrastructure.

When we run the agentcore deploy command, the CLI reads this file and uses the AWS CDK to synthesize and deploy CloudFormation resources. We add long term to our agent in the memory section using a resource identifier of "BriefingAgentMemory". This is the identifier that is referenced in our handler.

AgentCore Memory itself consists of several key components that work together to provide both short-term context and long-term intelligence for agents as shown in the diagram below:

The interactions with the user are stored in short term for 90 days, as specified in the event expiry duration attribute. We then specify two distinct memory strategies that transform these short term raw events into long-term memory. Note that all strategies by default ignore personally identifiable information (PII) data from long-term memory records.

We define the following strategies in the agentcore.json file:

SEMANTIC - this memory strategy identifies and extracts key pieces of factual information and contextual knowledge from conversational data. For example, a user is running AWS Lambda in production.
USER_PREFERENCE - this memory strategy is designed to automatically identify and extract user preferences, choices and styles from conversations. For example, a user is interested in serverless and containers.

Each strategy stores its long-term memory in a hierarchical structure within a namespace. These namespaces act as distinct logical containers. We segregate them using the special {actorId} placeholder variable, so that we guarantee separation between each user.

The complete relevant memory section in our agentcore.json file is shown below:

  "memories": [
    {
      "name": "BriefingAgentMemory",
      "eventExpiryDuration": 90,
      "strategies": [
        {
          "type": "SEMANTIC",
          "name": "semantic_facts",
          "namespaces": [
            "/users/{actorId}/facts"
          ]
        },
        {
          "type": "USER_PREFERENCE",
          "name": "user_preferences",
          "namespaces": [
            "/users/{actorId}/preferences"
          ]
        }
      ]
    }
  ],

Integrating Cognito and AgentCore Runtime

At this point, we need to do a segway into how we authenticate requests to our agent. AgentCore Runtime supports two inbound authentication mechanisms:

AWS IAM SigV4 - where the request to the InvokeAgentRuntime API is SigV4-signed with valid AWS credentials that have the bedrock-agentcore:InvokeAgentRuntime IAM permission.
JWT Bearer Token Auth - which is configured with an Inbound JWT authoriser

When our frontend invokes the agent, it is sending a request to the agent's public endpoint URL:

https://bedrock-agentcore.eu-west-1.amazonaws.com/runtimes/<arn>/invocations

This URL is a special public-facing endpoint that AgentCore Runtime exposes. We specify this in the agentcore.json file:

  "runtimes": [
    {
      "name": "AWSBriefingAgent",
      "build": "Container",
      "entrypoint": "handler.py",
      ...
      "authorizerType": "CUSTOM_JWT",
      "authorizerConfiguration": {
        "customJwtAuthorizer": {
          "discoveryUrl": "https://cognito-idp.eu-west-1.amazonaws.com/eu-west-1_dshjdhskj/.well-known/openid-configuration",
          "allowedClients": [
            "dhjhdjskhdjkshdjkhsd"
          ]
        }
      }
    }

The discoveryUrl points to Cognito's OpenID Connect discovery document for the AWS Cognito User Pool with the specified ID that is being used to authenticate users to the frontend. When AgentCore Runtime wants to validate the JWT token, it retrieves information from this endpoint such as the issuer and JWKS endpoint (contains the public keys used to verify the JWT signature).

The allowedClients shows the Cognito Application Client ID. When a user logs in, Cognito stamps the token with the client_id. AgentCore validates the JWT’s client_id claim, so only tokens issued for one of the permitted application clients can invoke the runtime.

When the user logs into our frontend application with their email address and password, the frontend calls Cognito directly to verify, and receives back

Access token — proves who you are and what you're allowed to do.
ID token — contains profile info (email, name). Used by the frontend to display the username.
Refresh token — used to get new access/ID tokens when they expire (usually after 1 hour). These tokens are stored by the frontend auth library.

When we send a request to the agent, the frontend attaches the access token as a bearer token

POST /invocations
Authorization: Bearer eyJraWQi...
Body: {"prompt": "Give me a briefing"}

This is the JWT token that gets validated by AgentCore Runtime.

Returning Memory records in Handler function

The following code snippet shows how we retrieve the memory records to display in the sidebar of the frontend.

@app.entrypoint
async def invoke(payload: Dict[str, Any], context: Any = None):
    message = payload.get("prompt", payload.get("message", ""))

    # Derive actor_id from the JWT 'sub' claim (source of truth)
    actor_id = _extract_sub_from_jwt(context) or payload.get("user_id", "default-user")

    # Sanitize actor_id for AgentCore Memory
    actor_id = re.sub(r"[^a-zA-Z0-9\-_/]", "_", actor_id)

    # Retrieve memory records to include in the stream
    memory_used = get_memory_records(actor_id, message)

The @app.entrypoint decorator registers a function as the handler for POST requests to /invocations. AgentCore Runtime calls this handler function when a client invokes the agent. Our handler function is an async generator, which means that it automatically streams the response as Server-Sent Events (SSE) delivered to the client in real-time (more around this in the next blog post).

Within the handler, we get the message that has been sent in the payload. We then extract the user's identity from the JWT token that Cognito issued. One of the claims in the JWT token is the sub or subject, which is the unique user ID assigned by Cognito to a user when they first register. We know that the JWT token has been cryptographically signed by Cognito and validated by AgentCore Runtime before it reaches the handler function. We assign this sub value to be the actor_id. We apply some regex to the actual value to ensure it has no characters in it that are not supported.

We then call our get_memory_records function. This function calls the AgentCore retrieve memory records API to search the long-term memory for facts and preferences relevant to the promt that has just been passed in. We retrieve the 5 highest scoring results from the vector search and store them in a records array, which is streamed back to the frontend to be displayed in the sidebar.

def get_memory_records(actor_id: str, prompt: str) -> List[Dict[str, Any]]:
    """Retrieve long-term memory records relevant to the user's prompt.

    Searches both the facts and preferences namespaces and returns
    the records the agent would have seen for this invocation.
    """
    if not MEMORY_ID:
        return []

    try:
        client = boto3.client("bedrock-agentcore", region_name=REGION)
        records = []

        for namespace in [
            f"users/{actor_id}/facts",
            f"users/{actor_id}/preferences",
        ]:
            try:
                response = client.retrieve_memory_records(
                    memoryId=MEMORY_ID,
                    namespace=namespace,
                    searchCriteria={
                        "searchQuery": prompt,
                        "topK": 5,
                    },
                    maxResults=5,
                )
                for r in response.get("memoryRecordSummaries", []):
                    records.append({
                        "memoryRecordId": r.get("memoryRecordId", ""),
                        "text": r.get("content", {}).get("text", ""),
                        "score": r.get("score"),
                        "memoryStrategyId": r.get("memoryStrategyId", ""),
                        "namespaces": r.get("namespaces", []),
                    })
            except Exception as exc:
                logger.warning("Failed to retrieve from %s: %s", namespace, exc)

        return records
    except Exception as exc:
        logger.warning("Failed to retrieve memory records: %s", exc)
        return []

We can see an example of the sidebar in the frontend below:

Setting up Memory with Strands

Both short-term and long-term memory are handled for us automatically through the AgentCore Memory session manager integration for Strands.

The memory ID is retrieved in a module-level constant:

MEMORY_ID = os.environ.get("MEMORY_BRIEFINGAGENTMEMORY_ID")

This reads the memory resource ID that AgentCore Runtime automatically injects as an environment variable into your container at runtime. The naming convention is: MEMORY__ID. Given the memory was given a name of "BriefingAgentMemory" in the agentcore.json file, AgentCore sets MEMORY_BRIEFINGAGENTMEMORY_ID to the actual memory resource ID (something like AWSBriefingAgent_BriefingAgentMemory-q2iBfL64BS).

The following function in our code is called on every request. A new stateless Strands Agent instance is created on each invocation, configured with the relevant session manager that loads conversation history from AgentCore Memory, tools and model settings.

def _create_agent(session_id: str, actor_id: str, gateway_tools: list = None) -> Agent:
    """Create a Strands Agent with KB retrieval, AgentCore Memory, and Gateway tools."""
    session_manager = None

    if MEMORY_ID:
        try:
            from bedrock_agentcore.memory.integrations.strands.config import (
                AgentCoreMemoryConfig,
                RetrievalConfig,
            )
            from bedrock_agentcore.memory.integrations.strands.session_manager import (
                AgentCoreMemorySessionManager,
            )

            config = AgentCoreMemoryConfig(
                memory_id=MEMORY_ID,
                session_id=session_id,
                actor_id=actor_id,
                retrieval_config={
                    f"users/{actor_id}/facts": RetrievalConfig(
                        top_k=5, relevance_score=0.5
                    ),
                    f"users/{actor_id}/preferences": RetrievalConfig(
                        top_k=5, relevance_score=0.5
                    ),
                },
            )
            session_manager = AgentCoreMemorySessionManager(
                agentcore_memory_config=config,
                region_name=REGION,
            )
        except Exception as exc:
            logger.warning("Failed to initialise memory session manager: %s", exc)

    tools = [retrieve, format_slack_message] + (gateway_tools or [])

    return Agent(
        system_prompt=_load_system_prompt(),
        model=_create_model(),
        tools=tools,
        session_manager=session_manager,
        conversation_manager=SlidingWindowConversationManager(
            window_size=20,
            should_truncate_results=True,
            per_turn=True,
        ),
        callback_handler=None,
    )

In our code, if memory has been set, then we import the AgentCoreMemorySessionManager. This session manager integrates Strands agents with AgentCore Memory, which synchronises the short-term and long-term memory capabilities. Some of its features include loading the conversation history from short-term memory during agent initialisation, and integrating with long-term memory for context injection into agent state.

Next we create a AgentCoreMemoryConfig configuration object which will be passed to the session manager telling it:

memory_id - which AgentCore Memory resource to connect to
session_id - the identifier for the conversation session
actor_id - the unique identifier for the user
retrieval_config - a dictionary mapping of namespaces to retrieval configurations. This tells the session manage to search the two namespaces for relevant long-term memories, and to get the 5 most relevant facts and user preferences

Our use of AgentCore Memory is now handled automatically by Strands Agents session manager. Before each turn, it will load recent events from the same session to populate the agent's conversation context. The short-term memory is the raw event stream. The agent will see the last 20 turns in its context window, as this has been configured with the Sliding Window Conversation Manager. After (and during) invocations of the agent, new conversation messages are automatically persisted to AgentCore Memory.

With this in place, we have now successfully added long-term memory to our agent, personalising the briefing for each user based on their preferences.

Biography

As Chief AWS Architect at IBM in the UK, I am responsible for growing the AWS capability and community within one of the fastest growing AWS consulting partners globally. This gives me the opportunity to try out the latest features in preview before they go into general availability. You'll often find me blogging about my experience, but please reach out if there are services you'd like to know more about.

Serverless applications on AWS with Lambda using Java 25, API Gateway and Aurora DSQL - Lambda performance optimization approaches

Vadym Kazulkin — Mon, 15 Jun 2026 14:51:59 +0000

Introduction

In the previous articles of the series about how to develop, run, and optimize Serverless applications on AWS with Lambda using Java 25, API Gateway, and Aurora DSQL database, we used:

Managed Java 25 runtime
GraalVM Native Image deployed as Lambda Custom Runtime

We also did Lambda performance (cold and warm starts) measurements with the following settings:

Lambda functions used 1024 MB of memory
Java compilation option "-XX:+TieredCompilation -XX:TieredStopAtLevel=1"
Lambda x86_64 architecture used

In this article, we'll introduce some additional Lambda performance (cold and warm starts) optimization approaches to apply to our sample application. You'll need to measure the performance by yourself to figure out whether they will provide the desired Lambda performance improvements.

Please keep in mind that you can also deploy our sample application on AWS Lambda as a (Docker) Container Image. I didn't cover this approach, but you can look into my article series Lambda function using Docker Container Image for a step-by-step introduction on how to do it. I used DynamoDB as a database in this example. The cold start will be quite big. Lambda SnapStart isn't available for the Lambda deployment as a Container Image. Instead, you can use Ahead-of-Time (AOT) and CDS caches for the Container Image and then measure the Lambda performance.

Lambda performance optimization approaches

We can apply the following approaches to the managed Java runtime and GraalVM Native Image. For the managed Java runtime, it includes SnapStart being enabled and applying the priming techniques on top:

Try out different Lambda memory settings. We performed all measurements with 1024 MB of memory for the Lambda function. With different memory settings, you might become better at the price-performance trade-off.
Try out setting Lambda arm64 architecture using AWS Graviton2 processor, which supports SnapStart since July 2024. This can provide a better cost-performance trade-off compared to x86 architecture.

We can apply the following approaches primarily only to the managed Java runtime on Lambda. This includes SnapStart being enabled and applying the priming techniques on top:

Try out different Java compilation options for the Lambda function. We performed all measurements until now with the compilation option "-XX:+TieredCompilation -XX:TieredStopAtLevel=1". We can provide other compilation options to the Lambda function using an environment variable called JAVA_TOOL_OPTIONS. This can have different cold and warm starts trade-offs. For GraalVM Native Image, the choice of Java compilation method doesn't have much impact on the Lambda performance. This is because our application is already compiled natively.
Further exclude unused dependencies. With that, we can especially reduce the cold start times (also for SnapStart enabled). In the case of GraalVM Native Image, only reachable Java classes, functions, and methods will become a part of the Native Image, so including unused dependencies may not help that much.

We can apply the following approach primarily to the managed Java runtime on Lambda with SnapStart enabled:

Search for further Lambda SnapStart priming potential in addition to those we introduced in this series. For this, you can use AWS Lambda Profiler Extension for Java. I described it in my article Improving Lambda performance with Lambda SnapStart and priming.

We can apply the following approach primarily to the GraalVM Native Image :

Try out Profile-Guided Optimizations to see whether you can further improve Lambda performance. The difficulty of trying out this technique is that you'll need to do some additional semi-automated steps to run your application either with the Lambda emulator locally or in an extra environment to obtain the profile of your application, which you'll then need to use to generate the optimized Native Image. You can use Lambda extension for it, but it still requires a lot of additional work. This is the work AWS did for us in case Lambda SnapStart is enabled. I really appreciate that I don't need to care about generating, encrypting, storing, and restoring the snapshots/profiles.

Conclusion

In this article, we introduced additional Lambda performance optimization approaches that we can use in our sample application. Try them out on your own to figure out whether they will provide the desired Lambda performance improvements.

Please also watch out for another series where I use a NoSQL serverless Amazon DynamoDB database instead of Aurora DSQL to do the same Lambda performance measurements.

If you like my content, please follow me on GitHub and give my repositories a star!

Please also check out my website for more technical content and upcoming public speaking activities.

Building Real-Time Fraud Detection with Amazon Bedrock and Claude AI - (Let's Build 🏗️ Series)

awedis — Fri, 12 Jun 2026 19:47:16 +0000

Imagine a fintech or e-commerce platform where you want to detect suspicious transactions in real time. Instead of writing complex rules only, you use AI + event-driven architecture.

In this article, I will design an architecture that detects fraudulent traffic and malicious activity in a system using AI and Amazon Bedrock.

The main parts of this article:
1- Architecture
2- Flow Step-by-Step, AWS Bedrock (Claude)
3- Key Takeaways

1- Architecture

I used Amazon EventBridge as an example here, but you might be using Amazon API Gateway or even AWS Step Functions in your architecture.

2- Flow Step-by-Step

A. Transaction Happens

A user makes a payment:

{
  "detail": {
    "user_id": "123",
    "amount": 2500,
    "country": "unknown VPN location",
    "device": "new device",
    "time": "03:12 AM"
  }
}

Event is sent to EventBridge.

B. Lambda Trigger

Lambda receives the transaction and prepares a prompt.

C. Amazon Bedrock (Claude)

import boto3
import json

bedrock = boto3.client("bedrock-runtime")

def lambda_handler(event, context):
    tx = event["detail"]

    prompt = f"""
    You are a fraud detection system.

    Analyze the transaction and return:
    - risk_score (0-100)
    - decision (ALLOW / REVIEW / BLOCK)
    - reason

    Transaction:
    {tx}
    """

    response = bedrock.invoke_model(
        modelId="eu.anthropic.claude-haiku-4-5-20251001-v1:0",
        body=json.dumps({
            "anthropic_version": "bedrock-2023-05-31",
            "messages": [
                {"role": "user", "content": prompt}
            ],
            "max_tokens": 200
        })
    )

    result = json.loads(response["body"].read())
    return result

D. AI Output Example

{
      "type": "text",
      "text": "# Fraud Detection Analysis\n\n**risk_score:** 78\n\n**decision:** REVIEW\n\n**reason:** Multiple risk factors detected:\n- **Unknown VPN location** - Unable to verify legitimate geographic origin; suggests attempted anonymization\n- **New device** - First transaction from unrecognized device; increases fraud probability\n- **Unusual transaction time** (03:12 AM) - Outside typical user activity windows\n- **Moderate-high amount** ($2,500) - Significant transaction value amplifies risk\n\n**Recommendation:** Require additional verification (2FA, identity confirmation, or customer contact) before processing."
}

E. Action Taken

Depending on response:

BLOCK → reject transaction
REVIEW → send to manual review queue
ALLOW → proceed normally

3- Key Takeaways

Together, these capabilities make AI-powered event-driven systems far more powerful than traditional rule-based approaches, as they can understand context instead of relying on static thresholds, make real-time decisions as events occur without batch delays, scale effortlessly from a few transactions to millions thanks to serverless architectures, and continuously adapt to new fraud patterns or behaviors. This combination enables systems that are not only scalable and efficient, but also intelligent, dynamic, and resilient to changing environments.

Happy coding 👨🏻‍💻

💡 Enjoyed this? Let’s connect and geek out some more on LinkedIn.

Serverless applications on AWS with Lambda using Java 25, API Gateway and DynamoDB - Part 7 Lambda performance optimization approaches

Vadym Kazulkin — Tue, 09 Jun 2026 15:04:06 +0000

Introduction

In the previous articles of the series about how to develop, run, and optimize Serverless applications on AWS with Lambda using Java 25, API Gateway, and DynamoDB, we used:

Managed Java 25 runtime
GraalVM Native Image deployed as Lambda Custom Runtime

We also did Lambda performance (cold and warm starts) measurements with the following settings:

Lambda functions used 1024 MB of memory
Java compilation option "-XX:+TieredCompilation -XX:TieredStopAtLevel=1"
Lambda x86_64 architecture used
Default Apache HTTP Client (version 4.5) used to connect to the DynamoDB

Please keep in mind that you can also deploy our sample application on AWS Lambda as a (Docker) Container Image. I didn't cover this approach, but you can look into my article series Lambda function using Docker Container Image for a step-by-step introduction on how to do it. The cold start will be quite big. Lambda SnapStart isn't available for the Lambda deployment as a Container Image. Instead, you can use Ahead-of-Time (AOT) and CDS caches for the Container Image and then measure the Lambda performance.

Lambda performance optimization approaches

To find a good balance between the cold and warm start times of the Lambda function, you can try out the optimization techniques introduced below. I have not taken any additional measurements with our sample application with Java and GraalVM 25, but have done so using older Java, GraalVM, and dependency versions. I'll provide references to my relevant articles. Measurements that I did back then might already be outdated, so I strongly recommend you to re-measure.

Try out different Lambda memory settings. We performed all measurements with 1024 MB of memory for the Lambda function. With different memory settings, you might become better at the price-performance trade-off. See my article Measuring cold and warm starts and deployment time with Java 21 using different Lambda memory settings for further examples, performance measurements, and conclusions.
Try out setting Lambda arm64 architecture using the AWS Graviton2 processor, which supports SnapStart since July 2024. This can provide a better cost-performance trade-off compared to x86 architecture. See my article AWS Lambda performance with Java 21: x86 vs arm64 - Initial measurements for some insights.
Try out different synchronous HTTP clients to establish an HTTP connection to DynamoDB. We performed all measurements until now with the default synchronous Apache HTTP Client version 4.5. There are other options like UrlConnection and AWS CRT HTTP clients, which provide different performance trade-offs for the cold and warm start. See my article Measuring cold and warm starts with Java 21 using different synchronous HTTP clients for further examples, performance measurements, and conclusions. GraalVM Native Image also supports the AWS CRT HTTP Client, and I did some measurements using a pure Java Lambda function in my article Measuring cold and warm starts with GraalVM 23 and AWS CRT HTTP Client. Recently, also Apache 5.x based HTTP client has been released, so you can try it out.
Explore whether an asynchronous HTTP client for DynamoDB is an option for your use case. The default asynchronous HTTP Client is NettyNio. There is another option, the AWS CRT async HTTP client, which provides different performance trade-offs for the cold and warm starts. See my article Measuring cold and warm starts with Java 21 using different asynchronous HTTP clients for further examples, performance measurements, and conclusions.

We can apply the following approaches primarily only to the managed Java runtime on Lambda. This includes SnapStart being enabled and applying the priming techniques on top:

Try out different Java compilation options for the Lambda function. We performed all measurements until now with the compilation option "-XX:+TieredCompilation -XX:TieredStopAtLevel=1". We can provide other compilation options to the Lambda function using an environment variable called JAVA_TOOL_OPTIONS. This can have different cold and warm starts trade-offs. See my article Measuring cold and warm starts with Java 21 using different compilation options for further examples, performance measurements, and conclusions. For GraalVM Native Image, the choice of Java compilation method doesn't have much impact on the Lambda performance. This is because our application is already compiled natively.
Further exclude unused dependencies. With that, we can especially reduce the cold start times (also for SnapStart enabled); see my article Measuring cold starts with Java 21 using different deployment artifact sizes. In the case of GraalVM Native Image, only reachable Java classes, functions, and methods will become a part of the Native Image, so including unused dependencies may not help that much.

We can apply the following approach primarily to the managed Java runtime on Lambda with SnapStart enabled:

Search for further Lambda SnapStart priming potential in addition to those we introduced in this series. For this, you can use AWS Lambda Profiler Extension for Java. I described it in my article Improving Lambda performance with Lambda SnapStart and priming.

We can apply the following approach primarily to the GraalVM Native Image :

Try out Profile-Guided Optimizations to see whether you can further improve Lambda performance. The difficulty of trying out this technique is that you'll need to do some additional semi-automated steps to run your application either with the Lambda emulator locally or in an extra environment to obtain the profile of your application, which you'll then need to use to generate the optimized Native Image. You can use Lambda extension for it, but it still requires a lot of additional work. This is the work AWS did for us in case Lambda SnapStart is enabled. I really appreciate that I don't need to care about generating, encrypting, storing, and restoring the snapshots/profiles.

Conclusion

Please also watch out for another series where I use a relational serverless Amazon Aurora DSQL database and additionally the Hibernate ORM framework instead of DynamoDB to do the same Lambda performance measurements.

If you like my content, please follow me on GitHub and give my repositories a star!

Please also check out my website for more technical content and upcoming public speaking activities.

Security for MCP Servers: Governed Access Beats Uploading Spreadsheets to ChatGPT

Guy — Sun, 07 Jun 2026 14:54:28 +0000

An analyst exports a spreadsheet with customer data, uploads it into a generic AI chat, asks for a sales summary, and gets an answer in seconds. It feels productive. It is also one of the least governed ways to bring AI into an enterprise. The model sees the raw file. The organization has little control over which fields were exposed, whether PII was included, whether the same data should have been aggregated first, or whether the user should have been allowed to access every row in the first place.

There is another failure mode that matters just as much in enterprise settings: inconsistent business aggregation. I saw this directly with one company where two managers prepared reports for the same management meeting from the same underlying report, but got dramatically different answers from ChatGPT. One manager instructed it to use the company's fiscal year, which starts in July; the other did not. One mentioned Canada as part of North America; the other forgot to ask for it. Both reports looked polished; both were based on the same raw data, but because the business definitions were stored in ad hoc prompts and sent to a probabilistic LLM rather than a governed interface, the meeting started with conflicting numbers.

That is the wrong operating model for enterprise AI.

The safer pattern is to connect the AI client to an MCP server instead. The MCP server becomes the controlled interface between the model and the underlying data systems. It determines which tools exist, what inputs they accept, what data shape they return, and which authenticated user they act on behalf of. The model does not get the whole spreadsheet. It gets the smallest, governed slice of data needed to answer the question.

A well-designed MCP server prevents this kind of semantic drift. The business analyst can define that fiscal years start in July, that North America includes Canada, that revenue is recognized by a specific accounting rule, and that management reports return a standard aggregation. Every user and every model then works from the same, governed business definition, rather than relitigating those choices in every chat.

This is where the recurring motif from the previous articles comes into play again. Good MCP design is always a balance of strengths across multiple parties. Security is not the job of one component. The business analyst defines the safe business surface. The business user brings the runtime question. The LLM interprets intent. The MCP server validates and executes deterministically. And when the surface becomes more powerful, as we saw in the code mode article, the IT administrator governs policy, approval, and audit. Each party covers the other party's weaknesses.

This article shows how that balance becomes a security architecture for production MCP servers, with layers of security that collectively enhance the overall trust in AI interactions.

What Is MCP? (The 30-Second Version)

The Model Context Protocol (MCP, spec 2025-11-25) is the interface layer between AI clients and external systems through tools, prompts, and resources. In enterprise deployments, the MCP server should be a thin, remote, mostly stateless interface layer over internal data systems. If HTTP-based applications are the human-facing interface to enterprise systems, MCP servers are the AI-facing interface.

That framing matters for security. Once you see the MCP server as an interface tier, the right controls become obvious: authenticated requests, typed inputs, least-privilege downstream access, output filtering, audit trails, rate limits, and active security testing. The server is not a shortcut around enterprise controls. It is where those controls should be made explicit for AI.

There is also a larger industry lesson here. We have already spent decades learning, often painfully, how vulnerable data-facing interfaces become when teams trust inputs too much, over-privilege backends, skip per-request authorization, or expose broad attack surfaces and hope the client behaves well. SQL injection, broken access control, credential leakage, over-broad service identities, tenant isolation failures, and data exfiltration were not abstract risks. They were the history of web security. MCP is the next layer of interfaces between users, models, and data systems. It should start with those lessons, not relearn them from scratch.

The Real Alternative Is Not "MCP vs. No MCP"

A lot of enterprise security discussions start from the wrong comparison. The practical comparison is not:

MCP server
nothing at all

The real comparison is usually:

a governed MCP server
business users pasting or uploading sensitive material into a general AI chat

When a user uploads a spreadsheet directly, the organization loses most of its control points:

no tracing of how fresh the data is
no typed or validated input contract
no server-side output shaping
no per-tool authorization boundary
no guaranteed row-level or field-level policy enforcement
no reliable audit trail of which business operation was governed

An MCP server restores those control points. Instead of exposing "the file," it can expose a tool such as quarterly_sales_summary, customer_health_report, or query_sales_cube, each with deliberately constrained inputs and outputs. That is the security benefit of MCP in enterprise settings: not secrecy by obscurity, but governance by interface design.

Security Starts With Tool Design

The first article argued that tool design is MCP's UX discipline. It is also one of MCP's first security layers.

The business analyst has a direct security role at design time. They decide:

which business actions should exist as tools at all
which inputs are valid for those actions
which outputs the model actually needs
which fields should never be returned
which workflows should be packaged as prompts rather than left to ad hoc exploration

This matters because most enterprise data exposure happens long before a cryptographic control fails. It happens when the wrong interface is published.

If the analyst defines a tool as "run any SQL against finance," the server surface is already too broad. If they define it as "summarize revenue by region for a selected quarter" with an enum for region scope, a validated quarter field, and a fixed output shape, most of the risk is removed before runtime.

That is the same balance we have used throughout this series. The analyst contributes domain judgment. The server contributes deterministic enforcement. The model contributes to language understanding. The user contributes the concrete business question. Security improves because the parties are not all doing the same job badly.

Typed Inputs Are A Security Boundary

One useful pattern, and one that Rust SDKs such as PMCP support well, is to make tool contracts both model-readable and runtime-enforced from the same Rust types.

In earlier articles, we used schemars constraints and deny_unknown_fields to improve tool usability. Those same patterns are security controls:

type safety rejects malformed inputs before business logic runs
range and length constraints reject obviously abusive input early
enums narrow the space of valid values
deny_unknown_fields prevents undeclared parameters from slipping through.

That is not the whole security story, but it is an important first layer. A business analyst can define the valid business shape, and the server implementation can enforce it consistently.

#[derive(Debug, Deserialize, JsonSchema)]
#[schemars(deny_unknown_fields)]
pub struct SalesSummaryInput {
    /// Fiscal quarter to summarize, for example, 2026-Q1
    #[schemars(length(min = 7, max = 7))]
    pub quarter: String,

    /// Business unit to analyze
    pub business_unit: BusinessUnit,

    /// Aggregation granularity
    pub group_by: GroupBy,

    /// Maximum number of rows to return
    #[schemars(range(min = 1, max = 100))]
    pub limit: Option<u32>,
}

This is more than schema decoration. It is the business contract. The LLM sees the same structure during tool discovery that the server enforces at runtime.

This is also where the business analyst's role becomes concrete. They are not writing Rust, but they are deciding that quarters should follow a fiscal format, that group_by should be an enum rather than free text, and that a sales summary tool should never return an unbounded result set. The engineer implements that contract in code. A good MCP server then turns it into a discoverable and enforceable interface.

Output Boundaries Matter More Than Most Teams Think

Enterprises often focus on who may call a tool and underinvest in what the tool can return. For AI workloads, output boundaries are just as important.

This is one reason MCP is safer than directly uploading documents. The server can expose a shaped result rather than raw records. It can return aggregates instead of rows. It can omit PII fields entirely. It can redact or mask values that are useful for joins or filtering, but should never be emitted back into the model context.

That distinction matters in the long-tail cases covered by the code mode article. A database query may legitimately need sensitive fields for internal computation or joins, while still forbidding those fields from appearing in the result.

For example, a code-mode policy can allow a query to join the customer and support tables on the server side, while blocking sensitive output fields such as ssn, salary, or even raw email addresses from appearing in the returned payload. The sensitive field can participate in the computation. It does not have to be exposed to the model.

That is a much better enterprise pattern than "upload the spreadsheet and ask the model to be careful."

The same principle applies outside code mode:

Use fixed output schemas for curated tools. You want to control the shape of the data returned by the data system to optimize AI flows for security, privacy, and cost (fewer tokens).
Prefer aggregates over row dumps. Don't trust the LLM to accurately calculate sum or other statistical measures. The MCP tools are much better for such symbolic computation.
Keep tool responses aligned to the actual business question for consistency, accuracy, and security.
block fields that are unnecessary for the answer

Security is not only about stopping bad requests. It is also about making oversharing hard by design. As a side effect, this usually saves tokens and processing time too.

OAuth Matters Because The Server Must Act On Behalf Of The User

This is the most important point of authentication when building AI agentics workflows.

An enterprise MCP server should not sit in front of a database or application using a single shared application API key, a single database username/password, or a single generic service identity that grants every end user the same access. That recreates the oldest enterprise security mistake: every user gets the power of the integration account.

The correct pattern is that the MCP client and MCP server work on behalf of the authenticated user.

That is why OAuth 2.0 and OIDC matter so much in a serious MCP security model:

The MCP client handles the OAuth flow and token refresh, allowing the users to log in once and then manage the secure handling of the access and refresh tokens.
The MCP server validates the access token on every request and extracts user identity, tenant context, groups, and scopes.
downstream systems enforce the user's own permissions whenever possible, with the pass-through of the user's access tokens.

In the strongest design, that delegated identity continues past the MCP boundary. If the downstream API or data platform supports OAuth, the MCP server should forward the user's token rather than substitute a broad application credential. If the backend uses a different enforcement model, the server should still propagate token-derived user and tenant context into that system so row-level, field-level, or tenant-level policies execute for the real user. The point is to preserve user identity end-to-end, not collapse it into a shared super-account in the middle.

The practical benefit is huge. When a company already uses Entra ID, Okta, Cognito, Auth0, or another identity provider, the MCP server can integrate with existing SSO, group membership, access reviews, and offboarding. When IT disables an employee account, MCP access is revoked.

That is categorically better than a static API key model.

In a well-factored implementation, the server code can stay provider-agnostic and work with an AuthContext rather than baking identity-provider details into every tool:

fn handle_request(auth: &AuthContext) -> Result<(), Error> {
    auth.require_auth()?;
    auth.require_scope("read:sales")?;

    let user_id = auth.user_id();
    let tenant_id = auth.tenant_id();

    // Pass identity downstream so backend policies act on behalf of the user
    tracing::info!(%user_id, ?tenant_id, "authorized sales request");
    Ok(())
}

The design principle underneath this is simple: the server should validate identity, not replace identity.

Token Validation Is Not Optional Plumbing

Because MCP servers follow the web server model, token validation must be treated as a first-class responsibility of the server.

For JWT access tokens, that means validating at least:

signature
algorithm
expiration
not-before time
issuer
audience
required scopes

The principle is straightforward. The MCP server should not invent a custom auth flow. It should validate the tokens it receives and return a stable authenticated context for the rest of the codebase.

This is another place where mature SDKs help enterprise teams. A provider-agnostic authentication model lets you switch between Cognito, Entra, Google, Okta, and Auth0 by configuring the tool rather than rewriting its logic. That keeps authentication a deployment concern rather than scattering auth conditionals throughout the business code.

Security Happens In Layers

Many teams miss an important point: you do not need to implement every security rule inside the MCP server itself. You need to place each rule in the correct layer.

For enterprise MCP, a clean mental model is:

Layer 1: Server access

Validate the token. Reject invalid, expired, or misissued requests.
Layer 2: Tool authorization

Check whether this user may call this tool or workflow at all.
Layer 3: Data-level security in the backend

Let the database, API gateway, GraphQL layer, or data platform enforce row-level security, field-level authorization, column masking, or tenant isolation.

That is the design you want because the MCP server is the interface tier, not the entire security platform. If your warehouse already supports column masking, or your database already supports row-level security, the MCP server should pass through the user identity and let the backend do what it is already good at.

This is also the answer to a common enterprise objection: "Do we need to reimplement all our data security logic in the AI layer?" No. The AI-facing layer should validate access, constrain the interface, and carry the user identity through. The data systems should continue enforcing the data rules closest to the data.

Security Testing Is Another Layer, Not A Final Checkbox

Security controls designed into the server still need to be tested as part of the release process.

That is where the testing article fits directly into the security story. We argued there that MCP production testing has five gates: smoke, conformance, scenarios, load, and pentest. Security is not separate from that stack. It is one of the gates, and it also cuts through the others.

For MCP workloads, pentesting matters because the client is programmable and partially adversarial by default. You are not only defending against a careless user. You are also defending against:

prompt-injection-shaped inputs
malformed parameters
tool misuse across role boundaries
schema edge cases
attempts to exfiltrate blocked fields
tenant boundary violations

So the enterprise posture should be layered:

design-time narrowing of tools and workflows
typed validation at the interface
OAuth-based user identity
backend-enforced data permissions
policy and approval for powerful surfaces
penetration testing before production rollout and after meaningful changes

That is how you turn "secure by design" from a slogan into a release discipline.

Do Not Throw Away Web Security Lessons

One of the easiest mistakes in AI infrastructure is to treat it as so new that older security practices no longer apply. That is usually how teams recreate old failures with new tooling.

MCP servers should start from the hard-learned lessons of web and API security:

never trust client input, even when the client is an LLM
authenticate every request
authorize every operation
prefer least-privilege credentials and delegated user identity
narrow the interface surface instead of wrapping the whole backends
validate and shape outputs, not only inputs
log and audit meaningful actions
assume attackers will probe every exposed edge
test for injection, exfiltration, broken access control, and tenant leaks before production

MCP changes the interface, but it does not repeal these rules. If anything, it makes some of them more important, because the caller is now a probabilistic system that can be induced to misuse the interface in unexpected ways.

Implementation Note: Rust Helps

This article is about MCP best practices, not a specific single SDK. Still, language and tooling choices do matter. Rust is a strong fit for this layer because memory safety, strong typing, and explicit contracts are useful properties for security-sensitive interface services. SDKs such as PMCP are valuable when they make those practices easier to apply consistently, but the architectural lesson comes first: narrow interfaces, user-scoped identity, layered authorization, and governed outputs.

The Core Security Argument

The core argument of this article is not that MCP automatically makes AI safe. Poorly designed MCP servers can absolutely be insecure.

A well-designed MCP server is a more secure enterprise AI pattern than allowing users to upload raw business documents to general AI chat interfaces.

Why?

It can limit the interface to approved business operations.
It can execute symbolic computation on the server side without exposing internal data to the outside world.
It can validate inputs before execution.
It can shape outputs before data reaches the model.
It can block sensitive fields from being returned.
It can act on behalf of the authenticated user instead of a shared super-account.
It can preserve existing backend security controls.
It can be tested, audited, and governed like any other production interface.

The Security Stack In One View

To wrap the article up, here is the layered model worth carrying forward:

Business design layer

The business analyst defines the approved operations, shared business definitions, and safe aggregation rules so users do not reinvent them in ad hoc prompts.
Interface contract layer

Tools expose typed inputs, bounded schemas, constrained outputs, and narrow outcome-oriented surfaces.
Authentication layer

The MCP client and server work on behalf of the authenticated user via OAuth and validated tokens, rather than shared integration credentials.
Authorization layer

Each tool, workflow, or code-mode action is checked against the user's scopes, roles, groups, and policy rules.
Data protection layer

Backend systems enforce row-level security, field-level permissions, masking, tenant isolation, and other controls closest to the data.
Governance layer

Powerful surfaces, such as code mode, add approval, policy administration, blocked fields, execution limits, and audit trails.
Validation and testing layer

Smoke tests, conformance tests, scenario tests, load tests, and penetration tests verify that the security design actually holds in production.

If enterprise teams start MCP with those seven layers, they will begin with the lessons the industry has already paid to learn, rather than paying for them again through avoidable AI-era security failures.

Redshift Check-In: Spring 2026

elliott cordo — Thu, 04 Jun 2026 21:26:55 +0000

I have a soft spot for Amazon Redshift.

One of my early clients was a pre-GA adopter, potentially a "Client #1" on the platform. We were mid-implementation Paraccel, the engine behind Redshift's early years, and decided to quickly pivot to this new shiny warehouse in the cloud.

At this time, cloud-based data warehouse engines were unheard of. Luckily my client was a scrappy media startup, with a tremendous amount of data, and a large aging Hadoop cluster, so they were willing to take the risk. Long story short, we had some early adopter hiccups but overall it went amazingly well, and the platform is still alive and well today!

Nearly 14 years later, I thought it would be interesting to step back and ask a simple question:

How is Redshift doing?

The answer is surprisingly well.

Although Redshift is certainly not the only cloud data warehouse, and attention has shifted toward lakehouses, open table formats, and AI platforms, Redshift has quietly continued to evolve. In many ways, the service has transformed from a cloud data warehouse into a broader analytics and AI platform.

What Has Changed Since The Early Days?

The original Redshift value proposition was straightforward:

Fast SQL analytics
Managed infrastructure
Commodity cloud economics

That was enough to disrupt traditional data warehousing.

Today's Redshift is solving a very different set of problems. Modern data teams are less concerned with standing up clusters and more concerned with operational complexity, data movement, governance, and AI enablement.

AWS has spent the last several years addressing those concerns directly.

Serverless Has Become The Default

One of the biggest shifts has been the maturation of Redshift Serverless.

In the early years, Redshift administration was a significant skillset. Teams spent time sizing clusters, planning node counts, managing concurrency, and tuning workloads.

Serverless dramatically changes that operating model by removing most infrastructure decisions from the equation. More recently, AWS introduced AI-driven scaling and optimization capabilities that automatically adjust resources based on workload patterns and query characteristics.

For many organizations, Redshift is no longer something that needs to be actively managed.

The Rise Of Zero-ETL

Perhaps the most strategically important development has been AWS's investment in Zero-ETL integrations.

Historically, a significant portion of data engineering effort was spent moving data from operational systems into analytical systems. Today, Redshift can consume near real-time data from Aurora, RDS, DynamoDB, and other AWS services through managed integrations.

As someone who has spent decades building data pipelines, I find this trend particularly interesting. And although I have concerns about potential anti-patterns such as tight-coupling, I’m largely happy to avoid the undifferentiated work of moving data from point A to point B.

Redshift Is Becoming More AI Native

Another notable shift is the growing integration between Redshift and generative AI services.

Redshift now supports Amazon Q assisted SQL generation, helping users author and understand queries through natural language. AWS has also integrated Amazon Bedrock capabilities directly into the platform, allowing organizations to leverage foundation models closer to their data.

This reflects a broader trend across the industry:

The analytical warehouse is no longer just where data lives. It is increasingly where AI is applied.

The Lakehouse Conversation

Several years ago, many observers predicted that open lakehouse architectures would make traditional data warehouses obsolete.

Instead, something more interesting happened.

Redshift evolved to participate in that ecosystem rather than compete against it. AWS has continued investing in Iceberg compatibility, open data architectures, and tighter integration across analytical services. The distinction between warehouse and lake continues to blur.

The result is that organizations increasingly have flexibility in how they store data while still leveraging Redshift's query engine, governance capabilities, and performance optimizations.

My general thesis is that eventually everything will be Iceberg, and it feels like the Redshift team is in on this bet.

The Operational Maturity Story

One thing that often gets overlooked is how much operational maturity Redshift has accumulated.

Features like automatic table optimization, materialized view improvements, federated permissions, enhanced security defaults, and continual query engine optimizations may not generate headlines, but they matter tremendously in production environments.

Many of these capabilities address lessons learned from operating thousands of customer deployments over more than a decade.

A Few Things To Watch

Not every change is additive.

Organizations running older Redshift environments should be aware of the ongoing retirement of Python UDFs, with AWS encouraging migration toward Lambda UDFs and external service integrations.

Similarly, teams adopting Serverless should continue to monitor workload economics carefully. Simpler operations do not eliminate the need for cost governance.

Final Thoughts

If you had asked me in 2012 what Redshift would look like in 2026, I would have guessed larger clusters, faster hardware, and lower costs.

What actually happened is more interesting.

Redshift evolved from a cloud data warehouse into a broader analytical platform that increasingly sits at the intersection of data, governance, and AI.

The core promise remains largely unchanged: help organizations derive value from their data.

The difference is that today's Redshift spends far less time asking engineers to manage infrastructure and far more time helping them solve business problems.

5 Multi-Agent Patterns in Strands Agents: Which One and When

ricardoceci — Wed, 03 Jun 2026 11:47:48 +0000

You have an agent that searches flights. Another one checks the weather. Another one enforces corporate policies. How do you make them work together?

Strands Agents offers 5 patterns for coordinating multiple agents. Each one solves a different problem. The key difference: who decides the execution order. The model, the agents themselves, or you in code.

In this post I walk through each pattern with code, show the differences in a comparison table, and give you a decision framework to pick the right one.

All examples use Strands Agents (June 2026). The complete code is at github.com/ricardoceci/curso-strands-agentcore-2026/tree/main/clase-3.

The Running Example: Corporate Travel Agent

Every example uses the same case: a corporate travel agent that coordinates flight search, weather lookup, and recommendation generation. Three capabilities, three specialized agents.

from strands import Agent, tool

@tool
def search_flights(origin: str, destination: str, date: str) -> dict:
    """Search one-way flights via Duffel sandbox."""
    # ... Duffel API call
    return {"route": f"{origin} -> {destination}", "offers": [...]}

@tool
def get_weather(city: str, target_date: str) -> dict:
    """Get weather forecast for a city on a specific date."""
    # ... Open-Meteo API call
    return {"city": city, "max_temp_c": 28, "precipitation_mm": 0.5}

Pattern 1: Agents as Tools

An orchestrator agent uses other agents as if they were tools. The orchestrator decides when to delegate and to whom.

How It Works

You pass the sub-agent directly in the main agent's tools array. The SDK converts it to a tool automatically. When the orchestrator's model decides it needs that capability, it invokes the sub-agent. Everything runs in the same Python process.

For more control, use .as_tool() to customize the tool name and description, or the @tool decorator to wrap the call entirely.

# Specialized sub-agent for weather
weather_agent = Agent(
    tools=[get_weather],
    system_prompt="You are a weather expert. Respond concisely.",
)

# The main agent receives the sub-agent directly in tools[]
travel_agent = Agent(
    tools=[search_flights, weather_agent],  # <-- the SDK converts it to a tool
    system_prompt="You are a corporate travel assistant.",
)

response = travel_agent("Flights from EZE to MIA on Aug 20. What's the weather like?")

When to Use It

When you have a few sub-agents with clearly distinct capabilities and you want the main model to decide when to call each one. This is the most direct pattern. It requires the least coordination.

When NOT to Use It

If you need agents to communicate with each other (not only with the orchestrator) or if execution order matters. In those cases, Graph or Swarm work better.

Pattern 2: Swarm

A group of agents that coordinate autonomously through handoffs (control transfers). Each agent decides when to pass work to another.

How It Works

Strands equips each agent in the Swarm with coordination tools: a handoff tool to transfer control, and a shared context that all agents can read. The agents decide the execution order themselves.

from strands.multiagent import Swarm

flight_agent = Agent(
    name="flight_agent",
    tools=[search_flights],
    system_prompt="You search flights. When done, hand off to weather_agent.",
)

weather_agent = Agent(
    name="weather_agent",
    tools=[get_weather],
    system_prompt="You check weather at the destination.",
)

summary_agent = Agent(
    name="summary_agent",
    system_prompt="You combine flights and weather into a clear recommendation.",
)

swarm = Swarm([flight_agent, weather_agent, summary_agent])

response = swarm("I need to travel from Buenos Aires to Miami on August 20")

When to Use It

When the problem breaks down into parts that different specialists handle better, and the order can vary by task. Swarm works best for problems where you don't know in advance which agent should act first.

When NOT to Use It

If you need a predictable, repeatable execution flow. Swarm decides the order at runtime. Two runs of the same prompt can follow different paths.

Pattern 3: Graph

A directed graph where each node is an agent and edges define the execution flow. You define the structure, the framework executes in order.

How It Works

GraphBuilder gives you an API to define nodes (agents) and edges (connections). The framework passes output from one node as input to the next. It supports acyclic graphs (pipelines) and cyclic graphs (refinement loops), giving you flexibility to implement review iterations between agents.

from strands.multiagent import GraphBuilder

graph = (
    GraphBuilder()
    .add_node("search", flight_agent)
    .add_node("weather", weather_agent)
    .add_node("summary", summary_agent)
    .add_edge("search", "weather")
    .add_edge("weather", "summary")
    .build()
)

response = graph("Travel from Buenos Aires to Miami, August 20")

When to Use It

When the workflow has a strict order that should not change. For example: always search flights first, then check weather, then generate the recommendation. If the pipeline is the same every time, Graph is the right pattern.

When NOT to Use It

If you need flexibility in execution order. A Graph with many depth levels also adds latency, because each node waits for the previous one.

Pattern 4: Workflow

A task graph with explicit dependencies and automatic parallel execution. Each task is assigned to an agent with a specific system_prompt.

How It Works

Unlike Graph (which operates with agents as nodes), Workflow operates with tasks. Each task has an ID, description, dependencies, and priority. The framework resolves execution order, parallelizes what it can, and passes results between tasks.

from strands import Agent
from strands_tools import workflow

agent = Agent(tools=[workflow])

agent.tool.workflow(
    action="create",
    workflow_id="travel_planning",
    tasks=[
        {
            "task_id": "search_flights",
            "description": "Find the 5 cheapest flights from EZE to MIA on Aug 20",
            "system_prompt": "You are a flight search expert.",
            "priority": 5,
        },
        {
            "task_id": "check_weather",
            "description": "Check weather in Miami for August 20",
            "system_prompt": "You are a weather expert.",
            "priority": 5,
        },
        {
            "task_id": "generate_report",
            "description": "Combine flights and weather into an executive report",
            "dependencies": ["search_flights", "check_weather"],
            "system_prompt": "You are a corporate travel analyst.",
            "priority": 3,
        },
    ],
)

agent.tool.workflow(action="execute", workflow_id="travel_planning")

In this example, search_flights and check_weather run in parallel (no dependencies between them). generate_report waits for both to finish.

When to Use It

When you have a repeatable process with steps that can run in parallel. Workflow is the closest pattern to a traditional data pipeline: explicit dependencies, deterministic execution, results flowing between tasks.

When NOT to Use It

If you need agents to make decisions about what to do next. Workflow executes exactly what you defined, with no autonomy.

Pattern 5: A2A (Agent-to-Agent)

An open protocol (created by Google, now at the Linux Foundation) for agents to communicate over HTTP. Agents can be written in different frameworks, on different servers.

How It Works

An agent is exposed as an A2A server with an Agent Card (JSON metadata at /.well-known/agent-card.json). Another agent consumes it as a client. Communication happens over HTTP/JSON.

Server:

from strands import Agent
from strands.multiagent.a2a import A2AServer
import uvicorn

weather_agent = Agent(
    name="weather_agent",
    description="Checks weather forecasts by city and date",
    tools=[get_weather],
)

a2a_server = A2AServer(agent=weather_agent)
app = a2a_server.to_fastapi_app()
uvicorn.run(app, host="0.0.0.0", port=9000)

Client:

from strands import Agent
from strands.agent.a2a_agent import A2AAgent

remote_weather = A2AAgent(
    agent_url="http://localhost:9000",
)

# Use it as a node in a Graph
graph = (
    GraphBuilder()
    .add_node("search", flight_agent)
    .add_node("weather", remote_weather)
    .add_edge("search", "weather")
    .build()
)

When to Use It

When agents live on different servers, are written in different frameworks (Strands, LangGraph, CrewAI), or belong to different teams or organizations. A2A is the only pattern that crosses the local process boundary.

When NOT to Use It

If all agents run in the same process. A2A adds network latency (50-1000ms per call, per community benchmarks). For local communication, the other patterns are orders of magnitude faster.

Comparing the 5 Multi-Agent Patterns

Aspect	Agents as Tools	Swarm	Graph	Workflow	A2A
Who decides order	Orchestrator model	Agents (handoffs)	You (edges)	You (dependencies)	N/A (protocol)
Communication	In-process	In-process	In-process	In-process	HTTP/JSON
Latency	Microseconds	Microseconds	Microseconds	Microseconds	50-1000ms
Parallel execution	Not native	Agent-decided	Supported	Automatic	N/A
Cyclic graphs	No	No	Yes	No	No
Cross-framework	No	No	No	No	Yes
Composable	Yes (as tool)	Yes (Graph node)	Yes (nested Graph)	No	Yes (Graph node)

Decision Framework

When choosing a pattern, follow these questions:

Are the agents in the same process?
If not: A2A (the only one that crosses the network).

Does execution order matter?
If yes, is it always the same?: Graph (fixed structure).
If yes, with parallel tasks?: Workflow (dependencies + parallelism).

Do agents need to decide on their own who acts?
If yes: Swarm (autonomous handoffs).

Is it direct delegation from an orchestrator to specialists?
If yes: Agents as Tools (the most direct).

None fits perfectly?
Patterns compose with each other. A Graph can have a Swarm as a node. An agent with tools can include a remote A2AAgent. Strands lets you mix patterns without constraints.

Frequently Asked Questions

Does Swarm use A2A internally?

No. Swarm uses Python function calls within the same process. The confusion comes from both involving "communication between agents," but Swarm is local (microseconds) and A2A is remote (HTTP).

Does Graph support conditional execution?

Yes. Edges can have conditions evaluated at runtime. This gives you graphs that behave like decision trees: based on a node's output, the flow takes one path or another.

Can I combine patterns?

Yes, and this is one of Strands' strengths. A Swarm can live as a node inside a Graph. A Graph can use a remote A2AAgent as a node. An agent with tools can include another agent as a tool. Composition is unrestricted.

Which one is most token-efficient?

Agents as Tools and Graph consume fewer tokens because coordination is deterministic. Swarm can consume more because agents reason about who to hand off to. A2A adds HTTP protocol overhead.

Does Workflow replace Graph?

No. Workflow is a Strands tool (from strands-agents-tools), while Graph is a native SDK orchestrator. Workflow operates with tasks, Graph operates with agents. If you need each node to be a full agent with its own system prompt and tools, use Graph. If you need a task pipeline with dependencies, use Workflow.

Conclusion

The 5 patterns don't compete with each other. Each one solves a different coordination need. The key is to start with the most direct one that works for your case (probably Agents as Tools) and scale toward more complex patterns when you need them.

If you want to see these patterns implemented live with the Corporate Travel Agent case, the full code is in the course repository:

github.com/ricardoceci/curso-strands-agentcore-2026

Resources

Meet Deliberation: 400+ models is easy, knowing which ones earn a place is hard.

Anton Babenko — Wed, 03 Jun 2026 08:54:04 +0000

A follow-up: how a three-model consensus tool grew into a configurable, measurable panel - and why I now make every model prove it pays for its slot.

It is open source and ready to use today: github.com/antonbabenko/deliberation. If you only do one thing, star the repo and install it in your own agent - it takes about two minutes, and the install section below has the exact command for Claude Code, Codex, Cursor, Kiro, and OpenCode.

Here is the part that surprises people: you are probably already paying to access a few models. A ChatGPT subscription includes Codex which allows you to use models like gpt-5.5 and gpt-5.3-codex from Codex CLI. A Claude subscription includes Claude Code. So you can wire GPT and Claude to review each other right now, at no extra cost, and add Gemini, Grok, or any OpenRouter model when you want a third opinion.

A few weeks ago I wrote that one model is a guess and three that agree is a plan. I still believe it. But it skipped the next problem, the one I hit the moment the trick worked: once you can ask three models, you can ask thirty seven. And more voices is not the same as more signal. A slow model that always agrees with a faster one is not a second opinion. It is a bill.

So the project grew up, and it got a name: Deliberation.

The word fits. Deliberation is the slow, careful weighing of a decision - the thing a jury or a council does before it returns a verdict. That is exactly the job here: a panel of models that weighs one artifact and argues its way to a call, instead of one model blurting the first thing that sounds right. It answers a different question than the first post did. The first post asked should you make models argue? This one is about which models, under what rules, and how do you know any of it was worth the wait?

It still runs the everyday work of compliance.tf, same as before. Most of what follows came from that: not from theory, from watching real runs and asking why one took four minutes to tell me what another said in twenty seconds.

A few words first

If you are new to this, here are the only terms you need. Plain version, with examples.

Agent host. The tool you already code in: Claude Code, the Codex CLI, Cursor, Kiro, OpenCode. Deliberation plugs into all of them and behaves the same way in each.
MCP. A standard plug that lets your agent host talk to outside tools. Deliberation is one MCP server, so any host that speaks MCP can use it. You install it once; you do not write glue code.
Provider / model. A provider is a company (OpenAI, Google, xAI, OpenRouter). A model is one brain from that provider (GPT, Gemini, Grok, or any of OpenRouter's 400-plus, like Qwen, DeepSeek, or Kimi).
Panel. The set of models you ask at once. You choose who is on it. Example: a panel of GPT + Gemini + Grok.
Expert (or persona). A hat a model wears for one job: Architect, Security Analyst, Code Reviewer, Plan Reviewer, and three more. The same model reviews differently depending on the hat.
Arbiter. The one who reads everybody's answers and makes the call. It can be a model, or it can be your own agent (you).

That is the whole vocabulary. The rest is just rules for how the panel talks.

Install it now

Deliberation is one MCP server with native packaging for each host. Set only the providers you use - a missing key just turns that one provider off. Full per-host guides: public-docs/hosts.

Claude Code: add the marketplace, install the plugin, run setup.

  /plugin marketplace add antonbabenko/agent-plugins
  /plugin install deliberation
  /deliberation:setup

Codex CLI: codex plugin marketplace add antonbabenko/deliberation, then install deliberation from /plugins.
Cursor: drop in the rule file and use the one-click MCP deeplink (guide).
Kiro: "Import power from GitHub" (guide).
OpenCode: add the opencode.json MCP snippet (guide).

Credentials come from your environment: GPT uses your Codex/OpenAI login (run codex login), Gemini signs in once through its new Antigravity CLI (run agy), Grok reads XAI_API_KEY, OpenRouter reads OPENROUTER_API_KEY. Start with GPT and Claude, since you likely already pay for both.

What actually changed

Three things.

It is one MCP server now, not a Claude-only plugin. It works in Claude Code, Cursor, Codex, Kiro, OpenCode - anything that speaks MCP. Your primary agent stays in charge and calls the panel when a decision is worth a second look.

The panel opened up. It used to be three fixed externals: GPT, Gemini, Grok. Those are still the built-in voices, but you can now add any of OpenRouter's 400-plus models (including Qwen, DeepSeek, Kimi, and others) as named records in a config file. You pick them. You say which ones join a quick fan-out, which ones sit on the consensus panel, and which expert hat each one is allowed to wear - not all are equally good for all tasks.

And the whole thing learned to measure itself. That last part is the real subject of this post.

I will walk through it as five categories. Each one has a switch you flip in config, and each one votes - or refuses to - in a specific way.

Quick one-offs: just ask

Before the heavy machinery, the everyday move. Most of the time you do not want a five-round debate - you want a fast second opinion from someone who is not the model you have been talking to all session.

So you just ask, in plain words: "Ask Grok and Gemini whether this retry loop can deadlock." Your agent routes that to the right voices and brings back the answers. Or use the explicit commands when you know who you want:

/ask-gpt does this IAM policy grant more than it should?
/ask-grok poke holes in this rollback plan
/ask-gemini summarize the risk in this migration in 5 bullets

These are single-shot and advisory: one question, one answer, no loop, no context contamination from your session. The exact same commands work in Claude Code, Codex, Cursor, Kiro, and OpenCode - the server is the same everywhere, so a prompt you learn in one host transfers to the rest unchanged.

And none of this is a separate ritual you have to schedule. Call /ask-all or /consensus at any point in the work - while you are still scoping a feature, halfway through writing a plan, or in the middle of a /grill-me session when you want a real outside voice in the room instead of arguing with only yourself. The panel is available the whole time, not just at the review at the end. The earlier you pull in a dissenting model, the cheaper the disagreement is to act on.

1. Fan-out with no cross-talk

The next cheapest move: ask several models the same question at once and read the answers side by side. Nobody sees anyone else's reply, so you get real independence instead of an echo.

Two examples where this works the best:

/ask-all can this Terraform state migration be rolled back safely, and what breaks if it cannot?
/ask-all what are the failure modes of switching this table's primary key on a live database?

/ask-all runs the parallel version. panel tells you exactly who would answer for your current config before you spend a token. ask-one lets you fire each member yourself, so each answer lands on screen as it finishes instead of waiting on the slowest one.

Switches: routing.maxFanout caps how many OpenRouter models join (default 3), and each model record has an askAll flag to opt in or out.

There is no voting here. It is parallel sampling. You and your agent (Claude Code, Codex, or anything else) get all the answers and decide.

2. The consensus loop, where the voting lives

This is the heart of it, and it is a state machine, not a vibe. One round goes:

Blind pre-commit. The arbiter writes down its own verdict - approve, request changes, or reject - before any other model sees the work. In writing, first. So its judgment cannot quietly drift to match the crowd later.
Parallel peer review. The panel reviews the same artifact, each in a fresh thread, none seeing another's review.
Blind cross-review. Each model then rates the others' answers with names stripped off (to avoid bias and deference). A "not viable" vote becomes a candidate problem the arbiter has to deal with. This catches the case where everyone looked like they agreed but were each walking past the same hole. (Pattern borrowed from karpathy/llm-council.)
Adjudication. The arbiter goes through every objection and accepts it, dismisses it with a written reason, or defers it. Then it revises the artifact and the round runs again.

It converges only when every responding model approves and the arbiter's pre-committed verdict agrees with them. If they cannot get there inside the round cap, it returns UNRESOLVED and says so. It does not fake a number to look finished.

Click for the detailed diagram with the bias guards and per-model flow.

The answers are not free text the arbiter has to guess at. Every voice returns a structured opinion, and the engine parses it into fixed fields:

a recommendation (the actual call),
a confidence label, so a weak "maybe" does not count the same as a firm "yes",
dissent points, assumptions, and tradeoffs as separate lists, so a disagreement is attached to a reason instead of buried in prose.

Reviews add a verdict (approve / request changes / reject) and a list of critical issues, each sorted into a closed six-category taxonomy - so "three reviewers, nine objections" collapses into a clean, deduplicated set the arbiter can act on. Models are asked to emit this as JSON; the parser is best-effort and never throws, so if a model returns slightly malformed JSON or plain text, the content is salvaged and tagged with how it was read (clean parse vs recovered). Structured where it can be, never brittle.

Switches: consensus.arbiter (auto, the host model, a named provider, or a dedicated model record), consensus.blindVote (add the blind pre-vote on the synthesis path), consensus.maxRounds (default 5, capped at 50), and a per-model consensus flag for who sits on the panel.

So there are four distinct votes in play: the arbiter's blind pre-commit, each peer's independent verdict, the anonymized cross-review rating, and the final convergence check. They are doing different jobs on purpose.

One honest note, since a reviewer pushed me on it: that convergence rule is a heuristic, not a proof. Unanimous approval means nobody in the room objected. It does not mean the answer is right. More on that below.

3. Synthesis, when there is no verdict to give

Not everything is approve-or-reject. "Which of these two designs should we pick?" has no yes or no. Flip synthesizeAlways: true and instead of the loop you get a single arbiter pass that reads every opinion and writes one combined answer - free text, no verdict, no rounds. Use the loop for go/no-go calls. Use synthesis for open questions. Same panel, two shapes.

4. Two drivers, one rulebook

The loop logic lives in one place - a single state machine - and two things drive it.

consensus runs the whole loop server-side with a model as the arbiter and hands you back the result in one call. consensus-step lets your own agent be the arbiter and drive the loop one step at a time, so every move shows up in your transcript where you can see it.

Two entry points is more surface area than one, and they do not behave identically - one is visible step by step, one is a single call. The win is that the rules (how rounds count, when it converges) are written once and shared, so the two paths cannot drift apart on the part that matters. That was worth the extra work.

5. Make the panel stay accountable

In a parallel fan-out, your wall-clock time is the slowest model, not the average. One slow member that rarely says anything new sets the clock for everyone. You will not notice by feel. You need numbers.

So there is an opt-in debug log. Turn on debug.enabled and it writes one line per model call and per round: latency (p50, p95, max), token counts, error rate, the reasoning effort used. It never records your prompts, the responses, or the issue text - only the timing and the outcome of each vote.

Turn on sessions.persist and runs are saved too, so you can ask how often a given model's verdict matched the final call - and, just as useful, how often it was the lone voice that caught something the others missed, or the lone voice that was simply wrong. A model that always agrees adds cost; a model that disagrees and is usually right is the one you keep.

Then an analyze tool reads both back and tells you, in plain terms, who is slow, who is redundant, and which config line to change. From my own panel last week: one model sat at a 200-second p95 while another finished in 15 (Grok 4.3 is often the fastest). analyze flagged it and suggested the one-line switch to drop it from the default fan-out. I had been waiting on that model for almost a week without realizing it.

There is also a small dedup cache, so an identical advisory question inside one session returns instantly instead of paying twice. Well, in tool-heavy work the prompts vary, so it hits less than you would hope.

This is the whole shift from the first post. Consensus was the idea. Making the panel prove it earns its seat is the engineering!

The parts I will not oversell

Multi-model review has real failure modes, and a post that hides them deserves the distrust it gets. So:

Agreement is not truth. Three models can be confidently, unanimously wrong - especially when they were trained on overlapping data and share the same blind spot. Consensus lowers the odds of a lone model's odd mistake. It does not discover facts. Treat it as risk reduction, not a truth oracle.
Pick models that actually differ. A panel of three close cousins is one model agreeing with itself in three accents. The value comes from genuinely different models arguing, which is exactly why the config lets you choose them by hand.
UNRESOLVED is a feature. When the room cannot agree, the honest output is "we could not." That is a signal to slow down and look, not a bug to smooth out. If you wire this into CI, decide on purpose what a deadlock means there - it should probably stop the line, not wave it through.
It is slow, and that is fine for the right job. A full consensus loop can take minutes. Run it on plans, designs, and reviews in the background. For a fast second opinion, use single-shot fan-out or synthesis. Do not put a five-round loop in an interactive path.
Mind where the text goes. Fanning a compliance artifact out to 400 third-party models is a data-boundary decision, not a free upgrade. The config defaults the long tail to off for exactly this reason. Turn models on deliberately.

None of this is specific to Terraform, AWS, or even to code. The loop runs on anything you can put in text - a plan, a runbook, a decision memo. That generality is the point. It just happens to have been built running a compliance product, which is a good place to learn that "looks done" and "is done" are different claims.

The takeaway

The first lesson was: do not trust one confident model, make a few argue. The second one is harder and more useful: adding voices is easy, and most of them will not earn their seat. Measure the panel like you would measure any other service or people, and cut what does not pay.

Deliberation is open source, it works in the agent you already use, and most of the models are ones you already pay for. It ships in the agent-plugins marketplace and as a standalone MCP server on npm (@antonbabenko/deliberation-mcp). Source, and a star button I would genuinely appreciate, are at github.com/antonbabenko/deliberation. Install it, point GPT and Claude at each other, and let me know where it breaks. The cheapest review still happens before you execute - just make sure every reviewer in the room is worth the wait and the cost.

AWS Lambda Managed Instances with Java 25 and AWS SAM – Part 7 Implement scheduled scaling

Vadym Kazulkin — Mon, 01 Jun 2026 15:36:09 +0000

Implement scheduled scaling

Recently, Amazon EventBridge Scheduler added 619 new SDK API actions. One of these actions adjusts the Lambda function's minimum and maximum execution environments for Lambda Managed Instances (LMI) on a recurring or one-time schedule. This is useful for predictable traffic patterns, such as scaling up before peak hours and scaling down during off-peak hours. We can use CloudFormation or the latest versions of AWS CDK or AWS CLI to perform this action. In our example, we'll use AWS CLI to adjust the Lambda Function Scaling Configuration, for which we'll use the PutFunctionScalingConfig API as a universal target. We'll use the Lambda function with the name GetProductByIdJava25WithLMI, which we introduced in part 1.

First of all, let's create an SQS dead-letter queue, which we'll use for our action:

aws sqs create-queue --queue-name scheduler-dlq

Next, let's create an EventBridge Scheduler IAM execution role with the name scale-lambda-managed-instances-eventbridge-scheduler-role. This role grants permission to call the lambda:PutFunctionScalingConfig and send the message to the dead-letter queue on our target function.

This is how the trusted policy looks for the IAM role:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "scheduler.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

And this is how the IAM policy looks (use the ARNs of the Lambda function and SQS dead-letter queue here):

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "lambda:PutFunctionScalingConfig"
            ],
            "Effect": "Allow",
            "Resource": "arn:aws:lambda:{aws_region}:{aws_account_id}:function:GetProductByIdJava25WithLMI:$LATEST.PUBLISHED"
        },
        {
            "Effect": "Allow",
            "Action": [
                "sqs:SendMessage"
            ],
            "Resource": [
                "arn:aws:sqs:{aws_region}:{aws_account_id}:scheduler-dlq"
            ]
        }
    ]
}

Please replace the values of {aws_region} and {aws_account_id} with your own values, and adjust the Lambda function and SQS Queue names if needed.

Now, let's create the scheduler:

aws scheduler create-schedule \
  --name "ScaleLambdaManagedInstances" \
  --schedule-expression "at(2026-05-18T08:10:00)" \
  --flexible-time-window '{"Mode": "OFF"}' \
  --target '{
     {"DeadLetterConfig": {"Arn": "arn:aws:sqs:{aws_region}:{aws_account_id}:scheduler-dlq"},
    "Arn": "arn:aws:scheduler:::aws-sdk:lambda:PutFunctionScalingConfig",
    "RoleArn": "arn:aws:iam::{aws_account_id}:role/scale-lambda-managed-instances-eventbridge-scheduler-role",
    "Input": "{\"FunctionName\": \"GetProductByIdJava25WithLMI\", \"Qualifier\": \"$LATEST.PUBLISHED\", \"FunctionScalingConfig\": {\"MinExecutionEnvironments\": 5, \"MaxExecutionEnvironments\": 10}}"
  }'

Let's explain what happens here:

First, we create the schedule with the name ScaleLambdaManagedInstances using a one-time schedule (executes at 08:10 on May 18, 2026). You can use other Schedule types in EventBridge Scheduler, like cron-based expressions.
Then, we target the PutFunctionScalingConfig Scheduler API as a universal target.
Next, we specify the SQS dead-letter queue ARN created above
Then, we specify the IAM execution Role ARN created above
Next, we specify the new MinExecutionEnvironments and MaxExecutionEnvironments values in the Input payload
Finally, as the scheduler input, we specify the name of the Lambda function and its qualifier (usually $LATEST.PUBLISHED), for which we'd like to change the Function Scaling Configuration. In our case, we set MinExecutionEnvironments to 5 and MaxExecutionEnvironments to 10.
We can also optionally set the retry policy and encryption.

After creating the schedule, we'll see something similar in the Amazon EventBridge Scheduler Service:

After the scheduler has run, we can verify that the Lambda function Scaling Configuration has changed:

It's also very important to configure the dead-letter queue. First, I didn't do it, and configured the Lambda function Resource ARN in the IAM policy like arn:aws:lambda:{aws_region}:{aws_account_id}:function:GetProductByIdJava25WithLMI. I observed that the scheduler hasn't been invoked and saw the errors in Amazon CloudWatch.
By configuring the dead-letter queue, I saw the exact error message:

ERROR_CODE: AccessDeniedException
ERROR_MESSAGE :

User: arn:aws:sts::{aws_account_id}:assumed-role/scale-lambda-managed-instances-eventbridge-scheduler-role/f375e2c757da339a8d593587ce800265 
is not authorized to perform: lambda:PutFunctionScalingConfig on resource: 
arn:aws:lambda:{aws_region}:{aws_account_id}:function:GetProductByIdJava25WithLMI:$LATEST.PUBLISHED 
because no identity-based policy allows the lambda:PutFunctionScalingConfig action

With that, it was clear to me that I needed to append $LATEST.PUBLISHED to the ARN of the Lambda function. Please also use Troubleshooting Amazon EventBridge Scheduler, in case you experience some issues.

Here, we scaled up the capacity at the given time, the same way we can scale it down. Things to pay attention to:

For workloads with predictable peaks, create multiple schedules to match your traffic pattern: one to scale up your function before peak hours, and another to scale down after peak hours. Each schedule follows the same pattern with updated MinExecutionEnvironments and MaxExecutionEnvironments values.
Scheduled scaling adjusts the provisioned floor and ceiling of execution environments, but actual scaling between min and max still responds to CPU utilization and concurrency saturation.
If your traffic more than doubles within 5 minutes of a scheduled scale-up, you might still experience throttling as capacity is provisioned.
When scaling to zero to deactivate a function, remember that reactivation requires an explicit PutFunctionScalingConfig call with non-zero values.

Lambda Managed Instances with Terraform: Multi-Concurrency, High Memory, and Compute Options

Darryl Ruggles — Fri, 29 May 2026 23:45:10 +0000

Lambda has always been one request at a time per execution environment. Your function starts, processes a single invocation, and sits idle until the next one arrives. If you need to handle a thousand concurrent requests, Lambda spins up a thousand execution environments - each with its own memory, its own cold start, and its own per-GB-second bill.

Lambda Managed Instances changes that model. Announced at re:Invent 2025 and expanded with 32 GB memory / 16 vCPU support in March 2026, LMI runs your functions on EC2 instances in your VPC with AWS handling provisioning, patching, scaling, and load balancing. Each execution environment handles multiple concurrent requests. You keep the Lambda programming model and gain EC2 hardware selection and pricing.

I built a product similarity engine to explore how this works in practice. The handler loads a product catalog with Nova embeddings via Bedrock into memory, uses Amazon Nova Multimodal Embeddings to embed incoming search queries, and computes cosine similarity across categories in parallel using ThreadPoolExecutor. It's the kind of workload that doesn't fit well on standard Lambda - sustained throughput, memory-intensive, with a mix of I/O (Bedrock API calls) and CPU (vector math) that benefits from multi-concurrency and configurable memory-to-vCPU ratios. The project uses Terraform for infrastructure, Python 3.14 with Powertools for observability, and the embedding model is configurable (Nova by default, Titan Text Embeddings V2 as an alternative).

The source code is on GitHub: lambda-managed-instances-similarity-engine

The AWS Compute Continuum

Before diving into the implementation, it helps to understand where Lambda Managed Instances fits in the AWS compute landscape. The options form a continuum from fully managed to fully self-managed:

	Standard Lambda	Lambda Managed Instances	ECS Express Mode	ECS Fargate	EKS
Scaling	Per-invocation, instant	Async, CPU-based and concurrency saturation	Traffic-based, auto	Task-based, minutes	Pod-based, minutes
Concurrency	1 per environment	Multiple per environment	Configurable	Configurable	Configurable
Pricing	Per-request + GB-second	Per-request + EC2 + 15% mgmt fee	Fargate + ALB	Per-vCPU-hour	EC2/Fargate + control plane
Commitment discounts	None	Savings Plans, Reserved Instances	Fargate Savings Plans	Fargate Savings Plans	EC2 Savings, RIs
Cold start	Milliseconds-seconds	Tens of seconds (new instances)	Minutes	Minutes	Minutes
Max invocation	15 minutes	15 minutes (environments long-lived, instances rotated by Lambda)	No limit	No limit	No limit
VPC	Optional	Required	Required	Required	Required
Memory	Up to 10 GB	Up to 32 GB (configurable vCPU ratio)	Configurable	Configurable	Configurable
Ops burden	Zero	Low	Low	Medium	High

When to choose Lambda Managed Instances:

Sustained, predictable throughput (hundreds or thousands of requests per second)
Workloads that benefit from specific EC2 instance types (Graviton4, high-bandwidth networking)
Memory-intensive functions that exceed standard Lambda's 10 GB limit or need configurable memory-to-vCPU ratios
Cost optimization at scale (10M+ invocations/month where EC2 pricing with Savings Plans beats per-GB-second)
Functions that load large datasets into memory and reuse them across requests (embeddings, models, reference data)

When standard Lambda is still better:

Bursty, unpredictable traffic patterns
Low to moderate throughput (standard Lambda's per-invocation pricing wins)
Functions that need instant scaling (LMI scales asynchronously based on CPU utilization and execution-environment saturation; if traffic more than doubles within 5 minutes you may see throttles while capacity catches up)

I've written about several of these compute options in previous projects. My ECS deep dive covers Fargate and ECS Express Mode. The Serverless Data Processor demonstrates Step Functions with both Lambda and Fargate. My Powertools best practices article covers the observability patterns used in this project.

Architecture

The architecture has three layers:

Capacity Provider - The foundation. Defines the VPC configuration, instance requirements (architecture, instance types), and scaling policies. Capacity providers define both the security boundary and the failure blast radius of your workload. All functions assigned to the same capacity provider share EC2 instances and must be mutually trusted. This uses container-based isolation, not Firecracker. A compromised function on a shared capacity provider can affect every other function on the same instances. Separate untrusted workloads, regulated workloads, and production from non-production into distinct capacity providers.

Managed Instances - EC2 instances launched and managed by Lambda in your VPC. They're visible in the EC2 console (tagged as managed by Lambda) but you don't SSH into them, patch them, or configure autoscaling groups - Lambda handles all of that. The lifecycle includes a 14-day rotation for security compliance.

Execution Environments - Containers running your function code on the managed instances. Each environment handles multiple concurrent requests. For Python, each concurrency slot is a separate process with its own memory space.

Networking - VPC connectivity is mandatory. Without proper outbound connectivity, functions execute but logs and traces are silently lost. This project uses private subnets with a NAT Gateway for telemetry transmission and Bedrock API access. For production, consider VPC endpoints to keep traffic on the AWS network.

Two-Level Concurrency

This is what makes Lambda Managed Instances architecturally different from standard Lambda. There are two levels of parallelism:

Level 1 - LMI manages for you: Multiple processes handle concurrent requests. Python's LMI runtime spawns a separate process for each concurrency slot (default: 16 per vCPU). Each process has its own memory space, its own global variables, and its own boto3 clients. No shared mutable state between processes. Scaling decisions are based on both execution environment saturation and CPU utilization, not request count alone.

Level 2 - You manage yourself: Within each request, you can use ThreadPoolExecutor to parallelize I/O operations. If your handler needs to search 5 product categories, you can search them in parallel rather than sequentially.

Combined, this means a single execution environment with 1 vCPU and 10 concurrent processes, each running 4 search threads, can have 40 category searches in flight concurrently. On standard Lambda, you'd need 10 separate execution environments to handle those 10 concurrent requests, each paying per-GB-second for its own copy of the catalog in memory.

Each process receives a request, calls Bedrock to embed the query text, then fans out across categories using ThreadPoolExecutor. The catalog data (loaded from DynamoDB at process init) stays in memory across all requests handled by that process.

Why LMI Instead of Standard Lambda

This workload is a poor fit for standard Lambda and a strong fit for LMI. Here's why:

In-memory catalog at scale. Each process loads the product catalog with embedding vectors into memory at initialization. A 100K product catalog with 384-dimensional vectors is roughly 150 MB per process. With 10 concurrent processes, that's 1.5 GB for catalog data alone. Standard Lambda's maximum is 10 GB total, and you pay per-GB-second for every millisecond of that memory. LMI gives you up to 32 GB with configurable memory-to-vCPU ratios, and you pay EC2 instance pricing regardless of how much memory your function uses.

Multi-concurrency amortizes catalog loading. On standard Lambda, 10 concurrent requests means 10 independent execution environments, each cold-starting and loading the catalog into its own memory, each paying per-GB-second. On LMI, those 10 requests run as 10 processes on one EC2 instance. The catalog loads once per process at init time and stays warm for all subsequent requests routed to that process. At sustained throughput, this eliminates the repeated cold-start penalty.

Sustained throughput economics. A product recommendation API serving a storefront has predictable, sustained traffic - hundreds of requests per second during business hours. Each request involves a Bedrock API call for query embedding (I/O), cosine similarity across categories (CPU), and structured logging (I/O). At 10M+ invocations per month, EC2 pricing with Savings Plans is 60-72% cheaper than standard Lambda's per-GB-second model.

Configurable memory-to-vCPU ratio. This workload is memory-heavy (large catalog) with moderate CPU needs (vector math on 384 dimensions). The 4:1 memory-to-vCPU ratio gives 4 GB of memory per vCPU - enough for the catalog plus Bedrock client overhead. Standard Lambda locks you into a fixed ratio where more memory always means proportionally more CPU and higher cost.

Why Not Fargate?

This project could run on ECS Fargate. The handler logic would move into a FastAPI app, the catalog would load at container startup, and an ALB would handle routing. It would work fine. But the infrastructure footprint would be significantly larger:

	Lambda Managed Instances	ECS Fargate
Application code	Single handler function	Web framework + Dockerfile + health checks
Infrastructure	Capacity provider + function	Cluster + task def + service + ALB + target group + listener rules
Auto-scaling	Built into capacity provider	Application Auto Scaling policies (target tracking, step scaling)
Event triggers	Native (SQS, EventBridge, API Gateway, S3)	Requires separate wiring per trigger
Terraform lines	~200 across 4 modules	~400-500 with ALB, ECR, auto-scaling
Container image	Not needed (zip deployment)	Required (Dockerfile, ECR push, image lifecycle)

For teams already comfortable with Lambda, LMI is the path of least resistance to get EC2 pricing and multi-concurrency without learning container orchestration. You keep the programming model you know and gain the hardware flexibility you need. The reverse is also true: for teams already invested in ECS, Fargate may remain the more operationally familiar choice - the muscle memory, dashboards, deployment pipelines, and on-call runbooks are already in place.

Where Fargate or EKS would be the better choice: custom native dependencies that exceed Lambda layer limits (PyTorch, large ML models), persistent connections (WebSocket, gRPC), specialized instance types not supported by LMI, or workloads that need the Kubernetes ecosystem. I covered Fargate patterns in my ECS deep dive and Kabob Store projects. My EKS Auto Mode article covers Karpenter-based scaling.

One specific area where EKS with Karpenter is significantly more sophisticated: scaling down and cost optimization at idle. LMI's scale-down is conservative - in my testing, 2 EC2 instances remained running overnight with zero traffic (1 per AZ). There's no minimum instance setting, no consolidation, and no way to force scale-to-zero short of deleting the function version or capacity provider. Karpenter, by contrast, actively consolidates workloads onto fewer nodes, replaces larger instances with smaller ones when demand drops, and can use Spot instances for fault-tolerant workloads. If your traffic has significant idle periods (nights, weekends), this difference matters for cost. LMI's simplicity comes at the price of less intelligent scaling.

Setting It Up with Terraform

The complete infrastructure is organized into four Terraform modules: networking, IAM, capacity provider, and Lambda. Every IAM policy follows least privilege, and the configuration follows the AWS Well-Architected Framework Security and Cost Optimization pillars. All resources use official HashiCorp providers (hashicorp/aws and hashicorp/archive where applicable) - no community modules or third-party providers.

For a fully production-hardened deployment, you'd also want to address the Reliability, Performance Efficiency, and Operational Excellence pillars more explicitly. The AWS Serverless Applications Lens emphasizes thinking in concurrent requests, sharing nothing, designing for failures and duplicates, and using versions and aliases for safe reversible deployments. Concretely:

Multi-AZ deployment - subnets in at least two AZs (this demo does)
Encryption at rest with customer-managed KMS keys - on the capacity provider (kms_key_arn on aws_lambda_capacity_provider), DynamoDB (server_side_encryption with kms_key_arn), and CloudWatch Logs (kms_key_id on the log group)
VPC endpoints instead of NAT Gateway (covered later in this article)
Invoke through an alias, not the published version directly - The demo invokes the qualified ARN of the published function version (function:name:1). For production, create an alias (prod, live, stable) pointing to a specific version and have callers invoke the alias ARN. Aliases enable instant rollback by updating one pointer, support traffic-shifting deployments (10% to a new version, then 50%, then 100%), and decouple caller code from version numbers.
Idempotency for downstream side effects - Because Lambda may retry or duplicate events, handlers must remain idempotent - even when using long-lived in-memory state. The Powertools idempotency utility uses DynamoDB to deduplicate requests by a configurable key. For this similarity engine the Bedrock embedding call is a read operation and the only state change is logging, so idempotency is less critical. For handlers that write to DynamoDB, send notifications, or charge a payment, idempotency is essential because LMI's at-least-once delivery semantics mean retries can produce duplicate side effects. The in-memory catalog is read-only and shared across requests, but any per-request state that produces side effects needs deduplication.
CloudWatch alarms on LMI-specific metrics (covered in the Observability section)

The demo includes the basics. The production hardening above is straightforward incremental work.

Provider Configuration

terraform {
  required_version = ">= 1.11.0"

  required_providers {
    aws = {
      # Validated with AWS provider v6.x (tested with 6.31+)
      source  = "hashicorp/aws"
      version = "~> 6.31"
    }
    archive = {
      source  = "hashicorp/archive"
      version = "~> 2.7"
    }
  }
}

provider "aws" {
  region  = var.aws_region
  profile = var.aws_profile  # Set via AWS_PROFILE env var or -var flag

  default_tags {
    tags = {
      Project     = "lambda-managed-instances"
      Environment = var.environment
      ManagedBy   = "terraform"
    }
  }
}

The ~> 6.31 constraint pins to the current stable major (6.31.0 at the time of writing) without locking too tightly. memory_size values above 10240 MB require hashicorp/aws 6.29.0 or later - earlier releases had a schema validator that capped memory_size at 10 GB even for LMI functions (fixed in #46065). Without a recent provider, attempting to set 16 GB or 32 GB on an LMI function fails at terraform plan with a confusing validation error.

IAM: The Two-Role Model

Lambda Managed Instances requires two separate IAM roles - a deliberate separation of concerns:

Operator Role - Allows Lambda to manage EC2 instances on your behalf. Your function code never gets these permissions.

resource "aws_iam_role" "operator" {
  name = "${var.project_name}-${var.environment}-operator"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Effect = "Allow"
      Principal = {
        Service = "lambda.amazonaws.com"
      }
      Action = "sts:AssumeRole"
      Condition = {
        StringEquals = {
          "aws:SourceAccount" = var.account_id
        }
      }
    }]
  })
}

resource "aws_iam_role_policy_attachment" "operator" {
  role       = aws_iam_role.operator.name
  policy_arn = "arn:aws:iam::aws:policy/service-role/AWSLambdaManagedEC2ResourceOperator"
}

Execution Role - Scoped to only what the function needs. No EC2 permissions, no wildcard resources. Bedrock access is limited to specific embedding model families.

# DynamoDB - least privilege: handler only does Query on the category-index GSI.
# The seed script runs locally with the operator's credentials and uses its own
# IAM identity for PutItem - not this role.
resource "aws_iam_role_policy" "execution_dynamodb" {
  name = "dynamodb-access"
  role = aws_iam_role.execution.id

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Effect = "Allow"
      Action = ["dynamodb:Query"]
      Resource = [
        var.dynamodb_table,
        "${var.dynamodb_table}/index/*"
      ]
    }]
  })
}

# Bedrock - scoped to the specific configured embedding model
resource "aws_iam_role_policy" "execution_bedrock" {
  name = "bedrock-embeddings"
  role = aws_iam_role.execution.id

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Effect   = "Allow"
      Action   = ["bedrock:InvokeModel"]
      Resource = [
        "arn:aws:bedrock:${var.aws_region}::foundation-model/${var.embedding_model_id}"
      ]
    }]
  })
}

The runtime function only needs dynamodb:Query because _load_catalog() queries the category-index GSI rather than scanning the table. No PutItem, no GetItem, no Scan. The seed script (scripts/seed_catalog.py) runs locally on the developer's machine with their own IAM identity - it never assumes the function execution role, so the runtime role doesn't need write permissions. The Bedrock policy is scoped to the exact model ARN configured via var.embedding_model_id, not a wildcard. This is what "least privilege" looks like when you actually walk through the code.

Capacity Provider

The capacity provider defines the EC2 infrastructure where your functions run. The scaling_mode = "Manual" with a target CPU utilization policy gives you control over scaling behavior while still letting Lambda handle the mechanics.

resource "aws_lambda_capacity_provider" "main" {
  name = "${var.project_name}-${var.environment}"

  vpc_config {
    subnet_ids         = var.subnet_ids
    security_group_ids = [var.security_group_id]
  }

  permissions_config {
    capacity_provider_operator_role_arn = var.operator_role_arn
  }

  instance_requirements {
    architectures = [var.instance_architecture]  # "arm64" for Graviton
  }

  capacity_provider_scaling_config {
    max_vcpu_count = var.max_vcpu_count
    scaling_mode   = "Manual"

    scaling_policies = [{
      predefined_metric_type = "LambdaCapacityProviderAverageCPUUtilization"
      target_value           = var.target_cpu_utilization
    }]
  }
}

The capacity provider supports two scaling modes: Auto and Manual. Auto mode is hands-off - Lambda picks an internal target CPU utilization and scales based on AWS-chosen defaults, with no explicit scaling_policies block needed. I chose Manual mode for this project because it lets me set an explicit target (50% in the demo config) so the scaling behavior is predictable and tunable. With a lower target, the capacity provider scales out faster and maintains more headroom for traffic bursts. For a production workload where you trust AWS to pick reasonable defaults, Auto mode is simpler and a valid choice.

Lambda Function with Capacity Provider

Four key differences from a standard Lambda function:

capacity_provider_config attaches the function to LMI
publish = true is required - LMI runs on published versions
memory_size minimum is 2048 MB (2 GB / 1 vCPU)
execution_environment_memory_gib_per_vcpu controls the memory-to-vCPU ratio (new in March 2026)

locals {
  powertools_layer_arn = (
    var.instance_architecture == "arm64"
    ? "arn:aws:lambda:${var.aws_region}:017000801446:layer:AWSLambdaPowertoolsPythonV3-python314-arm64:${var.powertools_layer_version}"
    : "arn:aws:lambda:${var.aws_region}:017000801446:layer:AWSLambdaPowertoolsPythonV3-python314-x86_64:${var.powertools_layer_version}"
  )

  powertools_env_vars = {
    POWERTOOLS_SERVICE_NAME      = "${var.project_name}-handler"
    POWERTOOLS_METRICS_NAMESPACE = var.metrics_namespace
    POWERTOOLS_LOG_LEVEL         = var.log_level
  }
}

resource "aws_lambda_function" "handler" {
  function_name = "${var.project_name}-${var.environment}-handler"
  role          = var.execution_role_arn
  handler       = "handler.lambda_handler"
  runtime       = "python3.14"
  architectures = [var.instance_architecture]
  memory_size   = var.lambda_memory_size
  timeout       = 30
  publish       = true

  filename         = data.archive_file.function.output_path
  source_code_hash = data.archive_file.function.output_base64sha256

  layers = [local.powertools_layer_arn]

  capacity_provider_config {
    lambda_managed_instances_capacity_provider_config {
      capacity_provider_arn                    = var.capacity_provider_arn
      execution_environment_memory_gib_per_vcpu = var.memory_gib_per_vcpu
      per_execution_environment_max_concurrency = var.max_concurrency_per_environment
    }
  }

  logging_config {
    log_format            = "JSON"
    application_log_level = var.log_level
    system_log_level      = "WARN"
    log_group             = aws_cloudwatch_log_group.function.name
  }

  tracing_config {
    mode = "Active"
  }

  environment {
    variables = merge(local.powertools_env_vars, {
      DYNAMODB_TABLE      = aws_dynamodb_table.products.name
      ENVIRONMENT         = var.environment
      EMBEDDING_MODEL_ID  = var.embedding_model_id
      EMBEDDING_DIMENSION = tostring(var.embedding_dimension)
    })
  }
}

The memory_gib_per_vcpu setting is powerful. LMI enforces a minimum of 1 vCPU per execution environment, so the ratio determines how much memory you get for that minimum. Examples at the 8 GB level:

2:1 ratio = 8 GB / 4 vCPUs (compute-heavy: batch processing, data crunching)
4:1 ratio = 8 GB / 2 vCPUs (balanced: API handlers, typical workloads)
8:1 ratio = 8 GB / 1 vCPU (memory-heavy: large in-memory datasets, caching)

The product similarity engine uses 4 GB at 4:1 - 1 vCPU per environment, which is the smallest balanced configuration that fits the catalog plus 10 worker processes.

A Note on Packaging Dependencies

The Powertools layer is pinned to a specific version (minimum 3.23.0 - the first release that officially supports LMI). For everything else, follow AWS's guidance for Python Lambda functions: package all dependencies, including boto3 and botocore, with the function rather than relying on the runtime's bundled copies. Even though boto3 is available in the runtime, package it with your function to avoid version drift. The runtime's boto3 is updated on AWS's schedule, not yours, and version drift between local development and the runtime can produce subtle bugs that are hard to reproduce. For production zip deployments, pip install --target build/ boto3 botocore and ship them in the zip. The demo uses the runtime's boto3 for simplicity, but production code should not.

Multi-Concurrency by Language

LMI supports five runtimes today: Python 3.13+, Node.js 22+, Java 21+, .NET 8+, and Rust on the OS-only runtime. All modern runtimes (Python 3.12+) are based on Amazon Linux 2023, replacing AL2 ahead of its June 2026 end-of-life. Every language handles multi-concurrency differently, and the differences matter - they change how you write the handler, what concurrency bugs you have to worry about, and how memory scales.

Runtime	Concurrency Model	What This Means for Your Handler
Python	Multiple processes per environment	Full isolation - each process has its own memory, globals, and boto3 clients. No thread-safety concerns. Memory multiplies linearly with concurrency.
Node.js	Worker threads with async dispatch	Each worker thread can also handle async requests concurrently. Requires safe handling of shared state.
Java	Single process with OS threads	Multiple threads execute the handler simultaneously in shared memory. Requires explicit thread-safe code: synchronized collections, no shared mutable state, atomic operations. The hardest model to get right.
.NET	.NET Tasks with async processing	Same patterns as ASP.NET Core - thread-safe data structures, no static mutable state.
Rust	Single process, Tokio async tasks	Compile-time enforcement: handlers must implement `Clone + Send`. The compiler catches concurrency bugs that other languages catch at runtime (or in production).

Python is the simplest model because there's no shared memory between concurrent requests. The trade-off is per-process memory multiplication. Java is the hardest because thread safety becomes a concern on every line that touches shared state. Rust is the safest because the compiler refuses to let you write non-thread-safe code in the first place.

This blog focuses on the Python implementation. The patterns shown here (process isolation, ThreadPoolExecutor for parallel I/O within a request, memory tuning around per_execution_environment_max_concurrency) are specific to how Python's LMI runtime works. The architecture concepts (capacity providers, scaling, networking, IAM) apply identically across all five languages, but the handler code patterns would differ if you were writing in Java or Rust.

The Python Handler

Python's LMI runtime uses multiple processes (not threads) for multi-concurrency. Each concurrent request runs in a separate process with its own memory space. Global variables, module-level caches, and boto3 clients are completely isolated between processes. This is simpler than the thread-based and async models above because there are no shared-memory concurrency concerns.

This blog uses Python 3.14, the newest supported version. Note that Lambda's Python 3.14 ships with the JIT and free-threaded mode disabled, so the GIL is still in effect.

The one shared resource: /tmp. All processes in an execution environment share the same /tmp directory. Use request-scoped filenames to prevent collisions.

Handler Structure with Powertools

Following the Powertools best practices pattern - Logger, Tracer, and Metrics decorators in the correct order:

import json
import math
import os
from concurrent.futures import ThreadPoolExecutor, as_completed
import boto3
from aws_lambda_powertools import Logger, Metrics, Tracer
from aws_lambda_powertools.metrics import MetricUnit
from aws_lambda_powertools.utilities.typing import LambdaContext

logger = Logger()
tracer = Tracer()
metrics = Metrics()

# Module-level init runs ONCE PER PROCESS.
# With 10 concurrent processes, this runs 10 times.
# Each process loads its own catalog copy and boto3 clients.
PROCESS_ID = os.getpid()
AWS_REGION = os.environ.get("AWS_REGION", "us-east-1")
EMBEDDING_MODEL_ID = os.environ.get(
    "EMBEDDING_MODEL_ID", "amazon.nova-2-multimodal-embeddings-v1:0"
)
EMBEDDING_DIMENSION = int(os.environ.get("EMBEDDING_DIMENSION", "384"))

dynamodb = boto3.resource("dynamodb", region_name=AWS_REGION)
table = dynamodb.Table(os.environ["DYNAMODB_TABLE"])
bedrock_runtime = boto3.client("bedrock-runtime", region_name=AWS_REGION)
_catalog: dict[str, list[dict]] = {}


@tracer.capture_method
def _load_catalog() -> None:
    """Load product catalog once per process. Uses Query (least privilege)."""
    if _catalog:  # already loaded in this process
        return
    # ... query DynamoDB by category and populate _catalog ...


@logger.inject_lambda_context(log_event=True)
@tracer.capture_lambda_handler
@metrics.log_metrics(capture_cold_start_metric=True)
def lambda_handler(event: dict, context: LambdaContext) -> dict:
    logger.append_keys(process_id=PROCESS_ID)
    _load_catalog()  # no-op after first call in this process

    # Extract params from event body or direct invocation
    body = event.get("body")
    params = json.loads(body) if isinstance(body, str) else (body or event)
    top_k = int(params.get("top_k", 5))

    # Step 1: Embed the query text via Bedrock (I/O-bound)
    query_embedding = _embed_query(params["query"])

    # Step 2: Search categories in parallel (CPU-bound)
    results: dict = {}
    categories = params.get("categories", list(_catalog.keys()))
    with ThreadPoolExecutor(max_workers=4) as executor:
        futures = {
            executor.submit(_search_category, cat, query_embedding, top_k): cat
            for cat in categories
        }
        for future in as_completed(futures):
            results[futures[future]] = future.result()

    metrics.add_metric(name="SearchRequests", unit=MetricUnit.Count, value=1)
    return {"statusCode": 200, "body": json.dumps({"results": results})}

Bedrock Embedding - Configurable Model

The query text is embedded via Amazon Bedrock before similarity search. The model is configurable via the EMBEDDING_MODEL_ID environment variable - Nova Multimodal Embeddings by default, with Titan Text Embeddings V2 as an alternative:

@tracer.capture_method
def _embed_query_nova(text: str) -> list[float]:
    """Nova Multimodal Embeddings request format."""
    request_body = {
        "taskType": "SINGLE_EMBEDDING",
        "singleEmbeddingParams": {
            "embeddingPurpose": "TEXT_RETRIEVAL",
            "embeddingDimension": EMBEDDING_DIMENSION,
            "text": {"truncationMode": "END", "value": text},
        },
    }
    response = bedrock_runtime.invoke_model(
        body=json.dumps(request_body),
        modelId=EMBEDDING_MODEL_ID,
        accept="application/json",
        contentType="application/json",
    )
    response_body = json.loads(response["body"].read())
    return response_body["embeddings"][0]["embedding"]

Product embeddings are generated at seed time using GENERIC_INDEX purpose and stored in DynamoDB alongside the product data. Query embeddings use TEXT_RETRIEVAL purpose at runtime. Nova supports 4 dimension sizes (256, 384, 1024, 3072) - trading off accuracy against memory and compute cost. The demo uses 384 dimensions as a practical balance.

Cosine Similarity - The CPU Bottleneck

The vector similarity computation is the compute-intensive core after the Bedrock call returns. For production, use NumPy - it's 10-50x faster than a pure Python loop and releases the GIL during C-level operations, which makes the ThreadPoolExecutor pattern actually parallel:

import numpy as np

def _cosine_similarity(query: np.ndarray, catalog: np.ndarray) -> np.ndarray:
    """Production version: batch operation across all products in a category."""
    # query shape: (D,), catalog shape: (N, D)
    norms = np.linalg.norm(catalog, axis=1) * np.linalg.norm(query)
    return np.dot(catalog, query) / np.where(norms == 0, 1, norms)

The pure Python version is included in the demo as an educational fallback (no NumPy dependency, easier to read):

def _cosine_similarity_pure(vec_a: list[float], vec_b: list[float]) -> float:
    """Educational version: shows the math, no dependencies."""
    dot = sum(a * b for a, b in zip(vec_a, vec_b))
    norm_a = math.sqrt(sum(a * a for a in vec_a))
    norm_b = math.sqrt(sum(b * b for b in vec_b))
    if norm_a == 0 or norm_b == 0:
        return 0.0
    return dot / (norm_a * norm_b)

The handler code, the capacity provider, the Terraform - none of it would need to change to run on an instance type with hardware-accelerated vector operations. The capacity provider's instance type selection is the only variable.

Process Memory Multiplication

This is the most important thing to understand about Python LMI. Each process loads its own copy of the catalog:

10 concurrent processes x 200 MB catalog = 2 GB just for catalog data

The MemoryUtilization CloudWatch metric tracks total memory consumption across all processes. If you're loading large datasets and running high concurrency, you'll hit memory limits. Tune with:

Reduce PerExecutionEnvironmentMaxConcurrency (fewer processes, less memory)
Increase memory_size (more memory per environment)
Use 8:1 memory_gib_per_vcpu ratio (more memory, fewer vCPUs)
Use shared /tmp as a cross-process cache (load once, read from all processes)

Observability

LMI publishes its own CloudWatch metrics in the AWS/Lambda namespace at 5-minute granularity. The capacity-provider-level metrics describe overall instance utilization; the execution-environment-level metrics describe per-function resource consumption.

Capacity provider metrics (dimensions: CapacityProviderName, InstanceType):

CPUUtilization - CPU usage across all instances in the capacity provider
MemoryUtilization - Memory usage across all instances
vCPUAllocated / vCPUAvailable - Used vs available vCPU count
MemoryAllocated / MemoryAvailable - Used vs available memory

Execution environment metrics (dimensions: FunctionName, CapacityProviderName, Resource):

ExecutionEnvironmentConcurrency - Active concurrent requests per environment
ExecutionEnvironmentConcurrencyLimit - Configured maximum concurrency per environment
ExecutionEnvironmentCPUUtilization - CPU usage of this function's environments
ExecutionEnvironmentMemoryUtilization - Memory usage of this function's environments

Alarms to Set First

If you only set three alarms when adopting LMI, set these:

Capacity provider CPU utilization - Alarm when sustained CPU exceeds your scaling target (e.g., > 80% for 10 minutes if your target is 50%). This indicates the capacity provider is failing to scale out fast enough or has hit max_vcpu_count.
Execution environment concurrency vs limit - Alarm when ExecutionEnvironmentConcurrency reaches ExecutionEnvironmentConcurrencyLimit for sustained periods. This means processes are saturated and incoming requests are being throttled or queued.
Execution environment memory utilization - Alarm when memory exceeds 80%. With Python's per-process memory multiplication, hitting memory limits causes new process spawns to fail (InitResourceExhausted) rather than gradual degradation. Catch this before it happens.

These three cover the LMI-specific failure modes that standard Lambda alarms (Errors, Throttles, Duration) won't catch.

Deployment

Prerequisites

AWS CLI configured with a profile (export AWS_PROFILE=your-profile)
Terraform >= 1.11
Python 3.14+ with boto3 (for the seed script)
Amazon Nova Multimodal Embeddings model enabled in your AWS account (Bedrock console, Model Access)

Deploy

# Clone the repo
git clone https://github.com/RDarrylR/lambda-managed-instances-similarity-engine.git
cd lambda-managed-instances-similarity-engine

# Configure
cp infrastructure/terraform.tfvars.example infrastructure/terraform.tfvars
# Edit terraform.tfvars with your values

# Deploy infrastructure
make init
make apply

# Seed the product catalog
make seed

# Invoke
make invoke

Cost Analysis

Lambda Managed Instances pricing is fundamentally different from standard Lambda. Understanding when each model wins is the key decision.

Standard Lambda pricing (arm64/Graviton):

$0.20 per million requests
$0.0000133334 per GB-second (arm64)
No minimum charge, no idle cost

Lambda Managed Instances pricing:

$0.20 per million requests (same)
EC2 on-demand instance pricing (varies by type)
15% management fee on the EC2 on-demand price
No per-invocation duration charge

The critical difference: standard Lambda charges per GB-second of execution. LMI charges for EC2 time regardless of how many requests you serve. At low volume, you're paying for idle EC2 capacity. At high volume, that fixed EC2 cost is amortized across millions of requests.

Break-Even: Standard Lambda vs LMI

Consider this workload: 4 GB memory, 200ms average duration, sustained traffic.

Standard Lambda cost per request (arm64):

Compute: 4 GB x 0.2s = 0.8 GB-seconds x $0.0000133334 = $0.00001067
Request: $0.0000002
Total: ~$0.0000109 per request

LMI on a c7g.medium (1 vCPU, 2 GB, ~$0.034/hr on-demand):

EC2 + 15% fee: $0.034 x 1.15 = $0.0391/hr
With 10 concurrent processes and 200ms per request, each process handles ~5 req/sec
Instance throughput: ~50 req/sec = ~180,000 req/hr
Cost per request: $0.0391 / 180,000 = ~$0.000000217

At this throughput, LMI is roughly 50x cheaper per request than standard Lambda. But the EC2 cost runs 24/7 whether you have traffic or not.

Monthly Cost Comparison

Monthly Requests	Instances Needed	Standard Lambda (arm64)	LMI On-Demand (c7g.medium)	LMI + 1yr Savings Plan
1M	1	$11	$28 + $0.20 = $28	~$18
10M	1	$109	$28 + $2.00 = $30	~$20
50M	1	$546	$28 + $10.00 = $38	~$28
100M	1	$1,091	$28 + $20.00 = $48	~$38
500M	4	$5,456	$112 + $100.00 = $212	~$172

A single c7g.medium tops out around ~130M requests/month at 50 req/sec sustained. Beyond that, instance count scales roughly linearly with load - 500M req/month requires approximately 4 instances. The LMI columns reflect the actual instance count needed at each volume.

The break-even is around 2.5M requests/month at this memory and duration profile. Below that, standard Lambda wins because you pay nothing when idle. Above that, LMI wins and the advantage grows with volume.

Commitment Discounts Change the Math

LMI supports EC2 Savings Plans and Reserved Instances. Standard Lambda supports Compute Savings Plans (up to 17% discount on duration). The discount gap is significant:

Commitment	Standard Lambda Discount	LMI Discount (EC2)
None (on-demand)	0%	0%
1-year Compute Savings Plan	Up to 17%	Up to 36%
3-year Compute Savings Plan	Up to 17%	Up to 56%
1-year EC2 Reserved Instance	N/A	Up to 40%
3-year EC2 Reserved Instance	N/A	Up to 60%

For predictable production workloads with steady traffic, a 3-year commitment on LMI can reduce costs by 60% on the EC2 portion. Standard Lambda's maximum discount is 17%. This difference widens the gap at scale.

Hidden Costs

Don't forget the supporting infrastructure that LMI requires and standard Lambda doesn't:

NAT Gateway: ~$32/month + $0.045/GB data transfer (required for VPC telemetry)
VPC endpoints (if used instead of NAT): ~$7.20/month per endpoint per AZ
DynamoDB: On-demand reads for catalog loading (minimal for small catalogs, significant at scale)
Bedrock: Nova Multimodal Embeddings per-token pricing for each query embedding
CloudWatch: Log storage and metric costs increase with concurrency

For low-volume workloads, these fixed costs can exceed the compute savings. Factor them into your total cost of ownership.

When Each Pricing Model Wins

Standard Lambda wins when:

Traffic is bursty or unpredictable (you pay nothing at zero traffic)
Monthly volume is below the break-even threshold (~2-3M requests for this workload profile)
You can't commit to 1-year or 3-year terms
You don't need VPC connectivity (avoids NAT Gateway cost)

LMI wins when:

Traffic is sustained and predictable (the EC2 cost is fully amortized)
Monthly volume exceeds 5-10M requests
You can commit to Savings Plans or Reserved Instances
You need more than 10 GB memory or specific instance types
You're already paying for VPC infrastructure

For this demo, expect to pay for:

NAT Gateway (~$0.045/hour + data transfer)
EC2 instances (varies by type, auto-selected by Lambda)
DynamoDB on-demand reads (minimal for this catalog size)
Bedrock embedding calls (per-token pricing for each query)

CLEANUP (IMPORTANT!!)

This infrastructure costs real money while running - approximately $2-4/day even with zero traffic (NAT Gateway + EC2 managed instances). Don't forget about it.

Make sure to destroy all resources when you're done:

make destroy

If the capacity provider fails to delete (it can take a few minutes to drain instances), wait and retry. Verify in the AWS console that no EC2 instances tagged with your project name are still running.

Networking: Three Supported Patterns

LMI requires VPC connectivity - the function execution environments need outbound network access for telemetry transmission and any AWS service calls. AWS documents three supported connectivity patterns:

Public subnets with an internet gateway - simplest, suitable for dev/test only
Private subnets with NAT Gateway - the pattern this demo uses
Private subnets with VPC endpoints - the most AWS-aligned production pattern

NAT Gateway (used in this demo)

Simple to set up - one resource, all outbound traffic routes through it
~$32/month base + $0.045/GB data transfer
Traffic leaves your VPC, crosses the public internet (encrypted), then re-enters AWS
Single point of failure unless you deploy one per AZ (~$64/month for 2-AZ HA)

VPC Endpoints (recommended for production)

For production, the most AWS-aligned pattern is one VPC endpoint per service per AZ. Traffic stays entirely on the AWS network and never touches the public internet. The endpoint set must cover every service the function calls - if you forget one, the function fails silently or hangs. For this workload, that means:

Endpoint	Type	Required For
`com.amazonaws.{region}.logs`	Interface	CloudWatch Logs (Powertools logger output)
`com.amazonaws.{region}.monitoring`	Interface	CloudWatch Metrics (Powertools metrics)
`com.amazonaws.{region}.xray`	Interface	X-Ray tracing (Powertools tracer)
`com.amazonaws.{region}.bedrock-runtime`	Interface	Bedrock embedding API calls
`com.amazonaws.{region}.dynamodb`	Gateway	DynamoDB catalog queries (free, no per-AZ charge)

Critical security group detail: Interface endpoints have their own security groups. They must allow inbound HTTPS (port 443) from the function's security group. The function security group must allow outbound HTTPS to the endpoint security groups. If you skip this, DNS resolves but the connection is silently blocked.

Endpoints should be deployed in each AZ used by the capacity provider to avoid cross-AZ latency and data transfer costs. If your capacity provider has subnets in us-east-1a and us-east-1b, every interface endpoint also needs ENIs in both AZs. This is the same Cross-AZ Tax pattern from my previous blog - cross-AZ data transfer charges apply when traffic from a function in us-east-1a hits an endpoint ENI in us-east-1b. Provision endpoints per AZ to keep traffic local.

Cost math: ~$7.20/month per interface endpoint per AZ. With 4 interface endpoints across 2 AZs, that's ~$58/month - roughly double the single NAT Gateway, but cheaper than 2-AZ NAT Gateway HA. The DynamoDB gateway endpoint is free. At high data transfer volumes (more than ~900 GB/month through the NAT Gateway), endpoints become cheaper because there's no per-GB data transfer surcharge for in-region traffic.

When endpoints win on security: Always. Traffic never leaves the AWS network. You can attach endpoint policies to restrict which resources each endpoint can access (e.g., limit the Bedrock endpoint to specific model ARNs). This aligns with the AWS Well-Architected Security Pillar - minimize the attack surface.

The Terraform for VPC endpoints is straightforward but verbose. I left it out of this demo to keep the focus on LMI itself. A follow-up project could add a networking_mode variable that switches between NAT Gateway and VPC endpoints.

A few things to watch for:

VPC connectivity isn't optional. Lambda Managed Instances requires a VPC. Without outbound connectivity (NAT Gateway or VPC endpoints), your function executes but logs and traces are silently lost. You'll debug a working function with no visible output. This is documented but easy to miss.
Scaling is asynchronous. LMI scales based on CPU utilization and execution-environment saturation, not per-invocation demand. Unlike standard Lambda, scaling isn't triggered by incoming requests - it's driven by resource consumption inside existing execution environments. Because scaling reacts to resource pressure instead of incoming traffic, inefficient code or high memory usage can delay scaling and increase throttling risk. The Scaler component decides when to add or remove instances, and instance launches aren't instant. Lambda maintains headroom so traffic can roughly double within minutes without immediate throttling, but if your traffic more than doubles within 5 minutes, you may see 429 throttles while capacity catches up. This is fundamentally different from standard Lambda's near-instant scaling. Plan for it with the target CPU utilization setting - lower values maintain more headroom.
Process memory multiplies. With Python, each concurrency slot is a separate process. Because Python uses process-based concurrency, memory usage scales linearly with concurrency - each worker process consumes its own memory. With Python, concurrency isn't "free" - each additional request increases memory consumption linearly. If your function uses 500 MB of memory and you set concurrency to 16, that's 8 GB of memory consumed per execution environment. Monitor the MemoryUtilization metric and tune accordingly.
publish = true is required. LMI runs on published function versions, not $LATEST. If you forget this, Terraform applies successfully but the function doesn't run on managed instances. Every code change needs a new published version.
Capacity providers are security boundaries, not isolation boundaries. Functions sharing a capacity provider run in containers on the same EC2 instances. This isn't Firecracker isolation. Separate untrusted workloads into separate capacity providers.
Powertools minimum version matters. Lambda Managed Instances requires Powertools for AWS Lambda (Python) version 3.23.0 or later. Pin the layer version in Terraform rather than using latest.
LMI doesn't scale to zero. Unlike standard Lambda where you pay nothing at zero traffic, LMI keeps a baseline of warm EC2 instances running for high availability. AWS launches a baseline of three managed instances for availability across AZs when you publish a function version with a capacity provider. In my testing with 2 AZs configured, 2 instances remained active overnight with zero traffic, but the documented baseline is three. There's no minimum instance setting, no Karpenter-style consolidation, and no way to force scale-to-zero short of deleting the function version or capacity provider. This is a meaningful cost difference for dev/test environments where you might leave infrastructure running between sessions. Run make destroy when you're not actively using the infrastructure, or design your dev environments to use standard Lambda where idle cost is zero.
Quotas to plan around. LMI has its own service quotas: 1 request per second on capacity provider write APIs (Create/Update/Delete - rate-limited to prevent infrastructure churn), 100 function versions per capacity provider, and 1,000 capacity providers per account per region. These are soft limits but worth knowing when you start automating capacity provider management or running multiple environments.

SAM Support

If you came in from the AWS Serverless plugin angle and are wondering whether SAM supports LMI - yes, it does. AWS::Serverless::CapacityProvider is the SAM resource equivalent to aws_lambda_capacity_provider. The SAM template syntax is more concise but follows the same model: capacity provider definition, function with CapacityProviderConfig property, and IAM roles. I chose Terraform for this project because the LMI Terraform path is less documented in the wild and I wanted to fill that gap, but SAM is a perfectly valid choice if your team already uses it.

Instance Type Selection

The capacity provider's instance_requirements block controls which EC2 instance types Lambda selects. By default, Lambda chooses the best fit automatically. You can constrain this with allowed_instance_types or excluded_instance_types.

Today, the interesting choice is between arm64 (Graviton4 - better price/performance for most workloads) and x86_64. But the architecture of Lambda Managed Instances - your function code running in containers on EC2 instances you specify - means the compute capabilities available to your functions expand with every new EC2 instance type AWS makes available for LMI.

The product similarity engine in this project calls Bedrock for query embeddings (I/O-bound) and then computes cosine similarity on CPU (compute-bound). The handler code isn't coupled to a specific compute architecture. The embedding call is behind a clean interface (_embed_query). The similarity computation is pure math. The instance type is a configuration parameter, not an application concern.

This is the practical difference between Lambda Managed Instances and standard Lambda. Standard Lambda abstracts the hardware entirely - you get what AWS gives you. Lambda Managed Instances lets you choose, and that choice extends to whatever EC2 instance types AWS makes available.

Wrapping Up

Lambda Managed Instances fills the gap between standard Lambda and ECS Fargate. The handler function and event-driven invocation pattern stay the same, but you gain EC2 hardware selection, multi-concurrency, configurable memory-to-vCPU ratios, and commitment-based pricing.

The key decisions:

Use it for sustained, predictable throughput where EC2 pricing beats per-GB-second
Choose your memory-to-vCPU ratio based on whether your workload is compute-bound or memory-bound
Understand the process model for your language - Python uses processes (simple, no shared-memory concerns), Java uses OS threads (requires thread-safe code), Node.js uses worker threads with async dispatch, .NET uses Tasks, and Rust uses Tokio async tasks (handlers must be Clone + Send)
Monitor MemoryUtilization because process memory multiplies with concurrency

The full Terraform configuration, Python handler, seed script, and Makefile are in the GitHub repository.

Resources

Lambda Managed Instances Documentation
Lambda Managed Instances - Python Runtime Guide
32 GB Memory / 16 vCPU Announcement (March 2026)
Capacity Provider Documentation
Amazon Nova Multimodal Embeddings - Embedding model used in this project
Terraform aws_lambda_capacity_provider
Powertools for AWS Lambda Best Practices - Observability patterns used in this project
Elastic Container Service - My Default Choice for Containers on AWS - ECS Fargate and Express Mode comparison
Serverless Data Processor - Step Functions with Lambda and Fargate

Connect with me on X, Bluesky, LinkedIn, GitHub, Medium, Dev.to, or the AWS Community. Check out more of my projects at darryl-ruggles.cloud and join the Believe In Serverless community.

DEV Community: AWS Heroes

Understanding AWS Blocks as a CDK Developer

Understanding AWS Blocks as a CDK Developer

What Is AWS Blocks?

The Same import Resolves to Different Files Depending on Context

Mechanism: conditional exports + a global variable

Setting Up a Project

Implementing a Sample

Deploying

AWS Blocks from a CDK Perspective

You Can Extend the CDK Definition

Checking the Generated CloudFormation with cdk synth

Integration Patterns

IAM Is Automatic and Least-Privilege per Block

Constraints You Should Know

The Architecture Is a Lambda-lith (All Methods in a Single Lambda)

Side Effects of new

When to Use It

AWS Blocks: Full-Stack Building Blocks That Run Locally Without an AWS Account

Why I built a Custom Kiro Power to ship faster with AWS Blocks

How AWS Blocks works

What changes for teams

What you can build

Teaching your AI agent a new framework

From local to production in two commands

Install Custom Kiro Power for AWS Blocks

Or Try it Manually

Adding Memory to the Agent

Configuring Memory in AgentCore

Integrating Cognito and AgentCore Runtime

Returning Memory records in Handler function

Setting up Memory with Strands

Biography

Serverless applications on AWS with Lambda using Java 25, API Gateway and Aurora DSQL - Lambda performance optimization approaches

Introduction

Lambda performance optimization approaches

Conclusion

Building Real-Time Fraud Detection with Amazon Bedrock and Claude AI - (Let's Build 🏗️ Series)

1- Architecture

2- Flow Step-by-Step

A. Transaction Happens

B. Lambda Trigger

C. Amazon Bedrock (Claude)

D. AI Output Example

E. Action Taken

3- Key Takeaways

Serverless applications on AWS with Lambda using Java 25, API Gateway and DynamoDB - Part 7 Lambda performance optimization approaches

Introduction

Lambda performance optimization approaches

Conclusion

Security for MCP Servers: Governed Access Beats Uploading Spreadsheets to ChatGPT

What Is MCP? (The 30-Second Version)

The Real Alternative Is Not "MCP vs. No MCP"

Security Starts With Tool Design

Typed Inputs Are A Security Boundary

Output Boundaries Matter More Than Most Teams Think

OAuth Matters Because The Server Must Act On Behalf Of The User

Token Validation Is Not Optional Plumbing

Security Happens In Layers

Security Testing Is Another Layer, Not A Final Checkbox

Do Not Throw Away Web Security Lessons

Implementation Note: Rust Helps

The Core Security Argument

The Security Stack In One View

Redshift Check-In: Spring 2026

What Has Changed Since The Early Days?

Serverless Has Become The Default

The Rise Of Zero-ETL

Redshift Is Becoming More AI Native

The Lakehouse Conversation

The Operational Maturity Story

A Few Things To Watch

Final Thoughts

5 Multi-Agent Patterns in Strands Agents: Which One and When

The Running Example: Corporate Travel Agent

Pattern 1: Agents as Tools

How It Works

When to Use It

When NOT to Use It

Pattern 2: Swarm

Side Effects of `new`