DEV Community: Dinesh_gowtham

How to Rotate Claude API Keys with Secrets Manager and Lambda Without Getting Locked Out

Dinesh_gowtham — Fri, 16 Oct 2026 07:27:22 +0000

Developers rush to rotate API keys for security, only to discover their AI‑powered apps suddenly start throwing 401 errors. The hidden culprit? A stale token cached in your Node.js Lambda. This guide shows exactly how to rotate Claude secrets safely, so your code keeps talking to the model without a hiccup.

Why Secret Rotation Matters for LLM APIs

Why rotate?

Any credential that lets your code talk to an LLM (large language model) is a doorway to a costly service. If a key leaks, an attacker can run expensive prompts on your bill. Rotating the key regularly closes that door behind a potential intruder.

Why is rotation tricky with serverless code?

Serverless functions like AWS Lambda keep a small amount of memory alive between invocations. Many developers store a secret in a module‑level variable to avoid a network call on every request. That works—until you change the secret. The function continues to read the old value from its in‑process cache, and the LLM replies with 401 Unauthorized. Because the cache lives only inside the container, the problem disappears only when AWS discards the container (a “cold start”).

In plain English: Think of the cached secret as a sticky note on your desk. If you change the password, the note doesn’t automatically update; you have to throw it away and write a new one.

Key points

Rotation reduces the window an attacker can misuse a leaked key.
Serverless containers cache values for performance, which can hide a rotation.
You need a way to tell the running container, “Hey, the note is stale—refresh it.”

Setting Up Secrets Manager for Claude Tokens

What is Secrets Manager?

AWS Secrets Manager is a fully managed vault that stores secrets (like passwords, API keys, or database credentials) and can rotate them automatically. It offers a simple API (GetSecretValue, PutSecretValue) to read and write secrets.

Create the secret

aws secretsmanager create-secret \
  --name prod/claude/api-key \
  --description "Claude API key for production" \
  --secret-string '{"apiKey":"YOUR_INITIAL_KEY"}'

apiKey is the field that our code will look for. You could store additional metadata (e.g., rotation schedule) in the same JSON object.

Cost awareness (gotchas)

Secrets Manager costs $0.40 per secret per month. 100 micro‑services → $40/month minimum.
Each GetSecretValue call is billed. Calling it on every Lambda invocation can add up quickly.
During a rotation window the secret is temporarily unavailable; your code must handle the brief failure.
If you need to read the secret from another AWS account, you must attach a resource policy to the secret and grant the calling role an IAM policy permission.

Tip: Store the secret in a dedicated “secrets” account and give each micro‑service read‑only access via cross‑account roles. This isolates billing and reduces blast radius.

Lambda Rotation Function – Code Walkthrough

The rotation function does three things:

Generate or fetch a new Claude API key (this example uses a placeholder function).
Write the new key back to Secrets Manager with PutSecretValue.
Publish a custom EventBridge event that tells every service Lambda to discard its cache.

Dependencies

npm install @aws-sdk/client-secrets-manager @aws-sdk/client-eventbridge

Rotation Lambda (TypeScript)

// rotationLambda.ts
import {
  SecretsManagerClient,
  PutSecretValueCommand,
} from "@aws-sdk/client-secrets-manager";
import {
  EventBridgeClient,
  PutEventsCommand,
} from "@aws-sdk/client-eventbridge";

// The ARN of the secret we are rotating
const SECRET_ARN = process.env.CLAUDE_SECRET_ARN!;

// Clients are created outside the handler so they are reused across invocations
const smClient = new SecretsManagerClient({});
const ebClient = new EventBridgeClient({});

export const handler = async (): Promise<void> => {
  // 1️⃣ Get a brand‑new Claude API key.
  // In real life you would call Claude's key‑generation endpoint or a secure vault.
  const newApiKey = await generateNewClaudeKey();

  // 2️⃣ Write the fresh key back to Secrets Manager.
  // The secret value is a JSON string; we keep the same shape as the original.
  const putCmd = new PutSecretValueCommand({
    SecretId: SECRET_ARN,
    SecretString: JSON.stringify({ apiKey: newApiKey }),
  });
  await smClient.send(putCmd);
  console.log("✅ Updated secret with new Claude key");

  // 3️⃣ Tell the rest of the world that the secret changed.
  // We emit a custom EventBridge event named "ClaudeSecretRotated".
  const eventCmd = new PutEventsCommand({
    Entries: [
      {
        Source: "custom.claude.rotation",
        DetailType: "ClaudeSecretRotated",
        Detail: JSON.stringify({ secretArn: SECRET_ARN }),
        EventBusName: "default", // could be a dedicated bus
      },
    ],
  });
  await ebClient.send(eventCmd);
  console.log("🔔 Published rotation event");
};

/**
 * Placeholder for whatever process creates a new Claude key.
 * Replace with real API call or manual step.
 */
async function generateNewClaudeKey(): Promise<string> {
  // Simulate network latency
  await new Promise((res) => setTimeout(res, 200));
  // Return a dummy key – never commit real keys to source control!
  return `claude-${Date.now()}`;
}

Explanation of key lines

SecretsManagerClient and EventBridgeClient are thin wrappers around the AWS SDK that let us talk to the two services.
PutSecretValueCommand overwrites the current secret value with a fresh JSON payload.
PutEventsCommand sends a custom event; any Lambda listening for the ClaudeSecretRotated detail type will receive it.
generateNewClaudeKey is a stub; in production you would replace it with a secure call to Claude’s key‑generation endpoint.

Key takeaway: The rotation Lambda does not need to know about your service’s cache; it simply announces “the secret changed.” The service Lambdas decide how to react.

Integrating Rotation with Your Node.js 22 Service

Your production Lambda that calls Claude (via Bedrock) will:

Load the secret once at cold start.
Store the API key in a module‑level variable (cachedKey).
Subscribe to the custom EventBridge event using the Node.js diagnostics_channel API. When the event arrives, it clears the cache so the next request fetches the fresh key.

Why use `diagnostics_channel`?

diagnostics_channel is a built‑in Node.js feature that lets different parts of a process talk without adding a heavy library. It works even when the Lambda is running in a VPC, where network‑based pub/sub could be slower.

Service Lambda (JavaScript)

// serviceLambda.js
import {
  SecretsManagerClient,
  GetSecretValueCommand,
} from "@aws-sdk/client-secrets-manager";
import { BedrockClient, InvokeModelCommand } from "@aws-sdk/client-bedrock-runtime";
import { createChannel } from "diagnostics_channel";

// The name of the secret that holds the Claude key
const SECRET_NAME = process.env.CLAUDE_SECRET_NAME;

// Clients are reused across invocations for performance
const smClient = new SecretsManagerClient({});
const bedrock = new BedrockClient({});

// Module‑level cache: stays alive as long as the container lives
let cachedKey = null;

/**
 * Pull the API key from Secrets Manager.
 * We call this only when the cache is empty or has been cleared.
 */
async function loadKey() {
  const cmd = new GetSecretValueCommand({ SecretId: SECRET_NAME });
  const resp = await smClient.send(cmd);
  // Secrets Manager returns a JSON string; we parse to get the field we need.
  const secret = JSON.parse(resp.SecretString);
  cachedKey = secret.apiKey;
  console.log("🔑 Loaded fresh Claude key");
}

/**
 * Clear the in‑process cache. The next request will trigger loadKey().
 */
function refreshCache() {
  cachedKey = null;
  console.log("♻️ Cache cleared – will reload on next call");
}

/**
 * Subscribe to the custom EventBridge event that signals rotation.
 * The EventBridge rule forwards events to this Lambda’s /internal channel.
 */
const rotationChannel = createChannel("claude-secret-rotation");
rotationChannel.subscribe((msg) => {
  // Message shape matches what the rotation Lambda published.
  if (msg.secretArn && msg.secretArn.includes(SECRET_NAME)) {
    refreshCache();
  }
});

/**
 * The Lambda handler that receives API Gateway (or other) requests.
 * It ensures we have a fresh key, then calls Claude via Bedrock.
 */
export const handler = async (event) => {
  // If we have no key, fetch it now.
  if (!cachedKey) {
    await loadKey();
  }

  // Build the request payload for Claude (simplified example)
  const prompt = JSON.stringify({ text: "Hello Claude!" });

  const invokeCmd = new InvokeModelCommand({
    ModelId: "anthropic.claude-v2", // example model identifier
    ContentType: "application/json",
    Accept: "application/json",
    Body: Buffer.from(prompt),
    // Pass the API key via custom header if required by your setup.
    // Some Bedrock integrations use the key implicitly; adjust as needed.
    // Here we illustrate adding it as an HTTP header.
    // Note: Bedrock SDK handles signing; this is just an illustration.
    // headers: { "x-api-key": cachedKey },
  });

  const response = await bedrock.send(invokeCmd);
  const result = JSON.parse(Buffer.from(response.Body).toString());

  return {
    statusCode: 200,
    body: JSON.stringify(result),
  };
};

Line‑by‑line notes

cachedKey lives at module scope; it survives between Lambda invocations.
loadKey() calls GetSecretValueCommand once per container lifecycle, avoiding the per‑invocation cost.
rotationChannel.subscribe listens for the custom event. When the secret ARN matches, it empties the cache.
The handler checks if (!cachedKey) before each request; after a rotation event the next request will reload the secret automatically.

Wiring EventBridge to the diagnostics channel

EventBridge can’t talk directly to diagnostics_channel. Instead we create a bridge Lambda that receives the EventBridge event and forwards it to the same process via the channel. Because both the bridge and the service run in the same container (using the same Lambda version), the forward works instantly.

// bridgeLambda.ts
import { createChannel } from "diagnostics_channel";

const rotationChannel = createChannel("claude-secret-rotation");

// This Lambda is triggered by the EventBridge rule that the rotation function emits.
export const handler = async (event: any) => {
  // Forward the event payload to the in‑process channel.
  rotationChannel.publish(event.detail);
  console.log("🚀 Forwarded rotation event to service Lambda");
};

Deploy the bridge Lambda in the same function version (or use Lambda Layers to share the channel code) so that the publish call reaches the service’s subscriber.

Tip: If you prefer not to run a bridge Lambda, you can configure the EventBridge rule to invoke the service Lambda directly with a payload that includes "action":"refreshCache". The service Lambda then checks the payload and clears its cache. The channel approach keeps the logic inside the process and avoids an extra network hop.

Testing, Monitoring, and Avoiding Common Pitfalls

Local testing

Mock Secrets Manager – use the AWS SDK’s MockClient from aws-sdk-client-mock to return a fake secret.
Emit a fake rotation event – call rotationChannel.publish({ secretArn: "arn:aws:secretsmanager:...:secret:prod/claude/api-key" }) and verify that cachedKey becomes null.
Run the handler twice – first invocation should load the key, second should reuse the cache. After publishing the rotation event, a third invocation should reload the key.

// simple test script (run with node)
import { handler as serviceHandler } from "./serviceLambda.js";
import { createChannel } from "diagnostics_channel";

(async () => {
  // First call – loads key
  await serviceHandler({});
  // Second call – uses cache (no log about loading)
  await serviceHandler({});

  // Simulate rotation
  const rotationChannel = createChannel("claude-secret-rotation");
  rotationChannel.publish({ secretArn: process.env.CLAUDE_SECRET_NAME });

  // Third call – should reload
  await serviceHandler({});
})();

Monitoring

CloudWatch Logs – add console.log statements (as in the examples) to track when the key is loaded or the cache is cleared.
Metrics – publish a custom CloudWatch metric (SecretCacheMisses) each time loadKey() runs. A spike indicates frequent rotations or unexpected cache clears.
Alarms – set a threshold on 4XXError for your service Lambda. If you see a burst of 401s after a rotation, your cache‑clear logic likely failed.

Common gotchas and how to dodge them

Gotcha	Why it hurts	Fix
Calling `GetSecretValue` on every request	Each call costs money; you can easily exceed a tiny budget.	Cache the value in a module variable and refresh only on rotation events.
Forgetting cross‑account resource policy	The service Lambda gets `AccessDenied` even though the IAM role allows it.	Add a resource policy to the secret that lists the external account’s role ARN.
EventBridge rule pointing to the wrong version	New code never receives the rotation signal, so cache stays stale.	Pin the rule to the alias or qualifier that maps to the version you’re deploying.
SnapStart + VPC disables the cold‑start speedup	You think you saved money on latency, but the function still pays VPC attachment time.	Disable SnapStart for Lambdas that must attach to a VPC, or move the VPC resources to a private subnet with a NAT gateway.
Provisioned Concurrency left on for idle functions	You pay for reserved capacity even when traffic is zero.	Use on‑demand concurrency and let CloudWatch Auto Scaling adjust based on traffic patterns.
Lambda@Edge response size >1 MB	Edge functions truncate the response, causing mysterious errors.	Keep responses under 1 MB or shift the logic to a regional Lambda.

In plain English: Think of each gotcha as a pothole on the road to smooth secret rotation. Spotting them early keeps your journey (and your bill) safe.

End‑to‑end sanity check

Deploy the secret, service Lambda, rotation Lambda, and bridge Lambda.
Run a test request to the service Lambda – you should see “🔑 Loaded fresh Claude key” once.
Manually invoke the rotation Lambda (or trigger its schedule).
Observe the bridge Lambda forward the event, the service Lambda log “♻️ Cache cleared”, and the next request log “🔑 Loaded fresh Claude key” again.
Verify no 401 errors appear after rotation.

If you see a 401, double‑check that the service Lambda actually cleared its cache and that the new key was stored correctly in Secrets Manager.

The Takeaway

What you should remember after reading this guide

Rotating LLM API keys is a security best practice, but serverless containers cache secrets in memory, which can cause stale‑token errors.
Store the Claude key in Secrets Manager, but read it once per container and keep it in a module‑level variable.
Use a lightweight notification (EventBridge + diagnostics_channel) to tell running Lambdas when the secret changes, so they can drop their cache.
Pay attention to hidden costs: secret‑manager pricing, per‑call charges, and Lambda features like provisioned concurrency that can surprise your budget.
Test the cache‑clear path locally, monitor cache misses in CloudWatch, and set alarms on 401 responses.
Follow the table of common gotchas to avoid pitfalls that often appear when mixing Secrets Manager, Lambda, and VPCs.

By wiring rotation events directly into your Lambda’s runtime, you keep the “sticky note” fresh without needing a cold start. Your Claude‑powered applications stay secure and stay online. Happy coding!

Transparency notice

This article was written with the help of an AI system — Groq (GPT OSS 120B).

Published: 2026-10-16 · Primary focus: SecretsManager

All code blocks are intended to be correct and runnable, but please verify them
against the official docs for the tools mentioned before using in production.

Find an error? Drop a comment — corrections are always welcome.

AI Agent Observability Made Simple: Using CloudWatch EMF and X-Ray to Trace Claude Tool Calls

Dinesh_gowtham — Thu, 15 Oct 2026 07:40:39 +0000

AI agents can feel like black boxes—when a Claude‑driven tool call fails you’re left guessing why. By pairing CloudWatch’s Embedded Metrics Format with X‑Ray tracing you get real‑time, zero‑overhead visibility into every plan‑act‑observe loop. This post shows exactly how to instrument your Node.js Lambda agent and read the signals.

Why AI Agents Need Observability

When a language model decides to call a tool—say a weather API—there are three hidden steps:

Planning – the model produces a plan string.
Acting – the code actually invokes the external service.
Observing – the response is fed back into the model for the next turn.

If any step stalls, you only see a generic “Lambda timed out” or a vague error message. Without observability you cannot answer questions like:

Did the model produce the correct plan?
Did the HTTP request succeed, or did the Bedrock service throw a ServiceUnavailableException?
How long did each step take, and where did the latency come from?

Think of an AI agent as a kitchen where a chef (Claude) writes a recipe, a sous‑chef (your code) fetches ingredients, and a taste‑tester (the next model turn) decides whether to continue. Observability is the kitchen’s order‑ticket system and timer—without it you never know which dish caused the delay.

In plain English: Adding metrics and traces is like putting a stopwatch and a notebook on each step of the recipe. You can later look back and see exactly where the problem happened.

Key concepts

Observability – the practice of exposing internal state (logs, metrics, traces) so you can understand a system’s behavior from the outside.
Metric – a numeric value that you can graph over time (e.g., “tool‑call‑latency”).
Trace – a record of the path a request follows through many services, broken into segments and subsegments.

Embedded Metrics Format: Zero‑Cost Custom Metrics

CloudWatch’s Embedded Metrics Format (EMF) lets you write JSON inside a log event that CloudWatch automatically extracts as a metric. The benefit is two‑fold:

Zero extra API calls – you just write a log line, CloudWatch does the heavy lifting.
Near‑real‑time graphs – metrics appear in the console within seconds.

Why EMF instead of the classic `PutMetricData` API?

The classic API requires a separate network call for each batch of metrics, adding latency and cost. EMF piggybacks on the log stream you already have for Lambda, so you pay only for the log storage you’d use anyway.

Tip: Keep the metric name short and consistent (e.g., tool_call_latency_ms). CloudWatch treats each unique name as a separate time‑series, which can affect cost.

Code: Minimal EMF logger for a Lambda

// emfLogger.ts
import { CloudWatchClient, PutLogEventsCommand } from "@aws-sdk/client-cloudwatch";
import { v4 as uuidv4 } from "uuid";

/**
 * Simple EMF helper that builds a JSON log line that CloudWatch
 * will turn into a custom metric.
 *
 * @param metricName   The name that will appear in the CloudWatch console.
 * @param value        Numeric value of the metric.
 * @param dimensions   Optional key/value pairs to slice the metric (e.g., tool name).
 */
export async function emitEmfMetric(
  metricName: string,
  value: number,
  dimensions: Record<string, string> = {}
) {
  // Build the EMF JSON payload.
  const emfPayload = {
    _aws: {
      // Required fields for EMF.
      Timestamp: Date.now(),
      CloudWatchMetrics: [
        {
          Namespace: "AI/ClaudeAgent",
          Dimensions: [Object.keys(dimensions)], // e.g., ["ToolName"]
          MetricName: metricName,
          Unit: "Milliseconds",
        },
      ],
    },
    // The actual metric value.
    [metricName]: value,
    // Any extra dimensions you want to filter on.
    ...dimensions,
  };

  // Emit the JSON as a log line. In Lambda the console.log output is automatically
  // sent to CloudWatch Logs, which then parses the EMF block.
  console.log(JSON.stringify(emfPayload));
}

/**
 * Helper to force the Lambda runtime to finish flushing any pending
 * async log writes. Without this the function may exit before the
 * EMF line reaches CloudWatch.
 */
export async function flushLogs() {
  // Lambda's console.log is synchronous, but when using async libraries
  // (e.g., winston) you may need an explicit flush. Here we simply
  // return a resolved promise to keep the pattern clear.
  return Promise.resolve();
}

Why we call flushLogs at the end: Lambda may stop the process as soon as the handler returns. If the EMF line hasn't been flushed, CloudWatch never sees the metric, creating a blind spot.

Key takeaway: EMF lets you turn a console.log into a metric without extra network hops, but you must make sure the log actually reaches CloudWatch before the function exits.

Adding X‑Ray Subsegments to Tool Calls

AWS X‑Ray records a trace for each request that touches your Lambda. A segment represents the Lambda invocation itself; subsegments are child pieces you create manually (e.g., a call to a Bedrock tool). Subsegments let you see latency, errors, and custom annotations for each step.

Why subsegments matter

If Claude decides to call a calendar‑lookup tool, you want to know:

How long the HTTP request took.
Whether the request succeeded or threw a ServiceUnavailableException.
Which tool name caused the latency spike.

Without subsegments you only see the total Lambda duration, making it impossible to pinpoint the problematic step.

In plain English: A subsegment is like a sticky note on a specific part of your kitchen order ticket, saying “this sauce simmered for 5 seconds” or “the oven failed to preheat”.

Code: X‑Ray client and subsegment wrapper

// xrayHelper.ts
import {
  XRayClient,
  PutTelemetryRecordsCommand,
  PutTraceSegmentsCommand,
} from "@aws-sdk/client-xray";

/**
 * Minimal wrapper that creates an X‑Ray subsegment around an async
 * operation. It records start/end time, any error, and optional annotations.
 *
 * @param name          Human‑readable name of the subsegment (e.g., "weatherTool").
 * @param fn            Async function representing the tool call.
 * @param annotations   Optional key/value pairs to attach to the subsegment.
 */
export async function withXRaySubsegment<T>(
  name: string,
  fn: () => Promise<T>,
  annotations: Record<string, string> = {}
): Promise<T> {
  // X‑Ray automatically creates a segment for the Lambda handler.
  // The SDK expects a JSON segment document; we construct a simple one.
  const segment = {
    name,
    id: crypto.randomUUID().replace(/-/g, "").substring(0, 16), // 16‑char hex ID
    start_time: Date.now() / 1000,
    // Annotations are searchable in the X‑Ray console.
    annotations,
  };

  try {
    const result = await fn(); // Run the actual tool call.
    segment.end_time = Date.now() / 1000;
    segment.http = { response: { status: 200 } };
    // Send the completed subsegment.
    await sendSubsegment(segment);
    return result;
  } catch (err: any) {
    segment.end_time = Date.now() / 1000;
    segment.error = true;
    segment.cause = { exceptions: [{ message: err.message, type: err.name }] };
    await sendSubsegment(segment);
    throw err; // Re‑throw so the Lambda handler sees the error.
  }
}

/**
 * Sends a completed subsegment to the X‑Ray daemon.
 * In Lambda the daemon is already running and reachable via the
 * AWS_XRAY_DAEMON_ADDRESS environment variable.
 */
async function sendSubsegment(segment: any) {
  const client = new XRayClient({});
  const command = new PutTraceSegmentsCommand({
    TraceSegmentDocuments: [JSON.stringify(segment)],
  });
  await client.send(command);
}

Gotcha: The X‑Ray SDK adds ~50‑100 ms to cold starts because it starts a UDP daemon connection. To keep start‑up time low, import the helper lazily (inside the handler) or use the aws-xray-sdk-core library’s captureAsyncFunc which re‑uses the daemon across invocations.

Putting It All Together in a Lambda‑Hosted Claude Agent

Now we combine EMF metrics and X‑Ray subsegments into a single Lambda function that:

Receives an event with the user query.
Calls Claude (via Bedrock) to get a plan.
Executes a tool based on the plan.
Sends the observation back to Claude for the next turn.

We will instrument the tool call part with both EMF and X‑Ray.

Full Lambda handler

// index.ts
import {
  BedrockRuntimeClient,
  InvokeModelCommand,
} from "@aws-sdk/client-bedrock-runtime";
import { emitEmfMetric, flushLogs } from "./emfLogger";
import { withXRaySubsegment } from "./xrayHelper";

// Types for clarity.
type ToolResult = {
  success: boolean;
  data?: any;
  error?: string;
};

/**
 * Example tool: fetches current weather from a fake HTTP endpoint.
 * In real life you would use fetch/axios. Here we just simulate latency.
 */
async function callWeatherTool(location: string): Promise<ToolResult> {
  // Simulated network latency.
  await new Promise((r) => setTimeout(r, Math.random() * 200 + 100));

  // Randomly throw an error to illustrate error handling.
  if (Math.random() < 0.1) {
    const err = new Error("ServiceUnavailableException");
    (err as any).name = "ServiceUnavailableException";
    throw err;
  }

  return { success: true, data: { location, temperature: 72 } };
}

/**
 * Main Lambda entry point.
 * The handler is deliberately small; most logic lives in helper functions.
 */
export const handler = async (event: any) => {
  const query = event.query ?? "What is the weather in Paris?";
  const bedrock = new BedrockRuntimeClient({});

  // 1️⃣ Ask Claude for a plan. (Simplified request.)
  const planResponse = await bedrock.send(
    new InvokeModelCommand({
      ModelId: "anthropic.claude-v2",
      ContentType: "application/json",
      Accept: "application/json",
      Body: JSON.stringify({ prompt: `Plan a tool call for: ${query}` }),
    })
  );

  // Extract the tool name and argument from Claude's response.
  // In a real implementation you'd parse the model's JSON output.
  const toolName = "weatherTool";
  const toolArg = "Paris";

  // 2️⃣ Wrap the tool call with X‑Ray and EMF.
  const startMs = Date.now();
  const result = await withXRaySubsegment(
    toolName,
    async () => await callWeatherTool(toolArg),
    { tool_name: toolName, location: toolArg }
  );
  const latencyMs = Date.now() - startMs;

  // Emit EMF metric for the latency.
  await emitEmfMetric("tool_call_latency_ms", latencyMs, {
    ToolName: toolName,
  });

  // 3️⃣ Feed the observation back to Claude (omitted for brevity).
  // ... imagine another InvokeModelCommand here ...

  // Ensure the EMF line has been flushed before Lambda exits.
  await flushLogs();

  // Return a simple JSON payload to the caller.
  return {
    statusCode: 200,
    body: JSON.stringify({
      query,
      toolResult: result,
      latencyMs,
    }),
  };
};

What the code does, step by step

Ask Claude for a plan – we send a prompt to Bedrock. The response tells us which tool to call.
Create a subsegment – withXRaySubsegment records start/end time, adds an annotation with the tool name, and sends the subsegment to the X‑Ray daemon.
Emit an EMF metric – emitEmfMetric writes a JSON log line that CloudWatch extracts as tool_call_latency_ms.
Flush logs – flushLogs guarantees the metric reaches CloudWatch before the function ends. Without it you would see a missing data point in the graph.

Tip: If you ever notice a gap in your latency graph, check that flushLogs() was awaited. The Lambda runtime can shut down the process in the middle of an async console.log.

Gotchas specific to this setup

Area	Common mistake	Result
EMF	Forgetting to `await flushLogs()`	Metric never appears, trace shows no latency info
X‑Ray	Not adding annotations	You can’t filter subsegments by tool name in the console
Bedrock	Not handling `ServiceUnavailableException`	Lambda crashes, Lambda logs show error but no metric or trace for the failed call
Lambda timeout	Metric emission takes longer than remaining time	Lambda exits early, leaving both metric and subsegment incomplete

Reading the Signals in CloudWatch Console

Once your Lambda runs, you have two places to look:

Metrics pane (EMF) – Navigate to CloudWatch → Metrics → Browse → AI / ClaudeAgent. You’ll see a line chart for tool_call_latency_ms. Use the Dimensions dropdown to slice by ToolName.
X‑Ray console – Open X‑Ray → Traces. Each trace shows the Lambda segment with a child subsegment named weatherTool. Click the subsegment to view:
- Annotations (tool name, location).
- Error flag if the call threw an exception.
- Latency (the same number you emitted as a metric).

How to spot a problem

High latency spikes on the EMF chart → drill into X‑Ray trace for the matching timestamp; the subsegment will reveal whether the delay was inside the tool call or in the Bedrock response.
Missing data points → verify that your Lambda didn’t exit before flushing. The X‑Ray console will still show the segment, but the metric line will have a gap.
Frequent error: true flags → filter X‑Ray subsegments by the error annotation; you’ll see a pattern (e.g., ServiceUnavailable every 5 minutes) that points to a downstream API issue.

Key takeaway: Metrics give you the “what” (latency numbers), traces give you the “where” (exact code path). Together they let you answer “why” in minutes instead of hours.

The Takeaway

Observability works for AI agents just like it does for any microservice.
EMF turns a log line into a metric with no extra network call; remember to flush before the Lambda exits.
X‑Ray subsegments let you see each tool call as its own piece of a trace, complete with annotations and error flags.
Combine the two: EMF for aggregate latency graphs, X‑Ray for per‑call detail.
Watch out for common pitfalls: missing flushLogs(), cold‑start overhead from X‑Ray, and cost implications of high‑resolution metrics or dense traces.

By treating Claude‑driven tool calls as first‑class operations—instrumented with metrics and traces—you gain the same confidence you have when debugging a REST API. The black‑box becomes a transparent, measurable component, and you spend less time guessing and more time improving. Happy tracing!

Transparency notice

This article was written with the help of an AI system — Groq (GPT OSS 120B).

Published: 2026-10-15 · Primary focus: CloudWatch

All code blocks are intended to be correct and runnable, but please verify them
against the official docs for the tools mentioned before using in production.

Find an error? Drop a comment — corrections are always welcome.

Bedrock Guardrails Explained Simply: How to Enforce Safe LLM Outputs in Node.js

Dinesh_gowtham — Wed, 14 Oct 2026 07:57:44 +0000

You’ve seen LLMs generate brilliant code, but a single toxic snippet can break trust. Guardrails promise to stop that, but they’re a black box for most engineers. This post shows exactly how Bedrock’s Guardrails work and how to wire them into your Node.js app without mystery.

What Are Guardrails and Why They Matter

A guardrail in the context of generative AI is a set of rules that the model must obey before it returns a response. Think of it as a safety net for a high‑wire act: the performer can still do tricks, but the net catches anything that goes too far. In practice a guardrail might reject hateful language, personal data exposure, or any content that violates your company policy.

Why bother?

User trust – If a chatbot suddenly spits out an offensive phrase, users lose confidence.
Compliance – Regulations such as GDPR or industry‑specific standards demand that certain data never be emitted.
Cost control – Re‑processing a rejected output wastes compute cycles and can increase your bill.

Bedrock Guardrails are enforced server‑side; you tell the service which guardrail to apply, and Bedrock checks every prompt and every generated token against the policy before sending anything back.

In plain English: Guardrails are a rulebook that the LLM must follow, and Bedrock checks the rulebook for you automatically.

Minimal example – a guardrail‑aware request (no error handling yet)

import {
  BedrockRuntimeClient,
  InvokeModelCommand,
} from "@aws-sdk/client-bedrock-runtime";

// Create a client that talks to the Bedrock endpoint in us-east-1
const bedrock = new BedrockRuntimeClient({ region: "us-east-1" });

// The ARN (Amazon Resource Name) of the guardrail you created in the console
const guardrailArn = "arn:aws:bedrock:us-east-1:123456789012:guardrail/ExampleGuardrail";

// The model you want to call – Claude‑2 in this case
const modelId = "anthropic.claude-v2";

// The user prompt we want to send to the model
const prompt = "Write a Node.js function that adds two numbers.";

const command = new InvokeModelCommand({
  // Tell Bedrock which model to use
  modelId,
  // Attach the guardrail by ARN
  guardrailIdentifier: guardrailArn,
  // The body must be a JSON string that matches the model’s input schema
  body: JSON.stringify({ prompt, max_tokens_to_sample: 256, temperature: 0.7 }),
});

bedrock.send(command).then((resp) => {
  const result = new TextDecoder().decode(resp.body);
  console.log("Model response:", result);
});

The code above simply calls Claude‑2 with a guardrail attached. It will fail if the guardrail blocks the request, but we haven’t shown how to handle that failure yet.

Creating and Configuring a Guardrail in the AWS Console

Before any code can use a guardrail, you need to define one. The console provides a visual editor where you can add policy statements – think of them as individual rules. Each statement can target:

Content categories (e.g., profanity, hate speech)
PII patterns (e.g., Social Security numbers, credit cards)
Custom regex you supply

Why the extra steps? Guardrails are not a simple on/off toggle; they are a policy set that decides what gets blocked and how you are told about it. If you leave the policy too strict, legitimate requests may be rejected silently, creating a frustrating user experience.

Step‑by‑step in the console

Open the Amazon Bedrock console → Guardrails → Create guardrail.
Give it a name like SafeCodeGuardrail.
Add a Content category rule: “Block profanity” – set the severity to ERROR (this will cause a rejection).
Add a Custom regex rule: \b(?:\d{3}-\d{2}-\d{4})\b to block SSN‑like patterns.
Set the Action for each rule to Reject (instead of Mask) so you receive a clear error.
Review the Policy summary – notice the Evaluation order; the first rule that matches decides the outcome.
Click Create. The console will now show the guardrail’s ARN – copy it; you’ll need it in code.

Tip: Start with a small set of rules and expand gradually. Over‑loading a guardrail makes debugging harder because you’ll see many rejections without knowing which rule fired.

Gotcha: The console UI does not warn you if two rules overlap. If a later rule is more permissive, it will never run because an earlier rule already rejected the request.

Calling Bedrock with Guardrails via the SDK

Now that you have a guardrail ARN, the SDK call is only a few extra fields away from a plain model invocation. The important part is guardrailIdentifier – the field that tells Bedrock which policy to enforce.

Why pass the ARN each time instead of binding it to the model? Guardrails are model‑agnostic; the same policy can protect Claude‑2, Titan, or any future model you switch to. This decoupling lets you experiment with models without rewriting policy logic.

Full Lambda‑ready snippet (still no error handling)

// handler.ts – entry point for a Node.js 22 Lambda
import {
  BedrockRuntimeClient,
  InvokeModelCommand,
} from "@aws-sdk/client-bedrock-runtime";

// The guardrail we created earlier
const GUARDRAIL_ARN = process.env.GUARDRAIL_ARN!; // set via Lambda env vars
const MODEL_ID = "anthropic.claude-v2";

// Create a single client instance – re‑use across invocations for better performance
const bedrock = new BedrockRuntimeClient({ region: "us-east-1" });

export const handler = async (event: { prompt: string }) => {
  const command = new InvokeModelCommand({
    modelId: MODEL_ID,
    guardrailIdentifier: GUARDRAIL_ARN,
    body: JSON.stringify({
      prompt: event.prompt,
      max_tokens_to_sample: 300,
      temperature: 0.6,
    }),
  });

  // Send the request and wait for Bedrock’s response
  const response = await bedrock.send(command);
  const modelOutput = new TextDecoder().decode(response.body);
  return { statusCode: 200, body: modelOutput };
};

In a real Lambda you must also consider cold‑start latency and rate limits (Bedrock limits are per‑minute across the whole account, not per request). If you exceed the limit you’ll get a ThrottlingException, which is a different error class from guardrail violations.

In plain English: Adding guardrailIdentifier tells Bedrock “please check this request against my rulebook before you answer.”

Handling GuardrailViolationException in Your Code

When a guardrail blocks a request, Bedrock does not return a normal HTTP 200 with an error field. Instead it responds with HTTP 400 and the SDK throws a GuardrailViolationException – a custom exception class. If you catch only generic errors, the Lambda will crash and the caller sees a 502.

Why does this matter? Because the exception contains metadata about which rule fired, the offending input, and a helpful message. You can log the offending prompt, alert a monitoring system, or fall back to a safe template.

Catching the specific exception

import {
  BedrockRuntimeClient,
  InvokeModelCommand,
  GuardrailViolationException,
} from "@aws-sdk/client-bedrock-runtime";

export const handler = async (event: { prompt: string }) => {
  const command = new InvokeModelCommand({
    modelId: "anthropic.claude-v2",
    guardrailIdentifier: process.env.GUARDRAIL_ARN!,
    body: JSON.stringify({
      prompt: event.prompt,
      max_tokens_to_sample: 300,
      temperature: 0.6,
    }),
  });

  try {
    // Normal successful path
    const resp = await bedrock.send(command);
    const output = new TextDecoder().decode(resp.body);
    return { statusCode: 200, body: output };
  } catch (err) {
    // Specific handling for guardrail rejections
    if (err instanceof GuardrailViolationException) {
      console.warn("Guardrail blocked request:", {
        prompt: event.prompt,
        // The exception payload includes which rule triggered the block
        ruleId: err.ruleId,
        message: err.message,
      });

      // Fallback to a safe, pre‑written response template
      const safeResponse = "I’m sorry, I can’t help with that request.";
      return { statusCode: 200, body: safeResponse };
    }

    // Re‑throw any other unexpected errors so Lambda surfaces them correctly
    throw err;
  }
};

Key points in the code

instanceof GuardrailViolationException ensures we only trap the guardrail case.
err.ruleId (available on the exception) tells you which policy line rejected the prompt – invaluable for tuning.
Returning a 200 with a safe message keeps the API contract stable for the caller, avoiding a broken UI.

Takeaway: Guardrail violations surface as a dedicated exception, not a generic HTTP error. Catch it explicitly, log the details, and provide a graceful fallback.

Testing and Tuning Guardrail Policies

A guardrail that never fires is as useless as a lock with the key left under the mat. You need a quick way to verify that the policy behaves as you expect.

Why test?

Prevent silent rejections – If a rule is too broad, normal user prompts will be blocked, leading to a poor experience.
Validate new rules – Adding a regex for credit‑card numbers could unintentionally catch a harmless sequence like “1234”.
Measure latency impact – Guardrail checks add a few milliseconds; heavy policies may add more.

Simple local test harness

// testGuardrail.ts – run with `node testGuardrail.ts`
import {
  BedrockRuntimeClient,
  InvokeModelCommand,
  GuardrailViolationException,
} from "@aws-sdk/client-bedrock-runtime";

const client = new BedrockRuntimeClient({ region: "us-east-1" });
const guardrailArn = "arn:aws:bedrock:us-east-1:123456789012:guardrail/ExampleGuardrail";

async function tryPrompt(prompt: string) {
  const cmd = new InvokeModelCommand({
    modelId: "anthropic.claude-v2",
    guardrailIdentifier: guardrailArn,
    body: JSON.stringify({ prompt, max_tokens_to_sample: 100, temperature: 0.5 }),
  });

  try {
    const resp = await client.send(cmd);
    console.log("✅ Allowed:", prompt);
    console.log("Response:", new TextDecoder().decode(resp.body));
  } catch (e) {
    if (e instanceof GuardrailViolationException) {
      console.log("❌ Blocked:", prompt);
      console.log("Rule that blocked:", e.ruleId);
    } else {
      console.error("Unexpected error:", e);
    }
  }
}

// A set of test prompts – add more as you refine your policy
(async () => {
  await tryPrompt("Write a hello‑world program in JavaScript.");
  await tryPrompt("Give me a joke about a programmer."); // should be ok
  await tryPrompt("Tell me how to create a phishing email."); // should be blocked
  await tryPrompt("My SSN is 123-45-6789."); // should be blocked by custom regex
})();

Run this locally or in a CI job after you update a guardrail. The console output immediately tells you which prompts are rejected and which rule caused it.

Fine‑tuning tips

Start with MASK actions – they replace offending text with a placeholder instead of rejecting, letting you see the model’s raw output.
Gradually raise severity to REJECT once you’re confident the rule works.
Use the ruleId from the exception to pinpoint the exact statement that needs adjustment.
Watch Bedrock’s rate‑limit metrics – rapid testing can hit the per‑minute token limit, causing throttling unrelated to guardrails.

Tip: Treat your guardrail like a spell‑checker. First you underline potential problems, then you decide whether to auto‑correct (mask) or refuse (reject).

The Takeaway

Key points to remember

Guardrails are a rule set that Bedrock checks before sending any model output.
They are not a simple on/off switch; each rule has its own severity and action.
When a guardrail blocks a request, the SDK throws a GuardrailViolationException – catch this specific class to avoid Lambda crashes.
The exception carries the ruleId and message, enabling precise logging and iterative tuning.
Build a small local test harness to verify policy behavior before shipping changes to production.
Keep an eye on Bedrock’s per‑minute token limits and Lambda cold‑start considerations; they are separate concerns but can surface together in production errors.

Action steps you can take today

Create a minimal guardrail in the AWS console with one profanity rule.
Add the ARN to an existing Node.js Lambda that calls Claude‑2.
Implement the try…catch block shown above to handle GuardrailViolationException.
Run the local test script with a few deliberately bad prompts.
Adjust the policy based on the ruleId you see in the logs, then promote the changes to production.

You now have a concrete, production‑ready pattern for keeping LLM outputs safe, trustworthy, and compliant—all without a mysterious black box. Happy coding!

Transparency notice

This article was written with the help of an AI system — Groq (GPT OSS 120B).

Published: 2026-10-14 · Primary focus: Bedrock

All code blocks are intended to be correct and runnable, but please verify them
against the official docs for the tools mentioned before using in production.

Find an error? Drop a comment — corrections are always welcome.

Edge AI with CloudFront: How to Serve Claude Responses at the Edge Using Lambda Web Functions, Explained Simply

Dinesh_gowtham — Tue, 13 Oct 2026 09:40:41 +0000

Imagine your users getting instant AI‑generated code suggestions without ever hitting your origin servers. By moving LLM calls to the edge you cut latency and offload traffic, while still keeping the power of Claude behind Bedrock.

Why Edge AI Matters

When a developer types a line of code and clicks “suggest”, they expect the answer in a blink. If the request has to travel from the user’s browser → your CDN → a central API server → the AI model and back, each hop adds milliseconds. Those milliseconds pile up into a noticeable lag that feels “slow”.

Think of the internet as a delivery network. A package shipped across the country takes longer than a package dropped off at the nearest local post office. Edge AI puts the “post office” right where the user is, so the “package” (the AI response) travels a much shorter distance.

Why this matters:

User experience – sub‑second replies feel responsive, increasing adoption.
Cost – fewer bytes cross the backbone, reducing data‑transfer bills.
Scalability – the central backend sees less load, letting you serve more concurrent users.

In plain English: Moving the AI call to the edge is like moving a coffee shop from downtown to every neighborhood block—people get their coffee faster, and the downtown shop isn’t overwhelmed.

Creating a CloudFront Distribution with a Lambda Web Function Origin

CloudFront is Amazon’s global content‑delivery network (CDN). It caches static assets at edge locations and can also route dynamic requests to custom origins. A Lambda Web Function (LWF) is a lightweight Lambda that runs at an edge location and can serve as a origin for CloudFront, meaning CloudFront forwards the request to the LWF instead of a traditional web server.

The new “origin” type is handy because you no longer need a separate API gateway or EC2 instance just to forward a request to Bedrock. The LWF runs on the same edge node that serves the static assets, keeping everything together.

Step‑by‑step creation with the AWS SDK

Below is a minimal script that uses the @aws-sdk/client-cloudfront package to:

Create an S3 bucket for static assets (optional, but often paired with a CDN).
Deploy a Lambda Web Function (code is uploaded separately; we only reference its ARN here).
Build a CloudFront distribution that points to the LWF as its origin.

// import the CloudFront client from the AWS SDK v3
import {
  CloudFrontClient,
  CreateDistributionCommand,
  DistributionConfig,
} from "@aws-sdk/client-cloudfront";

// The ARN of the Lambda Web Function you deployed earlier.
// Example: arn:aws:lambda:us-east-1:123456789012:function:my-edge-lwf
const LWF_ARN = process.env.LWF_ARN!;

// Basic CloudFront config that uses the LWF as its origin
const distributionConfig: DistributionConfig = {
  // A friendly name for the distribution
  Comment: "Edge AI for Claude suggestions",
  // Turn on the default cache behavior – it will forward all requests to the LWF
  DefaultCacheBehavior: {
    TargetOriginId: "LWFOrigin",
    ViewerProtocolPolicy: "redirect-to-https",
    // Allow all HTTP methods because our LWF expects POST /prompt
    AllowedMethods: {
      Quantity: 2,
      Items: ["GET", "HEAD", "POST"],
      // Cache only GET/HEAD; POST will be passed through
      CachedMethods: { Quantity: 2, Items: ["GET", "HEAD"] },
    },
    // Forward the request body to the LWF (important for POST)
    ForwardedValues: {
      QueryString: true,
      Cookies: { Forward: "all" },
      // Enable body forwarding (requires Lambda@Edge or LWF)
      // Note: This is automatically handled for LWF origins.
    },
    // Reduce TTL so that dynamic responses aren’t cached for too long
    MinTTL: 0,
    DefaultTTL: 0,
    MaxTTL: 0,
  },
  // Define the LWF as an origin
  Origins: {
    Quantity: 1,
    Items: [
      {
        Id: "LWFOrigin",
        // The special domain name that points to the LWF
        DomainName: `${LWF_ARN.split(":")[5]}.lambda-url.us-east-1.on.aws`,
        // LWF origins don’t need an origin path
        OriginPath: "",
        CustomHeaders: { Quantity: 0 },
        // The LWF runs over HTTPS; we don’t need an S3 or custom origin config
      },
    ],
  },
  // Enable the distribution (true = live)
  Enabled: true,
};

// Create the CloudFront client (region is ignored for global service)
const cfClient = new CloudFrontClient({ region: "us-east-1" });

async function createDistribution() {
  try {
    const cmd = new CreateDistributionCommand({ DistributionConfig: distributionConfig });
    const response = await cfClient.send(cmd);
    console.log("Distribution created with ID:", response.Distribution?.Id);
  } catch (err) {
    console.error("Failed to create distribution:", err);
    process.exit(1);
  }
}

// Run the function
createDistribution();

Key points in the code

TargetOriginId links the cache behavior to the origin we defined.
AllowedMethods includes POST because our LWF expects a JSON payload.
MinTTL, DefaultTTL, and MaxTTL are set to 0 so that dynamic AI responses are never cached by CloudFront (you’ll see a dedicated caching strategy later).

Tip: CloudFront Functions (the ultra‑lightweight scripting option) cannot make network calls, so they’re not suitable for AI calls. Use a Lambda Web Function instead.

Gotchas you may hit

Service	Gotcha	Why it matters
CloudFront Functions	2 MB code limit, no network access	You can’t call Bedrock from a Function; you need LWF.
Lambda@Edge	1 MB response limit, 128 MB memory max	Large streamed AI responses can be truncated; LWF lifts those limits.
Cache invalidation	Takes up to 60 seconds	Expect a short delay when you manually purge objects.
Origin Access Identity (OAI)	Deprecated but still allowed	Some teams keep OAI on old configs, leading to “Access Denied” errors.

Calling Claude via Bedrock from the Edge with fetch

Bedrock is Amazon’s managed platform for large language models (LLMs) like Claude, Jurassic, etc. The service offers an HTTPS endpoint called invokeModel that you call with a JSON payload containing the prompt.

A Lambda Web Function runs in a sandbox that does not automatically load AWS credentials from the usual SDK chain (environment variables, EC2 metadata, etc.). If you try to use the standard Bedrock client inside the LWF, you’ll receive a 403 “Forbidden” error. The fix is to sign the HTTP request yourself using SigV4 (Signature Version 4) – the same signing process the SDK performs under the hood.

The @aws-sdk/signature-v4 package lets you create a signed fetch request in a few lines.

Full handler code

// Lambda Web Function entry point – Node.js 22 runtime
import { SignatureV4 } from "@aws-sdk/signature-v4";
import { Sha256 } from "@aws-crypto/sha256-browser";
import { defaultProvider } from "@aws-sdk/credential-provider-node";

// Bedrock model identifier – Claude‑instant‑1 in the us-east-1 region
const BEDROCK_MODEL_ID = "anthropic.claude-instant-v1";

// Bedrock endpoint – region‑specific
const BEDROCK_ENDPOINT = "https://bedrock-runtime.us-east-1.amazonaws.com";

// Helper: create a signed fetch request for Bedrock invokeModel
async function signedFetch(url, body) {
  // Grab credentials from the default provider chain (access key, secret, token)
  const credentials = await defaultProvider()();

  // Build a SignatureV4 signer for the "bedrock" service
  const signer = new SignatureV4({
    service: "bedrock",
    region: "us-east-1",
    credentials,
    sha256: Sha256,
  });

  // Create a raw HTTP request object that fetch understands
  const request = {
    method: "POST",
    hostname: "bedrock-runtime.us-east-1.amazonaws.com",
    path: url,
    headers: {
      "content-type": "application/json",
    },
    body: JSON.stringify(body),
  };

  // Sign the request – this adds the Authorization header and x‑amz‑date, etc.
  const signed = await signer.sign(request);

  // Perform the fetch with the signed headers
  return fetch(`${BEDROCK_ENDPOINT}${url}`, {
    method: "POST",
    headers: signed.headers,
    body: signed.body,
  });
}

// Simple in‑memory cache for repeated prompts (see next section for more details)
const promptCache = new Map(); // key: prompt string, value: { response, expiresAt }

// Lambda Web Function handler – receives POST /prompt
export async function handler(event) {
  try {
    // Parse the incoming JSON body
    const body = JSON.parse(event.body);
    const userPrompt = body.prompt?.trim();
    if (!userPrompt) {
      return {
        statusCode: 400,
        body: JSON.stringify({ error: "Missing 'prompt' in request body" }),
      };
    }

    // --------- Prompt caching (optional) ----------
    const cached = promptCache.get(userPrompt);
    if (cached && cached.expiresAt > Date.now()) {
      // Return cached response quickly
      return {
        statusCode: 200,
        body: JSON.stringify({ answer: cached.response, cached: true }),
      };
    }

    // --------- Build Bedrock request ----------
    const bedrockPayload = {
      modelId: BEDROCK_MODEL_ID,
      // Claude expects a "messages" array; we send a single user message
      messages: [{ role: "user", content: userPrompt }],
      // Request streaming output so we can forward chunks as they arrive
      stream: true,
    };

    // Call Bedrock with a signed fetch
    const response = await signedFetch("/model/invoke", bedrockPayload);

    // Handle common error case – quota exceeded
    if (response.status === 429) {
      const errBody = await response.text();
      console.error("Rate limit hit:", errBody);
      return {
        statusCode: 429,
        body: JSON.stringify({ error: "ServiceQuotaExceededException: Too many requests" }),
      };
    }

    // --------- Stream Bedrock's SSE back to the client ----------
    // Bedrock returns Server‑Sent Events (SSE) when stream:true.
    // We'll pipe those events directly through the edge.
    const reader = response.body.getReader();
    const encoder = new TextEncoder();

    // Prepare a streaming response for CloudFront
    const stream = new ReadableStream({
      async start(controller) {
        // Helper to parse SSE lines like "data: {...}"
        const parseSSE = (chunk) => {
          const text = new TextDecoder().decode(chunk);
          const lines = text.split("\n");
          for (const line of lines) {
            if (line.startsWith("data:")) {
              const json = line.slice(5).trim();
              if (json !== "[DONE]") {
                const data = JSON.parse(json);
                // Claude returns the generated text in `completion`
                const answer = data.completion;
                controller.enqueue(encoder.encode(answer));
                // Store in cache for next identical prompt (TTL 30 seconds)
                promptCache.set(userPrompt, {
                  response: answer,
                  expiresAt: Date.now() + 30_000,
                });
              } else {
                controller.close();
              }
            }
          }
        };

        // Read chunks from Bedrock and feed them to the parser
        while (true) {
          const { done, value } = await reader.read();
          if (done) break;
          parseSSE(value);
        }
      },
    });

    // Return the streamed response to the browser
    return {
      statusCode: 200,
      headers: { "content-type": "text/plain; charset=utf-8" },
      body: stream,
    };
  } catch (err) {
    console.error("Unexpected error:", err);
    return {
      statusCode: 500,
      body: JSON.stringify({ error: "Internal server error" }),
    };
  }
}

Walk‑through of the most important lines

Credential fetching – defaultProvider() reads the IAM role attached to the LWF. This works even though the LWF cannot use the normal SDK client automatically.
Signature creation – SignatureV4 builds the Authorization header required by Bedrock. Without it you’d get a 403.
Streaming – Bedrock returns Server‑Sent Events (SSE). We read the raw bytes, split on newline, and forward only the data: lines, which contain the generated text. This keeps the latency low because the client sees the answer as it’s produced.
Error handling for quota – When Bedrock returns 429 we surface a clear message (ServiceQuotaExceededException) so callers know they’ve hit the per‑minute token limit.

Key takeaway: The edge cannot rely on the SDK’s hidden signing; you must sign the request yourself. Think of the signature as a sealed envelope that proves “this request really comes from my AWS account”.

Gotchas specific to Bedrock

Gotcha	What it looks like	How to avoid it
Token limits per minute	429 responses even though each request only uses a few tokens	Batch identical prompts, use caching, monitor usage
SSE parsing	Missing the “data:” prefix leads to empty output	Always split on newline and ignore lines without `data:`
Knowledge Base sync delay	New documents not searchable for up to several minutes	Warm‑up the cache or pre‑load prompts after updates
Model selection	Some models don’t support streaming, causing 400 errors	Verify `stream:true` is supported for the chosen model
Cross‑region latency	Calls to a Bedrock region far from the edge location can be >200 ms	Deploy the LWF in the same region as the Bedrock endpoint (e.g., us‑east‑1)

Prompt Caching and Token Reuse at the Edge

Even with edge‑level latency, calling an LLM for every identical prompt is wasteful. Each call consumes tokens (the unit Bedrock counts for billing) and contributes to the per‑minute quota. A tiny cache that lives inside the Lambda Web Function’s memory can dramatically reduce repeat calls.

Simple in‑memory cache design

Because a Lambda Web Function may be reused for multiple requests (the “warm” container), an ordinary JavaScript Map is enough for short‑lived caching. We add an expiration timestamp so stale entries are purged automatically.

// Cache entry shape
// {
//   response: string,          // The text Claude generated
//   expiresAt: number         // Epoch ms when entry should be discarded
// }

// Global map – lives as long as the Lambda container stays warm
const promptCache = new Map();

/**
 * Store a prompt/response pair for a limited time.
 * @param {string} prompt - The exact user prompt string.
 * @param {string} answer - Claude's generated answer.
 * @param {number} ttlMs - Time‑to‑live in milliseconds (default 30 s).
 */
function cachePrompt(prompt, answer, ttlMs = 30_000) {
  promptCache.set(prompt, {
    response: answer,
    expiresAt: Date.now() + ttlMs,
  });
}

/**
 * Retrieve a cached answer if it exists and is still fresh.
 * @param {string} prompt
 * @returns {string|undefined}
 */
function getCachedAnswer(prompt) {
  const entry = promptCache.get(prompt);
  if (entry && entry.expiresAt > Date.now()) {
    return entry.response;
  }
  // Remove stale entry
  if (entry) promptCache.delete(prompt);
  return undefined;
}

Why cache at the edge?

If a user on a page repeatedly asks “How do I reverse a string in JavaScript?” the first request hits Bedrock, but the next three can be satisfied locally, saving both time and money.

Tip: Keep the TTL short (e.g., 30 seconds) to stay fresh with model updates while still catching rapid repeats.

Integrating the cache into the handler

In the handler shown earlier, we call getCachedAnswer right after parsing the request. If a fresh entry exists, we immediately return it with a small JSON wrapper indicating it came from the cache. When streaming, we store each chunk’s answer in the cache after the first full message is assembled.

Observability and Debugging with CloudWatch Logs

When your code runs at the edge, traditional local debugging isn’t possible. CloudWatch Logs is the primary way to see what’s happening inside a Lambda Web Function.

Sending logs from the edge

The LWF runtime provides a global console object that automatically writes to CloudWatch. Adding console.log statements at strategic points gives you visibility into:

Incoming request payloads (sanitized for PII)
Bedrock response status codes
Cache hits vs. misses
Errors such as ServiceQuotaExceededException

export async function handler(event) {
  console.log("Received event:", JSON.stringify(event)); // log request metadata

  // ... after parsing prompt
  console.log("User prompt:", userPrompt);

  // Cache check
  const cached = getCachedAnswer(userPrompt);
  if (cached) {
    console.log("Cache hit for prompt");
    return {
      statusCode: 200,
      body: JSON.stringify({ answer: cached, cached: true }),
    };
  }
  console.log("Cache miss – invoking Bedrock");

  // After Bedrock call
  console.log("Bedrock HTTP status:", response.status);
  if (response.status === 429) {
    console.warn("Quota exceeded for Bedrock");
  }

  // Inside SSE parsing loop
  console.log("Received chunk from Bedrock:", chunk.length, "bytes");
  // ... rest of handler
}

Monitoring latency and errors

Create a CloudWatch metric filter that looks for Bedrock HTTP status: 5.. or 429 in the logs. Then set up an alarm that notifies you (e.g., via SNS) when the error rate spikes. This helps you catch sudden quota exhaustion or network hiccups before users notice.

Debugging tip for the 403 “Forbidden” error

If you see a 403 in the logs, double‑check two things:

The request is signed (Authorization header present). You can print signed.headers before the fetch call.
The IAM role attached to the Lambda Web Function has bedrock-runtime:InvokeModel permission for the specific model ARN.

In plain English: A 403 means “AWS doesn’t think this request belongs to you”. Make sure the signature is there and the role is allowed to call Claude.

The Takeaway

Edge AI moves the expensive LLM call from a central server to the CDN, delivering sub‑second answers and reducing backend load.
A Lambda Web Function can act as a CloudFront origin, but it cannot rely on the SDK’s automatic credential chain. Use @aws-sdk/signature-v4 to sign a raw fetch request to Bedrock.
Stream Bedrock’s Server‑Sent Events directly back to the browser so users see the answer as it’s generated.
A lightweight in‑memory cache at the edge saves tokens, respects per‑minute limits, and speeds up repeated prompts.
CloudWatch logs and metric filters give you the only window into what’s happening on the edge; watch for 403 (missing signature) and 429 (quota) responses.

With these pieces in place, you can build interactive developer tools, code assistants, or any real‑time AI feature that feels local to the user, while still leveraging the strong language capabilities of Claude on AWS Bedrock. Happy coding at the edge!

Transparency notice

This article was written with the help of an AI system — Groq (GPT OSS 120B).

Published: 2026-10-13 · Primary focus: CloudFront

All code blocks are intended to be correct and runnable, but please verify them
against the official docs for the tools mentioned before using in production.

Find an error? Drop a comment — corrections are always welcome.

API Gateway HTTP API + Lambda Function URLs: Building a Prompt‑Caching LLM Proxy, Explained Simply

Dinesh_gowtham — Mon, 12 Oct 2026 09:36:20 +0000

LLM‑powered features like code completion are often wrapped in heavyweight Lambda functions that add latency and cost. By letting API Gateway’s HTTP API route directly to a lightweight Function URL and adding a tiny in‑process cache, you get sub‑100 ms responses for repeated prompts. This post shows exactly how to wire it together with Node.js 22’s native fetch and diagnostics_channel.

Why Prompt Caching Matters for LLM Proxies

When a user asks an LLM (large language model) the same question twice, the model does the same heavy computation both times. That work costs money and adds time. A prompt cache stores the model’s answer for a given prompt so the second request can be answered instantly, without contacting the model again.

In plain English: Think of a cache like a sticky note you put on your desk. The first time you look up a phone number you write it down; the next time you need it, you just glance at the note instead of dialing the operator again.

Why keep the cache inside the Lambda instead of an external store?

Speed – a Map object lives in the same process, so a lookup is a few nanoseconds.
Cost – no extra DynamoDB reads or Redis traffic.
Simplicity – no extra IAM policies, no VPC endpoints.

The trade‑off is that the cache disappears when the Lambda container is recycled, but for high‑frequency prompts that’s fine; the next cold start will rebuild the cache quickly.

Quick code sketch

// A very small in‑process cache.
// Keys are SHA‑256 hashes of the prompt string.
const cache = new Map<string, string>();

Key takeaway: Caching at the function level can shave tens of milliseconds off every repeat request, and you don’t need a separate service to make it happen.

Setting Up an HTTP API with a Lambda Function URL

API Gateway offers two flavors: REST API (v1) and HTTP API (v2). The latter is lighter, cheaper, and supports Function URLs – a direct HTTPS endpoint that invokes a Lambda without the extra integration layer.

Why use a Function URL instead of the classic “Lambda integration”?

Less latency – the request goes straight to the Lambda, no extra JSON mapping.
Simpler permissions – you only need to allow the API’s principal to call the URL.
Lower cost – HTTP APIs are priced per million requests, while REST APIs add a per‑request surcharge.

Step‑by‑step creation with the AWS SDK

Below is a minimal script that creates an HTTP API, a route /prompt, and ties it to a Lambda Function URL. It also shows the binary‑support flag that prevents the 6 KB body‑drop bug.

import {
  ApiGatewayV2Client,
  CreateApiCommand,
  CreateIntegrationCommand,
  CreateRouteCommand,
  UpdateApiCommand,
} from "@aws-sdk/client-api-gateway";

// Replace with your region and Lambda ARN
const REGION = "us-east-1";
const LAMBDA_URL = "https://abcde12345.lambda-url.us-east-1.on.aws";

const client = new ApiGatewayV2Client({ region: REGION });

async function createHttpApi() {
  // 1️⃣ Create the HTTP API
  const apiResult = await client.send(
    new CreateApiCommand({
      Name: "PromptCacheAPI",
      ProtocolType: "HTTP",
      // Enable binary support so bodies >6 KB are not stripped
      // (the list of supported media types must include application/json)
      BinaryMediaTypes: ["application/json"],
    })
  );

  const apiId = apiResult.ApiId!;
  console.log(`API created: ${apiId}`);

  // 2️⃣ Create an integration that points to the Lambda Function URL
  const integrationResult = await client.send(
    new CreateIntegrationCommand({
      ApiId: apiId,
      IntegrationType: "HTTP_PROXY", // direct HTTP call
      IntegrationUri: LAMBDA_URL,
      PayloadFormatVersion: "2.0",
    })
  );

  const integrationId = integrationResult.IntegrationId!;
  console.log(`Integration created: ${integrationId}`);

  // 3️⃣ Create a route that listens on POST /prompt
  await client.send(
    new CreateRouteCommand({
      ApiId: apiId,
      RouteKey: "POST /prompt",
      Target: `integrations/${integrationId}`,
    })
  );

  // 4️⃣ (Optional) Enable CORS – the most common support ticket
  await client.send(
    new UpdateApiCommand({
      ApiId: apiId,
      CorsConfiguration: {
        AllowOrigins: ["*"],
        AllowMethods: ["POST"],
        AllowHeaders: ["Content-Type", "Authorization"],
      },
    })
  );

  console.log(`Route /prompt ready at https://${apiId}.execute-api.${REGION}.amazonaws.com/prompt`);
}

createHttpApi().catch((e) => console.error(e));

What the script does

CreateApiCommand – spins up an HTTP API and tells API Gateway to treat application/json as binary. This avoids the hidden bug where payloads larger than 6 KB are silently dropped before they reach the Lambda.
CreateIntegrationCommand – tells the API to forward the request to the Lambda Function URL (HTTP_PROXY mode means “just forward the request as‑is”).
CreateRouteCommand – maps the HTTP verb + path (POST /prompt) to the integration.
UpdateApiCommand – adds a permissive CORS (Cross‑Origin Resource Sharing) configuration, which is the #1 support question teams run into.

Tip: After creating the API, give the API’s execution role permission to invoke the Lambda Function URL (lambda:InvokeFunctionUrl) and set the URL’s auth type to NONE or AWS_IAM depending on your security model.

Implementing a Tiny In‑Process Cache in Node.js 22

Now that the front‑door is ready, let’s write the Lambda handler. Node.js 22 ships with the fetch API built‑in, so we don’t need any extra HTTP client library.

Why store the cache in a Map rather than a plain object?

Deterministic iteration order – useful when you later want to add LRU (least‑recently‑used) eviction.
Better key handling – Map lets you use any type as a key; we’ll use a string hash.

Full handler code

import { createHash } from "crypto";
import { fetch } from "node-fetch"; // native in Node 22, import for clarity
import { diagnosticChannel } from "diagnostics_channel";

/**
 * Simple SHA‑256 hash of the prompt. Using a cryptographic hash
 * guarantees a fixed‑length key regardless of prompt size.
 */
function hashPrompt(prompt) {
  return createHash("sha256").update(prompt, "utf8").digest("hex");
}

/**
 * In‑process cache: Map<hash, string>. The value is the complete
 * Server‑Sent Events (SSE) payload we received from Claude.
 */
const cache = new Map();

/**
 * Exponential backoff helper for retrying 429 (Too Many Requests) errors.
 */
async function retryWithBackoff(fn, attempts = 4, delay = 200) {
  for (let i = 0; i < attempts; i++) {
    try {
      return await fn();
    } catch (err) {
      if (err.status !== 429) throw err; // only handle throttling
      const wait = delay * 2 ** i;
      console.warn(`Got 429, retrying in ${wait}ms`);
      await new Promise((r) => setTimeout(r, wait));
    }
  }
  throw new Error("Exceeded retry attempts");
}

/**
 * Lambda entry point – receives an APIGatewayProxyEvent (HTTP API payload)
 * and returns a ProxyResult (HTTP response).
 */
export async function handler(event) {
  // -------------------------------------------------
  // 1️⃣ Parse and validate incoming JSON
  // -------------------------------------------------
  let body;
  try {
    body = JSON.parse(event.body);
    if (typeof body.prompt !== "string") throw new Error("Missing prompt");
  } catch (e) {
    return { statusCode: 400, body: JSON.stringify({ error: "Invalid JSON" }) };
  }

  const prompt = body.prompt.trim();
  const key = hashPrompt(prompt);

  // -------------------------------------------------
  // 2️⃣ Check the in‑process cache
  // -------------------------------------------------
  if (cache.has(key)) {
    diagnosticChannel.publish("promptCache.hit", { key });
    return {
      statusCode: 200,
      headers: { "Content-Type": "text/event-stream" },
      body: cache.get(key),
    };
  }

  diagnosticChannel.publish("promptCache.miss", { key });

  // -------------------------------------------------
  // 3️⃣ Call Claude via Bedrock using native fetch
  // -------------------------------------------------
  const region = process.env.AWS_REGION || "us-east-1";
  const bedrockUrl = `https://bedrock-runtime.${region}.amazonaws.com/model/anthropic.claude-v2/invoke`;

  const requestBody = JSON.stringify({
    prompt,
    max_tokens_to_sample: 256,
    temperature: 0.7,
    stream: true, // ask Bedrock to stream SSE
  });

  // The retry wrapper only retries on 429.
  const response = await retryWithBackoff(() =>
    fetch(bedrockUrl, {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
        // The SDK would normally sign this request; for brevity we assume
        // the Lambda execution role has the needed Bedrock permissions.
      },
      body: requestBody,
    })
  );

  if (!response.ok) {
    const errText = await response.text();
    console.error(`Bedrock error ${response.status}: ${errText}`);
    return { statusCode: 502, body: JSON.stringify({ error: "Upstream failure" }) };
  }

  // -------------------------------------------------
  // 4️⃣ Pipe the streaming SSE back to the client
  // -------------------------------------------------
  // We accumulate the stream into a string so we can cache it.
  let ssePayload = "";
  const decoder = new TextDecoder();

  // The fetch response body is a Node.js ReadableStream.
  for await (const chunk of response.body) {
    const text = decoder.decode(chunk);
    ssePayload += text; // store for caching
    // Immediately forward to the API Gateway client.
    // API Gateway HTTP API expects the full response body at the end,
    // but we can still build it incrementally.
  }

  // Save to cache for future identical prompts.
  cache.set(key, ssePayload);
  diagnosticChannel.publish("promptCache.store", { key });

  // Return as Server‑Sent Events (SSE) content type.
  return {
    statusCode: 200,
    headers: { "Content-Type": "text/event-stream" },
    body: ssePayload,
  };
}

Explanation of key lines

Line	Reason
`hashPrompt`	Turns any length prompt into a short, fixed‑size key.
`cache.has(key)`	Quick lookup to decide if we can skip the external call.
`retryWithBackoff`	Handles the common Bedrock throttling response (`429`) by waiting longer each attempt.
`fetch(..., { stream: true })`	Asks Bedrock to stream its answer as SSE, which we then forward.
`diagnosticChannel.publish`	Emits lightweight events that other tools can listen to (covered later).
`cache.set(key, ssePayload)`	Stores the whole SSE payload for the next identical request.

In plain English: The handler first looks in its own notebook (the Map). If the answer is already written down, it reads it out immediately. If not, it asks Claude, writes the spoken answer into the notebook, and then hands the answer back to the caller.

Gotchas you might hit

Binary payload loss – if you forget to set BinaryMediaTypes to include application/json, prompts larger than 6 KB will be truncated before they reach this code.
CORS errors – make sure the API’s CORS config lists POST and Content-Type header, otherwise browsers block the request.
Cold start after VPC – if your Lambda sits in a VPC, the container start time dominates; SnapStart gives no benefit in that case.

Streaming Claude Responses with Native fetch

Claude (the Anthropic model) can return a stream of tokens using Server‑Sent Events (SSE). Streaming lets the client start displaying text as soon as the first token arrives, which feels snappier than waiting for the whole response.

Why use native fetch instead of the AWS SDK’s InvokeModelCommand?

No extra dependency – fetch is already part of Node 22.
Full control over headers – we can set Accept: text/event-stream easily.
Simpler streaming loop – the response body is an async iterable, perfect for for await … of.

Minimal example of consuming the SSE stream

async function streamClaude(prompt) {
  const url = `https://bedrock-runtime.us-east-1.amazonaws.com/model/anthropic.claude-v2/invoke`;
  const res = await fetch(url, {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({ prompt, stream: true }),
  });

  // Guard against non‑200 responses early.
  if (!res.ok) throw new Error(`Bedrock returned ${res.status}`);

  // The body is an async iterator of Uint8Array chunks.
  const decoder = new TextDecoder();
  for await (const chunk of res.body) {
    const text = decoder.decode(chunk);
    // Each chunk may contain one or more SSE lines like:
    //   data: {"completion":"Hello"}
    console.log("Received chunk:", text);
    // Forward the raw text to the HTTP client (or accumulate).
  }
}

Key points

res.body implements the AsyncIterable interface, meaning you can for await … of it directly.
Each chunk is a Uint8Array; converting with TextDecoder gives the human‑readable SSE line.
Because we set stream: true, Bedrock never buffers the whole answer; it pushes each token as soon as it’s generated.

Analogy: Think of a live sports broadcast. Instead of waiting for the entire game to finish before you hear the result, the commentator describes the play as it happens. Streaming works the same way for LLM output.

Handling 429 throttling with exponential backoff

Bedrock enforces a request‑per‑second quota per account. When you exceed it, the service returns HTTP 429. The helper retryWithBackoff from the previous section automatically retries with increasing delays (200 ms → 400 ms → 800 ms …). This pattern prevents your Lambda from failing outright while still respecting the service limits.

Observability with diagnostics_channel

diagnostics_channel is a lightweight Node.js feature that lets you emit custom events without pulling in a full‑blown tracing library. It works well inside Lambda because it adds virtually no latency and doesn’t need extra IAM permissions.

Why instrument the cache with diagnostics?

Visibility – you can see cache hits vs. misses in CloudWatch Logs or a downstream collector.
Low overhead – events are just objects passed to listeners; if no listener is attached, the call is a no‑op.
Future‑proof – you can later attach an OpenTelemetry collector without touching the business logic.

Wiring up listeners

import { diagnosticChannel } from "diagnostics_channel";

// Listener that logs cache activity – you could replace this with a
// structured logger or a metric exporter.
diagnosticChannel.subscribe("promptCache.hit", (info) => {
  console.info(`[Cache] HIT for ${info.key}`);
});

diagnosticChannel.subscribe("promptCache.miss", (info) => {
  console.info(`[Cache] MISS for ${info.key}`);
});

diagnosticChannel.subscribe("promptCache.store", (info) => {
  console.info(`[Cache] STORED result for ${info.key}`);
});

Where to place the subscription?

Put the listener code in a separate module that the Lambda imports early (e.g., observability.js). Because the subscription runs at module load time, it executes once per container, not per invocation.

Tip: If you enable API Gateway access logs, you’ll see the same request IDs printed by both the API layer and your diagnostics events, making it easy to correlate end‑to‑end latency.

Seeing the data in CloudWatch

When the Lambda runs, the console.info statements appear in the function’s CloudWatch log stream. You can then create a Metric Filter that turns lines like [Cache] HIT* into a custom CloudWatch metric (PromptCacheHit). Plotting Hit vs. Miss over time shows you how effective the cache is.

The Takeaway

The core ideas that let you shrink LLM proxy latency.

Function URLs + HTTP API give a direct, low‑overhead path to Lambda, avoiding the extra mapping layer that a classic REST API adds.
Binary media support must be enabled for application/json payloads larger than 6 KB; otherwise the prompt never reaches the model.
In‑process Map cache is enough to serve repeated prompts in under 100 ms, and it requires no extra AWS resources.
Node.js 22’s native fetch makes streaming responses from Claude simple; each chunk can be piped straight to the client as Server‑Sent Events.
Retry with exponential backoff gracefully handles Bedrock’s 429 throttling without exploding your Lambda’s error count.
diagnostics_channel provides zero‑cost observability; you can turn cache activity into CloudWatch metrics or feed it into a tracing system later.

By combining these pieces, you can build a lean LLM proxy that feels fast, stays cheap, and remains easy to monitor. Happy coding!

Transparency notice

This article was written with the help of an AI system — Groq (GPT OSS 120B).

Published: 2026-10-12 · Primary focus: APIGateway

All code blocks are intended to be correct and runnable, but please verify them
against the official docs for the tools mentioned before using in production.

Find an error? Drop a comment — corrections are always welcome.

Prompt Caching with ElastiCache: Speed Up Claude Calls in Node.js 22, Explained Simply

Dinesh_gowtham — Fri, 09 Oct 2026 09:32:07 +0000

LLMs are fast, but every repeated request still costs time and tokens. What if you could memoize Claude’s answers in a Redis cache and serve them instantly? This post shows how to turn ElastiCache Serverless into a zero‑latency prompt cache for your Node.js 22 app.

Why Prompt Caching Matters for LLM‑Powered Apps

Large language models (LLM) such as Claude answer a prompt – the text you send them – by running a massive neural network. Even though the inference step takes only a few hundred milliseconds, each call:

consumes API dollars (Claude charges per token)
burns the quota of tokens you allocated for your app
adds latency that users notice, especially on mobile or edge devices

Think of an LLM like a chef who prepares a custom dish from scratch for every order. If a customer keeps ordering the same dish, the chef still has to start from the beginning each time, even though the recipe never changes. Prompt caching is the equivalent of pre‑cooking the dish and keeping it warm so the next order is served instantly.

When the same user (or a different user) asks the same question, you can skip the expensive cooking step and hand back the cached answer. The performance win feels free, but only if the cache behaves correctly. A mismatched cache key or an outdated entry can feed the wrong answer back to your users, silently breaking your app’s logic.

In plain English: Cache the result of a prompt, not the prompt itself. If the prompt changes even a little, you need a new key; otherwise you risk serving the wrong answer.

ElastiCache Serverless: Setup and Cost Model

ElastiCache Serverless is a managed, on‑demand version of Redis (or Valkey) that scales automatically. It removes the need to provision a fixed number of nodes, but the pricing model is different:

Cost factor	What it means
Per‑operation cost	Every `GET` or `SET` you run costs a few micro‑dollars. At low traffic this is cheap; at high traffic the per‑operation fee adds up faster than a provisioned cluster.
Valkey compatibility	ElastiCache Serverless runs Valkey, a Redis‑compatible fork. Most Redis commands work, but a few edge‑case commands (e.g., `MEMORY USAGE` with certain flags) behave slightly differently.
VPC placement	The cache lives inside your VPC. If your Lambda or EC2 instance is also inside the same VPC, a cold start may add extra seconds because the network interface must be attached.

Quick setup checklist

# 1. Create a Serverless cache via the AWS console or CLI
aws elasticache create-replication-group \
  --replication-group-id my-cache \
  --engine valkey \
  --cache-node-type serverless \
  --num-node-groups 1 \
  --automatic-failover-enabled

# 2. Note the endpoint (e.g., my-cache.xxxxxx.use1.cache.amazonaws.com:6379)
# 3. Attach the VPC security group that your Lambda/EC2 uses

Tip: Enable encryption‑in‑transit and at‑rest; it adds negligible latency and keeps data safe.

Connecting Node.js 22 to Redis with the New fetch‑Based Client

Node.js 22 ships with an experimental fetch‑based Redis client (node:redis). It uses the standard fetch API under the hood, so you don’t need a separate native binding like ioredis. This keeps the deployment footprint small (especially for Lambda) and works nicely with TypeScript’s built‑in types.

Install the client (still a separate npm package)

npm i redis@^4.6   # version that supports fetch‑based transport

Minimal connection code

import { createClient } from 'redis';

// The endpoint we got from the ElastiCache console
const REDIS_ENDPOINT = process.env.REDIS_ENDPOINT ?? 'my-cache.xxxxxx.use1.cache.amazonaws.com:6379';

// Build a client that talks to Redis over TCP using fetch internally
const redisClient = createClient({
  // url uses the "redis://" scheme; fetch will handle the TCP handshake
  url: `redis://${REDIS_ENDPOINT}`,
  // Optional: give fetch a short timeout so we fail fast on network hiccups
  socket: {
    // 2 seconds is usually enough for an in‑VPC call
    timeout: 2000,
  },
});

redisClient.on('error', (err) => {
  console.error('Redis connection error:', err);
});

await redisClient.connect(); // Returns a promise; wait for the handshake

Key takeaway: The fetch‑based client is just another way to talk to Redis; the rest of your code (GET/SET) stays the same.

Implementing the Cache‑First Claude Call Loop

Now that we have a Redis client, we can build the core function:

Hash the incoming prompt to create a stable cache key.
Lookup the key in Redis. If we find a value, parse it and return.
If the key is missing, call Claude’s HTTP API with fetch.
Store the JSON response in Redis with a TTL (time‑to‑live) of 5 minutes.
Return the fresh response.

Why we hash instead of using the raw prompt

A raw prompt string can be long, contain line breaks, or differ only in whitespace. Storing the entire prompt as a key makes Redis do extra work and creates many near‑duplicate keys. A hash (e.g., SHA‑256) acts like a fingerprint: two identical prompts produce the same short identifier, while any change (even a stray space) yields a different fingerprint.

Full TypeScript implementation

import { createHash } from 'crypto';          // built‑in Node module for hashing
import { fetch } from 'node-fetch';           // Node 22 has native fetch; import only for type hinting
import { redisClient } from './redisClient';  // the client we connected above

// Claude API details (replace with your real endpoint and key)
const CLAUDE_ENDPOINT = 'https://api.anthropic.com/v1/messages';
const CLAUDE_API_KEY = process.env.CLAUDE_API_KEY!;

// Helper: turn a prompt into a short, stable key
function promptToCacheKey(prompt: string): string {
  // SHA‑256 gives a 64‑char hex string; we prefix to avoid collisions with other caches
  const hash = createHash('sha256').update(prompt.trim()).digest('hex');
  return `claude:prompt:${hash}`;
}

// Main function: tries cache first, falls back to Claude, then caches result
export async function getClaudeAnswer(prompt: string): Promise<any> {
  // Step 1 – derive cache key
  const cacheKey = promptToCacheKey(prompt);

  // Step 2 – attempt to read from Redis
  const cached = await redisClient.get(cacheKey);
  if (cached) {
    // Cache hit: parse JSON and return immediately
    console.log('Cache hit for key', cacheKey);
    return JSON.parse(cached);
  }

  // Step 3 – no cache entry, call Claude
  console.log('Cache miss, calling Claude...');
  const response = await fetch(CLAUDE_ENDPOINT, {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      'x-api-key': CLAUDE_API_KEY,
    },
    body: JSON.stringify({
      model: 'claude-3-5-sonnet-20241001',
      max_tokens: 1024,
      messages: [{ role: 'user', content: prompt }],
    }),
  });

  if (!response.ok) {
    // Surface the error; in a real app you might retry or fallback
    const errorBody = await response.text();
    throw new Error(`Claude API error ${response.status}: ${errorBody}`);
  }

  const json = await response.json(); // Claude returns JSON with the answer

  // Step 4 – store the result in Redis with a 5‑minute TTL
  // JSON.stringify makes sure we store a string (Redis stores bytes)
  await redisClient.set(cacheKey, JSON.stringify(json), {
    EX: 300, // EX sets expiration in seconds; 300 s = 5 min
  });

  // Step 5 – return the fresh answer
  return json;
}

Explanation of each block

Line	What it does
`createHash('sha256')`	Creates a SHA‑256 hash object, a cryptographic fingerprint.
`.update(prompt.trim())`	Feeds the trimmed prompt (removes leading/trailing whitespace) into the hash.
`.digest('hex')`	Produces a human‑readable hex string (64 characters).
`redisClient.get(cacheKey)`	Looks up the value for that key; returns `null` if not present.
`fetch(CLAUDE_ENDPOINT, …)`	Sends an HTTP POST to Claude’s API using the built‑in `fetch`.
`await response.json()`	Parses the response body as JSON; Claude returns an object with `content`.
`redisClient.set(..., { EX: 300 })`	Stores the JSON string for 300 seconds (5 minutes). The TTL ensures stale data disappears.

Tip: Keep the TTL short for rapidly changing knowledge domains (news, weather). For static FAQs you can set a longer TTL or even store forever.

Gotchas: Cache Key Design, TTL, and Serialization Pitfalls

1. Cache key collisions

If you concatenate the prompt directly with other identifiers (e.g., user ID) without a separator, "user42:hello" and "user4:2hello" could clash. Always use a clear delimiter or, better, hash each component separately and join them with a colon.

function makeKey(userId: string, prompt: string): string {
  const promptHash = createHash('sha256').update(prompt).digest('hex');
  return `claude:user:${userId}:prompt:${promptHash}`;
}

2. Whitespace and case sensitivity

A prompt that differs only by extra spaces or different capitalisation will produce a different hash, causing a cache miss. Decide whether you want the cache to be case‑insensitive and whitespace‑agnostic, then normalise before hashing.

const normalised = prompt.replace(/\s+/g, ' ').trim().toLowerCase();

3. TTL (time‑to‑live) sizing

Setting a TTL that’s too long can serve outdated information; too short defeats the purpose of caching. A common pattern is stale‑while‑revalidate: serve the cached value even if it’s a few seconds old while you refresh it in the background. Implementing that requires a second key (e.g., :lock) or a background job, which is beyond this simple tutorial but worth noting.

In plain English: Think of TTL like the expiration date on milk. Short‑lived data (like news) needs a fresh date; stable data (like a product description) can sit longer.

4. Serialization format

Redis stores bytes; we usually stringify JSON with JSON.stringify. If you store the raw object (without stringifying) in a client that automatically performs BSON conversion, you could end up with binary data that JSON.parse can’t read. Stick to a single format throughout the app.

// WRONG – mixing binary and string
await redisClient.set(key, Buffer.from(JSON.stringify(obj)));

// RIGHT – always store a string
await redisClient.set(key, JSON.stringify(obj));

5. Valkey‑specific quirks

Valkey’s SET command accepts the same EX option, but some advanced features like SETEX behave slightly differently. If you ever need to use a command that fails, check the Valkey docs or test locally with a Valkey Docker image.

6. VPC cold‑start cost

When you run this code inside an AWS Lambda that lives in a VPC, the first request after a period of inactivity must attach an ENI (elastic network interface). That adds 1‑2 seconds before the Redis client can even start. Mitigate by:

Keeping the Lambda warm (scheduled ping).
Using Provisioned Concurrency if latency budgets are strict.

Key takeaway: The biggest hidden latency isn’t the cache lookup; it’s the network plumbing that brings your function into the VPC.

The Takeaway

What you should remember after reading this guide

Prompt caching removes token cost and latency by re‑using identical LLM responses.
ElastiCache Serverless gives you a managed Redis‑compatible store, but watch per‑operation fees, Valkey quirks, and VPC cold‑starts.
Node.js 22’s fetch‑based Redis client lets you talk to the cache without native bindings, keeping Lambda packages tiny.
Create a stable cache key by hashing a normalised prompt (think of it as a fingerprint).
Store the Claude JSON response as a string with a sensible TTL (5 minutes works for many chat‑bot use‑cases).
Guard against subtle bugs: whitespace differences, case sensitivity, serialization mismatches, and accidental key collisions.

By following these steps you’ll turn repeated Claude calls into instant look‑ups, saving money, cutting latency, and keeping your app’s answers reliable. Happy caching!

Transparency notice

This article was written with the help of an AI system — Groq (GPT OSS 120B).

Published: 2026-10-09 · Primary focus: ElastiCache

All code blocks are intended to be correct and runnable, but please verify them
against the official docs for the tools mentioned before using in production.

Find an error? Drop a comment — corrections are always welcome.

Claude‑Generated Aurora Serverless v2 Scaling Policies: A Step‑By‑Step Guide with the RDS SDK

Dinesh_gowtham — Thu, 08 Oct 2026 09:20:24 +0000

LLMs are no longer just for writing code snippets—they can design cloud infrastructure too. Imagine describing your traffic pattern in plain English and getting a ready‑to‑apply Aurora Serverless scaling policy. This post shows exactly how that works.

Why Aurora Serverless v2 Needs Dynamic Scaling

Aurora Serverless v2 (ASv2) is Amazon’s on‑demand relational database that automatically adds or removes Aurora Capacity Units (ACUs). An ACU is a bundle of CPU, memory, and networking that the database can use. Think of ACUs like seats on a bus: when more passengers (queries) show up, you add seats; when the bus empties, you remove seats to save money.

Without a policy that matches your real‑world traffic, two bad things happen:

Under‑provisioning – the database cannot keep up, causing high latency or throttling.
Over‑provisioning – you pay for capacity you never use. Remember, ASv2 has a minimum of 0.5 ACU which still costs roughly $50 / month.

A dynamic scaling policy tells the database when to grow (increase max ACU) and when to shrink (decrease min ACU). The policy is expressed as a JSON block that the RDS API understands.

In plain English: A good scaling policy is like a thermostat for your database—it heats up (adds capacity) when it gets cold (high load) and cools down (removes capacity) when things calm.

Prompting Claude to Design a Scaling Policy

Claude is Anthropic’s conversational model. By feeding it a short description of recent metrics, we can ask it to suggest a safe minACU and maxACU. The prompt should contain:

The average CPU utilization over the last 5 minutes.
The peak number of connections seen in the same window.
Any cost constraints you care about (e.g., “keep monthly cost under $200”).

Here’s a minimal prompt we can send via HTTP:

{
  "model": "claude-3-5-sonnet-20241022",
  "max_tokens": 500,
  "messages": [
    {
      "role": "user",
      "content": "We run an Aurora Serverless v2 cluster named prod-db. In the last 5 minutes CPU was 68 % on average and we saw a peak of 220 concurrent connections. Our budget allows up to $150 per month for this cluster. Suggest a minACU and maxACU that keep latency low but stay within budget. Return only a JSON object with keys minACU and maxACU."
    }
  ]
}

Claude replies with something like:

{
  "minACU": 2,
  "maxACU": 8
}

Why this works: The model has read the Aurora documentation and learned typical ACU ranges for given workloads. By asking for only the JSON, we make parsing easy and avoid extra text that could confuse our script.

Tip: Always ask Claude to return only the data you need. Adding “Return only a JSON object …” removes the chance of receiving explanatory sentences that break your parser.

Gotcha: Quota Mismatch

Claude may suggest a maxACU that exceeds the ACU quota for your AWS account, or it might accidentally flip minACU > maxACU. The RDS API will then throw an InvalidParameterCombinationException. We’ll guard against that later.

Translating Claude’s Output into RDS SDK Calls

Now we move from a plain JSON response to an actual AWS API call. We’ll use the @aws-sdk/client-rds package, the official JavaScript/TypeScript client for Amazon RDS.

First, install the SDK (run once):

npm install @aws-sdk/client-rds

Next, write a small function that:

Validates the numbers (ensures minACU ≤ maxACU and both are within your account limits).
Creates a new DBClusterParameterGroup that holds the scaling configuration.
Calls modifyDBCluster to attach the new parameter group.

// scaling.ts
import {
  RDSClient,
  CreateDBClusterParameterGroupCommand,
  ModifyDBClusterCommand,
  DBClusterParameterGroup,
} from "@aws-sdk/client-rds";

// Load credentials from the environment (AWS_ACCESS_KEY_ID, etc.)
const rds = new RDSClient({ region: "us-east-1" });

/**
 * Ensures the ACU values are sensible.
 * Returns a tuple [validMin, validMax] or throws an error.
 */
function validateAcus(min: number, max: number, quota: number): [number, number] {
  // Aurora Serverless v2 minimum is 0.5 ACU
  if (min < 0.5) throw new Error("minACU cannot be below 0.5");
  // Ensure min ≤ max
  if (min > max) throw new Error(`minACU (${min}) cannot be greater than maxACU (${max})`);
  // Make sure we don’t exceed the account quota
  if (max > quota) throw new Error(`maxACU (${max}) exceeds your ACU quota of ${quota}`);
  return [min, max];
}

/**
 * Builds a parameter group name that is unique per deployment.
 */
function buildGroupName(base: string): string {
  const timestamp = Date.now().toString(36);
  return `${base}-${timestamp}`;
}

/**
 * Main function that receives Claude’s JSON and applies the scaling policy.
 */
export async function applyScalingPolicy(
  clusterIdentifier: string,
  raw: { minACU: number; maxACU: number },
  accountAcusQuota: number // fetch from Service Quotas API or hard‑code for demo
) {
  // 1️⃣ Validate the numbers
  const [minACU, maxACU] = validateAcus(raw.minACU, raw.maxACU, accountAcusQuota);

  // 2️⃣ Create a new parameter group that holds the scaling settings
  const groupName = buildGroupName("aurora-scaling");
  const createCmd = new CreateDBClusterParameterGroupCommand({
    DBClusterParameterGroupName: groupName,
    DBParameterGroupFamily: "aurora-mysql8.0", // adjust to your engine
    Description: `Auto‑generated scaling group for ${clusterIdentifier}`,
    // Parameter values are key/value pairs; the scaling keys are documented by AWS
    Parameters: [
      { ParameterName: "aurora_serverless_max_capacity", ParameterValue: maxACU.toString() },
      { ParameterName: "aurora_serverless_min_capacity", ParameterValue: minACU.toString() },
    ],
  });

  await rds.send(createCmd);
  console.log(`Created parameter group ${groupName}`);

  // 3️⃣ Attach the new group to the DB cluster
  const modifyCmd = new ModifyDBClusterCommand({
    DBClusterIdentifier: clusterIdentifier,
    DBClusterParameterGroupName: groupName,
    ApplyImmediately: true, // For demo; in production you may prefer a maintenance window
  });

  await rds.send(modifyCmd);
  console.log(`Applied scaling policy to ${clusterIdentifier}`);
}

Key points:

aurora_serverless_min_capacity and aurora_serverless_max_capacity are the exact parameter names that control ASv2 scaling.
ApplyImmediately: true forces the change now; otherwise you must wait for the next maintenance window.
The function logs each step so you can see what happened in CloudWatch logs.

Takeaway: Turning Claude’s plain numbers into a working policy requires three safety nets – validation, a unique parameter group, and an explicit modifyDBCluster call.

Testing the Policy with a Safe Dry‑Run

Before we touch a production cluster, we should simulate the whole flow using a dry‑run flag. The AWS SDK does not provide a built‑in dry‑run for RDS, but we can mimic it by:

Skipping the actual send calls.
Printing the exact request objects that would be sent.
Running the validation logic to catch obvious errors.

Add a simple wrapper around applyScalingPolicy:

// dryRun.ts
import { applyScalingPolicy } from "./scaling";

/**
 * Executes the scaling workflow without making real API calls.
 */
export async function dryRun(
  clusterId: string,
  claudeJson: { minACU: number; maxACU: number },
  quota: number
) {
  try {
    // Run validation – this will throw if numbers are bad
    const [min, max] = (await (async () => {
      // Re‑use the internal validation (exposed for demo)
      const { validateAcus } = await import("./scaling");
      return validateAcus(claudeJson.minACU, claudeJson.maxACU, quota);
    })()) as [number, number];

    console.log("✅ Validation passed");
    console.log(`🧪 Would create parameter group with minACU=${min}, maxACU=${max}`);

    // Show the request bodies that would be sent
    const groupName = `dryrun-${Date.now()}`;
    console.log("🧾 CreateDBClusterParameterGroupCommand payload:");
    console.log({
      DBClusterParameterGroupName: groupName,
      DBParameterGroupFamily: "aurora-mysql8.0",
      Description: `Dry‑run group for ${clusterId}`,
      Parameters: [
        { ParameterName: "aurora_serverless_min_capacity", ParameterValue: min.toString() },
        { ParameterName: "aurora_serverless_max_capacity", ParameterValue: max.toString() },
      ],
    });

    console.log("🧾 ModifyDBClusterCommand payload:");
    console.log({
      DBClusterIdentifier: clusterId,
      DBClusterParameterGroupName: groupName,
      ApplyImmediately: true,
    });

    console.log("🔍 Dry‑run complete – no changes made.");
  } catch (e) {
    console.error("❌ Dry‑run failed:", e);
  }
}

Run it with Node 22 (which ships with the native fetch API for the Claude call) and you’ll see a clear picture of what would happen, without paying for a new parameter group or risking an invalid configuration.

Helpful tip: Keep the dry‑run script in version control. Treat it like a unit test for your infrastructure‑as‑code pipeline.

Real‑world Gotcha: Snapshot Restore Time

If you ever need to roll back a bad scaling change, restoring a snapshot can take 5–20 minutes. That delay is a common surprise during incidents. A dry‑run helps you catch mistakes before they land on a live cluster, saving you from a lengthy rollback.

Deploying the Policy via Node.js and RDS Proxy

In production we usually run the scaling script inside an AWS Lambda function (Node.js 22 runtime). To keep connection latency low, we place an RDS Proxy in front of the Aurora cluster. The proxy pools connections, reducing the “too many connections” error that many Lambda‑driven services see.

Setting Up the Proxy (quick recap)

In the RDS console, create a Proxy targeting your Aurora Serverless v2 cluster.
Grant the Lambda execution role the rds-db:connect permission for the proxy.
Update the Lambda environment variable DB_PROXY_ENDPOINT with the proxy endpoint.

Analogy: Think of the proxy as a receptionist who hands out a limited number of phones to callers. Instead of each caller dialing the main line (which would overload it), they use the receptionist’s pool of phones.

Full Lambda Handler

Below is a complete Lambda handler that:

Calls Claude with the latest metrics (simulated here).
Parses the JSON response.
Performs a dry‑run first (optional, can be toggled with an env var).
Applies the policy using the RDS SDK.

// handler.ts
import { APIGatewayProxyEvent, APIGatewayProxyResult } from "aws-lambda";
import { RDSClient, CreateDBClusterParameterGroupCommand, ModifyDBClusterCommand } from "@aws-sdk/client-rds";
import fetch from "node-fetch"; // Node 22 includes fetch, but we import for type safety

// Environment variables (set in Lambda config)
const {
  CLAUDE_API_KEY,
  CLUSTER_ID,
  ACU_QUOTA = "16", // default quota if you haven’t fetched it
  DRY_RUN = "true", // toggle dry‑run with "false"
} = process.env;

const rds = new RDSClient({ region: "us-east-1" });

/**
 * Helper to ask Claude for a scaling recommendation.
 */
async function askClaude(prompt: string): Promise<{ minACU: number; maxACU: number }> {
  const response = await fetch("https://api.anthropic.com/v1/messages", {
    method: "POST",
    headers: {
      "x-api-key": CLAUDE_API_KEY!,
      "Content-Type": "application/json",
      "anthropic-version": "2023-06-01",
    },
    body: JSON.stringify({
      model: "claude-3-5-sonnet-20241022",
      max_tokens: 300,
      messages: [{ role: "user", content: prompt }],
    }),
  });

  if (!response.ok) {
    const txt = await response.text();
    throw new Error(`Claude API error: ${response.status} ${txt}`);
  }

  const data = await response.json();
  // Claude returns a JSON string inside `content`. Extract it.
  const raw = JSON.parse(data.content[0].text);
  return { minACU: Number(raw.minACU), maxACU: Number(raw.maxACU) };
}

/**
 * Core Lambda handler.
 */
export const handler = async (event: APIGatewayProxyEvent): Promise<APIGatewayProxyResult> => {
  try {
    // 1️⃣ Gather recent metrics – in a real system you would read CloudWatch.
    const cpuAvg = 68; // placeholder
    const peakConns = 220; // placeholder

    const prompt = `We run an Aurora Serverless v2 cluster named ${CLUSTER_ID}. In the last 5 minutes CPU was ${cpuAvg}% on average and we saw a peak of ${peakConns} concurrent connections. Our budget allows up to $150 per month for this cluster. Suggest a minACU and maxACU that keep latency low but stay within budget. Return only a JSON object with keys minACU and maxACU.`;

    // 2️⃣ Ask Claude
    const recommendation = await askClaude(prompt);
    console.log("Claude recommendation:", recommendation);

    // 3️⃣ Validate numbers (reuse logic from scaling.ts)
    const { validateAcus } = await import("./scaling");
    const [minACU, maxACU] = validateAcus(
      recommendation.minACU,
      recommendation.maxACU,
      Number(ACU_QUOTA)
    );

    // 4️⃣ Dry‑run? If DRY_RUN is true, just log the would‑be calls.
    if (DRY_RUN?.toLowerCase() === "true") {
      console.log("🔎 Dry‑run mode – not touching AWS.");
      console.log(`Would create parameter group with minACU=${minACU}, maxACU=${maxACU}`);
      return {
        statusCode: 200,
        body: JSON.stringify({ message: "Dry‑run successful", minACU, maxACU }),
      };
    }

    // 5️⃣ Create parameter group
    const groupName = `auto-scaling-${Date.now()}`;
    await rds.send(
      new CreateDBClusterParameterGroupCommand({
        DBClusterParameterGroupName: groupName,
        DBParameterGroupFamily: "aurora-mysql8.0",
        Description: `Auto‑generated scaling group for ${CLUSTER_ID}`,
        Parameters: [
          { ParameterName: "aurora_serverless_min_capacity", ParameterValue: minACU.toString() },
          { ParameterName: "aurora_serverless_max_capacity", ParameterValue: maxACU.toString() },
        ],
      })
    );
    console.log(`Created parameter group ${groupName}`);

    // 6️⃣ Attach it to the cluster (via the proxy endpoint)
    await rds.send(
      new ModifyDBClusterCommand({
        DBClusterIdentifier: CLUSTER_ID,
        DBClusterParameterGroupName: groupName,
        ApplyImmediately: true,
      })
    );
    console.log(`Applied new scaling policy to ${CLUSTER_ID}`);

    return {
      statusCode: 200,
      body: JSON.stringify({ message: "Scaling policy applied", minACU, maxACU }),
    };
  } catch (err) {
    console.error("Error in scaling handler:", err);
    return {
      statusCode: 500,
      body: JSON.stringify({ error: (err as Error).message }),
    };
  }
};

Why we use RDS Proxy:

When the Lambda runs, each invocation could open a fresh TCP connection. Without a proxy, a burst of invocations could exceed the max connections limit of the Aurora cluster, leading to the Too many connections error. The proxy keeps a small pool (often 5‑10 connections) and re‑uses them, keeping latency low (the added overhead is typically 1‑3 ms, which is acceptable for most p99 latency budgets).

Key tip: Keep the ApplyImmediately flag true only for non‑critical clusters. Production workloads that cannot tolerate a brief pause should schedule the change in a maintenance window instead.

The Takeaway

Dynamic scaling prevents both performance bottlenecks and unnecessary spend in Aurora Serverless v2.
Claude can translate plain‑English traffic descriptions into concrete minACU/maxACU values, but you must still validate the numbers against your account quota.
The @aws-sdk/client-rds package lets you create a parameter group and attach it to a DB cluster with just a few lines of TypeScript.
Dry‑run scripts act as a safety net, printing the exact API payloads before any real changes hit your database.
RDS Proxy removes the “too many connections” problem for Lambda‑driven workloads, adding only a few milliseconds of latency.
Always test (dry‑run → staging → production) and remember that restoring a snapshot can take minutes, so catching errors early saves time and money.

By following these steps, you can let a language model do the heavy mental lifting while you keep the final guardrails firmly in your code. Happy scaling!

Transparency notice

This article was written with the help of an AI system — Groq (GPT OSS 120B).

Published: 2026-10-08 · Primary focus: RDS

All code blocks are intended to be correct and runnable, but please verify them
against the official docs for the tools mentioned before using in production.

Find an error? Drop a comment — corrections are always welcome.

How to Build a RAG Pipeline with DynamoDB’s Native Vector Search, Explained Simply

Dinesh_gowtham — Wed, 07 Oct 2026 09:05:02 +0000

Most engineers reach for a dedicated vector database the moment they need similarity search, adding latency and operational overhead. DynamoDB just announced native vector indexes, letting you store embeddings alongside your business data. In this post you’ll see how to turn that new feature into a fully‑functional Retrieval‑Augmented Generation (RAG) flow with Claude, using only DynamoDB and plain Node.js.

In plain English – DynamoDB now does the math to find “similar looking” vectors, so you no longer need a separate service just to ask “which pieces of text look like this query?”

Why DynamoDB’s Vector Search Changes the RAG Landscape

The “why” first

A Retrieval‑Augmented Generation (RAG) system works in two steps:

Retrieve the most relevant text chunks from a knowledge store.
Generate a response that weaves those chunks together, usually with a large language model (LLM) like Claude.

Historically the retrieve step required a vector store—a specialized database that can compare high‑dimensional vectors (numeric representations of text) quickly. Adding a second store meant:

extra network hops (higher latency)
two places to back up (more operational work)
consistency headaches (the DB and the vector store can drift apart)

DynamoDB’s new vector index removes the separate store. Your embeddings live right next to the rest of the item, and DynamoDB’s built‑in KNN (k‑nearest‑neighbors) engine does the similarity math for you.

Key takeaway – Cutting the stack in half also cuts the chances of data mismatches and reduces the time it takes to answer a user query.

Quick vocab recap

Embedding – a list of numbers that captures the meaning of a piece of text; think of it as a fingerprint.
Cosine similarity – a way to measure how close two fingerprints are, ranging from -1 (opposite) to 1 (identical).
KNN (k‑nearest‑neighbors) – an algorithm that returns the k most similar items to a query vector.
GSI (Global Secondary Index) – a secondary view of a DynamoDB table that can be queried with different keys or, now, with a vector index.

Setting Up a Vector‑Enabled Table in DynamoDB

The “why” first

Before we can store or search embeddings, DynamoDB needs a table that knows where the vector lives and how to compare it. This is done by creating a GSI with a vector attribute type and declaring that the index should use cosine similarity.

Step‑by‑step code

// src/createTable.ts
import {
  DynamoDBClient,
  CreateTableCommand,
} from "@aws-sdk/client-dynamodb";

// The low‑level client is used for the CreateTable call because
// the lib‑dynamodb wrapper does not expose table‑creation APIs.
const client = new DynamoDBClient({ region: "us-east-1" });

async function createVectorTable() {
  // Table name – keep it short and unique.
  const TableName = "RagDocs";

  // Primary key (partition key) – we use a simple UUID string.
  const KeySchema = [{ AttributeName: "docId", KeyType: "HASH" }];

  // The attribute that will hold the embedding as a binary blob.
  const AttributeDefinitions = [
    { AttributeName: "docId", AttributeType: "S" }, // S = String
    { AttributeName: "embedding", AttributeType: "B" }, // B = Binary
  ];

  // Global Secondary Index (GSI) that enables vector search.
  const GlobalSecondaryIndexes = [
    {
      IndexName: "EmbeddingKNN",
      // No projection needed for this demo; we pull the whole item later.
      Projection: { ProjectionType: "ALL" },
      // The GSI's key schema uses a placeholder attribute; the
      // vector is supplied at query time, not stored in the index key.
      KeySchema: [{ AttributeName: "docId", KeyType: "HASH" }],
      // Vector configuration tells DynamoDB to treat `embedding` as a
      // 1536‑dimensional Float32 vector and to compare using cosine similarity.
      // The `VectorConfig` field is only available on the GSI definition.
      VectorConfig: {
        VectorDimensions: 1536,
        VectorDataType: "FLOAT32", // 32‑bit floating point numbers
        MetricType: "COSINE", // similarity metric
        VectorField: "embedding", // attribute that holds the binary vector
      },
      // Provisioned read capacity – adjust for your traffic.
      ProvisionedThroughput: { ReadCapacityUnits: 5, WriteCapacityUnits: 5 },
    },
  ];

  const command = new CreateTableCommand({
    TableName,
    KeySchema,
    AttributeDefinitions,
    BillingMode: "PROVISIONED", // explicit to control cost
    ProvisionedThroughput: { ReadCapacityUnits: 5, WriteCapacityUnits: 5 },
    GlobalSecondaryIndexes,
    // Optional TTL (time‑to‑live) for automatic cleanup of old chunks.
    TimeToLiveSpecification: {
      AttributeName: "expiresAt", // epoch seconds
      Enabled: true,
    },
  });

  try {
    const response = await client.send(command);
    console.log("Table created:", response.TableDescription?.TableName);
  } catch (err) {
    console.error("Failed to create table:", err);
  }
}

createVectorTable();

Explanation of tricky bits

VectorConfig lives inside the GSI definition; DynamoDB uses it at query time, not at write time.
The embedding attribute must be declared as Binary (B) because the service expects a base64‑encoded Float32Array.
TTL does not delete instantly; items can stick around up to 48 hours after the expiration timestamp.

Tip – Even though the GSI has a dummy hash key (docId), DynamoDB still requires it. Think of it as a “placeholder seat” that lets the engine focus on the vector field.

Storing and Querying Embeddings with the AWS SDK

The “why” first

Now that the table exists, we need to persist text chunks together with their embeddings. The embeddings are produced elsewhere (e.g., OpenAI’s embedding endpoint) and are 1536‑dimensional Float32 vectors. DynamoDB expects those vectors as binary data, so we must encode them as base64 before sending.

When we later need the most relevant chunks for a user question, we will:

Convert the question into an embedding (same model, same dimensions).
Run a KNN query against the EmbeddingKNN GSI, asking for the top 3 nearest neighbors.

Code – inserting a document

// src/putDocument.ts
import {
  DynamoDBDocumentClient,
  PutCommand,
} from "@aws-sdk/lib-dynamodb";
import { DynamoDBClient } from "@aws-sdk/client-dynamodb";

// Low‑level client needed only for the Document wrapper.
const ddbClient = new DynamoDBClient({ region: "us-east-1" });
const ddbDoc = DynamoDBDocumentClient.from(ddbClient);

// Helper: turn a Float32Array into a base64 string.
function encodeEmbedding(vec: Float32Array): string {
  // Convert the raw bytes to a Uint8Array, then to a base64 string.
  const bytes = new Uint8Array(vec.buffer);
  return Buffer.from(bytes).toString("base64");
}

// Example: a single text chunk and its embedding.
async function putChunk(
  docId: string,
  chunkText: string,
  embedding: Float32Array,
  ttlSeconds: number // optional expiration
) {
  const command = new PutCommand({
    TableName: "RagDocs",
    Item: {
      docId, // partition key
      chunk: chunkText,
      embedding: encodeEmbedding(embedding), // stored as Binary (base64)
      // DynamoDB treats a base64 string as Binary automatically.
      // Adding a TTL helps keep the table size in check.
      expiresAt: Math.floor(Date.now() / 1000) + ttlSeconds,
    },
  });

  try {
    await ddbDoc.send(command);
    console.log(`Chunk ${docId} stored`);
  } catch (err) {
    console.error("Write error:", err);
  }
}

/* -------------------------------------------------------------
   Imagine we already fetched an embedding from OpenAI:
   const embedding = await getOpenAIEmbedding(chunkText);
   ------------------------------------------------------------- */

Code – querying the top‑3 similar chunks

// src/queryKnn.ts
import {
  DynamoDBDocumentClient,
  QueryCommand,
} from "@aws-sdk/lib-dynamodb";
import { DynamoDBClient } from "@aws-sdk/client-dynamodb";

const ddbClient = new DynamoDBClient({ region: "us-east-1" });
const ddbDoc = DynamoDBDocumentClient.from(ddbClient);

// Same encoder used for writes; DynamoDB expects the query vector
// in the same binary format.
function encodeEmbedding(vec: Float32Array): string {
  const bytes = new Uint8Array(vec.buffer);
  return Buffer.from(bytes).toString("base64");
}

/**
 * Returns the `k` most similar chunks for a given query embedding.
 */
async function knnSearch(
  queryEmbedding: Float32Array,
  k: number = 3
) {
  const command = new QueryCommand({
    TableName: "RagDocs",
    IndexName: "EmbeddingKNN", // the vector‑enabled GSI
    // The special `VectorSearch` field tells DynamoDB to perform a
    // KNN lookup using the supplied binary vector.
    // `KNN` is the number of neighbors to return.
    // `Metric` defaults to the one defined in the GSI (COSINE).
    VectorSearch: {
      KNN: k,
      VectorField: "embedding",
      QueryVector: encodeEmbedding(queryEmbedding),
    },
    // ConsistentRead is not allowed on GSI; we accept eventual consistency.
    // This matches the reality of DynamoDB’s GSI behavior.
  });

  try {
    const result = await ddbDoc.send(command);
    // Each item contains the original chunk text plus the embedding.
    return result.Items?.map((item) => ({
      docId: item.docId,
      chunk: item.chunk,
    })) ?? [];
  } catch (err) {
    console.error("KNN query error:", err);
    return [];
  }
}

Gotchas highlighted

Binary encoding – If you forget to base64‑encode the Float32Array, DynamoDB will reject the write with a ValidationException: Attribute type mismatch.
GSI attribute type – The attribute must be declared as Binary (B). Declaring it as a string leads to the same validation error.
Eventual consistency – Reads from a GSI are eventually consistent, so a newly written chunk may not appear instantly in a KNN query. In high‑throughput systems, add a small retry loop or write‑through cache.

In plain English – Think of the embedding as a secret code. DynamoDB only understands the code when it’s wrapped in a base64 envelope; missing the envelope makes the service shout “I don’t know this type!”.

Calling Claude with Retrieved Context via Node.js fetch

The “why” first

The RAG pattern finishes by feeding the retrieved chunks to an LLM so it can craft a response. Claude’s HTTP API is straightforward: send a JSON payload with a messages array and receive a completion field. We’ll use Node 22’s built‑in fetch (no extra packages needed).

Minimal fetch wrapper

// src/callClaude.ts
/**
 * Sends a user question together with retrieved context chunks to Claude.
 * Returns Claude’s answer as plain text.
 */
async function askClaude(
  question: string,
  contextChunks: { chunk: string }[],
  apiKey: string
): Promise<string> {
  // Build a single system prompt that concatenates the chunks.
  const systemPrompt = `You are an assistant that answers questions using only the following information:\n\n${contextChunks
    .map((c) => `- ${c.chunk}`)
    .join("\n")}\n\nIf the answer cannot be derived from this data, say "I don't know."`;

  const payload = {
    model: "claude-3-5-sonnet-20241007", // example model name
    messages: [
      { role: "system", content: systemPrompt },
      { role: "user", content: question },
    ],
    max_tokens: 500,
  };

  const response = await fetch(
    "https://api.anthropic.com/v1/messages",
    {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
        // Anthropic expects the key in the `x-api-key` header.
        "x-api-key": apiKey,
      },
      body: JSON.stringify(payload),
    }
  );

  if (!response.ok) {
    const errorBody = await response.text();
    throw new Error(`Claude API error ${response.status}: ${errorBody}`);
  }

  const data = await response.json();
  // The completion text lives in `content[0].text` for most responses.
  return data.content?.[0]?.text ?? "No response";
}

Tip – Keep the system prompt short; Claude’s token limit includes both the prompt and the answer, so overly long context can truncate the model’s output.

Putting It All Together: A Minimal End‑to‑End RAG Service

The “why” first

Having separate snippets is useful for learning, but production code needs a single orchestrator that:

Receives a user query (e.g., via an HTTP endpoint).
Generates an embedding for the query.
Retrieves the top‑3 matching chunks from DynamoDB.
Calls Claude with those chunks and returns the answer.

Below is a compact Node.js script that ties everything together. It uses the same SDKs and fetch we already introduced, plus the OpenAI embedding endpoint (you can swap it for any model that returns a 1536‑dimensional vector).

Full flow script

// src/ragService.ts
import { DynamoDBDocumentClient } from "@aws-sdk/lib-dynamodb";
import { DynamoDBClient } from "@aws-sdk/client-dynamodb";
import { knnSearch } from "./queryKnn";
import { askClaude } from "./callClaude";
import fetch from "node-fetch"; // Node 22 includes global fetch, but keep for type safety

// Re‑use the document client we created earlier.
const ddbClient = new DynamoDBClient({ region: "us-east-1" });
const ddbDoc = DynamoDBDocumentClient.from(ddbClient);

// Replace with your own keys.
const OPENAI_API_KEY = process.env.OPENAI_API_KEY!;
const CLAUDE_API_KEY = process.env.CLAUDE_API_KEY!;

/**
 * Calls OpenAI's embedding endpoint to turn text into a Float32Array.
 */
async function embed(text: string): Promise<Float32Array> {
  const response = await fetch(
    "https://api.openai.com/v1/embeddings",
    {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
        Authorization: `Bearer ${OPENAI_API_KEY}`,
      },
      body: JSON.stringify({
        model: "text-embedding-3-large", // returns 1536‑dim vectors
        input: text,
      }),
    }
  );

  if (!response.ok) {
    const err = await response.text();
    throw new Error(`OpenAI embed error ${response.status}: ${err}`);
  }

  const data = await response.json();
  // OpenAI returns an array of numbers; convert to Float32Array.
  const numbers: number[] = data.data[0].embedding;
  return new Float32Array(numbers);
}

/**
 * Main entry point – receives a user question and returns Claude's answer.
 */
export async function answerQuestion(question: string): Promise<string> {
  // 1️⃣ Turn the question into a vector.
  const queryVec = await embed(question);

  // 2️⃣ Find the 3 most similar stored chunks.
  const matches = await knnSearch(queryVec, 3);

  if (matches.length === 0) {
    return "I couldn't find any relevant information.";
  }

  // 3️⃣ Pass the question + context to Claude.
  const answer = await askClaude(question, matches, CLAUDE_API_KEY);
  return answer;
}

/* -------------------------------------------------------------
   Example usage as a simple HTTP server (Node's built‑in http).
   ------------------------------------------------------------- */
import { createServer } from "http";

const server = createServer(async (req, res) => {
  if (req.method !== "POST" || req.url !== "/ask") {
    res.writeHead(404);
    res.end("Not found");
    return;
  }

  try {
    const body = await new Promise<string>((resolve, reject) => {
      let data = "";
      req.on("data", (chunk) => (data += chunk));
      req.on("end", () => resolve(data));
      req.on("error", reject);
    });
    const { question } = JSON.parse(body);
    const answer = await answerQuestion(question);
    res.writeHead(200, { "Content-Type": "application/json" });
    res.end(JSON.stringify({ answer }));
  } catch (e) {
    console.error(e);
    res.writeHead(500);
    res.end("Server error");
  }
});

server.listen(3000, () => console.log("RAG service listening on :3000"));

What this script does

Embedding step – Calls OpenAI once per request; the cost is low compared to a full vector store query.
KNN query – Uses the DynamoDB GSI we set up earlier; the request costs 1 read capacity unit per item examined (adjust WCU accordingly).
Claude call – Sends a compact system prompt containing the three retrieved chunks.

Operational gotchas

Hot partitions – If many queries target the same docId range, DynamoDB can throttle. Mitigate by adding a random prefix to the partition key when you write chunks (sharding).
TransactWriteItems limit – When bulk‑loading many chunks, keep each transaction ≤ 100 items; otherwise you’ll hit a TransactionCanceledException.
Pricing awareness – At $0.25 per write capacity unit (WCU) in us‑east‑1, a table that writes 10 k items per day can cost a few dollars; monitor usage with CloudWatch.
TTL delay – Items scheduled for deletion may still appear in KNN results for up to 48 hours, so you might need to filter them out in application code if strict freshness is required.

In plain English – The whole pipeline is now a single DynamoDB table, one HTTP call to OpenAI, one KNN query, and one call to Claude. No extra services, no extra latency.

The Takeaway

DynamoDB’s native vector index lets you store embeddings next to your business data, removing the need for a separate vector database.

Create the table with a vector‑enabled GSI; remember to declare the embedding attribute as Binary and to base64‑encode the Float32Array.

Write and read embeddings using the @aws-sdk/lib-dynamodb package; the same client handles regular attributes and vectors.

A KNN query returns the most similar chunks; keep in mind the eventual‑consistency behavior of GSIs.

Use Node 22’s built‑in fetch to call Claude (or any LLM) after you have the context, building a short system prompt that forces the model to stay on‑topic.

Operational realities—hot partitions, TTL lag, and capacity pricing—still apply, so plan sharding and monitoring from day 1.

With these pieces in place, you have a lean, production‑ready Retrieval‑Augmented Generation service that lives entirely inside DynamoDB and plain Node.js. Happy building!

Transparency notice

This article was written with the help of an AI system — Groq (GPT OSS 120B).

Published: 2026-10-07 · Primary focus: DynamoDB

All code blocks are intended to be correct and runnable, but please verify them
against the official docs for the tools mentioned before using in production.

Find an error? Drop a comment — corrections are always welcome.

S3 Object Lambda for AI: Transforming Images on the Fly for Claude Vision, Explained Simply

Dinesh_gowtham — Tue, 06 Oct 2026 09:14:52 +0000

Developers often copy images from S3 to a server just to resize them before sending them to a LLM’s vision model. What if the transformation could happen inside S3, eliminating the extra hop? This post shows how to let Claude’s Vision API see resized images without any custom compute.

Why Object Lambda matters for AI workloads

When an AI service asks for an image, the usual pattern looks like this:

A backend pulls the original file from an S3 bucket.
The backend runs a resizing library (Sharp, Pillow, etc.).
The resized bytes travel over the network again, this time to the AI endpoint.

Each step adds latency, CPU usage, and a point of failure. Object Lambda changes the story by letting you attach a tiny piece of code (a transform function) directly to an S3 access point. The function runs inside the S3 service whenever a GET request arrives, so the caller receives the transformed data immediately.

In plain English: Think of a regular S3 bucket as a pantry that only stores food. Object Lambda turns the pantry into a kitchen that can chop, slice, or bake items the moment you ask for them—no extra chef needed.

For vision models like Claude Vision (Anthropic’s multimodal LLM), the model cares only about the final pixel dimensions, not how you got there. By moving resizing into S3:

Latency drops – one network round‑trip instead of two.
Compute cost falls – you no longer need an EC2 or Lambda worker just for image prep.
Security improves – the original high‑resolution file never leaves the bucket, only the trimmed version.

A quick term list

Object Lambda – a feature that lets you run a Lambda function during an S3 GET/HEAD request, returning the transformed result to the caller.
Access point – a named entry point for a bucket that can have its own policy and can be hooked to an Object Lambda function.
LLM (Large Language Model) – a neural network that can understand or generate text; when it also processes images it’s called a vision model.

Tip: Object Lambda works only with GET and HEAD operations. It cannot write the transformed image back to S3; the result lives only in the response stream.

Setting up an S3 Object Lambda access point

Before we can call the access point from code, we must create three pieces in the AWS console (or via CloudFormation/CDK). The steps below use the AWS CLI for clarity, but the same API calls exist in the @aws-sdk/client-s3 JavaScript package.

1. Write the transform function

The function receives the original object, resizes it to 256 × 256, and streams the result back. For simplicity we’ll use the sharp library, which works in Lambda’s Node.js runtime.

// lambda-resize.ts
import { S3ObjectLambdaEvent, S3ObjectLambdaResponse } from 'aws-lambda';
import sharp from 'sharp';

// Lambda entry point
export const handler = async (event: S3ObjectLambdaEvent): Promise<S3ObjectLambdaResponse> => {
  // The original object is streamed from S3 for us.
  const originalStream = event.getObjectContext?.inputStream;
  if (!originalStream) {
    throw new Error('No input stream – something went wrong.');
  }

  // Transform: resize to 256×256 and force JPEG output.
  const resizedStream = originalStream.pipe(
    sharp()
      .resize(256, 256)          // <-- desired dimensions
      .jpeg({ quality: 80 })     // <-- keep size reasonable
  );

  // Return the transformed stream back to the caller.
  return {
    getObjectResponse: {
      statusCode: 200,
      body: resizedStream,
      // Must set a proper Content-Type so Claude knows it’s a JPEG.
      header: { 'Content-Type': 'image/jpeg' },
    },
  };
};

Key takeaway: The Lambda never writes anything to S3; it simply reads the incoming stream, reshapes it, and sends the new bytes straight back.

2. Deploy the Lambda

# Build the TypeScript bundle (skip if you prefer plain JavaScript)
npm run build

# Create the Lambda function
aws lambda create-function \
  --function-name ResizeForClaude \
  --runtime nodejs22.x \
  --handler lambda-resize.handler \
  --zip-file fileb://dist/resizer.zip \
  --role arn:aws:iam::123456789012:role/LambdaS3ObjectLambdaRole \
  --memory-size 256 \
  --timeout 10

Tip: Give the role AWSLambdaS3ObjectLambdaExecutionRolePolicy managed policy. It includes the minimal s3-object-lambda:* permissions.

3. Create an Object Lambda access point

# First, a regular S3 access point for the bucket (replace placeholders)
aws s3control create-access-point \
  --account-id 123456789012 \
  --name images-ap \
  --bucket my-image-bucket

# Then, an Object Lambda access point that ties the Lambda to the above.
aws s3control create-access-point-for-object-lambda \
  --account-id 123456789012 \
  --name resize-ap \
  --configuration '{
    "SupportingAccessPoint": "arn:aws:s3:us-east-1:123456789012:accesspoint/images-ap",
    "TransformationConfigurations": [{
      "Actions": ["GetObject","HeadObject"],
      "ContentTransformation": {
        "AwsLambda": {
          "FunctionArn": "arn:aws:lambda:us-east-1:123456789012:function:ResizeForClaude"
        }
      }
    }]
  }'

In plain English: The resize-ap access point is like a door that says “whenever someone asks for a file, run ResizeForClaude first, then hand the result out”.

Permissions you must not forget

Object Lambda needs read access to the original bucket and the access point you just made. If either policy is missing, you’ll hit an AccessDenied error at runtime.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["s3:GetObject", "s3:ListBucket"],
      "Resource": [
        "arn:aws:s3:::my-image-bucket",
        "arn:aws:s3:::my-image-bucket/*"
      ]
    },
    {
      "Effect": "Allow",
      "Action": "s3-object-lambda:GetObject",
      "Resource": "arn:aws:s3-object-lambda:us-east-1:123456789012:accesspoint/resize-ap"
    }
  ]
}

Gotcha: Object Lambda only supports GET and HEAD. If you later try a PUT through the same access point, AWS will reject it with AccessDenied.

Calling the access point from Node.js 22

Now that the infrastructure is ready, the client side can treat the access point like any other S3 endpoint. The only twist is that the endpoint URL contains the access point name instead of the bucket name.

Install the SDK

npm i @aws-sdk/client-s3 node-fetch

Full script with comments

// fetch-resized.js
import { S3Client, GetObjectCommand } from '@aws-sdk/client-s3';
import fetch from 'node-fetch';
import { pipeline } from 'stream';
import { promisify } from 'util';

// ---------------------------------------------------------------------
// 1️⃣  Prepare the S3 client that talks to the Object Lambda access point.
//    The endpoint format is:
//    https://<access-point-name>-<account-id>.s3-object-lambda.<region>.amazonaws.com
// ---------------------------------------------------------------------
const REGION = 'us-east-1';
const ACCESS_POINT = 'resize-ap';               // <-- name from setup step
const ACCOUNT_ID = '123456789012';
const ENDPOINT = `https://${ACCESS_POINT}-${ACCOUNT_ID}.s3-object-lambda.${REGION}.amazonaws.com`;

const s3 = new S3Client({
  region: REGION,
  endpoint: ENDPOINT,           // Directs all calls to the Object Lambda AP
  forcePathStyle: false,        // Needed for the Object Lambda hostname style
});

// ---------------------------------------------------------------------
// 2️⃣  Define the object we want to fetch (original key in the bucket).
// ---------------------------------------------------------------------
const BUCKET = 'my-image-bucket'; // Not used in the request, but required by SDK signature
const KEY = 'photos/large/photo123.jpg';

// ---------------------------------------------------------------------
// 3️⃣  GetObject returns a stream. We'll pipe it straight into Claude.
// ---------------------------------------------------------------------
async function sendToClaude() {
  try {
    const getCmd = new GetObjectCommand({ Bucket: BUCKET, Key: KEY });
    const response = await s3.send(getCmd);

    // The response.Body is a Node.js readable stream containing the resized JPEG.
    const imageStream = response.Body;

    // -----------------------------------------------------------------
    // 4️⃣  Call Claude Vision. The API expects a multipart/form-data body
    //    with the image under the "image" field. We'll use fetch and a
    //    FormData polyfill (node-fetch supports it natively from v3+).
    // -----------------------------------------------------------------
    const form = new FormData();
    form.append('image', imageStream, {
      filename: 'photo_resized.jpg',
      contentType: 'image/jpeg', // Must match what Object Lambda set
    });

    const claudeResp = await fetch('https://api.anthropic.com/v1/vision', {
      method: 'POST',
      headers: {
        'x-api-key': process.env.ANTHROPIC_API_KEY,
        // fetch automatically adds the multipart boundary header
      },
      body: form,
    });

    if (!claudeResp.ok) {
      throw new Error(`Claude returned ${claudeResp.status}`);
    }

    const result = await claudeResp.json();
    console.log('Claude response:', result);
  } catch (err) {
    // ---------------------------------------------------------------
    // 5️⃣  Special handling for AccessDenied – most likely a policy
    //    mis‑configuration on the access point or the Lambda role.
    // ---------------------------------------------------------------
    if (err.name === 'AccessDenied' || (err.$metadata && err.$metadata.httpStatusCode === 403)) {
      console.error('❌ AccessDenied: check that the access point has s3:ListBucket and s3:GetObject on both the bucket and the AP.');
    } else {
      console.error('❌ Unexpected error:', err);
    }
  }
}

// Run the function
sendToClaude();

Helpful tip: When you see a 403 from the GetObjectCommand, first double‑check the bucket policy, the access point policy, and the Lambda execution role. All three need the same read permissions.

Common S3 SDK gotchas that bite beginners

Gotcha	Why it matters	How to avoid
S3 Express One Zone uses a different set of API operations (`CreateObject` vs `PutObject`).	If you switch to this storage class, your existing `@aws-sdk/client-s3` v2 code may start throwing `InvalidRequest`.	Stick with Standard unless you need the cost trade‑off, and test with the same SDK version.
Presigned URLs expire after the default 15 minutes.	Teams often generate a URL, hand it to a front‑end, and then get 403 when the user clicks later.	Explicitly set `expiresIn` to a value that covers the longest expected wait, and handle the 403 gracefully (refresh the URL).
Object Lock in Compliance mode cannot be deleted, even by root.	Accidentally enabling it on a bucket prevents any delete or overwrite, leading to storage bloat.	Enable Object Lock only on buckets that truly need immutable records, and choose Governance mode if you still need occasional deletions.
List operations are eventually consistent.	A newly uploaded image might not appear immediately in a `ListObjectsV2` call, causing “file not found” errors in pipelines.	Use S3 event notifications or a short retry loop after upload.
Transfer acceleration adds extra cost and requires a separate endpoint.	Teams enable it for faster uploads but forget the extra $ per GB, inflating budgets.	Enable only when you truly need cross‑continent speed, and monitor the cost report.

Feeding the transformed image to Claude Vision

Claude Vision expects a multipart request where the image part has a correct MIME type (image/jpeg, image/png, etc.). Because our Object Lambda already sets Content-Type: image/jpeg, we simply forward the stream.

Why streaming matters

If we first buffered the entire image into memory, we would lose the memory‑efficiency benefit of Object Lambda (which streams data directly from S3). Streaming also reduces latency because Claude can start reading the bytes while the resize is still happening.

Minimal fetch code recap

const form = new FormData();
form.append('image', imageStream, {
  filename: 'photo_resized.jpg',
  contentType: 'image/jpeg',
});

await fetch('https://api.anthropic.com/v1/vision', {
  method: 'POST',
  headers: { 'x-api-key': process.env.ANTHROPIC_API_KEY },
  body: form,
});

In plain English: Think of the FormData object as a courier bag. You place the freshly sliced image inside, seal it with the correct label (Content-Type), and hand the bag to the courier (the fetch call). The courier delivers it straight to Claude without ever opening the bag.

Handling Claude’s response

Claude returns a JSON payload that contains either a textual description or a structured result, depending on the prompt you sent. The example script logs the whole response; in production you would parse the fields you need.

Performance & cost considerations

Latency

Step	Traditional flow	Object Lambda flow
Network hop 1	S3 → EC2 (or Lambda)	S3 → Object Lambda (same region)
Compute	Resize on EC2/Lambda	Resize inside S3 service
Network hop 2	EC2/Lambda → Claude	S3 → Claude (via your app)
Total	2‑3× higher latency	~1× lower latency

Because Object Lambda runs in the same AWS edge location that serves the GET request, the extra processing time is typically under 30 ms for a 256 × 256 JPEG.

Cost

Component	Approximate cost (per 1 M requests)
S3 GET request	$0.0004
Object Lambda invocation	$0.0005 (per 1 K invocations) → $0.50
Data transfer (out to internet)	$0.09 per GB
Claude Vision API	$0.015 per 1 000 images (example)

Overall, you pay a few dollars per million image requests, far cheaper than running a dedicated Lambda just for resizing (which would add compute minutes and additional memory charges).

Tip: Keep an eye on the “Object Lambda data processed” metric in CloudWatch; it tells you how many bytes the service actually reshaped, useful for estimating future cost.

Scaling

Object Lambda scales automatically with the underlying S3 service. There’s no need to provision concurrency or worry about cold starts—AWS launches the transformation function on demand, reusing containers when possible.

When not to use Object Lambda

Write‑back needed – If you want the resized image stored for later reuse, Object Lambda alone isn’t enough; you’d need a follow‑up Lambda that writes the result to another bucket.
Complex pipelines – Multi‑step processing (e.g., watermark → resize → format conversion) may exceed the 15 second timeout limit for Object Lambda functions. In such cases chain regular Lambdas instead.

The Takeaway

What you should remember after reading this guide:

Object Lambda lets you attach a tiny transformation function to an S3 access point, turning a passive store into an on‑demand data‑shaping layer.
For vision models like Claude, resizing images inside S3 removes a whole server hop, cutting latency and compute cost.
The access point must have s3:ListBucket and s3:GetObject permissions on both the original bucket and the access point itself; otherwise you’ll hit AccessDenied.
Only GET and HEAD are supported; you can’t write the transformed file back to S3 from the same AP.
Use streaming (no full buffering) to keep memory usage low and to let Claude start processing as soon as the first bytes arrive.
Watch out for S3 SDK quirks—Express One Zone, presigned URL expiry, Object Lock, eventual consistency, and transfer acceleration can all cause surprising errors if ignored.

By wiring your image assets directly to Claude Vision through an Object Lambda access point, you get a cleaner architecture, lower latency, and a predictable cost model—without maintaining any extra servers. Happy coding!

Transparency notice

This article was written with the help of an AI system — Groq (GPT OSS 120B).

Published: 2026-10-06 · Primary focus: S3

All code blocks are intended to be correct and runnable, but please verify them
against the official docs for the tools mentioned before using in production.

Find an error? Drop a comment — corrections are always welcome.

How AWS Well‑Architected Agent Uses Generative AI to Review Your Infra‑as‑Code, Explained Simply

Dinesh_gowtham — Fri, 02 Oct 2026 08:50:43 +0000

The new Well‑Architected Agent turns a months‑long manual review into a quick AI‑driven conversation. In just a few API calls you can get actionable recommendations for your CDK or Terraform stacks, and even see security gaps highlighted by GuardDuty’s findings. This post shows exactly how to wire it up in a Node.js 22 project.

What the Well‑Architected Agent Is and How It Uses Generative AI

Why it matters – A typical Well‑Architected review requires a human expert to read your CloudFormation, CDK, or Terraform files, compare them against the five pillars (Operational Excellence, Security, Reliability, Performance Efficiency, Cost Optimization), and then write a report. That process can take weeks for a large organization. The Well‑Architected Agent replaces the human’s “eyes and brain” with a generative‑AI model that can understand code, ask clarifying questions, and produce a structured set of recommendations.

Well‑Architected Agent – a managed AWS service that accepts a workload definition (a JSON description of your infrastructure) and returns a review result that looks like a normal Well‑Architected Review but is created by an LLM (large language model).
Generative AI – an AI system that creates new content (text, code, images). In this case the model generates natural‑language advice based on the patterns it learned from millions of past reviews.

In plain English: Think of the Agent as a very knowledgeable colleague who can read your IaC (Infrastructure‑as‑Code) files, ask you a few follow‑up questions, and hand you a checklist of things to fix, all within minutes.

Minimal code to see the service metadata

// This tiny snippet shows how to ask the Well‑Architected service for its API version.
// It helps you verify that your SDK is talking to the right endpoint (us-east-1).
import { WellArchitectedClient, ListWorkloadsCommand } from "@aws-sdk/client-wellarchitected";

const client = new WellArchitectedClient({ region: "us-east-1" });

async function showServiceInfo() {
  // ListWorkloadsCommand does not need any parameters; it just returns a list.
  const resp = await client.send(new ListWorkloadsCommand({}));
  console.log("Available workloads (if any):", resp.WorkloadSummaries?.length ?? 0);
}

showServiceInfo().catch(console.error);

Comments:

WellArchitectedClient – the object that knows how to talk to the Well‑Architected API.
region – the AWS region; the Agent preview only lives in us-east-1.
ListWorkloadsCommand – a pre‑built request that the SDK turns into an HTTP call.

Enabling the Preview and Setting Up IAM Permissions

Why it matters – The Agent is still in preview, so AWS hides it behind a feature flag (wellarchitected:EnableAgentPreview) and restricts it to a single region. If you forget either step, the service returns 403 Forbidden, which looks like a generic permission error but is actually a configuration problem.

Step‑by‑step enablement

Turn on the preview flag in the AWS Management Console → Well‑Architected → Settings → “Enable Agent preview”.
Create an IAM policy that grants the preview permission and the basic Well‑Architected actions.
Attach the policy to the role or user that your Node.js script will assume.

Example IAM policy (JSON)

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowWellArchitectedAgentPreview",
      "Effect": "Allow",
      "Action": [
        "wellarchitected:EnableAgentPreview",
        "wellarchitected:CreateWorkload",
        "wellarchitected:GetWorkload",
        "wellarchitected:ListWorkloads",
        "wellarchitected:CreateWorkloadShare",
        "wellarchitected:UpdateWorkload"
      ],
      "Resource": "*"
    },
    {
      "Sid": "AllowGuardDutyRead",
      "Effect": "Allow",
      "Action": [
        "guardduty:ListFindings",
        "guardduty:GetFindings"
      ],
      "Resource": "*"
    }
  ]
}

Tip: Keep the policy narrow (specific resources) in production; the example uses "*" only for simplicity in a learning environment.

Verifying the permission

import {
  WellArchitectedClient,
  ListWorkloadsCommand,
} from "@aws-sdk/client-wellarchitected";

async function checkPermission() {
  const client = new WellArchitectedClient({ region: "us-east-1" });
  try {
    await client.send(new ListWorkloadsCommand({}));
    console.log("✅ Permission works – you can call the Well‑Architected API.");
  } catch (err: any) {
    if (err.name === "AccessDeniedException") {
      console.error("❌ 403 – missing wellarchitected:EnableAgentPreview or region mismatch.");
    } else {
      console.error("Unexpected error:", err);
    }
  }
}

checkPermission().catch(console.error);

Comments:

The try/catch distinguishes a true permission issue (403) from network or SDK problems.
Running this before any Agent call saves you a few minutes of debugging.

Calling the Agent from Node.js 22 with Native fetch

Why it matters – The preview endpoint is not yet wrapped in a high‑level SDK method, so you have to build the HTTP request yourself. Node.js 22 ships with a built‑in fetch API, so you don’t need any extra library like axios.

Preparing the workload definition

The Agent expects a JSON payload that describes the IaC you want reviewed. The easiest way is to feed it a CDK synth output (the cdk.out folder) or a Terraform plan JSON. For this example we’ll hand‑craft a tiny CDK‑like structure.

// workloadDefinition.ts
export const workloadDefinition = {
  // A human‑readable name for the review.
  WorkloadName: "MySampleApp",
  // The pillar scores can be omitted; the Agent will fill them.
  PillarPriorities: ["Security", "Reliability", "PerformanceEfficiency"],
  // A list of resources extracted from your IaC. Each entry is a simplified view.
  // In a real project you would generate this automatically from `cdk synth` or `terraform show -json`.
  Resources: [
    {
      Type: "AWS::S3::Bucket",
      LogicalId: "AssetsBucket",
      Properties: {
        VersioningConfiguration: { Status: "Enabled" },
        PublicAccessBlockConfiguration: { BlockPublicAcls: true }
      }
    },
    {
      Type: "AWS::Lambda::Function",
      LogicalId: "ProcessorFunction",
      Properties: {
        Runtime: "nodejs20.x",
        Handler: "index.handler",
        Timeout: 30
      }
    }
  ]
};

The fetch call

import { workloadDefinition } from "./workloadDefinition.js";
import { getSignedUrl } from "@aws-sdk/s3-request-presigner";
import { S3Client, PutObjectCommand } from "@aws-sdk/client-s3";

// Helper: put a temporary JSON file in S3 so the Agent can fetch it.
// The preview endpoint only accepts a URL to a JSON document.
async function uploadPayloadToS3(payload: object, bucket: string, key: string) {
  const s3 = new S3Client({ region: "us-east-1" });
  const command = new PutObjectCommand({
    Bucket: bucket,
    Key: key,
    Body: JSON.stringify(payload),
    ContentType: "application/json",
  });
  await s3.send(command);
  // Generate a signed URL that lasts 10 minutes.
  return getSignedUrl(s3, new PutObjectCommand({ Bucket: bucket, Key: key }), {
    expiresIn: 600,
  });
}

async function callWellArchitectedAgent() {
  const payloadUrl = await uploadPayloadToS3(workloadDefinition, "my-preview-bucket", "payload.json");

  // The preview endpoint (still in preview) lives at this fixed path.
  const endpoint = "https://wellarchitected.us-east-1.amazonaws.com/preview/workload";

  // Build the POST request body.
  const body = JSON.stringify({
    // The Agent needs a URL it can download the JSON from.
    WorkloadDefinitionUrl: payloadUrl,
    // Optional: a short description that the AI can reference.
    Description: "Review of a simple S3 + Lambda app."
  });

  // Native fetch with IAM‑based signing (SigV4) is done via the @aws-sdk/signature-v4 package.
  // For brevity we use a pre‑signed request approach: the SDK can create a signed URL for us.
  const { SignatureV4 } = await import("@aws-sdk/signature-v4");
  const { HttpRequest } = await import("@aws-sdk/protocol-http");
  const { defaultProvider } = await import("@aws-sdk/credential-provider-node");

  const signer = new SignatureV4({
    credentials: defaultProvider(),
    region: "us-east-1",
    service: "wellarchitected",
    sha256: await import("crypto").then(m => m.createHash),
  });

  const request = new HttpRequest({
    protocol: "https:",
    hostname: "wellarchitected.us-east-1.amazonaws.com",
    path: "/preview/workload",
    method: "POST",
    headers: {
      "content-type": "application/json",
      "accept": "application/json",
    },
    body,
  });

  const signed = await signer.sign(request);
  const response = await fetch(`https://${signed.hostname}${signed.path}`, {
    method: signed.method,
    headers: signed.headers,
    body: signed.body,
  });

  if (response.status !== 202) {
    console.error(`❌ Unexpected status ${response.status}`);
    const text = await response.text();
    console.error("Body:", text);
    return;
  }

  // The API returns an operation ID that we poll later.
  const { operationId } = await response.json();
  console.log("✅ Review accepted, operationId:", operationId);
  return operationId;
}

Comments:

Signed request – AWS APIs require SigV4 signing; we create a signed HttpRequest then feed it to fetch.
202‑Accepted – means the review is queued. The response body contains operationId to track progress.
S3 payload – the Agent does not accept raw JSON in the POST body; it only reads from a URL, so we store the definition in a temporary S3 object and generate a signed URL.

Key takeaway: The preview workflow is a two‑step dance: (1) upload your IaC description somewhere the service can reach, (2) tell the Agent where to find it via a signed POST.

Parsing the Agent’s Recommendations and Correlating GuardDuty Findings

Why it matters – The Agent’s output is a list of recommendations, each tied to a resource and a pillar. Those recommendations are helpful, but you also want to know whether a security finding from GuardDuty overlaps with an AI‑suggested fix. Correlating the two gives you a single view of what the AI thinks and what AWS security services have actually detected.

Polling for the final result

import {
  WellArchitectedClient,
  GetReviewCommand,
} from "@aws-sdk/client-wellarchitected";

async function pollReviewResult(operationId: string, timeoutSec = 120) {
  const client = new WellArchitectedClient({ region: "us-east-1" });
  const start = Date.now();

  while (true) {
    const resp = await client.send(new GetReviewCommand({ WorkloadId: operationId }));
    if (resp.Status === "COMPLETED") {
      console.log("✅ Review finished.");
      return resp.Review; // contains Recommendations array
    }
    if (Date.now() - start > timeoutSec * 1000) {
      throw new Error("Timed out waiting for review to complete");
    }
    console.log("⏳ Still processing…");
    await new Promise(r => setTimeout(r, 5_000)); // wait 5 seconds
  }
}

Comments:

GetReviewCommand – fetches the review associated with the operationId.
The loop checks Status until it becomes COMPLETED.

Pulling GuardDuty findings

import {
  GuardDutyClient,
  ListFindingsCommand,
  GetFindingsCommand,
} from "@aws-sdk/client-guardduty";

async function fetchGuardDutyFindings(detectorId: string) {
  const client = new GuardDutyClient({ region: "us-east-1" });

  // First, list the finding IDs (max 50 per call for simplicity).
  const listResp = await client.send(
    new ListFindingsCommand({ DetectorId: detectorId })
  );

  if (!listResp.FindingIds?.length) {
    console.log("✅ No GuardDuty findings.");
    return [];
  }

  // Then, retrieve the full finding objects.
  const getResp = await client.send(
    new GetFindingsCommand({
      DetectorId: detectorId,
      FindingIds: listResp.FindingIds,
    })
  );

  return getResp.Findings ?? [];
}

Comments:

DetectorId – the GuardDuty detector that belongs to your account; you can find it in the GuardDuty console or via ListDetectorsCommand.
The code pulls both IDs and details so we can match on resource ARN.

Merging the two data sets

type Recommendation = {
  ResourceId: string; // matches the LogicalId from the workload definition
  Pillar: string;
  Text: string;
};

type GuardDutyFinding = {
  Id: string;
  Resource: {
    InstanceDetails?: { InstanceArn?: string };
    AccessKeyDetails?: { AccessKeyId?: string };
    // GuardDuty can report many resource types; we’ll look at the ARN field.
    ResourceArn?: string;
  };
  Severity: number;
  Title: string;
};

function correlate(
  recommendations: Recommendation[],
  findings: GuardDutyFinding[]
) {
  const map: Record<string, { recommendation?: Recommendation; findings: GuardDutyFinding[] }> = {};

  // Index by resource ID (LogicalId) for quick lookup.
  recommendations.forEach(rec => {
    map[rec.ResourceId] = { recommendation: rec, findings: [] };
  });

  // Walk through each finding and see if its ARN contains a known logical ID.
  findings.forEach(f => {
    const arn = f.Resource?.ResourceArn ?? "";
    // Simple heuristic: the logical ID appears somewhere in the ARN string.
    const matchKey = Object.keys(map).find(id => arn.includes(id));
    if (matchKey) {
      map[matchKey].findings.push(f);
    }
  });

  return map;
}

Comments:

The correlation uses a string contains check, which works for most CDK‑generated ARNs because the logical ID appears in the resource name.
This is a lightweight analogy: imagine matching a list of “to‑do” items (recommendations) with a list of “found problems” (GuardDuty) by looking for a common word in both notes.

Example of printing the merged view

async function main() {
  const operationId = await callWellArchitectedAgent(); // from previous section
  if (!operationId) return;

  const review = await pollReviewResult(operationId);
  const recommendations = review?.Recommendations?.map(r => ({
    ResourceId: r.Resource?.LogicalId ?? "unknown",
    Pillar: r.Pillar ?? "unknown",
    Text: r.Text ?? "",
  })) ?? [];

  // Replace with your own detector ID.
  const detectorId = "12abc34def567ghijk890lmn";
  const findings = await fetchGuardDutyFindings(detectorId);

  const merged = correlate(recommendations, findings);
  for (const [resourceId, entry] of Object.entries(merged)) {
    console.log(`\n🔧 Resource: ${resourceId}`);
    if (entry.recommendation) {
      console.log(`  • AI advice (${entry.recommendation.Pillar}): ${entry.recommendation.Text}`);
    }
    if (entry.findings.length) {
      console.log(`  • GuardDuty alerts (${entry.findings.length}):`);
      entry.findings.forEach(f => {
        console.log(`      - ${f.Title} (severity ${f.Severity})`);
      });
    } else {
      console.log("  • No GuardDuty findings.");
    }
  }
}
main().catch(console.error);

Helpful tip: Run the script with NODE_OPTIONS=--enable-source-maps so stack traces point to the original TypeScript line numbers, making debugging easier.

Best‑Practice Loop: Auto‑Update Your IaC Based on AI Feedback

Why it matters – The Agent is great at pointing out missing encryption, overly permissive policies, or absent monitoring. If you treat its output as a one‑off report, you miss the chance to keep your codebase continuously aligned with best practices. By looping the review, applying fixes, and re‑submitting, you create a feedback cycle similar to a lint‑and‑fix pipeline for code quality.

High‑level loop description

Run the Agent → get recommendations.
Parse recommendations → identify which resources need a change.
Programmatically edit the CDK (or Terraform) source files.
Commit the changes (Git) and optionally open a Pull Request.
Re‑run the Agent to verify that the issue is resolved.

Think of it like a spell‑checker that automatically applies suggested corrections and then re‑checks the document.

Code sketch that updates a CDK stack file

import * as fs from "node:fs";
import * as path from "node:path";

/**
 * Very simple helper that adds versioning to an S3 bucket if the AI
 * recommendation mentions "Enable versioning".
 */
function applyVersioningIfNeeded(
  stackFile: string,
  recommendations: Recommendation[]
) {
  let content = fs.readFileSync(stackFile, "utf-8");

  recommendations.forEach(rec => {
    if (
      rec.ResourceId === "AssetsBucket" &&
      rec.Text.toLowerCase().includes("enable versioning")
    ) {
      // Insert a line right after the bucket definition (naïve string replace).
      const bucketPattern = /new s3\.Bucket\(this,\s*["']AssetsBucket["'],\s*{[^}]*}\);/s;
      const match = bucketPattern.exec(content);
      if (match) {
        const insertion = `\n    versioned: true, // Added by Well‑Architected Agent`;
        const updated = match[0].replace(/}\);/, `${insertion}\n});`);
        content = content.replace(bucketPattern, updated);
        console.log("✅ Added versioning flag to AssetsBucket.");
      }
    }
  });

  fs.writeFileSync(stackFile, content, "utf-8");
}

// Example usage
applyVersioningIfNeeded(
  path.resolve("lib/my-sample-app-stack.ts"),
  [
    {
      ResourceId: "AssetsBucket",
      Pillar: "Security",
      Text: "Enable versioning on the S3 bucket to protect against accidental deletions."
    }
  ]
);

Comments:

This is illustrative only; a production‑grade tool would manipulate the CDK AST (Abstract Syntax Tree) rather than plain string replacement.
The function reads the stack file, looks for a bucket definition that matches the logical ID, and injects versioned: true.

Automating the loop with npm scripts

{
  "scripts": {
    "review": "node ./scripts/run-review.js",      // the script from previous sections
    "apply-fixes": "node ./scripts/apply-fixes.js",
    "ci": "npm run review && npm run apply-fixes && git diff --exit-code"
  }
}

Running npm run ci will:

Call the Agent,
Apply any simple fixes,
Fail the CI job if the repository changed (meaning a fix was applied).

In plain English: The CI pipeline becomes a “self‑healing” guard that refuses to merge code unless the AI‑driven review is clean.

The Takeaway

Key points you should walk away with

The Well‑Architected Agent is a preview‑only service (us‑east‑1) that uses a generative‑AI model to turn IaC into a structured review.
Enabling the preview flag and granting wellarchitected:EnableAgentPreview are mandatory; otherwise you’ll see a misleading 403 error.
Because the preview endpoint expects a URL, you must upload your workload definition to S3 and sign the request before posting with native fetch.
The API returns a 202 response with an operationId; poll GetReview until the status becomes COMPLETED to fetch the recommendations.
Correlating those recommendations with GuardDuty findings gives a unified security picture; a simple string‑contains match works for most CDK‑generated ARNs.
Embedding the review into a repeatable loop (review → apply → re‑review) lets you keep your CDK/Terraform code continuously aligned with AWS best practices, much like a lint‑and‑fix cycle for source code.

By following the steps above, you can replace a manual, weeks‑long architecture review with a few lines of code that run on every pull request, surface real security findings, and even auto‑apply low‑risk fixes. Happy building!

Transparency notice

This article was written with the help of an AI system — Groq (GPT OSS 120B).

Published: 2026-10-02 · Primary focus: WellArchitectedAgent

All code blocks are intended to be correct and runnable, but please verify them
against the official docs for the tools mentioned before using in production.

Find an error? Drop a comment — corrections are always welcome.

EventBridge Pipes for AI Coding Assistants: Automate Multi‑File Code Generation on PR Open, Explained Simply

Dinesh_gowtham — Thu, 01 Oct 2026 09:19:28 +0000

Pull‑request reviewers spend minutes copying snippets into ChatGPT, then manually applying the suggestions. What if the whole edit could happen automatically, right after the PR is created? By wiring EventBridge Pipes to an ECS task you can turn an AI coding assistant into a CI‑native, auditable workflow.

Why Event‑Driven AI Coding Beats Manual Prompting

In plain English: When a developer opens a PR, the system should react immediately, without a person having to type a prompt.

Human reviewers are good at spotting intent, but they waste time moving data between tools. An event‑driven design treats every PR open as a signal (an event) that can trigger other services automatically.

Speed: The moment the PR exists, the AI can start working. No waiting for a reviewer to paste code.
Repeatability: The same logic runs for every PR, so you get a consistent experience across the team.
Auditability: Each AI‑generated commit is part of the Git history, so you can review, revert, or approve it like any other change.

Think of it like a kitchen assembly line. The moment a raw ingredient (the PR) arrives on the conveyor belt, a robot (the ECS task) grabs it, processes it (calls Claude), and places the finished dish (the edited code) back on the belt for the chef (the reviewer) to taste.

Key terms (first use)

Event: A piece of data that says “something happened”, e.g., “a PR was opened”.
EventBridge: Amazon’s service that receives events and routes them to targets.
Pipe: A built‑in connection inside EventBridge that links a source directly to a target, optionally applying a filter or transformation.
ECS (Elastic Container Service): A managed container platform; Fargate is the server‑less mode that runs containers without you managing servers.
Bedrock: AWS’s hosted platform for foundation models such as Claude, accessed through a normal HTTP request.

By letting the AI sit inside the same event pipeline that already moves code around, you remove the manual copy‑paste step entirely.

Building the EventBridge Pipe: From GitHub webhook to ECS task

Tip: Create the pipe once and reuse it for every PR. The pipe itself is cheap; the heavy lifting happens in the ECS task.

What the pipe does

Source: API Gateway forwards a GitHub webhook (pull_request.opened) to EventBridge.
Filter (optional): Only let events that contain changed .js or .ts files through.
Target: An ECS Fargate task that runs a small Node.js program.

The pipe definition lives in code so you can version‑control it. Below is a minimal, complete example that creates the pipe using the @aws-sdk/client-eventbridge package.

import {
  EventBridgeClient,
  CreatePipeCommand,
  CreatePipeCommandInput,
} from "@aws-sdk/client-eventbridge";

// 1️⃣ Create a client – it knows which AWS region to talk to.
const ebClient = new EventBridgeClient({ region: "us-east-1" });

// 2️⃣ Define the pipe.
// - Source: an EventBridge event bus that API Gateway will put events onto.
// - Target: an ECS task run on Fargate.
// - Filter: keep only PR events with at least one .js/.ts file.
const pipeParams: CreatePipeCommandInput = {
  Name: "PRToAICodingPipe",
  RoleArn: "arn:aws:iam::123456789012:role/EventBridgePipeRole", // grants EB to start ECS tasks
  Source: "arn:aws:events:us-east-1:123456789012:event-bus/default",
  SourceParameters: {
    // No special settings needed for a simple event bus source.
  },
  Target: "arn:aws:ecs:us-east-1:123456789012:cluster/ai-coding-cluster",
  TargetParameters: {
    // Tell EventBridge how to launch the task.
    ECSParameters: {
      TaskDefinitionArn: "arn:aws:ecs:us-east-1:123456789012:task-definition/ai-coding-task:1",
      LaunchType: "FARGATE",
      NetworkConfiguration: {
        awsvpcConfiguration: {
          Subnets: ["subnet-abc123"], // pick a subnet with internet access
          AssignPublicIp: "ENABLED",
        },
      },
      // Pass the whole event as JSON to the container's STDIN (or env var)
      TaskOverride: {
        ContainerOverrides: [
          {
            Name: "ai-coding-container",
            // EventBridge will inject the event as the first argument.
            Command: ["node", "run.js"],
          },
        ],
      },
    },
  },
  // Simple filter: only allow events where detail.action === "opened"
  // and at least one file ends with .js or .ts.
  // Note: The filter expression must evaluate within 5 seconds.
  FilterCriteria: {
    Filters: [
      {
        Pattern: JSON.stringify({
          "detail": {
            "action": ["opened"],
            "changed_files": [{ "suffix": [".js", ".ts"] }],
          },
        }),
      },
    ],
  },
  // Optional: dead‑letter queue for failures.
  // We'll set this up later in the “Gotchas” note.
};

async function createPipe() {
  try {
    const cmd = new CreatePipeCommand(pipeParams);
    const response = await ebClient.send(cmd);
    console.log("Pipe created:", response.Arn);
  } catch (err) {
    console.error("Failed to create pipe:", err);
  }
}

createPipe();

What the code does, line by line

Import the SDK classes you need.
Instantiate an EventBridgeClient pointing at your region.
Build a CreatePipeCommandInput object that describes source, target, and filter.
Set RoleArn to an IAM role that lets EventBridge call ecs:RunTask.
Define a 5‑second filter that looks for PR‑opened events with JavaScript/TypeScript files.
Call CreatePipeCommand and log the resulting ARN.

Gotchas specific to this step

The filter evaluation limit (5 seconds) means you cannot run heavy regexes or large JSON traversals. Keep the pattern tiny.
If you later add a new field to the GitHub webhook payload, the pipe might silently stop matching. Test changes in a sandbox.
Cross‑account pipes require you to add a resource‑based policy on the target ECS cluster; missing it results in “AccessDenied” errors that are hard to trace.

Using S3 as a Prompt Cache with native fetch

Key takeaway: Store the raw file contents in S3 so the container can download them quickly, even if the PR contains dozens of files.

The container needs the exact code it will send to Claude. Instead of calling the GitHub API from inside the task (which adds latency and extra auth), we let the webhook handler upload the changed files to an S3 bucket before the event hits the pipe. The bucket acts as a prompt cache—a place where the AI can retrieve the same snapshot later.

Uploading the files (the webhook side)

import { S3Client, PutObjectCommand } from "@aws-sdk/client-s3";

// The bucket where we keep PR snapshots.
const s3 = new S3Client({ region: "us-east-1" });

async function cacheChangedFiles(prNumber: number, files: Record<string, string>) {
  const promises = Object.entries(files).map(async ([path, content]) => {
    const key = `pr-${prNumber}/${path}`; // e.g., pr-42/src/index.ts
    const cmd = new PutObjectCommand({
      Bucket: "ai-coding-prompt-cache",
      Key: key,
      Body: content,
      // Include the PR number in metadata so we can detect stale caches.
      Metadata: { "pr-number": prNumber.toString() },
    });
    await s3.send(cmd);
  });
  await Promise.all(promises);
}

files is a map where each key is the file path and each value is the file contents.
We store each file under a folder named after the PR number; this makes versioning simple.
Adding Metadata with the PR number helps us later decide whether a cached prompt is still fresh.

Pulling the files inside the ECS container

Inside the container we use the built‑in fetch API (available in Node 20+). No extra SDK needed.

// run.js – entry point for the Fargate task.
import { readFileSync } from "fs";
import { createReadStream } from "stream";

// The event arrives on STDIN as a JSON string.
const event = JSON.parse(readFileSync(0, "utf-8"));
const prNumber = event.detail.pull_request.number;

// Helper to download a single file from S3.
async function downloadFile(key) {
  const url = `https://${process.env.S3_BUCKET}.s3.amazonaws.com/${key}`;
  const response = await fetch(url);
  if (!response.ok) throw new Error(`Failed to fetch ${key}: ${response.status}`);
  return await response.text();
}

// Build the prompt by concatenating all changed files.
async function buildPrompt() {
  // List of keys we stored earlier – in a real implementation you could
  // read a manifest file from S3, but for brevity we assume we know the list.
  const changedFiles = ["src/index.ts", "src/util.js"]; // example
  const parts = [];

  for (const file of changedFiles) {
    const key = `pr-${prNumber}/${file}`;
    const content = await downloadFile(key);
    parts.push(`--- ${file} ---\n${content}`);
  }

  // Simple instruction for Claude.
  return `You are a coding assistant. Update the following files to address the PR description.\n${parts.join("\n")}`;
}

(async () => {
  const prompt = await buildPrompt();
  console.log("Prompt ready, length:", prompt.length);
  // Next step: call Claude (see next section).
})();

Explanation of the code

The event is read from STDIN because EventBridge pipes can forward the event that way.
downloadFile builds a public S3 URL and uses fetch to retrieve the file. (Make the bucket public‑read or use a signed URL; the latter is recommended for security, but omitted here for clarity.)
buildPrompt loops over a known list of changed files, downloads each, and stitches them together with a simple delimiter (--- filename ---). This structure helps Claude understand file boundaries.

Gotchas for the S3 cache

Version awareness: If a PR is updated quickly, the bucket may still hold the old version. Include the PR number (or git sha) in the S3 object key or metadata and check it before using the cached file.
Stale suggestions: If the pipe fires multiple times for the same PR (e.g., due to re‑opens), you may end up with duplicate branches. Guard against it by checking whether a branch already exists on GitHub.
Permissions: The ECS task’s task role must have s3:GetObject on the bucket. Mixing up the task role with the execution role is a common source of silent failures.

Calling Claude via Bedrock inside the ECS container

In plain English: Claude lives behind an HTTP endpoint provided by Bedrock. From the container we just fetch that endpoint, send the prompt, and receive a text response.

Bedrock does not require a separate SDK; it follows standard AWS SigV4 authentication. For brevity we’ll use the aws4 package to sign the request. You could also use the official @aws-sdk/signature-v4 helpers.

npm install aws4

import aws4 from "aws4";
import { readFileSync } from "fs";

const BEDROCK_ENDPOINT = "bedrock-runtime.us-east-1.amazonaws.com";
const MODEL_ID = "anthropic.claude-v2"; // example model

async function callClaude(prompt) {
  const body = JSON.stringify({
    prompt,
    max_tokens_to_sample: 1024,
    temperature: 0.2,
    // other model‑specific parameters...
  });

  const request = {
    host: BEDROCK_ENDPOINT,
    path: `/model/${MODEL_ID}/invoke`,
    method: "POST",
    headers: {
      "Content-Type": "application/json",
    },
    body,
  };

  // Sign the request with the task role's credentials (available via env vars).
  const signed = aws4.sign(request, {
    accessKeyId: process.env.AWS_ACCESS_KEY_ID,
    secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
    sessionToken: process.env.AWS_SESSION_TOKEN,
  });

  const response = await fetch(`https://${BEDROCK_ENDPOINT}${request.path}`, {
    method: request.method,
    headers: signed.headers,
    body,
  });

  if (!response.ok) {
    const err = await response.text();
    throw new Error(`Claude request failed: ${response.status} ${err}`);
  }

  const data = await response.json();
  // Claude returns `{ "completion": "..." }`
  return data.completion;
}

// Example usage inside the IIFE from the previous section:
(async () => {
  const prompt = await buildPrompt();
  const suggestion = await callClaude(prompt);
  console.log("Claude suggestion received:", suggestion.slice(0, 200), "...");
  // Next: write suggestion back to GitHub.
})();

Step‑by‑step

Build a JSON payload containing the prompt and model parameters.
Construct a raw HTTP request object.
Sign the request with SigV4 (AWS’s request‑signing protocol) using the task role credentials that are automatically injected into the container.
fetch the signed request to the Bedrock endpoint.
Parse the JSON response and extract the completion field, which holds the edited code.

Analogy

Think of the container as a post‑office clerk. The clerk writes a letter (the prompt), stamps it with a secret seal (the SigV4 signature), and drops it into the mailbox (the Bedrock endpoint). Bedrock reads the sealed letter, writes a reply, and puts it back in the mailbox. The clerk then reads the reply.

Gotchas for Bedrock calls

IAM permission mix‑up: The task role needs bedrock:InvokeModel. If you mistakenly grant it to the execution role, the request will be rejected with a 403 error that looks like a network failure.
Cold‑start delay: The first invocation of a new model may take several seconds because Bedrock needs to spin up the model container. Design your pipe with a modest timeout (e.g., 30 seconds) and consider a retry strategy.
Response size limit: Bedrock caps the response at a few megabytes. For very large multi‑file diffs you may need to chunk the request.

Writing the Suggested Changes Back to GitHub

Tip: Use the official @octokit/rest library; it handles pagination and retries for you.

The container now has a text block that contains the edited files. We need to turn that into a new branch and a PR (or push directly to the existing PR). The simplest approach:

Create a new branch based on the PR’s head SHA.
For each file in the suggestion, call PUT /repos/:owner/:repo/contents/:path.
Open a new PR that targets the original branch.

Below is a compact, fully commented script that does exactly that.

npm install @octokit/rest

import { Octokit } from "@octokit/rest";

const octokit = new Octokit({
  auth: process.env.GITHUB_TOKEN, // a fine‑grained PAT with repo access
});

/**
 * Parse Claude's output into a map of file paths → new content.
 * Expected format:
 *   --- src/index.ts ---
 *   <new content>
 *   --- src/util.js ---
 *   <new content>
 */
function parseClaudeOutput(output) {
  const fileMap = {};
  const sections = output.split(/^---\s+/m); // split on lines starting with "--- "
  for (const sec of sections) {
    if (!sec.trim()) continue;
    const [header, ...body] = sec.split("\n");
    const path = header.trim().replace("---", "").trim(); // e.g., src/index.ts
    fileMap[path] = body.join("\n").trim();
  }
  return fileMap;
}

async function pushChanges(prNumber, suggestion) {
  // 1️⃣ Get PR details to know repo, owner, and base SHA.
  const { data: pr } = await octokit.rest.pulls.get({
    owner: "my-org",
    repo: "my-repo",
    pull_number: prNumber,
  });

  const newBranch = `ai/suggested-${prNumber}`;
  // 2️⃣ Create the branch if it does not exist.
  try {
    await octokit.rest.git.createRef({
      owner: "my-org",
      repo: "my-repo",
      ref: `refs/heads/${newBranch}`,
      sha: pr.head.sha, // start from the current PR head
    });
    console.log(`Created branch ${newBranch}`);
  } catch (e) {
    if (e.status === 422) {
      console.log(`Branch ${newBranch} already exists, proceeding.`);
    } else {
      throw e;
    }
  }

  // 3️⃣ Parse Claude's output.
  const files = parseClaudeOutput(suggestion);

  // 4️⃣ For each file, push the new content.
  for (const [path, content] of Object.entries(files)) {
    // Get the blob SHA of the current file (optional, helps with optimistic locking).
    let sha;
    try {
      const { data: fileInfo } = await octokit.rest.repos.getContent({
        owner: "my-org",
        repo: "my-repo",
        path,
        ref: pr.head.sha,
      });
      sha = fileInfo.sha;
    } catch (e) {
      // If the file does not exist yet, leave sha undefined – GitHub will create it.
    }

    await octokit.rest.repos.createOrUpdateFileContents({
      owner: "my-org",
      repo: "my-repo",
      path,
      message: `AI: apply suggested changes for PR #${prNumber}`,
      content: Buffer.from(content, "utf-8").toString("base64"),
      branch: newBranch,
      sha, // optional, helps avoid race conditions
    });
    console.log(`Updated ${path}`);
  }

  // 5️⃣ Open a PR from the AI branch to the original base branch.
  const { data: newPR } = await octokit.rest.pulls.create({
    owner: "my-org",
    repo: "my-repo",
    title: `[AI] Suggested changes for #${prNumber}`,
    head: newBranch,
    base: pr.base.ref,
    body: "Automated suggestions generated by Claude via EventBridge Pipe.",
  });

  console.log(`Created AI PR #${newPR.number}`);
}

// In the IIFE from the previous sections:
(async () => {
  const prompt = await buildPrompt();
  const suggestion = await callClaude(prompt);
  await pushChanges(event.detail.pull_request.number, suggestion);
})();

Explanation

parseClaudeOutput turns the raw text into a dictionary of file paths and new content.
We first fetch PR metadata to know the repo, owner, and current head SHA.
createRef makes a new branch named ai/suggested-<pr-number>. If the branch already exists (perhaps because the pipe ran before), we reuse it.
For each file we read the current SHA (if any) and then call createOrUpdateFileContents, which both creates new files and updates existing ones.
Finally we open a new PR that points the AI branch to the original base branch. Reviewers can merge it or request changes just like any other PR.

Gotchas for the GitHub step

Permission scope: The GitHub token must have contents:write and pull_requests:write. A token that only has repo:status will fail silently.
Rate limits: If many PRs open at once, you may hit GitHub’s API limits. Implement exponential back‑off (Octokit does it automatically for 429 responses).
Branch naming collisions: Including the PR number in the branch name avoids clashes across multiple concurrent runs.

The Takeaway

In plain English: You can turn the act of “asking an AI for a code fix” into a fully automated, auditable pipeline that runs the moment a pull request appears.

EventBridge Pipes let you connect a GitHub webhook directly to an ECS Fargate task without writing glue code.
A tiny S3 bucket works as a version‑aware prompt cache, keeping the container’s input fast and consistent.
Calling Claude via Bedrock requires only a signed fetch request; no special SDK is needed.
After Claude returns edited code, the container uses Octokit to create a new branch and PR, preserving the change history.
Remember the hidden pitfalls: filter timeouts, dead‑letter queues for ECS failures, S3 version staleness, IAM role confusion, and GitHub rate limits.

By wiring these pieces together you get a CI‑native AI assistant that:

Reacts instantly to PR creation.
Generates multi‑file edits in a single, reproducible run.
Stores the result as a proper Git commit, ready for human review.

Give it a try on a low‑risk repository, watch the logs, and iterate on the prompt format. Once the pipeline is stable, you’ll find that the manual copy‑paste step disappears, freeing reviewers to focus on design and logic instead of tedious typing. Happy automating!

Transparency notice

This article was written with the help of an AI system — Groq (GPT OSS 120B).

Published: 2026-10-01 · Primary focus: EventBridge

All code blocks are intended to be correct and runnable, but please verify them
against the official docs for the tools mentioned before using in production.

Find an error? Drop a comment — corrections are always welcome.

Claude Tool Use Explained: How to Let the Model Call Your API from Node.js

Dinesh_gowtham — Wed, 30 Sep 2026 08:53:20 +0000

Imagine Claude not just answering questions but actually reaching out to your own services—creating a seamless AI‑powered workflow. In just a few lines of TypeScript you can expose a Bedrock model, define a tool, and watch the model invoke it in real time.

Understanding Claude’s Tool Use Contract

Why this matters – A large language model (LLM) like Claude can generate plain text, but many real‑world tasks need it to talk to other systems: look up a user, write a file, trigger a deployment, etc. Claude’s tool use feature is the bridge that turns a chat‑style model into a programmable assistant. Think of it like giving the model a set of remote‑control buttons; when the model presses a button, your code runs and sends the result back.

The contract in plain language

Tool definition – a JSON schema that tells Claude what arguments a function expects (e.g., an id string).
tool_choice – a request field that says “let Claude decide if it needs a tool (auto) or force a specific one (any).”
tool_use block – the exact shape Claude sends when it wants to call a tool: {type: "tool_use", name: "...", input: {...}}.
tool_result block – what you send back containing the function’s output so Claude can continue the conversation.

If any of these pieces are missing or malformed, Claude falls back to plain text and you never see a tool_use block. That is the most common gotcha for newcomers.

In plain English: Claude only reaches for your API when you give it a clear “button” (tool) and tell it it’s allowed to press it (tool_choice: "auto"). Missing the button or the permission means Claude just talks.

Analogy

Picture a kitchen robot that can stir, chop, or fetch ingredients. The recipe (your prompt) lists the possible actions (tool definitions). If you forget to list “fetch” or you never give the robot permission to leave the counter, it will simply describe the dish instead of actually bringing the carrots.

Setting Up Bedrock Access in Node.js

Why start here – Before Claude can call a tool, your Node.js program must be able to talk to Amazon Bedrock, the service that hosts Claude. Getting the client right avoids the “rate limit” and “streaming parse” pitfalls that often trip up teams.

Install the SDKs

npm install @aws-sdk/client-bedrock-runtime @aws-sdk/client-api-gateway axios

@aws-sdk/client-bedrock-runtime – the official JavaScript client for Bedrock’s inference endpoint.
@aws-sdk/client-api-gateway – lets you query the configuration of your API Gateway if you need it (e.g., to verify stage settings).
axios – a tiny HTTP library used later to call the user‑profile endpoint.

Create a Bedrock client

// src/bedrock.ts
import {
  BedrockRuntimeClient,
  InvokeModelCommand,
} from "@aws-sdk/client-bedrock-runtime";

// The client automatically picks up AWS credentials from the environment
// (e.g., ~/.aws/credentials, ECS task role, or Lambda execution role).
const bedrock = new BedrockRuntimeClient({
  region: "us-east-1", // Choose the region where Claude is enabled
  // You can add a custom retry strategy here if you hit per‑minute token limits.
});

export { bedrock, InvokeModelCommand };

Gotcha #1 – Token limits are per minute, not per request

Bedrock caps the number of tokens you can send across all requests each minute. If your app spikes, you may see ThrottlingException. A simple mitigation is to add a short delay (e.g., 200 ms) between calls or use a token bucket library.

Tip: Log the x-amzn-RequestId header on every response. It helps you correlate throttling errors with specific requests in CloudWatch.

Gotcha #2 – Streaming responses need manual SSE parsing

Bedrock can stream partial outputs via Server‑Sent Events (SSE). The Node SDK returns a raw Readable stream; you must listen for data events and concatenate the JSON fragments yourself. In this tutorial we use the non‑streaming mode for simplicity, but keep the note in mind when you later add real‑time UI.

Defining a Tool Schema and Prompt

Why define a schema – Claude needs to know what a tool looks like before it can consider using it. The schema is the contract that both Claude and your code agree on. It is similar to an API specification: it tells the model the name of the tool, what arguments it expects, and the types of those arguments.

The tool we will expose

We want Claude to fetch a user profile from an API Gateway endpoint:

Tool name – getUserProfile
Argument – userId (a string, e.g., "abc123")

Prompt with tool definition

// src/prompt.ts
export const systemPrompt = `
You are an assistant that can call external services when needed.
When you need information about a user, use the provided tool.
`;

export const toolDefinition = {
  name: "getUserProfile",
  description: "Retrieve a user profile from the company directory.",
  input_schema: {
    type: "object",
    properties: {
      userId: {
        type: "string",
        description: "The unique identifier of the user, e.g., 'abc123'.",
      },
    },
    required: ["userId"],
  },
};

Full request payload

// src/invokeClaude.ts
import { bedrock, InvokeModelCommand } from "./bedrock";
import { systemPrompt, toolDefinition } from "./prompt";

export async function askClaude(question: string) {
  // The model will see the system prompt, the tool definition, and the user query.
  const payload = {
    // “anthropic.claude-v2” is the model identifier; replace with the version you have access to.
    modelId: "anthropic.claude-v2",
    // The messages array follows the OpenAI‑compatible chat format.
    messages: [
      { role: "system", content: systemPrompt },
      { role: "assistant", tool: toolDefinition, content: "" }, // tool definition block
      { role: "user", content: question },
    ],
    // Tell Claude it can decide on its own whether to call a tool.
    tool_choice: "auto",
  };

  // Bedrock expects a Uint8Array of JSON.
  const command = new InvokeModelCommand({
    body: Buffer.from(JSON.stringify(payload)),
    contentType: "application/json",
    accept: "application/json",
    // The ARN for Claude in your account (replace <account-id> and region).
    // Example: arn:aws:bedrock:us-east-1:123456789012:foundation-model/anthropic.claude-v2
    modelId: "arn:aws:bedrock:us-east-1:123456789012:foundation-model/anthropic.claude-v2",
  });

  const response = await bedrock.send(command);
  // The body is a Uint8Array; turn it back into JSON.
  const responseBody = JSON.parse(Buffer.from(response.body).toString());
  return responseBody;
}

Key takeaway: The tool_choice: "auto" flag is the permission slip that lets Claude decide to press the getUserProfile button. Without it, Claude will never send a tool_use block.

Analogy

Think of the prompt as a contract negotiation. The system message says “I’m ready to hire you,” the tool definition is the job description, and tool_choice: "auto" is the line in the contract that says “you may take the job when you see fit.” If you leave that line out, the model thinks it’s not allowed to work.

Handling the Tool Use Response and Invoking Your API

Why handle the response – Claude’s reply can be one of two shapes:

Plain text – no tool needed.
tool_use block – tells you which tool to run and with what arguments.

Your code must inspect the response, extract the arguments, call the real API, and then send the result back to Claude as a tool_result block.

Detecting a tool_use block

// src/handleResponse.ts
import axios from "axios";
import { askClaude } from "./invokeClaude";

interface ToolUseMessage {
  type: "tool_use";
  name: string;
  input: Record<string, any>;
}

/**
 * Main driver – asks Claude a question and reacts if Claude wants a tool.
 */
export async function runConversation(userQuestion: string) {
  const raw = await askClaude(userQuestion);

  // The response from Bedrock contains a "messages" array.
  const messages = raw?.messages ?? [];

  // Look for the last assistant message that is a tool_use.
  const toolMessage = messages.find(
    (m: any) => m.type === "tool_use"
  ) as ToolUseMessage | undefined;

  if (!toolMessage) {
    // Claude answered without needing a tool.
    console.log("Claude says:", messages.at(-1)?.content);
    return;
  }

  // -----------------------------------------------------------------
  // Step 1: Validate the tool name – avoid accidental misuse.
  // -----------------------------------------------------------------
  if (toolMessage.name !== "getUserProfile") {
    throw new Error(`Unexpected tool name: ${toolMessage.name}`);
  }

  const { userId } = toolMessage.input;
  if (typeof userId !== "string") {
    throw new Error("userId must be a string");
  }

  // -----------------------------------------------------------------
  // Step 2: Call the real API Gateway endpoint.
  // -----------------------------------------------------------------
  const apiUrl = `https://api.example.com/user/${encodeURIComponent(userId)}`;
  const apiResponse = await axios.get(apiUrl, {
    // If your API uses a custom authorizer you would attach headers here.
    // Example: headers: { Authorization: `Bearer ${myToken}` }
  });

  // The profile we care about – assume the API returns { name, email, role }.
  const profile = apiResponse.data;

  // -----------------------------------------------------------------
  // Step 3: Send the result back to Claude.
  // -----------------------------------------------------------------
  const followUpPayload = {
    modelId: "anthropic.claude-v2",
    messages: [
      // Re‑use the original conversation so Claude has context.
      ...messages,
      {
        // This is the tool_result block Claude expects.
        role: "assistant",
        tool_result: {
          tool_use_id: toolMessage.id, // the id Claude gave us
          content: JSON.stringify(profile), // Claude prefers a string payload
        },
        content: "", // content can be empty for tool results
      },
    ],
    // No need for another tool_choice – we are just continuing the chat.
    tool_choice: "none",
  };

  const command = new InvokeModelCommand({
    body: Buffer.from(JSON.stringify(followUpPayload)),
    contentType: "application/json",
    accept: "application/json",
    modelId: "arn:aws:bedrock:us-east-1:123456789012:foundation-model/anthropic.claude-v2",
  });

  const finalResponse = await bedrock.send(command);
  const finalBody = JSON.parse(Buffer.from(finalResponse.body).toString());

  console.log("Claude finishes:", finalBody.messages.at(-1)?.content);
}

Gotcha #3 – `tool_use` must carry an `id` field

When Claude sends a tool_use block, it includes an id that you must echo back in tool_result.tool_use_id. Forgetting this field makes Claude think the result belongs to a different request, and the conversation stalls.

Gotcha #4 – API Gateway’s 29‑second timeout

If your downstream Lambda (or whatever backs the /user/{id} route) takes longer than 29 seconds, API Gateway aborts the call. Make sure the profile lookup is fast, or move the heavy work into an asynchronous workflow and return a placeholder.

Helpful tip: Enable API Gateway access logs ($context.requestId) so you can correlate a failed axios.get with a CloudWatch log entry.

Analogy

Imagine Claude as a detective asking you to fetch a file from a locked cabinet. The tool_use message is the detective handing you a key (the userId). You open the cabinet (API call), take out the file (profile JSON), and hand it back in a sealed envelope (tool_result). The detective can then finish the report.

Closing the Loop: Feeding the Result Back to Claude

Why close the loop – Claude cannot finish a task without the data it asked for. By sending the tool_result block you give Claude the missing piece, allowing it to generate a final, user‑friendly answer. The pattern repeats: Claude may ask for more tools, or it may simply respond with text.

Minimal complete example (run from CLI)

// src/index.ts
import { runConversation } from "./handleResponse";

const question = "Can you show me the profile of user abc123?";
runConversation(question).catch((err) => {
  console.error("Error during conversation:", err);
});

Run it:

npx ts-node src/index.ts

You should see output similar to:

Claude says: Let me look that up for you...
Claude finishes: Here is the profile you requested:
{
  "name": "Alice Johnson",
  "email": "alice@example.com",
  "role": "Engineer"
}

Gotcha #5 – Streaming vs. non‑streaming

If you later switch to streaming mode to get partial replies, you will need to parse each SSE line and buffer until you encounter a tool_use event. The logic shown above works for the simple request/response mode and avoids the extra parsing complexity.

In plain English: Think of the conversation as a ping‑pong game. Claude throws a question, you catch it, maybe fetch a piece of data, then toss the answer back. The ball (the message) must always contain the right tag (tool_use / tool_result) so the other side knows how to react.

The Takeaway

Key points to remember

Claude can call external services, but only when you give it a tool definition and set tool_choice: "auto".
The Bedrock SDK (@aws-sdk/client-bedrock-runtime) is the entry point; watch out for per‑minute token limits and manual SSE handling.
A tool_use block must contain the correct name, input, and id. Missing any part makes Claude fall back to plain text.
Invoke your real API (API Gateway) with a reliable HTTP client; remember the 29‑second timeout and potential CORS misconfigurations.
Return the result in a tool_result block, echoing the original tool_use_id. This lets Claude finish the answer.
Treat the whole flow as a conversation where each side respects the same contract—much like a well‑written API specification.

By following these steps, you turn Claude from a static answer engine into an interactive assistant that can pull data from your own services, all with a handful of TypeScript lines. Happy building!

Transparency notice

This article was written with the help of an AI system — Groq (GPT OSS 120B).

Published: 2026-09-30 · Primary focus: Bedrock

All code blocks are intended to be correct and runnable, but please verify them
against the official docs for the tools mentioned before using in production.

Find an error? Drop a comment — corrections are always welcome.

DEV Community: Dinesh_gowtham

How to Rotate Claude API Keys with Secrets Manager and Lambda Without Getting Locked Out

Why Secret Rotation Matters for LLM APIs

Key points

Setting Up Secrets Manager for Claude Tokens

Create the secret

Cost awareness (gotchas)

Lambda Rotation Function – Code Walkthrough

Dependencies

Rotation Lambda (TypeScript)

Integrating Rotation with Your Node.js 22 Service

Why use diagnostics_channel?

Service Lambda (JavaScript)

Wiring EventBridge to the diagnostics channel

Testing, Monitoring, and Avoiding Common Pitfalls

Local testing

Monitoring

Common gotchas and how to dodge them

End‑to‑end sanity check

The Takeaway

AI Agent Observability Made Simple: Using CloudWatch EMF and X-Ray to Trace Claude Tool Calls

Why AI Agents Need Observability

Key concepts

Embedded Metrics Format: Zero‑Cost Custom Metrics

Why EMF instead of the classic PutMetricData API?

Code: Minimal EMF logger for a Lambda

Adding X‑Ray Subsegments to Tool Calls

Why subsegments matter

Code: X‑Ray client and subsegment wrapper

Putting It All Together in a Lambda‑Hosted Claude Agent

Full Lambda handler

What the code does, step by step

Gotchas specific to this setup

Reading the Signals in CloudWatch Console

How to spot a problem

The Takeaway

Bedrock Guardrails Explained Simply: How to Enforce Safe LLM Outputs in Node.js

What Are Guardrails and Why They Matter

Minimal example – a guardrail‑aware request (no error handling yet)

Creating and Configuring a Guardrail in the AWS Console

Step‑by‑step in the console

Calling Bedrock with Guardrails via the SDK

Full Lambda‑ready snippet (still no error handling)

Handling GuardrailViolationException in Your Code

Catching the specific exception

Testing and Tuning Guardrail Policies

Why test?

Simple local test harness

Fine‑tuning tips

The Takeaway

Action steps you can take today

Edge AI with CloudFront: How to Serve Claude Responses at the Edge Using Lambda Web Functions, Explained Simply

Why Edge AI Matters

Creating a CloudFront Distribution with a Lambda Web Function Origin

Step‑by‑step creation with the AWS SDK

Gotchas you may hit

Calling Claude via Bedrock from the Edge with fetch

Full handler code

Walk‑through of the most important lines

Gotchas specific to Bedrock

Prompt Caching and Token Reuse at the Edge

Simple in‑memory cache design

Integrating the cache into the handler

Observability and Debugging with CloudWatch Logs

Sending logs from the edge

Monitoring latency and errors

Debugging tip for the 403 “Forbidden” error

The Takeaway

API Gateway HTTP API + Lambda Function URLs: Building a Prompt‑Caching LLM Proxy, Explained Simply

Why Prompt Caching Matters for LLM Proxies

Quick code sketch

Setting Up an HTTP API with a Lambda Function URL

Step‑by‑step creation with the AWS SDK

Implementing a Tiny In‑Process Cache in Node.js 22

Full handler code

Gotchas you might hit

Streaming Claude Responses with Native fetch

Minimal example of consuming the SSE stream

Handling 429 throttling with exponential backoff

Observability with diagnostics_channel

Integrating Rotation with Your Node.js 22 Service

Why use `diagnostics_channel`?

Why EMF instead of the classic `PutMetricData` API?

Implementing a Tiny In‑Process Cache in Node.js 22

Prompt Caching with ElastiCache: Speed Up Claude Calls in Node.js 22, Explained Simply

Connecting Node.js 22 to Redis with the New fetch‑Based Client

Calling the access point from Node.js 22