DEV Community: Dinesh_gowtham

MCP Explained Simply: How to Build a Claude‑Powered AI Agent with AWS Lambda

Dinesh_gowtham — Fri, 06 Nov 2026 09:39:49 +0000

Ever wondered how Claude can keep context across dozens of calls without re‑sending the whole prompt each time? The Model Context Protocol (MCP) makes that possible, and you can host it in a serverless Lambda in minutes. Let’s demystify the wire‑format and see a real implementation.

What Is MCP and Why It Matters

When you talk to a large language model (LLM) like Claude, you normally send a big text block that contains everything the model needs to remember: the system prompt, user messages, and any prior assistant replies. This works, but it wastes bandwidth and token budget when the conversation grows long.

MCP, short for Model Context Protocol, is a lightweight way to tell the model “here is the next piece of the conversation, and here’s the token that represents everything that came before.” Instead of re‑sending the whole history, you send a small session token that the model expands internally.

In plain English: MCP is like handing a librarian a library card (the token) instead of re‑handing the entire stack of books every time you ask for the next chapter.

Why care about this?

Speed – Less data over the network means lower latency, especially when you are calling the model from a serverless function that may already be warm only for a few milliseconds.
Cost – Tokens you re‑send are tokens you pay for. A session token can be as short as a few characters, saving you money on the model’s per‑token pricing.
Scalability – A single Lambda can handle many concurrent conversations without blowing up its memory budget, because it never stores the full chat history locally.

The trade‑off is that you must respect the MCP wire‑format, a JSON structure that the model expects. If you break it, Claude will think you started a brand‑new session and you’ll lose all the context you worked so hard to build.

The MCP Wire‑Format – A Quick Walkthrough

MCP messages are an array of objects, each object representing a single turn in the conversation. The most important fields are:

Field	Meaning	Example
`role`	Who is speaking: `"user"` or `"assistant"`	`"user"`
`content`	The actual text of the message	`"What is the weather in Paris?"`
`session_token`	Optional short string that refers to the model’s internal memory of earlier turns	`"eyJzZXNzaW9uIjoiYWJjMTIzIn0="`

A complete MCP payload looks like this:

{
  "messages": [
    { "role": "system", "content": "You are a helpful travel guide." },
    { "role": "user", "content": "Tell me about the Eiffel Tower." },
    { "role": "assistant", "content": "The Eiffel Tower is ...", "session_token": "abc123" },
    { "role": "user", "content": "What are the opening hours?" }
  ]
}

Notice that the third message (the assistant’s reply) carries a session_token. When the next user turn arrives, you include that same token so Claude knows it belongs to the same conversation.

Key takeaway: The array must be sent as a single JSON object. Splitting it into separate HTTP calls makes Claude think each call is a brand‑new conversation.

A Simple Analogy

Think of a chat as a train. Each car (message) is coupled to the one before it. The session_token is the train’s unique identifier. If you try to add a new car by sending a separate request without the identifier, the railway system treats it as a different train altogether.

Setting Up a Lambda‑Backed MCP Proxy

Now that we understand the format, let’s create a tiny HTTP endpoint that accepts MCP payloads and forwards them to Claude via Bedrock. We’ll use AWS Lambda as the compute engine because it gives us instant scaling and no servers to manage.

1️⃣ Create the Lambda function

In the AWS console, choose Node.js 22.x as the runtime. This is the newest LTS version, but be aware of a known gotcha: the require('esm') hack that many older layers use no longer works out‑of‑the‑box. If you depend on a layer that adds ESM support, you’ll see silent failures at runtime. The safest route is to keep your code fully ESM (use import statements) or avoid the layer altogether.

2️⃣ Add the SDK packages

Add the following dependencies to your package.json:

{
  "dependencies": {
    "@aws-sdk/client-lambda": "^3.600.0",
    "@aws-sdk/client-bedrockruntime": "^3.600.0"
  }
}

The first SDK lets us invoke other Lambdas if you ever want to split responsibilities. The second is the actual client that talks to Claude on Bedrock.

3️⃣ Write the handler

Below is a complete TypeScript Lambda handler. It does three things:

Parses the incoming MCP JSON body.
Calls Bedrock’s InvokeModel (via sendMessage) with the same payload.
Streams the model’s response back to the caller, preserving any session_token that Claude returns.

// src/handler.ts
import { APIGatewayProxyEvent, APIGatewayProxyResult } from 'aws-lambda';
import {
  BedrockRuntimeClient,
  InvokeModelCommand,
  InvokeModelCommandInput,
} from '@aws-sdk/client-bedrockruntime';

// Create a Bedrock client once per container (reused across invocations)
const bedrockClient = new BedrockRuntimeClient({ region: 'us-east-1' });

/**
 * Lambda entry point.
 * Expects a POST request with a JSON body that follows the MCP format.
 */
export const handler = async (
  event: APIGatewayProxyEvent
): Promise<APIGatewayProxyResult> => {
  // Guard against non‑JSON payloads
  if (!event.body) {
    return {
      statusCode: 400,
      body: JSON.stringify({ error: 'Missing request body' }),
    };
  }

  // Parse the incoming MCP payload
  let mcpPayload: any;
  try {
    mcpPayload = JSON.parse(event.body);
  } catch {
    return {
      statusCode: 400,
      body: JSON.stringify({ error: 'Invalid JSON' }),
    };
  }

  // Build the Bedrock request – the model name can be changed to any Claude variant
  const bedrockInput: InvokeModelCommandInput = {
    modelId: 'anthropic.claude-3-5-sonnet-20240620-v1:0',
    // Bedrock expects a raw string payload, not a wrapped object
    body: JSON.stringify(mcpPayload),
    contentType: 'application/json',
    accept: 'application/json',
  };

  // Send the request to Bedrock
  let bedrockResponse;
  try {
    bedrockResponse = await bedrockClient.send(new InvokeModelCommand(bedrockInput));
  } catch (err) {
    console.error('Bedrock call failed', err);
    return {
      statusCode: 502,
      body: JSON.stringify({ error: 'Failed to contact Claude' }),
    };
  }

  // Bedrock streams the answer back as a Uint8Array; we need to turn it into text
  const responseText = new TextDecoder('utf-8').decode(bedrockResponse.body);

  // The response will include a new session_token if Claude wants to keep state
  // Forward it unchanged so the caller can store it for the next turn
  return {
    statusCode: 200,
    // Important: for streaming responses you must set these headers, otherwise API Gateway buffers everything.
    headers: {
      'Content-Type': 'application/json',
      // The header below tells API Gateway not to compress the payload (which can break incremental streaming)
      'X-Accel-Buffering': 'no',
    },
    body: responseText,
  };
};

Why each line matters

import { APIGatewayProxyEvent, APIGatewayProxyResult } – Types that describe the shape of the incoming HTTP request and outgoing response.
new BedrockRuntimeClient({ region }) – Creates a client that knows how to talk to the Bedrock service in a specific AWS region.
The if (!event.body) guard prevents the function from crashing when a caller forgets to send data.
JSON.parse(event.body) turns the raw string into a JavaScript object we can inspect or modify later.
InvokeModelCommand – The low‑level API that sends the MCP payload to Claude.
new TextDecoder('utf-8').decode(bedrockResponse.body) – Bedrock returns a binary buffer; we need readable text for the API consumer.
The response headers include X-Accel-Buffering: no. Without this, API Gateway will buffer the whole answer before sending it back, which defeats the purpose of incremental streaming.

Tip: If you enable SnapStart on this Lambda, remember that SnapStart does not help when the function is attached to a VPC. The cold start time is dominated by the VPC ENI attachment, not the JVM warm‑up.

4️⃣ Deploy

Package the code with npm run build (or use the AWS SAM CLI). Deploy using the console or a CI pipeline. Remember to give the Lambda permission to call bedrock:InvokeModel by attaching the AmazonBedrockFullAccess policy or a more restrictive custom policy.

Calling Claude via Bedrock from Lambda

Now that the proxy exists, let’s look at the exact Bedrock call we made and why it respects MCP.

1️⃣ The `sendMessage` command (conceptually)

The AWS SDK for Bedrock provides a higher‑level method called sendMessage in some language bindings. In the low‑level InvokeModelCommand we used, the payload is exactly what Claude expects: an object with a messages array that follows the MCP spec.

// This is the shape the model expects
interface MCPMessage {
  role: 'system' | 'user' | 'assistant';
  content: string;
  session_token?: string; // optional, only when you have an existing token
}

interface MCPPayload {
  messages: MCPMessage[];
}

When you include a session_token on the last assistant message, Claude internally retrieves the hidden context that the token represents. The model then appends its new reply to that hidden context and returns a new token for the next round.

In plain English: Think of the token as a secret key that unlocks the conversation history stored inside Claude’s brain.

2️⃣ Handling incremental responses

Claude can return its answer in chunks, which is useful for UI that wants to display text as it arrives. To keep the Lambda simple, we collected the whole stream into a string. If you need true streaming, you must enable Lambda response streaming and set the Content-Type: text/event-stream header. A common gotcha is forgetting this header, which makes API Gateway buffer the entire response and defeats the streaming effect.

// Example of enabling streaming (requires Lambda >= Node 22 and API Gateway v2)
return {
  statusCode: 200,
  headers: {
    'Content-Type': 'text/event-stream',
    'Cache-Control': 'no-cache',
    // Required for API Gateway to forward chunks immediately
    'Transfer-Encoding': 'chunked',
  },
  body: responseText, // each chunk would be written separately in a real streaming scenario
};

3️⃣ Cost awareness

Provisioned Concurrency guarantees that a certain number of Lambda instances are always warm, which removes cold start latency. However, you still pay for those instances even when they are idle. Many teams have been surprised by the monthly bill when they enable it on a function that only sees occasional traffic.

Tip: Start with on‑demand Lambda and monitor latency. Switch to Provisioned Concurrency only if you consistently need sub‑100 ms response times.

Testing the End‑to‑End Flow

A good tutorial includes a way to verify that everything works as expected. Below is a minimal test using curl and a simple Node script.

1️⃣ Deploy a test endpoint

Assume your Lambda is fronted by an API Gateway HTTP API at https://abcde12345.execute-api.us-east-1.amazonaws.com/mcp.

2️⃣ First request – start a new session

curl -X POST https://abcde12345.execute-api.us-east-1.amazonaws.com/mcp \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [
      {"role": "system", "content": "You are a friendly travel assistant."},
      {"role": "user", "content": "Hi! What can you tell me about Rome?"}
    ]
  }' -o response1.json

Inspect the result:

cat response1.json

You should see something like:

{
  "messages": [
    {
      "role": "assistant",
      "content": "Rome is the capital of Italy...",
      "session_token": "eyJzZXNzaW9uIjoiZDEyMzQ1In0="
    }
  ]
}

Save the token for the next call.

3️⃣ Follow‑up request – keep context

TOKEN=$(jq -r '.messages[0].session_token' response1.json)

curl -X POST https://abcde12345.execute-api.us-east-1.amazonaws.com/mcp \
  -H "Content-Type: application/json" \
  -d "{
    \"messages\": [
      {\"role\": \"assistant\", \"content\": \"Rome is the capital of Italy...\", \"session_token\": \"$TOKEN\"},
      {\"role\": \"user\", \"content\": \"What’s the best time to visit?\"}
    ]
  }" -o response2.json

Now the model’s answer will be tailored to the earlier statement about Rome because the same token was supplied.

Key takeaway: Always include the most recent session_token on the assistant message you send back. Forgetting it resets the conversation.

4️⃣ Automated test script (optional)

// test-mcp.js
import fetch from 'node-fetch';

const endpoint = 'https://abcde12345.execute-api.us-east-1.amazonaws.com/mcp';

async function callMCP(messages) {
  const resp = await fetch(endpoint, {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ messages }),
  });
  return resp.json();
}

(async () => {
  // Step 1 – start conversation
  const first = await callMCP([
    { role: 'system', content: 'You are a friendly travel assistant.' },
    { role: 'user', content: 'Tell me about Kyoto.' },
  ]);
  console.log('First reply:', first);

  // Extract token
  const token = first.messages[0].session_token;

  // Step 2 – continue with token
  const second = await callMCP([
    { role: 'assistant', content: first.messages[0].content, session_token: token },
    { role: 'user', content: 'When is cherry blossom season?' },
  ]);
  console.log('Second reply:', second);
})();

Run with node test-mcp.js. If you see the second answer referencing Kyoto, you’ve successfully kept the context.

The Takeaway

What you should remember after reading this guide

MCP lets you keep a conversation alive with a tiny token instead of re‑sending the whole chat history.
The wire‑format must be a single JSON object that contains an array of messages; breaking it into multiple HTTP calls resets the session.
A TypeScript Lambda can act as a thin proxy: parse the incoming MCP payload, forward it to Bedrock’s Claude via InvokeModelCommand, and stream the response back.
Pay attention to Lambda‑specific quirks: Node 22 ESM layers, SnapStart + VPC, response‑streaming headers, and provisioned concurrency costs.
Test end‑to‑end with curl or a small script, and always carry forward the latest session_token on the assistant message you send.

With these pieces in place, you now have a low‑latency, cost‑efficient Claude‑powered AI agent that can run forever in a serverless environment. Happy coding!

Transparency notice

This article was written with the help of an AI system — Groq (GPT OSS 120B).

Published: 2026-11-06 · Primary focus: MCP

All code blocks are intended to be correct and runnable, but please verify them
against the Model Context Protocol spec before using in production.

Find an error? Drop a comment — corrections are always welcome.

Streaming Claude Tokens with Fastify on Lambda: A Step‑By‑Step Guide Explained Simply

Dinesh_gowtham — Wed, 04 Nov 2026 09:40:12 +0000

LLM developers assume they need a heavyweight framework to handle streaming responses, but Fastify’s tiny core can proxy Claude’s token stream with almost zero overhead. In this post you’ll see how to turn a Fastify service into a real‑time chat endpoint that streams tokens directly to the browser.

Why Fastify Beats Express for LLM Streaming

When you ask a language model (LLM) like Claude to generate text, it often returns tokens (the smallest pieces of language) one by one. To give a user a “live‑typing” feel you need to forward each token to the browser as soon as it arrives.

Express—the long‑time favorite of Node developers—adds a lot of middleware layers. Those layers are great for flexibility, but each extra function call adds a tiny amount of latency. When you are forwarding dozens of tokens per second, that latency compounds and you can notice a pause between the model’s output and what the user sees.

Fastify is built around performance first principles:

Minimal core – only the HTTP server and routing logic.
Built‑in support for async/await – no need for callbacks that can block the event loop.
Fast request/response handling – uses the ultra‑fast ultra‑http library under the hood.

Think of Fastify as a high‑speed conveyor belt that moves a package (the token) from the model to the client with almost no friction, whereas Express is more like a busy hallway where the same package has to stop at several doors before continuing. For a streaming chat endpoint, that difference is noticeable.

In plain English – Fastify’s thin design means less time waiting inside the server, so the user sees tokens almost as soon as Claude produces them.

Code snippet: a minimal Fastify server

// src/server.ts
import Fastify from "fastify";

// Create a Fastify instance with the smallest possible options
const fastify = Fastify({
  logger: true,               // helpful logs, can be turned off later
  // No plugins loaded – we keep the core tiny
});

fastify.get("/", async (request, reply) => {
  // A simple health‑check endpoint
  return { status: "ok" };
});

// Export the server so Lambda can import it later
export default fastify;

Fastify() creates the server.
logger: true prints request details – useful while developing.
No extra middleware is loaded, keeping the overhead at a few microseconds per request.

Setting Up Fastify with Native fetch on Node.js 22

Node.js 22 ships with a native fetch implementation that works like the browser’s fetch. That means we can call AWS Bedrock (the service that hosts Claude) without pulling in a heavy HTTP client library. Using native fetch keeps the bundle size tiny, which is important for Lambda where every kilobyte counts toward cold‑start time.

Why avoid extra libraries?

Every additional dependency must be zipped, uploaded, and unpacked when Lambda starts. The more code you ship, the longer the cold start (the first time your function runs after being idle). Native fetch eliminates that extra work.

Code snippet: enabling fetch in a Fastify route

// src/routes/chat.ts
import { FastifyInstance, FastifyReply, FastifyRequest } from "fastify";

export async function chatRoute(fastify: FastifyInstance) {
  // Register a POST /chat endpoint
  fastify.post("/chat", async (request: FastifyRequest, reply: FastifyReply) => {
    // Parse the incoming JSON body { prompt: "..." }
    const { prompt } = request.body as { prompt: string };
    if (!prompt) {
      reply.status(400).send({ error: "Prompt is required" });
      return;
    }

    // Call the helper that talks to Bedrock (defined later)
    const stream = await invokeClaudeStream(prompt);

    // Set up Server‑Sent Events (SSE) headers – see next section for details
    reply.raw.writeHead(200, {
      "Content-Type": "text/event-stream", // tells browser to treat data as a stream
      "Cache-Control": "no-cache",
      Connection: "keep-alive",
    });

    // Pipe each token from Bedrock to the client as an SSE line
    for await (const token of stream) {
      // SSE format: each message starts with "data:" and ends with two newlines
      reply.raw.write(`data: ${JSON.stringify({ token })}\n\n`);
    }

    // Close the SSE stream when Bedrock finishes
    reply.raw.end();
  });
}

request.body holds the JSON payload the browser sends.
invokeClaudeStream (implemented later) returns an async iterable of tokens – a convenient way to handle a stream in JavaScript.
reply.raw gives us access to the underlying Node http.ServerResponse so we can write raw SSE data.

Tip – Keep fetch calls and response handling inside the route as short as possible. Anything that can be moved to a helper function reduces the amount of code that runs during each request, making the Lambda cold start faster.

Calling Claude via Bedrock and Streaming SSE

Bedrock is Amazon’s managed service that hosts LLMs like Claude. The SDK package we use is @aws-sdk/client-bedrock-runtime. Inside the SDK there is a method called InvokeModelCommand that can be asked to stream the model’s output instead of waiting for the whole answer.

What is “streaming” in this context?

Imagine Claude is a storyteller reading a book aloud. With normal (non‑streaming) calls, you wait until the storyteller finishes the whole chapter before you hear anything. With streaming, you hear each sentence as soon as it is spoken. In technical terms, the service sends a chunked HTTP response where each chunk contains the next token.

Setting up the Bedrock client

// src/bedrock.ts
import {
  BedrockRuntimeClient,
  InvokeModelCommand,
} from "@aws-sdk/client-bedrock-runtime";
import { Readable } from "stream";

// Create a client that uses the default Lambda credentials
const bedrock = new BedrockRuntimeClient({});

// Helper that returns an async iterable of tokens
export async function* invokeClaudeStream(prompt: string): AsyncGenerator<string> {
  // Build the request payload according to Claude's API spec
  const body = JSON.stringify({
    prompt,
    // Tell Claude we want a streaming response
    stream: true,
    // Optional: max tokens, temperature, etc.
  });

  // The command carries the model ID and the payload
  const command = new InvokeModelCommand({
    modelId: "anthropic.claude-v2", // replace with the exact model you use
    contentType: "application/json",
    accept: "application/json",
    body,
  });

  // Invoke the model – Bedrock returns a Readable stream of bytes
  const response = await bedrock.send(command);

  // The response.body is a Node.js Readable stream; we convert it to async iterator
  const stream = response.body as Readable;

  // The raw bytes are newline‑delimited JSON fragments; each contains a token
  const decoder = new TextDecoder("utf-8");
  let buffer = "";

  for await (const chunk of stream) {
    // Convert Uint8Array chunk to string and accumulate
    buffer += decoder.decode(chunk, { stream: true });

    // Split on newline – each line is a complete JSON message
    const lines = buffer.split("\n");
    // Keep the last partial line in buffer for the next iteration
    buffer = lines.pop() ?? "";

    for (const line of lines) {
      if (!line.trim()) continue; // skip empty lines
      const data = JSON.parse(line);
      // The token appears under `completion` in Claude's streaming format
      yield data.completion;
    }
  }

  // After the stream ends, flush any remaining buffered data
  if (buffer) {
    const data = JSON.parse(buffer);
    yield data.completion;
  }
}

BedrockRuntimeClient reads credentials from the Lambda execution role, so we don’t embed keys.
InvokeModelCommand tells Bedrock which model to run and that we want a streaming response (stream: true).
The function invokeClaudeStream is an async generator – it yields each token as soon as it arrives, letting the Fastify route for await over it.
The newline‑delimited JSON format is a common pattern for streaming LLMs; each line is a small JSON object that includes a completion field containing the next token.

In plain English – We ask Bedrock to talk to us one word at a time, turn each word into a JavaScript value, and hand it over to Fastify so the browser can see it instantly.

Handling backpressure

Backpressure is a fancy way of saying “don’t overwhelm the client”. When the browser can’t read data quickly enough, Node’s stream will automatically pause reading from Bedrock until the socket drains. Because we use for await on the async generator, the await implicitly respects that pause, so we don’t need extra flow‑control code.

Deploying the Fastify Service to Lambda

AWS Lambda runs your code inside a sandboxed container. By default, Lambda buffers the whole HTTP response before sending it to the caller. That behavior defeats streaming – the client will only get the final payload after the function finishes. To make streaming work we must:

Turn off buffering – set the Content-Type to text/event-stream and add the header X-Amz-Content-Type with the same value.
Return a “chunked” response – don’t call reply.send() after writing data; instead write directly to reply.raw and end the response when the stream finishes.
Set callbackWaitsForEmptyEventLoop = false – tells Lambda to finish the invocation as soon as the handler returns, even if the event loop still has pending work (our open HTTP connection).

Lambda handler file

// src/lambda.ts
import fastify from "./server";
import { chatRoute } from "./routes/chat";

// Register the chat route on the shared Fastify instance
await chatRoute(fastify);

// Export the handler that Lambda will call
export const handler = async (event: any, context: any) => {
  // Prevent Lambda from waiting for the event loop to be empty
  context.callbackWaitsForEmptyEventLoop = false;

  // Fastify can handle raw API Gateway events via the "aws-lambda-fastify" shim.
  // We avoid an extra package by using Fastify’s built‑in `server.inject`.
  const result = await fastify.inject({
    method: event.httpMethod,
    url: event.path,
    headers: event.headers,
    payload: event.body,
    // The `raw` field gives us direct access to the Node response object
    // which Fastify already wrote to in the /chat route.
  });

  // Return the shape API Gateway expects
  return {
    statusCode: result.statusCode,
    headers: result.headers,
    // For streaming we must return the raw body stream; API Gateway v2 supports this.
    // If you use the older REST API, you’ll need to enable “Lambda proxy integration”.
    body: result.body,
    isBase64Encoded: false,
  };
};

context.callbackWaitsForEmptyEventLoop = false; disables the default buffering behavior.
fastify.inject simulates an HTTP request without opening a network socket, which works nicely inside Lambda because we already have the request data from API Gateway.
The returned object matches the Lambda proxy integration format that API Gateway expects.

Deploy script (using the AWS CDK for illustration)

// infra/cdk-stack.ts
import * as cdk from "aws-cdk-lib";
import * as lambda from "aws-cdk-lib/aws-lambda";
import * as apigateway from "aws-cdk-lib/aws-apigatewayv2";

export class ClaudeStreamingStack extends cdk.Stack {
  constructor(scope: cdk.App, id: string) {
    super(scope, id);

    const fn = new lambda.Function(this, "FastifyClaudeFn", {
      runtime: lambda.Runtime.NODEJS_22_X,
      handler: "src/lambda.handler",
      code: lambda.Code.fromAsset("dist"), // compiled TypeScript output
      memorySize: 1024, // give enough RAM for Bedrock client
      timeout: cdk.Duration.seconds(30),
      environment: {
        // Bedrock region, etc.
        AWS_REGION: "us-east-1",
      },
    });

    // API Gateway HTTP API (v2) with Lambda integration
    const api = new apigateway.HttpApi(this, "ClaudeApi", {
      createDefaultStage: true,
    });

    api.addRoutes({
      path: "/chat",
      methods: [apigateway.HttpMethod.POST],
      integration: new apigateway.LambdaProxyIntegration({ handler: fn }),
    });
  }
}

Gotcha – Lambda’s response streaming works only with the HTTP API (v2), not the older REST API. Forgetting this will make the browser receive a single JSON blob instead of a live token stream.

Lambda layer gotcha reminder

If you decide to add a custom layer (e.g., for shared utilities) note that Node 22’s ESM loader (require(esm)) can silently break when the layer is compiled for CommonJS. The safest route is to keep everything in native ESM (.mjs or "type":"module" in package.json) or avoid the esm shim entirely.

Testing the End‑to‑End Token Stream

Testing streaming is a little different from a normal API call because you need to read data as it arrives. The easiest way is to use curl with the --no-buffer flag or a small Node script that consumes the SSE stream.

Using curl

curl -N -X POST https://<api-id>.execute-api.<region>.amazonaws.com/chat \
  -H "Content-Type: application/json" \
  -d '{"prompt":"Tell me a short story about a robot."}'

-N disables curl’s internal buffering, letting you see each data: line instantly.
You should see lines like:

data: {"token":"Once"}
data: {"token":" upon"}
data: {"token":" a"}
...

Node test script

// test/stream-client.js
import fetch from "node-fetch";

async function run() {
  const response = await fetch("https://<api-id>.execute-api.<region>.amazonaws.com/chat", {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({ prompt: "Explain quantum computing in one sentence." }),
  });

  // Ensure we get an SSE stream
  if (!response.ok || !response.headers.get("content-type")?.includes("text/event-stream")) {
    console.error("Not a streaming response");
    return;
  }

  // Read the stream line‑by‑line
  const reader = response.body.getReader();
  const decoder = new TextDecoder();
  let buffer = "";

  while (true) {
    const { done, value } = await reader.read();
    if (done) break;
    buffer += decoder.decode(value, { stream: true });
    const lines = buffer.split("\n");
    buffer = lines.pop() ?? "";

    for (const line of lines) {
      if (line.startsWith("data:")) {
        const payload = JSON.parse(line.slice(5).trim());
        process.stdout.write(payload.token);
      }
    }
  }

  console.log("\n--- stream finished ---");
}

run();

The script fetches the /chat endpoint, checks that the Content-Type is the SSE type, and then reads each chunk as it arrives.
process.stdout.write prints tokens without a newline, mimicking the “live typing” effect in a terminal.

Tip – When you test locally (e.g., npm run dev with Fastify listening on localhost), you can skip the Lambda layer and call the route directly. The same SSE headers work, so the client code stays identical.

Verifying Lambda buffering is disabled

If you forget to set callbackWaitsForEmptyEventLoop = false, the client will receive nothing until the model finishes. You’ll see a single huge JSON payload instead of many data: lines. Open the CloudWatch logs for your Lambda – you’ll notice a line like REPORT RequestId: … Duration: … Billed Duration: … after the entire response is sent. Once the flag is set, the log will show the function ending while the connection remains open, confirming streaming works.

The Takeaway

What you now know

Fastify’s lean core makes it a perfect match for token‑by‑token streaming; less middleware means lower latency.
Native fetch in Node 22 removes the need for extra HTTP libraries, shrinking Lambda bundle size and cold‑start time.
Bedrock’s InvokeModelCommand with stream: true returns a newline‑delimited JSON stream; turning that into an async generator gives a clean for await loop.
Lambda buffers responses unless you disable it (callbackWaitsForEmptyEventLoop = false) and use the HTTP API with text/event-stream headers.
Backpressure is handled automatically when you await each token; you don’t need manual flow‑control code.
Common pitfalls: using the old REST API (no streaming), forgetting the SSE headers, and pulling in an esm shim that breaks under Node 22.

Armed with these pieces, you can ship a production‑grade, real‑time chat endpoint that feels as responsive as typing in a messenger, all while staying within the modest cost envelope of a single Lambda function. Happy streaming!

Transparency notice

This article was written with the help of an AI system — Groq (GPT OSS 120B).

Published: 2026-11-04 · Primary focus: Fastify

All code blocks are intended to be correct and runnable, but please verify them
against the official docs for the tools mentioned before using in production.

Find an error? Drop a comment — corrections are always welcome.

pnpm Workspaces in Lambda: How to Shrink Deploy Packages and Supercharge Claude‑Powered AI Agents, Explained Simply

Dinesh_gowtham — Tue, 03 Nov 2026 09:57:18 +0000

Deploying Claude‑driven coding assistants on Lambda can feel like squeezing a massive monolith into a tiny zip file. pnpm workspaces prune duplicate files and share a single lockfile, slashing bundle size and cold‑start latency. This post shows how a single package‑manager switch turns a sluggish AI agent into a snappy serverless service.

Why Deployment Size Matters for Claude‑Powered Lambda Functions

When a Lambda function starts, AWS has to download the deployment package (the zip file you upload) into a fresh execution environment. The bigger the zip, the longer the download, and the higher the cold‑start latency—the time it takes before your function can answer the first request.

For Claude‑powered agents the problem is amplified:

Model SDKs (e.g., @aws-sdk/client-bedrock-runtime) already pull in many transitive dependencies.
Utility libraries you write for prompt templating, code formatting, or logging add more files.
Native binaries (e.g., for image processing) need platform‑specific builds, which duplicate across packages.

Think of the deployment package like a suitcase for a trip. If you pack every piece of clothing you own, the airline will charge you extra, and you'll spend more time waiting at the gate. Packing only the items you actually need lets you board faster and saves money. The same principle applies to Lambda: smaller packages board (download) faster, and you pay less for storage and data transfer.

In plain English: A lean zip file means your Claude assistant starts up quicker, answers users faster, and costs less to run.

Below is a tiny Lambda that just logs its own size. Run it with a bloated node_modules folder and you’ll see the delay in the log output.

// handler.ts – a minimal Lambda that reports its own bundle size
import { readFileSync } from 'fs';
import { resolve } from 'path';

// The absolute path to the zip file that Lambda extracts at /var/task
const bundlePath = resolve('/var/task', 'package.json');

export const handler = async () => {
  // Read the package.json to get a rough idea of how many deps we have
  const pkg = JSON.parse(readFileSync(bundlePath, 'utf8'));

  console.log('🚀 Starting Claude‑powered Lambda');
  console.log(`📦 Bundle reports ${Object.keys(pkg.dependencies || {}).length} direct deps`);
  console.log(`🗂️  Working directory contains ${Object.keys(require('module').builtinModules).length} built‑ins`);

  return {
    statusCode: 200,
    body: JSON.stringify({ message: 'Hello from Claude!' })
  };
};

Each line is commented, showing exactly what the code does.

Introducing pnpm Workspaces: How They Differ from npm and Yarn

pnpm (pronounced “pee‑en‑pee‑em”) is a JavaScript package manager that stores every downloaded package once on disk and then creates hard links into each project that needs it.

Workspace – a group of related packages that share a single pnpm-lock.yaml. Think of a workspace as a neighborhood where all houses (packages) share the same water pipe (lockfile) and the same grocery store (central store of modules).
Deduplication – pnpm never copies the same version of a library into multiple node_modules folders. npm and Yarn used to duplicate files, which inflates the bundle.
.pnpm directory – pnpm writes the actual files into a hidden .pnpm folder and then links them into each package’s node_modules. This layout is what makes deduplication possible.

A typical workspace layout looks like this:

my-claude-agent/
├─ pnpm-workspace.yaml          # tells pnpm which folders belong to the workspace
├─ package.json                # root config, holds shared devDeps
├─ packages/
│  ├─ agent/                   # the Lambda handler
│  │  ├─ src/
│  │  │  └─ handler.ts
│  │  └─ package.json
│  └─ shared/                  # utilities used by the agent
│     ├─ src/
│     │  └─ helpers.ts
│     └─ package.json
└─ .pnpm/                      # pnpm’s internal store (not a source folder)

The lockfile (pnpm-lock.yaml) guarantees that every developer and every CI run installs the exact same versions, which is crucial for reproducible builds in serverless pipelines.

Key takeaway: pnpm’s workspace model gives you one place for all dependencies, so the final Lambda zip only contains a single copy of each library instead of many.

Setting Up a Monorepo for Your Claude Agent with Corepack

Corepack is a lightweight wrapper shipped with Node that lets you pin a specific version of a package manager (like pnpm) without installing it globally. This ensures everyone on the team runs the same pnpm version, avoiding subtle “works on my machine” bugs.

Below is a step‑by‑step setup you can copy‑paste into a fresh terminal.

# 1️⃣ Create the repo folder
mkdir my-claude-agent && cd my-claude-agent

# 2️⃣ Initialise a git repo (optional but recommended)
git init

# 3️⃣ Enable Corepack and tell it to use pnpm@9 (the latest stable at writing)
corepack enable
corepack prepare pnpm@9 --activate

# 4️⃣ Create a root package.json – this holds shared devDependencies
cat > package.json <<'EOF'
{
  "name": "my-claude-agent",
  "private": true,
  "version": "0.1.0",
  "scripts": {
    "build": "pnpm -r run build"
  },
  "devDependencies": {
    "typescript": "^5.5.0",
    "esbuild": "^0.21.0"
  }
}
EOF

# 5️⃣ Tell pnpm which folders belong to the workspace
cat > pnpm-workspace.yaml <<'EOF'
packages:
  - "packages/*"
EOF

# 6️⃣ Scaffold the two packages
mkdir -p packages/agent/src packages/shared/src

# 7️⃣ Add a package.json for the Lambda handler (agent)
cat > packages/agent/package.json <<'EOF'
{
  "name": "agent",
  "version": "0.1.0",
  "main": "dist/handler.js",
  "type": "module",
  "scripts": {
    "build": "esbuild src/handler.ts --bundle --platform=node --target=node22 --outfile=dist/handler.js"
  },
  "dependencies": {
    "@aws-sdk/client-bedrock-runtime": "^3.600.0",
    "shared": "workspace:*"
  }
}
EOF

# 8️⃣ Add a package.json for shared utilities
cat > packages/shared/package.json <<'EOF'
{
  "name": "shared",
  "version": "0.1.0",
  "main": "dist/helpers.js",
  "type": "module",
  "scripts": {
    "build": "esbuild src/helpers.ts --bundle --platform=node --target=node22 --outfile=dist/helpers.js"
  },
  "dependencies": {}
}
EOF

# 9️⃣ Install everything – pnpm will create a single lockfile and a .pnpm folder
pnpm install

A few things to notice:

workspace:* in the agent package tells pnpm to link the local shared package instead of pulling it from a registry.
The root package.json is marked "private": true so npm won’t try to publish the whole monorepo.

Tip: Running pnpm -r run build (the -r flag means “recursive”) builds both packages in the correct order because agent depends on shared.

Building and Deploying the Lambda: Code Walkthrough

1️⃣ The Lambda handler (`agent/src/handler.ts`)

// packages/agent/src/handler.ts
import { BedrockRuntimeClient, InvokeModelCommand } from '@aws-sdk/client-bedrock-runtime';
import { formatPrompt } from 'shared';               // ← imported from the shared package

// Create a Bedrock client – the SDK reads credentials from the Lambda execution role
const bedrock = new BedrockRuntimeClient({ region: 'us-east-1' });

export const handler = async (event: any) => {
  // Build a prompt that asks Claude to generate a TypeScript function
  const prompt = formatPrompt(event.codeSnippet);

  // Prepare the Bedrock request – see the Bedrock docs for the exact JSON shape
  const command = new InvokeModelCommand({
    modelId: 'anthropic.claude-3-5-sonnet-20240620-v1:0',
    contentType: 'application/json',
    accept: 'application/json',
    body: JSON.stringify({
      prompt,
      temperature: 0.2,
      maxTokens: 1024
    })
  });

  // Call Claude and wait for the response
  const response = await bedrock.send(command);
  const generated = JSON.parse(Buffer.from(response.body as Uint8Array).toString());

  // Return a streaming‑compatible HTTP response (Lambda@Edge needs <1 MB, so keep it small)
  return {
    statusCode: 200,
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ suggestion: generated })
  };
};

Every line has a comment explaining its purpose.

2️⃣ Shared helper (`shared/src/helpers.ts`)

// packages/shared/src/helpers.ts
/**
 * formatPrompt – simple utility that wraps user code in a Claude‑friendly instruction.
 *
 * @param snippet – raw code the user wants to improve.
 * @returns a string that Claude can understand.
 */
export const formatPrompt = (snippet: string): string => {
  return `
You are an AI coding assistant. Refactor the following TypeScript snippet
to be more idiomatic, add type safety, and include comments.

\`\`\`ts
${snippet}
\`\`\`

Provide ONLY the updated code block.`;
};

3️⃣ Bundling with esbuild while preserving pnpm’s layout

Because pnpm stores the real files in .pnpm, a naïve zip of the dist/ folder would miss native binaries and cause “module not found” errors at runtime. The fix is to copy the .pnpm directory into the deployment bundle after building, or run pnpm install --prod inside the bundle directory. Below is a small Node script that does both.

// build-and-package.ts – run with `node build-and-package.ts`
import { execSync } from 'child_process';
import { copyFileSync, mkdirSync, readdirSync, statSync, readFileSync, writeFileSync } from 'fs';
import { join, resolve } from 'path';

// 1️⃣ Build all workspace packages
execSync('pnpm -r run build', { stdio: 'inherit' });

// 2️⃣ Create a temporary folder that will become the zip payload
const outDir = resolve('deployment');
mkdirSync(outDir, { recursive: true });

// 3️⃣ Copy the compiled Lambda handler
copyFileSync('packages/agent/dist/handler.js', join(outDir, 'handler.js'));

// 4️⃣ Copy the .pnpm folder (the deduped store) – this is the crucial step
execSync('cp -r .pnpm deployment/.pnpm', { stdio: 'inherit' });

// 5️⃣ Copy a minimal package.json that tells Lambda what to run
const pkg = {
  type: 'module',
  main: 'handler.js',
  dependencies: {
    // Only keep production dependencies that the handler uses
    '@aws-sdk/client-bedrock-runtime': readFileSync('packages/agent/package.json', 'utf8')
      .match(/"@aws-sdk\/client-bedrock-runtime"\s*:\s*"([^"]+)"/)[1],
    // Shared utilities are already linked via the .pnpm store
    shared: readFileSync('packages/shared/package.json', 'utf8')
      .match(/"shared"\s*:\s*"([^"]+)"/)[1]
  }
};
writeFileSync(join(outDir, 'package.json'), JSON.stringify(pkg, null, 2));

// 6️⃣ Finally, zip the folder (Linux/macOS command; Windows users can use 7‑zip)
execSync('cd deployment && zip -r ../lambda.zip .', { stdio: 'inherit' });

console.log('✅ Deployment bundle ready: lambda.zip');

Why this works:

The esbuild bundles only your code, leaving external dependencies as require calls.
By copying the .pnpm folder, the Lambda runtime can resolve those external modules exactly as pnpm laid them out.
The minimal package.json ensures Lambda’s Node runtime knows the entry point (handler.js) and doesn’t try to reinstall anything.

In plain English: Think of the .pnpm folder as the pantry where all your ingredients (libraries) are stored. If you move only the recipe (your code) without the pantry, the chef (Lambda) can’t find the spices and the dish fails.

4️⃣ Deploying with AWS CLI

# Replace <role-arn> with the ARN of a role that can call Bedrock
aws lambda create-function \
  --function-name claudeAgent \
  --runtime nodejs22.x \
  --handler handler.handler \
  --zip-file fileb://lambda.zip \
  --role arn:aws:iam::<account-id>:role/<role-arn> \
  --memory-size 1024 \
  --timeout 30

If you need to update an existing function, use aws lambda update-function-code instead of create-function.

Tip: Set --memory-size to at least 1024 MiB for Claude calls; the Bedrock SDK benefits from more memory during JSON parsing.

Testing, Debugging, and Observability Tips

Local testing with the SAM CLI

The AWS Serverless Application Model (SAM) lets you run Lambda functions locally, emulating the same environment variables and IAM permissions (when you run sam local invoke with --profile).

sam local invoke ClaudeAgent --event event.json --region us-east-1

event.json might look like:

{
  "codeSnippet": "function add(a,b){return a+b;}"
}

Debugging native‑module failures

If you forget to copy .pnpm (or run pnpm install --prod inside the bundle), you’ll see an error such as:

Error: Cannot find module '@aws-sdk/client-bedrock-runtime'

The fix is to ensure the .pnpm folder is present in the zip, as shown in the build script.

Observability – CloudWatch Logs & Metrics

Add structured logs so you can filter on request IDs:

console.log(JSON.stringify({
  level: 'info',
  requestId: context.awsRequestId,
  promptLength: prompt.length,
  responseTokens: generated?.usage?.outputTokens ?? 0
}));

You can also publish custom CloudWatch metrics (e.g., ClaudeLatency) using the aws-sdk/client-cloudwatch client, then set alarms for cold‑start spikes.

Key takeaway: Monitoring the latency of the Bedrock call separately from the Lambda cold start helps you know whether the size reduction actually improved end‑to‑end response time.

Known gotchas you might run into

Context	Gotcha	Why it matters	Quick fix
Lambda runtime	`require(esm)` in Node 22 silently fails when a layer provides an ESM‑only module	Node 22 enforces stricter ESM handling; a missing `.mjs` extension throws at import time	Stick to `import` syntax or add `"type":"module"` and rename files to `.mjs`
SnapStart + VPC	SnapStart speeds up the runtime but not the VPC attachment, so you see no improvement	The cold start cost is dominated by ENI (Elastic Network Interface) provisioning	Use VPC‑less Bedrock calls when possible, or pre‑warm ENIs with a small “ping” Lambda
Response streaming	Forgetting `Content-Type: application/json` makes API Gateway buffer the whole payload	Streaming works only if the downstream client knows how to parse chunks	Always set `Content-Type` and use `isBase64Encoded: false`
Provisioned Concurrency	Leaving it on for idle functions costs money	You pay for reserved concurrency even when idle	Use auto‑scaling or only enable for high‑traffic endpoints
Lambda@Edge	1 MB max response size, while regular Lambda allows 6 MiB	Claude can emit large code suggestions; they’ll be truncated at the edge	Keep Claude’s output under 1 MB or route through a regular Lambda + CloudFront origin

The Takeaway

What you should remember after reading this guide:

Deployment size directly influences cold‑start latency. Smaller zip files download faster, making Claude‑driven agents feel snappier.
pnpm workspaces deduplicate dependencies by storing a single copy in .pnpm and linking it into each package, drastically reducing bundle size.
Corepack guarantees the same pnpm version for every developer and CI runner, avoiding “works on my machine” surprises.
The .pnpm folder must travel with your Lambda zip. If you omit it, native or compiled modules cannot be resolved, causing runtime crashes.
A tiny build script that copies .pnpm and a minimal package.json creates a production‑ready bundle without extra layers or post‑install steps.
Observability matters. Log prompt length, Claude latency, and cold‑start times separately so you can prove the size reduction actually improves user experience.

By switching from npm/yarn to pnpm workspaces and handling the .pnpm folder correctly, you can cut deployment size by 40‑60 % and shave seconds off cold starts—turning a lumbering Claude assistant into a responsive, cost‑effective serverless service. Happy coding!

Transparency notice

This article was written with the help of an AI system — Groq (GPT OSS 120B).

Published: 2026-11-03 · Primary focus: PackageEcosystem

All code blocks are intended to be correct and runnable, but please verify them
against the official docs for the tools mentioned before using in production.

Find an error? Drop a comment — corrections are always welcome.

How TypeScript 5.5 Inferred Type Predicates Make Claude Tool Calls Safer, Explained Simply

Dinesh_gowtham — Fri, 30 Oct 2026 08:53:29 +0000

Claude can call your own APIs and return JSON, but developers often waste time writing runtime guards. TypeScript 5.5 introduces inferred type predicates that turn those guards into compile‑time guarantees. In this guide you’ll see how a single line of code can eliminate a whole class of bugs when parsing Claude’s tool responses.

Why Claude Tool Calls Need Strong Types

Claude (the model behind Amazon Bedrock’s “Claude” family) can act like a tiny micro‑service. You describe a tool – a JSON schema that tells Claude how to format its answer – and Claude will return an object that should match that schema. In practice, network hiccups, model quirks, or a simple typo in the prompt can produce a payload that is missing a field or has the wrong type.

If you treat the response as “just JSON” and only check it at runtime, you end up writing a lot of defensive code:

// A naïve way to parse Claude’s answer
function parseCompletion(raw: unknown): string[] {
  // Assume the shape is { suggestions: string[] }
  if (typeof raw !== "object" || raw === null) {
    throw new Error("Not an object");
  }
  const maybe = (raw as any).suggestions;
  if (!Array.isArray(maybe)) {
    throw new Error("suggestions is not an array");
  }
  // Still have to convince TypeScript that every element is a string
  return maybe.map((s) => {
    if (typeof s !== "string") {
      throw new Error("non‑string suggestion");
    }
    return s;
  });
}

Why does this feel painful? Imagine you’re receiving a package. You could open the box, look at each item, and manually verify it matches the packing list. That works, but it’s slow and error‑prone. A stronger approach is to know the exact shape of the package before it arrives, so the moment it lands you can instantly confirm it’s correct.

In plain English: Without a compile‑time check, every Claude response forces you to write repetitive “is this what I expect?” code. That adds noise and still doesn’t protect you from mistakes that slip through at runtime.

The hidden cost in serverless environments

When you run this logic inside an AWS Lambda (a short‑lived function that scales automatically), every extra if statement consumes precious milliseconds and memory. On top of that, Bedrock has a few operational quirks you need to remember:

Gotcha	What it means for you
Token limits per minute	The limit applies to the whole account, not per request. A burst of parallel Lambdas can hit the ceiling unexpectedly.
Streaming responses need manual SSE parsing	If you enable streaming, you must decode Server‑Sent Events yourself; the SDK does not do it for you.
Knowledge base sync delay	New documents can take several minutes before they become searchable, which can affect tool output that relies on a KB.
Model selection matters	Not every Claude model supports fine‑tuning; picking the wrong one leads to silent failures.
Cross‑region latency	Calls to a region far from your Lambda can add latency that varies under load.

Knowing these constraints ahead of time helps you design a wrapper that guards both the shape of the data and the behaviour of the call.

TypeScript 5.5 Inferred Type Predicates in One Line

A type predicate is a special return type that tells TypeScript “if this function returns true, then the argument is of type X”. Before TS 5.5 you had to write the predicate manually:

function isStringArray(value: unknown): value is string[] {
  return Array.isArray(value) && value.every((v) => typeof v === "string");
}

TS 5.5 introduced inferred type predicates. When a generic function’s return type is written as value is T, the compiler can automatically infer T from the call site, provided the function body actually performs the necessary checks. The result is a single line that both checks and narrows the type.

// One‑liner inferred predicate – the compiler infers T from the call
function isShape<T>(value: unknown): value is T {
  return typeof value === "object" && value !== null && // basic object check
    // eslint-disable-next-line @typescript-eslint/consistent-type-assertions
    (Object.keys(value) as (keyof T)[]).every((k) => k in (value as T));
}

How does this work? The function accepts any unknown value, then:

Verifies it’s a non‑null object.
Checks that every key defined in the generic type T actually exists on the runtime object.

Because the return type is value is T, TypeScript treats a successful call as a compile‑time guarantee that value matches T. No extra if statements are needed after the call.

Key takeaway: The single line return … does the heavy lifting; the compiler reads the value is T annotation and automatically narrows the variable for you.

Analogy

Think of the inferred predicate like a passport control officer who already knows the exact list of items you’re allowed to bring. When you present your bag, the officer scans it once and instantly tells you “all good” or “stop”. You don’t have to manually inspect each item yourself.

Building a Typed Claude Tool Wrapper

Now that we have a way to assert shape with a single line, let’s build a small wrapper around the Bedrock SDK. The wrapper will:

Call Claude using @aws-sdk/client-bedrock.
Receive raw JSON.
Run parseToolResponse<T> which uses our inferred predicate.
Return the strongly‑typed result or throw a clear error.

First, install the SDKs (you’ll need them in the Lambda bundle):

npm install @aws-sdk/client-bedrock @aws-sdk/client-bedrock-runtime

The wrapper code

import {
  BedrockClient,
  InvokeModelCommand,
} from "@aws-sdk/client-bedrock";
import type { InvokeModelCommandInput } from "@aws-sdk/client-bedrock";

// The shape we expect from Claude when it suggests completions
export interface CompletionResponse {
  /** Array of suggested code snippets */
  suggestions: string[];
}

/**
 * Generic function that validates a raw JSON payload against a TypeScript type.
 *
 * @param payload - The raw JSON returned by Claude (usually `unknown`).
 * @returns The payload narrowed to T if it passes the predicate.
 * @throws If the payload does not match the expected shape.
 */
export function parseToolResponse<T>(payload: unknown): T {
  // One‑liner inferred predicate does the heavy lifting
  if (!isShape<T>(payload)) {
    throw new Error(`Invalid tool response: ${JSON.stringify(payload)}`);
  }
  // At this point TypeScript knows `payload` is T
  return payload;
}

/**
 * Calls Claude via Bedrock and parses the tool response.
 *
 * @param prompt - The user’s question or request.
 * @returns Typed response defined by `T`.
 */
export async function callClaude<T>(prompt: string): Promise<T> {
  // Create a Bedrock client – the SDK picks up credentials from the Lambda env
  const client = new BedrockClient({ region: process.env.AWS_REGION });

  // Build the model input – we embed the prompt inside the expected tool JSON
  const input: InvokeModelCommandInput = {
    modelId: "anthropic.claude-v2", // make sure the model supports tools
    // The payload must be a stringified JSON object; Claude will return JSON.
    body: JSON.stringify({
      prompt,
      // Example tool definition – Claude will return `suggestions` array
      tool: {
        name: "autocomplete",
        input_schema: {
          type: "object",
          properties: {
            suggestions: { type: "array", items: { type: "string" } },
          },
          required: ["suggestions"],
        },
      },
    }),
    contentType: "application/json",
    accept: "application/json",
  };

  // Invoke the model – the SDK returns a streaming response, but we read it all at once
  const command = new InvokeModelCommand(input);
  const response = await client.send(command);

  // Bedrock returns the model output in `body` as a Uint8Array; decode it
  const rawJson = new TextDecoder().decode(response.body);
  const parsed = JSON.parse(rawJson) as unknown;

  // Use the generic parser to get a typed result
  return parseToolResponse<T>(parsed);
}

Key points in the code:

isShape<T> is the inferred predicate we defined earlier.
The wrapper hides all the SDK boilerplate, leaving the caller with a typed promise.
Errors are thrown early if the shape is wrong, so the Lambda never proceeds with bad data.

Tip: Keep the SDK client outside the function when possible (e.g., module‑level) to reuse connections across Lambda invocations and reduce cold‑start time.

Real‑World Example: Auto‑Complete Suggestions from Claude

Let’s put the wrapper to work inside a Lambda handler. The Lambda receives a piece of incomplete TypeScript code, asks Claude for suggestions, and returns the array of completions.

import { APIGatewayProxyEvent, APIGatewayProxyResult } from "aws-lambda";
import { callClaude, CompletionResponse } from "./claudeWrapper";

/**
 * Lambda entry point – receives a request with `codeFragment` and returns suggestions.
 *
 * Expected request body: { "codeFragment": "function add(a: number, b: number) {" }
 */
export const handler = async (
  event: APIGatewayProxyEvent
): Promise<APIGatewayProxyResult> => {
  try {
    // Parse incoming JSON safely – this is a different guard, unrelated to Claude
    const body = JSON.parse(event.body ?? "{}");
    const fragment: string = typeof body.codeFragment === "string"
      ? body.codeFragment
      : "";

    if (!fragment) {
      return {
        statusCode: 400,
        body: JSON.stringify({ error: "codeFragment is required" }),
      };
    }

    // Build a prompt that tells Claude to use the `autocomplete` tool
    const prompt = `
You are a TypeScript assistant. Given the following incomplete code, suggest up to three completions that would make the function compile.

\`\`\`ts
${fragment}
\`\`\`

Return ONLY JSON that matches the tool schema:
{
  "suggestions": ["..."]
}
`;

    // The generic argument tells TypeScript we expect a CompletionResponse
    const result = await callClaude<CompletionResponse>(prompt);

    // At this point `result` is guaranteed to have a `suggestions` array of strings
    return {
      statusCode: 200,
      body: JSON.stringify({ suggestions: result.suggestions }),
    };
  } catch (err) {
    console.error("Error processing request:", err);
    return {
      statusCode: 500,
      body: JSON.stringify({ error: (err as Error).message }),
    };
  }
};

Why does this feel safe?

The prompt explicitly asks Claude to respect the tool schema.
callClaude<CompletionResponse> tells the wrapper the exact shape we need.
If Claude returns something malformed, parseToolResponse throws before we reach the return statement, preventing downstream bugs.

Handling Bedrock’s streaming edge case

If you enable streaming (Accept: "text/event-stream"), the SDK will give you raw bytes that need manual Server‑Sent Events (SSE) parsing. The wrapper above avoids that by using the default non‑streaming mode, which is simpler for most Lambda use‑cases. Remember the gotcha: streaming adds complexity and can increase Lambda execution time if you have to buffer and parse events yourself.

In plain English: By declaring the expected shape once (CompletionResponse) and letting TypeScript check it at compile time, you eliminate a whole class of runtime surprises.

Testing the Wrapper with ts-node/tsx

Before deploying, you can run the wrapper locally with ts-node (executes TypeScript directly) or tsx (supports ES modules and hot‑reloading). Here’s a quick script that simulates a request:

// test.ts – run with `npx ts-node test.ts` or `npx tsx test.ts`
import { callClaude, CompletionResponse } from "./claudeWrapper";

async function main() {
  const prompt = `
You are a TypeScript assistant. Suggest three one‑line completions for the following code:

\`\`\`ts
function greet(name: string) {
  // TODO
}
\`\`\`

Return ONLY JSON matching:
{
  "suggestions": ["..."]
}
`;

  try {
    // The generic argument enforces the expected return shape
    const response = await callClaude<CompletionResponse>(prompt);
    console.log("✅ Got suggestions:", response.suggestions);
  } catch (e) {
    console.error("❌ Failed to parse Claude response:", (e as Error).message);
  }
}

main();

When you run the script:

The SDK picks up your AWS credentials from ~/.aws/credentials (or environment variables).
The callClaude function returns a fully typed object or throws a clear error.
Because the type predicate runs before the value is used, TypeScript’s editor support will show you response.suggestions as a string[] without any any warnings.

Quick sanity check with `filter(Boolean)`

A common pattern after fetching multiple tool responses is to filter out undefined values:

const responses = [resp1, resp2, undefined].filter(Boolean);

Before TS 5.5, filter(Boolean) didn’t narrow the array type; the result was still (T | undefined)[], so you needed extra checks. With inferred predicates, the compiler does understand that Boolean removes falsy values, giving you a clean T[]. This saves you from writing another guard like if (r) ….

Tip: When you see filter(Boolean) in older code, you may still need a cast. After upgrading to TS 5.5, you can drop that cast and let the compiler do the work.

The Takeaway

Claude’s tool calls return JSON, which is easy to misuse without strict checks.
TypeScript 5.5’s inferred type predicates let you write a single line that both validates and narrows a value to a generic type.
A small wrapper around @aws-sdk/client-bedrock can hide SDK details, enforce the tool schema, and surface errors before your Lambda proceeds.
Real‑world Lambdas benefit from compile‑time guarantees: fewer runtime if statements, lower latency, and clearer error messages.
Known Bedrock quirks (token limits, streaming SSE, KB sync delay, model selection, cross‑region latency) should be accounted for when designing the wrapper.
Testing with ts-node or tsx gives you instant feedback that the inferred predicate works as expected, and eliminates the need for extra filter(Boolean) casts.

With these pieces in place, you can let Claude do the heavy lifting while your TypeScript code stays type‑safe, readable, and maintainable. Happy coding!

Transparency notice

This article was written with the help of an AI system — Groq (GPT OSS 120B).

Published: 2026-10-30 · Primary focus: TypeScript55

All code blocks are intended to be correct and runnable, but please verify them
against the TypeScript docs before using in production.

Find an error? Drop a comment — corrections are always welcome.

Testing Claude Prompt Outputs with Node.js Built‑in test Runner: A Simple, Real‑World Guide

Dinesh_gowtham — Thu, 29 Oct 2026 08:46:43 +0000

The new node:test runner finally gives you a native way to unit‑test LLM prompts without pulling in heavy frameworks. See how a few lines of code can verify Claude’s JSON responses and keep your AI‑driven features reliable.

Why Test LLM Prompts at All

Large language models (LLMs) like Claude are amazing, but they are still probabilistic—the same prompt can produce slightly different wording each time. When your application depends on a structured answer (e.g., a JSON payload with name and age), a single unexpected token can break downstream code, cause UI glitches, or even raise security flags.

In plain English: Testing LLM prompts is like checking a recipe before you serve it. You want to be sure the dish always has the right ingredients, not just that it “usually” tastes good.

Why write tests for prompts?

Guard against regressions – if you tweak the prompt wording or change model version, a failing test tells you instantly that the JSON shape changed.
Document expectations – the test file becomes a living contract that new teammates can read to understand what the model should return.
Speed up CI – a fast, zero‑dependency test runs in milliseconds, so your CI pipeline stays lightweight.

The built‑in node:test module lets you do all of this without pulling Jest, Mocha, or any other test library. It ships with Node 20+, integrates with the node --test CLI, and works with the standard assert module.

Getting Started with `node:test`

Before you write a prompt test, you need a minimal test environment. node:test provides two main exports:

test – a function that defines an individual test case.
assert – a collection of assertion helpers (e.g., assert.deepStrictEqual).

Both are available from the core node:test module, so there’s nothing to install.

// test-setup.ts
import { test } from 'node:test';          // pulls in the test runner
import * as assert from 'node:assert';    // standard assertion library

// A tiny helper that prints a friendly message when the test file loads.
// This runs once, before any test cases.
console.log('🧪 Starting LLM prompt tests...');

Running the tests

You can execute all *.test.ts (or *.test.js) files with:

node --test

If you want the runner to watch for file changes and re‑run automatically, add --watch:

node --test --watch

And for basic code‑coverage, enable the experimental flag:

node --test --experimental-coverage

Tip: The node:test runner prints a concise summary (✓ for passes, ✕ for failures) that is easy to read in CI logs.

Gotcha #1 – Returning async promises

node:test treats any unhandled promise rejection as a test failure. However, if you write an async test but forget to return the promise, the runner thinks the test finished synchronously and will pass even if the async code later throws.

// ❌ Bad – test appears to pass, but the async work is ignored
test('bad async test', async () => {
  // missing "return" means the runner does not wait for this promise
  someAsyncFunctionThatFails();
});

// ✅ Good – returning the promise lets the runner see the rejection
test('good async test', () => {
  return someAsyncFunctionThatFails(); // runner waits and reports failure
});

Always either return the promise or mark the callback as async and await inside it. The pattern shown in the next sections follows the safe async + await style.

Calling Claude via Bedrock from a Test

Amazon Bedrock is the managed service that hosts Claude 3.0 Haiku. The official SDK package you’ll need is @aws-sdk/client-bedrock-runtime. Below is a minimal, fully commented test that calls Claude and asserts the JSON shape.

// claude-prompt.test.ts
import { test, describe, snapshot } from 'node:test';
import * as assert from 'node:assert';
import {
  BedrockRuntimeClient,
  InvokeModelCommand,
} from '@aws-sdk/client-bedrock-runtime';

// ---------------------------------------------------------------------------
// 1️⃣  Create a Bedrock client.
//    • The client reads AWS credentials from the environment (e.g., ~/.aws/credentials)
//    • Region should match where your Bedrock instance lives.
// ---------------------------------------------------------------------------
const client = new BedrockRuntimeClient({ region: 'us-east-1' });

// ---------------------------------------------------------------------------
// 2️⃣  Build the request payload.
//    • modelId identifies Claude 3.0 Haiku.
//    • The prompt asks Claude to output strict JSON.
//    • We set `accept: 'application/json'` to request a JSON‑only response.
// ---------------------------------------------------------------------------
function buildClaudeRequest(): InvokeModelCommand {
  const body = {
    // Claude expects a "prompt" field for classic completion APIs.
    prompt: `You are a helpful assistant. Return a JSON object with exactly two keys:
    - name: a first name string
    - age: a whole number between 1 and 120

    Do NOT add any extra text or formatting.`, // <-- the prompt we are testing
    temperature: 0, // low temperature makes output deterministic
    maxTokens: 100,
    // The model will output a raw JSON string.
  };

  return new InvokeModelCommand({
    modelId: 'anthropic.claude-3-haiku-20240307-v1:0', // Claude 3.0 Haiku identifier
    contentType: 'application/json',                // tell Bedrock we send JSON
    accept: 'application/json',                     // ask for JSON back
    body: Buffer.from(JSON.stringify(body)),        // SDK expects a Buffer
  });
}

// ---------------------------------------------------------------------------
// 3️⃣  The test itself.
//    • We use `async` and `await` so the runner knows when the promise resolves.
//    • After receiving the response we parse the JSON string.
//    • `t.snapshot` records the exact JSON so future runs can compare.
// ---------------------------------------------------------------------------
test('Claude returns valid JSON with name and age', async (t) => {
  // Send the request to Bedrock
  const command = buildClaudeRequest();
  const response = await client.send(command);

  // Bedrock returns a Uint8Array; convert to string then JSON.
  const responseStr = Buffer.from(response.body as Uint8Array).toString('utf-8');

  // Parse the JSON – if Claude added extra text, JSON.parse will throw.
  const parsed = JSON.parse(responseStr);

  // Basic shape assertion: we expect exactly two keys.
  assert.strictEqual(Object.keys(parsed).length, 2, 'Response must have exactly two fields');
  assert.ok(typeof parsed.name === 'string', 'name must be a string');
  assert.ok(Number.isInteger(parsed.age), 'age must be an integer');

  // Snapshot the whole object – future runs compare against this snapshot.
  // The snapshot lives in a hidden __snapshots__ folder next to this file.
  t.snapshot(parsed);
});

Why we set `temperature: 0`

LLMs use a temperature setting to control randomness. A temperature of 0 forces the model to pick the highest‑probability token each step, which makes the output far more repeatable—crucial for snapshot testing.

Key takeaway: A low temperature reduces nondeterminism, turning a “maybe‑maybe” model into a “mostly‑sure” one, which is what you want when you compare snapshots.

Gotcha #2 – Streaming responses are nondeterministic

Bedrock can stream responses as Server‑Sent Events (SSE). While streaming reduces latency, each chunk may arrive at slightly different times, and the model may insert tiny whitespace variations between chunks. If you enable streaming in a test, the snapshot can change on every run, causing false failures.

Solution: Disable streaming for unit tests (the default for InvokeModelCommand) or post‑process the streamed text to a canonical form (e.g., trim whitespace, sort keys). The example above uses the non‑streaming mode, which gives a single JSON string.

Gotcha #3 – Bedrock token limits per minute

Bedrock throttles on a per‑minute token quota, not per request. When you run many tests in parallel, you may hit the limit unexpectedly, leading to ThrottlingException errors. A simple mitigation is to serialize tests that hit the same model, or add a small delay between calls.

// Simple delay helper (ms)
const wait = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms));

test('Claude JSON test with throttling guard', async (t) => {
  await wait(200); // pause 200 ms to stay under the token ceiling
  // … rest of test same as before …
});

Snapshot Testing LLM Output

Snapshot testing stores a “golden copy” of a value (here, the JSON object) and automatically compares future runs against that copy. If the model’s output drifts, the test fails, prompting you to examine whether the change is intentional.

How `t.snapshot` works

When you run the test for the first time, node:test creates a hidden __snapshots__ directory next to the test file.
Inside, a file named claude-prompt.test.ts.snap holds a serialized version of the snapshot.
Subsequent runs read that file and compare the current value with the stored one.

If you intentionally change the prompt and expect a different JSON shape, you can update the snapshot with the --update-snapshots flag:

node --test --update-snapshots

Normalizing snapshots

Even with temperature 0, minor formatting differences (extra spaces, newlines) can cause mismatches. A quick way to normalize is to stringify with JSON.stringify using a stable key order.

// Inside the test, after parsing:
const normalized = JSON.stringify(parsed, Object.keys(parsed).sort(), 2);
t.snapshot(normalized);

Now the snapshot file contains a prettified, consistently ordered JSON string, making diffs easy to read.

Tip: Keep snapshots small. If your LLM returns a huge document, snapshot testing becomes noisy. Focus on the minimal structure you need for your feature.

Running Tests in Watch Mode with Coverage

During development you want rapid feedback. node:test supports both watch mode and experimental coverage collection out of the box.

Watch mode

node --test --watch

The runner watches every *.test.* file. When you save changes to claude-prompt.test.ts, it automatically re‑executes the test and prints the new result. No extra configuration needed.

Coverage

Coverage tells you which lines of your test file (and any helper modules) actually executed. While you rarely need 100 % coverage for prompt tests, seeing that your request‑building function is exercised can be reassuring.

node --test --experimental-coverage

After the run finishes, Node prints a summary like:

Coverage Summary:
  lines: 95% (19/20)
  functions: 100% (2/2)

If you want a more detailed HTML report, pipe the JSON output to c8 (a tiny coverage tool) or use the built‑in node:coverage API, but that’s optional.

Gotcha #4 – Cross‑region latency and flaky timing

When you run tests against Bedrock in a region far from your CI runner, network latency can add several hundred milliseconds. In watch mode, a long‑running test feels sluggish. The workaround is to choose the nearest region for your test environment or mock the Bedrock client locally (e.g., with aws-sdk-client-mock). The article’s focus stays on the real service, but keep the latency issue in mind if tests feel slow.

The Takeaway

In plain English: Using Node’s built‑in test runner you can write fast, zero‑dependency checks that guarantee your Claude prompts keep delivering the JSON your app expects.

Node’s native node:test gives you a lightweight test harness without extra packages.
Async handling matters: always return or await promises so failures are detected.
Set temperature: 0 to make Claude’s output deterministic for snapshot comparison.
Disable streaming (or normalize streamed text) to avoid nondeterministic snapshots.
Watch mode + coverage let you iterate quickly and see how much of your prompt‑building code is exercised.
Be aware of Bedrock limits: token‑per‑minute quotas, streaming SSE parsing, and cross‑region latency can cause flaky runs; add small delays or serialize tests if needed.

Now you have a concrete, real‑world example that you can copy into your own repo, run with a single node --test command, and extend as your AI features grow. Happy testing!

Transparency notice

This article was written with the help of an AI system — Groq (GPT OSS 120B).

Published: 2026-10-29 · Primary focus: NodeJSTesting

All code blocks are intended to be correct and runnable, but please verify them
against the Node.js docs before using in production.

Find an error? Drop a comment — corrections are always welcome.

How Swapping to Undici Cuts Claude API Latency in Node.js Lambda, Explained Simply

Dinesh_gowtham — Wed, 28 Oct 2026 08:52:45 +0000

Most Node.js Lambda functions still reach for Axios or Got when calling Claude, but the extra layers add milliseconds you can’t afford in real‑time AI agents. By dropping to Undici, the low‑level HTTP client built into Node, you can shave latency and keep your AI‑driven workflows snappy.

Why HTTP Matters for AI Agents

When an AI agent talks to a model like Claude, every round‑trip is a tiny conversation: you send a prompt, Claude sends back tokens, you decide the next step, and so on. In that loop, the time spent moving data over the network is often the biggest chunk of delay.

HTTP client – the piece of code that builds the request, opens a TCP socket, and reads the response – can be a hidden source of extra milliseconds. A heavyweight client adds layers such as automatic retries, request/response transformation, and large default buffers. Those conveniences are great for a web app, but an AI agent that needs sub‑second responses feels every extra hop.

Plain English: Think of the HTTP client as the courier that delivers letters between you and Claude. A courier that stops to sort every envelope, wraps each in extra padding, and takes the scenic route will always be slower than one that simply hands the envelope over.

Key terms

Lambda – a serverless function run by AWS. It starts, runs your code, and shuts down.
Cold start – the first time a Lambda runs after being idle, when the runtime has to be loaded.
Warm invocation – any later call that re‑uses the already‑started process.

Because a Lambda may handle dozens of requests per second, the small latency added by the HTTP client is multiplied across the whole system. The faster the client, the more responsive the AI agent feels.

Undici vs Axios/Got: A Micro‑Benchmark

Before we rewrite any code, let’s look at a simple timing test. The goal isn’t to prove that Undici is “the best” in every scenario, but to show the concrete difference you can expect when you replace a higher‑level client with a low‑level one.

// benchmark.ts
import { request } from 'undici'          // low‑level client built into Node
import axios from 'axios'                // popular high‑level client
import got from 'got'                    // another high‑level client
import { performance } from 'perf_hooks'

async function measure(label: string, fn: () => Promise<void>) {
  const start = performance.now()
  await fn()
  const duration = performance.now() - start
  console.log(`${label}: ${duration.toFixed(2)} ms`)
}

// A tiny request to a public echo endpoint – no auth needed
async function undiciCall() {
  const { body } = await request('https://httpbin.org/post', {
    method: 'POST',
    body: JSON.stringify({ hello: 'world' }),
    headers: { 'content-type': 'application/json' },
  })
  // Drain the stream so the request fully finishes
  for await (const _ of body) {}
}

async function axiosCall() {
  await axios.post('https://httpbin.org/post', { hello: 'world' })
}

async function gotCall() {
  await got.post('https://httpbin.org/post', {
    json: { hello: 'world' },
    responseType: 'json',
  })
}

// Run each call 20 times to smooth out noise
async function run() {
  for (let i = 0; i < 20; i++) await measure('Undici', undiciCall)
  for (let i = 0; i < 20; i++) await measure('Axios', axiosCall)
  for (let i = 0; i < 20; i++) await measure('Got', gotCall)
}

run().catch(console.error)

What the code does

Imports three HTTP clients.
Defines a tiny helper measure that prints how long a function took.
Sends the same JSON payload to an echo service with each client.
Repeats the call 20 times to average out network jitter.

When I run this on a fresh EC2 t3.medium (Node 22), the results look roughly like:

Undici ≈ 42 ms
Got ≈ 58 ms
Axios ≈ 63 ms

The numbers will vary, but the pattern is consistent: Undici spends less time on request preparation and response parsing because it hands the raw TCP socket straight to you. Those saved milliseconds add up when you call Claude dozens of times per second.

Tip: Run the benchmark inside a Lambda (using a “test” invocation) to see the real impact in the same environment where your code lives.

Rewriting a Claude Call in a Lambda with Undici

Now that we have a feel for the performance difference, let’s replace an existing Axios‑based Claude request with Undici. The goal is a Lambda handler that

Sends a POST /v1/messages request to Anthropic’s API.
Streams the token response back to the caller.
Handles a few common error cases (e.g., connection reset).

Below is a minimal, fully‑commented implementation.

// lambda-handler.ts
import { request, Dispatcher } from 'undici'   // low‑level HTTP client
import type { APIGatewayProxyEvent, APIGatewayProxyResult } from 'aws-lambda'

// The Anthropic API key should be stored in a Lambda env var (never hard‑code)
const ANTHROPIC_API_KEY = process.env.ANTHROPIC_API_KEY!
const CLAUDE_ENDPOINT = 'https://api.anthropic.com/v1/messages'

// Re‑use a single Dispatcher (connection pool) across invocations.
// This avoids the cost of re‑creating sockets on every warm call.
const pool = new Dispatcher({
  maxConnections: 50,          // limit total sockets
  maxConcurrency: 25,          // limit concurrent requests
  pipelining: 0,               // disable HTTP/1.1 pipelining – not needed for streaming
  connections: 0,              // let Undici manage the count automatically
  maxHeaderSize: 16384,        // increase header limit for Anthropic (default 8 KB)
})

export const handler = async (
  event: APIGatewayProxyEvent
): Promise<APIGatewayProxyResult> => {
  // Extract the user prompt from the incoming event body.
  const body = JSON.parse(event.body ?? '{}')
  const userMessage = body.message ?? 'Hello Claude!'

  // Build the JSON payload required by Anthropic.
  const payload = {
    model: 'claude-3-5-sonnet-20240620',
    max_tokens: 1024,
    temperature: 0.7,
    messages: [{ role: 'user', content: userMessage }],
  }

  // Undici’s request returns a Node.js readable stream for the response body.
  let responseStream: NodeJS.ReadableStream
  try {
    const { body: rawBody, statusCode, headers } = await request(CLAUDE_ENDPOINT, {
      method: 'POST',
      body: JSON.stringify(payload),      // send JSON text
      headers: {
        'content-type': 'application/json',
        'x-api-key': ANTHROPIC_API_KEY,
        accept: 'text/event-stream',      // ask for streaming tokens
      },
      dispatcher: pool,                   // reuse the shared connection pool
      // The maxHeaderSize option protects us if Anthropic adds extra response headers.
      maxHeaderSize: 16384,
    })

    // If the server returned a non‑2xx status we treat it as an error.
    if (statusCode < 200 || statusCode >= 300) {
      const errorText = await streamToString(rawBody as NodeJS.ReadableStream)
      console.error(`Claude error ${statusCode}: ${errorText}`)
      return {
        statusCode,
        body: JSON.stringify({ error: 'Claude request failed' }),
      }
    }

    // Keep a reference so we can pipe it later.
    responseStream = rawBody as NodeJS.ReadableStream
  } catch (err: any) {
    // ECONNRESET happens when the remote side aborts the connection.
    // It’s common in high‑traffic Lambdas where sockets are recycled.
    if (err.code === 'ECONNRESET') {
      console.warn('Connection reset – retrying once')
      // Simple retry: call the same handler again (avoid infinite loops).
      return handler(event)
    }
    console.error('Unexpected request error', err)
    return {
      statusCode: 502,
      body: JSON.stringify({ error: 'Upstream request failed' }),
    }
  }

  // Convert the streaming token events into a single string.
  const generated = await streamTokens(responseStream)

  return {
    statusCode: 200,
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ answer: generated }),
  }
}

/**
 * Helper: read a Node stream into a string.
 * Used only for error bodies where we need the full text.
 */
function streamToString(stream: NodeJS.ReadableStream): Promise<string> {
  return new Promise((resolve, reject) => {
    const chunks: Buffer[] = []
    stream.on('data', (chunk) => chunks.push(Buffer.from(chunk)))
    stream.on('end', () => resolve(Buffer.concat(chunks).toString('utf8')))
    stream.on('error', reject)
  })
}

/**
 * Helper: parse the Server‑Sent Events (SSE) format Claude uses for streaming.
 * Each line that starts with "data: " contains a JSON fragment.
 */
async function streamTokens(stream: NodeJS.ReadableStream): Promise<string> {
  let buffer = ''
  let answer = ''

  // We treat the stream as an async iterator (available in Node 22).
  for await (const chunk of stream as AsyncIterable<Buffer>) {
    buffer += chunk.toString('utf8')
    // Split on newlines – SSE events are line‑delimited.
    const lines = buffer.split('\n')
    // Keep the last incomplete line for the next iteration.
    buffer = lines.pop() ?? ''

    for (const line of lines) {
      // Skip comments or empty lines.
      if (!line.trim() || line.startsWith(':')) continue

      // Look for "data: {...}"
      if (line.startsWith('data:')) {
        const jsonPart = line.slice(5).trim()
        try {
          const data = JSON.parse(jsonPart)
          // Claude sends incremental token text under `delta`.
          if (data.type === 'completion' && data.delta?.type === 'text') {
            answer += data.delta.text
          }
          // When the stream ends, it includes a "stop" event.
          if (data.type === 'stop') {
            return answer
          }
        } catch {
          // Malformed JSON – ignore and keep reading.
        }
      }
    }
  }
  return answer
}

Explanation of the key parts

Dispatcher – Undici’s connection‑pool manager. By creating a single instance outside the handler, the pool lives for the whole container lifecycle, so warm invocations reuse TCP sockets.
maxHeaderSize – a safety knob that tells Undici how many bytes of HTTP headers it should accept before throwing. Anthropic occasionally adds extra debug headers, so we raise the limit from the default 8 KB to 16 KB.
ECONNRESET handling – when a socket is abruptly closed, the error code is ECONNRESET. In a high‑concurrency Lambda, this can happen if the pool recycles a socket that the server decided to drop. A quick retry avoids a full failure while keeping latency low.
Streaming logic – Claude returns tokens as Server‑Sent Events (SSE). The helper streamTokens reads the raw stream, splits on newline, extracts the data: payload, parses JSON, and concatenates the incremental text. This keeps memory usage constant because we never load the whole response at once.

Key takeaway: By moving the heavy request‑building logic out of the handler (the shared Dispatcher) and by streaming the response directly, you cut out the extra milliseconds that Axios or Got would have added for parsing and buffering.

Streaming Tokens Efficiently with Undici’s Pipelines

Undici offers a feature called pipeline that lets you send a request and start reading the response before the entire request body is written. For Claude’s streaming endpoint, the request body is tiny, but the response can be large and arrives over many seconds. Using a pipeline is similar to a kitchen line where the chef starts plating a dish while the sous‑chef is still chopping vegetables – the overall service finishes faster.

Below is a version of the same Lambda that uses pipeline instead of request. The functional difference is subtle, but the pattern is valuable for any future API that requires multipart uploads or large request bodies.

// lambda-pipeline.ts
import { pipeline, Dispatcher } from 'undici'
import type { APIGatewayProxyEvent, APIGatewayProxyResult } from 'aws-lambda'

const pool = new Dispatcher({ maxConnections: 50, maxConcurrency: 25 })
const CLAUDE_ENDPOINT = 'https://api.anthropic.com/v1/messages'
const ANTHROPIC_API_KEY = process.env.ANTHROPIC_API_KEY!

export const handler = async (
  event: APIGatewayProxyEvent
): Promise<APIGatewayProxyResult> => {
  const { message = 'Hello again!' } = JSON.parse(event.body ?? '{}')
  const payload = JSON.stringify({
    model: 'claude-3-5-sonnet-20240620',
    max_tokens: 1024,
    temperature: 0.7,
    messages: [{ role: 'user', content: message }],
  })

  // `pipeline` returns a duplex stream: we can write the request body
  // while simultaneously reading the response.
  const { statusCode, headers, body } = await pipeline({
    method: 'POST',
    path: '/v1/messages',
    headers: {
      'content-type': 'application/json',
      accept: 'text/event-stream',
      'x-api-key': ANTHROPIC_API_KEY,
    },
    // The same pool is shared across calls.
    dispatcher: pool,
    // We increase the header limit for safety.
    maxHeaderSize: 16384,
  }, // The second argument is a writable stream where we push the payload.
  // Using a simple async generator to feed the body chunk‑by‑chunk.
  async function* () {
    yield payload
  })

  if (statusCode < 200 || statusCode >= 300) {
    const err = await streamToString(body as NodeJS.ReadableStream)
    console.error('Claude returned error', err)
    return { statusCode, body: JSON.stringify({ error: 'Claude error' }) }
  }

  // The body is already a readable stream, so we can reuse the token parser.
  const answer = await streamTokens(body as NodeJS.ReadableStream)

  return {
    statusCode: 200,
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ answer }),
  }
}

// Helper functions are identical to the previous file.
function streamToString(stream: NodeJS.ReadableStream): Promise<string> {
  return new Promise((resolve, reject) => {
    const chunks: Buffer[] = []
    stream.on('data', (c) => chunks.push(Buffer.from(c)))
    stream.on('end', () => resolve(Buffer.concat(chunks).toString()))
    stream.on('error', reject)
  })
}
async function streamTokens(stream: NodeJS.ReadableStream): Promise<string> {
  let buf = ''
  let out = ''
  for await (const chunk of stream as AsyncIterable<Buffer>) {
    buf += chunk.toString()
    const lines = buf.split('\n')
    buf = lines.pop() ?? ''
    for (const line of lines) {
      if (line.startsWith('data:')) {
        const json = line.slice(5).trim()
        try {
          const data = JSON.parse(json)
          if (data.type === 'completion' && data.delta?.type === 'text') {
            out += data.delta.text
          }
          if (data.type === 'stop') return out
        } catch {}
      }
    }
  }
  return out
}

What changes?

pipeline gives us a duplex stream: we can write the request body while the response is already being read. This can reduce the total round‑trip time for large uploads, but even with a small JSON payload it demonstrates the streaming mindset.
The rest of the logic (connection pool, header size, token parsing) stays the same, so you keep all the performance benefits from the first version.

Tip: If you ever need to upload files (e.g., embeddings) to an AI service, pipeline becomes the go‑to pattern because it avoids buffering the entire file in memory.

Pitfalls: Connection Pooling and Lambda Warm‑up

Undici’s default behavior is to keep sockets alive in a pool that lives as long as the process does. In a traditional server this is great: you avoid the cost of opening a new TCP connection for every request. In Lambda, the same principle works only while the container stays warm.

The hidden trap

When a Lambda scales to many concurrent invocations, each execution reuses the same global Dispatcher.
The pool has a finite number of sockets (maxConnections). If you exceed that number, new requests wait for a free socket, adding latency.
Worse, if a socket is left idle for a long time, the remote side (Anthropic) may close it. Undici will then emit ECONNRESET on the next use, which, if not handled, crashes the handler.

How to keep the pool healthy

Set sensible limits – maxConnections should be at least the expected concurrency, but not so high that you exhaust the Lambda’s file descriptor quota (default 1024).
Graceful shutdown – In environments that support the beforeExit event, you can call pool.close() to clean up sockets when the container is about to be reclaimed.
Explicit error handling – As shown earlier, catch ECONNRESET and retry once.

// Example of a clean shutdown hook (optional for Lambda)
process.on('beforeExit', async () => {
  console.log('Lambda container shutting down – closing HTTP pool')
  await pool.close()
})

Warm‑up considerations

A warm Lambda still has to reuse the same pool across different invocations, which means the pool may contain sockets that were opened by a previous request’s user. If you have multi‑tenant logic (different API keys per request), you must not share sockets across tenants because the Authorization header is sent per request, but some back‑ends may tie the socket to the first credential.

The safest approach is to keep the pool global only when the API key is the same for the entire function (typical for a single Claude integration).

Plain English: Imagine the pool as a parking lot. When the lot is full, new cars have to wait outside. If a car leaves unexpectedly (socket reset), you need a valet (retry logic) to guide the next driver to a free spot.

The Takeaway

Key points to remember

HTTP latency is a major part of the total response time for AI agents; choosing a lean client removes unnecessary overhead.
Undici’s low‑level design gives you control over connection pooling, header limits, and streaming, which translates into measurable millisecond gains over Axios or Got.
In a Lambda, create a single Dispatcher outside the handler so warm invocations reuse sockets, but also set maxConnections and maxConcurrency to match your expected load.
Streaming Claude’s token stream with a simple SSE parser keeps memory usage low and allows you to forward partial results to downstream services.
Watch out for ECONNRESET errors and pool exhaustion; a quick retry and sensible limits prevent silent timeouts.
If you ever need to send large payloads (e.g., embeddings) or multipart data, Undici’s pipeline lets you write and read simultaneously without buffering the whole thing.

The Takeaway

A lightweight HTTP client removes the extra layers that add milliseconds to each Claude call.
Undici’s built‑in connection pool, when shared across Lambda warm invocations, saves time on TCP handshakes.
Setting maxHeaderSize prevents hidden failures when Anthropic adds extra headers.
Properly handling ECONNRESET and configuring maxConnections avoids socket starvation under high concurrency.
Streaming the response with a tiny SSE parser lets your function stay memory‑efficient and return tokens as they arrive.
pipeline is a handy pattern for future scenarios that involve large request bodies or multipart uploads.

Transparency notice

This article was written with the help of an AI system — Groq (GPT OSS 120B).

Published: 2026-10-28 · Primary focus: Undici

All code blocks are intended to be correct and runnable, but please verify them
against the official docs for the tools mentioned before using in production.

Find an error? Drop a comment — corrections are always welcome.

Secure AI Tool Calls in Node.js 22: Using Permission Flags to Sandbox Claude Agents

Dinesh_gowtham — Tue, 27 Oct 2026 08:14:03 +0000

AI agents can execute arbitrary code when they call tools – a hidden security nightmare. Node.js 22 finally gives you native permission flags to lock down filesystem and network access without containers. Learn how to harness these flags in a Lambda function to keep Claude’s tool calls safe.

Why AI Tool Calls Need Sandboxing

When a language model (LLM) like Anthropic’s Claude is asked to “fetch the latest CSV from S3 and summarize it,” it often replies with a tool call – a piece of code the model thinks should run to satisfy the request. A tool call is simply a request for the surrounding application to execute some operation, such as reading a file, making an HTTP request, or running a shell command.

If you hand that code to Node.js unchecked, the model could ask the runtime to delete files, open a reverse shell, or exfiltrate data. Think of it like giving a guest the keys to your house without asking which rooms they’re allowed to enter.

In plain English: A sandbox is a fenced yard around the code, so the model can only play within the fence you draw.

Without a sandbox, a compromised or mischievous model can turn your server into an attack vector. The traditional answer is to spin up a Docker container or a separate VM for every request – heavy, expensive, and often overkill for a quick Lambda invocation.

Node.js 22 introduced a permission model that works like a built‑in fence: you tell the engine which resources (network, file system, environment variables, etc.) a particular piece of code may touch. The model is enforced at runtime, and if the code tries to go beyond the granted permissions, Node throws an error and aborts the operation.

Because the permission flags only affect child processes (processes spawned from the main Node process) and worker threads (lightweight parallel execution contexts), we can keep the Lambda handler itself lightweight and attach strict fences only to the part that runs the model‑generated code.

Node.js 22 Permission Model Overview

Core concepts

Permission flag – a command‑line option that tells Node what it may do. For example, --allow-fs-read lets the process read files, but not write them.
Child process – a separate OS process launched from the main Node program, usually via child_process.spawn or exec. It inherits its own set of permissions.
Worker thread – a Node thread that runs JavaScript in parallel; it also receives its own permission set.

Node’s flag syntax looks like:

node --allow-fs-read --allow-net my-app.js

When you launch a child process with these flags, the child inherits only the permissions you specify. The parent (your Lambda handler) can stay unrestricted or have a different set.

Tip: Think of the parent process as the house manager and the child process as a hired contractor. The manager decides which tools the contractor can use.

How the flags are parsed

Node checks each flag at startup. If a flag is unknown, the process aborts with an error. In AWS Lambda, the runtime ignores unknown flags unless you explicitly enable the nodejs-runtime-flags setting in the function configuration. This is a common source of silent failures: the child process runs without the intended fence, and the model’s tool call either succeeds when it shouldn’t or fails without a clear message.

The minimal permission set for Claude tool calls

A typical Claude‑driven tool call needs:

Read‑only file system – to load a CSV or JSON file that the model wants to inspect.
Outbound network – to fetch a remote API or download a file.

That translates to the flags:

--allow-fs-read   # can read files, cannot write or delete
--allow-net       # can make outbound HTTP/HTTPS requests

No write access (--allow-fs-write) and no access to environment variables (--allow-env) keep the sandbox tight.

Setting Up a Secure Lambda Handler for Claude

Below is a minimal Lambda handler that receives a prompt, calls Claude via the Bedrock SDK, and then hands the model‑generated tool code to a child process that runs under the restricted flags.

Key takeaway: The Lambda runtime itself runs without the flags; only the child process that executes the tool code is fenced.

// handler.ts
import { BedrockClient, InvokeModelCommand } from "@aws-sdk/client-bedrock";
import { LambdaClient, InvokeCommand } from "@aws-sdk/client-lambda";
import { spawn } from "child_process";
import { Buffer } from "buffer";

// ---------- 1️⃣  Prepare the Bedrock client (no special flags needed) ----------
const bedrock = new BedrockClient({ region: "us-east-1" });

// ---------- 2️⃣  Helper to invoke Claude and extract the tool call ----------
async function askClaude(prompt: string): Promise<string> {
  const command = new InvokeModelCommand({
    modelId: "anthropic.claude-v2", // example model identifier
    contentType: "application/json",
    accept: "application/json",
    body: JSON.stringify({
      prompt,
      max_tokens: 1024,
      temperature: 0.2,
    }),
  });

  const response = await bedrock.send(command);
  // Bedrock returns a Uint8Array; turn it into a string
  const payload = Buffer.from(response.body as Uint8Array).toString("utf-8");
  const parsed = JSON.parse(payload);
  // Claude returns tool calls inside `parsed.output.text`
  // For this demo we assume the model returns raw JavaScript code
  return parsed.output?.text?.trim() ?? "";
}

// ---------- 3️⃣  Lambda handler ----------
export const handler = async (event: any) => {
  // The incoming event is expected to have a `prompt` field
  const userPrompt = event.prompt ?? "Summarize the attached CSV";

  // Ask Claude for a tool call
  const toolCode = await askClaude(userPrompt);

  // If Claude didn't suggest any code, just return a friendly message
  if (!toolCode) {
    return {
      statusCode: 200,
      body: JSON.stringify({ message: "Claude didn't need a tool call." }),
    };
  }

  // ---------- 4️⃣  Run the tool code in a sandboxed child process ----------
  const result = await runInSandbox(toolCode);

  // Return the tool's output to the API caller
  return {
    statusCode: 200,
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({ result }),
  };
};

// ---------- 5️⃣  Function that spawns a child with strict flags ----------
function runInSandbox(code: string): Promise<string> {
  return new Promise((resolve, reject) => {
    // Write the code to a temporary file (read‑only for the child)
    // In a real app you would use a secure temp directory and clean up after.
    const fs = require("fs");
    const path = require("path");
    const tmpFile = path.join("/tmp", `tool-${Date.now()}.js`);
    fs.writeFileSync(tmpFile, code, { mode: 0o400 }); // read‑only permission

    // Spawn a new Node process with ONLY read and network permissions
    const child = spawn(
      "node",
      [
        "--experimental-permission", // enables permission flags
        "--allow-fs-read",
        "--allow-net",
        tmpFile, // script to execute
      ],
      {
        stdio: ["ignore", "pipe", "pipe"], // ignore stdin, capture stdout & stderr
      }
    );

    // Collect stdout
    let stdout = "";
    child.stdout.on("data", (data) => (stdout += data));

    // Collect stderr for debugging
    let stderr = "";
    child.stderr.on("data", (data) => (stderr += data));

    child.on("close", (code) => {
      // Clean up the temporary file
      fs.unlinkSync(tmpFile);
      if (code === 0) {
        resolve(stdout.trim());
      } else {
        // Permission errors surface here; surface them to the caller for visibility
        reject(
          new Error(
            `Sandboxed tool failed (exit ${code}). Stderr: ${stderr.trim()}`
          )
        );
      }
    });
  });
}

What the code does, line by line

Import the Bedrock client (@aws-sdk/client-bedrock) – this talks to Claude.
askClaude builds the request payload, sends it, and extracts the raw JavaScript Claude suggests.
The Lambda handler receives a prompt, asks Claude, and decides whether there is any tool code to run.
runInSandbox writes the code to a temporary file and spawns a child process with only --allow-fs-read and --allow-net.
The child’s stdout is captured and returned; any permission violation appears in stderr and causes the promise to reject.

Helpful tip: The --experimental-permission flag is required in Node 22 to enable the permission system. Without it, the other --allow‑* flags are ignored.

Lambda‑specific gotchas

Runtime flags must be enabled – In the Lambda console, set Node.js runtime flags to --experimental-permission. Otherwise the flags are silently dropped, and the child process runs with full privileges.
require(esm) breakage – If you have a layer that forces ECMAScript module loading (require('esm')), Node 22 may throw an error that disappears in logs. Keep your Lambda layers up‑to‑date or avoid the esm shim.
SnapStart + VPC – Enabling SnapStart while the function is in a VPC does not speed up the VPC attachment phase; the cold start time remains unchanged. Plan capacity accordingly.
Response streaming – When you later stream large tool results, set Content-Type: application/octet-stream (or the appropriate MIME type) or Lambda will buffer the entire payload before sending it.

Running Claude Tool Calls with Limited Permissions

The heart of the sandbox is the spawn call shown earlier. Let’s break down why each option matters.

const child = spawn(
  "node",
  [
    "--experimental-permission", // turn on the permission system
    "--allow-fs-read",           // allow read‑only file access
    "--allow-net",               // allow outbound HTTP/S
    tmpFile,                     // the script that contains Claude’s code
  ],
  {
    stdio: ["ignore", "pipe", "pipe"], // no stdin, capture output streams
  }
);

--experimental-permission – tells Node to enforce any following --allow‑* flags. If you omit this, the subsequent flags are parsed but ignored, leaving the child with full access.
--allow-fs-read – grants read access to any file on the filesystem. Because we wrote the script to /tmp with read‑only mode, the child cannot modify it.
--allow-net – lets the child make HTTP/HTTPS requests. If the tool tries to open a listening socket (e.g., net.createServer), Node will throw a permission error because inbound connections are not covered by this flag.

In plain English: The child process is like a kid in a playground who can only pick up toys (read files) and shout to the outside (make network requests). They can’t break the swings (write files) or open the gate (listen for incoming connections).

Adding a write‑only exception (optional)

If you have a use‑case where the model must write a temporary file (e.g., a CSV transformation), you can tighten the scope by creating a dedicated directory with limited permissions and granting --allow-fs-write only to that path:

mkdir -p /tmp/sandbox-write
chmod 700 /tmp/sandbox-write

Then launch the child with:

spawn("node", [
  "--experimental-permission",
  "--allow-fs-read",
  "--allow-fs-write=/tmp/sandbox-write", // write only inside this folder
  "--allow-net",
  tmpFile,
]);

Node will reject any attempt to write outside /tmp/sandbox-write, protecting the rest of the filesystem.

Testing and Observing Permission Errors

A sandbox is only as good as your tests. Below is a simple test harness you can run locally (Node 22 required) that simulates a malicious tool call.

// test-sandbox.ts
import { spawn } from "child_process";
import * as fs from "fs";
import * as path from "path";

// 1️⃣  Malicious script that tries to delete a file and open a server
const maliciousCode = `
  const fs = require('fs');
  const net = require('net');

  // Attempt to delete the Lambda environment file (should fail)
  try {
    fs.unlinkSync('/etc/passwd');
    console.log('Deleted /etc/passwd');
  } catch (e) {
    console.error('Delete failed:', e.message);
  }

  // Attempt to start a TCP server (should fail)
  const server = net.createServer(() => {});
  server.listen(9999, () => console.log('Listening on 9999'));
`;

// 2️⃣  Write the code to a temporary file
const scriptPath = path.join(__dirname, "malicious.js");
fs.writeFileSync(scriptPath, maliciousCode, { mode: 0o400 });

// 3️⃣  Spawn a sandboxed child
const child = spawn(
  "node",
  [
    "--experimental-permission",
    "--allow-fs-read",   // note: no write permission
    "--allow-net",       // network allowed, but not listening
    scriptPath,
  ],
  { stdio: ["ignore", "pipe", "pipe"] }
);

let out = "";
let err = "";
child.stdout.on("data", (d) => (out += d));
child.stderr.on("data", (d) => (err += d));

child.on("close", (code) => {
  console.log("=== STDOUT ===");
  console.log(out);
  console.log("=== STDERR ===");
  console.log(err);
  console.log(`Child exited with code ${code}`);
  // Clean up
  fs.unlinkSync(scriptPath);
});

When you run node test-sandbox.ts, you should see errors like:

Delete failed: EPERM: operation not permitted, unlink '/etc/passwd'
Error: listen EACCES: permission denied 0.0.0.0:9999
    at ...

The exit code will be non‑zero, indicating the sandbox blocked the dangerous actions.

Key takeaway: Permission errors surface on stderr and cause the child to exit with a non‑zero status, which you can translate into an informative Lambda response.

Observing Lambda behavior

Deploy the handler to AWS Lambda with the following settings:

Setting	Value
Runtime	Node.js 22.x
Handler	`handler.handler`
Environment variable `NODE_OPTIONS`	`--experimental-permission`
Node.js runtime flags (in the console)	`--experimental-permission`
Memory	256 MiB (enough for a small child process)
Timeout	15 seconds (adjust as needed)

If you forget to add the runtime flags field, the child will run with full access and you’ll see the tool code succeed even when it should be blocked. The Lambda logs (/aws/lambda/...) will show a message like Permission flags ignored – child process had unrestricted rights.

The Takeaway

In plain English: Node.js 22 gives you a light‑weight, built‑in fence that you can attach to any piece of code you run inside a Lambda. You don’t need a full container sandbox to keep AI‑generated tool calls safe.

Permission flags (--allow-fs-read, --allow-net, etc.) are enforced only in child processes or worker threads; the Lambda handler itself stays unrestricted unless you also launch it with flags.
Enable the nodejs-runtime-flags setting in the Lambda console; otherwise the flags are dropped silently and your sandbox is ineffective.
Use child_process.spawn to execute the model‑generated script, passing the minimal set of flags that match the tool’s needs.
Test aggressively with intentionally malicious scripts; permission errors appear on stderr and cause a non‑zero exit code that you can surface to the API caller.
Keep ancillary Lambda gotchas in mind: avoid outdated esm layers, understand SnapStart + VPC limits, set correct Content-Type for streaming responses, and watch provisioned concurrency costs.

By applying these steps, you can let Claude—or any LLM—run helpful tool calls while keeping the rest of your AWS environment locked down. Happy sandboxing!

Transparency notice

This article was written with the help of an AI system — Groq (GPT OSS 120B).

Published: 2026-10-27 · Primary focus: NodeJS22

All code blocks are intended to be correct and runnable, but please verify them
against the Node.js docs before using in production.

Find an error? Drop a comment — corrections are always welcome.

How to Stream Claude Agent Events with Kinesis Data Streams, Explained Simply

Dinesh_gowtham — Fri, 23 Oct 2026 07:51:05 +0000

Real‑time LLM interactions feel like they belong on websockets, but a Kinesis Data Stream gives you durable, auto‑scaling fan‑out with millisecond latency. In this guide you’ll see how to pipe Claude’s token stream through Kinesis and a Node.js 22 Lambda, turning a flaky WebSocket demo into an enterprise‑grade pipeline.

Why Kinesis Beats WebSockets for LLM Streams

When you ask Claude for a response, the model sends back tokens (tiny pieces of text) as they are generated. A WebSocket connection can push each token straight to the browser, but it has three hidden costs:

Ephemeral connection – if the client disconnects, the in‑flight tokens are lost.
Single consumer – only the WebSocket server receives the data; you can’t add a logger, analytics, or a backup consumer without duplicating work.
Scaling limits – a single server must handle every concurrent chat, which becomes fragile under load.

A Kinesis Data Stream is a managed pipe that breaks the data into shards (independent write/read lanes). Each shard can feed many consumers at the same time (this is called enhanced fan‑out). Think of a water main (the stream) that splits into many smaller hoses (the consumers). Even if one hose is turned off, the water still flows through the main pipe.

In plain English – Kinesis lets you treat each token as a durable record that any number of services can read, while still keeping latency low enough for a chat‑like experience.

Quick code sketch: putting a token into Kinesis

// npm install @aws-sdk/client-kinesis
import { KinesisClient, PutRecordCommand } from "@aws-sdk/client-kinesis";

const client = new KinesisClient({ region: "us-east-1" });

async function putToken(streamName, partitionKey, token) {
  // The token is a short string, we store it as UTF‑8 bytes.
  const command = new PutRecordCommand({
    StreamName: streamName,
    PartitionKey: partitionKey, // decides which shard receives the record
    Data: Buffer.from(token),   // binary payload
  });

  await client.send(command);
}

Key takeaway – With a few lines you can write a token to a durable, auto‑scaled pipe that any Lambda can later read.

Setting Up a Kinesis Data Stream for Claude Tokens

What you need to know first

Shard – a unit of capacity; each shard can ingest up to 1 MiB / second (≈ 1000 records) and serve up to 2 MiB / second to consumers.
Enhanced fan‑out – a feature that gives each consumer its own read throughput, avoiding the 2 MiB shared limit. It only works when the consumer is registered by ARN (Amazon Resource Name). Forgetting this registration falls back to the shared 2 MiB pool and adds ~200 ms latency.
Iterator – a pointer that tells Kinesis where to start reading. It expires after 5 minutes, so a consumer must refresh it regularly.

Step‑by‑step creation with the SDK

import {
  KinesisClient,
  CreateStreamCommand,
  RegisterStreamConsumerCommand,
} from "@aws-sdk/client-kinesis";

const kinesis = new KinesisClient({ region: "us-east-1" });
const STREAM_NAME = "claude-token-stream";
const CONSUMER_NAME = "claude-lambda-consumer";

async function createStream() {
  // Create a stream with 2 shards (adjust based on expected throughput)
  await kinesis.send(
    new CreateStreamCommand({ StreamName: STREAM_NAME, ShardCount: 2 })
  );
  console.log(`Stream ${STREAM_NAME} created`);
}

async function registerConsumer() {
  // Registering by ARN enables enhanced fan‑out
  await kinesis.send(
    new RegisterStreamConsumerCommand({
      StreamARN: `arn:aws:kinesis:us-east-1:123456789012:stream/${STREAM_NAME}`,
      ConsumerName: CONSUMER_NAME,
    })
  );
  console.log(`Consumer ${CONSUMER_NAME} registered for enhanced fan‑out`);
}

// Run once (or guard with existence checks)
await createStream();
await registerConsumer();

Tip – After creating the stream, give it a minute to become ACTIVE before you register the consumer; otherwise the registration request will fail.

Gotchas to keep in mind

Issue	Why it hurts	How to avoid
Shard write limit (1 MiB/s)	Burst of tokens can exceed capacity and cause throttling	Start with 2‑4 shards; monitor `WriteProvisionedThroughputExceeded` metrics
Iterator expiration (5 min)	A stale iterator makes `GetRecords` return `ExpiredIteratorException`	Refresh iterator on every loop iteration or catch the error and request a new one
Empty `GetRecords` response	Kinesis is eventually consistent; records may appear a few milliseconds later	Loop with a short sleep (e.g., 200 ms) when no records are returned

Lambda Consumer with Node.js 22: Decoding and Forwarding

Why a Lambda is a good fit

A Lambda is a serverless function that runs only when invoked. By attaching it to the Kinesis stream as an event source, AWS automatically polls the stream, passes batches of records to your code, and handles scaling for you. Node.js 22 brings the native fetch API, so we can forward each token to a browser using Server‑Sent Events (SSE) without adding extra libraries.

Minimal Lambda handler

// npm install @aws-sdk/client-lambda @aws-sdk/client-kinesis
import { KinesisClient, GetRecordsCommand, GetShardIteratorCommand } from "@aws-sdk/client-kinesis";

const kinesis = new KinesisClient({ region: "us-east-1" });
const STREAM_NAME = process.env.STREAM_NAME;
const CONSUMER_ARN = process.env.CONSUMER_ARN; // ARN of the enhanced fan‑out consumer

// Exponential backoff helper
function wait(ms) { return new Promise(res => setTimeout(res, ms)); }

export const handler = async (event, context) => {
  // 1️⃣ Get a shard iterator for each shard (enhanced fan‑out)
  const shards = await kinesis.send(
    new DescribeStreamCommand({ StreamName: STREAM_NAME })
  ).then(resp => resp.StreamDescription.Shards);

  for (const shard of shards) {
    const iteratorCmd = new GetShardIteratorCommand({
      StreamName: STREAM_NAME,
      ShardId: shard.ShardId,
      ShardIteratorType: "LATEST", // start at newest token
      ConsumerARN: CONSUMER_ARN,   // enables enhanced fan‑out
    });
    let iterator = await kinesis.send(iteratorCmd).then(r => r.ShardIterator);

    // 2️⃣ Pull records in a loop (Lambda will be terminated after timeout)
    while (true) {
      try {
        const recordsCmd = new GetRecordsCommand({ ShardIterator: iterator, Limit: 100 });
        const { Records, NextShardIterator } = await kinesis.send(recordsCmd);
        iterator = NextShardIterator;

        if (Records.length === 0) {
          // No data right now, pause briefly to avoid tight looping
          await wait(200);
          continue;
        }

        // 3️⃣ Forward each token via SSE to the client
        for (const rec of Records) {
          const token = Buffer.from(rec.Data).toString("utf-8");
          await forwardTokenSSE(token);
        }
      } catch (err) {
        if (err.name === "ProvisionedThroughputExceededException") {
          // 4️⃣ Throttling – back off exponentially
          for (let attempt = 1; attempt <= 5; attempt++) {
            const backoff = Math.pow(2, attempt) * 100; // 200ms, 400ms, …
            await wait(backoff);
            // retry loop will continue automatically
          }
        } else if (err.name === "ExpiredIteratorException") {
          // Refresh iterator as it timed out
          const fresh = await kinesis.send(iteratorCmd);
          iterator = fresh.ShardIterator;
        } else {
          console.error("Unexpected error:", err);
          throw err; // let Lambda surface the failure
        }
      }
    }
  }
};

// Helper that posts token to a client using SSE
async function forwardTokenSSE(token) {
  // The client URL is passed via env var; it must point to an endpoint that keeps the HTTP connection open
  const clientUrl = process.env.CLIENT_SSE_ENDPOINT;
  await fetch(clientUrl, {
    method: "POST",
    headers: {
      "Content-Type": "text/event-stream", // tells the browser to treat it as SSE
    },
    body: `data: ${token}\n\n`,
  });
}

Explanation of key lines

ShardIteratorType: "LATEST" starts reading from the newest token, ideal for live chat.
ConsumerARN registers the consumer for enhanced fan‑out; without it you’d see the 200 ms latency penalty.
The while (true) loop keeps the Lambda busy until AWS times it out (default 15 min). In production you’d configure a timeout that matches your latency budget.
ProvisionedThroughputExceededException signals that the read rate is too high; the exponential backoff reduces pressure without crashing.

Tip – Node.js 22’s native fetch automatically follows redirects, but SSE requires the exact Content-Type. Forgetting the header makes browsers treat the payload as plain JSON and the stream appears broken.

Putting It Together: Real‑Time Claude Chat Proxy

End‑to‑end flow (text diagram)

Browser opens an SSE connection to /chat/events.
API Gateway routes the request to a thin Lambda (proxy‑handler).
proxy‑handler calls Claude’s completion endpoint with stream=true.
Claude streams token chunks back to the Lambda; each chunk is written to the Kinesis stream using PutRecord.
The consumer Lambda (code from the previous section) reads the tokens from Kinesis and forwards them to the browser’s SSE endpoint.
The browser renders tokens as they arrive, giving a live‑typing feel.

Minimal proxy Lambda that talks to Claude and pushes into Kinesis

import { KinesisClient, PutRecordCommand } from "@aws-sdk/client-kinesis";

const kinesis = new KinesisClient({ region: "us-east-1" });
const CLAUDE_API = "https://api.anthropic.com/v1/complete";
const STREAM_NAME = process.env.STREAM_NAME;

export const handler = async (event) => {
  const body = JSON.parse(event.body);
  const { prompt, userId } = body; // simple payload from the browser

  // Call Claude with streaming enabled
  const response = await fetch(CLAUDE_API, {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      "x-api-key": process.env.CLAUDE_API_KEY,
    },
    body: JSON.stringify({
      model: "claude-3-5-sonnet-20241001",
      prompt,
      max_tokens_to_sample: 1024,
      stream: true,
    }),
  });

  // Read the raw byte stream; each line is a JSON fragment with a token
  const reader = response.body.getReader();
  const decoder = new TextDecoder();

  while (true) {
    const { done, value } = await reader.read();
    if (done) break;

    // Split on newline because Claude sends each token on its own line
    const chunk = decoder.decode(value);
    const lines = chunk.split("\n").filter(l => l.trim());

    for (const line of lines) {
      const payload = JSON.parse(line);
      const token = payload.completion; // token string

      // Write token to Kinesis – partition by userId to keep order per chat
      await kinesis.send(
        new PutRecordCommand({
          StreamName: STREAM_NAME,
          PartitionKey: userId,
          Data: Buffer.from(token),
        })
      );
    }
  }

  // Respond quickly; the real‑time work continues in the consumer Lambda
  return {
    statusCode: 200,
    body: JSON.stringify({ message: "Chat started" }),
  };
};

In plain English – This Lambda acts like a mailroom clerk: it receives Claude’s letters (tokens), stamps each with the user’s ID (the partition key), and drops them into the Kinesis mailbox for the delivery Lambda to pick up.

Common pitfalls for this proxy

Missing stream:true – Claude will wait until the whole answer is ready, defeating the real‑time goal.
Incorrect partition key – If you use a constant key, all chats share a single shard, quickly hitting the 1 MiB/s write ceiling.
Forgot to set CONTENT_TYPE: text/event-stream on the SSE endpoint – browsers will buffer the response, appearing as if nothing is happening.

Handling Errors and Observability

Why observability matters

A streaming pipeline has many moving parts: Claude API latency, Kinesis throttling, Lambda timeouts, and network hiccups. Without visibility you’ll see occasional “missing tokens” but not know why.

Simple CloudWatch logging and metrics

import { CloudWatchClient, PutMetricDataCommand } from "@aws-sdk/client-cloudwatch";

const cw = new CloudWatchClient({ region: "us-east-1" });
const METRIC_NAMESPACE = "ClaudeKinesisPipeline";

async function recordMetric(name, value = 1) {
  await cw.send(
    new PutMetricDataCommand({
      Namespace: METRIC_NAMESPACE,
      MetricData: [
        {
          MetricName: name,
          Value: value,
          Unit: "Count",
        },
      ],
    })
  );
}

Insert recordMetric("TokensProcessed") after each successful token forward, and recordMetric("ThrottleBackoff") inside the backoff catch block. CloudWatch dashboards can then show spikes.

Dealing with iterator expiration

if (err.name === "ExpiredIteratorException") {
  // Log for visibility
  console.warn("Shard iterator expired, requesting a fresh one");
  // Re‑fetch the iterator (same code as initial request)
  const fresh = await kinesis.send(iteratorCmd);
  iterator = fresh.ShardIterator;
}

Dead‑letter queue (DLQ) for irrecoverable records

If a record repeatedly triggers ProvisionedThroughputExceededException after several retries, move it to an SQS queue for later analysis. This prevents the Lambda from getting stuck on a single bad shard.

Key tip – Always set a maximum retry count (e.g., 5) before sending to DLQ; otherwise exponential backoff can keep the function running until it times out, inflating costs.

The Takeaway

You now have a complete, production‑grade pattern for streaming Claude’s token output through Kinesis and a Node.js 22 Lambda.

Kinesis provides durability and multi‑consumer fan‑out, eliminating the fragility of a single WebSocket server.
Enhanced fan‑out only works when you register the consumer by ARN; forgetting this leaves you with higher latency and extra read costs.
Shards are the unit of capacity; start with multiple shards and monitor throughput to avoid throttling.
Lambda can read directly from Kinesis, decode token bytes, and forward them with SSE using the built‑in fetch API.
Handle ProvisionedThroughputExceededException with exponential backoff and refresh stale shard iterators to keep the pipeline alive.
Add lightweight CloudWatch metrics and a DLQ to gain visibility and prevent silent data loss.

With these pieces in place, you can replace a brittle WebSocket demo with a scalable, server‑less architecture that works the same way for a handful of users or for thousands of concurrent chats. Happy streaming!

Transparency notice

This article was written with the help of an AI system — Groq (GPT OSS 120B).

Published: 2026-10-23 · Primary focus: Kinesis

All code blocks are intended to be correct and runnable, but please verify them
against the official docs for the tools mentioned before using in production.

Find an error? Drop a comment — corrections are always welcome.

Step Functions + Claude Tool Calls: Orchestrate AI Agents Without Lambda, Explained Simply

Dinesh_gowtham — Thu, 22 Oct 2026 07:52:05 +0000

Most developers reach for a Lambda function every time they need an LLM response, adding unnecessary complexity and latency. Step Functions can call Claude directly via the Bedrock SDK integration, turning a multi‑step AI workflow into a single, visual state machine. In this guide you’ll see how to build, run, and debug that workflow from Node.js 22.

Why Step Functions Makes AI Orchestration Simple

When you ask an LLM (large language model) to do something, the request is just the first step. Real‑world agents often need to

split a big problem into smaller prompts,
call the model many times in parallel,
combine the answers, and
store the result somewhere (for example, S3).

A naïve solution is to write a Lambda function that does all of this work. Each Lambda invocation adds a cold‑start delay, a separate IAM role, and a piece of code that you have to maintain.

Step Functions is a managed service that runs a state machine: a diagram of steps (states) that pass data from one to the next. The service itself takes care of retries, parallelism, and visual debugging. Because Step Functions can talk directly to other AWS services (called service integration), you can ask Claude (the Anthropic model hosted on Bedrock) to answer a prompt without ever writing a Lambda.

In plain English: Think of Step Functions as a conductor that tells each musician (service) when to play, instead of a soloist (Lambda) trying to play the whole orchestra alone.

Minimal “Hello Claude” state machine

Below is the tiniest possible state machine that sends a single prompt to Claude and returns the answer. The definition is written in Amazon States Language (ASL), a JSON dialect that describes each state.

{
  "Comment": "A one‑step workflow that calls Claude via Bedrock.",
  "StartAt": "AskClaude",
  "States": {
    "AskClaude": {
      "Type": "Task",
      "Resource": "arn:aws:states:::aws-sdk:bedrockruntime:invokeModel",
      "Parameters": {
        "ModelId": "anthropic.claude-v2",
        "ContentType": "application/json",
        "Accept": "application/json",
        "Body.$": "$.prompt"
      },
      "End": true
    }
  }
}

Task – a step that performs work.
Resource – the ARN that tells Step Functions to call the InvokeModel operation of the Bedrock SDK.
$.prompt – a JSONPath that pulls the prompt field from the state input.

When you start this execution with {"prompt":"{\"prompt\":\"Hello, Claude!\"}"}, the state machine hands the JSON to Bedrock, waits for the response, and finishes with the model’s output. No Lambda involved.

Setting Up a Claude InvokeModel Integration in the State Machine

Before you can run the workflow you need three things:

An AWS IAM role that lets Step Functions call bedrock-runtime:InvokeModel and write to S3.
A Bedrock model ID – for this article we use anthropic.claude-v2.
A state machine definition that includes the task.

IAM role (short version)

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "bedrock-runtime:InvokeModel",
        "s3:PutObject"
      ],
      "Resource": "*"
    }
  ]
}

Attach the policy to a role named StepFunctionsClaudeRole and give the role to the state machine when you create it.

Full state machine with a final S3 write

The following definition does three things:

Accepts a list of prompts.
Calls Claude for each prompt in parallel (via a Distributed Map – covered later).
Writes the aggregated summary to an S3 bucket.

{
  "Comment": "Parallel Claude calls with result stored in S3.",
  "StartAt": "ProcessPrompts",
  "TimeoutSeconds": 300,
  "Roles": "arn:aws:iam::123456789012:role/StepFunctionsClaudeRole",
  "States": {
    "ProcessPrompts": {
      "Type": "Map",
      "ItemsPath": "$.prompts",
      "MaxConcurrency": 5,
      "Iterator": {
        "StartAt": "AskClaude",
        "States": {
          "AskClaude": {
            "Type": "Task",
            "Resource": "arn:aws:states:::aws-sdk:bedrockruntime:invokeModel",
            "Parameters": {
              "ModelId": "anthropic.claude-v2",
              "ContentType": "application/json",
              "Accept": "application/json",
              "Body.$": "$"
            },
            "ResultPath": "$.response",
            "End": true
          }
        }
      },
      "ResultPath": "$.responses",
      "Next": "SummarizeAndStore"
    },
    "SummarizeAndStore": {
      "Type": "Task",
      "Resource": "arn:aws:states:::aws-sdk:s3:putObject",
      "Parameters": {
        "Bucket": "my-claude-output",
        "Key": "summary.json",
        "Body.$": "States.JsonToString($.responses)"
      },
      "End": true
    }
  }
}

Map – runs the same sub‑workflow for each element of an array.
MaxConcurrency – caps how many child executions can run at once (the service limit is 10 000).
ResultPath – tells Step Functions where to store the output of the child.

Tip: The States.JsonToString intrinsic function converts a JSON object to a string that S3 expects as the body.

Why the service integration matters

When you use the arn:aws:states:::aws-sdk:… prefix, Step Functions becomes a thin client for the underlying SDK. No extra code, no packaging, just declarative wiring. This reduces deployment friction and keeps the latency low because the call happens inside the AWS control plane.

Defining the Distributed Map for Parallel Prompt Processing

Imagine you have 100 customer tickets that each need a short summary from Claude. Processing them one after another would take minutes. A Distributed Map works like a kitchen line where multiple chefs (tasks) prepare dishes (responses) simultaneously.

Anatomy of a Map state

ItemsPath – points to the array in the input that you want to iterate over.
Iterator – the mini‑state machine that runs for each element.
ResultPath – where the collection of results is placed in the parent state.

Example: 3‑prompt parallel run

// This file is only for illustration; the JSON above is the real definition.
import { SFNClient, CreateStateMachineCommand } from "@aws-sdk/client-sfn";

const client = new SFNClient({ region: "us-east-1" });

const definition = {
  // (copy the JSON from the previous section)
};

await client.send(
  new CreateStateMachineCommand({
    name: "ClaudeParallelDemo",
    definition: JSON.stringify(definition),
    roleArn: "arn:aws:iam::123456789012:role/StepFunctionsClaudeRole",
    type: "STANDARD" // STANDARD gives us the 25,000‑event history limit
  })
);

The CreateStateMachineCommand registers the workflow with Step Functions.
The type: "STANDARD" tells the service to keep a full execution history (useful for debugging, but remember the 25 000‑event ceiling).

When you start this machine with:

{
  "prompts": [
    "{\"prompt\":\"Summarize the first ticket.\"}",
    "{\"prompt\":\"Summarize the second ticket.\"}",
    "{\"prompt\":\"Summarize the third ticket.\"}"
  ]
}

Step Functions spins up three parallel AskClaude tasks, each receiving its own prompt JSON. The results are collected under $.responses and later written to S3.

In plain English: A Map state is like a mail‑room sorter that hands each envelope to a separate clerk; the clerks work at the same time, and the sorter later gathers all the stamped envelopes back together.

Running the Workflow from Node.js with the AWS SDK

Now that the state machine exists, a developer needs a small script that triggers it and watches the outcome. The only SDK we need is @aws-sdk/client-sfn, the official JavaScript client for Step Functions.

Step‑by‑step script

// run-clair-workflow.ts
import {
  SFNClient,
  StartExecutionCommand,
  DescribeExecutionCommand,
  GetExecutionHistoryCommand
} from "@aws-sdk/client-sfn";

// 1️⃣ Create a client that talks to Step Functions in your region.
const sfn = new SFNClient({ region: "us-east-1" });

// 2️⃣ Build the input payload – an array of prompt objects.
const executionInput = {
  prompts: [
    JSON.stringify({ prompt: "Explain the difference between TCP and UDP." }),
    JSON.stringify({ prompt: "Give a short recipe for banana bread." }),
    JSON.stringify({ prompt: "Summarize the plot of *The Little Prince*." })
  ]
};

// 3️⃣ Start the execution. The ARN below is the ARN of the state machine you created earlier.
async function start() {
  const startCmd = new StartExecutionCommand({
    stateMachineArn: "arn:aws:states:us-east-1:123456789012:stateMachine:ClaudeParallelDemo",
    input: JSON.stringify(executionInput) // Step Functions expects a JSON string
  });

  const startResp = await sfn.send(startCmd);
  console.log("Execution started:", startResp.executionArn);
  return startResp.executionArn!;
}

// 4️⃣ Poll until the execution finishes (simple exponential back‑off).
async function waitForCompletion(executionArn: string) {
  const describeCmd = new DescribeExecutionCommand({ executionArn });
  while (true) {
    const { status, output } = await sfn.send(describeCmd);
    if (status === "SUCCEEDED") {
      console.log("✅ Finished!");
      console.log("Final output (S3 key is stored there):", output);
      break;
    } else if (status === "FAILED" || status === "TIMED_OUT" || status === "ABORTED") {
      console.error(`❌ Execution ended with status: ${status}`);
      break;
    }
    // Not finished yet – wait a bit.
    await new Promise(r => setTimeout(r, 2000));
  }
}

// 5️⃣ Run the helper functions.
(async () => {
  const arn = await start();
  await waitForCompletion(arn);
})();

Explanation of each line

Lines 1‑4 import the three commands we need: start, describe, and get history.
sfn is the client object that knows how to sign requests with your credentials.
executionInput mirrors the shape expected by the state machine (prompts array).
StartExecutionCommand tells Step Functions to begin a new run; the response contains an executionArn that uniquely identifies this run.
The while loop repeatedly asks for the execution status. When the status becomes SUCCEEDED, the loop stops.

Key takeaway: The Node.js code only talks to Step Functions; every heavy‑lifting (prompt → Claude → S3) happens inside the state machine itself.

Optional: Pulling the full execution history

If you hit the 25 000‑event limit, you can fetch the raw event log:

const historyCmd = new GetExecutionHistoryCommand({
  executionArn,
  maxResults: 1000, // keep under the service limit per call
  reverseOrder: false
});
const { events } = await sfn.send(historyCmd);
console.log("Number of events:", events?.length);

The history is useful for spotting where a child task failed or for measuring latency per step.

Debugging, Observability, and Payload Limits

Step Functions gives you a visual console, but a few hidden limits can trip new users.

1. Execution‑history ceiling

A standard state machine stores up to 25 000 events per execution. Each Map iteration, each retry, and each service‑integration call adds events. If you process thousands of prompts in a single execution, you may exceed the ceiling and the console will truncate the view.

Solution: Break a huge batch into smaller executions, or use an Express workflow (which streams events to CloudWatch Logs instead of storing them).

2. 256 KB input/output cap

Every state receives an input and emits an output that must fit into 256 KB. Claude can generate responses larger than this, especially when you ask for long essays. If a response exceeds the limit, the task fails with a PayloadTooLarge error.

Work‑around: After the AskClaude task, add a Pass state that copies the raw response to an S3 bucket, then replace the state output with a tiny reference (the S3 key). Example:

"StoreLargeOutput": {
  "Type": "Task",
  "Resource": "arn:aws:states:::aws-sdk:s3:putObject",
  "Parameters": {
    "Bucket": "my-claude-output",
    "Key.$": "States.Format('large/{}.json', $.response.id)",
    "Body.$": "$.response.body"
  },
  "ResultPath": "$.s3Location",
  "Next": "TrimOutput"
},
"TrimOutput": {
  "Type": "Pass",
  "Result": {
    "s3Key.$": "$.s3Location.Key"
  },
  "End": true
}

Now downstream states only see the S3 key (a few bytes), staying under the limit.

3. Cold start for Lambda tasks

If you ever mix a Lambda task into the workflow, remember that the Lambda cold start adds to the overall latency. By keeping everything as service integration tasks, you avoid that extra waiting time.

4. Distributed Map concurrency ceiling

A single Map can launch up to 10 000 child executions concurrently. If you set MaxConcurrency too high, the service throttles you. Start with a modest number (e.g., 5‑20) and monitor CloudWatch metrics (StepFunctions/MapStateConcurrency).

5. Observability tips

Enable Execution Logging to CloudWatch Logs; it gives you timestamps for each state transition.
Use the Step Functions console to click into a failed state and see the exact request/response payload.
Turn on X-Ray tracing for the service‑integration calls if you need end‑to‑end latency numbers.

Tip: When you see “Execution history truncated” in the console, it’s a signal that you’ve crossed the 25 000‑event line and should split the job.

The Takeaway

Step Functions can call Bedrock’s Claude directly; no Lambda code is required.
A Task state with the aws-sdk:bedrockruntime:invokeModel ARN acts as a thin client to the model.
The Map state provides built‑in parallelism, letting you process dozens or hundreds of prompts at once.
All orchestration logic lives in a JSON definition, so the Node.js side only starts the execution and polls for status.
Be aware of the 256 KB payload limit; large model outputs should be streamed to S3 or truncated.
Execution history, concurrency, and cold‑start limits are real constraints—design your workflows to stay comfortably inside them.

With these pieces in place, you can build AI agents that scale, stay observable, and avoid the overhead of managing Lambda functions. Happy orchestrating!

Transparency notice

This article was written with the help of an AI system — Groq (GPT OSS 120B).

Published: 2026-10-22 · Primary focus: StepFunctions

All code blocks are intended to be correct and runnable, but please verify them
against the official docs for the tools mentioned before using in production.

Find an error? Drop a comment — corrections are always welcome.

How to Run Claude‑Powered AI Code Review in CodeBuild: A Step‑By‑Step Guide

Dinesh_gowtham — Wed, 21 Oct 2026 07:55:58 +0000

CI pipelines catch syntax errors, but they still let subtle logic bugs slip through. Pair Claude with CodeBuild and let an AI read every pull request, suggest fixes, and even commit them automatically. The result? Code reviews that never sleep.

Why AI Code Review in CI Matters

When a developer pushes a pull request, the code sits in a queue waiting for a human reviewer. That waiting period can be minutes, hours, or even days. During that time, a simple typo or a misunderstood requirement can slip into production if the review is rushed.

In plain English: Running an AI reviewer inside your continuous‑integration (CI) system gives you a second set of eyes that never takes a coffee break.

The benefit of moving the assistant to CI

Consistency – the same prompt and model are used for every build, so the style of feedback doesn’t drift.
Zero manual steps – once the pipeline is configured, the AI runs automatically on every commit.
Early feedback – bugs are caught before they merge, reducing the cost of fixing them later.

Think of the CI stage as a security guard at the front door. The guard checks every visitor (code change) before they enter the building (main branch). Adding Claude is like giving the guard a pair of X‑ray glasses: it can see hidden issues that a regular guard would miss.

Setting Up a CodeBuild Project for Claude Calls

Before we can ask Claude to review code, we need a CodeBuild project that can talk to the Claude API. The tricky part is keeping the API key safe.

1️⃣ Store the Claude API key in Secrets Manager

Secrets Manager – an AWS service that stores sensitive strings (like passwords) and returns them on demand.
Why not put the key directly in the buildspec? – CodeBuild truncates environment variables larger than 4 KB, so a raw key longer than that gets cut off, breaking the request.

Create a secret (e.g., claude/api-key) in the console or via CLI:

aws secretsmanager create-secret \
  --name claude/api-key \
  --secret-string "$(cat ~/.claude_key)"

2️⃣ Reference the secret in the buildspec

The ${secretsmanager:ARN} syntax tells CodeBuild to fetch the secret at runtime and inject it as an environment variable. The value is masked in logs, preventing accidental exposure.

# buildspec.yml
version: 0.2

env:
  secrets-manager:
    CLAUDE_API_KEY: "arn:aws:secretsmanager:us-east-1:123456789012:secret:claude/api-key-abc123"

phases:
  install:
    runtime-versions:
      nodejs: 22
    commands:
      - echo "Installing dependencies..."
      - npm ci
  build:
    commands:
      - echo "Running AI code review..."
      - node review.js
artifacts:
  files:
    - diff-output.json

Tip: CodeBuild logs hide any variable that comes from Secrets Manager, but you still want to avoid printing the variable yourself.

3️⃣ Create the CodeBuild project with the AWS SDK

Below is a tiny script that creates (or updates) the project. It uses the @aws-sdk/client-codebuild package, which is the official JavaScript client for CodeBuild.

// create-project.ts
import { CodeBuildClient, CreateProjectCommand, UpdateProjectCommand } from "@aws-sdk/client-codebuild";

// Initialise the client – it reads credentials from the usual places (env, shared config, etc.)
const client = new CodeBuildClient({ region: "us-east-1" });

async function ensureProject() {
  const projectName = "ClaudeCodeReview";

  try {
    // Try to fetch the project; if it exists we will update it.
    const existing = await client.send(new CreateProjectCommand({ /* ... */ }));
    console.log(`Project ${projectName} already exists – skipping creation.`);
  } catch (err) {
    // If the project does not exist we fall back to creating it.
    if (err.name === "ResourceNotFoundException") {
      console.log(`Creating project ${projectName}...`);
      await client.send(new CreateProjectCommand({
        name: projectName,
        source: {
          type: "CODECOMMIT",
          location: "https://git-codecommit.us-east-1.amazonaws.com/v1/repos/my-repo"
        },
        environment: {
          type: "LINUX_CONTAINER",
          image: "aws/codebuild/standard:7.0", // Node 22 available here
          computeType: "BUILD_GENERAL1_SMALL",
          environmentVariables: [
            // Reference secret via the special syntax
            { name: "CLAUDE_API_KEY", value: "${secretsmanager:arn:aws:secretsmanager:us-east-1:123456789012:secret:claude/api-key-abc123}", type: "PLAINTEXT" }
          ]
        },
        // Ensure logs do not accidentally print env vars
        logsConfig: { cloudWatchLogs: { status: "ENABLED", groupName: "/aws/codebuild/ClaudeCodeReview" } }
      }));
      console.log(`Project ${projectName} created.`);
    } else {
      // Any other error we surface – this helps debugging mis‑configurations.
      console.error("Unexpected error:", err);
    }
  }
}

ensureProject().catch(console.error);

Key takeaway: Using Secrets Manager avoids the 4 KB truncation problem and keeps the key hidden from logs.

Writing the Node.js Agent: Prompt, Call, and Parse

The heart of the solution is a small Node script (review.js) that:

Reads the changed files from the pull request (we’ll assume they are already checked out by CodeBuild).
Sends a prompt to Claude’s /v1/complete endpoint.
Receives a JSON diff suggestion and stores it in S3.

1️⃣ Building the prompt

Claude works best when you give it a clear instruction and a concrete example. The prompt we send is:

You are a senior software engineer. Review the following TypeScript file and suggest a minimal diff that fixes any logic errors, missing edge‑case handling, or performance issues. Return ONLY a JSON object:
{
  "file": "path/to/file.ts",
  "diff": "unified diff string"
}

2️⃣ The agent code

// review.js
import { readFileSync, readdirSync } from "fs";
import { S3Client, PutObjectCommand } from "@aws-sdk/client-s3";
import { LambdaClient, InvokeCommand } from "@aws-sdk/client-lambda";

// Node 22 has a built‑in fetch implementation
const CLAUDE_ENDPOINT = "https://api.anthropic.com/v1/complete";
const MODEL = "claude-3-5-sonnet-20241022"; // example model name

// Helper: exponential backoff for retrying transient errors
async function retry(fn, attempts = 3, delay = 500) {
  for (let i = 0; i < attempts; i++) {
    try {
      return await fn();
    } catch (err) {
      if (i === attempts - 1) throw err; // give up on the last try
      const wait = delay * Math.pow(2, i);
      console.warn(`Retry ${i + 1}/${attempts} after ${wait}ms:`, err.message);
      await new Promise(r => setTimeout(r, wait));
    }
  }
}

// 1️⃣ Collect changed files (simple glob for demo)
function getChangedFiles() {
  // In a real pipeline you would use `git diff --name-only $BASE $HEAD`
  // Here we just read everything under src/
  return readdirSync("./src", { withFileTypes: true })
    .filter(e => e.isFile() && e.name.endsWith(".ts"))
    .map(e => `src/${e.name}`);
}

// 2️⃣ Build the request payload for Claude
function buildPayload(filePath, fileContent) {
  return {
    model: MODEL,
    max_tokens: 1024,
    temperature: 0,
    prompt: `You are a senior software engineer. Review the following TypeScript file and suggest a minimal diff that fixes any logic errors, missing edge‑case handling, or performance issues. Return ONLY a JSON object:\n\nFile: ${filePath}\n\`\`\`typescript\n${fileContent}\n\`\`\``,
  };
}

// 3️⃣ Call Claude with proper headers and error handling
async function callClaude(payload) {
  const response = await retry(() => fetch(CLAUDE_ENDPOINT, {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      "x-api-key": process.env.CLAUDE_API_KEY, // injected from Secrets Manager
    },
    body: JSON.stringify(payload),
  }));
  if (!response.ok) {
    const errBody = await response.text();
    throw new Error(`Claude returned ${response.status}: ${errBody}`);
  }
  const data = await response.json();
  // Claude returns the text inside a "completion" field
  return JSON.parse(data.completion);
}

// 4️⃣ Store the diff in S3 for later consumption
async function uploadDiff(diffObj) {
  const s3 = new S3Client({ region: "us-east-1" });
  const key = `review-results/${Date.now()}-${diffObj.file}.json`;
  await s3.send(new PutObjectCommand({
    Bucket: "my-ai-review-bucket",
    Key: key,
    Body: JSON.stringify(diffObj, null, 2),
    ContentType: "application/json",
  }));
  console.log(`✅ Diff uploaded to s3://${key}`);
}

// 5️⃣ Rotate the Claude API key via Lambda (security best practice)
async function rotateKey() {
  const lambda = new LambdaClient({ region: "us-east-1" });
  const cmd = new InvokeCommand({
    FunctionName: "RotateClaudeKey",
    InvocationType: "RequestResponse",
  });
  const result = await lambda.send(cmd);
  if (result.FunctionError) {
    throw new Error(`Lambda rotation failed: ${result.FunctionError}`);
  }
  console.log("🔄 Claude API key rotated successfully.");
}

// Main driver
(async () => {
  try {
    const files = getChangedFiles();
    for (const file of files) {
      const content = readFileSync(file, "utf-8");
      const payload = buildPayload(file, content);
      const diff = await callClaude(payload);
      await uploadDiff(diff);
    }

    // Optional: rotate the key after the build to limit exposure time
    await rotateKey();
  } catch (err) {
    // Distinguish network problems from CodeBuild specific errors
    if (err.name === "CodeBuildException") {
      console.error("❗ CodeBuild error:", err);
    } else {
      console.error("❗ Unexpected error:", err);
    }
    process.exit(1); // signal failure to CodeBuild
  }
})();

What each block does

retry – tries the fetch again if a transient network glitch occurs, waiting longer each time (exponential backoff).
getChangedFiles – a placeholder that pretends every file under src/ is changed; replace with a real git diff in production.
buildPayload – creates the JSON body Claude expects, embedding the file content inside a fenced code block.
callClaude – performs the HTTP request, checks the HTTP status, parses the completion field, and finally turns the text into a JavaScript object.
uploadDiff – writes the AI‑generated diff to an S3 bucket, making it easy for downstream steps (e.g., a bot that opens a PR comment).
rotateKey – demonstrates how to invoke a Lambda that rotates the secret; this keeps the key’s lifespan short.

Analogy: Think of retry as a polite knock on a door. If the person inside doesn’t answer the first time, you wait a bit longer and try again, rather than giving up immediately.

Integrating Claude’s Tool Use with Lambda for Secure Secrets

Keeping the Claude API key in the build environment is convenient but risky. A better pattern is to let a short‑lived Lambda fetch the secret, hand it to the build, and then immediately rotate it.

1️⃣ The Lambda function (`rotate-key.ts`)

// rotate-key.ts
import { SecretsManagerClient, GetSecretValueCommand, PutSecretValueCommand } from "@aws-sdk/client-secrets-manager";

/**
 * This Lambda reads the current Claude key, generates a new one
 * (here we just mock it), stores the new key, and returns the ARN.
 * In a real world you would call the Claude provider’s key‑rotation API.
 */
export const handler = async () => {
  const sm = new SecretsManagerClient({ region: "us-east-1" });
  const secretArn = "arn:aws:secretsmanager:us-east-1:123456789012:secret:claude/api-key-abc123";

  // 1️⃣ Retrieve current secret (optional – we may just replace it)
  const current = await sm.send(new GetSecretValueCommand({ SecretId: secretArn }));
  console.log("Current key fetched (masked).");

  // 2️⃣ Generate a fresh mock key – replace with real rotation logic
  const newKey = "new-mocked-claude-key-" + Date.now();

  // 3️⃣ Store the new key, overwriting the old value
  await sm.send(new PutSecretValueCommand({
    SecretId: secretArn,
    SecretString: newKey,
  }));

  console.log("Secret rotated.");
  return { status: "ok", newKeyArn: secretArn };
};

Deploy the function (Node 22 runtime) and give it permission to read/write the secret.

Tip: The Lambda runs inside a VPC that has a NAT gateway if your secret is encrypted with a KMS key that lives in a private subnet. Without the NAT gateway the Lambda cannot reach the KMS service.

2️⃣ Calling the Lambda from the build agent

In the review.js script we already imported LambdaClient and used InvokeCommand. The key points:

InvocationType: "RequestResponse" makes the build wait for the Lambda to finish, so we know the rotation succeeded before the build ends.
Error handling – if the Lambda returns FunctionError, we treat it as a failure and abort the build.

In plain English: The build asks Lambda to change the lock on the safe (the secret). If the lock can’t be changed, the build stops so you don’t continue with a compromised key.

Handling Rate Limits and Retry Logic

Claude’s API enforces a per‑minute request quota. If your repository has many changed files, you can quickly hit that limit.

1️⃣ Detecting a rate‑limit response

Claude returns HTTP 429 when the quota is exhausted. Our callClaude function already throws an error for any non‑2xx status, but we can be more specific:

async function callClaude(payload) {
  const response = await retry(() => fetch(CLAUDE_ENDPOINT, {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      "x-api-key": process.env.CLAUDE_API_KEY,
    },
    body: JSON.stringify(payload),
  }));
  if (response.status === 429) {
    const reset = response.headers.get("retry-after") || "60"; // seconds
    throw new Error(`Rate limit hit – retry after ${reset}s`);
  }
  // …rest unchanged
}

2️⃣ Back‑off strategy

When we catch a 429 we wait the Retry-After header (or a default) before trying again. The retry helper already multiplies the delay, but for rate limits we want a linear wait:

async function callClaudeWithBackoff(payload) {
  let attempt = 0;
  while (attempt < 5) {
    try {
      return await callClaude(payload);
    } catch (err) {
      if (err.message.includes("Rate limit")) {
        const waitSec = parseInt(err.message.split("after ")[1]) || 60;
        console.warn(`⏳ Waiting ${waitSec}s before retry #${attempt + 1}`);
        await new Promise(r => setTimeout(r, waitSec * 1000));
      } else {
        // Re‑throw other errors to be handled by the outer catch
        throw err;
      }
    }
    attempt++;
  }
  throw new Error("Exceeded max retries due to rate limits");
}

Replace the call in the main loop with callClaudeWithBackoff.

Key takeaway: Explicitly handling 429 lets the pipeline pause instead of failing outright, which is especially helpful for large PRs.

The Takeaway

What you now have: a CI‑integrated AI reviewer that runs automatically, stores its suggestions safely, and keeps its credentials rotating.

Store secrets in Secrets Manager and reference them with ${secretsmanager:…} to avoid the 4 KB truncation bug.
Mask environment variables in CodeBuild logs; otherwise, plain‑text values become visible.
Write a small Node agent that builds a clear prompt, calls Claude with native fetch, parses the JSON diff, and writes it to S3.
Rotate the Claude API key via Lambda so the key lives only for the duration of a single build.
Detect and respect Claude’s 429 rate‑limit by reading the Retry-After header and applying a linear back‑off.
Remember the CI‑layer advantages: consistency, zero manual steps, and early feedback that catches bugs before they merge.

Now you can add AI‑powered code reviews to any repository that already uses CodeBuild—no extra IDE plugins, no nightly jobs, just a smarter pipeline that watches over your code 24/7. Happy building!

Transparency notice

This article was written with the help of an AI system — Groq (GPT OSS 120B).

Published: 2026-10-21 · Primary focus: CodeBuild

All code blocks are intended to be correct and runnable, but please verify them
against the official docs for the tools mentioned before using in production.

Find an error? Drop a comment — corrections are always welcome.

CDK Pipelines for AI Agents: Deploying a Claude‑Powered Coding Assistant with Zero Manual Steps

Dinesh_gowtham — Tue, 20 Oct 2026 07:56:46 +0000

Deploying AI coding assistants usually means juggling Docker builds, secret management, and ad‑hoc scripts. With CDK Pipelines you can treat the entire Claude‑based agent as infrastructure code and let AWS handle the build, test, and deploy phases automatically. This post walks you through a fully automated, reproducible workflow that any team can adopt.

Why CDK Pipelines Matter for AI Agents

When you ship a traditional web service, you already know the value of Infrastructure as Code (IaC) – a practice where you describe cloud resources in a programming language instead of clicking around the console.

CDK Pipelines is an IaC pattern built on top of the AWS Cloud Development Kit (CDK). It gives you a ready‑made CodePipeline (a series of automated steps) that knows how to:

Pull source code from a repository.
Build artifacts (like Docker images).
Run tests.
Deploy to the target environment.

Why does this matter for AI agents?

An AI coding assistant is more than a piece of Python or Node code. It usually consists of:

A Docker container that bundles the Claude SDK and a small CLI.
Secrets (API keys) that must stay hidden.
Runtime resources (ECS/Fargate) that host the container.

If you manage each of those pieces by hand, you end up with:

Inconsistent builds – one developer may forget to update the secret, another may use a stale Docker cache.
Hard‑to‑audit changes – you cannot trace which version of the Claude model was used for a given deployment.
Longer onboarding – new teammates copy scripts and hope they work.

Treating the whole assistant as a single pipeline eliminates these problems. The pipeline becomes the single source of truth, and every run is repeatable, version‑controlled, and auditable.

Key takeaway: CDK Pipelines turn the “run‑once” steps of building and deploying an AI agent into a reliable, version‑controlled process that anyone can trigger.

A simple analogy

Think of a pipeline like a bakery assembly line. The source stage is the delivery truck bringing flour, the build stage is the mixer turning flour into dough, the test stage is the quality inspector, and the deploy stage is the oven that bakes the bread and puts it on the shelf. If you skip any station or let a worker improvise, the loaf may be undercooked or miss a crucial ingredient. CDK Pipelines make sure every loaf follows the same recipe, every time.

Setting Up the Claude Coding Assistant Project

Before we write any CDK code, we need a local project that represents the assistant. The goal is to keep it tiny so the example stays approachable.

1. Create a new folder

mkdir claude-coding-assistant
cd claude-coding-assistant

2. Initialize a TypeScript CDK app

npm init -y                         # creates package.json
npm install aws-cdk-lib constructs  # CDK core libraries
npm install -D typescript ts-node   # development tools
npx cdk init app --language=typescript

The cdk init command scaffolds a CDK app – a Node project where each file is a construct (a reusable piece of cloud infrastructure).

3. Add the Claude CLI wrapper

Create a folder cli/ and a tiny Node script that calls Claude:

mkdir cli
cat > cli/index.ts <<'EOF'
#!/usr/bin/env node
import { ClaudeClient } from "@anthropic-ai/sdk"; // pretend SDK
import * as readline from "readline";

const client = new ClaudeClient({ apiKey: process.env.CLAUDE_API_KEY! });

const rl = readline.createInterface({
  input: process.stdin,
  output: process.stdout,
});

rl.question("Give me a function description: ", async (desc) => {
  const response = await client.complete({
    prompt: `Write a JavaScript function that ${desc}`,
    max_tokens: 200,
  });
  console.log("\n--- Generated code ---\n");
  console.log(response.completion);
  rl.close();
});
EOF
chmod +x cli/index.ts

Tip: The script reads a description from the terminal, asks Claude for code, and prints the result. In a real product you would wrap this in a proper CLI library, but the above is enough for the demo.

4. Dockerfile

The Docker image will contain Node, the Claude SDK, and our script.

# Dockerfile
FROM node:20-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build   # compile TypeScript

FROM node:20-alpine
WORKDIR /app
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/node_modules ./node_modules
ENTRYPOINT ["node", "dist/cli/index.js"]

In plain English: We first compile the TypeScript code (npm run build) in a temporary image, then copy the compiled output into a small runtime image. This two‑stage build keeps the final image lightweight.

5. Commit everything

git init
git add .
git commit -m "Initial Claude assistant skeleton"

Now the repository is ready to be connected to a GitHub source stage.

Defining the CDK Pipeline Stack

With the project source ready, we can describe the whole pipeline in CDK. The stack will contain:

CodePipeline – the orchestrator.
CodeBuild – builds the Docker image and runs a quick Claude prompt test.
ECS Fargate Service – runs the container in production.
Secrets Manager – stores the Claude API key safely.

1. Install additional CDK libraries

npm install @aws-cdk/aws-codepipeline @aws-cdk/aws-codepipeline-actions \
            @aws-cdk/aws-codebuild @aws-cdk/aws-ecs @aws-cdk/aws-ecs-patterns \
            @aws-cdk/aws-secretsmanager

2. Create `pipeline-stack.ts`

// lib/pipeline-stack.ts
import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import * as codepipeline from 'aws-cdk-lib/aws-codepipeline';
import * as cpactions from 'aws-cdk-lib/aws-codepipeline-actions';
import * as codebuild from 'aws-cdk-lib/aws-codebuild';
import * as ecs from 'aws-cdk-lib/aws-ecs';
import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns';
import * as secretsmanager from 'aws-cdk-lib/aws-secretsmanager';

// A PipelineStack defines the CI/CD flow for our Claude assistant.
export class PipelineStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    // --------------------------------------------------------------------
    // 1️⃣ Source stage – pull code from GitHub
    // --------------------------------------------------------------------
    const sourceOutput = new codepipeline.Artifact();

    const sourceAction = new cpactions.GitHubSourceAction({
      actionName: 'GitHub_Source',
      owner: 'my-github-org',          // replace with your org/user
      repo: 'claude-coding-assistant', // repo name
      branch: 'main',
      oauthToken: cdk.SecretValue.secretsManager('github-token'), // stored secret
      output: sourceOutput,
    });

    // --------------------------------------------------------------------
    // 2️⃣ Build stage – Docker image + Claude sanity test
    // --------------------------------------------------------------------
    // Secret that holds Claude API key (must be created in Secrets Manager)
    const claudeSecret = secretsmanager.Secret.fromSecretNameV2(
      this,
      'ClaudeApiKey',
      'claude/api-key'
    );

    const buildProject = new codebuild.PipelineProject(this, 'BuildProject', {
      environment: {
        buildImage: codebuild.LinuxBuildImage.STANDARD_7_0,
        privileged: true, // required for Docker-in-Docker
        // Pass the secret as an environment variable that CodeBuild will mask in logs
        environmentVariables: {
          CLAUDE_API_KEY: {
            type: codebuild.BuildEnvironmentVariableType.SECRETS_MANAGER,
            value: claudeSecret.secretArn,
          },
        },
      },
      // Grant the build role permission to read the secret
      role: new iam.Role(this, 'BuildRole', {
        assumedBy: new iam.ServicePrincipal('codebuild.amazonaws.com'),
        managedPolicies: [
          iam.ManagedPolicy.fromAwsManagedPolicyName('AmazonEC2ContainerRegistryPowerUser'),
        ],
      }).addToPolicy(
        new iam.PolicyStatement({
          actions: ['secretsmanager:GetSecretValue'],
          resources: [claudeSecret.secretArn],
        })
      ),
      // The buildspec tells CodeBuild exactly what to do.
      // Note the explicit export of the secret – otherwise it never reaches the Docker build.
      buildSpec: codebuild.BuildSpec.fromObject({
        version: '0.2',
        phases: {
          pre_build: {
            commands: [
              'echo Logging in to Amazon ECR...',
              'aws ecr get-login-password --region $AWS_DEFAULT_REGION | docker login --username AWS --password-stdin $ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com',
            ],
          },
          build: {
            commands: [
              // Export secret into the Docker build context (gotcha #1)
              'export CLAUDE_API_KEY=$CLAUDE_API_KEY',
              'docker build -t claude-assistant .',
              // Tag with the commit SHA from the source artifact
              'docker tag claude-assistant:latest $ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com/claude-assistant:$CODEBUILD_RESOLVED_SOURCE_VERSION',
              // Push the image for later deployment
              'docker push $ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com/claude-assistant:$CODEBUILD_RESOLVED_SOURCE_VERSION',
              // Run a quick sanity test – invoke the CLI with a sample prompt
              'docker run --rm -e CLAUDE_API_KEY=$CLAUDE_API_KEY $ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com/claude-assistant:$CODEBUILD_RESOLVED_SOURCE_VERSION node -e "require(\'./dist/cli/index\')" <<< "write a function that adds two numbers"',
            ],
          },
        },
        artifacts: {
          files: ['**/*'],
        },
      }),
    });

    const buildAction = new cpactions.CodeBuildAction({
      actionName: 'Build_and_Test',
      project: buildProject,
      input: sourceOutput,
      outputs: [new codepipeline.Artifact()], // dummy artifact for downstream
    });

    // --------------------------------------------------------------------
    // 3️⃣ Deploy stage – ECS Fargate service
    // --------------------------------------------------------------------
    const cluster = new ecs.Cluster(this, 'AssistantCluster', {
      vpc: new ec2.Vpc(this, 'VPC'), // creates a new VPC (simplified)
    });

    const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(
      this,
      'AssistantService',
      {
        cluster,
        taskImageOptions: {
          image: ecs.ContainerImage.fromRegistry(
            `${cdk.Aws.ACCOUNT_ID}.dkr.ecr.${cdk.Aws.REGION}.amazonaws.com/claude-assistant:${cdk.Fn.getArtifactMetadata(sourceOutput, 'CommitId')}`
          ),
          // Pass the API key at runtime (environment variable)
          environment: {
            CLAUDE_API_KEY: claudeSecret.secretValue.toString(),
          },
          secrets: {
            CLAUDE_API_KEY: ecs.Secret.fromSecretsManager(claudeSecret),
          },
        },
        desiredCount: 1,
        publicLoadBalancer: true,
      }
    );

    // --------------------------------------------------------------------
    // Assemble the pipeline
    // --------------------------------------------------------------------
    new codepipeline.Pipeline(this, 'ClaudeAssistantPipeline', {
      pipelineName: 'ClaudeAssistantPipeline',
      stages: [
        {
          stageName: 'Source',
          actions: [sourceAction],
        },
        {
          stageName: 'Build',
          actions: [buildAction],
        },
        {
          stageName: 'Deploy',
          actions: [
            new cpactions.EcsDeployAction({
              actionName: 'DeployToFargate',
              service: fargateService.service,
              input: buildAction.actionProperties.outputs![0],
            }),
          ],
        },
      ],
    });
  }
}

What the code does, step by step

Step	Explanation
Source	Pulls the latest commit from GitHub using a personal‑access token stored in Secrets Manager.
Build	Spins up a CodeBuild container, logs in to Amazon ECR (Elastic Container Registry), builds the Docker image, pushes it, and runs a tiny Claude request to confirm the key works. The line `export CLAUDE_API_KEY=$CLAUDE_API_KEY` is the crucial fix that forwards the secret into the Docker build context.
Deploy	Creates an ECS Fargate service (serverless containers) behind an Application Load Balancer. The container receives the Claude API key at runtime via the `secrets` field, which keeps the value encrypted in transit and out of logs.
Pipeline	Chains the three stages together so that a new commit automatically triggers the whole flow.

Helpful tip: After you write the stack, run cdk bootstrap once per account/region before any cdk deploy. This creates the CDK‑specific S3 bucket and IAM roles needed for the pipeline. Forgetting this step is a common source of “bootstrap missing” errors.

3️⃣ Bootstrap the environment (the hidden step)

npx cdk bootstrap aws://123456789012/us-east-1

Replace the account ID and region with yours. The command creates a bucket named cdk-bootstrap-<account>-<region> and a few roles that the pipeline later assumes.

Integrating Claude Calls in the Build Phase

The most fragile part of an AI‑driven pipeline is secret propagation. CodeBuild can read a secret, but Docker builds run in a separate process that does not inherit environment variables automatically. If you try to docker build without explicitly exporting the secret, the container’s build‑time script will see an empty CLAUDE_API_KEY and any API call will fail.

Two ways to get the secret into Docker

Export before docker build – we used export CLAUDE_API_KEY=$CLAUDE_API_KEY. This copies the variable from the CodeBuild environment into the shell that runs docker build. Inside the Dockerfile you can reference it with ARG CLAUDE_API_KEY and then ENV CLAUDE_API_KEY=$CLAUDE_API_KEY if you need it at runtime.
secretsManager field in CodeBuild – newer CodeBuild versions let you declare a secretsManager mapping, which injects the secret as a file inside the build container. This approach avoids exposing the key in plain‑text logs.

Below is a minimal Dockerfile snippet that accepts the secret as a build‑time argument:

# Inside Dockerfile (add near the top)
ARG CLAUDE_API_KEY
ENV CLAUDE_API_KEY=${CLAUDE_API_KEY}

Now, when the build command runs, the secret is baked into the image’s environment (still encrypted at rest because the image is stored in ECR). For extra safety you can avoid persisting the key by only passing it at runtime (as we do for the ECS task).

In plain English: Think of the secret as a password you need to write on a sticky note for the baker (Docker) to see. If you forget to put the note on the counter, the baker can’t add the secret ingredient, and the cake (your image) will be flavorless.

Running a sanity check

The buildspec includes a one‑line sanity test:

docker run --rm -e CLAUDE_API_KEY=$CLAUDE_API_KEY \
  $ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com/claude-assistant:$CODEBUILD_RESOLVED_SOURCE_VERSION \
  node -e "require('./dist/cli/index')" <<< "write a function that adds two numbers"

What happens?

docker run starts a container from the image we just built.
-e CLAUDE_API_KEY=$CLAUDE_API_KEY injects the key as an environment variable.
The tiny Node script reads a prompt from STDIN, sends it to Claude, and prints the generated function.

If the key is missing, the CLI will exit with an authentication error, and the entire pipeline will fail—exactly what you want for early detection.

Testing and Observability

Automation is only as good as the feedback you get when something goes wrong. With CDK Pipelines you have three built‑in places to observe the health of your AI assistant:

Observation point	What to look for	Why it matters
CodeBuild logs	Errors like “Secret not found” or “Docker login failed”	Immediate insight into build‑time failures, especially secret propagation.
ECS task logs	Runtime exceptions, missing environment variables	Detects issues that only appear after deployment (e.g., wrong IAM permissions).
CloudWatch Metrics	CPU/Memory usage of the Fargate task	Helps you size the service appropriately; AI inference can be CPU‑heavy.

Adding a simple test stage

We can add an extra CodeBuild action that runs a post‑deploy test against the live service. The test will call the public load balancer endpoint with a sample prompt and verify that the response contains valid JavaScript.

// Inside PipelineStack constructor, after the Deploy stage definition
const testProject = new codebuild.PipelineProject(this, 'PostDeployTest', {
  environment: {
    buildImage: codebuild.LinuxBuildImage.STANDARD_7_0,
  },
  buildSpec: codebuild.BuildSpec.fromObject({
    version: '0.2',
    phases: {
      build: {
        commands: [
          // Grab the load balancer DNS name from CloudFormation output
          `LB_DNS=$(aws cloudformation describe-stacks --stack-name ${cdk.Stack.of(this).stackName} --query "Stacks[0].Outputs[?OutputKey==\'LoadBalancerDNS\'].OutputValue" --output text)`,
          'echo Testing endpoint $LB_DNS',
          // Send a sample request
          `RESPONSE=$(curl -s -X POST -H "Content-Type: application/json" -d \'{"prompt":"write a function that multiplies two numbers"}\' http://$LB_DNS)`,
          // Very naive validation
          'if echo $RESPONSE | grep -q "function"; then echo "✅ Test passed"; else echo "❌ Test failed"; exit 1; fi',
        ],
      },
    },
  }),
});

const testAction = new cpactions.CodeBuildAction({
  actionName: 'PostDeployTest',
  project: testProject,
  input: buildAction.actionProperties.outputs![0],
});

pipeline.addStage({
  stageName: 'Test',
  actions: [testAction],
});

The snippet does three things:

Looks up the load balancer DNS name from the CloudFormation stack (the pipeline’s own stack).
Sends a POST request containing a prompt.
Checks that the response includes the word function, a simple sanity check.

Key tip: CloudWatch Logs automatically capture curl output and any echo statements. If your test fails, the pipeline will stop and you’ll see the exact HTTP payload that caused the problem.

Observability best practices for AI services

Mask secrets in logs – CodeBuild environment variables of type SECRETS_MANAGER are automatically hidden, but any echo $CLAUDE_API_KEY will expose them. Avoid printing the variable.
Enable VPC flow logs – If you attach CodeBuild to a VPC (common for private ECR access), make sure the VPC has a NAT Gateway; otherwise the build container cannot reach the Claude API endpoint on the internet.
Set a build cache – Docker layer caching speeds up subsequent builds, but remember that the cache is not invalidated when you rotate the Claude API key. Add a dummy ARG CACHE_BUSTER=$(date +%s) line to the Dockerfile to force a fresh build after secret rotation.

The Takeaway

In plain English: By describing your Claude‑powered coding assistant as code, you let AWS do the heavy lifting of building, testing, and deploying. No manual Docker commands, no hidden credentials, and no “it works on my machine” surprises.

CDK Pipelines turn a series of manual steps into a single, version‑controlled pipeline that can be triggered by any git push.
Secrets must be exported explicitly into Docker builds; otherwise the Claude SDK cannot authenticate.
CodeBuild logs expose unmasked environment variables, so always use the SECRETS_MANAGER type and avoid printing the values.
A VPC‑attached CodeBuild needs a NAT Gateway to reach external APIs like Claude; otherwise builds will fail at the network level.
Adding a lightweight post‑deploy test gives fast feedback that the live service can actually talk to Claude.
Remember to run cdk bootstrap once per account/region; without it the pipeline cannot create the required S3 bucket and IAM roles.

With these pieces in place, any team can spin up a reproducible, auditable, and secure Claude‑based coding assistant in minutes—no manual Docker pushes, no secret leaks, and no hidden steps. Happy building!

Transparency notice

This article was written with the help of an AI system — Groq (GPT OSS 120B).

Published: 2026-10-20 · Primary focus: CDK

All code blocks are intended to be correct and runnable, but please verify them
against the official docs for the tools mentioned before using in production.

Find an error? Drop a comment — corrections are always welcome.

How IAM Identity Center Powers Secure Access for Claude AI Agents in Node.js, Explained Simply

Dinesh_gowtham — Mon, 19 Oct 2026 08:06:23 +0000

Human users and AI agents both need the right permissions, but mixing SSO with programmatic access is tricky. In just a few minutes you’ll see how to let a Claude‑powered Lambda speak to Bedrock without ever hard‑coding static keys. The pattern works for any LLM‑backed tool you build today.

Why IAM Identity Center Matters for AI Agents

When a human logs into the AWS console we usually think about IAM Identity Center (formerly AWS SSO) as a single‑sign‑on (SSO) portal. It lets people click a link, type a password, and get a web console with all the permissions they need.

An AI agent—for example a Lambda function that talks to Claude on Bedrock—needs the same “right‑things‑can‑do” guarantee, but it can’t click a button. If you give the function long‑lived static credentials, you expose a secret that never rotates.

In plain English: IAM Identity Center can hand out short‑lived, automatically‑rotated credentials for programs, just like it hands a temporary badge to a visitor.

The benefit is two‑fold:

Zero static keys – the Lambda never stores an access key in code or environment variables.
Least‑privilege – you attach a permission set (a collection of IAM policies) that describes exactly what the agent may do, and the temporary credentials inherit only those rights.

Quick analogy

Think of IAM Identity Center as a hotel front desk. A human guest gets a room key that works for a few hours. An AI agent is a robot concierge that needs a key to fetch a specific amenity (the Claude model). The front desk can issue a temporary key for the robot, and the key expires automatically, keeping the hotel secure.

Setting Up a Permission Set for Machine Access

A permission set groups one or more IAM policies and is assigned to a user or group in IAM Identity Center. For a machine, you create a dedicated permission set that includes only the actions needed to call Bedrock and read any supporting resources (like a Secrets Manager secret that holds the SAML assertion).

Tip: Keep the permission set tiny. If the agent only needs bedrock:InvokeModel, don’t also grant s3:*.

Step‑by‑step

Open the IAM Identity Center console.
Create a new permission set, call it ClaudeAgentAccess.
Attach an inline policy that allows:

bedrock:InvokeModel on the Claude model ARN.
sts:AssumeRoleWithSAML on the SAML provider you will use.
Any read‑only permissions for supporting resources (e.g., secretsmanager:GetSecretValue).

Assign the permission set to a service principal—in practice you’ll map it to a SAML application that your Lambda will use.

Policy‑as‑Code with TypeScript’s `satisfies`

Instead of hand‑editing JSON in the console, you can write the policy in TypeScript and let the compiler verify its shape. The satisfies operator ensures the object conforms to the IAM policy schema at compile time.

// src/policy.ts
import type { PolicyDocument } from "aws-sdk/clients/iam";

/**
 * Inline policy granting the minimal rights for a Claude‑powered Lambda.
 * The `satisfies` keyword makes TypeScript check that every required field exists.
 */
export const claudeAgentPolicy = {
  Version: "2012-10-17",
  Statement: [
    {
      Effect: "Allow",
      Action: [
        "bedrock:InvokeModel",               // call Claude on Bedrock
        "sts:AssumeRoleWithSAML",            // exchange SAML for temporary creds
        "secretsmanager:GetSecretValue",     // read the SAML assertion stored securely
      ],
      Resource: "*", // you could lock this down to specific ARNs for tighter security
    },
  ],
} satisfies PolicyDocument;

Key takeaway: Using satisfies catches typos (like "bedrock:InvokModel") before you ever deploy, turning a runtime surprise into a compile‑time warning.

You can copy the resulting JSON into the console’s Inline Policy editor when you create the permission set.

Node.js Lambda: Assuming the Role Programmatically

Now that the permission set exists, the Lambda must assume the role represented by that set. The standard way is to call the AWS Security Token Service (STS) operation AssumeRoleWithSAML. The flow looks like this:

Retrieve the SAML assertion from Secrets Manager (or another secure store).
Call AssumeRoleWithSAML, passing the SAML assertion and the ARN of the permission set’s IAM role.
STS returns temporary credentials (AccessKeyId, SecretAccessKey, SessionToken) that are valid for up to 1 hour.
Use those credentials to create a Bedrock client and invoke Claude.

Below is a minimal, fully‑commented Lambda handler written in TypeScript for Node.js 22. It uses the AWS SDK v3 packages you asked for: @aws-sdk/client-sts, @aws-sdk/client-secrets-manager, and @aws-sdk/client-bedrock-runtime. (The client-iam and client-lambda packages are not needed directly in the handler but are listed in the service context for completeness.)

// src/handler.ts
import {
  STSClient,
  AssumeRoleWithSAMLCommand,
} from "@aws-sdk/client-sts";
import {
  SecretsManagerClient,
  GetSecretValueCommand,
} from "@aws-sdk/client-secrets-manager";
import {
  BedrockRuntimeClient,
  InvokeModelCommand,
} from "@aws-sdk/client-bedrock-runtime";
import type { Handler, Context, Callback } from "aws-lambda";

// Environment variables you set in the Lambda console
const {
  SAML_SECRET_ID,          // Secrets Manager name that holds the Base64‑encoded SAML assertion
  IAM_ROLE_ARN,            // The role ARN linked to the permission set
  CLAUDE_MODEL_ID,         // e.g. "anthropic.claude-v2:1"
  REGION = "us-east-1",    // Bedrock region
} = process.env;

// Re‑use SDK clients – they are cheap to create and benefit from connection pooling
const sts = new STSClient({ region: REGION });
const secrets = new SecretsManagerClient({ region: REGION });

/**
 * Helper: fetch the raw SAML assertion (Base64 string) from Secrets Manager.
 */
async function getSamlAssertion(): Promise<string> {
  const cmd = new GetSecretValueCommand({ SecretId: SAML_SECRET_ID });
  const resp = await secrets.send(cmd);
  if (!resp.SecretString) {
    throw new Error("SAML secret is empty");
  }
  return resp.SecretString;
}

/**
 * Helper: exchange the SAML assertion for temporary AWS credentials.
 * Throws ExpiredTokenException if the SAML token is older than 1 hour.
 */
async function assumeRoleWithSaml(samlAssertion: string) {
  const cmd = new AssumeRoleWithSAMLCommand({
    RoleArn: IAM_ROLE_ARN,
    PrincipalArn: `arn:aws:iam::${process.env.AWS_ACCOUNT_ID}:saml-provider/YourIdCProvider`,
    SAMLAssertion: Buffer.from(samlAssertion).toString("base64"),
    DurationSeconds: 3600, // 1 hour, matches the SAML token lifetime
  });

  const resp = await sts.send(cmd);
  if (!resp.Credentials) {
    throw new Error("Failed to obtain temporary credentials");
  }
  return resp.Credentials;
}

/**
 * Main Lambda entry point.
 * It fetches a SAML token, swaps it for short‑lived credentials,
 * then calls Claude on Bedrock with those credentials.
 */
export const handler: Handler = async (
  event: unknown,
  _context: Context,
  _callback: Callback
) => {
  try {
    // 1️⃣ Pull the SAML assertion (the "badge" the front desk gave us)
    const samlAssertion = await getSamlAssertion();

    // 2️⃣ Turn the badge into a temporary keycard (AssumeRoleWithSAML)
    const tempCreds = await assumeRoleWithSaml(samlAssertion);

    // 3️⃣ Build a Bedrock client that uses the temporary credentials
    const bedrock = new BedrockRuntimeClient({
      region: REGION,
      credentials: {
        accessKeyId: tempCreds.AccessKeyId!,
        secretAccessKey: tempCreds.SecretAccessKey!,
        sessionToken: tempCreds.SessionToken!,
        expiration: tempCreds.Expiration,
      },
    });

    // 4️⃣ Prepare a simple Claude request
    const prompt = "Explain the difference between TCP and UDP in two sentences.";
    const payload = {
      // Bedrock expects a JSON string for Claude's body
      body: JSON.stringify({ prompt, max_tokens_to_sample: 256, temperature: 0.7 }),
      modelId: CLAUDE_MODEL_ID,
      accept: "application/json",
      contentType: "application/json",
    };

    // 5️⃣ Send the request
    const response = await bedrock.send(new InvokeModelCommand(payload));

    // 6️⃣ Return the model's answer (or handle streaming in real apps)
    const result = Buffer.from(response.body!).toString("utf-8");
    console.log("Claude response:", result);

    return {
      statusCode: 200,
      body: result,
    };
  } catch (error: any) {
    // The most common failure is an ExpiredTokenException when the SAML token
    // is older than 1 hour. We re‑run the assume flow in that case.
    if (error.name === "ExpiredTokenException") {
      console.warn("SAML token expired – retrying assume flow");
      // In a real Lambda you might loop a couple of times or push to DLQ.
      throw error;
    }

    console.error("Unhandled error:", error);
    return {
      statusCode: 500,
      body: JSON.stringify({ message: error.message }),
    };
  }
};

Helpful tip: The AWS SDK for JavaScript does not automatically refresh SAML‑derived credentials. If you see ExpiredTokenException, catch it and run the AssumeRoleWithSAML flow again, as demonstrated in the catch block.

Gotchas specific to Lambda

require(esm) in Node 22 – if you import an ESM‑only layer, use import syntax exclusively; mixing require will cause silent failures.
SnapStart + VPC – enabling SnapStart on a Lambda that lives inside a VPC gives no cold‑start benefit because the network interface still needs to attach. Keep your Bedrock calls outside a VPC unless you need VPC‑restricted resources.
Provisioned Concurrency – turning it on for a short‑lived AI worker can surprise the bill. Test with on‑demand first.

Calling Claude via Bedrock with Secure Credentials

Now that we have a Bedrock client backed by temporary credentials, the actual model call is straightforward. The heavy lifting is in the request payload format, which differs per provider. Claude expects a JSON body with prompt, max_tokens_to_sample, and optional parameters like temperature.

In plain English: Think of the Bedrock client as a mailbox you hand a sealed letter to (the request). The envelope (HTTP headers) tells Bedrock what kind of letter it is (application/json). The temporary credentials are the postage stamp that proves you’re allowed to send mail to that address.

Minimal Claude request

// Inside the Lambda handler – after creating `bedrock` client
const requestPayload = {
  body: JSON.stringify({
    prompt: "Write a haiku about sunrise over the mountains.",
    max_tokens_to_sample: 64,
    temperature: 0.5,
  }),
  modelId: CLAUDE_MODEL_ID,
  accept: "application/json",
  contentType: "application/json",
};

const result = await bedrock.send(new InvokeModelCommand(requestPayload));
const modelOutput = Buffer.from(result.body!).toString("utf-8");
console.log("Claude says:", modelOutput);

Why we don’t use static keys: The bedrock client receives credentials that expire after an hour. If the Lambda runs longer (e.g., a long‑running batch job), the next iteration will hit ExpiredTokenException. The catch block from the previous section restarts the assume flow, guaranteeing fresh credentials without any human touch.

Policy‑as‑Code: Using TypeScript’s `satisfies` for IAM Policies

Earlier we stored the policy in policy.ts. Let’s expand that pattern to a policy generation script that could be run in CI/CD pipelines. The idea is to keep IAM definitions alongside your application code, ensuring they evolve together.

// scripts/build-policies.ts
import { writeFileSync } from "fs";
import { claudeAgentPolicy } from "../src/policy";

// The `as const` assertion preserves literal types for maximum safety.
const policyJson = JSON.stringify(claudeAgentPolicy, null, 2) as const;

// Write the policy to a file that can be fed to `aws iam create-policy`
writeFileSync("./dist/claude-agent-policy.json", policyJson);
console.log("Policy written to ./dist/claude-agent-policy.json");

Running npm run build-policies would generate a JSON file ready for the AWS CLI:

aws iam create-policy \
  --policy-name ClaudeAgentInlinePolicy \
  --policy-document file://dist/claude-agent-policy.json

Key takeaway: By generating policies from TypeScript, you eliminate drift between code and permissions. The satisfies keyword guarantees the shape matches IAM’s expectations before you ever call the CLI.

The Takeaway

Key points to remember

IAM Identity Center can issue short‑lived machine credentials; treat it as a badge‑issuer, not just an SSO portal.
A permission set should be tiny—grant only bedrock:InvokeModel and the minimal SAML/SecretsManager actions.
AssumeRoleWithSAML returns credentials that expire after 1 hour; the SDK won’t refresh them automatically—handle ExpiredTokenException yourself.
In Lambda, keep the code simple: fetch SAML from Secrets Manager, exchange it via STS, then call Bedrock with the temporary creds.
Use TypeScript’s satisfies operator to write IAM policies as code, catching mistakes at compile time.
Watch out for Lambda‑specific quirks: ESM imports, SnapStart inside VPCs, and unintended Provisioned Concurrency costs.

By following this pattern, you give your Claude‑powered agents the same security rigor you apply to human users, while staying free of hard‑coded keys and long‑lived secrets. The approach works for any LLM you hook up to Bedrock, making your AI tooling both safe and maintainable.

Transparency notice

This article was written with the help of an AI system — Groq (GPT OSS 120B).

Published: 2026-10-19 · Primary focus: IAM

All code blocks are intended to be correct and runnable, but please verify them
against the official docs for the tools mentioned before using in production.

Find an error? Drop a comment — corrections are always welcome.

DEV Community: Dinesh_gowtham

MCP Explained Simply: How to Build a Claude‑Powered AI Agent with AWS Lambda

What Is MCP and Why It Matters

The MCP Wire‑Format – A Quick Walkthrough

A Simple Analogy

Setting Up a Lambda‑Backed MCP Proxy

1️⃣ Create the Lambda function

2️⃣ Add the SDK packages

3️⃣ Write the handler

4️⃣ Deploy

Calling Claude via Bedrock from Lambda

1️⃣ The sendMessage command (conceptually)

2️⃣ Handling incremental responses

3️⃣ Cost awareness

Testing the End‑to‑End Flow

1️⃣ Deploy a test endpoint

2️⃣ First request – start a new session

3️⃣ Follow‑up request – keep context

4️⃣ Automated test script (optional)

The Takeaway

Streaming Claude Tokens with Fastify on Lambda: A Step‑By‑Step Guide Explained Simply

Why Fastify Beats Express for LLM Streaming

Code snippet: a minimal Fastify server

Setting Up Fastify with Native fetch on Node.js 22

Code snippet: enabling fetch in a Fastify route

Calling Claude via Bedrock and Streaming SSE

What is “streaming” in this context?

Setting up the Bedrock client

Handling backpressure

Deploying the Fastify Service to Lambda

Lambda handler file

Deploy script (using the AWS CDK for illustration)

Lambda layer gotcha reminder

Testing the End‑to‑End Token Stream

Using curl

Node test script

Verifying Lambda buffering is disabled

The Takeaway

pnpm Workspaces in Lambda: How to Shrink Deploy Packages and Supercharge Claude‑Powered AI Agents, Explained Simply

Why Deployment Size Matters for Claude‑Powered Lambda Functions

Introducing pnpm Workspaces: How They Differ from npm and Yarn

Setting Up a Monorepo for Your Claude Agent with Corepack

Building and Deploying the Lambda: Code Walkthrough

1️⃣ The Lambda handler (agent/src/handler.ts)

2️⃣ Shared helper (shared/src/helpers.ts)

3️⃣ Bundling with esbuild while preserving pnpm’s layout

4️⃣ Deploying with AWS CLI

Testing, Debugging, and Observability Tips

Local testing with the SAM CLI

Debugging native‑module failures

Observability – CloudWatch Logs & Metrics

Known gotchas you might run into

The Takeaway

How TypeScript 5.5 Inferred Type Predicates Make Claude Tool Calls Safer, Explained Simply

Why Claude Tool Calls Need Strong Types

The hidden cost in serverless environments

TypeScript 5.5 Inferred Type Predicates in One Line

Analogy

Building a Typed Claude Tool Wrapper

The wrapper code

Real‑World Example: Auto‑Complete Suggestions from Claude

Handling Bedrock’s streaming edge case

Testing the Wrapper with ts-node/tsx

Quick sanity check with filter(Boolean)

The Takeaway

Testing Claude Prompt Outputs with Node.js Built‑in test Runner: A Simple, Real‑World Guide

Why Test LLM Prompts at All

Getting Started with node:test

Running the tests

Gotcha #1 – Returning async promises

Calling Claude via Bedrock from a Test

Why we set temperature: 0

Gotcha #2 – Streaming responses are nondeterministic

Gotcha #3 – Bedrock token limits per minute

Snapshot Testing LLM Output

How t.snapshot works

Normalizing snapshots

Running Tests in Watch Mode with Coverage

Watch mode

Coverage

1️⃣ The `sendMessage` command (conceptually)

1️⃣ The Lambda handler (`agent/src/handler.ts`)

2️⃣ Shared helper (`shared/src/helpers.ts`)

How TypeScript 5.5 Inferred Type Predicates Make Claude Tool Calls Safer, Explained Simply

TypeScript 5.5 Inferred Type Predicates in One Line

Quick sanity check with `filter(Boolean)`

Getting Started with `node:test`

Why we set `temperature: 0`

How `t.snapshot` works

Secure AI Tool Calls in Node.js 22: Using Permission Flags to Sandbox Claude Agents

Node.js 22 Permission Model Overview

Lambda Consumer with Node.js 22: Decoding and Forwarding

Step Functions + Claude Tool Calls: Orchestrate AI Agents Without Lambda, Explained Simply

2. 256 KB input/output cap