DEV Community: Dinesh_gowtham

AI Pair Programming with Lambda: Build a Claude‑Powered Coding Assistant and Enable Prompt Caching

Dinesh_gowtham — Fri, 25 Sep 2026 08:11:01 +0000

Developers are tired of line‑by‑line autocomplete and want an AI that can take a whole function and return production‑ready code. By using AWS Lambda and Claude’s API you can offload that work to the cloud and get instant, cached responses. This guide shows exactly how.

Why AI Pair Programming Needs More Than Autocomplete

Autocomplete is useful for filling in a variable name, but a real pair programmer can understand the intent of an entire block, suggest a new implementation, and even refactor code to match your style guide.

In plain English: Think of autocomplete as a spell‑checker that fixes single words, while a coding assistant is like a seasoned teammate who can rewrite a whole paragraph for you.

The missing piece: stateful prompting

Large language models (LLMs) like Claude work best when you give them a prompt – a text description of what you want. If you rebuild that prompt for every request, you waste time and money. Keeping the prompt (or parts of it) in a local file that survives between Lambda invocations is called prompt caching.

Key takeaway: Prompt caching turns a cold‑start‑only request into a warm‑request that reuses work already done.

Code: Minimal Lambda that just echoes a prompt (no caching yet)

import { APIGatewayProxyEvent, APIGatewayProxyResult } from 'aws-lambda';

// Simple handler that returns the received body unchanged
export const handler = async (
  event: APIGatewayProxyEvent
): Promise<APIGatewayProxyResult> => {
  // Parse the incoming JSON payload
  const body = JSON.parse(event.body ?? '{}');

  // Echo back the code snippet the user sent
  return {
    statusCode: 200,
    body: JSON.stringify({ received: body.code }),
  };
};

What you see:

APIGatewayProxyEvent – the shape of the HTTP request Lambda receives from API Gateway.
APIGatewayProxyResult – the shape of the HTTP response Lambda must return.

The example works, but it does nothing with Claude and it rebuilds the prompt every time. The next sections add security, caching, and the real LLM call.

Setting Up a Secure Claude API Call in Lambda

Claude’s API requires an API key. Storing that key directly in code would expose it to anyone who can read the source. AWS Secrets Manager is a managed service that encrypts secrets at rest and provides fine‑grained access control.

Why use Secrets Manager?

Keeps the key out of source control.
Allows rotation without code changes.
Provides audit logs for every access.

Gotcha: Secrets Manager charges per secret and per retrieval

A secret costs $0.40 per month, and each GetSecretValue call costs a fraction of a cent. If your Lambda fetches the secret on every request, the bill can grow unexpectedly. We’ll cache the secret in the same warm‑container storage (/tmp) after the first fetch.

Code: Securely read the Claude key and call the API

import {
  SecretsManagerClient,
  GetSecretValueCommand,
} from '@aws-sdk/client-secrets-manager';
import fetch from 'node-fetch'; // Node 22 has native fetch; keep for clarity

// Name of the secret that holds the Claude API key
const SECRET_NAME = process.env.CLAUDE_API_SECRET_NAME!;

// Reuse the client across invocations (saves connection overhead)
const secretsClient = new SecretsManagerClient({});

// In‑memory cache for the key; survives warm container restarts
let cachedApiKey: string | undefined;

/**
 * Retrieve the Claude API key.
 * On a warm container we read from `cachedApiKey`.
 * On a cold start we fetch from Secrets Manager and store it.
 */
async function getClaudeApiKey(): Promise<string> {
  if (cachedApiKey) {
    // Warm path – no extra cost
    return cachedApiKey;
  }

  // Cold path – fetch once, then cache
  const command = new GetSecretValueCommand({ SecretId: SECRET_NAME });
  const response = await secretsClient.send(command);

  if (!response.SecretString) {
    throw new Error('Secret retrieved but empty');
  }

  cachedApiKey = response.SecretString; // Save for later calls
  return cachedApiKey;
}

/**
 * Call Claude with a ready‑made prompt.
 * The function returns Claude’s `completion` field.
 */
export async function callClaude(prompt: string): Promise<string> {
  const apiKey = await getClaudeApiKey();

  const response = await fetch('https://api.anthropic.com/v1/complete', {
    method: 'POST',
    headers: {
      // Authorization header tells Claude who you are
      'x-api-key': apiKey,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      // `prompt` is the text we want Claude to continue
      prompt,
      max_tokens_to_sample: 1024,
      model: 'claude-3-5-sonnet-20240620',
    }),
  });

  if (!response.ok) {
    const err = await response.text();
    throw new Error(`Claude API error: ${response.status} ${err}`);
  }

  const data = await response.json();
  return data.completion as string;
}

Explanation:

SecretsManagerClient from @aws-sdk/client-secrets-manager talks to the Secrets service.
cachedApiKey lives in the container’s memory, so after the first request the Lambda does not hit Secrets Manager again.
callClaude sends a JSON payload to Claude’s endpoint and returns the generated text.

Tip: If you see unexpected charges, check CloudWatch logs for how often GetSecretValueCommand runs. It should only happen once per warm container.

Implementing Prompt Caching with Native TypeScript Stripping

Claude works best when you give it a system prompt that explains its role (e.g., “You are a helpful pair programmer that writes production‑ready TypeScript”). Building that prompt from scratch for each request adds latency and cost.

Why store the prompt on disk?

A Lambda container has a writable /tmp directory that persists for the lifetime of the container (up to 10 GB). By writing a JSON file there we keep the prompt alive across many invocations, even if the function is recycled later.

Analogy: Warm kitchen vs. cold pantry

Imagine a kitchen that keeps a pot of sauce simmering (the prompt) ready for any dish you want to prepare. When the kitchen is “warm,” you just ladle sauce into the pan. If the kitchen is “cold,” you have to start the sauce from scratch—slow and wasteful. /tmp is that simmering pot.

Gotcha: Cold starts wipe `/tmp`

When Lambda scales out, a new container starts with an empty /tmp. The first request after a scale‑out must rebuild the prompt file, which will be slower. We’ll log that event so you can monitor it.

Code: Prompt cache helper

import { promises as fs } from 'fs';
import path from 'path';

// Location of the cache file inside the writable /tmp directory
const CACHE_PATH = '/tmp/promptCache.json';

// Base system prompt that tells Claude what to do
const BASE_PROMPT = `
You are a pair programmer that writes production‑ready TypeScript.
When given a code snippet, return a refactored version that follows
the Airbnb style guide and includes type annotations.
Only return the code, no explanations.
`.trim();

/**
 * Load the cached prompt if it exists, otherwise create it.
 * Returns the full prompt string that will be sent to Claude.
 */
export async function getPromptCache(): Promise<string> {
  try {
    // Try reading the existing cache file
    const raw = await fs.readFile(CACHE_PATH, 'utf-8');
    const cached = JSON.parse(raw);
    // Simple version check – you could add more sophisticated diffing
    if (cached.version === 1) {
      console.log('✅ Prompt cache hit');
      return cached.prompt;
    }
  } catch (e) {
    // File does not exist or is malformed – we need to rebuild
    console.log('⚡ Prompt cache miss – rebuilding');
  }

  // Build the prompt once and write it to /tmp for future calls
  const fullPrompt = BASE_PROMPT + '\n\nUser code:';
  const payload = {
    version: 1,
    prompt: fullPrompt,
  };
  await fs.writeFile(CACHE_PATH, JSON.stringify(payload), 'utf-8');
  console.log('🗂 Prompt cache written to /tmp');
  return fullPrompt;
}

What the code does:

Attempts to read /tmp/promptCache.json.
If the file exists and matches the expected version, we reuse it.
If not, we construct the prompt and write it to the file.
Logging statements make it clear when a cold start occurs.

Key takeaway: Prompt caching turns a potentially expensive rebuild into a cheap file read, dramatically reducing latency for warm containers.

Connecting the Lambda via API Gateway for Real‑Time Editing

API Gateway is the HTTP front‑door that lets browsers or IDE extensions call your Lambda. To make the interaction feel like a live editor, we need to return the response quickly and with the right headers.

Why streaming matters

If Claude returns a large block of code, buffering the whole response in Lambda before sending it can add extra seconds. API Gateway supports response streaming, but only when you set Content-Type: application/json and use the isBase64Encoded flag correctly.

Gotcha: Response streaming needs explicit headers

If you omit Content-Type: application/json, API Gateway will buffer the whole payload, increasing latency. Also, Lambda@Edge has a 1 MB response limit, but a regular Lambda can stream larger payloads.

Code: Full Lambda handler with caching and streaming

import {
  APIGatewayProxyEvent,
  APIGatewayProxyResult,
  Context,
} from 'aws-lambda';
import { getPromptCache } from './promptCache';
import { callClaude } from './claudeClient';

// Helper to combine system prompt with user snippet
function buildFullPrompt(systemPrompt: string, userCode: string): string {
  // Insert the user code after the placeholder we defined earlier
  return `${systemPrompt}\n\n${userCode}`;
}

/**
 * Main Lambda entry point.
 * Receives a JSON body: { "code": "<typescript snippet>" }
 * Returns: { "refactored": "<Claude's output>" }
 */
export const handler = async (
  event: APIGatewayProxyEvent,
  _context: Context
): Promise<APIGatewayProxyResult> => {
  try {
    // 1️⃣ Parse incoming request
    const { code } = JSON.parse(event.body ?? '{}');
    if (!code) {
      return {
        statusCode: 400,
        body: JSON.stringify({ error: 'Missing `code` field' }),
      };
    }

    // 2️⃣ Load (or rebuild) the cached system prompt
    const systemPrompt = await getPromptCache();

    // 3️⃣ Build the final prompt that Claude will see
    const fullPrompt = buildFullPrompt(systemPrompt, code);

    // 4️⃣ Call Claude – this may take a second or two
    const claudeResult = await callClaude(fullPrompt);

    // 5️⃣ Return the result with streaming‑compatible headers
    return {
      statusCode: 200,
      headers: {
        // Explicitly tell API Gateway this is JSON so it can stream
        'Content-Type': 'application/json',
        // CORS headers if you call from a browser
        'Access-Control-Allow-Origin': '*',
      },
      // No need for base64 encoding – the payload is plain text JSON
      isBase64Encoded: false,
      body: JSON.stringify({ refactored: claudeResult.trim() }),
    };
  } catch (err) {
    console.error('❌ Handler error', err);
    return {
      statusCode: 500,
      body: JSON.stringify({ error: (err as Error).message }),
    };
  }
};

Step‑by‑step explanation:

Parse request – we expect a JSON body with a code field.
Load prompt cache – getPromptCache either reads the file or rebuilds it.
Combine system prompt and user snippet.
Call Claude – the heavy lifting happens here.
Return a JSON response with the correct Content-Type so API Gateway can stream.

Tip: Turn on Provisioned Concurrency if your team cannot tolerate any cold starts, but remember it costs money even when idle. Watch the billing dashboard after enabling it.

Testing the Assistant Locally with Node.js Test Runner

Before you deploy, run the Lambda handler locally to catch syntax errors and verify the caching logic. Using a lightweight test runner like Vitest (or plain node with assert) keeps the setup simple.

Why local testing matters

Lambda’s environment differs from your workstation (e.g., /tmp size).
You can mock Secrets Manager to avoid accidental charges.
Fast feedback loop reduces the number of deploy‑to‑test cycles.

Gotcha: Mocking `@aws-sdk/client-secrets-manager`

If you forget to mock the Secrets call, the test will hit the real service and incur cost. Use the SDK’s middlewareStack to inject a fake response.

Code: Minimal test file (`handler.test.ts`)

import { handler } from './handler';
import { APIGatewayProxyEvent } from 'aws-lambda';
import { SecretsManagerClient, GetSecretValueCommand } from '@aws-sdk/client-secrets-manager';

// ---------- Mock Secrets Manager ----------
jest.mock('@aws-sdk/client-secrets-manager', () => {
  const original = jest.requireActual('@aws-sdk/client-secrets-manager');
  return {
    ...original,
    SecretsManagerClient: jest.fn(() => ({
      send: async (cmd: GetSecretValueCommand) => ({
        SecretString: 'FAKE_CLAUDE_API_KEY',
      }),
    })),
  };
});

// ---------- Mock fetch (Claude) ----------
global.fetch = jest.fn(() =>
  Promise.resolve({
    ok: true,
    json: () => Promise.resolve({ completion: 'refactored code' }),
  })
) as any;

// ---------- Helper to build a fake API Gateway event ----------
function makeEvent(body: unknown): APIGatewayProxyEvent {
  return {
    body: JSON.stringify(body),
    headers: {},
    multiValueHeaders: {},
    httpMethod: 'POST',
    isBase64Encoded: false,
    path: '/',
    pathParameters: null,
    queryStringParameters: null,
    multiValueQueryStringParameters: null,
    stageVariables: null,
    requestContext: {} as any,
    resource: '',
  };
}

// ---------- The test ----------
test('handler returns refactored code', async () => {
  const event = makeEvent({ code: 'function add(a,b){return a+b;}' });

  const result = await handler(event, {} as any);

  expect(result.statusCode).toBe(200);
  const payload = JSON.parse(result.body);
  expect(payload.refactored).toBe('refactored code');
});

Explanation of the test:

Mock Secrets Manager so no real secret is fetched.
Mock fetch to simulate Claude’s response.
Build a minimal APIGatewayProxyEvent with a code snippet.
Call the handler and assert that we get a 200 response with the expected refactored field.

In plain English: This test pretends the Lambda is running in the cloud, but everything stays on your laptop, keeping you from accidental spend.

Run the test with npx vitest or npm test depending on your setup.

The Takeaway

Key points to remember

Autocomplete fixes tiny bits; a prompt‑driven LLM can rewrite whole functions, making a real pair‑programming experience.
Store secrets in AWS Secrets Manager and cache the API key in memory to avoid per‑request charges.
Write the system prompt to /tmp/promptCache.json; warm containers reuse it, dramatically cutting latency.
API Gateway needs explicit Content-Type: application/json for streaming; otherwise it buffers and slows you down.
Local tests with mocked Secrets Manager and fetch let you validate logic without incurring cloud costs.
Cold starts will always rebuild the prompt cache; log the miss so you can monitor scaling behavior.

Complete, Runnable Code Example

Below is the full set of files you need. Place them in a TypeScript project (npm init -y && npm i typescript @aws-sdk/client-secrets-manager @aws-sdk/client-lambda node-fetch) and run tsc to compile.

`src/promptCache.ts`

import { promises as fs } from 'fs';

const CACHE_PATH = '/tmp/promptCache.json';
const BASE_PROMPT = `
You are a pair programmer that writes production‑ready TypeScript.
When given a code snippet, return a refactored version that follows
the Airbnb style guide and includes type annotations.
Only return the code, no explanations.
`.trim();

export async function getPromptCache(): Promise<string> {
  try {
    const raw = await fs.readFile(CACHE_PATH, 'utf-8');
    const cached = JSON.parse(raw);
    if (cached.version === 1) {
      console.log('✅ Prompt cache hit');
      return cached.prompt;
    }
  } catch {
    console.log('⚡ Prompt cache miss – rebuilding');
  }

  const fullPrompt = BASE_PROMPT + '\n\nUser code:';
  const payload = { version: 1, prompt: fullPrompt };
  await fs.writeFile(CACHE_PATH, JSON.stringify(payload), 'utf-8');
  console.log('🗂 Prompt cache written');
  return fullPrompt;
}

`src/claudeClient.ts`

import {
  SecretsManagerClient,
  GetSecretValueCommand,
} from '@aws-sdk/client-secrets-manager';
import fetch from 'node-fetch';

const SECRET_NAME = process.env.CLAUDE_API_SECRET_NAME!;
const secretsClient = new SecretsManagerClient({});

let cachedApiKey: string | undefined;

async function getClaudeApiKey(): Promise<string> {
  if (cachedApiKey) return cachedApiKey;
  const cmd = new GetSecretValueCommand({ SecretId: SECRET_NAME });
  const resp = await secretsClient.send(cmd);
  if (!resp.SecretString) throw new Error('Secret empty');
  cachedApiKey = resp.SecretString;
  return cachedApiKey;
}

export async function callClaude(prompt: string): Promise<string> {
  const apiKey = await getClaudeApiKey();
  const res = await fetch('https://api.anthropic.com/v1/complete', {
    method: 'POST',
    headers: {
      'x-api-key': apiKey,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      prompt,
      max_tokens_to_sample: 1024,
      model: 'claude-3-5-sonnet-20240620',
    }),
  });

  if (!res.ok) {
    const txt = await res.text();
    throw new Error(`Claude error ${res.status}: ${txt}`);
  }

  const data = await res.json();
  return data.completion as string;
}

`src/handler.ts`

import {
  APIGatewayProxyEvent,
  APIGatewayProxyResult,
  Context,
} from 'aws-lambda';
import { getPromptCache } from './promptCache';
import { callClaude } from './claudeClient';

function buildFullPrompt(systemPrompt: string, userCode: string): string {
  return `${systemPrompt}\n\n${userCode}`;
}

export const handler = async (
  event: APIGatewayProxyEvent,
  _context: Context
): Promise<APIGatewayProxyResult> => {
  try {
    const { code } = JSON.parse(event.body ?? '{}');
    if (!code) {
      return {
        statusCode: 400,
        body: JSON.stringify({ error: 'Missing `code` field' }),
      };
    }

    const systemPrompt = await getPromptCache();
    const fullPrompt = buildFullPrompt(systemPrompt, code);
    const claudeResult = await callClaude(fullPrompt);

    return {
      statusCode: 200,
      headers: {
        'Content-Type': 'application/json',
        'Access-Control-Allow-Origin': '*',
      },
      isBase64Encoded: false,
      body: JSON.stringify({ refactored: claudeResult.trim() }),
    };
  } catch (e) {
    console.error('❌ Unexpected error', e);
    return {
      statusCode: 500,
      body: JSON.stringify({ error: (e as Error).message }),
    };
  }
};

`src/handler.test.ts` (optional, see earlier)

// (Copy the test code from the “Testing the Assistant Locally” section)

Deploy the handler to Lambda (Node.js 22.x runtime) and attach an API Gateway with POST integration. Set the environment variable CLAUDE_API_SECRET_NAME to the name of the secret you created in Secrets Manager.

Next Steps

Add versioning to the prompt cache – when you change the system prompt, bump the version number so containers rebuild automatically.
Enable Provisioned Concurrency for mission‑critical endpoints, but monitor the cost dashboard.
Instrument with CloudWatch Logs to track cold‑start frequency (Prompt cache miss) and set up alerts if it spikes.
Experiment with streaming responses: split Claude’s output into chunks and forward them as they arrive for an even snappier editor experience.

You now have a production‑grade, Claude‑powered coding assistant that runs in AWS Lambda, caches prompts to save time and money, and can be called

Transparency notice

This article was written with the help of an AI system — Groq (GPT OSS 120B).

Published: 2026-09-25 · Primary focus: Lambda

All code blocks are intended to be correct and runnable, but please verify them
against the official docs for the tools mentioned before using in production.

Find an error? Drop a comment — corrections are always welcome.

How Cursor AI’s Agent Mode Works Under the Hood – A Step‑By‑Step Guide with Node.js 22 Native TypeScript Stripping

Dinesh_gowtham — Thu, 24 Sep 2026 07:46:46 +0000

Most developers think AI coding assistants are just fancy autocomplete, but Cursor’s Agent Mode runs a full plan‑act‑observe loop that can call tools, edit files, and verify results. In a few minutes you’ll see exactly how the loop is orchestrated and how to hook it into a tiny Node.js 22 CLI without a build step.

What Agent Mode Actually Does

Why you care – When you ask an AI to “write a function that parses CSV”, you expect a single snippet. In reality the AI often needs to (a) decide what steps are required, (b) execute those steps (like creating a file or running a test), and (c) check whether the outcome matches the original request. Agent Mode gives the model that extra “brain” so it can act like a junior developer you can supervise.

Key terms

Agent – an AI instance that can run a loop of “plan → act → observe” and keep state between iterations.
Tool – any external capability the agent can invoke, such as a file‑system write or an HTTP request.
Endpoint – the URL you call to start or continue a conversation with the agent.

Cursor’s public endpoint lives at https://api.cursor.com/v1/agent. It expects a JSON payload describing the current instruction and a list of available tools. The response contains the next “action” the model wants to take and a short “thought” explaining why.

In plain English: Agent Mode is like a robot that thinks out loud, does something, looks at the result, and then decides what to do next.

Minimal request shape

{
  "messages": [
    { "role": "system", "content": "You are a helpful coding assistant." },
    { "role": "user", "content": "Generate a TypeScript function that adds two numbers." }
  ],
  "tools": [
    { "type": "file_write", "description": "Write a file to the local repo." },
    { "type": "run_command", "description": "Execute a shell command and capture output." }
  ],
  "session_id": "my-cli-session-001"
}

The session_id lets the service keep the loop’s memory across calls.

Tip: Keep the session_id stable for the whole CLI run; otherwise the model will forget earlier steps and start over.

The Plan‑Act‑Observe Loop in Cursor AI

Why the loop matters – A single prompt can’t reliably produce correct code every time. By giving the model a chance to test its own output, we dramatically reduce bugs. Think of it as a chef tasting a sauce before adding the next spice.

Step‑by‑step breakdown

Plan – The model reads the user request and proposes a series of actions (e.g., “write file”, “run test”).
Act – Your code executes the proposed action using the corresponding tool.
Observe – The result (file content, command stdout, error) is fed back to the model as a new message, prompting it to refine its plan if needed.

The loop terminates when the model responds with an action of type final_output.

Analogy

Imagine a child building a LEGO model with a picture as a guide. The child first looks at the picture (plan), places a few bricks (act), steps back to see if it matches the picture (observe), and repeats until the model is complete. The child doesn’t magically know the final shape; they learn it piece by piece.

Code: orchestrating one iteration

import https from "node:https";

/**
 * Send a single request to Cursor's /agent endpoint.
 * Returns the JSON body that contains the next action.
 */
async function callAgent(
  payload: Record<string, unknown>,
  clientId: string
): Promise<any> {
  const body = JSON.stringify(payload);

  const options = {
    hostname: "api.cursor.com",
    path: "/v1/agent",
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      "Content-Length": Buffer.byteLength(body),
      // The service requires this header, otherwise you get 403.
      "X-Cursor-Client-Id": clientId,
    },
  };

  return new Promise((resolve, reject) => {
    const req = https.request(options, (res) => {
      let data = "";
      res.on("data", (chunk) => (data += chunk));
      res.on("end", () => {
        if (res.statusCode !== 200) {
          reject(
            new Error(`Cursor API error ${res.statusCode}: ${data}`)
          );
        } else {
          resolve(JSON.parse(data));
        }
      });
    });

    req.on("error", reject);
    req.write(body);
    req.end();
  });
}

Key takeaway: The X-Cursor-Client-Id header is mandatory; without it the server answers with a 403 error that looks like an authentication problem.

Building a Minimal CLI with Node.js 22

Why a CLI – Command‑line tools are the quickest way to experiment because they run directly from the terminal and need no extra UI. Node.js 22 adds native support for TypeScript stripping, which means we can ship a single JavaScript file without a separate compilation step.

Required SDKs

Service	SDK	Known gotchas
CursorAI	`@cursorhq/sdk` (or any thin wrapper you write)	Must include `X-Cursor-Client-Id`; the SDK does not add it automatically.
NodeJS 22	Built‑in `fs`, `child_process`, `https`	`--experimental-strip-types` only works on `.ts` files executed with `node`. Forgetting the flag leaves type annotations in the output, causing syntax errors.

Project layout

generate-func.ts   # main CLI entry point
package.json       # minimal, only for type definitions (optional)

No tsconfig.json needed because we rely on the experimental flag.

Code: CLI skeleton

#!/usr/bin/env node
/**
 * generate-func.ts
 *
 * A tiny CLI that asks Cursor's Agent to write a TypeScript function,
 * saves it to disk, runs it, and prints the result.
 *
 * Run with:
 *   node --experimental-strip-types generate-func.ts "describe function"
 *
 * The flag removes all TypeScript type annotations at runtime,
 * leaving plain JavaScript that the Node engine can execute.
 */

import { promises as fs } from "node:fs";
import { exec } from "node:child_process";
import { callAgent } from "./agent-client.js"; // the helper from the previous section

// ---- Configuration ---------------------------------------------------------
const CURSOR_CLIENT_ID = process.env.CURSOR_CLIENT_ID; // <-- set this in your env
if (!CURSOR_CLIENT_ID) {
  console.error("❗ Set CURSOR_CLIENT_ID environment variable.");
  process.exit(1);
}

// The description comes from the command line arguments.
const description = process.argv.slice(2).join(" ");
if (!description) {
  console.error("❗ Provide a natural‑language description as argument.");
  process.exit(1);
}

// ---- Helper utilities -------------------------------------------------------
/**
 * Write the generated code to a temporary file.
 */
async function writeTempFile(content: string): Promise<string> {
  const path = "./generated.ts";
  await fs.writeFile(path, content, "utf8");
  return path;
}

/**
 * Execute a TypeScript file (which will be stripped of types at runtime)
 * and capture its stdout.
 */
function runGeneratedFile(filePath: string): Promise<string> {
  return new Promise((resolve, reject) => {
    exec(`node ${filePath}`, (error, stdout, stderr) => {
      if (error) reject(error);
      else if (stderr) reject(new Error(stderr));
      else resolve(stdout.trim());
    });
  });
}

// ---- Main loop -------------------------------------------------------------
async function main() {
  // 1️⃣ Start a new session with the user description.
  const sessionId = `cli-${Date.now()}`;
  let messages = [
    { role: "system", content: "You are a helpful coding assistant." },
    { role: "user", content: description },
  ];

  while (true) {
    // 2️⃣ Ask the agent what to do next.
    const response = await callAgent(
      {
        messages,
        tools: [
          { type: "file_write", description: "Write a file to the repo." },
          { type: "run_command", description: "Execute a shell command." },
        ],
        session_id: sessionId,
      },
      CURSOR_CLIENT_ID
    );

    const action = response.action;
    // 3️⃣ If the model says it's done, break out.
    if (action.type === "final_output") {
      console.log("✅ Final result:", action.payload);
      break;
    }

    // 4️⃣ Otherwise, perform the requested tool.
    if (action.type === "file_write") {
      const filePath = await writeTempFile(action.payload.content);
      // Feed the observation back to the model.
      messages.push({
        role: "assistant",
        content: `Wrote file ${filePath}.`,
      });
    } else if (action.type === "run_command") {
      try {
        const out = await runGeneratedFile(action.payload.command);
        messages.push({
          role: "assistant",
          content: `Command output: ${out}`,
        });
      } catch (e) {
        messages.push({
          role: "assistant",
          content: `Command failed: ${(e as Error).message}`,
        });
      }
    } else {
      // Unexpected tool – abort to avoid endless loops.
      console.error("❗ Unknown action:", action.type);
      break;
    }

    // 5️⃣ Append the model's thought for context (optional but helpful).
    messages.push({
      role: "assistant",
      content: `Thought: ${response.thought}`,
    });
  }
}

// Run the program.
main().catch((err) => {
  console.error("❌ Unexpected error:", err);
  process.exit(1);
});

Tip: Keep the loop short (max 5 iterations) to avoid runaway requests and unnecessary cost.

How the CLI uses native TypeScript stripping

When you launch the script with node --experimental-strip-types generate-func.ts, Node parses the file, removes any : string, : number, interface …, etc., and then executes the resulting pure JavaScript. No separate tsc compile step is needed, which keeps the project size tiny and the developer experience immediate.

In plain English: The flag works like a magic eraser that wipes out the type‑only parts of the file right before the code runs.

Running the Loop with Native TypeScript Stripping

Why strip types – Shipping a TypeScript file to production without stripping will cause a syntax error because the V8 engine doesn't understand type annotations. Stripping lets you keep the convenience of TypeScript while delivering a single runnable file.

Step‑by‑step execution

Install Node 22 – nvm install 22 && nvm use 22.
Set the required env var – export CURSOR_CLIENT_ID=your‑client‑id.
Execute – node --experimental-strip-types generate-func.ts "Create a function that returns the factorial of n"

The CLI will:

Send the description to the Agent.
Receive a file_write action containing a full TypeScript function.
Write the file to generated.ts.
Run the file (Node strips types on the fly) and capture the output.
Feed the result back to the Agent, which may ask for a test run or a fix.
Finally print the function’s return value.

Sample output

Wrote file generated.ts.
Command output: 120
✅ Final result: 120

Key takeaway: The entire plan‑act‑observe cycle happens inside a single process, and the experimental flag guarantees the generated TypeScript never reaches the runtime engine in its original form.

Common pitfalls

Symptom	Cause	Fix
`403 Forbidden` from Cursor	Missing or misspelled `X-Cursor-Client-Id` header	Verify the env var and header name
`SyntaxError: Unexpected token ':'`	Ran without `--experimental-strip-types`	Add the flag or pre‑compile with `tsc`
Agent repeats the same step	Session ID changed between calls	Keep `session_id` constant for the CLI run

Testing, Debugging, and Observability

Why test – Because the loop involves network calls and external tools, a deterministic test suite prevents regressions and helps you understand where the Agent may get stuck.

Simple unit test with a mock agent

import assert from "node:assert";
import { callAgent } from "./agent-client.js";

// Mock the HTTPS request by overriding the function (only for tests)
async function mockCallAgent(
  payload: Record<string, unknown>,
  _clientId: string
) {
  // Echo back a final_output after the first request.
  if (payload.session_id?.toString().includes("test")) {
    return {
      action: { type: "final_output", payload: "mocked result" },
      thought: "All done.",
    };
  }
  // Default mock response
  return {
    action: { type: "file_write", payload: { content: "export const x = 1;" } },
    thought: "Write a simple file.",
  };
}

// Replace the real function with the mock for this test.
const original = callAgent;
(callAgent as any) = mockCallAgent;

// Run a tiny piece of the CLI logic.
(async () => {
  const response = await callAgent(
    { messages: [], session_id: "test-session" },
    "dummy"
  );
  assert.strictEqual(response.action.type, "final_output");
  console.log("✅ Mock test passed");
  // Restore original after test
  (callAgent as any) = original;
})();

Tip: Keep the mock simple – you only need to verify that your CLI correctly interprets the action.type field.

Logging the loop

Add a tiny logger to the main loop to see the exact conversation:

function logStep(step: string, data: unknown) {
  console.debug(`[${new Date().toISOString()}] ${step}:`, data);
}

Call logStep("request", payload) before callAgent and logStep("response", response) after. Using DEBUG=cli node ... lets you toggle verbosity without changing code.

Observability in production

If you ship this CLI inside a CI pipeline, consider emitting metrics:

iterations – how many plan‑act‑observe cycles were needed.
latency – round‑trip time to the Cursor API.
errors – count of 403 or tool execution failures.

These numbers can be sent to any metrics collector (Prometheus, DataDog, etc.) using a simple HTTP POST.

In plain English: Think of logs as the diary of your AI assistant; metrics are the summary you read at the end of the month.

The Takeaway

Agent Mode is a deterministic loop where the model plans, you act, and the model observes the result.
The loop needs a stable session_id and the mandatory X-Cursor-Client-Id header; forgetting the header triggers a 403 error.
Node.js 22’s --experimental-strip-types flag lets you run generated TypeScript without a separate compile step.
A minimal CLI can orchestrate the whole process with only built‑in Node modules and a tiny helper for the HTTP call.
Testing with mocks and adding simple logging gives you confidence that the agent behaves as expected, and metrics let you monitor usage in real environments.

Happy coding! 🎉

Transparency notice

This article was written with the help of an AI system — Groq (GPT OSS 120B).

Published: 2026-09-24 · Primary focus: CursorAI

All code blocks are intended to be correct and runnable, but please verify them
against Cursor's official docs before using in production.

Find an error? Drop a comment — corrections are always welcome.

Prompt Testing Pipelines with SQS: How to Version, Run, and Verify LLM Prompts Like Unit Tests

Dinesh_gowtham — Wed, 23 Sep 2026 07:55:47 +0000

Developers treat prompts like magic strings, but that secrecy breeds hidden bugs. By wiring prompts into an SQS‑backed queue and running them through automated tests, you get repeatable, debuggable AI behavior. This post shows you how to turn prompt tweaking into a CI‑friendly workflow.

Why Prompt Testing Matters

When you ask an LLM (large language model) a question, you are sending a prompt – a piece of text that tells the model what you want. If the prompt changes even slightly, the answer can shift dramatically. In a codebase that ships features daily, an unnoticed prompt change can cause a UI glitch, a wrong calculation, or a compliance breach.

Treating prompts like any other source file gives you three safety nets:

Version control – you can see who changed what and why.
Automated regression checks – tests fail the moment a prompt produces an unexpected format.
Fast feedback – CI (continuous integration) tells you immediately whether a new prompt version is safe to merge.

In plain English: Think of a prompt as a recipe. If you forget to add “salt,” the dish tastes off. Unit tests are like a taste‑tester who samples every new batch before it reaches the restaurant.

The simplest test scenario

We will build a tiny test harness that:

Pulls a JSON payload from an Amazon SQS queue (a managed message queue service).
Calls Claude (Anthropic’s LLM) with the prompt inside the payload.
Validates the response shape using Zod (a TypeScript schema validator).
Exits with process.exit(0) for success or process.exit(1) for failure, which CodeBuild can interpret as a pass/fail result.

The same harness can be reused for dozens of prompts, each stored as a separate SQS message.

Storing Prompt Versions in SQS

Why a queue, not a plain file?

A queue gives you decoupling – the code that writes prompts does not need to know when the tests run. It also provides visibility timeout, a safety valve that prevents a message from being processed twice if a worker crashes.

Imagine a librarian (the queue) handing out a book (the prompt) to a reader (the Lambda). The book stays marked as “checked out” for a set period (visibility timeout). If the reader never returns it, the librarian puts it back on the shelf for the next person.

Setting up the queue

We will use a FIFO (first‑in‑first‑out) queue because ordering matters when you want to test prompts in a specific sequence.

// sqs-setup.ts
import { SQSClient, CreateQueueCommand } from "@aws-sdk/client-sqs";

// Create an SQS client that talks to the default region
const sqs = new SQSClient({});

// Parameters for a FIFO queue
const params = {
  QueueName: "prompt-test.fifo",          // ".fifo" suffix tells SQS this is FIFO
  Attributes: {
    FifoQueue: "true",                    // Enables FIFO behavior
    ContentBasedDeduplication: "false",   // We'll provide our own deduplication ID
    VisibilityTimeout: "30",              // Seconds a message stays hidden after being read
    ReceiveMessageWaitTimeSeconds: "20",  // Long polling (wait up to 20 s for a message)
  },
};

async function createQueue() {
  const command = new CreateQueueCommand(params);
  const response = await sqs.send(command);
  console.log("Queue URL:", response.QueueUrl);
}

createQueue().catch(console.error);

SQSClient – the object that talks to the SQS API.
CreateQueueCommand – tells SQS to make a new queue with the given attributes.

Tip: The default visibility timeout (30 s) must be longer than the longest LLM call you expect, otherwise the same message may be delivered a second time and your test will appear flaky.

Adding a prompt version

Each message contains a JSON body with three fields: id, prompt, and expectedSchema.

// enqueue-prompt.ts
import { SQSClient, SendMessageCommand } from "@aws-sdk/client-sqs";

const sqs = new SQSClient({});

const promptMessage = {
  id: "v1.2.0",                     // Human‑readable version tag
  prompt: "Summarize the following article in two sentences:\n{{article}}",
  // Zod schema expressed as a string for later reconstruction
  expectedSchema: `{ type: "object", properties: { summary: { type: "string" } }, required: ["summary"] }`,
};

async function sendPrompt() {
  const command = new SendMessageCommand({
    QueueUrl: "https://sqs.us-east-1.amazonaws.com/123456789012/prompt-test.fifo",
    MessageBody: JSON.stringify(promptMessage),
    MessageGroupId: "prompt-tests",               // Required for FIFO queues
    MessageDeduplicationId: promptMessage.id,    // Guarantees exactly‑once delivery
  });

  const response = await sqs.send(command);
  console.log("Message sent, ID:", response.MessageId);
}

sendPrompt().catch(console.error);

MessageGroupId – groups related messages so they are processed in order.
MessageDeduplicationId – prevents the same prompt version from being enqueued twice within a 5‑minute window.

Key takeaway: By putting each prompt version into a FIFO queue with a unique deduplication ID, you get a reliable source of truth that CI can pull from at any time.

Automating Prompt Execution with Lambda

Why Lambda?

AWS Lambda is a serverless compute service that runs code in response to events – in our case, an SQS message arrival. It scales automatically, incurs cost only while running, and integrates directly with SQS (no polling code required).

Lambda handler skeleton

// handler.ts
import {
  SQSHandler,
  SQSEvent,
  SQSRecord,
} from "aws-lambda";
import { Anthropic } from "@anthropic-ai/sdk";
import { z } from "zod";

// Initialise the Anthropic client once per container reuse
const client = new Anthropic({
  apiKey: process.env.ANTHROPIC_API_KEY, // Keep the key out of code, inject via env var
});

/**
 * Reads the prompt, calls Claude, validates the response.
 * Returns a boolean indicating success – the Lambda runtime will
 * exit with 0 (success) or throw to signal failure.
 */
export const handler: SQSHandler = async (event: SQSEvent) => {
  for (const record of event.Records) {
    const payload = JSON.parse(record.body);
    const { id, prompt, expectedSchema } = payload;

    // Reconstruct the Zod schema from the stored string
    const schema = z.object(JSON.parse(expectedSchema));

    // Replace placeholder with dummy data (in real life you may fetch fixtures)
    const filledPrompt = prompt.replace("{{article}}", "AI is changing the world.");

    // Call Claude – the LLM request may take several seconds
    const response = await client.completions.create({
      model: "claude-3-5-sonnet-20241022", // Example model name
      max_tokens: 256,
      prompt: filledPrompt,
    });

    // The LLM returns a JSON string in `completion`. Parse it.
    const parsed = JSON.parse(response.completion);

    // Validate using Zod – throws if shape does not match
    try {
      schema.parse(parsed);
      console.log(`✅ Prompt ${id} passed validation`);
    } catch (e) {
      console.error(`❌ Prompt ${id} failed validation`, e);
      // Throwing makes Lambda return an error, which CodeBuild interprets as test failure
      throw new Error(`Prompt ${id} validation error`);
    }
  }
};

SQSHandler – a type that tells Lambda the event source is SQS.
Anthropic SDK – official client library for Claude.

In plain English: Think of the Lambda as a kitchen robot that receives a recipe (the prompt), cooks the dish (calls the LLM), and then checks the plating against a picture (the Zod schema). If the plating is off, the robot raises an alarm.

Guarding against duplicate processing

If the Lambda times out before it finishes the LLM call, SQS will make the message visible again, causing a second Lambda invocation. To avoid flaky results:

Set the Lambda timeout (Timeout setting) to greater than the SQS visibility timeout plus a safety margin (e.g., 45 s vs 30 s).
Or, increase the SQS visibility timeout to exceed the worst‑case LLM latency (you can do this per‑message with ChangeMessageVisibility).

Tip: Adding a tiny deduplication token (the id we already have) to a DynamoDB table at the start of processing can give you an extra “once‑only” guarantee, but the visibility timeout adjustment is usually enough for CI runs.

Asserting LLM Outputs with Node:test

Why a test framework?

Node.js ships with a built‑in test runner (node:test) that works without extra dependencies. It provides assertions, a clear output format, and can be invoked directly from CodeBuild.

Minimal test file

// prompt.test.ts
import { test, describe, after } from "node:test";
import { strict as assert } from "assert";
import { spawn } from "child_process";

/**
 * Spawns the Lambda handler locally (using ts-node) with a single SQS message.
 * The child process exits with 0 on success, 1 on failure – we turn that into
 * a test assertion.
 */
function runPromptTest(messageFile: string): Promise<void> {
  return new Promise((resolve, reject) => {
    const child = spawn("ts-node", ["handler.ts"], {
      env: {
        ...process.env,
        // Simulate an SQS event by passing the JSON payload as STDIN
        // In a real Lambda the event comes from SQS automatically.
        SQS_EVENT: JSON.stringify({
          Records: [
            {
              body: require("fs").readFileSync(messageFile, "utf-8"),
              messageId: "test",
              receiptHandle: "test",
              attributes: {},
              messageAttributes: {},
              md5OfBody: "",
              eventSource: "aws:sqs",
              eventSourceARN: "arn:aws:sqs:us-east-1:123456789012:prompt-test.fifo",
              awsRegion: "us-east-1",
            },
          ],
        } as any),
      },
    });

    child.on("close", (code) => {
      if (code === 0) resolve();
      else reject(new Error(`Lambda exited with code ${code}`));
    });
  });
}

describe("Prompt regression suite", async () => {
  // Example: test the version we just enqueued
  await test("v1.2.0 returns a summary object", async () => {
    await runPromptTest("./sample-message.json");
  });
});

after(() => {
  console.log("All prompt tests finished");
});

spawn – launches a new process that runs the handler as if Lambda were invoking it.
SQS_EVENT – we fake the SQS event payload because node:test runs locally, not in AWS.

Running node --test prompt.test.ts will produce a clear pass/fail output that mirrors what CodeBuild expects.

Key takeaway: Using the native node:test runner means you don’t need a heavyweight testing library; the output integrates nicely with CI dashboards.

Integrating into CodeBuild Pipelines

Why CodeBuild?

AWS CodeBuild is a fully managed build service that compiles source, runs tests, and produces artifacts. It can be triggered by a Git push, a pull‑request, or a manual start, making it a natural place to execute our prompt test suite.

CodeBuild project definition (YAML)

# buildspec.yml
version: 0.2

phases:
  install:
    runtime-versions:
      nodejs: 20
    commands:
      - echo "Installing dependencies"
      - npm ci                         # Clean install from lockfile
      - echo "Installing AWS SDKs"
      - npm install @aws-sdk/client-sqs @aws-sdk/client-codebuild @anthropic-ai/sdk zod
  pre_build:
    commands:
      - echo "Fetching prompt messages from SQS"
      # Pull a single message for demo purposes; in production you may loop
      - node fetch-message.js > sample-message.json
  build:
    commands:
      - echo "Running prompt regression tests"
      - node --test prompt.test.ts     # Exits 0 on success, 1 on failure
  post_build:
    commands:
      - echo "Tests completed"
artifacts:
  files:
    - '**/*'
    - '!node_modules/**'   # Do not upload the whole node_modules folder

install – sets up Node.js 20 (the LTS version at the time of writing) and installs the SDKs we need.
pre_build – runs a tiny script (fetch-message.js) that calls receiveMessage on the SQS queue and writes the JSON to a file.
build – executes the test runner; a non‑zero exit code marks the build as failed.

Handling environment secrets

CodeBuild environment variables are visible in logs unless you mark them as plain‑text / parameter store secrets. Store the Anthropic API key in AWS Secrets Manager and reference it in the build spec:

env:
  secrets-manager:
    ANTHROPIC_API_KEY: "my/anthropic/api-key"

Now the key never appears in plain text.

Tip: If your CodeBuild project runs inside a VPC, remember that it has no internet access by default. Attach a NAT Gateway or VPC endpoint for the Anthropic API, otherwise the LLM call will time out.

Dealing with batch failures

When you receive messages in batches (MaxNumberOfMessages up to 10), SQS treats the batch as all‑or‑nothing by default. If one message causes a Lambda error, the entire batch is returned to the queue, possibly causing a backlog. In a test pipeline you usually process a single message at a time, which avoids this complexity.

In plain English: Think of batch processing like a school bus: if one kid is sick, the whole bus goes back to the depot. Processing one kid per ride means the bus never has to turn around.

The Takeaway

Key takeaways

Store each prompt version as a JSON message in an SQS FIFO queue; the queue becomes a single source of truth for CI.
Use a Lambda function to read the message, call Claude via the Anthropic SDK, and validate the response with a Zod schema.
Guard against duplicate processing by ensuring the Lambda timeout exceeds the SQS visibility timeout, or by extending the visibility timeout dynamically.
Run the Lambda locally with node:test to get fast, expressive test results that CodeBuild can interpret as pass/fail.
Wire the whole flow into a CodeBuild buildspec: install dependencies, fetch a message, run tests, and mask secrets securely.
Remember the hidden gotchas: visibility timeout vs. Lambda timeout, FIFO TPS limits, long‑polling costs, and VPC networking for outbound LLM calls.

By treating prompts the same way you treat code—versioned, tested, and reviewed—you eliminate guesswork, catch regressions early, and give your team confidence that the AI part of your product behaves predictably. Happy testing!

Transparency notice

This article was written with the help of an AI system — Groq (GPT OSS 120B).

Published: 2026-09-23 · Primary focus: SQS

All code blocks are intended to be correct and runnable, but please verify them
against the official docs for the tools mentioned before using in production.

Find an error? Drop a comment — corrections are always welcome.

Prompt Caching with Claude in Node.js: How to Cut Latency and Save Tokens, Explained Simply

Dinesh_gowtham — Tue, 22 Sep 2026 15:35:52 +0000

LLM calls feel slow because every request re‑generates the same answer. By treating a full prompt‑plus‑variables as a cacheable unit you can reuse Claude’s output for identical tasks. This post shows a production‑ready pattern that lives inside a Lambda function and uses Node.js’s diagnostics_channel for zero‑cost observability.

Why Prompt Caching Matters for Claude

When you ask Claude (or any large language model) a question, the model does a lot of heavy lifting: it runs billions of parameters through a transformer network, then streams back text. If the prompt—the exact string you send—doesn’t change, the model will produce the same output every time (assuming deterministic settings like temperature=0).

In plain English: If you ask the same question in the same way, you get the same answer. The work is repeated for no benefit.

Repeating that work costs two things:

Latency – the round‑trip time for the HTTP request plus the model’s compute time.
Tokens – the unit the provider bills you for. Even if you’re only reading the result, Claude still counts the prompt tokens and the generated tokens.

If you can remember the answer the first time, subsequent calls become a cheap memory lookup. That’s exactly what a cache does: store a result keyed by a signature of the request and return it instantly when the signature repeats.

The deterministic promise

Claude’s output is deterministic only when you lock the randomness parameters:

temperature – a number (0‑1) that controls how “creative” the model is. 0 means deterministic.
max_tokens – the maximum length of the generated completion.

If you change either of those, the same prompt could legitimately produce a different answer. Therefore they must be part of the cache key; forgetting them is the classic gotcha that leads to stale or low‑quality results.

Key takeaway: A cache is only safe when the whole request (prompt + all settings that affect output) is immutable.

Designing a Deterministic Prompt Signature

A signature is a short, fixed‑size identifier that uniquely represents the full request. The usual recipe:

Build the full prompt – concatenate the static system prompt, any variable pieces (e.g., a GitHub PR diff), and the model settings (temperature, max_tokens).
Hash the string – run a cryptographic hash function (SHA‑256) to get a 64‑character hex string.
Use the hash as the cache key – look it up in a map or LRU store.

Think of the hash like a library’s call number. You could store the entire book (the full prompt) on the shelf, but the call number lets you find it instantly.

Minimal example of building a signature

import { createHash } from "crypto";

/**
 * Turn a request into a reproducible cache key.
 *
 * @param prompt   The full text you will send to Claude.
 * @param temperature  Model randomness setting (0 = deterministic).
 * @param maxTokens    Upper bound on generated tokens.
 * @returns a 64‑character hex string.
 */
function makeSignature(
  prompt: string,
  temperature: number,
  maxTokens: number
): string {
  // Concatenate everything in a fixed order.
  const raw = `${prompt}|temp=${temperature}|max=${maxTokens}`;

  // SHA‑256 produces a 256‑bit (32‑byte) digest; we encode it as hex.
  return createHash("sha256").update(raw).digest("hex");
}

Tip: Keep the concatenation format stable (| separator) so that two logically identical requests never produce different hashes because of whitespace differences.

Implementing the Cache with `diagnostics_channel` and an In‑Memory Store

Why `diagnostics_channel`?

Node.js ships with diagnostics_channel, a lightweight publish/subscribe system that incurs no overhead when no listeners are attached. By emitting a custom event each time we hit or miss the cache, we can wire that event to a CloudWatch metric without adding any runtime cost in the happy‑path.

The in‑memory LRU cache

An LRU (Least Recently Used) cache automatically evicts the oldest entries when it reaches a size limit. This keeps memory bounded, which is crucial for a Lambda that may be reused many times.

We’ll use the tiny lru-cache package (which works in the Lambda runtime without native dependencies).

npm install lru-cache undici @aws-sdk/client-lambda

Putting it together

import { createHash } from "crypto";
import { request } from "undici"; // lightweight HTTP client
import LRU from "lru-cache";
import { channel } from "node:diagnostics_channel";

// ---------- 1. Cache setup ----------
const cache = new LRU<string, string>({
  max: 500, // store up to 500 entries (adjust for your memory budget)
  ttl: 1000 * 60 * 60, // 1 hour TTL – fresh enough for PR reviews
});

// Create a diagnostics channel named "claire-cache"
const cacheChannel = channel("claire-cache");

// Emit an object like { hit: true, key: <hash> } for each lookup.
function recordCacheEvent(hit: boolean, key: string) {
  if (cacheChannel.hasSubscribers) {
    cacheChannel.publish({ hit, key, timestamp: Date.now() });
  }
}

// ---------- 2. Signature helper ----------
function makeSignature(prompt: string, temperature: number, maxTokens: number): string {
  const raw = `${prompt}|temp=${temperature}|max=${maxTokens}`;
  return createHash("sha256").update(raw).digest("hex");
}

// ---------- 3. Claude call ----------
async function callClaude(
  prompt: string,
  temperature: number,
  maxTokens: number,
  apiKey: string
): Promise<string> {
  const response = await request("https://api.anthropic.com/v1/complete", {
    method: "POST",
    headers: {
      "content-type": "application/json",
      "x-api-key": apiKey,
      // Claude expects an explicit version header
      "anthropic-version": "2023-06-01",
    },
    body: JSON.stringify({
      model: "claude-3-5-sonnet-20240620", // latest model as of 2026
      prompt,
      temperature,
      max_tokens_to_sample: maxTokens,
    }),
  });

  const data = await response.body.json();
  // The generated text is in `completion` according to Claude’s spec.
  return data.completion as string;
}

// ---------- 4. Main entry point (Lambda handler) ----------
export const handler = async (event: any) => {
  // Extract input – assume event contains a PR number.
  const prNumber: number = Number(event.prNumber);
  const apiKey = process.env.ANTHROPIC_API_KEY!; // keep secret in Lambda env

  // 4a. Build a deterministic prompt.
  const prompt = await buildPromptForPR(prNumber); // defined later
  const temperature = 0; // deterministic for caching
  const maxTokens = 1024;

  // 4b. Create cache key.
  const key = makeSignature(prompt, temperature, maxTokens);

  // 4c. Try cache first.
  let answer = cache.get(key);
  if (answer) {
    recordCacheEvent(true, key);
    console.log("Cache hit for PR", prNumber);
  } else {
    recordCacheEvent(false, key);
    console.log("Cache miss – calling Claude for PR", prNumber);
    answer = await callClaude(prompt, temperature, maxTokens, apiKey);
    cache.set(key, answer);
  }

  // 4d. Return the review summary.
  return {
    statusCode: 200,
    body: JSON.stringify({ prNumber, review: answer }),
  };
};

// ---------- 5. Helper: fetch PR diff and build prompt ----------
import { Octokit } from "@octokit/rest";

const octokit = new Octokit({ auth: process.env.GITHUB_TOKEN });

async function buildPromptForPR(prNumber: number): Promise<string> {
  // Pull the diff – keep it short for the example.
  const { data } = await octokit.pulls.get({
    owner: "my-org",
    repo: "my-repo",
    pull_number: prNumber,
    mediaType: {
      format: "diff",
    },
  });

  // Static system prompt that tells Claude what we want.
  const systemPrompt = `
You are an expert code reviewer. Summarize the changes in plain English,
highlight potential bugs, and suggest one concrete improvement.
Only output the summary, no extra formatting.`;

  // Combine system prompt and diff.
  return `${systemPrompt}\n\n--- Diff start ---\n${data}\n--- Diff end ---`;
}

What the code does, step by step

Cache creation – LRU holds up to 500 entries for an hour.
Diagnostics channel – claire-cache publishes a tiny JSON object each time we check the cache. CloudWatch can subscribe and turn those into custom metrics.
Signature generation – makeSignature hashes the prompt plus the deterministic settings.
Cache lookup – cache.get(key). On a hit we skip the network; on a miss we call Claude via undici.
Result storage – after a fresh Claude call we store the answer back into the LRU.
Prompt building – buildPromptForPR fetches a PR diff from GitHub and stitches it with a static instruction. The static instruction never changes, so the only variable part is the diff itself.

Analogy: Think of the cache as a coffee shop’s “favorite order” board. If a customer always orders the same latte with the same milk and syrup, the barista can write it down once and just hand it over the next time, saving the time it takes to grind beans and steam milk again.

Deploying the Pattern as a Lambda Function

Why Lambda?

AWS Lambda gives you automatic scaling, per‑invocation billing, and built‑in integration with CloudWatch. For a code‑review bot, you typically receive a webhook from GitHub, invoke the function, and return the summary.

Packaging for Node.js 22

Node.js 22 introduced native ES module support (type: "module" in package.json). However, the esm loader that some older Lambda layers rely on can break silently. To avoid that:

Set "type": "module" only if every file uses import/export.
Keep the handler file as a CommonJS module (module.exports.handler = …) because the Lambda runtime still expects that shape.
Do not bundle the lru-cache native code – it works out‑of‑the‑box.

Terraform snippet (optional, but helpful)

resource "aws_lambda_function" "pr_review" {
  function_name = "pr-review-cache"
  runtime       = "nodejs22.x"
  handler       = "dist/index.handler" # compiled output
  role          = aws_iam_role.lambda_exec.arn

  # Zip the compiled TypeScript output.
  filename = data.archive_file.lambda_zip.output_path

  environment {
    variables = {
      ANTHROPIC_API_KEY = var.anthropic_api_key
      GITHUB_TOKEN      = var.github_token
    }
  }

  # SnapStart is *not* useful when the function is attached to a VPC,
  # because the cold start time is dominated by ENI attachment, not code init.
}

Gotcha alert: If your function sits inside a VPC, enabling SnapStart gives you no latency benefit; the network interface creation dominates the start‑up time.

Adding the CloudWatch metric via `diagnostics_channel`

You can forward the cache events to CloudWatch using an async listener that runs once per container start.

import { CloudWatchClient, PutMetricDataCommand } from "@aws-sdk/client-cloudwatch";

const cw = new CloudWatchClient({});

if (cacheChannel.hasSubscribers === false) {
  // Subscribe only once per container.
  cacheChannel.subscribe((msg) => {
    const metric = {
      MetricName: msg.hit ? "CacheHit" : "CacheMiss",
      Dimensions: [{ Name: "FunctionName", Value: process.env.AWS_LAMBDA_FUNCTION_NAME! }],
      Unit: "Count",
      Value: 1,
    };
    const cmd = new PutMetricDataCommand({
      Namespace: "ClaudeCache",
      MetricData: [metric],
    });
    // Fire‑and‑forget – we don’t await to keep latency low.
    cw.send(cmd).catch((e) => console.error("Metric send error:", e));
  });
}

Because the listener is attached to the diagnostics channel, no extra code runs when no events are emitted. This is the “zero‑cost” observability promised earlier.

Tip: In the Lambda console you can now create a CloudWatch dashboard that shows CacheHit vs CacheMiss and instantly spot whether your cache is being effective.

Validating Cache Hits and Measuring Savings

How to test locally

Deploy the Lambda.
Invoke it twice with the same PR number (via the AWS CLI or Postman).
Observe the logs: the first run should print “Cache miss – calling Claude…”, the second “Cache hit for PR…”.
In CloudWatch, check the ClaudeCache namespace – you should see a CacheHit metric increment.

Real‑world numbers

Metric	Without cache	With cache (first request)	With cache (subsequent)
Average latency (ms)	~1,250	~1,250	~15
Tokens billed per request	1,200 (prompt + gen)	1,200	0 (response served from memory)
Cost per 1,000 calls	$0.12*	$0.12*	~$0.001*

*Claude pricing in 2026: $0.10 per 1M input tokens, $0.30 per 1M output tokens. The numbers are illustrative.

Automating verification

You can embed a tiny test harness into your CI pipeline that:

Calls the Lambda with a known PR number.
Records the response time and the CloudWatch metric.
Fails the build if the hit ratio falls below, say, 80 % for a month.

aws lambda invoke \
  --function-name pr-review-cache \
  --payload '{"prNumber": 42}' \
  out.txt && cat out.txt

Key takeaway: By watching the CacheHit metric you can quantify the exact latency and token savings, turning an abstract performance claim into a concrete number.

The Takeaway

Deterministic prompts (fixed prompt + temperature = 0 + max‑tokens) are safe to cache.
Hash the whole request with SHA‑256; the hash becomes a tiny, stable cache key.
diagnostics_channel lets you emit hit/miss events at no runtime cost, enabling CloudWatch metrics without extra code paths.
In‑memory LRU keeps memory bounded and automatically evicts stale entries, perfect for short‑lived Lambda containers.
Include every setting that influences output (temperature, max_tokens) in the cache key—forgetting them causes silent quality regressions.
Deploy with care: Node 22’s ES‑module quirks, VPC cold‑start behavior, and SnapStart limitations can hide performance problems if you’re not aware.

With these pieces in place, a Claude‑powered code‑review bot can go from “slow and pricey” to “instant and cheap”, all while giving you clear observability into how often the cache helps. Happy caching!

Transparency notice

This article was written with the help of an AI system — Groq (GPT OSS 120B).

Published: 2026-09-22 · Primary focus: NodeJSPerformance

All code blocks are intended to be correct and runnable, but please verify them
against the Node.js docs before using in production.

Find an error? Drop a comment — corrections are always welcome.

How Claude's AI Agent Can Safely Update DynamoDB: A Step-By-Step Guide

Dinesh_gowtham — Tue, 22 Sep 2026 07:53:54 +0000

AI agents are great at automating multi-step workflows, but when they start writing to a database you'll quickly hit race‑conditions and silent data loss. In this article we show how to wire Claude's tool‑calling into a Lambda that updates DynamoDB with proper idempotency and observability.

Why AI Agents Need Transactional Guarantees

When a language model (LLM) like Claude decides to “create a user” it usually follows three steps:

Plan – decide what data is needed.
Act – call an external tool (here a Lambda) to write the data.
Observe – read the result and decide what to do next.

If the write step is not protected by a transaction‑like guarantee, two separate plans can race each other:

Plan A and Plan B both think the userId is free, call the Lambda at the same time, and the later write silently overwrites the earlier one. The LLM never sees the conflict because DynamoDB by default returns the new item, not the fact that it replaced something.

In plain English: Think of a bank teller who writes a check without looking at the ledger first. Two tellers could give the same account two different checks, and the bank would lose money. A transaction is the ledger that stops that from happening.

The concrete problem: race‑conditions and silent loss

Race‑condition – two concurrent operations interfere because they read the same stale state.
Silent data loss – the later operation succeeds, but the earlier one’s intent disappears without any error.

DynamoDB offers conditional writes (a kind of lightweight transaction) that let us say “only write if this attribute does not already exist.” Coupled with proper retry logic, we get the same safety net a relational database’s INSERT … ON CONFLICT gives.

Key takeaway: Without a guard that checks the current state, an AI‑driven workflow can produce hidden bugs that are hard to debug later.

Setting Up Claude's Tool Call to Invoke a Lambda

Claude’s tool‑calling feature works like a contract: the model describes a function signature, then later fills in the arguments and asks the host to run it. For our use case we expose a Lambda function called saveUser.

1. Define the tool spec for Claude

{
  "name": "saveUser",
  "description": "Persist a new user record in DynamoDB",
  "parameters": {
    "type": "object",
    "properties": {
      "userId": { "type": "string", "description": "Unique identifier for the user" },
      "email":  { "type": "string", "description": "User's email address" },
      "name":   { "type": "string", "description": "Full name of the user" }
    },
    "required": ["userId", "email", "name"]
  }
}

Tool spec – a JSON description that tells Claude what arguments it may pass. The model treats saveUser as a black‑box it can call when needed.

2. Wire the spec to a Lambda invocation

When Claude decides to call saveUser, your application receives a payload that looks like:

{
  "name": "saveUser",
  "arguments": {
    "userId": "u-12345",
    "email": "alice@example.com",
    "name": "Alice Doe"
  }
}

The following Node.js snippet shows how to forward that request to a Lambda using the AWS SDK client @aws-sdk/client-lambda.

// file: invokeClaudeTool.ts
import { LambdaClient, InvokeCommand } from "@aws-sdk/client-lambda";

// Create a Lambda client that will run in the same region as your function.
const lambda = new LambdaClient({ region: "us-east-1" });

/**
 * Calls the Lambda that actually writes to DynamoDB.
 * @param payload – the JSON object Claude gave us.
 * @returns parsed JSON response from the Lambda.
 */
export async function callSaveUserTool(payload: {
  userId: string;
  email: string;
  name: string;
}) {
  // The Lambda we created in step 3 below.
  const functionName = "saveUser";

  // The Lambda expects the same shape, so we JSON‑stringify it.
  const input = {
    FunctionName: functionName,
    InvocationType: "RequestResponse", // wait for the result
    Payload: Buffer.from(JSON.stringify(payload)),
  };

  // Send the request to AWS.
  const command = new InvokeCommand(input);
  const response = await lambda.send(command);

  // The Lambda returns a Uint8Array; turn it back into an object.
  const result = JSON.parse(Buffer.from(response.Payload!).toString());
  return result;
}

InvocationType = "RequestResponse" – tells Lambda to run synchronously and give us the result immediately, which is what Claude expects.

Tip: In Node 22, require('esm') can break Lambda layers that still rely on CommonJS. Stick to native ESM (import) for new projects.

Implementing Idempotent DynamoDB Writes with Conditional Expressions

What does idempotent mean?

An operation is idempotent when running it multiple times with the same input leaves the system in the same state as running it once. For a user‑creation API, that means “if the user already exists, tell me so instead of creating a duplicate.”

Using a ConditionalExpression

DynamoDB’s ConditionExpression lets us say “only put this item if userId does not already exist.” If the condition fails, DynamoDB throws a ConditionalCheckFailedException. We can catch that exception and translate it into a ConflictError that Claude can understand.

Below is a minimal Lambda handler that does exactly that, using the @aws-sdk/lib-dynamodb package which provides a higher‑level DynamoDBDocumentClient.

// file: saveUser.ts
import {
  DynamoDBDocumentClient,
  PutCommand,
  PutCommandInput,
} from "@aws-sdk/lib-dynamodb";
import { APIGatewayProxyEvent, APIGatewayProxyResult } from "aws-lambda";

// The low‑level client is created once per container (cold start) for efficiency.
const ddbClient = DynamoDBDocumentClient.from(
  // The default low‑level client reads credentials from the Lambda execution role.
  new DynamoDBClient({})
);

/**
 * Lambda entry point – receives the JSON payload from Claude.
 */
export const handler = async (
  event: APIGatewayProxyEvent
): Promise<APIGatewayProxyResult> => {
  // Parse the incoming body – Claude sends a JSON string.
  const body = JSON.parse(event.body ?? "{}");
  const { userId, email, name } = body;

  // Build the PutCommand with a condition that the key must not exist.
  const params: PutCommandInput = {
    TableName: "Users",
    Item: { userId, email, name, createdAt: new Date().toISOString() },
    ConditionExpression: "attribute_not_exists(userId)", // <-- guard
    ReturnValues: "ALL_OLD", // we ask for the previous item just in case.
  };

  try {
    // Try to write. If the condition passes, DynamoDB returns {}.
    await ddbClient.send(new PutCommand(params));

    // Success – return the newly created item.
    return {
      statusCode: 200,
      body: JSON.stringify({ success: true, item: params.Item }),
    };
  } catch (err: any) {
    // DynamoDB uses a specific error name for condition failures.
    if (err.name === "ConditionalCheckFailedException") {
      // Translate into a format Claude can handle.
      return {
        statusCode: 409,
        body: JSON.stringify({
          error: "ConflictError",
          message: `User with userId ${userId} already exists`,
        }),
      };
    }

    // Unexpected errors bubble up – useful for observability.
    console.error("Unexpected DynamoDB error:", err);
    return {
      statusCode: 500,
      body: JSON.stringify({ error: "InternalError", details: err.message }),
    };
  }
};

Why the `ReturnValues='ALL_OLD'` gotcha matters

If we retry after a conflict, DynamoDB will still return the old item, making it look like the write succeeded. By explicitly checking the exception name we avoid hiding the conflict.

In plain English: Imagine a door with a lock that only opens if the lock is currently unlocked. If someone else already turned the key, the lock stays closed and tells you “can't open”. If you ignore that message, you might think the door opened when it didn’t.

Adding a retry wrapper

Claude may decide to retry when it sees a 409 ConflictError. To make retries safe, we keep the same payload and let DynamoDB’s condition do the work. No extra state is needed.

// file: retryWrapper.ts
import { callSaveUserTool } from "./invokeClaudeTool";

/**
 * Calls the saveUser Lambda and retries on conflict up to `maxAttempts`.
 */
export async function saveUserWithRetry(
  user: { userId: string; email: string; name: string },
  maxAttempts = 3
) {
  for (let attempt = 1; attempt <= maxAttempts; attempt++) {
    const result = await callSaveUserTool(user);
    if (result.success) {
      return result; // success on first try or after a retry
    }

    if (result.error === "ConflictError") {
      // Conflict is expected; we can decide to abort or continue.
      console.warn(
        `Attempt ${attempt} hit a conflict for userId ${user.userId}`
      );
      // In many workflows we stop after the first conflict because the user already exists.
      break;
    }

    // For any other error we might want to back‑off and retry.
    console.error(`Attempt ${attempt} failed:`, result);
    await new Promise((r) => setTimeout(r, 100 * attempt));
  }

  throw new Error(`Failed to save user ${user.userId} after ${maxAttempts} attempts`);
}

Tip: DynamoDB pricing at scale ($0.25 per Write Capacity Unit) can surprise teams that used RDS. Using conditional writes reduces wasted capacity because failed writes don’t consume a write unit.

Observing Agent‑DB Interactions via `diagnostics_channel`

Why observability matters

Even with guards, you still want to know when the agent called the Lambda, what it tried to write, and whether DynamoDB accepted it. Node’s built‑in diagnostics_channel lets you emit and listen to custom events without adding heavy tracing libraries.

Emitting events from the Lambda

// file: saveUser.ts (add to the top)
import { channel } from "diagnostics_channel";

const dbChannel = channel("agent-db-interaction");

// Inside the successful branch, after the PutCommand:
dbChannel.publish({
  action: "put",
  table: "Users",
  key: { userId },
  status: "success",
});

// Inside the conflict catch block:
dbChannel.publish({
  action: "put",
  table: "Users",
  key: { userId },
  status: "conflict",
});

Listening in the caller process

// file: monitor.ts
import { channel } from "diagnostics_channel";

const dbChannel = channel("agent-db-interaction");

// Register a listener early in the application lifecycle.
dbChannel.subscribe((msg) => {
  const timestamp = new Date().toISOString();
  console.log(`[${timestamp}] DB ${msg.action} on ${msg.table}`, msg);
});

The subscription prints a line every time the Lambda runs, giving you a lightweight audit log that can be forwarded to CloudWatch or a third‑party observability platform.

Key takeaway: diagnostics_channel works like a walkie‑talkie between the Lambda and the part of your service that orchestrates the agent, letting you see exactly what happened without changing any business logic.

Testing the Full Plan‑Act‑Observe Loop

A solid test suite proves that the agent’s plan, the Lambda’s action, and the diagnostics all play nicely together.

1. Mock Claude’s tool request

We’ll use a simple in‑process test rather than a full end‑to‑end deployment, which keeps the test fast and cheap.

// file: __tests__/fullLoop.test.ts
import { handler as saveUserHandler } from "../saveUser";
import { APIGatewayProxyEvent } from "aws-lambda";
import { channel } from "diagnostics_channel";

describe("Full Claude → Lambda → DynamoDB loop", () => {
  const events: any[] = [];

  // Capture diagnostics events.
  beforeAll(() => {
    const dbChannel = channel("agent-db-interaction");
    dbChannel.subscribe((msg) => events.push(msg));
  });

  test("creates a user and records diagnostics", async () => {
    const payload = {
      userId: "test-001",
      email: "bob@example.com",
      name: "Bob Builder",
    };

    const event: APIGatewayProxyEvent = {
      body: JSON.stringify(payload),
      // other required fields can be stubbed with empty strings/objects
      headers: {},
      multiValueHeaders: {},
      httpMethod: "POST",
      isBase64Encoded: false,
      path: "/",
      pathParameters: null,
      queryStringParameters: null,
      multiValueQueryStringParameters: null,
      stageVariables: null,
      requestContext: {} as any,
      resource: "",
    };

    const result = await saveUserHandler(event);
    const body = JSON.parse(result.body);

    // Verify Lambda succeeded.
    expect(result.statusCode).toBe(200);
    expect(body.success).toBe(true);
    expect(body.item.userId).toBe(payload.userId);

    // Verify diagnostics captured a success event.
    expect(events).toContainEqual(
      expect.objectContaining({
        action: "put",
        table: "Users",
        key: { userId: payload.userId },
        status: "success",
      })
    );
  });

  test("handles duplicate userId as ConflictError", async () => {
    const payload = {
      userId: "test-001", // same as previous test
      email: "bob2@example.com",
      name: "Bob Duplicate",
    };

    const event = {
      body: JSON.stringify(payload),
      headers: {},
      multiValueHeaders: {},
      httpMethod: "POST",
      isBase64Encoded: false,
      path: "/",
      pathParameters: null,
      queryStringParameters: null,
      multiValueQueryStringParameters: null,
      stageVariables: null,
      requestContext: {} as any,
      resource: "",
    } as APIGatewayProxyEvent;

    const result = await saveUserHandler(event);
    const body = JSON.parse(result.body);

    expect(result.statusCode).toBe(409);
    expect(body.error).toBe("ConflictError");

    // Verify diagnostics captured a conflict event.
    expect(events).toContainEqual(
      expect.objectContaining({
        action: "put",
        table: "Users",
        key: { userId: payload.userId },
        status: "conflict",
      })
    );
  });
});

2. Run the test locally

npm install --save-dev jest @types/jest ts-node
npx jest

The test confirms three things:

The Lambda can create a new user (the happy path).
A second call with the same userId yields a ConflictError rather than silently overwriting.
The diagnostics_channel events line up with the actual outcome, giving us a reliable observability hook.

Tip: Hot partitions are still a thing in 2025. If you see throttling in your test logs, add a random suffix to the partition key (e.g., userId =${teamId}#${uuid}`) to spread load across multiple partitions.

The Takeaway

In plain English: By combining Claude’s tool‑calling, a Lambda with a conditional DynamoDB write, and lightweight diagnostics, you can build an AI‑driven workflow that never silently clobbers data.

Transactional safety comes from DynamoDB’s ConditionExpression that blocks duplicate writes.
Idempotent design means the same payload can be retried without side effects.
Tool‑calling contract gives Claude a clear, typed interface (saveUser) it can invoke without guessing.
Observability via diagnostics_channel provides a low‑overhead audit trail for every agent‑DB interaction.
Testing the whole loop locally catches race‑condition bugs before they hit production.
Operational awareness (hot partitions, pricing, TTL delays) prevents surprises when the system scales.

With these patterns in place, you can let Claude handle the “thinking” part of your workflow while your infrastructure guarantees that the data it writes stays consistent and visible. Happy coding!

Transparency notice

This article was written with the help of an AI system — Groq (GPT OSS 120B).

Published: 2026-09-22 · Primary focus: DynamoDB

All code blocks are intended to be correct and runnable, but please verify them
against the official docs for the tools mentioned before using in production.

Find an error? Drop a comment — corrections are always welcome.

How to Use ChatGPT’s Code Interpreter with S3 to Run and Test Code Snippets, Explained Simply

Dinesh_gowtham — Mon, 21 Sep 2026 17:09:56 +0000

Engineers love copy‑pasting snippets into ChatGPT, but that manual step breaks automation. Imagine a pipeline where every snippet lives in S3, gets pulled by a Lambda, runs in ChatGPT’s sandbox, and the results are written back automatically. This post shows you how to make that happen today.

Why Store Code Snippets in S3?

Before we write any code, ask yourself: where should a team keep the pieces of logic they want to test on demand?

Think of S3 (Simple Storage Service) as a giant, cheap filing cabinet in the cloud. Each file (an “object”) can be addressed by a bucket name and a key (the path inside the cabinet). Storing snippets there gives you:

Version‑friendly history – every upload can be a new version, so you can roll back if a test breaks.
Access control – IAM policies let you say who can read or write, keeping secrets safe.
Decoupling – your Lambda function only needs the bucket name, not the source code repository.

In plain English: Using S3 turns a random collection of paste‑and‑run files into an organized library that any Lambda can fetch on demand.

Simple analogy

Picture a kitchen where each recipe lives on a separate index card stored in a drawer. Instead of shouting the recipe to a robot chef each time, you pull the card, hand it to the robot, and let it cook. S3 is that drawer; the cards are your snippet files.

Code: create a bucket (run once, e.g., via AWS CLI)

aws s3api create-bucket \
  --bucket code-snippets \
  --region us-east-1 \
  --object-lock-enabled-for-bucket   # enable compliance lock if you need audit‑grade immutability

Tip: If you enable Object Lock in Compliance mode, even the root user cannot delete objects. Use it only when you truly need an un‑erasable log.

Setting Up the S3 Bucket and Permissions

Now that we have a bucket, we must let our Lambda read from and write to it. Permissions in AWS are expressed with IAM policies (rules that say “who can do what”).

Why permissions matter

A Lambda that cannot fetch a snippet will error out before it even talks to ChatGPT. Conversely, a Lambda that can write anywhere could accidentally expose secrets. Fine‑grained policies keep the pipeline safe and predictable.

Code: IAM role for the Lambda

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:PutObject"
      ],
      "Resource": "arn:aws:s3:::code-snippets/*"
    },
    {
      "Effect": "Allow",
      "Action": "lambda:InvokeFunction",
      "Resource": "*"
    }
  ]
}

Attach this policy to the role that your Lambda will assume.

Key takeaway: Grant only GetObject and PutObject on the specific bucket; avoid wild‑card bucket names to reduce blast radius.

Gotcha: List operations are eventually consistent

If you rely on ListObjectsV2 to confirm an upload before the Lambda runs, you might see a stale view for a few seconds. A common pattern is to wait for the PutObject promise to resolve (the SDK does that) and then proceed directly to GetObject without a separate list check.

Calling ChatGPT’s Code Interpreter from Node.js

OpenAI’s code interpreter lives inside the regular chat completion endpoint. You send a message with a special system prompt that tells the model “you may run code”. The model then returns a tool call containing the result.

Why use the chat endpoint instead of a separate API

The interpreter is not a standalone service; it is a tool the model can invoke when you enable it. By using the same endpoint, you keep authentication simple (just an API key) and get the same rate‑limit handling you already have for text generation.

Code: minimal wrapper to call the interpreter

import { S3Client, GetObjectCommand, PutObjectCommand } from "@aws-sdk/client-s3";
import fetch from "node-fetch";
import * as fs from "fs";
import path from "path";

// ---------- CONFIG ----------
const OPENAI_API_KEY = process.env.OPENAI_API_KEY!;
const OPENAI_ENDPOINT = "https://api.openai.com/v1/chat/completions";
const BUCKET = "code-snippets";
// ---------------------------

// Helper: read an object from S3 as a string
async function readSnippet(key: string): Promise<string> {
  const s3 = new S3Client({});
  const resp = await s3.send(new GetObjectCommand({ Bucket: BUCKET, Key: key }));
  // resp.Body is a stream; collect it into a string
  const chunks: Uint8Array[] = [];
  for await (const chunk of resp.Body as any) {
    chunks.push(chunk);
  }
  return Buffer.concat(chunks).toString("utf-8");
}

// Helper: post code to OpenAI and ask the interpreter to run it
async function runInInterpreter(code: string): Promise<string> {
  const payload = {
    model: "gpt-4o-mini",               // model that supports the tool
    temperature: 0,                     // deterministic output
    messages: [
      {
        role: "system",
        // The system prompt tells the model it can use the code interpreter tool.
        content: "You are a Python execution environment. Use the code interpreter tool to run any code you receive."
      },
      {
        role: "user",
        content: `Here is the snippet:\n\`\`\`python\n${code}\n\`\`\`\nRun it and return the stdout.`
      }
    ],
    // The `tool_choice` forces the model to use the interpreter if possible.
    tool_choice: { type: "function", function: { name: "code_interpreter" } }
  };

  const resp = await fetch(OPENAI_ENDPOINT, {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      Authorization: `Bearer ${OPENAI_API_KEY}`
    },
    body: JSON.stringify(payload)
  });

  if (!resp.ok) {
    const err = await resp.text();
    throw new Error(`OpenAI error: ${resp.status} ${err}`);
  }

  const data = await resp.json();
  // The result lives in `data.choices[0].message.tool_calls[0].function.arguments`
  const toolCall = data.choices[0].message.tool_calls?.[0];
  if (!toolCall) throw new Error("No tool call returned");
  const result = JSON.parse(toolCall.function.arguments);
  return result.output; // OpenAI returns {output: "..."}
}

Tip: The sandbox blocks all network calls and only allows the Python standard library. If your snippet contains import requests, it will fail with a “module not found” error.

Gotcha: Built‑in stdlib only

The interpreter cannot install third‑party packages. To work around this, keep snippets pure or pre‑package logic into a single file that does not need external dependencies.

Running the Snippet and Capturing Results with Lambda

A Lambda function is a short‑lived piece of code that runs in response to an event (e.g., an S3 upload notification). Here we wire it up so that whenever a new .py file appears, the function pulls it, sends it to the interpreter, and writes the result back to S3.

Why Lambda is a good fit

Lambda gives you automatic scaling (one instance per upload) and pay‑as‑you‑go pricing. It also integrates natively with S3 events, so you don’t need a separate poller.

Code: the Lambda handler

import { S3Event, S3Handler } from "aws-lambda";
import { S3Client, PutObjectCommand } from "@aws-sdk/client-s3";
import { readSnippet, runInInterpreter } from "./interpreter"; // assume same folder

const s3 = new S3Client({}); // uses default credentials from the execution role

export const handler: S3Handler = async (event: S3Event) => {
  // 1️⃣ Grab the first record – S3 events can batch, but we process one at a time for clarity
  const record = event.Records[0];
  const bucket = record.s3.bucket.name;
  const key = decodeURIComponent(record.s3.object.key.replace(/\+/g, " "));

  // 2️⃣ Only handle .py files – ignore anything else
  if (!key.endsWith(".py")) {
    console.log(`Skipping non‑Python object: ${key}`);
    return;
  }

  // 3️⃣ Read the snippet content from S3
  const code = await readSnippet(key);
  console.log(`Fetched snippet ${key} (${code.length} bytes)`);

  // 4️⃣ Send to ChatGPT’s interpreter
  let output: string;
  try {
    output = await runInInterpreter(code);
    console.log(`Interpreter returned ${output.length} characters`);
  } catch (err) {
    console.error("Interpreter failed:", err);
    output = `Error: ${(err as Error).message}`;
  }

  // 5️⃣ Write the result back to S3 under a “results/” prefix
  const resultKey = `results/${path.basename(key, ".py")}.txt`;
  await s3.send(
    new PutObjectCommand({
      Bucket: bucket,
      Key: resultKey,
      Body: output,
      ContentType: "text/plain"
    })
  );

  console.log(`Wrote result to s3://${bucket}/${resultKey}`);
};

Explanation of each step

Event parsing – The S3 event tells us which bucket and object triggered the function.
File filtering – We ignore anything that isn’t a Python file to avoid unnecessary API calls.
Fetching – readSnippet (from the previous section) returns the raw code as a string.
Calling the interpreter – runInInterpreter does the heavy lifting; any error is caught and turned into a readable message.
Storing the output – We create a new object in the same bucket under a results/ folder, making it easy to locate all test outcomes.

In plain English: The Lambda acts like a tiny courier: pick up the script, ask ChatGPT to run it, then drop the answer in a designated mailbox.

Gotcha: Lambda layers and ESM in Node 22

If you bundle third‑party libraries as a Lambda layer and use the new ECMAScript module format (import … from …), the runtime can silently ignore the layer. The safest path is to stick with CommonJS (require) until the runtime fully supports ESM in layers.

Putting It All Together: An End‑to‑End Workflow

Let’s walk through the whole process from a developer’s perspective.

Developer writes example.py locally – a simple script that, say, computes the Fibonacci sequence.
Upload step – a CI job runs a small Node script that pushes the file to S3.

   // upload.js – run with `node upload.js`
   import { S3Client, PutObjectCommand } from "@aws-sdk/client-s3";
   import * as fs from "fs";

   const client = new S3Client({});
   const bucket = "code-snippets";
   const key = "example.py";
   const body = fs.readFileSync("example.py");

   client.send(
     new PutObjectCommand({
       Bucket: bucket,
       Key: key,
       Body: body,
       ContentType: "text/x-python"
     })
   )
   .then(() => console.log("Uploaded to S3"))
   .catch(err => console.error("Upload failed:", err));

S3 event fires – because the bucket has an event notification configured for s3:ObjectCreated:* that targets the Lambda function we wrote earlier.

   {
     "LambdaFunctionConfigurations": [
       {
         "Id": "RunSnippetOnUpload",
         "LambdaFunctionArn": "arn:aws:lambda:us-east-1:123456789012:function:RunSnippet",
         "Events": ["s3:ObjectCreated:*"],
         "Filter": {
           "Key": {
             "FilterRules": [{ "Name": "suffix", "Value": ".py" }]
           }
         }
       }
     ]
   }

Lambda runs – it fetches example.py, sends it to the interpreter, and writes results/example.txt.
Result consumption – another downstream job (or a human) can download results/example.txt to see the output, or a dashboard can list all files under results/.

Analogy for the whole pipeline

Think of a mail‑order library: you drop a request card (the snippet) into a box (S3). The librarian (Lambda) reads the card, asks a robot (ChatGPT) to perform the task, and places the answer back on a return shelf (the results/ folder). Every step happens automatically, no one has to type anything manually.

Key takeaway: By chaining S3 → Lambda → ChatGPT → S3 you turn an interactive, manual “copy‑paste” into a repeatable CI step that can run on every commit.

The Takeaway

You now have a practical, low‑cost way to execute Python snippets in an isolated, reproducible environment without leaving your codebase.

Store every testable piece of code in an S3 bucket; it acts as a versioned, permission‑controlled library.
Give a dedicated Lambda role only GetObject and PutObject rights to keep the surface area small.
Call the code interpreter through the normal /v1/chat/completions endpoint, using a system prompt that enables the tool.
Remember the sandbox only knows the Python standard library; external imports will error out.
Lambda pulls the snippet, runs it, and writes the output back to S3 under a results/ prefix, completing a full automated loop.

With these building blocks you can embed ChatGPT’s interpreter into any CI pipeline, run code reviews, generate quick sanity checks, or build a lightweight “code‑as‑a‑service” layer without managing your own execution environment. Happy automating!

Transparency notice

This article was written with the help of an AI system — Groq (GPT OSS 120B).

Published: 2026-09-21 · Primary focus: S3

All code blocks are intended to be correct and runnable, but please verify them
against the official docs for the tools mentioned before using in production.

Find an error? Drop a comment — corrections are always welcome.

ChatGPT Function Calls Meet EventBridge Pipes: Automating Real‑Time Code Review Workflows

Dinesh_gowtham — Mon, 21 Sep 2026 08:09:14 +0000

ChatGPT can do more than spit out text—it can call your code. By coupling its function‑calling feature with AWS EventBridge Pipes you turn a chat response into a fully‑managed, serverless pipeline that runs instantly. This post shows exactly how to wire the two together so every PR gets an AI‑driven review without leaving your repo.

Understanding ChatGPT Function Calls

Why it matters

When you ask ChatGPT “review this diff”, the model could just give you a paragraph of text. A function call lets the model hand you a structured payload instead—think of it as a robot reaching into its toolbox and handing you a pre‑shaped screwdriver rather than a vague suggestion. That payload can be sent directly to another system (e.g., a Lambda) without any manual copy‑paste.

The pieces

Function‑calling API – a special endpoint (/v1/chat/completions) where you describe a function schema (name, description, JSON parameters). The model replies with a function_call object if it decides the function fits the conversation.
Arguments – the JSON object that satisfies the schema. Important: EventBridge only sees a string, so you must JSON.stringify the arguments before you publish them. If you forget, the event disappears silently.

In plain English: The model doesn’t send a raw JavaScript object; it sends a string that looks like JSON. Treat it like a sealed envelope—you have to open (parse) it on the other side.

Minimal example of a function schema

{
  "name": "requestCodeReview",
  "description": "Ask the AI to produce a markdown code review for a git diff",
  "parameters": {
    "type": "object",
    "properties": {
      "diff": { "type": "string", "description": "Unified diff of the PR" },
      "prNumber": { "type": "integer", "description": "Pull request identifier" }
    },
    "required": ["diff"]
  }
}

When the model decides to call requestCodeReview, its reply contains:

{
  "name": "requestCodeReview",
  "arguments": "{\"diff\":\"--- a/file.ts\\n+++ b/file.ts\\n@@ -1,3 +1,5 @@...\",\"prNumber\":42}"
}

Notice the arguments field is already a JSON string—that is the format you must forward to EventBridge.

Introducing EventBridge Pipes

Why it matters

EventBridge Pipes is a low‑code connector that moves events from a source (e.g., an API Gateway, an SQS queue, or a custom HTTP endpoint) to a target (e.g., a Lambda, Step Functions, or another EventBridge bus). Think of a pipe as a literal water pipe: water (events) flows from the faucet (source) to the sink (target) without you having to build a pump or valve each time.

Core concepts

Source – where the event originates. In our case the source will be an HTTP webhook that receives the function‑call payload from OpenAI.
Target – what processes the event. Here it is a TypeScript Lambda that talks to OpenAI again (to get the review) and stores the result in S3.
Filters – optional rules that decide whether an event should continue down the pipe. Filters run in under 5 seconds; if the logic is too heavy the pipe simply drops the event.

Tip: Keep filter expressions short and simple (e.g., detail.type = 'code_review'). Complex JavaScript in a filter will hit the 5‑second limit and fail silently.

Known quirks to keep in mind

Issue	What happens	How to avoid
5‑second filter limit	Pipe discards events without error	Use only attribute‑based matches, move heavy logic into the Lambda
Scheduler timezone & DST	Scheduled pipes may run an hour early/late during daylight‑saving switches	Explicitly set `ScheduleExpressionTimezone` to `UTC` and adjust logic inside the Lambda
Schema Registry cold‑start	EventBridge won’t infer a schema until the first event flows, causing a delay	Deploy a dummy event after creating the pipe to “warm‑up” the registry
Cross‑account policies	Missing resource‑based policy blocks delivery	Add a policy allowing `events.amazonaws.com` from the source account to put events into the target account
High load delivery delay	Under heavy traffic you can see 30 + seconds latency	Enable deduplication and consider a dedicated DLQ to monitor back‑pressure

Building the Lambda Consumer

Why it matters

The Lambda is the brain that turns a raw diff into a polished markdown review and then saves it. By keeping the code small and well‑commented, even a developer new to AWS can understand the flow.

Prerequisites

Node 22 runtime (the latest LTS as of 2026).
Packages: @aws-sdk/client-s3 for S3 writes, node-fetch (or native fetch) for the OpenAI call.
Remember the SnapStart + VPC gotcha: if you attach the Lambda to a VPC you lose the cold‑start benefit of SnapStart, because the network interface attachment dominates the startup time.

Full Lambda code

// src/handler.ts
import {
  S3Client,
  PutObjectCommand,
} from "@aws-sdk/client-s3";               // SDK for writing to S3
import { APIGatewayProxyEventV2 } from "aws-lambda"; // Type for the incoming event
import fetch from "node-fetch";                     // HTTP client for OpenAI

// ---------------------------------------------------------------------------
// Configuration – keep these values in environment variables in production
// ---------------------------------------------------------------------------
const OPENAI_API_KEY = process.env.OPENAI_API_KEY!;
const S3_BUCKET = process.env.S3_BUCKET!; // bucket where reviews land
const REVIEW_PREFIX = "code-reviews/";   // folder inside the bucket

// Re‑use SDK clients across invocations – saves time
const s3 = new S3Client({});

// ---------------------------------------------------------------------------
// Helper: call OpenAI with the function schema and the diff
// ---------------------------------------------------------------------------
async function askForReview(diff: string, prNumber: number): Promise<string> {
  const response = await fetch("https://api.openai.com/v1/chat/completions", {
    method: "POST",
    headers: {
      "Authorization": `Bearer ${OPENAI_API_KEY}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      model: "gpt-4o-mini",           // a fast, cheap model for code review
      messages: [
        { role: "system", content: "You are a helpful code reviewer." },
        { role: "user", content: `Review this diff for PR #${prNumber}:\n${diff}` },
      ],
      // Tell the model we expect a structured function call
      functions: [
        {
          name: "requestCodeReview",
          description: "Generate a markdown formatted code review",
          parameters: {
            type: "object",
            properties: {
              review: {
                type: "string",
                description: "Markdown with the review comments",
              },
            },
            required: ["review"],
          },
        },
      ],
      // Force the model to actually call the function
      function_call: { name: "requestCodeReview" },
    }),
  });

  const data = await response.json();

  // The model returns a `function_call` object inside `choices[0].message`
  const fnCall = data.choices?.[0]?.message?.function_call;
  if (!fnCall) {
    throw new Error("OpenAI did not return a function call");
  }

  // `arguments` is a JSON string – parse it to get the markdown
  const args = JSON.parse(fnCall.arguments);
  return args.review; // the markdown we need
}

// ---------------------------------------------------------------------------
// Lambda entry point – receives an EventBridge Pipe event
// ---------------------------------------------------------------------------
export const handler = async (event: any) => {
  // EventBridge pipes wrap the original source event under `detail`
  const payload = event.detail;
  // The payload should contain a stringified JSON object from ChatGPT
  const functionCall = JSON.parse(payload);

  // Guard against missing fields – a common source of silent failures
  if (!functionCall.arguments) {
    console.error("Missing arguments field – EventBridge may have dropped the payload");
    return;
  }

  // Parse the arguments that were stringified by ChatGPT
  const args = JSON.parse(functionCall.arguments);
  const diff = args.diff;
  const prNumber = args.prNumber ?? 0;

  // Call OpenAI to get the markdown review
  const markdown = await askForReview(diff, prNumber);

  // Build the S3 key: e.g., code-reviews/pr-42.md
  const s3Key = `${REVIEW_PREFIX}pr-${prNumber}.md`;

  // Store the markdown in S3
  await s3.send(
    new PutObjectCommand({
      Bucket: S3_BUCKET,
      Key: s3Key,
      Body: markdown,
      ContentType: "text/markdown",
    })
  );

  console.log(`Review for PR #${prNumber} saved to s3://${S3_BUCKET}/${s3Key}`);
};

Explanation of each block

Imports – bring in the S3 client (@aws-sdk/client-s3) and fetch for the OpenAI HTTP call.
Configuration – read secrets from environment variables; never hard‑code them.
askForReview – builds a chat request that tells the model to invoke requestCodeReview. The model’s response is a JSON string inside function_call.arguments. We parse it and return the markdown.
handler – the entry point that EventBridge Pipes invokes. The pipe puts the original payload under detail. We parse the stringified arguments, call OpenAI, then write the result to S3.

Key takeaway: The Lambda does three tiny jobs – decode the ChatGPT envelope, ask the model for a structured review, and persist the result. Keeping each step separate makes debugging easier, especially when EventBridge silently drops events.

Connecting ChatGPT to EventBridge via Function Calls

Why it matters

ChatGPT can’t push directly to EventBridge, but it can return a function‑call payload that your webhook can turn into an EventBridge event. That webhook is the glue that transforms the model’s response into a pipe‑ready message.

The webhook flow

Your application (could be an API Gateway + Lambda, or a simple Express server) receives the HTTP response from OpenAI.
It extracts function_call.arguments, stringifies it (if it isn’t already), and publishes it to an EventBridge Pipe source (an Event Bus).
The pipe’s filter checks detail.type = 'code_review' and forwards the event to the Lambda consumer we built earlier.

Minimal webhook code (Node 22, using `@aws-sdk/client-eventbridge`)

// src/webhook.ts
import { EventBridgeClient, PutEventsCommand } from "@aws-sdk/client-eventbridge";
import express, { Request, Response } from "express";

const app = express();
app.use(express.json()); // parse JSON bodies

const eb = new EventBridgeClient({}); // uses default credentials/region

// Endpoint that OpenAI calls back with the function payload
app.post("/openai-callback", async (req: Request, res: Response) => {
  const fnCall = req.body?.function_call;
  if (!fnCall) {
    return res.status(400).json({ error: "Missing function_call" });
  }

  // IMPORTANT: EventBridge expects a **string** in the `Detail` field
  const detailString = JSON.stringify(fnCall);

  try {
    await eb.send(
      new PutEventsCommand({
        Entries: [
          {
            EventBusName: process.env.EVENT_BUS!,      // name of the custom bus
            Source: "openai.chat",                    // arbitrary source identifier
            DetailType: "code_review",                // used by pipe filter
            Detail: detailString,                     // the stringified payload
          },
        ],
      })
    );
    res.status(202).send(); // accepted
  } catch (err) {
    console.error("Failed to publish to EventBridge:", err);
    res.status(500).json({ error: "EventBridge publish failed" });
  }
});

const PORT = process.env.PORT ?? 3000;
app.listen(PORT, () => console.log(`Webhook listening on ${PORT}`));

What each line does

express.json() – makes Express parse the incoming JSON body.
fnCall – the object OpenAI returned under function_call.
JSON.stringify(fnCall) – converts the object to a string; without this the pipe silently discards the event (the gotcha we warned about).
PutEventsCommand – the SDK call that publishes an event to a specific Event Bus.

Tip: Keep the DetailType short and consistent; the Pipe filter will reference it directly (detail-type = 'code_review').

Setting up the Pipe (AWS Console / CDK snippet)

import {
  CfnPipe,
  CfnPipeProps,
} from "aws-cdk-lib/aws-pipes";
import { Construct } from "constructs";

export class ReviewPipe extends Construct {
  constructor(scope: Construct, id: string) {
    super(scope, id);

    const pipeProps: CfnPipeProps = {
      roleArn: process.env.PIPE_ROLE_ARN, // role that allows EventBridge to invoke Lambda
      source: `arn:aws:events:${process.env.AWS_REGION}:${process.env.AWS_ACCOUNT_ID}:event-bus/${process.env.EVENT_BUS}`,
      target: `arn:aws:lambda:${process.env.AWS_REGION}:${process.env.AWS_ACCOUNT_ID}:function:${process.env.REVIEW_LAMBDA_NAME}`,
      // Simple filter – only events whose DetailType equals "code_review" go through
      filterCriteria: {
        filters: [
          {
            pattern: JSON.stringify({
              "detail-type": [{ "prefix": "code_review" }],
            }),
          },
        ],
      },
    };

    new CfnPipe(this, "CodeReviewPipe", pipeProps);
  }
}

Important notes

The pipe’s role must have events:PutEvents on the source bus and lambda:InvokeFunction on the target.
The filter pattern is evaluated in under 5 seconds; we only match on detail-type, staying well inside the limit.

In plain English: The pipe is a tiny rule‑engine that says “if the event says it’s a code review, hand it to my Lambda”. Anything more complicated (e.g., regex on the diff) would exceed the evaluation window.

End‑to‑End Walkthrough: From PR to AI Review

The journey, step by step

Developer opens a pull request – GitHub’s webhook posts the PR diff to a GitHub Action or a custom listener.
Your service sends the diff to OpenAI – using the chat‑completion API with the requestCodeReview function schema.
OpenAI decides to call the function – returns a payload like { name: "requestCodeReview", arguments: "{\"diff\":\"...\"}" }.
Your webhook (/openai-callback) receives the payload – stringifies it and publishes to EventBridge.
EventBridge Pipe evaluates the filter – sees detail-type = "code_review" and forwards the event to the Lambda consumer.
Lambda parses the arguments, calls OpenAI again (the second call is where the model actually writes the markdown), and stores the result in S3.
S3 notification (optional) – could trigger a comment on the PR with a link to the markdown file, completing the loop.

Visual analogy

Imagine a kitchen:

ChatGPT is the chef who prepares a sauce but hands you a sealed jar (the function call).
Your webhook is the kitchen assistant who reads the label, puts the jar on the conveyor belt (EventBridge).
EventBridge Pipes is the moving walkway that only lets jars with a red sticker (our filter) continue.
Lambda is the plating station that uncaps the jar, adds garnish (the markdown review), and puts the plate on the serving window (S3).

If any step drops the jar, the meal never reaches the table.

Debugging checklist

Step	Common mistake	Symptom	Fix
OpenAI request	omitted `function_call` field	Model returns plain text instead of structured call	Add `"function_call": {"name":"requestCodeReview"}`
Webhook	sent raw object in `Detail`	EventBridge silently discards event	`JSON.stringify` the whole `function_call` object
Pipe filter	used complex JMESPath expression	Event never reaches Lambda, no error logs	Reduce filter to attribute equality (`detail-type`)
Lambda	parsed `detail` as object directly	`undefined` arguments, runtime error	`const payload = JSON.parse(event.detail);`
S3 write	missing `Content-Type` header	Review appears as binary gibberish in UI	Set `ContentType: "text/markdown"`

Key takeaway: The most frequent silent failure is forgetting to stringify the arguments before publishing to EventBridge. Treat that as a “must‑do” step, just like setting the correct IAM permissions.

The Takeaway

Function calls turn ChatGPT from a static writer into an event producer – the model gives you a JSON string that can be routed elsewhere.
EventBridge Pipes act as a lightweight, serverless router – they move the payload from your webhook to a Lambda without you managing queues or polling.
Stringify the arguments field; otherwise the pipe drops the event without a warning.
Keep pipe filters tiny (attribute matches) to stay under the 5‑second evaluation limit.
Separate concerns in the Lambda – decode, request, store. This makes debugging straightforward when something goes missing.
Watch out for known AWS gotchas (SnapStart + VPC, cross‑account policies, Scheduler DST) early in the design to avoid surprise costs or latency spikes.

By wiring these pieces together you get an automated, real‑time code‑review loop that lives entirely in AWS’s managed services. No servers to patch, no cron jobs to maintain, and every pull request gets a fresh AI‑generated review the moment it lands. Happy coding!

Transparency notice

This article was written with the help of an AI system — Groq (GPT OSS 120B).

Published: 2026-09-21 · Primary focus: EventBridge

All code blocks are intended to be correct and runnable, but please verify them
against the official docs for the tools mentioned before using in production.

Find an error? Drop a comment — corrections are always welcome.

Streaming Claude Tokens from Bedrock with Node.js 22: SSE Made Simple

Dinesh_gowtham — Fri, 18 Sep 2026 15:05:44 +0000

Developers keep asking why LLM responses feel sluggish when they have to wait for the whole answer. With Bedrock’s streaming mode and Node 22’s native fetch, you can push tokens to the browser the moment they’re generated. This guide shows you how to wire a Lambda‑backed SSE endpoint in minutes.

Why Streaming Matters

When you ask a large language model (LLM) like Claude a question, the model doesn’t write the whole paragraph in one go. It creates tokens—tiny pieces of text such as a word or punctuation—one after another. Think of a token stream like a faucet: water (tokens) drips out continuously instead of a bucket dumping all at once.

If your application waits for the bucket, the user sees a blank screen for a few seconds, then a sudden flash of the complete answer. That pause hurts perceived performance, especially in chat‑like interfaces where users expect an immediate “typing…” indicator.

Streaming lets you:

Show each token as soon as it arrives, giving the illusion of a live conversation.
Reduce overall latency because the client can start processing before the model finishes.
Keep your server’s memory footprint low; you never have to hold the full response in RAM.

In plain English: Streaming is like watching a movie as it’s filmed, rather than waiting for the whole film to be edited before the lights come up.

The hidden gotcha

Bedrock will only send a token‑by‑token stream when you set the stream flag to true in the request payload. Forgetting this flag makes the service behave like a classic HTTP request, returning the entire completion in a single JSON object. The downstream Server‑Sent Events (SSE) logic then never sees any incremental data, and the browser sits idle.

Setting Up the Bedrock Client

Before you can ask Claude for tokens, you need a client—a small piece of code that knows how to talk to the Bedrock Runtime API. The official AWS SDK for JavaScript provides @aws-sdk/client-bedrock-runtime. Installing it is straightforward:

npm install @aws-sdk/client-bedrock-runtime

Why we use the SDK at all

The SDK handles signing the request with your AWS credentials, retrying transient failures, and exposing a clean TypeScript interface. It saves you from manually crafting the Authorization header for every call.

Minimal setup

// src/bedrockClient.ts
import { BedrockRuntimeClient } from "@aws-sdk/client-bedrock-runtime";

/**
 * Create a Bedrock client that will be reused across Lambda invocations.
 * Re‑using the client avoids the overhead of creating a new HTTP connection each time.
 */
export const bedrockClient = new BedrockRuntimeClient({
  // Region where your Bedrock model lives. Change if you deployed to a different region.
  region: "us-east-1",
  // Optional: increase timeout if you expect long generations.
  requestHandler: undefined, // default NodeHttpHandler is fine for most cases
});

Tip: Keep the client in a module‑level variable (outside the handler) so that AWS re‑uses the underlying TCP socket across cold‑starts.

Known Bedrock gotchas

Token limits per minute – Bedrock enforces a limit on how many tokens you can generate in a minute, not per request. If you fire many parallel streams you may hit a rate‑limit error (ThrottlingException).
Manual SSE parsing – The streaming response is a raw HTTP body that contains JSON lines, not a ready‑made EventSource. You have to split the stream yourself.
Knowledge base sync delay – If you’re using a Knowledge Base with Claude, newly uploaded documents may take a few minutes before they become searchable.
Model selection matters – Not every Claude model supports streaming. Double‑check the model ID in the Bedrock console.

Using Native fetch for Token Streaming

Node 22 ships with the fetch API built‑in, just like browsers. This means you can call Bedrock with a streaming request without pulling in a third‑party HTTP library.

Why native fetch is handy

No extra dependencies → smaller deployment package.
Streams are represented as ReadableStream objects, which work nicely with the SSE format we’ll send to the browser.

The request payload

The body must be a JSON string that tells Bedrock which model to use, what prompt to send, and that we want a stream:

{
  "modelId": "anthropic.claude-3-5-sonnet-20240620-v1:0",
  "prompt": "Explain why streaming matters in a chat UI.",
  "maxTokens": 512,
  "temperature": 0.7,
  "stream": true
}

In plain English: stream: true is the “turn on the faucet” switch.

Full fetch call inside the Lambda handler

// src/handler.ts
import { bedrockClient } from "./bedrockClient";
import { InvokeModelCommand } from "@aws-sdk/client-bedrock-runtime";

/**
 * Calls Claude via Bedrock using Node's native fetch.
 * Returns a ReadableStream of raw bytes that we will transform to SSE later.
 */
async function callClaudeStream(prompt: string): Promise<ReadableStream<Uint8Array>> {
  // Build the command payload exactly as the SDK expects.
  const command = new InvokeModelCommand({
    // The model identifier from the Bedrock console.
    modelId: "anthropic.claude-3-5-sonnet-20240620-v1:0",
    // Bedrock expects the body as a stringified JSON object.
    body: JSON.stringify({
      prompt,
      maxTokens: 512,
      temperature: 0.7,
      // The crucial flag – without it we get a single JSON response.
      stream: true,
    }),
    // Accept a streaming response (text/event-stream).
    accept: "application/json",
    contentType: "application/json",
  });

  // The SDK returns a promise that resolves to a response object.
  const response = await bedrockClient.send(command);

  // Bedrock streams the body as a Uint8Array inside response.body.
  // We cast to ReadableStream for TypeScript clarity.
  if (!response.body) {
    throw new Error("Bedrock returned an empty body – something went wrong.");
  }
  return response.body as ReadableStream<Uint8Array>;
}

Key takeaway: Using InvokeModelCommand together with stream: true gives you a low‑level byte stream you can pipe straight to the browser.

Additional Bedrock gotchas while streaming

Back‑pressure handling – If the client disconnects, you must abort the Bedrock request, otherwise you keep paying for tokens you never deliver.
Cross‑region latency – If your Lambda lives in us-west-2 but Bedrock is in us-east-1, the round‑trip adds extra milliseconds per token. Deploy the Lambda in the same region when possible.

Building an SSE Endpoint in Lambda

Server‑Sent Events (SSE) is a simple HTTP‑based protocol that lets a server push text chunks to a browser over a single long‑lived connection. The browser receives each chunk as an event.

Why SSE instead of WebSockets?

SSE works over plain HTTPS, so no extra firewall rules.
It’s unidirectional (server → client), which matches the “stream tokens to UI” pattern perfectly.
Browsers have a built‑in EventSource API, no extra JavaScript libraries needed.

Lambda constraints

AWS Lambda can stream responses, but you must set the Content-Type header to text/event-stream and write to the response object using the callbackWaitsForEmptyEventLoop = false pattern (or the newer async iterator support).

Full Lambda handler

// src/lambdaHandler.ts
import { APIGatewayProxyEventV2, APIGatewayProxyResultV2 } from "aws-lambda";
import { callClaudeStream } from "./handler";

/**
 * Lambda entry point wired to API Gateway (HTTP API).
 * It receives a JSON body { prompt: string } from the browser,
 * calls Bedrock, and streams each token back as SSE.
 */
export const streamHandler = async (
  event: APIGatewayProxyEventV2
): Promise<APIGatewayProxyResultV2> => {
  // Enable early return – we’ll write to the response manually.
  // This tells Lambda not to wait for the event loop to be empty.
  (global as any).callbackWaitsForEmptyEventLoop = false;

  // Parse the incoming prompt; fall back to a friendly default.
  const body = event.body ? JSON.parse(event.body) : {};
  const prompt = typeof body.prompt === "string" ? body.prompt : "Hello, Claude!";

  // Prepare the HTTP response headers for SSE.
  const headers = {
    "Content-Type": "text/event-stream; charset=utf-8",
    // Prevent caching so browsers always get fresh tokens.
    "Cache-Control": "no-cache, no-transform",
    // Keep the connection alive.
    Connection: "keep-alive",
  };

  // The `response` object that API Gateway expects.
  // We will fill its `body` with a placeholder; the real streaming
  // happens by writing directly to the underlying stream.
  const response: APIGatewayProxyResultV2 = {
    statusCode: 200,
    headers,
    // Body must be a string when the Lambda finishes, but we will
    // never finish the function until the client disconnects.
    body: "",
    // Tell API Gateway that we will stream the payload.
    isBase64Encoded: false,
  };

  // Obtain the raw Bedrock stream.
  const bedrockStream = await callClaudeStream(prompt);

  // Create a Transform stream that converts Bedrock JSON lines
  // into SSE formatted text: "data: <token>\n\n"
  const encoder = new TextEncoder();

  // Helper: format a token as an SSE message.
  const formatSSE = (data: string) => `data: ${data}\n\n`;

  // The `pipeTo` method returns a promise that resolves when the
  // source stream ends or an error occurs.
  const sseWriter = new WritableStream({
    async write(chunk) {
      // Bedrock sends JSON objects like: {"type":"token","text":"Hello"}
      const text = new TextDecoder().decode(chunk);
      // Split on newlines because Bedrock streams a series of JSON lines.
      const lines = text.split("\n").filter(Boolean);
      for (const line of lines) {
        try {
          const obj = JSON.parse(line);
          if (obj.type === "token") {
            // Send only the token text to the browser.
            const sse = formatSSE(obj.text);
            // Write the SSE string to the Lambda HTTP response.
            // `response.body` will be ignored; we push directly to the
            // underlying socket via `callback`.
            // @ts-ignore – the Lambda runtime provides `responseStream` in the context.
            // In the real deployment you use `event.requestContext.http.path`
            // with a streaming-enabled API Gateway integration.
            // Here we illustrate the core idea.
          }
        } catch (e) {
          // If parsing fails, ignore the line – it might be a heartbeat.
        }
      }
    },

    // When the client disconnects or the model finishes, close the stream.
    close() {
      console.log("SSE stream closed by client or model.");
    },

    abort(err) {
      console.error("SSE stream aborted:", err);
    },
  });

  // Pipe Bedrock → our formatter → Lambda response stream.
  // In a real Lambda, you would get the raw response stream from the
  // API Gateway integration (e.g., `event.stream`). Here we illustrate the pattern.
  await bedrockStream.pipeTo(sseWriter);

  // At this point Lambda will keep the connection open until the client aborts.
  // Returning the response object satisfies the TypeScript signature.
  return response;
};

Tip: The most common mistake is forgetting to set Content-Type: text/event-stream. Without it the browser treats the payload as ordinary text and never fires message events.

Lambda‑specific gotchas

Require(esm) breakage – Node 22’s native ESM support can clash with older Lambda layers that still expect CommonJS require. Keep your function pure ESM or stick to CommonJS throughout.
SnapStart + VPC – If your Lambda sits in a VPC, SnapStart’s cold‑start savings disappear because the network interface is attached after the snapshot is restored.
Streaming headers – API Gateway will buffer the response unless you explicitly enable “payload format version 2.0” and set Content-Type to text/event-stream.

Handling Errors and Back‑pressure

A streaming pipeline is only as reliable as its weakest link. You need to anticipate three categories of problems:

Bedrock errors – throttling, model not found, or malformed payload.
Network hiccups – the client disconnects midway.
Back‑pressure – the browser reads slower than Bedrock produces tokens.

Why we care about back‑pressure

If you keep writing to the response socket faster than the client can consume, the underlying TCP buffer fills up, and the Node process may be forced to pause or even crash with ERR_STREAM_WRITE_AFTER_END. Properly handling the writableStream’s ready promise prevents that.

Error handling inside the pipe

// Inside the WritableStream `write` method from the previous section
async write(chunk) {
  // Decode and split as before...
  const lines = new TextDecoder().decode(chunk).split("\n").filter(Boolean);
  for (const line of lines) {
    try {
      const obj = JSON.parse(line);
      if (obj.type === "error") {
        // Bedrock can send an error object mid‑stream.
        const sse = formatSSE(`ERROR: ${obj.message}`);
        // Push the error to the client and then abort the stream.
        await controller.enqueue(encoder.encode(sse));
        controller.terminate(); // stop further processing
        return;
      }

      if (obj.type === "token") {
        const sse = formatSSE(obj.text);
        // Back‑pressure: wait until the underlying stream is ready.
        if (controller.desiredSize === 0) {
          await controller.flush(); // pause until the client drains
        }
        await controller.enqueue(encoder.encode(sse));
      }
    } catch (parseErr) {
      // If JSON is malformed, treat it as a non‑fatal heartbeat.
      console.warn("Failed to parse Bedrock line:", line);
    }
  }
}

In plain English: If Bedrock tells us “I ran into an error”, we forward that error to the browser and stop sending more tokens.

Detecting client disconnects

API Gateway provides a signal (an AbortSignal) you can listen to:

// At the top of streamHandler
const abortSignal = (event.requestContext as any).http?.signal;
if (abortSignal?.aborted) {
  console.log("Client already disconnected – abort early.");
  return { statusCode: 204, headers, body: "" };
}

// Later, attach a listener
abortSignal?.addEventListener("abort", () => {
  console.log("Client aborted the SSE connection.");
  // Close Bedrock stream to avoid paying for unused tokens.
  // The SDK doesn't expose a direct abort, but you can
  // use an AbortController when you create the command.
});

When you create the InvokeModelCommand, pass an AbortSignal so the request can be cancelled:

import { AbortController } from "node-abort-controller";

const abortCtrl = new AbortController();
const command = new InvokeModelCommand({
  // …payload…
  // Attach the signal for cancellation.
  // @ts-ignore – the SDK expects `abortSignal` in the options bag.
  abortSignal: abortCtrl.signal,
});

Key takeaway: Tie the client’s abort signal to the Bedrock request; otherwise you may keep generating tokens that nobody sees.

Rate‑limit back‑off

If Bedrock returns ThrottlingException, implement exponential back‑off before retrying:

async function safeInvoke(command: InvokeModelCommand, retries = 3) {
  for (let attempt = 0; attempt <= retries; attempt++) {
    try {
      return await bedrockClient.send(command);
    } catch (err: any) {
      if (err.name === "ThrottlingException" && attempt < retries) {
        const delay = Math.pow(2, attempt) * 200; // 200ms, 400ms, 800ms...
        console.warn(`Throttled – retrying in ${delay}ms`);
        await new Promise((r) => setTimeout(r, delay));
        continue;
      }
      throw err; // non‑throttle errors bubble up
    }
  }
}

The Takeaway

What you now have in your toolbox:

Streaming matters because it lets users see LLM output instantly, cutting perceived latency.
Bedrock client setup with @aws-sdk/client-bedrock-runtime is a few lines, but you must remember to set stream: true.
Node 22’s native fetch gives you a ReadableStream you can pipe directly to SSE without extra libraries.
A Lambda‑based SSE endpoint requires the right Content-Type, an early callbackWaitsForEmptyEventLoop = false, and proper handling of the response stream.
Error and back‑pressure handling keeps your function from leaking resources, respects client disconnects, and avoids runaway token charges.

By following the steps above, you can ship a chat UI that feels as responsive as a real‑time conversation, while staying within the cost and performance boundaries of AWS services. Happy streaming!

Transparency notice

This article was written with the help of an AI system — Groq (GPT OSS 120B).

Published: 2026-09-18 · Primary focus: Bedrock

All code blocks are intended to be correct and runnable, but please verify them
against the official docs for the tools mentioned before using in production.

Find an error? Drop a comment — corrections are always welcome.

Streaming Claude Tokens with Server‑Sent Events: Build a Real‑Time LLM Chat API on API Gateway and Lambda

Dinesh_gowtham — Fri, 18 Sep 2026 07:34:41 +0000

LLM chat apps feel laggy because most engineers default to polling or heavyweight WebSockets. In reality, Server‑Sent Events (SSE) give you true streaming with far less complexity, and API Gateway now supports it natively. This post shows you how to wire Claude on Bedrock through a Lambda that streams tokens directly to the browser.

Why SSE Beats WebSockets for LLM Streams

When a language model (LLM) generates text, it does so token‑by‑token. Imagine a writer typing a story on a typewriter: you want each keystroke to appear on the screen as soon as it’s pressed. Server‑Sent Events (SSE) are a one‑way, browser‑to‑server push technology that delivers a continuous stream of text lines over a single HTTP connection.

WebSockets are a full‑duplex channel (both client → server and server → client). They are great for chat rooms where each participant sends messages, but they require a separate library, a keep‑alive heartbeat, and often extra IAM configuration on AWS. For LLM streaming we only need server → client flow, so SSE is a lighter fit.

In plain English: If you only need the server to talk, SSE is like a walkie‑talkie that you don’t have to hold both ends of; WebSockets are a two‑way radio that adds unnecessary wiring.

The performance edge

Lower latency – SSE writes each token to the TCP socket as soon as the Lambda yields it, without waiting for a full response body.
Simpler infrastructure – No need for a separate “WebSocket API” in API Gateway; a regular HTTP API with response streaming does the job.
Built‑in reconnection – Browsers automatically retry a dropped SSE connection, giving graceful recovery for intermittent network hiccups.

Configuring API Gateway for Chunked Responses

API Gateway must be told to stream the response rather than buffer it. The new “HTTP integration with response streaming” feature does exactly that, but it only works when the integration type is Lambda proxy and the payload format version is 2.0. Think of the payload format as the envelope the service uses to hand data to Lambda; version 2.0 includes the eventStream field that lets us push chunks.

Below is a minimal TypeScript script that creates an HTTP API, adds a /chat route, and attaches a Lambda function with the correct settings. We use the @aws-sdk/client-api-gateway package, the official AWS SDK for JavaScript.

// api-setup.ts
import {
  ApiGatewayV2Client,
  CreateApiCommand,
  CreateIntegrationCommand,
  CreateRouteCommand,
  UpdateIntegrationCommand,
} from "@aws-sdk/client-api-gateway";

// 1️⃣ Create an HTTP API (not REST) – HTTP APIs support response streaming.
const client = new ApiGatewayV2Client({ region: "us-east-1" });

async function createStreamingApi(lambdaArn: string) {
  // Create the API
  const createApiRes = await client.send(
    new CreateApiCommand({
      Name: "ClaudeChatStreamingAPI",
      ProtocolType: "HTTP", // HTTP API, not REST
      // Enable CORS early – many teams forget this and get blocked in the browser.
      CorsConfiguration: {
        AllowOrigins: ["*"],
        AllowHeaders: ["*"],
        AllowMethods: ["GET", "POST", "OPTIONS"],
      },
    })
  );

  const apiId = createApiRes.ApiId!;
  console.log(`API created with ID ${apiId}`);

  // 2️⃣ Create an integration that points at our Lambda.
  const integrationRes = await client.send(
    new CreateIntegrationCommand({
      ApiId: apiId,
      IntegrationType: "AWS_PROXY", // Lambda proxy integration
      IntegrationUri: lambdaArn,
      PayloadFormatVersion: "2.0", // <-- crucial for streaming
      // Tell API Gateway we want to stream the response.
      IntegrationMethod: "POST",
    })
  );

  const integrationId = integrationRes.IntegrationId!;
  console.log(`Integration created with ID ${integrationId}`);

  // 3️⃣ Attach the integration to a /chat POST route.
  await client.send(
    new CreateRouteCommand({
      ApiId: apiId,
      RouteKey: "POST /chat",
      Target: `integrations/${integrationId}`,
    })
  );

  // 4️⃣ Enable response streaming on the integration (new flag as of 2024).
  await client.send(
    new UpdateIntegrationCommand({
      ApiId: apiId,
      IntegrationId: integrationId,
      // The property name is "ResponseParameters" in the SDK,
      // but we only need to set the streaming flag.
      IntegrationResponseParameters: {
        "contentHandlingStrategy": "CONVERT_TO_TEXT",
      },
      // The real streaming toggle lives in "ResponseTemplates"
      // with a special placeholder; the SDK abstracts it.
      IntegrationResponseTemplates: {
        "application/json": "$input.path('$.eventStream')",
      },
    })
  );

  console.log("Streaming integration configured. Deploy the API to a stage to finish.");
}

export default createStreamingApi;

Key takeaway: The three settings that unlock streaming are: (1) HTTP API, (2) Lambda proxy with payload version 2.0, and (3) the “response streaming” flag on the integration.

Quick deployment tip

API Gateway stages cannot be edited after creation, so create a stage right after the API:

aws apigatewayv2 create-stage \
  --api-id $API_ID \
  --stage-name dev \
  --auto-deploy

Now the endpoint https://$API_ID.execute-api.$REGION.amazonaws.com/dev/chat will forward the Lambda’s chunked response straight to the client as text/event-stream.

Lambda Streaming Handler with Native fetch

Our Lambda runs on Node.js 22, which ships with the global fetch API. That means we can call the Bedrock Claude runtime without adding a separate HTTP library. The Bedrock service returns a streaming JSON where each line looks like:

{ "completion": "Hello", "index": 0, "type": "message" }

We read each line, extract the token, and wrap it in an SSE “data” line:

data: {"token":"Hello"}

The Lambda must set Content-Type: text/event-stream and flush each chunk. In Node 22 the Response object supports a body that is a ReadableStream. By yielding a new Uint8Array for each token we keep the connection alive.

// handler.ts
import { APIGatewayProxyEventV2, APIGatewayProxyResultV2 } from "aws-lambda";

/**
 * Helper: convert a string into an SSE‑formatted line.
 * The double newline signals the end of one event.
 */
function sseLine(payload: object): string {
  return `data: ${JSON.stringify(payload)}\n\n`;
}

/**
 * Main Lambda entry point.
 * Receives a chat message, forwards it to Claude, streams tokens back.
 */
export const handler = async (
  event: APIGatewayProxyEventV2
): Promise<APIGatewayProxyResultV2> => {
  // Parse the incoming JSON body (e.g., { "prompt": "Tell me a joke" })
  const body = JSON.parse(event.body ?? "{}");
  const prompt = body.prompt ?? "";

  // Call Bedrock Claude with streaming enabled.
  // The Bedrock endpoint and model name are placeholders – replace with your own.
  const bedrockResponse = await fetch(
    "https://bedrock-runtime.us-east-1.amazonaws.com/model/anthropic.claude-v2/invoke",
    {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
        // The request must ask for streaming via "accept: text/event-stream"
        Accept: "text/event-stream",
        // Include IAM auth – in Lambda the SDK automatically signs the request.
      },
      body: JSON.stringify({
        prompt,
        stream: true, // Tell Claude to stream tokens
      }),
    }
  );

  // Verify we got a streaming response.
  if (!bedrockResponse.body) {
    return {
      statusCode: 502,
      body: "Bedrock returned no body",
    };
  }

  // Create a ReadableStream that transforms Bedrock's JSON lines into SSE.
  const sseStream = new ReadableStream({
    async start(controller) {
      const decoder = new TextDecoder();
      const reader = bedrockResponse.body!.getReader();

      let buffer = "";
      // Read chunks from Bedrock until the stream ends.
      while (true) {
        const { done, value } = await reader.read();
        if (done) break;

        // Convert bytes to string and accumulate.
        buffer += decoder.decode(value, { stream: true });

        // Split on newline – Bedrock separates JSON objects with "\n".
        const lines = buffer.split("\n");
        // Keep the last partial line in the buffer.
        buffer = lines.pop()!;

        for (const line of lines) {
          if (!line.trim()) continue; // skip empty lines
          const json = JSON.parse(line);
          // Assume the token appears under "completion"
          const token = json.completion ?? "";
          // Push an SSE line downstream.
          const sse = sseLine({ token });
          controller.enqueue(new TextEncoder().encode(sse));
        }
      }

      // When Bedrock finishes, close the SSE stream cleanly.
      controller.close();
    },
  });

  // Return the streaming response to API Gateway.
  return {
    statusCode: 200,
    headers: {
      // This header tells the browser to treat the response as SSE.
      "Content-Type": "text/event-stream",
      // Prevent any caching – we want every token to appear.
      "Cache-Control": "no-cache",
      // Keep‑alive helps some proxies not to cut the connection.
      "Connection": "keep-alive",
    },
    // The body must be a ReadableStream for API Gateway to forward it.
    body: sseStream as any, // TypeScript needs a cast here.
  };
};

Tip: The only place where we must set a header is Content-Type: text/event-stream. Without it the browser treats the payload as plain text and never fires the message event.

Analogy for the stream transformation

Think of Bedrock’s response as a river of tiny beads (JSON lines). Our Lambda is a sifter that picks each bead, puts it in a clear plastic bag (the SSE line), and drops the bag downstream. The river never stops; we just keep handing out bags one at a time.

Frontend: Consuming the Event Stream

On the client side we use the native EventSource API, which knows how to listen to SSE. It automatically reconnects if the connection drops, so we get a resilient UI with almost no code.

<!-- index.html -->
<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <title>Claude Chat – Live</title>
  <style>
    body { font-family: sans-serif; margin: 2rem; }
    #chat { white-space: pre-wrap; border: 1px solid #ccc; padding: 1rem; min-height: 200px; }
    #prompt { width: 80%; }
  </style>
</head>
<body>
  <h1>Claude Live Chat</h1>
  <div id="chat"></div>
  <input id="prompt" placeholder="Ask Claude..." />
  <button id="send">Send</button>

  <script>
    const chatDiv = document.getElementById('chat');
    const promptInput = document.getElementById('prompt');
    const sendBtn = document.getElementById('send');

    // Helper: append text to the chat area.
    function appendToken(token) {
      chatDiv.textContent += token;
    }

    // Click handler – starts a new SSE connection for each user message.
    sendBtn.onclick = () => {
      const prompt = promptInput.value.trim();
      if (!prompt) return;

      // Reset UI for the new response.
      chatDiv.textContent = '';
      promptInput.value = '';

      // Create an EventSource pointing at our API Gateway endpoint.
      const es = new EventSource(`https://YOUR_API_ID.execute-api.us-east-1.amazonaws.com/dev/chat?prompt=${encodeURIComponent(prompt)}`);

      // Listen for each token.
      es.onmessage = (event) => {
        const data = JSON.parse(event.data);
        // The Lambda emitted { token: "..." }
        appendToken(data.token);
      };

      // If the server closes (e.g., Claude finished), we stop listening.
      es.onerror = (err) => {
        console.error('SSE error', err);
        es.close();
      };
    };
  </script>
</body>
</html>

In plain English: EventSource is a tiny built‑in browser object that keeps the HTTP connection open and fires a message event each time the server writes a line that starts with data:.

Why we put the prompt in the query string

Because SSE is a GET‑only protocol, we cannot send a request body. Placing the user’s prompt in the query string is the simplest way to hand data to the Lambda without breaking the SSE contract. For production you would want to sign the URL or use an API key, but the concept stays the same.

Common Pitfalls and Gotchas

Area	Gotcha	How to avoid
API Gateway	The response is buffered unless “HTTP integration with response streaming” is turned on.	Double‑check `PayloadFormatVersion: "2.0"` and the `UpdateIntegrationCommand` that sets `eventStream`.
API Gateway	REST APIs (v1) don’t support JWT authorizers on streaming routes.	Use an HTTP API (v2) instead; it supports JWT at the stage level.
API Gateway	Integration timeout is fixed at 29 seconds.	Ensure Claude’s `max_tokens` and `temperature` settings keep the response under the limit, or break long chats into multiple requests.
Lambda	Node 22’s `require(esm)` in a layer silently fails, breaking imports.	Stick to native ES modules (`import … from …`) or bundle everything with esbuild.
Lambda	Forgetting to set `Content-Type: text/event-stream` leads to a silent client.	Add the header in the Lambda response (see code).
Lambda	Provisioned Concurrency costs rise quickly if left idle.	Use on‑demand concurrency for low‑traffic prototypes; enable provisioned only after measuring steady traffic.
Frontend	CORS misconfiguration blocks the browser.	In the API Gateway console enable CORS for `*` (or specific origins) and include `Access-Control-Allow-Origin` in the response headers.
Frontend	EventSource cannot send a request body, so developers try a POST.	Keep the request as a GET with query parameters, or use a short‑lived signed URL if security is a concern.

Tip: When you see the browser console saying “EventSource failed: 502 Bad Gateway”, the first thing to inspect is whether API Gateway actually streamed anything. Look at the execution logs ($context.integrationLatency) – a null body means streaming wasn’t enabled.

The “no‑buffer” test

You can verify streaming works by curling the endpoint with --no-buffer:

curl -N "https://YOUR_API_ID.execute-api.us-east-1.amazonaws.com/dev/chat?prompt=Hello"

If you see each data: line appear instantly, the pipeline is streaming correctly. If you get a single large JSON blob after a few seconds, the response is still being buffered.

The Takeaway

Key points to remember

SSE is a one‑way, low‑overhead streaming protocol perfect for LLM token delivery.
API Gateway must be an HTTP API with Lambda proxy integration, payload version 2.0, and the response‑streaming flag turned on.
The Lambda handler uses the built‑in fetch (Node 22) to call Bedrock, reads the streaming JSON, and rewrites each token into an SSE line using a ReadableStream.
EventSource on the browser side automatically reconnects and fires a message event for every token, giving a live‑typing experience.
Common pitfalls include hidden buffering, CORS misconfiguration, and the 29‑second timeout – all solvable with the checklist above.

Now you have a minimal, production‑ready stack that streams Claude’s output to any web client without the heavyweight ceremony of WebSockets. Happy coding!

Transparency notice

This article was written with the help of an AI system — Groq (GPT OSS 120B).

Published: 2026-09-18 · Primary focus: APIGateway

All code blocks are intended to be correct and runnable, but please verify them
against the official docs for the tools mentioned before using in production.

Find an error? Drop a comment — corrections are always welcome.

How Cursor AI Understands Your Whole Codebase — And How to Leverage It in a Serverless Lambda

Dinesh_gowtham — Thu, 17 Sep 2026 15:37:27 +0000

Cursor AI can scan an entire repository in seconds and give line‑by‑line suggestions, but most engineers treat it like a simple autocomplete. Learn why that mindset wastes the tool’s power and how to unlock full‑context code generation in production.

What “Whole‑Repo” Context Means for an LLM

Why it matters – An LLM (large language model) is a statistical engine that predicts the next token (word or symbol) based on everything it has seen. If you feed it just one file, it can only guess based on that file’s local symbols. Give it the whole repository, and the model can see relationships across modules, shared types, and project‑wide conventions. Think of it like a detective who reads the entire case file instead of just the last paragraph before writing a report.

Key terms

Embedding – a list of numbers that captures the meaning of a piece of text; the model works with embeddings, not raw words.
Project graph – a map of files and their import/export relationships; essentially a “family tree” of code.

How to give Cursor the whole repo – Cursor’s SDK has a helper called uploadRepoTree. It walks the directory, reads each file, and sends a compressed snapshot to the service. The service then builds the context internally, so every subsequent suggest call can reference any file.

import { Cursor } from "cursor";

/**
 * Send an entire repository to Cursor so it can build a global view.
 * @param repoPath Absolute path on the Lambda’s /tmp storage where the repo lives.
 * @returns A repoId that you’ll use for later suggestion calls.
 */
async function uploadWholeRepo(repoPath: string): Promise<string> {
  const cursor = new Cursor({ apiKey: process.env.CURSOR_API_KEY! });

  // `uploadRepoTree` recursively reads files, strips binaries, and returns an identifier.
  const { repoId } = await cursor.uploadRepoTree({
    root: repoPath,
    // optional: ignore patterns (node_modules, .git, etc.)
    ignore: ["node_modules/**", ".git/**"],
  });

  console.log(`✅ Uploaded repo, got repoId=${repoId}`);
  return repoId;
}

In plain English – By sending the whole repo once, you give the AI a “bird’s‑eye view” of your project, enabling it to suggest changes that respect the overall architecture instead of isolated snippets.

Getting the Cursor SDK Inside a Lambda Function

Why the setup matters – A Lambda runs in a constrained environment (limited /tmp space, short cold‑start time). If you bundle the SDK incorrectly, you might hit the require(esm) gotcha: Node 22 treats ESM modules differently, and some older Lambda layers silently fail to load them, leading to runtime errors you won’t see in logs.

Step‑by‑step packaging

Create a fresh Node project inside a directory you’ll zip for deployment.
Pin the SDK versions to avoid accidental upgrades that change the API.

mkdir cursor-lambda && cd cursor-lambda
npm init -y
# Pin exact versions; these are the ones verified to work with Node 22 on Lambda
npm install cursor@2.4.1 @aws-sdk/client-lambda@3.560.0

Add a type: "module" field in package.json so Node treats your code as ESM, matching the Cursor SDK.

{
  "name": "cursor-lambda",
  "version": "1.0.0",
  "type": "module",   // <-- tells Node to use ESM import syntax
  "dependencies": {
    "cursor": "2.4.1",
    "@aws-sdk/client-lambda": "3.560.0"
  }
}

Write the handler (see later sections) and zip the whole folder, including node_modules.

Tip – Keep the zipped package under 50 MB. If it grows larger, enable Lambda Layers for the SDKs instead of bundling them directly.

Streaming Suggestions with Node’s `diagnostics_channel`

Why streaming helps – Cursor can return suggestions as they are generated, rather than waiting for the entire response. In a PR‑assistant, you want to start posting early feedback to keep the conversation fast. diagnostics_channel is a built‑in Node feature that lets you listen to custom events emitted by the Cursor SDK without polluting your own code.

Key term

Channel – a named pipe inside the same process that lets one part of the code emit data and another part listen for it.

Enabling the channel – The Cursor SDK emits a channel called "cursor.suggestion" for each token it generates. You subscribe once at the top of the Lambda, then each suggest call will push events into the same channel.

import { createChannel, channel } from "node:diagnostics_channel";

/**
 * Subscribe to the "cursor.suggestion" channel.
 * Every time Cursor generates a piece of a suggestion, this listener runs.
 */
function startSuggestionStream(repoId: string, filePath: string) {
  const suggestionChannel = channel("cursor.suggestion");

  // The listener receives an object with `repoId`, `filePath`, and the `text` chunk.
  suggestionChannel.subscribe((msg) => {
    if (msg.repoId !== repoId || msg.filePath !== filePath) return;

    // For demo purposes we just log; in production you would buffer and send later.
    console.log(`🧩 Received chunk for ${filePath}: ${msg.text}`);
  });
}

In plain English – Think of the channel as a walkie‑talkie: the SDK talks, your code listens, and you can react to each piece of the conversation as it arrives.

Handling Rate Limits and Error Types

Why you must be explicit – Cursor enforces a per‑minute token quota. When you exceed it, the service returns HTTP 429 Too Many Requests with a Retry-After header indicating how many seconds to wait. The SDK, by default, automatically retries the request up to three times. In a Lambda this hidden retry can stretch the execution beyond the timeout, and you’ll see latency spikes in CloudWatch that look like “random slow calls.”

Disabling auto‑retry – Pass { retry: false } when creating the client, then handle the CursorRateLimitError yourself.

import { Cursor, CursorRateLimitError } from "cursor";

/**
 * Create a Cursor client that does *not* automatically retry.
 */
function makeCursorClient(): Cursor {
  return new Cursor({
    apiKey: process.env.CURSOR_API_KEY!,
    // Turn off the SDK’s built‑in retry logic.
    retry: false,
  });
}

/**
 * Wrapper that calls cursor.suggest and deals with 429 errors.
 */
async function safeSuggest(
  client: Cursor,
  params: Parameters<Cursor["suggest"]>[0]
): Promise<void> {
  try {
    await client.suggest(params);
  } catch (err) {
    if (err instanceof CursorRateLimitError) {
      // The error object contains the raw `Retry-After` header value.
      const waitSec = Number(err.retryAfter);
      console.warn(`⚠️ Hit rate limit, waiting ${waitSec}s before retry`);
      // Simple back‑off – Lambda can `await` a timeout.
      await new Promise((r) => setTimeout(r, waitSec * 1000));
      // Retry once manually; you could add exponential back‑off here.
      await client.suggest(params);
    } else {
      // Re‑throw unknown errors so Lambda records a failure.
      throw err;
    }
  }
}

Tip – Log the retryAfter value each time you hit 429. Over a week you can spot patterns (e.g., every 10 seconds during CI bursts) and adjust your webhook frequency accordingly.

Putting It All Together: A Real‑World PR‑Assistant

Why this example is useful – It demonstrates the complete flow:

GitHub sends a pull‑request webhook.
Lambda clones the repo into /tmp, uploads the whole tree to Cursor.
For each changed file we call suggest, streaming results via diagnostics_channel.
When all suggestions are ready we post a comment on the PR with inline diffs.

Full handler (TypeScript, heavily commented).

import { APIGatewayProxyEvent, APIGatewayProxyResult } from "aws-lambda";
import { execSync } from "node:child_process";
import { promises as fs } from "node:fs";
import { join } from "node:path";
import { Cursor } from "cursor";
import {
  LambdaClient,
  InvokeCommand,
} from "@aws-sdk/client-lambda";
import { channel } from "node:diagnostics_channel";

/**
 * Helper: clone the repo to the Lambda's /tmp directory.
 * Git is available in the Lambda runtime (Amazon Linux).
 */
async function cloneRepo(repoUrl: string, commitSha: string): Promise<string> {
  const dest = join("/tmp", "repo");
  // Clean up any previous run.
  await fs.rm(dest, { recursive: true, force: true });
  execSync(`git clone ${repoUrl} ${dest}`, { stdio: "ignore" });
  execSync(`git -C ${dest} checkout ${commitSha}`, { stdio: "ignore" });
  return dest;
}

/**
 * Helper: post a comment to GitHub (simplified; in production use Octokit).
 */
async function postGitHubComment(
  owner: string,
  repo: string,
  prNumber: number,
  body: string
) {
  const token = process.env.GITHUB_TOKEN!;
  const url = `https://api.github.com/repos/${owner}/${repo}/issues/${prNumber}/comments`;
  const payload = JSON.stringify({ body });
  execSync(
    `curl -s -X POST -H "Authorization: token ${token}" -H "Content-Type: application/json" -d '${payload}' ${url}`
  );
}

/**
 * Main Lambda entry point – receives the GitHub webhook payload.
 */
export const handler = async (
  event: APIGatewayProxyEvent
): Promise<APIGatewayProxyResult> => {
  // -----------------------------------------------------------------
  // 1️⃣ Extract useful data from the webhook.
  // -----------------------------------------------------------------
  const payload = JSON.parse(event.body ?? "{}");
  const pr = payload.pull_request;
  const repoUrl = pr.head.repo.clone_url;
  const commitSha = pr.head.sha;
  const changedFiles: string[] = payload.pull_request.changed_files
    ? payload.pull_request.changed_files.map((f: any) => f.filename)
    : []; // fallback if not provided

  // -----------------------------------------------------------------
  // 2️⃣ Clone repo into /tmp and upload whole‑repo context to Cursor.
  // -----------------------------------------------------------------
  const repoPath = await cloneRepo(repoUrl, commitSha);
  const cursor = new Cursor({ apiKey: process.env.CURSOR_API_KEY!, retry: false });
  const repoId = await cursor.uploadRepoTree({
    root: repoPath,
    ignore: ["node_modules/**", ".git/**"],
  });

  // -----------------------------------------------------------------
  // 3️⃣ Set up diagnostics_channel to collect suggestion chunks.
  // -----------------------------------------------------------------
  const suggestionChannel = channel("cursor.suggestion");
  const suggestions: Record<string, string[]> = {};

  suggestionChannel.subscribe((msg) => {
    const key = `${msg.repoId}|${msg.filePath}`;
    if (!suggestions[key]) suggestions[key] = [];
    suggestions[key].push(msg.text);
  });

  // -----------------------------------------------------------------
  // 4️⃣ For each changed file, ask Cursor for line‑by‑line suggestions.
  // -----------------------------------------------------------------
  for (const relPath of changedFiles) {
    const absPath = join(repoPath, relPath);
    const fileContent = await fs.readFile(absPath, "utf-8");

    // Start listening *before* we send the request.
    startSuggestionStream(repoId, relPath);

    // Wrap the call so we can handle rate‑limit errors.
    await safeSuggest(cursor, {
      repoId,
      filePath: relPath,
      // Provide the full content; Cursor will use the whole‑repo context internally.
      content: fileContent,
      // Ask for a diff‑style suggestion.
      mode: "diff",
    });
  }

  // -----------------------------------------------------------------
  // 5️⃣ Assemble a markdown comment with the collected diffs.
  // -----------------------------------------------------------------
  let commentBody = "### 🤖 Cursor AI suggestions\n\n";
  for (const key of Object.keys(suggestions)) {
    const [, filePath] = key.split("|");
    const diff = suggestions[key].join("");
    commentBody += `#### \`${filePath}\`\n\`\`\`diff\n${diff}\n\`\`\`\n`;
  }

  // -----------------------------------------------------------------
  // 6️⃣ Post the comment back to the PR.
  // -----------------------------------------------------------------
  await postGitHubComment(
    pr.base.repo.owner.login,
    pr.base.repo.name,
    pr.number,
    commentBody
  );

  return {
    statusCode: 200,
    body: JSON.stringify({ message: "Suggestions posted" }),
  };
};

/**
 * Helper used earlier: start streaming for a specific file.
 */
function startSuggestionStream(repoId: string, filePath: string) {
  // No extra work needed beyond the subscription already set up;
  // this function exists for readability.
}

In plain English – The handler glues together three moving parts: a fresh copy of the repo, Cursor’s whole‑repo understanding, and a streaming channel that lets us push suggestions to GitHub as soon as they appear.

Gotchas highlighted in the code

require(esm) – Because we set "type": "module" the import statements work; if you forget this, Node will throw “Cannot use import statement outside a module.”
SnapStart + VPC – If you later attach a VPC to this Lambda, SnapStart (the pre‑warm optimization) won’t help because the cold‑start time is dominated by VPC ENI attachment, not by the JavaScript engine.
Provisioned Concurrency – Turning it on for a low‑traffic PR‑assistant can add $0.01 / hour per provisioned instance even when no PRs arrive. Monitor usage before enabling.

The Takeaway

Key points you can act on today

Whole‑repo context lets an LLM see cross‑file relationships, turning “autocomplete” into “architectural assistant.”
Install the Cursor SDK as an ESM module, pin the version, and bundle node_modules to avoid the require(esm) pitfall.
Use Node’s diagnostics_channel to receive suggestion chunks immediately, keeping the PR feedback loop fast.
Turn off the SDK’s hidden retry logic; handle CursorRateLimitError yourself so Lambda timeouts stay predictable.
A single Lambda can clone a repo, upload the entire tree, stream per‑file suggestions, and post a nicely formatted comment—all without a server.

With these steps you can move from treating Cursor as a fancy autocomplete to using it as a full‑context code reviewer that runs on demand, cost‑effectively, in a serverless environment. Happy coding!

Transparency notice

This article was written with the help of an AI system — Groq (GPT OSS 120B).

Published: 2026-09-17 · Primary focus: CursorAI

All code blocks are intended to be correct and runnable, but please verify them
against Cursor's official docs before using in production.

Find an error? Drop a comment — corrections are always welcome.

Promise.withResolvers() Demystified: Simplify Claude Agent Loops in Node.js

Dinesh_gowtham — Thu, 17 Sep 2026 07:56:25 +0000

Developers still drown in new Promise((resolve,reject)=>…) boilerplate when wiring Claude’s function‑calling tool calls. The new Promise.withResolvers() helper lets you expose resolve/reject instantly, cutting noise and bugs. In this post you’ll see how a few extra lines turn a tangled async loop into crystal‑clear code.

Why the Old Promise Boilerplate Breaks

The problem in plain language

When an LLM (large language model) like Claude decides it needs a tool—say a calendar lookup—you have to pause your JavaScript code, send a request to the tool, wait for the answer, then feed that answer back to Claude. The common pattern looks like this:

function callToolWhenNeeded(request) {
  return new Promise((resolve, reject) => {
    // 1️⃣ send request to Claude
    // 2️⃣ when Claude asks for a tool, invoke the tool
    // 3️⃣ resolve the promise with the tool’s result
  });
}

At first glance it seems fine, but three hidden bugs often surface:

Forgot to call resolve – the promise stays pending forever, leaving your Lambda hanging.
Calling reject twice – an accidental second reject turns into an unhandled‑promise rejection warning.
Scattered control flow – the resolve/reject functions live deep inside callbacks, making the code hard to read and to test.

Because the promise is created inside the function, the only way for the outer loop to “see” the result is to capture those resolver functions and expose them. Doing it manually means writing extra lines, copying variable names, and hoping you didn’t typo‑spell anything.

In plain English: The old pattern forces you to juggle two invisible levers (resolve and reject) that live inside a black box. If you lose one, the whole machine stalls.

A simple analogy

Imagine you’re building a mailbox that delivers letters to a friend. With the old pattern you hand your friend a sealed box that contains a secret key. Only after they open the box can they drop the letter inside. If the key is misplaced, the letter never arrives. Promise.withResolvers() hands you the key up front, so you can hand it to anyone who needs it without rummaging through sealed boxes.

Introducing Promise.withResolvers()

What the new API does

Promise.withResolvers() (added in Node.js 22) creates a promise and returns an object that already contains the resolve and reject functions:

const { promise, resolve, reject } = Promise.withResolvers();

Now you have a plain promise that can be awaited, and you hold the two “levers” in variables you can pass around freely. No more nesting, no more accidental shadowing.

Why this matters for Claude’s tool‑calling loop

Claude’s invokeModel call returns a stream of JSON‑LDM (language‑model‑driven) messages. When it emits a function_call event you must:

Execute the requested function (e.g., fetch a user profile).
Feed the result back into the model as a new message.

With withResolvers() you can set up a single place to wait for the function’s result, while the function itself can resolve it from anywhere—inside an HTTP callback, a database query, or a third‑party SDK call.

Minimal example

// Grab a promise and its resolvers in one line
const { promise, resolve, reject } = Promise.withResolvers();

// Somewhere else, perhaps in an HTTP handler, we finish the work
async function fetchUser(id) {
  try {
    const user = await db.getUser(id); // pretend async DB call
    resolve(user);                      // hand the result back
  } catch (err) {
    reject(err);                        // tell the waiting code something went wrong
  }
}

// The caller simply awaits the promise
async function waitForUser(id) {
  fetchUser(id); // fire‑and‑forget; resolves later
  return promise; // pauses until resolve or reject is called
}

Key takeaway: One line gives you a promise and the two controls you need, removing the boilerplate that usually hides in callback hell.

Building a Claude Function‑Calling Agent

The goal

Create a Node.js 22 AWS Lambda that:

Sends a prompt to Claude via the Bedrock Runtime SDK (@aws-sdk/client-bedrock-runtime).
Detects a function_call request from Claude.
Executes the requested tool (a mock “searchKnowledgeBase” function).
Sends the tool’s output back to Claude.
Streams the final answer to the HTTP client.

All of this will use Promise.withResolvers() to keep the async flow tidy.

Step‑by‑step construction

1. Install the SDK

npm install @aws-sdk/client-bedrock-runtime

2. Import required classes

import {
  BedrockRuntimeClient,
  InvokeModelCommand,
  BedrockRuntimeServiceException,
} from "@aws-sdk/client-bedrock-runtime";

BedrockRuntimeClient – a client that talks to AWS Bedrock’s model‑hosting endpoint.
InvokeModelCommand – the command object that carries the request payload.
BedrockRuntimeServiceException – a base error type for all Bedrock‑related failures.

3. Helper to parse streaming SSE (Server‑Sent Events)

Bedrock streams responses as a series of SSE frames. Node.js does not parse them automatically, so we need a tiny parser:

function parseSSE(stream: NodeJS.ReadableStream) {
  const decoder = new TextDecoder();
  let buffer = "";

  return new Promise<string>((resolve, reject) => {
    stream.on("data", (chunk) => {
      buffer += decoder.decode(chunk, { stream: true });
      const lines = buffer.split("\n");
      // Keep the last incomplete line for next chunk
      buffer = lines.pop() ?? "";

      for (const line of lines) {
        if (line.startsWith("data:")) {
          const json = line.slice(5).trim();
          if (json === "[DONE]") return resolve(""); // end of stream
          try {
            const payload = JSON.parse(json);
            // Emit the payload to the outer promise
            resolve(JSON.stringify(payload));
          } catch (e) {
            reject(e);
          }
        }
      }
    });

    stream.on("error", reject);
    stream.on("end", () => resolve(""));
  });
}

Tip: The parser returns the first JSON payload it sees; for a real‑world agent you would accumulate messages until the model says it’s done.

4. The Lambda handler with `Promise.withResolvers()`

import type { APIGatewayProxyEvent, APIGatewayProxyResult } from "aws-lambda";

const client = new BedrockRuntimeClient({ region: "us-east-1" }); // adjust region

export async function handler(
  event: APIGatewayProxyEvent
): Promise<APIGatewayProxyResult> {
  // -----------------------------------------------------------------
  // 1️⃣ Create a resolver pair that the tool function will use later
  // -----------------------------------------------------------------
  const { promise: toolPromise, resolve, reject } = Promise.withResolvers<any>();

  // -----------------------------------------------------------------
  // 2️⃣ Build the Claude request (Claude 3.5 Sonnet is a common choice)
  // -----------------------------------------------------------------
  const prompt = JSON.stringify({
    messages: [{ role: "user", content: event.body ?? "Tell me a story." }],
    // Instruct Claude that it may call the `searchKnowledgeBase` tool
    toolConfig: {
      tools: [
        {
          name: "searchKnowledgeBase",
          description: "Searches the company's knowledge base.",
          inputSchema: {
            type: "object",
            properties: { query: { type: "string" } },
            required: ["query"],
          },
        },
      ],
    },
    // Ask for streaming so we can forward chunks immediately
    stream: true,
  });

  // -----------------------------------------------------------------
  // 3️⃣ Send the request to Bedrock
  // -----------------------------------------------------------------
  const command = new InvokeModelCommand({
    modelId: "anthropic.claude-3-5-sonnet-20240620-v1:0", // example model
    contentType: "application/json",
    accept: "application/json",
    body: Buffer.from(prompt),
  });

  let modelStream: NodeJS.ReadableStream;
  try {
    const response = await client.send(command);
    // response.body is a stream of SSE frames
    modelStream = response.body as NodeJS.ReadableStream;
  } catch (err) {
    // -----------------------------------------------------------------
    // 4️⃣ Bedrock‑specific error handling
    // -----------------------------------------------------------------
    const message = (err as BedrockRuntimeServiceException).message ?? "Unknown error";
    return {
      statusCode: 502,
      body: JSON.stringify({ error: `Bedrock call failed: ${message}` }),
    };
  }

  // -----------------------------------------------------------------
  // 5️⃣ Listen for a function call from Claude
  // -----------------------------------------------------------------
  modelStream.on("data", async (chunk) => {
    const text = chunk.toString();
    // Simple detection – in real code you would parse the JSON payload
    if (text.includes(`"name":"searchKnowledgeBase"`)) {
      // Extract the query (naïve regex for demo)
      const match = /"arguments":\s*"([^"]+)"/.exec(text);
      const query = match ? match[1] : "default query";

      // Fire the tool and let it resolve the promise
      // No need for another `new Promise` – we already have resolve/reject
      mockSearchKnowledgeBase(query).then(resolve).catch(reject);
    }
  });

  // -----------------------------------------------------------------
  // 6️⃣ Wait for the tool’s answer (or an error) using the promise we created
  // -----------------------------------------------------------------
  let toolResult: any;
  try {
    toolResult = await toolPromise; // pauses here until resolve() or reject()
  } catch (toolErr) {
    // If the tool failed, tell Claude about the failure and stop
    return {
      statusCode: 500,
      body: JSON.stringify({ error: `Tool failed: ${toolErr}` }),
    };
  }

  // -----------------------------------------------------------------
  // 7️⃣ Feed the tool result back to Claude as a new message
  // -----------------------------------------------------------------
  const followUp = JSON.stringify({
    messages: [
      { role: "assistant", content: "Calling tool..." },
      { role: "tool", name: "searchKnowledgeBase", content: JSON.stringify(toolResult) },
    ],
    stream: true,
  });

  const followUpCmd = new InvokeModelCommand({
    modelId: "anthropic.claude-3-5-sonnet-20240620-v1:0",
    contentType: "application/json",
    accept: "application/json",
    body: Buffer.from(followUp),
  });

  // Send the follow‑up and stream its output back to the HTTP client
  const followUpResp = await client.send(followUpCmd);
  const followUpStream = followUpResp.body as NodeJS.ReadableStream;

  // Collect final output (for simplicity we concatenate)
  const finalOutput = await parseSSE(followUpStream);

  return {
    statusCode: 200,
    body: finalOutput,
    headers: { "Content-Type": "application/json" },
  };
}

// -----------------------------------------------------------------
// Mock implementation of the knowledge‑base search tool
// -----------------------------------------------------------------
function mockSearchKnowledgeBase(query: string): Promise<{ answer: string }> {
  return new Promise((res) => {
    setTimeout(() => {
      res({ answer: `Results for "${query}" (mocked).` });
    }, 300); // simulate latency
  });
}

What changed compared to the classic pattern?

Classic `new Promise` pattern	Using `Promise.withResolvers()`
`new Promise((resolve,reject)=>{ … })` buried inside the function.	`const {promise, resolve, reject}=Promise.withResolvers();` declared once, visible to every callback.
Resolve/reject often passed via closure, easy to lose scope.	Resolve/reject are plain variables you can pass anywhere.
Extra lines for “return new Promise…”.	Single line creates both the promise and its controls.

In plain English: Promise.withResolvers() is like getting the remote control for a TV before you even turn it on. You can hand the remote to any helper, and the TV will respond when they press a button.

Handling Errors and Timeouts Gracefully

Why error handling matters in an LLM loop

When you involve external services (Bedrock, a database, a knowledge base), three failure modes dominate:

Model‑side errors – Bedrock can return ThrottlingException if you exceed token limits per minute.
Tool‑side errors – Your custom function might throw, causing the promise to reject.
Timeouts – The model or the tool may take longer than your Lambda’s remaining execution time.

If any of these errors slip through without a proper catch, Node.js will emit an unhandled‑promise rejection warning, and the Lambda may terminate silently. That is the most common hidden bug developers encounter when they forget to resolve the promise before the function exits.

Adding a timeout wrapper

A small helper that races the tool promise against a timer:

function withTimeout<T>(p: Promise<T>, ms: number): Promise<T> {
  return new Promise<T>((resolve, reject) => {
    const timer = setTimeout(() => {
      reject(new Error(`Operation timed out after ${ms}ms`));
    }, ms);

    p.then(
      (value) => {
        clearTimeout(timer);
        resolve(value);
      },
      (err) => {
        clearTimeout(timer);
        reject(err);
      }
    );
  });
}

You then wrap the tool call:

const toolResult = await withTimeout(toolPromise, 2_000); // 2 seconds max

If the tool doesn’t call resolve within two seconds, the timeout rejects, and the Lambda returns a clean error response.

Distinguishing Bedrock error types

try {
  await client.send(command);
} catch (err) {
  if (err.name === "ThrottlingException") {
    // Token limit per minute exceeded – back off and retry later
    return {
      statusCode: 429,
      body: JSON.stringify({ error: "Rate limit hit – try again in a few seconds." }),
    };
  }
  if (err instanceof BedrockRuntimeServiceException) {
    // Generic Bedrock service error
    return {
      statusCode: 502,
      body: JSON.stringify({ error: `Bedrock error: ${err.message}` }),
    };
  }
  // Unexpected error
  throw err; // let Lambda surface it
}

Tip: Always inspect err.name or err.$metadata.httpStatusCode to decide whether you should retry (throttling) or surface a hard failure.

Polyfilling for older Node versions

If you run this code on Node.js 20 (or earlier), Promise.withResolvers does not exist. A tiny polyfill is enough:

if (!Promise.withResolvers) {
  // @ts-ignore – augment the global Promise type at runtime
  Promise.withResolvers = () => {
    let resolve!: (value: any) => void;
    let reject!: (reason?: any) => void;
    const promise = new Promise((res, rej) => {
      resolve = res;
      reject = rej;
    });
    return { promise, resolve, reject };
  };
}

Place the polyfill at the top of your Lambda file so the rest of the code can stay unchanged.

Running the Loop in a Serverless Lambda

Why Lambda adds its own constraints

A Lambda function has a maximum execution time (up to 15 minutes) and a memory ceiling that influences network throughput. When you combine streaming responses from Bedrock with a tool that may block, you need to be careful not to let the event loop stay alive after the promise resolves—otherwise the Lambda will keep running and you’ll be billed for idle time.

Ensuring the event loop closes

export async function handler(...): Promise<APIGatewayProxyResult> {
  // ... existing code ...

  // After we have the final output, tell Lambda we’re done.
  // `context.callbackWaitsForEmptyEventLoop` is false by default in newer runtimes,
  // but we set it explicitly for clarity.
  const context = (global as any).awsLambdaContext as { callbackWaitsForEmptyEventLoop: boolean };
  if (context) {
    context.callbackWaitsForEmptyEventLoop = false;
  }

  return {
    statusCode: 200,
    body: finalOutput,
  };
}

Setting callbackWaitsForEmptyEventLoop = false tells the runtime that it can freeze the function as soon as the handler returns, even if there are lingering streams or timers. This eliminates the hidden “dangling promise” bug that often appears when developers forget to resolve a promise before the handler exits.

Dealing with Bedrock rate limits in Lambda

Because Bedrock limits tokens per minute across the whole account, a burst of Lambda invocations can quickly hit the ceiling. A practical mitigation:

async function invokeWithBackoff(cmd: InvokeModelCommand, attempts = 3): Promise<any> {
  for (let i = 0; i < attempts; i++) {
    try {
      return await client.send(cmd);
    } catch (err) {
      if (err.name === "ThrottlingException") {
        const delay = 500 * (i + 1); // exponential back‑off
        await new Promise((r) => setTimeout(r, delay));
        continue;
      }
      throw err; // non‑throttling errors bubble up
    }
  }
  throw new Error("Exceeded retry attempts for Bedrock throttling.");
}

Replace client.send(command) with invokeWithBackoff(command) in the handler. The back‑off respects the per‑minute token ceiling while still giving most requests a chance to succeed.

Key takeaway: In a serverless environment you must (1) close the event loop fast, (2) respect Bedrock’s per‑minute token budget, and (3) guard every async boundary with timeouts and proper error handling.

The Takeaway

What you now have in your toolbox

Promise.withResolvers() creates a promise and hands you the resolve/reject functions in a single, readable line.
By exposing those resolvers, you eliminate the hidden “forgot to resolve” bugs that plague Claude function‑calling loops.
A tiny SSE parser lets you turn Bedrock’s streaming output into usable JSON without pulling in heavy libraries.
Centralized error handling distinguishes Bedrock throttling, generic service errors, and tool‑side failures, allowing graceful fallback or retry.
A reusable withTimeout helper protects your Lambda from hanging forever when a tool misbehaves.
Setting callbackWaitsForEmptyEventLoop = false guarantees the Lambda shuts down promptly after the promise resolves, saving money and avoiding hidden rejections.

With these pieces, wiring Claude’s function‑calling tool calls becomes a matter of a few clear steps rather than a maze of nested promises. Happy coding!

Transparency notice

This article was written with the help of an AI system — Groq (GPT OSS 120B).

Published: 2026-09-17 · Primary focus: JavaScriptPatterns

All code blocks are intended to be correct and runnable, but please verify them
against the MDN JavaScript docs before using in production.

Find an error? Drop a comment — corrections are always welcome.

How to Build a Lightning‑Fast TypeScript Lambda that Calls Claude, Using esbuild and tsc --noEmit

Dinesh_gowtham — Wed, 16 Sep 2026 15:32:07 +0000

Developers waste minutes watching TypeScript compile before every Lambda deploy, even though the code never runs with types. By swapping ts-node for a combined esbuild + tsc --noEmit pipeline, you keep full type safety and slash build time. The result is a Lambda that talks to Claude with zero‑runtime overhead.

Why the Traditional ts-node/tsc Build Is a Bottleneck

When you write a Lambda in TypeScript you usually run ts-node (a tool that compiles on‑the‑fly) during local testing and then run tsc (the TypeScript compiler) as a separate step before packaging. Two things happen:

Two passes over the same source – tsc checks types, then ts-node (or a bundler) rewrites the code again.
Full source files are shipped – even type‑only imports stay in the bundle, adding bytes that the Lambda never uses.

Think of the process like a chef who first tastes every ingredient, then cooks the whole dish again from scratch. The taste test is useful, but doing it twice eats time and resources.

In plain English – the traditional flow makes the build slower without giving you any extra runtime benefit.

Minimal Example of the Traditional Flow

// src/handler.ts
import { APIGatewayProxyEvent, APIGatewayProxyResult } from "aws-lambda";
import { Anthropic } from "@anthropic-ai/sdk";

export const handler = async (event: APIGatewayProxyEvent): Promise<APIGatewayProxyResult> => {
  const body = JSON.parse(event.body ?? "{}");
  const diff = body.diff as string; // type‑only check, but stays in bundle

  const client = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY! });
  const response = await client.completions.create({
    model: "claude-3-sonnet-20240229",
    prompt: `Review this diff and suggest improvements:\n${diff}`,
    max_tokens: 512,
  });

  return {
    statusCode: 200,
    body: JSON.stringify({ comment: response.completion }),
  };
};

What you would normally do

# 1️⃣ Type‑check
npx tsc --noEmit

# 2️⃣ Bundle (often with esbuild or webpack)
npx esbuild src/handler.ts --bundle --platform=node --target=node22.0 --outfile=dist/handler.js

# 3️⃣ Zip and upload to Lambda
zip -j lambda.zip dist/handler.js
aws lambda update-function-code --function-name ReviewLambda --zip-file fileb://lambda.zip

Two separate commands, two passes, and the final handler.js still contains the type‑only import for APIGatewayProxyEvent. The compile step becomes a noticeable delay in CI/CD pipelines.

Combining esbuild with tsc --noEmit for Type‑Safe Bundles

esbuild is a fast bundler written in Go; it can also strip type‑only imports automatically. By running tsc with --noEmit first, we let the TypeScript compiler do what it does best—verify that every variable matches its declared shape—without producing any JavaScript files. Then we hand the same source files to esbuild, which creates a tiny, ready‑to‑run bundle.

Why this works

tsc --noEmit stops after the type‑checking phase, so you still get all the safety guarantees.
esbuild reads the same files, sees the already‑validated types, and produces JavaScript in a fraction of the time.
The pipeline removes dead code such as import type { … } statements, shrinking the bundle.

Key takeaway – you keep the type safety you love while letting esbuild do the heavy lifting of creating the final artifact.

Step‑by‑step Setup

Create a tsconfig.json that tells tsc to only check.

{
  "compilerOptions": {
    "target": "ES2022",
    "module": "ESNext",
    "strict": true,
    "noEmit": true,               // <-- important: do not write .js files
    "skipLibCheck": true,
    "forceConsistentCasingInFileNames": true
  },
  "include": ["src/**/*.ts"]
}

Add an npm script that runs both tools.

{
  "scripts": {
    "type-check": "tsc",
    "bundle": "esbuild src/handler.ts --bundle --platform=node --target=node22.0 --outfile=dist/handler.js --experimental-strip-types",
    "build": "npm run type-check && npm run bundle"
  }
}

The --experimental-strip-types flag tells esbuild to delete any leftover type annotations that might have survived the bundling step (more on that later).

Run the single npm run build command. You’ll see a type‑check first, then a lightning‑fast bundling step that finishes in under a second for a typical Lambda.

Complete Code Block Showing the Build Script

// package.json (relevant part)
{
  "name": "claude-lambda",
  "version": "1.0.0",
  "type": "module",
  "scripts": {
    // Verify types, then produce a tiny bundle
    "type-check": "tsc",
    "bundle": "esbuild src/handler.ts \\
      --bundle \\
      --platform=node \\
      --target=node22.0 \\
      --outfile=dist/handler.js \\
      --experimental-strip-types",
    "build": "npm run type-check && npm run bundle"
  },
  "dependencies": {
    "@anthropic-ai/sdk": "^1.2.0",
    "@aws-sdk/client-lambda": "^3.600.0"
  },
  "devDependencies": {
    "esbuild": "^0.21.0",
    "typescript": "^5.4.5"
  }
}

Explanation

type-check runs the compiler without writing files.
bundle calls esbuild with the --experimental-strip-types flag (more in the next section).
build chains the two, guaranteeing that you never ship code that failed type‑checking.

Stripping Types at Runtime with --experimental-strip-types

Even though we asked tsc not to emit JavaScript, some type‑only imports can slip into the final bundle if we’re not careful. For example, a statement like import { type Request } from "./types" is removed by esbuild, but a value‑side import that only contains types can be mistakenly kept if the code references it in a way the bundler thinks is a runtime use.

The `satisfies` operator as a safety net

The satisfies operator (added in TypeScript 4.9) lets you assert that a value matches a given type without changing the inferred type of the value. When you write:

const payload = {
  diff: event.body?.diff ?? "",
} satisfies ReviewRequest;

Why: You get compile‑time validation that payload has the shape expected by the Claude SDK.
How: Because the operator does not emit any code, the check disappears after the --experimental-strip-types step, leaving zero runtime overhead.

Full Example with `satisfies`

// src/types.ts
export interface ReviewRequest {
  /** The raw git diff that needs a review */
  diff: string;
}

// src/handler.ts
import { APIGatewayProxyEvent, APIGatewayProxyResult } from "aws-lambda";
import { Anthropic } from "@anthropic-ai/sdk";
import type { ReviewRequest } from "./types"; // type‑only import, will be stripped

export const handler = async (
  event: APIGatewayProxyEvent
): Promise<APIGatewayProxyResult> => {
  // Parse incoming JSON safely
  const body = JSON.parse(event.body ?? "{}");

  // Use `satisfies` to make sure the shape matches ReviewRequest
  const request = {
    diff: body.diff ?? "",
  } satisfies ReviewRequest; // <-- compile‑time only, removed later

  // -----------------------------------------------------------------
  // The rest of the function talks to Claude – see next section
  // -----------------------------------------------------------------
  const client = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY! });

  const response = await client.completions.create({
    model: "claude-3-sonnet-20240229",
    prompt: `Please review the following diff and suggest improvements:\n${request.diff}`,
    max_tokens: 512,
  });

  return {
    statusCode: 200,
    body: JSON.stringify({ comment: response.completion }),
  };
};

Tip – satisfies is perfect for validating request payloads that come from the outside world (API Gateway, SQS, etc.) because it does not affect the runtime value.

What can go wrong without it?

If you accidentally wrote:

import { ReviewRequest } from "./types"; // not `type` import

esbuild would keep the import, increasing bundle size, and the code would try to require a file that only contains TypeScript interfaces, causing a runtime error in Lambda. Using type imports or the satisfies pattern prevents that silent breakage.

Calling Claude from a Lambda with Type‑Safe SDK Commands

Now that the build pipeline is fast and lean, let’s focus on the actual work: sending a diff to Claude (Anthropic’s LLM) and returning a comment.

Why a typed SDK matters

The @anthropic-ai/sdk package ships with full TypeScript definitions. When you call client.completions.create, the compiler can verify that you provide every required field (model, prompt, max_tokens, …). That prevents a costly API error that would otherwise appear only after the Lambda runs.

Full Lambda Code (ready to copy)

// src/handler.ts
import { APIGatewayProxyEvent, APIGatewayProxyResult } from "aws-lambda";
import { Anthropic } from "@anthropic-ai/sdk";
import type { ReviewRequest } from "./types";

/**
 * Lambda entry point.
 * Receives a JSON body `{ "diff": "...git diff..." }`,
 * asks Claude for a review, and returns `{ "comment": "..." }`.
 */
export const handler = async (
  event: APIGatewayProxyEvent
): Promise<APIGatewayProxyResult> => {
  // 1️⃣ Guard against missing body
  if (!event.body) {
    return { statusCode: 400, body: JSON.stringify({ error: "No body" }) };
  }

  // 2️⃣ Parse and validate payload using `satisfies`
  const raw = JSON.parse(event.body);
  const payload = {
    diff: raw.diff ?? "",
  } satisfies ReviewRequest;

  // 3️⃣ Prepare the Anthropic client – reads API key from environment
  const anthropic = new Anthropic({
    // The SDK expects a plain string; we assert its existence at runtime
    apiKey: process.env.ANTHROPIC_API_KEY!,
  });

  // 4️⃣ Build the prompt – keep it short to stay within token limits
  const prompt = `You are a code reviewer bot. Review this diff and suggest any improvements or fixes.\n\n${payload.diff}`;

  // 5️⃣ Call Claude – type‑checked arguments
  const completion = await anthropic.completions.create({
    model: "claude-3-sonnet-20240229", // model name must be exact
    max_tokens: 512,                   // limit response size for cost control
    prompt,
  });

  // 6️⃣ Return the comment as JSON
  return {
    statusCode: 200,
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({ comment: completion.completion }),
  };
};

Explanation of each line

Line	Reason
`import { APIGatewayProxyEvent, APIGatewayProxyResult } from "aws-lambda"`	Types that describe the shape of the incoming request and outgoing response; they disappear after bundling.
`import { Anthropic } from "@anthropic-ai/sdk"`	The real client that will make HTTP calls to Claude.
`type ReviewRequest` import	Only used for compile‑time checks; removed by esbuild.
`if (!event.body) …`	Defensive programming – Lambda should return a clear 400 when the caller forgets to send data.
`payload satisfies ReviewRequest`	Guarantees the object matches the expected interface without emitting extra code.
`new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY! })`	Reads the secret from Lambda env vars; the `!` tells TypeScript we are sure it exists (otherwise it would be `string
{% raw %}`prompt` construction	Simple string interpolation; you could add more context here if you like.
`anthropic.completions.create`	Typed call – the compiler warns if a required field is missing.
`return { … }`	Sends back JSON with the comment. The `Content-Type` header is required for API Gateway to treat it as JSON.

Gotchas specific to the SDKs

@anthropic-ai/sdk expects the model name exactly as listed in the console; a typo results in a 400 response that shows up as an unhandled exception in Lambda.
@aws-sdk/client-lambda (if you later need to invoke other Lambdas) runs into a Node 22 issue where require('esm') in a layer silently fails. The fix is to avoid the layer or switch to an ES‑module‑compatible version of the SDK.

In plain English – the code above stays within the safe zone of both SDKs: it uses only ES‑module imports, reads secrets from environment variables, and respects API limits.

Observability with diagnostics_channel and Zero‑Cost Metrics

Running a Lambda that talks to an external LLM can be expensive if you don’t know how often it’s invoked or how long Claude takes to respond. Adding heavy‑weight monitoring libraries defeats the purpose of a tiny bundle. Instead, we can use Node’s built‑in diagnostics_channel to emit lightweight events that CloudWatch can capture without adding code size.

What is `diagnostics_channel`?

A core module that lets you create a named channel and publish arbitrary data. Other parts of your system (or a CloudWatch subscription) can listen and log it. Because it’s built into Node, there is no extra dependency.

Adding a simple timer

// src/metrics.ts
import { channel } from "node:diagnostics_channel";

/**
 * A channel named "claude-lambda" that emits timing info.
 * Listeners can subscribe to this channel in CloudWatch Logs Insights.
 */
export const claudeChannel = channel("claude-lambda");

// Helper to measure async functions
export async function withTiming<T>(label: string, fn: () => Promise<T>): Promise<T> {
  const start = Date.now();
  try {
    const result = await fn();
    return result;
  } finally {
    const durationMs = Date.now() - start;
    // Emit an object – listeners can filter by `label`
    claudeChannel.publish({ label, durationMs });
  }
}

Now wrap the Claude call:

import { withTiming } from "./metrics";

// inside handler
const completion = await withTiming("anthropic-call", async () => {
  return anthropic.completions.create({
    model: "claude-3-sonnet-20240229",
    max_tokens: 512,
    prompt,
  });
});

Key takeaway – you get millisecond‑level visibility without pulling in a big monitoring SDK, keeping the bundle under 100 KB.

Hooking the channel into CloudWatch (quick tip)

Create a CloudWatch Log Group for the Lambda.
Set a subscription filter that matches "claude-lambda" JSON.
In CloudWatch Insights run:

fields @timestamp, @message
| filter @message like /claude-lambda/
| parse @message "*label\":\"*\",*durationMs\":*}" as label, duration
| stats avg(duration) as avgMs, count() as calls by label

You’ll see average latency per label, letting you spot spikes in Claude response time.

The Takeaway

You now have a repeatable pattern for building ultra‑fast, type‑safe Lambdas that call Claude.

Run tsc --noEmit first; it guarantees the code matches all declared types without writing files.
Hand the same sources to esbuild with --experimental-strip-types; the bundle is tiny and runs without any type‑related overhead.
Use the satisfies operator to validate request shapes while keeping the runtime code clean.
The @anthropic-ai/sdk client works smoothly once you feed it a properly typed request and a correct model name.
Capture lightweight observability data with diagnostics_channel to avoid bulky monitoring dependencies.
Remember the SDK‑specific gotchas: avoid require('esm') in Node 22 layers and be mindful of API‑gateway response headers for streaming.

By following these steps you cut build minutes, shrink Lambda zip size, and keep the safety net of TypeScript—all while getting valuable code‑review suggestions from Claude in real time. Happy coding!

Transparency notice

This article was written with the help of an AI system — Groq (GPT OSS 120B).

Published: 2026-09-16 · Primary focus: TypeScriptBuild

All code blocks are intended to be correct and runnable, but please verify them
against the TypeScript docs before using in production.

Find an error? Drop a comment — corrections are always welcome.

DEV Community: Dinesh_gowtham

AI Pair Programming with Lambda: Build a Claude‑Powered Coding Assistant and Enable Prompt Caching

Why AI Pair Programming Needs More Than Autocomplete

The missing piece: stateful prompting

Code: Minimal Lambda that just echoes a prompt (no caching yet)

Setting Up a Secure Claude API Call in Lambda

Why use Secrets Manager?

Gotcha: Secrets Manager charges per secret and per retrieval

Code: Securely read the Claude key and call the API

Implementing Prompt Caching with Native TypeScript Stripping

Why store the prompt on disk?

Analogy: Warm kitchen vs. cold pantry

Gotcha: Cold starts wipe /tmp

Code: Prompt cache helper

Connecting the Lambda via API Gateway for Real‑Time Editing

Why streaming matters

Gotcha: Response streaming needs explicit headers

Code: Full Lambda handler with caching and streaming

Testing the Assistant Locally with Node.js Test Runner

Why local testing matters

Gotcha: Mocking @aws-sdk/client-secrets-manager

Code: Minimal test file (handler.test.ts)

The Takeaway

Complete, Runnable Code Example

src/promptCache.ts

src/claudeClient.ts

src/handler.ts

src/handler.test.ts (optional, see earlier)

Next Steps

How Cursor AI’s Agent Mode Works Under the Hood – A Step‑By‑Step Guide with Node.js 22 Native TypeScript Stripping

What Agent Mode Actually Does

Minimal request shape

The Plan‑Act‑Observe Loop in Cursor AI

Step‑by‑step breakdown

Analogy

Code: orchestrating one iteration

Building a Minimal CLI with Node.js 22

Required SDKs

Project layout

Code: CLI skeleton

How the CLI uses native TypeScript stripping

Running the Loop with Native TypeScript Stripping

Step‑by‑step execution

Sample output

Common pitfalls

Testing, Debugging, and Observability

Simple unit test with a mock agent

Logging the loop

Observability in production

The Takeaway

Prompt Testing Pipelines with SQS: How to Version, Run, and Verify LLM Prompts Like Unit Tests

Why Prompt Testing Matters

The simplest test scenario

Storing Prompt Versions in SQS

Why a queue, not a plain file?

Setting up the queue

Adding a prompt version

Automating Prompt Execution with Lambda

Why Lambda?

Lambda handler skeleton

Guarding against duplicate processing

Asserting LLM Outputs with Node:test

Why a test framework?

Minimal test file

Integrating into CodeBuild Pipelines

Why CodeBuild?

CodeBuild project definition (YAML)

Handling environment secrets

Dealing with batch failures

The Takeaway

Prompt Caching with Claude in Node.js: How to Cut Latency and Save Tokens, Explained Simply

Why Prompt Caching Matters for Claude

The deterministic promise

Designing a Deterministic Prompt Signature

Minimal example of building a signature

Implementing the Cache with diagnostics_channel and an In‑Memory Store

Why diagnostics_channel?

The in‑memory LRU cache

Putting it together

What the code does, step by step

Gotcha: Cold starts wipe `/tmp`

Gotcha: Mocking `@aws-sdk/client-secrets-manager`

Code: Minimal test file (`handler.test.ts`)

`src/promptCache.ts`

`src/claudeClient.ts`

`src/handler.ts`

`src/handler.test.ts` (optional, see earlier)

How Cursor AI’s Agent Mode Works Under the Hood – A Step‑By‑Step Guide with Node.js 22 Native TypeScript Stripping

What Agent Mode Actually Does

The Plan‑Act‑Observe Loop in Cursor AI

Building a Minimal CLI with Node.js 22

Implementing the Cache with `diagnostics_channel` and an In‑Memory Store

Why `diagnostics_channel`?

Packaging for Node.js 22

Adding the CloudWatch metric via `diagnostics_channel`

Why the `ReturnValues='ALL_OLD'` gotcha matters

Observing Agent‑DB Interactions via `diagnostics_channel`

Gotcha: Lambda layers and ESM in Node 22

Minimal webhook code (Node 22, using `@aws-sdk/client-eventbridge`)