Lambda response streaming: live AI answers from Bedrock

Lambda response streaming is, for me, the difference between an AI feature that feels sluggish and one that feels alive. If you have ever stared at an empty screen for eight seconds until a language model had finished its complete answer, you know what I mean. The model has been writing all along; nobody gets to see it.

In this article I will show you how a Lambda function passes Claude’s answer from Amazon Bedrock to the caller piece by piece, as soon as the first words are available. We build everything in TypeScript, deploy with the Serverless Framework and test with curl and a small Node script. We will also look at where things get tricky: errors after the first byte, costs when connections drop, and the way into the browser.

In short

With response streaming, a Lambda function sends its answer to the client while it is still working, instead of buffering it completely first. For AI answers from Bedrock this means the first words appear after a fraction of a second rather than after the full generation time. The biggest pitfall: the HTTP status is already sent when something fails later, and the function keeps running even after the client has left.

What you will learn in this article

• When response streaming pays off and when a normal Lambda invocation is enough
• How to write a streaming handler with streamifyResponse and Bedrock ConverseStream
• How to switch the function URL to streaming in the Serverless Framework (and in CDK)
• How to read the stream with curl, the AWS SDK and in the browser
• How to keep errors, aborted connections and costs under control

What is Lambda response streaming?

By default, Lambda works in buffered mode: the function computes, assembles the complete response in memory, returns it, and only then is it sent to the caller. That fits most APIs, because a JSON document with a few fields is ready quickly. A language model (LLM, large language model) is different. It produces text token by token, meaning in small word fragments, and a longer answer easily takes several seconds.

With response streaming, your handler additionally receives a writable stream. Everything you write into it goes straight to the client. The metric that improves is called time to first byte (TTFB): the time until the first byte of the response reaches the user. The total duration stays the same, but the application feels much faster because something happens right away.

According to the AWS documentation, there are a few key limits you should know:

Property	Buffered (BUFFERED)	Streamed (RESPONSE_STREAM)
Maximum response size	6 MB	200 MB
Bandwidth	not relevant	first 6 MB uncapped, then max. 2 MBps
Runtimes	all	Node.js natively, others via custom runtime or Lambda Web Adapter
Invocation paths	Invoke, function URL, API Gateway	Function URL, InvokeWithResponseStream, API Gateway (REST, proxy integration)
Function URL inside a VPC	yes	no

For text from a language model, the bandwidth limit hardly matters, because even long answers stay far below 6 MB. The runtime row is more interesting: natively, Lambda response streaming only works with Node.js. That makes TypeScript the obvious choice here.

When does Lambda response streaming pay off?

Streaming is not an end in itself. I reach for it when at least one of these cases applies:

• AI answers for humans: chat, assistants, summaries. This is where the effect is biggest.
• Large responses: exports or reports over 6 MB that would not fit through in buffered mode.
• Long processing with progress: the client should see progress instead of running into a timeout.

For a classic REST API that returns a small JSON document in 80 milliseconds, streaming brings nothing except extra complexity. The same applies when another system processes the answer automatically and waits for the end anyway. A batch job that generates product descriptions does not need a stream.

🎬 How Lambda response streaming works with Amazon Bedrock – this animated diagram only runs on my site: watch it on sascha-ms.de

The example: a tour advisor for a bike rental shop

As an example I use a fictional bike rental shop at Lake Constance. Customers ask a question on the website, such as “Which tour suits families with children?”, and a small AI tour advisor answers. The architecture is deliberately lean: one Lambda function with a function URL that calls Claude via the Bedrock Converse API and passes the answer through.

A function URL is a dedicated HTTPS endpoint attached directly to the function, without API Gateway. To start with, we protect it with IAM authentication. How the browser can still get access is covered further below.

Prerequisites

• An AWS account with permissions for Lambda, IAM, CloudFormation and Bedrock
• Access to a Claude model enabled in the Bedrock console for your region (here: Frankfurt, eu-central-1)
• Node.js 24 and npm installed locally
• Serverless Framework v4 (npm i -g serverless) with configured AWS credentials
• curl version 7.75 or later for signed requests (--aws-sigv4)

Lambda response streaming step by step

Step 1: Create the project

First we create an empty project and install the dependencies. The Bedrock SDK is already included in the Lambda Node.js runtime, but we still need it locally for types and tests.

mkdir touradvisor && cd touradvisor
npm init -y
npm install @aws-sdk/client-bedrock-runtime
npm install -D typescript tsx @types/node @types/aws-lambda @aws-sdk/client-lambda

Afterwards, the package.json looks roughly like this:

{
  "name": "touradvisor-streaming",
  "version": "1.0.0",
  "private": true,
  "type": "module",
  "scripts": {
    "typecheck": "tsc --noEmit",
    "deploy": "serverless deploy",
    "ask": "tsx client/ask.ts"
  },
  "dependencies": {
    "@aws-sdk/client-bedrock-runtime": "^3.920.0"
  },
  "devDependencies": {
    "@aws-sdk/client-lambda": "^3.920.0",
    "@types/aws-lambda": "^8.10.152",
    "@types/node": "^24.0.0",
    "tsx": "^4.20.0",
    "typescript": "^5.9.0"
  }
}

Add a simple tsconfig.json. The Serverless Framework builds with esbuild later; TypeScript only checks the types:

{
  "compilerOptions": {
    "target": "ES2023",
    "module": "NodeNext",
    "moduleResolution": "NodeNext",
    "strict": true,
    "skipLibCheck": true,
    "noEmit": true
  },
  "include": ["src", "client"]
}

Step 2: Write the streaming handler

Now for the core. The handler is wrapped with awslambda.streamifyResponse(). The Lambda runtime provides the awslambda object globally, and @types/aws-lambda supplies the types. Besides event and context, the handler receives the responseStream that we write into.

// src/handler.ts
import { Readable } from 'node:stream';
import { pipeline } from 'node:stream/promises';
import type { Context, LambdaFunctionURLEvent } from 'aws-lambda';
import {
  BedrockRuntimeClient,
  ConverseStreamCommand,
} from '@aws-sdk/client-bedrock-runtime';

const bedrock = new BedrockRuntimeClient({});
const MODEL_ID = process.env.MODEL_ID ?? '';
const MAX_QUESTION = 2000; // characters

const SYSTEM_PROMPT =
  'You are the tour advisor of a bike rental shop at Lake Constance. ' +
  'Answer in English, friendly and concise (at most 150 words). ' +
  'If you are not sure about something, say so openly.';

type Question = { question: string };

// Reads the question from the request body (function URL or direct invoke)
export function readQuestion(event: Pick<LambdaFunctionURLEvent, 'body' | 'isBase64Encoded'>): Question | string {
  if (!event.body) return 'Please send JSON like {"question": "..."}.';
  const raw = event.isBase64Encoded
    ? Buffer.from(event.body, 'base64').toString('utf8')
    : event.body;
  try {
    const data = JSON.parse(raw) as Partial<Question>;
    const question = typeof data.question === 'string' ? data.question.trim() : '';
    if (!question) return 'The field "question" is missing or empty.';
    if (question.length > MAX_QUESTION) return `The question is longer than ${MAX_QUESTION} characters.`;
    return { question };
  } catch {
    return 'The body is not valid JSON.';
  }
}

// Yields the model's answer piece by piece as text
export async function* answerInChunks(
  question: string,
  signal: AbortSignal,
  client: Pick<BedrockRuntimeClient, 'send'> = bedrock,
): AsyncGenerator<string> {
  try {
    const response = await client.send(
      new ConverseStreamCommand({
        modelId: MODEL_ID,
        system: [{ text: SYSTEM_PROMPT }],
        messages: [{ role: 'user', content: [{ text: question }] }],
        inferenceConfig: { maxTokens: 600, temperature: 0.3 },
      }),
      { abortSignal: signal },
    );

    for await (const event of response.stream ?? []) {
      const text = event.contentBlockDelta?.delta?.text;
      if (text) yield text;

      if (event.messageStop?.stopReason === 'max_tokens') {
        yield '\n\n[Answer truncated: token limit reached]';
      }
      if (event.metadata?.usage) {
        const { inputTokens, outputTokens } = event.metadata.usage;
        console.log(JSON.stringify({ msg: 'usage', inputTokens, outputTokens }));
      }
    }
  } catch (error) {
    // Status code 200 is already on its way, so mark the error clearly in the text
    console.error('Streaming aborted', error);
    yield '\n\n[Error: The answer could not be generated completely.]';
  }
}

export const handler = awslambda.streamifyResponse(
  async (event: LambdaFunctionURLEvent, responseStream, context: Context) => {
    const result = readQuestion(event);
    const isError = typeof result === 'string';

    const stream = awslambda.HttpResponseStream.from(responseStream, {
      statusCode: isError ? 400 : 200,
      headers: {
        'Content-Type': 'text/plain; charset=utf-8',
        'Cache-Control': 'no-cache',
      },
    });

    if (isError) {
      await pipeline(Readable.from([result]), stream);
      return;
    }

    // Abort cleanly five seconds before the Lambda timeout
    const abort = new AbortController();
    const timer = setTimeout(
      () => abort.abort(),
      Math.max(context.getRemainingTimeInMillis() - 5000, 1000),
    );

    try {
      await pipeline(
        Readable.from(answerInChunks(result.question, abort.signal)),
        stream,
      );
    } finally {
      clearTimeout(timer);
    }
  },
);

A few spots deserve a second look. HttpResponseStream.from() sets the status code and headers before the first byte goes out. That is your only chance to do so; you cannot change the status later.

The function answerInChunks is an async generator. It reads the events from ConverseStream and only passes on the text pieces from contentBlockDelta. The metadata event at the end contains the consumed tokens, which we write to the log. That way you can see in CloudWatch what each request cost.

For writing I use pipeline() instead of individual write() calls. AWS recommends this too, because pipeline() handles backpressure: if the client is slower than the model, the generator waits instead of filling up memory. In addition, pipeline() ends the stream automatically.

Step 3: The serverless.yml

In the Serverless Framework, two lines under url are enough to switch the function URL to streaming. Without invokeMode, the framework assumes BUFFERED, and the answer arrives in one piece despite the streaming code.

# serverless.yml
service: touradvisor
frameworkVersion: '4'

provider:
  name: aws
  runtime: nodejs24.x
  architecture: arm64
  region: eu-central-1
  memorySize: 512
  timeout: 60
  logRetentionInDays: 14
  environment:
    # EU inference profile; available IDs are listed in the Bedrock console
    MODEL_ID: eu.anthropic.claude-sonnet-4-5-20250929-v1:0
  iam:
    role:
      statements:
        - Effect: Allow
          Action:
            - bedrock:InvokeModelWithResponseStream
          Resource:
            - arn:aws:bedrock:${aws:region}:${aws:accountId}:inference-profile/eu.anthropic.*
            - arn:aws:bedrock:*::foundation-model/anthropic.*

functions:
  ask:
    handler: src/handler.handler
    url:
      authorizer: aws_iam
      invokeMode: RESPONSE_STREAM

The IAM action for ConverseStream is bedrock:InvokeModelWithResponseStream. Because the model ID is a cross-region inference profile (recognizable by the eu. prefix), the role needs permissions on the profile and on the models in the destination regions. Check the exact ID in the Bedrock console, as model versions change regularly.

Step 4: Deploy and test with curl

npx tsc --noEmit
serverless deploy

The function URL is printed at the end of the output. Since it is protected with IAM, we sign the request with our own credentials. The --no-buffer flag is important; otherwise curl holds back the output until everything has arrived:

export URL="https://<your-url-id>.lambda-url.eu-central-1.on.aws/"

curl --no-buffer "$URL" \
  --aws-sigv4 "aws:amz:eu-central-1:lambda" \
  --user "$AWS_ACCESS_KEY_ID:$AWS_SECRET_ACCESS_KEY" \
  -H "x-amz-security-token: $AWS_SESSION_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"question":"Which tour suits families with children?"}'

If everything works, the answer trickles into the terminal sentence by sentence. If it arrives in one piece, invokeMode is almost always missing or --no-buffer was left out. By the way, the Lambda console does not help with testing here: it always displays responses as buffered.

Step 5: Read the stream in your own code

For tests, scripts or other backends there is the direct route via the AWS SDK with InvokeWithResponseStream. It returns an event stream with PayloadChunk pieces and a final InvokeComplete.

// client/ask.ts
import {
  LambdaClient,
  InvokeWithResponseStreamCommand,
} from '@aws-sdk/client-lambda';

const FUNCTION = process.env.FUNCTION_NAME ?? 'touradvisor-dev-ask';
const question = process.argv.slice(2).join(' ') || 'Which tour suits families with children?';

const lambda = new LambdaClient({});
const decoder = new TextDecoder();

// HttpResponseStream.from puts a JSON prelude plus eight null bytes
// in front of the actual data. Function URLs remove it, the SDK does not.
function preludeStripper() {
  let buffer = new Uint8Array(0);
  let done = false;
  return (chunk: Uint8Array): Uint8Array => {
    if (done) return chunk;
    const merged = new Uint8Array(buffer.length + chunk.length);
    merged.set(buffer);
    merged.set(chunk, buffer.length);
    buffer = merged;
    for (let i = 0; i + 8 <= buffer.length; i++) {
      if (buffer.subarray(i, i + 8).every((b) => b === 0)) {
        done = true;
        return buffer.subarray(i + 8);
      }
    }
    return new Uint8Array(0);
  };
}

const response = await lambda.send(
  new InvokeWithResponseStreamCommand({
    FunctionName: FUNCTION,
    Payload: JSON.stringify({ body: JSON.stringify({ question }) }),
  }),
);

const stripPrelude = preludeStripper();
for await (const event of response.EventStream ?? []) {
  if (event.PayloadChunk?.Payload) {
    const data = stripPrelude(event.PayloadChunk.Payload);
    process.stdout.write(decoder.decode(data, { stream: true }));
  }
  if (event.InvokeComplete) {
    const { ErrorCode, ErrorDetails } = event.InvokeComplete;
    if (ErrorCode) console.error(`\nError: ${ErrorCode} ${ErrorDetails ?? ''}`);
  }
}
process.stdout.write('\n');

One detail that is easy to miss: HttpResponseStream.from() writes the metadata (status, headers) as a JSON prelude into the stream, separated by eight null bytes. The function URL evaluates this prelude and removes it. If you invoke the function directly via the SDK, it ends up in your data instead. That is exactly what preludeStripper() is for.

FUNCTION_NAME=touradvisor-dev-ask npx tsx client/ask.ts "How long is the loop around the Untersee?"

And with AWS CDK?

If you use CDK instead of the Serverless Framework, the switch is just as short. The handler stays identical; only the function URL gets the invokeMode:

// lib/tour-advisor-stack.ts (excerpt)
import { Duration } from 'aws-cdk-lib';
import * as lambda from 'aws-cdk-lib/aws-lambda';
import { NodejsFunction } from 'aws-cdk-lib/aws-lambda-nodejs';

const ask = new NodejsFunction(this, 'Ask', {
  entry: 'src/handler.ts',
  runtime: lambda.Runtime.NODEJS_24_X,
  architecture: lambda.Architecture.ARM_64,
  memorySize: 512,
  timeout: Duration.seconds(60),
});

const url = ask.addFunctionUrl({
  authType: lambda.FunctionUrlAuthType.AWS_IAM,
  invokeMode: lambda.InvokeMode.RESPONSE_STREAM,
});

Add the Bedrock permissions as usual with ask.addToRolePolicy(). Which tool I pick and when is described in my comparison Serverless Framework vs. CDK vs. SST.

How does the browser get to the stream?

A function URL with IAM protection is ideal for server-to-server calls, but impractical in the browser, where (fortunately) no AWS keys live. Still, I would not put a public URL without authentication in front of a language model: every call costs money, and a script on the internet will find such an endpoint faster than you would like.

In practice there are two good options. First, CloudFront in front of the function URL with origin access control (OAC). CloudFront then signs the requests to Lambda, and you can add rate limits with AWS WAF in front. For POST requests, however, the client must send the SHA-256 hash of the body in the x-amz-content-sha256 header. Second, an API Gateway REST API with Lambda proxy integration and the response transfer mode STREAM. There you get authorizers, API keys and usage plans. Mind the limits: a maximum stream duration of 15 minutes, and caching as well as VTL transformations do not work in streaming mode.

In the frontend itself you do not need a library. fetch() delivers the body as a ReadableStream that you read chunk by chunk:

// web/read-stream.ts
export async function askQuestion(url: string, question: string, target: HTMLElement) {
  const response = await fetch(url, {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ question }),
  });
  if (!response.ok || !response.body) {
    target.textContent = `Error ${response.status}`;
    return;
  }
  const reader = response.body.pipeThrough(new TextDecoderStream()).getReader();
  target.textContent = '';
  for (;;) {
    const { value, done } = await reader.read();
    if (done) break;
    target.textContent += value;
  }
}

For a chat with several event types (text, errors, token usage), a structured format such as server-sent events (text/event-stream) is worth it. For a simple tour advisor, plain text is enough.

Common mistakes when streaming

Errors after the first byte

The status code is gone as soon as the stream starts. If Bedrock fails in the middle of an answer, for example due to throttling (throttlingException), you can no longer send a 500. That is why answerInChunks catches errors itself and writes a clearly recognizable marker into the text. If machines process the output, choose a format with its own error event, such as server-sent events.

The function keeps running after the client has left

If someone closes the browser tab, Lambda does not notice. The documentation is clear about it: the stream is not interrupted, and you pay for the full duration. So limit the answer length via maxTokens, set a realistic timeout and abort the Bedrock call with an AbortController shortly before the Lambda timeout, as shown in the handler.

Unresolved promises from Node.js 24 on

Starting with Node.js 24, Lambda stops waiting for pending promises once the handler or the stream is done. A log shipment or metrics call that you only start without await may get lost. Anything that has to happen after the stream belongs in the handler and must be awaited.

Function URL inside a VPC

If your function is attached to a VPC, the function URL does not stream. Then your options are invoking via InvokeWithResponseStream with an interface VPC endpoint for Lambda, or putting API Gateway in front.

Costs and security

Streaming does not make the model cheaper. On Bedrock you pay per input and output token, streamed or not. On Lambda you pay for duration, and with AI calls that is mostly waiting time for the model. An answer that takes six seconds to generate costs six seconds of Lambda duration. On top comes a fee for the streamed data volume, which hardly matters for plain text. With 512 MB on ARM this is little per call, but it adds up with many users. You can quickly play through your scenario with the Lambda cost calculator.

On the security side: limit the input (2,000 characters in the example), set maxTokens, log token usage and protect the endpoint. A budget alarm in AWS Budgets takes five minutes and saves you nasty surprises. How to cap tokens per user or agent is shown in Token budget for AI agents on Amazon Bedrock, and you can make recurring system prompts cheaper with prompt caching on Bedrock.

My take

I consider Lambda response streaming one of the underrated Lambda features. It has been around for years, but only AI applications turned it from a niche topic into a standard tool. For every interface where a person waits for generated text, I would stream today. The effort is small; the difference for users is large.

Where I disagree: streaming is often sold as a performance optimization. That is only true for perceived speed. Compute time, costs and total duration stay the same. If you want to make a slow application faster, you still have to work on the prompt, the model choice and the answer length. And error handling gets harder rather than easier with streaming, because the clean HTTP status is no longer available as a signal.

FAQ

Which runtimes support Lambda response streaming?

Natively, only the managed Node.js runtimes. For Python, Java and other languages it works via a custom runtime or the Lambda Web Adapter, which can stream a FastAPI or Spring Boot project, for example.

Can API Gateway stream Lambda responses?

Yes, REST APIs with Lambda proxy integration support the response transfer mode STREAM. Separate limits apply, such as a maximum stream duration of 15 minutes, and caching and VTL transformations are not possible.

How large can a streamed Lambda response be?

Up to 200 MB instead of 6 MB for buffered responses. The first 6 MB go out uncapped, after that at a maximum of 2 MBps.

Is streaming more expensive than a normal Lambda invocation?

Duration costs the same, plus a small fee for the streamed data volume. It mainly gets expensive when dropped connections keep the function running. So limit answer length and timeout.

Your next step

Take an existing Lambda function that currently returns an AI answer in buffered mode and switch it to streaming with the five steps above. Measure the time to the first word before and after, for example with curl -w "%{time_starttransfer}". The difference is usually bigger than expected. If you need structured outputs, Bedrock structured outputs with Claude and TypeScript covers the counterpart.

Now I am curious: do you already stream AI answers in your projects, and if so, via function URL, API Gateway or something else entirely? Let me know in the comments.

Sources

• AWS Lambda: Response streaming for Lambda functions
• AWS Lambda: Writing response streaming-enabled Lambda functions
• AWS Lambda: Invoking a response streaming function using function URLs
• AWS Lambda: Tutorial – Creating a response streaming Lambda function with a function URL
• Amazon Bedrock API Reference: ConverseStream
• Amazon API Gateway: Response streaming for REST APIs
• Serverless Framework: AWS Lambda Functions (Function URLs)

This post was created with AI assistance.

---

👉 Next step: Curious what your own Lambda functions cost? Try my Lambda cost calculator – it runs right in your browser.

📬 Stay updated: New articles on serverless, AI and leading tech teams via RSS, browser push or the weekly newsletter (in German).

Originally published on sascha-ms.de. More articles on serverless, AI in everyday development and leading tech teams: sascha-ms.de/en