Developers keep asking why LLM responses feel sluggish when they have to wait for the whole answer. With Bedrock’s streaming mode and Node 22’s native fetch, you can push tokens to the browser the moment they’re generated. This guide shows you how to wire a Lambda‑backed SSE endpoint in minutes.
Why Streaming Matters
When you ask a large language model (LLM) like Claude a question, the model doesn’t write the whole paragraph in one go. It creates tokens—tiny pieces of text such as a word or punctuation—one after another. Think of a token stream like a faucet: water (tokens) drips out continuously instead of a bucket dumping all at once.
If your application waits for the bucket, the user sees a blank screen for a few seconds, then a sudden flash of the complete answer. That pause hurts perceived performance, especially in chat‑like interfaces where users expect an immediate “typing…” indicator.
Streaming lets you:
- Show each token as soon as it arrives, giving the illusion of a live conversation.
- Reduce overall latency because the client can start processing before the model finishes.
- Keep your server’s memory footprint low; you never have to hold the full response in RAM.
In plain English: Streaming is like watching a movie as it’s filmed, rather than waiting for the whole film to be edited before the lights come up.
The hidden gotcha
Bedrock will only send a token‑by‑token stream when you set the stream flag to true in the request payload. Forgetting this flag makes the service behave like a classic HTTP request, returning the entire completion in a single JSON object. The downstream Server‑Sent Events (SSE) logic then never sees any incremental data, and the browser sits idle.
Setting Up the Bedrock Client
Before you can ask Claude for tokens, you need a client—a small piece of code that knows how to talk to the Bedrock Runtime API. The official AWS SDK for JavaScript provides @aws-sdk/client-bedrock-runtime. Installing it is straightforward:
npm install @aws-sdk/client-bedrock-runtime
Why we use the SDK at all
The SDK handles signing the request with your AWS credentials, retrying transient failures, and exposing a clean TypeScript interface. It saves you from manually crafting the Authorization header for every call.
Minimal setup
// src/bedrockClient.ts
import { BedrockRuntimeClient } from "@aws-sdk/client-bedrock-runtime";
/**
* Create a Bedrock client that will be reused across Lambda invocations.
* Re‑using the client avoids the overhead of creating a new HTTP connection each time.
*/
export const bedrockClient = new BedrockRuntimeClient({
// Region where your Bedrock model lives. Change if you deployed to a different region.
region: "us-east-1",
// Optional: increase timeout if you expect long generations.
requestHandler: undefined, // default NodeHttpHandler is fine for most cases
});
Tip: Keep the client in a module‑level variable (outside the handler) so that AWS re‑uses the underlying TCP socket across cold‑starts.
Known Bedrock gotchas
-
Token limits per minute – Bedrock enforces a limit on how many tokens you can generate in a minute, not per request. If you fire many parallel streams you may hit a rate‑limit error (
ThrottlingException). - Manual SSE parsing – The streaming response is a raw HTTP body that contains JSON lines, not a ready‑made EventSource. You have to split the stream yourself.
- Knowledge base sync delay – If you’re using a Knowledge Base with Claude, newly uploaded documents may take a few minutes before they become searchable.
- Model selection matters – Not every Claude model supports streaming. Double‑check the model ID in the Bedrock console.
Using Native fetch for Token Streaming
Node 22 ships with the fetch API built‑in, just like browsers. This means you can call Bedrock with a streaming request without pulling in a third‑party HTTP library.
Why native fetch is handy
- No extra dependencies → smaller deployment package.
- Streams are represented as
ReadableStreamobjects, which work nicely with the SSE format we’ll send to the browser.
The request payload
The body must be a JSON string that tells Bedrock which model to use, what prompt to send, and that we want a stream:
{
"modelId": "anthropic.claude-3-5-sonnet-20240620-v1:0",
"prompt": "Explain why streaming matters in a chat UI.",
"maxTokens": 512,
"temperature": 0.7,
"stream": true
}
In plain English:
stream: trueis the “turn on the faucet” switch.
Full fetch call inside the Lambda handler
// src/handler.ts
import { bedrockClient } from "./bedrockClient";
import { InvokeModelCommand } from "@aws-sdk/client-bedrock-runtime";
/**
* Calls Claude via Bedrock using Node's native fetch.
* Returns a ReadableStream of raw bytes that we will transform to SSE later.
*/
async function callClaudeStream(prompt: string): Promise<ReadableStream<Uint8Array>> {
// Build the command payload exactly as the SDK expects.
const command = new InvokeModelCommand({
// The model identifier from the Bedrock console.
modelId: "anthropic.claude-3-5-sonnet-20240620-v1:0",
// Bedrock expects the body as a stringified JSON object.
body: JSON.stringify({
prompt,
maxTokens: 512,
temperature: 0.7,
// The crucial flag – without it we get a single JSON response.
stream: true,
}),
// Accept a streaming response (text/event-stream).
accept: "application/json",
contentType: "application/json",
});
// The SDK returns a promise that resolves to a response object.
const response = await bedrockClient.send(command);
// Bedrock streams the body as a Uint8Array inside response.body.
// We cast to ReadableStream for TypeScript clarity.
if (!response.body) {
throw new Error("Bedrock returned an empty body – something went wrong.");
}
return response.body as ReadableStream<Uint8Array>;
}
Key takeaway: Using
InvokeModelCommandtogether withstream: truegives you a low‑level byte stream you can pipe straight to the browser.
Additional Bedrock gotchas while streaming
- Back‑pressure handling – If the client disconnects, you must abort the Bedrock request, otherwise you keep paying for tokens you never deliver.
-
Cross‑region latency – If your Lambda lives in
us-west-2but Bedrock is inus-east-1, the round‑trip adds extra milliseconds per token. Deploy the Lambda in the same region when possible.
Building an SSE Endpoint in Lambda
Server‑Sent Events (SSE) is a simple HTTP‑based protocol that lets a server push text chunks to a browser over a single long‑lived connection. The browser receives each chunk as an event.
Why SSE instead of WebSockets?
- SSE works over plain HTTPS, so no extra firewall rules.
- It’s unidirectional (server → client), which matches the “stream tokens to UI” pattern perfectly.
- Browsers have a built‑in
EventSourceAPI, no extra JavaScript libraries needed.
Lambda constraints
AWS Lambda can stream responses, but you must set the Content-Type header to text/event-stream and write to the response object using the callbackWaitsForEmptyEventLoop = false pattern (or the newer async iterator support).
Full Lambda handler
// src/lambdaHandler.ts
import { APIGatewayProxyEventV2, APIGatewayProxyResultV2 } from "aws-lambda";
import { callClaudeStream } from "./handler";
/**
* Lambda entry point wired to API Gateway (HTTP API).
* It receives a JSON body { prompt: string } from the browser,
* calls Bedrock, and streams each token back as SSE.
*/
export const streamHandler = async (
event: APIGatewayProxyEventV2
): Promise<APIGatewayProxyResultV2> => {
// Enable early return – we’ll write to the response manually.
// This tells Lambda not to wait for the event loop to be empty.
(global as any).callbackWaitsForEmptyEventLoop = false;
// Parse the incoming prompt; fall back to a friendly default.
const body = event.body ? JSON.parse(event.body) : {};
const prompt = typeof body.prompt === "string" ? body.prompt : "Hello, Claude!";
// Prepare the HTTP response headers for SSE.
const headers = {
"Content-Type": "text/event-stream; charset=utf-8",
// Prevent caching so browsers always get fresh tokens.
"Cache-Control": "no-cache, no-transform",
// Keep the connection alive.
Connection: "keep-alive",
};
// The `response` object that API Gateway expects.
// We will fill its `body` with a placeholder; the real streaming
// happens by writing directly to the underlying stream.
const response: APIGatewayProxyResultV2 = {
statusCode: 200,
headers,
// Body must be a string when the Lambda finishes, but we will
// never finish the function until the client disconnects.
body: "",
// Tell API Gateway that we will stream the payload.
isBase64Encoded: false,
};
// Obtain the raw Bedrock stream.
const bedrockStream = await callClaudeStream(prompt);
// Create a Transform stream that converts Bedrock JSON lines
// into SSE formatted text: "data: <token>\n\n"
const encoder = new TextEncoder();
// Helper: format a token as an SSE message.
const formatSSE = (data: string) => `data: ${data}\n\n`;
// The `pipeTo` method returns a promise that resolves when the
// source stream ends or an error occurs.
const sseWriter = new WritableStream({
async write(chunk) {
// Bedrock sends JSON objects like: {"type":"token","text":"Hello"}
const text = new TextDecoder().decode(chunk);
// Split on newlines because Bedrock streams a series of JSON lines.
const lines = text.split("\n").filter(Boolean);
for (const line of lines) {
try {
const obj = JSON.parse(line);
if (obj.type === "token") {
// Send only the token text to the browser.
const sse = formatSSE(obj.text);
// Write the SSE string to the Lambda HTTP response.
// `response.body` will be ignored; we push directly to the
// underlying socket via `callback`.
// @ts-ignore – the Lambda runtime provides `responseStream` in the context.
// In the real deployment you use `event.requestContext.http.path`
// with a streaming-enabled API Gateway integration.
// Here we illustrate the core idea.
}
} catch (e) {
// If parsing fails, ignore the line – it might be a heartbeat.
}
}
},
// When the client disconnects or the model finishes, close the stream.
close() {
console.log("SSE stream closed by client or model.");
},
abort(err) {
console.error("SSE stream aborted:", err);
},
});
// Pipe Bedrock → our formatter → Lambda response stream.
// In a real Lambda, you would get the raw response stream from the
// API Gateway integration (e.g., `event.stream`). Here we illustrate the pattern.
await bedrockStream.pipeTo(sseWriter);
// At this point Lambda will keep the connection open until the client aborts.
// Returning the response object satisfies the TypeScript signature.
return response;
};
Tip: The most common mistake is forgetting to set
Content-Type: text/event-stream. Without it the browser treats the payload as ordinary text and never firesmessageevents.
Lambda‑specific gotchas
-
Require(esm) breakage – Node 22’s native ESM support can clash with older Lambda layers that still expect CommonJS
require. Keep your function pure ESM or stick to CommonJS throughout. - SnapStart + VPC – If your Lambda sits in a VPC, SnapStart’s cold‑start savings disappear because the network interface is attached after the snapshot is restored.
-
Streaming headers – API Gateway will buffer the response unless you explicitly enable “payload format version 2.0” and set
Content-Typetotext/event-stream.
Handling Errors and Back‑pressure
A streaming pipeline is only as reliable as its weakest link. You need to anticipate three categories of problems:
- Bedrock errors – throttling, model not found, or malformed payload.
- Network hiccups – the client disconnects midway.
- Back‑pressure – the browser reads slower than Bedrock produces tokens.
Why we care about back‑pressure
If you keep writing to the response socket faster than the client can consume, the underlying TCP buffer fills up, and the Node process may be forced to pause or even crash with ERR_STREAM_WRITE_AFTER_END. Properly handling the writableStream’s ready promise prevents that.
Error handling inside the pipe
// Inside the WritableStream `write` method from the previous section
async write(chunk) {
// Decode and split as before...
const lines = new TextDecoder().decode(chunk).split("\n").filter(Boolean);
for (const line of lines) {
try {
const obj = JSON.parse(line);
if (obj.type === "error") {
// Bedrock can send an error object mid‑stream.
const sse = formatSSE(`ERROR: ${obj.message}`);
// Push the error to the client and then abort the stream.
await controller.enqueue(encoder.encode(sse));
controller.terminate(); // stop further processing
return;
}
if (obj.type === "token") {
const sse = formatSSE(obj.text);
// Back‑pressure: wait until the underlying stream is ready.
if (controller.desiredSize === 0) {
await controller.flush(); // pause until the client drains
}
await controller.enqueue(encoder.encode(sse));
}
} catch (parseErr) {
// If JSON is malformed, treat it as a non‑fatal heartbeat.
console.warn("Failed to parse Bedrock line:", line);
}
}
}
In plain English: If Bedrock tells us “I ran into an error”, we forward that error to the browser and stop sending more tokens.
Detecting client disconnects
API Gateway provides a signal (an AbortSignal) you can listen to:
// At the top of streamHandler
const abortSignal = (event.requestContext as any).http?.signal;
if (abortSignal?.aborted) {
console.log("Client already disconnected – abort early.");
return { statusCode: 204, headers, body: "" };
}
// Later, attach a listener
abortSignal?.addEventListener("abort", () => {
console.log("Client aborted the SSE connection.");
// Close Bedrock stream to avoid paying for unused tokens.
// The SDK doesn't expose a direct abort, but you can
// use an AbortController when you create the command.
});
When you create the InvokeModelCommand, pass an AbortSignal so the request can be cancelled:
import { AbortController } from "node-abort-controller";
const abortCtrl = new AbortController();
const command = new InvokeModelCommand({
// …payload…
// Attach the signal for cancellation.
// @ts-ignore – the SDK expects `abortSignal` in the options bag.
abortSignal: abortCtrl.signal,
});
Key takeaway: Tie the client’s abort signal to the Bedrock request; otherwise you may keep generating tokens that nobody sees.
Rate‑limit back‑off
If Bedrock returns ThrottlingException, implement exponential back‑off before retrying:
async function safeInvoke(command: InvokeModelCommand, retries = 3) {
for (let attempt = 0; attempt <= retries; attempt++) {
try {
return await bedrockClient.send(command);
} catch (err: any) {
if (err.name === "ThrottlingException" && attempt < retries) {
const delay = Math.pow(2, attempt) * 200; // 200ms, 400ms, 800ms...
console.warn(`Throttled – retrying in ${delay}ms`);
await new Promise((r) => setTimeout(r, delay));
continue;
}
throw err; // non‑throttle errors bubble up
}
}
}
The Takeaway
What you now have in your toolbox:
- Streaming matters because it lets users see LLM output instantly, cutting perceived latency.
-
Bedrock client setup with
@aws-sdk/client-bedrock-runtimeis a few lines, but you must remember to setstream: true. -
Node 22’s native fetch gives you a
ReadableStreamyou can pipe directly to SSE without extra libraries. -
A Lambda‑based SSE endpoint requires the right
Content-Type, an earlycallbackWaitsForEmptyEventLoop = false, and proper handling of the response stream. - Error and back‑pressure handling keeps your function from leaking resources, respects client disconnects, and avoids runaway token charges.
By following the steps above, you can ship a chat UI that feels as responsive as a real‑time conversation, while staying within the cost and performance boundaries of AWS services. Happy streaming!
Transparency notice
This article was written with the help of an AI system — Groq (GPT OSS 120B).
Published: 2026-09-18 · Primary focus: Bedrock
All code blocks are intended to be correct and runnable, but please verify them
against the official docs for the tools mentioned before using in production.Find an error? Drop a comment — corrections are always welcome.
Top comments (0)