LLM chat apps feel laggy because most engineers default to polling or heavyweight WebSockets. In reality, Server‑Sent Events (SSE) give you true streaming with far less complexity, and API Gateway now supports it natively. This post shows you how to wire Claude on Bedrock through a Lambda that streams tokens directly to the browser.
Why SSE Beats WebSockets for LLM Streams
When a language model (LLM) generates text, it does so token‑by‑token. Imagine a writer typing a story on a typewriter: you want each keystroke to appear on the screen as soon as it’s pressed. Server‑Sent Events (SSE) are a one‑way, browser‑to‑server push technology that delivers a continuous stream of text lines over a single HTTP connection.
WebSockets are a full‑duplex channel (both client → server and server → client). They are great for chat rooms where each participant sends messages, but they require a separate library, a keep‑alive heartbeat, and often extra IAM configuration on AWS. For LLM streaming we only need server → client flow, so SSE is a lighter fit.
In plain English: If you only need the server to talk, SSE is like a walkie‑talkie that you don’t have to hold both ends of; WebSockets are a two‑way radio that adds unnecessary wiring.
The performance edge
- Lower latency – SSE writes each token to the TCP socket as soon as the Lambda yields it, without waiting for a full response body.
- Simpler infrastructure – No need for a separate “WebSocket API” in API Gateway; a regular HTTP API with response streaming does the job.
- Built‑in reconnection – Browsers automatically retry a dropped SSE connection, giving graceful recovery for intermittent network hiccups.
Configuring API Gateway for Chunked Responses
API Gateway must be told to stream the response rather than buffer it. The new “HTTP integration with response streaming” feature does exactly that, but it only works when the integration type is Lambda proxy and the payload format version is 2.0. Think of the payload format as the envelope the service uses to hand data to Lambda; version 2.0 includes the eventStream field that lets us push chunks.
Below is a minimal TypeScript script that creates an HTTP API, adds a /chat route, and attaches a Lambda function with the correct settings. We use the @aws-sdk/client-api-gateway package, the official AWS SDK for JavaScript.
// api-setup.ts
import {
ApiGatewayV2Client,
CreateApiCommand,
CreateIntegrationCommand,
CreateRouteCommand,
UpdateIntegrationCommand,
} from "@aws-sdk/client-api-gateway";
// 1️⃣ Create an HTTP API (not REST) – HTTP APIs support response streaming.
const client = new ApiGatewayV2Client({ region: "us-east-1" });
async function createStreamingApi(lambdaArn: string) {
// Create the API
const createApiRes = await client.send(
new CreateApiCommand({
Name: "ClaudeChatStreamingAPI",
ProtocolType: "HTTP", // HTTP API, not REST
// Enable CORS early – many teams forget this and get blocked in the browser.
CorsConfiguration: {
AllowOrigins: ["*"],
AllowHeaders: ["*"],
AllowMethods: ["GET", "POST", "OPTIONS"],
},
})
);
const apiId = createApiRes.ApiId!;
console.log(`API created with ID ${apiId}`);
// 2️⃣ Create an integration that points at our Lambda.
const integrationRes = await client.send(
new CreateIntegrationCommand({
ApiId: apiId,
IntegrationType: "AWS_PROXY", // Lambda proxy integration
IntegrationUri: lambdaArn,
PayloadFormatVersion: "2.0", // <-- crucial for streaming
// Tell API Gateway we want to stream the response.
IntegrationMethod: "POST",
})
);
const integrationId = integrationRes.IntegrationId!;
console.log(`Integration created with ID ${integrationId}`);
// 3️⃣ Attach the integration to a /chat POST route.
await client.send(
new CreateRouteCommand({
ApiId: apiId,
RouteKey: "POST /chat",
Target: `integrations/${integrationId}`,
})
);
// 4️⃣ Enable response streaming on the integration (new flag as of 2024).
await client.send(
new UpdateIntegrationCommand({
ApiId: apiId,
IntegrationId: integrationId,
// The property name is "ResponseParameters" in the SDK,
// but we only need to set the streaming flag.
IntegrationResponseParameters: {
"contentHandlingStrategy": "CONVERT_TO_TEXT",
},
// The real streaming toggle lives in "ResponseTemplates"
// with a special placeholder; the SDK abstracts it.
IntegrationResponseTemplates: {
"application/json": "$input.path('$.eventStream')",
},
})
);
console.log("Streaming integration configured. Deploy the API to a stage to finish.");
}
export default createStreamingApi;
Key takeaway: The three settings that unlock streaming are: (1) HTTP API, (2) Lambda proxy with payload version 2.0, and (3) the “response streaming” flag on the integration.
Quick deployment tip
API Gateway stages cannot be edited after creation, so create a stage right after the API:
aws apigatewayv2 create-stage \
--api-id $API_ID \
--stage-name dev \
--auto-deploy
Now the endpoint https://$API_ID.execute-api.$REGION.amazonaws.com/dev/chat will forward the Lambda’s chunked response straight to the client as text/event-stream.
Lambda Streaming Handler with Native fetch
Our Lambda runs on Node.js 22, which ships with the global fetch API. That means we can call the Bedrock Claude runtime without adding a separate HTTP library. The Bedrock service returns a streaming JSON where each line looks like:
{ "completion": "Hello", "index": 0, "type": "message" }
We read each line, extract the token, and wrap it in an SSE “data” line:
data: {"token":"Hello"}
The Lambda must set Content-Type: text/event-stream and flush each chunk. In Node 22 the Response object supports a body that is a ReadableStream. By yielding a new Uint8Array for each token we keep the connection alive.
// handler.ts
import { APIGatewayProxyEventV2, APIGatewayProxyResultV2 } from "aws-lambda";
/**
* Helper: convert a string into an SSE‑formatted line.
* The double newline signals the end of one event.
*/
function sseLine(payload: object): string {
return `data: ${JSON.stringify(payload)}\n\n`;
}
/**
* Main Lambda entry point.
* Receives a chat message, forwards it to Claude, streams tokens back.
*/
export const handler = async (
event: APIGatewayProxyEventV2
): Promise<APIGatewayProxyResultV2> => {
// Parse the incoming JSON body (e.g., { "prompt": "Tell me a joke" })
const body = JSON.parse(event.body ?? "{}");
const prompt = body.prompt ?? "";
// Call Bedrock Claude with streaming enabled.
// The Bedrock endpoint and model name are placeholders – replace with your own.
const bedrockResponse = await fetch(
"https://bedrock-runtime.us-east-1.amazonaws.com/model/anthropic.claude-v2/invoke",
{
method: "POST",
headers: {
"Content-Type": "application/json",
// The request must ask for streaming via "accept: text/event-stream"
Accept: "text/event-stream",
// Include IAM auth – in Lambda the SDK automatically signs the request.
},
body: JSON.stringify({
prompt,
stream: true, // Tell Claude to stream tokens
}),
}
);
// Verify we got a streaming response.
if (!bedrockResponse.body) {
return {
statusCode: 502,
body: "Bedrock returned no body",
};
}
// Create a ReadableStream that transforms Bedrock's JSON lines into SSE.
const sseStream = new ReadableStream({
async start(controller) {
const decoder = new TextDecoder();
const reader = bedrockResponse.body!.getReader();
let buffer = "";
// Read chunks from Bedrock until the stream ends.
while (true) {
const { done, value } = await reader.read();
if (done) break;
// Convert bytes to string and accumulate.
buffer += decoder.decode(value, { stream: true });
// Split on newline – Bedrock separates JSON objects with "\n".
const lines = buffer.split("\n");
// Keep the last partial line in the buffer.
buffer = lines.pop()!;
for (const line of lines) {
if (!line.trim()) continue; // skip empty lines
const json = JSON.parse(line);
// Assume the token appears under "completion"
const token = json.completion ?? "";
// Push an SSE line downstream.
const sse = sseLine({ token });
controller.enqueue(new TextEncoder().encode(sse));
}
}
// When Bedrock finishes, close the SSE stream cleanly.
controller.close();
},
});
// Return the streaming response to API Gateway.
return {
statusCode: 200,
headers: {
// This header tells the browser to treat the response as SSE.
"Content-Type": "text/event-stream",
// Prevent any caching – we want every token to appear.
"Cache-Control": "no-cache",
// Keep‑alive helps some proxies not to cut the connection.
"Connection": "keep-alive",
},
// The body must be a ReadableStream for API Gateway to forward it.
body: sseStream as any, // TypeScript needs a cast here.
};
};
Tip: The only place where we must set a header is
Content-Type: text/event-stream. Without it the browser treats the payload as plain text and never fires themessageevent.
Analogy for the stream transformation
Think of Bedrock’s response as a river of tiny beads (JSON lines). Our Lambda is a sifter that picks each bead, puts it in a clear plastic bag (the SSE line), and drops the bag downstream. The river never stops; we just keep handing out bags one at a time.
Frontend: Consuming the Event Stream
On the client side we use the native EventSource API, which knows how to listen to SSE. It automatically reconnects if the connection drops, so we get a resilient UI with almost no code.
<!-- index.html -->
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>Claude Chat – Live</title>
<style>
body { font-family: sans-serif; margin: 2rem; }
#chat { white-space: pre-wrap; border: 1px solid #ccc; padding: 1rem; min-height: 200px; }
#prompt { width: 80%; }
</style>
</head>
<body>
<h1>Claude Live Chat</h1>
<div id="chat"></div>
<input id="prompt" placeholder="Ask Claude..." />
<button id="send">Send</button>
<script>
const chatDiv = document.getElementById('chat');
const promptInput = document.getElementById('prompt');
const sendBtn = document.getElementById('send');
// Helper: append text to the chat area.
function appendToken(token) {
chatDiv.textContent += token;
}
// Click handler – starts a new SSE connection for each user message.
sendBtn.onclick = () => {
const prompt = promptInput.value.trim();
if (!prompt) return;
// Reset UI for the new response.
chatDiv.textContent = '';
promptInput.value = '';
// Create an EventSource pointing at our API Gateway endpoint.
const es = new EventSource(`https://YOUR_API_ID.execute-api.us-east-1.amazonaws.com/dev/chat?prompt=${encodeURIComponent(prompt)}`);
// Listen for each token.
es.onmessage = (event) => {
const data = JSON.parse(event.data);
// The Lambda emitted { token: "..." }
appendToken(data.token);
};
// If the server closes (e.g., Claude finished), we stop listening.
es.onerror = (err) => {
console.error('SSE error', err);
es.close();
};
};
</script>
</body>
</html>
In plain English:
EventSourceis a tiny built‑in browser object that keeps the HTTP connection open and fires amessageevent each time the server writes a line that starts withdata:.
Why we put the prompt in the query string
Because SSE is a GET‑only protocol, we cannot send a request body. Placing the user’s prompt in the query string is the simplest way to hand data to the Lambda without breaking the SSE contract. For production you would want to sign the URL or use an API key, but the concept stays the same.
Common Pitfalls and Gotchas
| Area | Gotcha | How to avoid |
|---|---|---|
| API Gateway | The response is buffered unless “HTTP integration with response streaming” is turned on. | Double‑check PayloadFormatVersion: "2.0" and the UpdateIntegrationCommand that sets eventStream. |
| API Gateway | REST APIs (v1) don’t support JWT authorizers on streaming routes. | Use an HTTP API (v2) instead; it supports JWT at the stage level. |
| API Gateway | Integration timeout is fixed at 29 seconds. | Ensure Claude’s max_tokens and temperature settings keep the response under the limit, or break long chats into multiple requests. |
| Lambda | Node 22’s require(esm) in a layer silently fails, breaking imports. |
Stick to native ES modules (import … from …) or bundle everything with esbuild. |
| Lambda | Forgetting to set Content-Type: text/event-stream leads to a silent client. |
Add the header in the Lambda response (see code). |
| Lambda | Provisioned Concurrency costs rise quickly if left idle. | Use on‑demand concurrency for low‑traffic prototypes; enable provisioned only after measuring steady traffic. |
| Frontend | CORS misconfiguration blocks the browser. | In the API Gateway console enable CORS for * (or specific origins) and include Access-Control-Allow-Origin in the response headers. |
| Frontend | EventSource cannot send a request body, so developers try a POST. | Keep the request as a GET with query parameters, or use a short‑lived signed URL if security is a concern. |
Tip: When you see the browser console saying “EventSource failed: 502 Bad Gateway”, the first thing to inspect is whether API Gateway actually streamed anything. Look at the execution logs (
$context.integrationLatency) – anullbody means streaming wasn’t enabled.
The “no‑buffer” test
You can verify streaming works by curling the endpoint with --no-buffer:
curl -N "https://YOUR_API_ID.execute-api.us-east-1.amazonaws.com/dev/chat?prompt=Hello"
If you see each data: line appear instantly, the pipeline is streaming correctly. If you get a single large JSON blob after a few seconds, the response is still being buffered.
The Takeaway
Key points to remember
- SSE is a one‑way, low‑overhead streaming protocol perfect for LLM token delivery.
- API Gateway must be an HTTP API with Lambda proxy integration, payload version 2.0, and the response‑streaming flag turned on.
- The Lambda handler uses the built‑in
fetch(Node 22) to call Bedrock, reads the streaming JSON, and rewrites each token into an SSE line using aReadableStream. -
EventSource on the browser side automatically reconnects and fires a
messageevent for every token, giving a live‑typing experience. - Common pitfalls include hidden buffering, CORS misconfiguration, and the 29‑second timeout – all solvable with the checklist above.
Now you have a minimal, production‑ready stack that streams Claude’s output to any web client without the heavyweight ceremony of WebSockets. Happy coding!
Transparency notice
This article was written with the help of an AI system — Groq (GPT OSS 120B).
Published: 2026-09-18 · Primary focus: APIGateway
All code blocks are intended to be correct and runnable, but please verify them
against the official docs for the tools mentioned before using in production.Find an error? Drop a comment — corrections are always welcome.
Top comments (0)