AI SDK Streaming Text from Lambda

#programming #node #javascript #typescript

The AI SDK has two primary modes to interact with a language model: generation and streaming.

Generation means the model returns a complete message. That can be a word, a sentence, paragraph, or complete article.
In streaming mode, the model starts to return multiple partial messages - chunks - while it is generating a response. This is how we usually interact with ChatGPT and other chatbots.

The AI SDK docs already provide lots of examples. While its primary focus might be the usage via Next.js deployed on Vercel, it is built in a vendor-agnostic way so it should run on any Node.js platform.

In this post, I'll port the official example Node.js HTTP text streaming to work on Lambda.

Lambda Response Streaming

AWS introduced response streaming for Lambda functions a while ago, but it looks like it didn't really get into mainstream yet. A reason could be that it is still hard to find examples for using it correctly. Every example I found seems to derive from the official announcement Introducing AWS Lambda response streaming.

exports.handler = awslambda.streamifyResponse(
  async (event, responseStream, context) => {
    responseStream.setContentType("text/plain");
    responseStream.write("Hello, world!");
    responseStream.end();
  }
);

The biggest pain point is the missing type-safety. A few months ago I went through all available docs and examples and extended the official @types/aws-lambda package with types for streaming:

feat(aws-lambda): add response streaming types #72417

zirkelc posted on Apr 07, 2025

Please fill in this template.

[X] Use a meaningful title for the pull request. Include the name of the package modified.
[X] Test the change in your own code. (Compile and run.)
[X] Add or edit tests to reflect the change.
[X] Follow the advice from the readme.
[X] Avoid common mistakes.
[X] Run pnpm test aws-lambda.

Select one of these and delete the others:

If changing an existing definition:

[x] Provide a URL to documentation or source code which provides context for the suggested changes: https://aws.amazon.com/blogs/compute/introducing-aws-lambda-response-streaming/

View on GitHub

Installing the package will make the global awslambda namespace available, with the streamifyResponse() function and the HttpResponseStream class.

import { StreamifyHandler } from 'aws-lambda';

const streamifyResponseHandler: StreamifyHandler = (event, responseStream, context) => {
  const metadata = {
    statusCode: 200,
    headers: {
      "Content-Type": "application/json",
      "CustomHeader": "outerspace",
    },
  };
  responseStream = awslambda.HttpResponseStream.from(responseStream, metadata);
  responseStream.setContentType("text/plain");
  responseStream.write("Hello, world!");
  responseStream.end();
};

export const handler = awslambda.streamifyResponse(streamifyResponseHandler);

AI SDK Text Streaming

The official AI SDK example for Node.js uses the pipeTextStreamToResponse function to stream the text back to the client:

import { openai } from '@ai-sdk/openai';
import { streamText } from 'ai';
import { createServer } from 'http';

createServer(async (req, res) => {
  const result = streamText({
    model: openai('gpt-4o'),
    prompt: 'Invent a new holiday and describe its traditions.',
  });

  result.pipeTextStreamToResponse(res);
}).listen(8080);

Looking at the implementation of pipeTextStreamToResponse, it reads the textStream from the streamText() result and writes it to the ServerResponse from node:http.

However, this does not work on Lambda because we cannot return the ServerResponse object; instead, we must write to the output stream responseStream. Since we're dealing with streams, we can use Node.js pipeline() to simply pipe result.textStream to the Lambda response stream.

Here is the Node.js example from above, converted to work on Lambda:

import { bedrock } from '@ai-sdk/amazon-bedrock';
import { streamText } from 'ai';
import { pipeline } from 'node:stream/promises';
import type { APIGatewayProxyEventV2 } from 'aws-lambda';

export const handler = awslambda.streamifyResponse<APIGatewayProxyEventV2>(
  async (event, responseStream) => {
    responseStream.setContentType('text/plain; charset=utf-8');

    const result = streamText({
      model: bedrock('us.anthropic.claude-sonnet-4-5-20250929-v1:0'),
      prompt: 'Invent a new holiday and describe its traditions.',
    });

    await pipeline(result.textStream, responseStream);
  },
);

It's important to set the right content type text/plain via responseStream.setContentType() at the beginning, otherwise your browser or client won't render the incoming text chunks correctly.

Lambda Function URL

Lambda response streaming works only with Function URLs, not with the API Gateway. When you deploy your Lambda function, you must set the InvokeMode to RESPONSE_STREAM:

export class AISDKChatStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    const streamFunction = new NodejsFunction(this, 'aisdk-stream', {
      functionName: 'aisdk-stream',
      entry: 'src/functions/stream.ts',
      handler: 'handler',
      runtime: Runtime.NODEJS_22_X,
      timeout: Duration.seconds(60),
      memorySize: 1024,
    });

    const streamFunctionUrl = streamFunction.addFunctionUrl({
      authType: FunctionUrlAuthType.NONE,
      invokeMode: InvokeMode.RESPONSE_STREAM,
      cors: {
        allowedOrigins: Cors.ALL_ORIGINS,
      },
    });
  }
}