Package your NodeJS Lambda functions individually with esbuild for faster cold-start times

Learn about packaging your NodeJS-based AWS Lambda functions with esbuild for improved cold start times and reduced deployment package size

This article was originally posted on my development blog, where you can find other articles about TypeScript, AWS, Lambda and open source.

Introduction

AWS SAM is a great way to package and deploy your AWS Lambda functions, but as your project grows, it's easy to become frustrated with increasing deployment times from an ever-growing deployment package containing all your functions and NodeJS dependencies.

The serverless framework supports an excellent workflow with the serverless-webpack plugin to package your functions individually with webpack. For AWS SAM, it's probably worth checking out the aws-sam-webpack-plugin, but I've found it far simpler to configure esbuild to perform the same task.

Why package my code using a bundler?

• Improved cold start time - smaller deployment packages generally means less time copying your deployment package to the AWS Lambda container each time it has to be started
• TypeScript support - NodeJS doesn't support TypeScript natively, so converting it to JavaScript can be done as part of a bundilng step
• Tree-shaking and reduced deployment package size - a bundler will only include the node dependencies your function imports, and if enabled correctly for tree-shaking, your overall bundle size will be reduced to just the blocks of code that are needed (which is great for multi-functional libraries like lodash which are rarely used as a whole).

Disadvantages

Using a bundler does come with some downsides, including:

• Increased complexity - bundling your code means more steps before running your code locally using aws sam local and more deployment time steps
• Inceased packaging time - as your functions are packaged individually, a unique deployment package is generated for each one, instead of uploading a single deployment package with all of your functions. However, this may still be acceptable as each function's deployment package will be considerably smaller after bundling
• Poor stack trace support - getting decent stack trace support with bundled code can be very difficult, especially for exceptions. In many cases, you'll still be debugging by console.log() or guessing based on the function names in your stack trace.
• Module support - some node modules are not compatible with bundling, either because they assume the existence of the package.json and node_modules folder (looking for filesystem artefacts in their code), or they are trying to perform auto-instrumentation for telemetry (which bundling interferes with).

esbuild vs webpack

I've used webpack before for bundling, and while it is a mature and flexible bundler, you may run into problems with its memory usage and execution when you use it on very large projects with dozens of AWS Lambda functions. This is because it is written in JavaScript and must execute the same bundling process for each Lambda function.

esbuild is a newer bundler built in Go, which is gaining considerable popularity for its speedy execution time and simple configuration, as well as removing the need for Babel with its native TypeScript support. However, it should be noted it has less features and is less mature than webpack (and probably always will because it is deliberately targeting a reduced feature set).

I've chosen esbuild because I've found it mature enough to use in production for large, established AWS Lambda based applications, and it reduced bundling times from over 5 minutes to less than 30 seconds.

Using esbuild with AWS SAM

Configuring your project

In this setup, we will bypass sam build and set up an esbuild bundling step that must be run before every sam package/sam deploy.

Install esbuild

First get esbuild installed with npm or yarn:

npm i esbuild -D

# OR

yarn add esbuild -D

Source code layout

Next structure your function entry points to be hosted in individual directories - this helps with ensuring that bundled output for each function gets its own directory, and hence can be deployed individually by SAM.

e.g.

src/say_hello/index.ts
src/send_email/index.ts

(I've assumed TypeScript here, but you can use JavaScript by simply changing the file extension. Remember that you should have typescript installed in your project and a tsconfig.json set up for your Node version if you're transpiling TS).

If you have any other shared code, it should be put into another directory e.g /shared or /util and simply imported.

Setup esbuild configuration

Although esbuild can be run from the command line, a quick JavaScript configuration file is easier to maintain and read with the number of options we will use.

const fs = require('fs');
const path = require('path');
const esbuild = require('esbuild');

const functionsDir = `src`;
const outDir = `dist`;
const entryPoints = fs
  .readdirSync(path.join(__dirname, functionsDir))
  .map(entry => `${functionsDir}/${entry}/index.ts`);

esbuild
  .build({
    entryPoints,
    bundle: true,
    outdir: path.join(__dirname, outDir),
    outbase: functionsDir, 
    platform: 'node',
    sourcemap: 'inline',
  });

This assumes your Lambda function entry points are located in ./src/<function_name>/index.ts and will be built to ./dist/<function_name>/index.js.

If you would prefer to do the above using the command line, the following esbuild line should be equivalent:

npx esbuild \
  --bundle src/*/index.ts \
  --outdir=dist \
  --outbase=src/ \
  --sourcemap=inline \
  --platform=node

Setting up your AWS SAM template

Each function in your AWS SAM template will need to be updated to use the new package layout:

Resources:
  HelloFunction:
    CodeUri: dist/say_hello
    # assumes your function is at `src/say_hello/index.ts` or 
    # `src/say_hello/index.js` with an exported entry point 
    # function called `handler`
    Handler: index.handler
    ...
  SendFunction:
    CodeUri: dist/send_email
    Handler: index.handler
    ...

Building and deploying your function

You can now build and deploy your function by running esbuild before each sam package and sam deploy step i.e.

node esbuild.js 
sam package
sam deploy

Running locally

Your bundled code can be executed locally, but you need to run node esbuild.js before you call sam local invoke.

If you want to re-run esbuild in watch mode, you can modify your esbuild.js accordingly:

esbuild.
  build({
    ...
    watch: process.argv.includes('--watch'),
  });

and run node esbuild.js --watch in the background (or add --watch to the command line variant given in Setup esbuild configuration).