How we keep our Serverless deploy times short and avoid headaches

At BubblyDoo we're building the world's most powerful product personalization platform, and we've gotten this far by using open-source software through all of our projects.

We're using Serverless Framework to deploy most of our backend. AWS Lambda, Cloudflare Workers and Deno Deploy are the Serverless platforms we've been using. Unfortunately, not all projects can be deployed to isolate-based platforms like Cloudflare Workers and Deno Deploy, as many still have binary dependencies or need filesystem access. That's why most of our infrastructure is deployed on AWS Lambda.

But how do you deploy a large Node.js project with hundreds of dependencies, and avoid long deploy times?

We've encountered this problem as well, and we've come up with a solution: the Serverless Externals Plugin.

Without any plugins

You create a Javascript file (lambda.js) which requires some Node modules. You include the whole node_modules folder in the Serverless deployment.
Serverless has some built-in optimizations: it can exclude your dev dependencies, which already helps reduce the size.

# serverless.yml
package:
  excludeDevDependencies: true

However, there's no tree-shaking, and a lot of unnecessary files are uploaded (e.g. documentation). For some of our deployments this would create zips of 100MB+.

Next to that excludeDevDependencies is inefficient and takes a very long time.

With a bundler

You use a bundler like Webpack, Rollup or esbuild to turn your code and all node_modules into a single bundled file (bundle.js).

You then exclude all node_modules from the deployment.

# serverless.yml
package:
  excludeDevDependencies: false
  patterns:
    - '!node_modules/**'

But there's a problem! Not all Node modules can be bundled. There are issues in bundlers, issues in packages, but there are also inherent problems: what if a Node module includes a binary file? In that case, it can't be bundled.

To solve this, we need a way to exclude some modules from the bundle, and keep them external. We can then upload only these modules in the deployment package.

With Serverless Externals Plugin

We don't like plugins that add magic, so you'll have to configure a few things.

Let's say we made a function that uses readable-stream, a module that can't be bundled.

const { Readable } = require('readable-stream');
const _ = require('lodash');

module.exports.handler = () => {
  ... // code using _ and Readable
};

The desired result is a bundle that has bundled lodash, but keeps the call to require('readable-stream').

You use Rollup, a bundler, to create a single bundled file.

In rollup.config.js:

import { rollupPlugin as externals } from "serverless-externals-plugin";

export default {
  input: { file: "src/lambda.js" },
  output: { file: "dist/bundle.js" },
  ...,
  plugins: [
    externals(__dirname, {
      modules: ["readable-stream"] // <- list external modules
    }),
    commonjs(),
    nodeResolve({ preferBuiltins: true, exportConditions: ["node"] }),
    ...
  ],
}

After running rollup -c, you'll have your bundle inside dist/bundle.js, and a report inside dist/node-externals-report.json:

{
  "isReport": true,
  "importedModuleRoots": [
    "node_modules/readable-stream"
  ],
  ...
}

Using this report, Serverless knows which node_modules it needs to upload.

In serverless.yml:

plugins:
  - serverless-externals-plugin

functions:
  handler:
    handler: dist/bundle.handler
    package:
      patterns:
        # include only dist
        - "!./**"
        - ./dist/**
    externals:
      report: dist/node-externals-report.json

Advantages of using this plugin

• Node spends a lot of time resolving the correct Node module because it is I/O-bound. This is not great for cold starts. By inlining all code, a bundler basically removes this problem.
• The bundled code is much smaller than the raw files. It's also tree-shaken, meaning unused code is removed.
• The plugin can be added incrementally. If you're already bundling your code but you have one node_module you can't bundle, this plugin is for you.

How does it do that?

1. The Rollup plugin looks at your package-lock.json or your yarn.lock and builds a dependency tree for your application.

2. It uses your configuration to mark the right modules and all of their production dependencies as external.

3. It looks at the bundled file and checks which modules are actually imported. If a module isn't imported, it's not packaged.

This is why it doesn't matter if you add too many dependencies to the modules array, the unused ones will be filtered out.

The dependency tree is quite complicated when you take different versions into account, see our README for an example. This plugin handles different versions correctly.

Example

Let's say you have two modules in your package.json, pkg2 and pkg3. pkg3 is a module with native binaries, so it can't be bundled.

root
+-- pkg3@2.0.0
+-- pkg2@0.0.1
    +-- pkg3@1.0.0

Because pkg3 can't be bundled, both ./node_modules/pkg3 and ./node_modules/pkg2/node_modules/pkg3 should be included in the bundle. pkg2 can just be bundled, but should import pkg3 as follows: require('pkg2/node_modules/pkg3'). It cannot just do require('pkg3') because pkg3 has a different version than pkg2/node_modules/pkg3.

In the Serverless package, only ./node_modules/pkg3/** and ./node_modules/pkg2/node_modules/pkg3/** will be included, all the other contents of node_modules are already bundled.

When uploading the whole node_modules folder, all requires from ./node_modules/pkg2 to pkg3 would already require pkg2/node_modules/pkg3 because of the Node resolution algorithm. Because Rollup isn't made to make only subdependencies external, this plugin rewrites those calls to require('pkg2/node_modules/pkg3').

How does this compare to other plugins?

Serverless Jetpack

Jetpack is great but it doesn't go the bundling way. It does something like a bundler and analyzes the files on which the Lambda code depends, and generates include patterns from there. (in trace mode)
Because of this it doesn't have the benefits of bundling, namely fast module resolution and tree-shaking.

Serverless Webpack

By default, Serverless Webpack doesn't support externals, but Webpack can use Webpack Node Externals to exclude all modules from the bundle. All included modules have to be allowlisted, but this plugin doesn't look at subdependencies.
When used with custom.webpack.includeModules, the non-allowlisted modules are added to the deployment zip.

Serverless Plugin Tree Shake

There's not much documentation about this plugin, but it also doesn't use bundling. However, it uses @vercel/nft to analyze the files on which the Lambda code depends. It seems to support Yarn PnP, which this plugin doesn't.
It overrides the zip function of Serverless to achieve this.

Used in production

This plugin is used for all our AWS Lambda deployments, using a wide range of Node modules, some with more quirks than others. We use it together with Lambda Layer Sharp and Chrome AWS Lambda.

Webpack and esbuild Plugin

Although Rollup is great, Webpack and esbuild are more feature-rich and faster, respectively. I'd like to create plugins for these bundlers as well if the community is interested. Feel free to open an issue or comment here!