Reduce your package size when uploading a Lambda function

How do you deploy your lambdas?

Hopefully not by writing code in the AWS UIConsole nor by manually uploading a zip file.
Even a small shell script to zip your lambda folder and upload it with aws cli would not be such an improvement:

zip -r function.zip .
aws lambda update-function-code --function-name my-function --zip-file fileb://function.zip

Much better if you are using Serverless Framework (or SAM or Terraform) or the AWS CDK: they will take care of bundling the lambda and its dependencies and upload them.

Whatever the approach, we must always remember the limits of Lambda package size:

Deployment package size

• 50 MB (zipped, for direct upload)
• 250 MB (unzipped, including layers)
• 3 MB (console editor)

How to check the content of currently deployed lambda?

Remember that the size is not only a problem because you cannot see the content of your lambda in the UI console (see also this post to know how to check the content of currently deployed lambda) or because you might not be able at all to upload it. Size and number of dependencies could also affect the time the Lambda needs in order to start and initialize! [Actually while writing this, I made some research and found out that it is not really proven.. see this article]

How to keep the package size you are uploading small?

Basically you have to keep your project clean and tidy and specify exactly what you are "zipping" and uploading.
This means making sure that all the dev dependencies and all the additional stuff ( docs, test, fixtures, shell scripts) that might be in your repository should not be part of the package.

In serverless to make sure you are deploying just what is necessary to your lambda on production, you don't have to do much, sls deploy (or if you just edited on lambda in your stack even sls deploy -f YOUR_FUNCTION_NAME) already takes care of excluding all devDependencies.
But what about tests, internal docs, .env files, fixtures, and other support files, to be bundled?

When publishing a package on NPM, everything that is listed in the .gitignore or in the more specific .npmignore is, well, ignored
(You can check what would end up in your package by running npm pack).
With serverless framework we can specify a similar list in the serverless.yml.

package:
  exclude:
    - 'tests/**'
    - 'tests-integration/**'
    - '.idea/**'
    - 'serverless-configs/**'
    - 'envs/**'
    - 'support/**'
    - 'node_modules/.cache/**'
    - 'scripts/**'

But how can it be done if you are using the AWS CDK ( or if you really want to zip the content and upload it manually?)

An approach could be using a script that:

• removes all the devDependencies: npm install --production or even better (and faster!) npm prune --production see the docs.
• then we can specify the files to be excluded directly in the lambda.Code.FromAsset(AssetOptions of the CDK ) lambda.Code.fromAsset("./resources/lambda", {exclude: ["package-lock.json", "package.json", "tests/**","support/idontwhantthis.txt" ]}),
• then we basically edit a script in our package.json file so that whenever we want to cdk deploy we first "package the lambda"

Not very handy..

The best solution I could find so far, which is also anyway very good in case you write your lambdas in Typescript is bundling your lambda with Webpack.
This will basically solve all issues at once: No DevDependencies, No unnecessary files, just an easy lambda.Code.fromAsset("my_bundle.js")without a long complex exclude list - one drawback could be that in the UIConsole you will see one big ugly file instead of your beautiful code and folder structure, but hey.. )

In your package.json, at cdk root you will just need a couple of scripts to bundel the lambda and to deploy your stack:

    "bundle-lambda": "webpack",
    "deploy": "npm run bundle-lambda && cdk deploy",

Your webpack config is just specifying where to grab your lambda code and where to put it after bundling all dependencies and imports.

const path = require('path');

const SRC_DIR = path.resolve(__dirname, 'resources/lambda');
const OUT_DIR = path.resolve(__dirname, 'resources/lambda/deployment');

const config = {
    entry: {
        mylambda: path.resolve(SRC_DIR, 'mylambda.js')
    },
    // aws-sdk is already available in the Node.js Lambda environment
    //  so it can be excluded from function bundles
    externals: [
        'aws-sdk'
    ],
    output: {
        path: OUT_DIR,
        filename: '[name].js',
        library: '[name]',
        libraryTarget: 'umd'
    },
    target: 'node',
    mode: 'production'
};

module.exports = config;

then inside your stack-definition.ts, where you define what code upload to Lambda just point to the deployment or dist ( or whatever you added as OUT_DIR in webpack config)

const myLambda = new lambda.Function(this, "MyAwesomeLambda",{
  name:...,
  code: lambda.Code.fromAsset("./resources/lambda/deployment"),
  runtime: NODEJS_10_X
  etc...
})

Hope it helps.

---

Photo by Brandless on Unsplash

Comments

[Vincent Voyer]

Hey there, I created an aws cdk construct based on Rollup.js that I think could interest you. It's here: github.com/vvo/aws-lambda-nodejs-r.... I would love to have your feedback on it. Thanks!