How to setup AWS Lambda Layers (Nodejs)

Lambda layers were introduced in 2018 for flexibel code/data sharing within same or different AWS accounts. AWS Lambda supports multiple environments but here we will talk about only Nodejs. I will describe the whole process of making lambda layers & how to use them in Lambda functions.

So, whenever, we want to use custom code in lambda functions supporting nodejs environment. We write that code in form of node modules and to make that code shared through lambda layers, we have to make a build of those node modules. That build will be uploaded to a Lambda Layer and layer can be attached to any lambda function which can ultimately access that code.

Common code as Node Modules

In Nodejs supported AWS Lambda environment, lambda layers are usually made for following usecases:

1. NPM Packages

   There are only a few public npm packages which are available natively in lambda environment. For all other npm packages, you will have to create common lambda layers that can be used in your whole AWS environment.

2. Custom Node Modules

   Custom node modules which you have made for your system specific requirements.

Link Custom Node Modules

For npm packages from npm repository, you just need to install them using command: npm install { package-name }.
But you have to convert your custom nodejs code to node modules. Every node module has mostly 3 common items:

1. node_modules folder having all modules.
2. index.js file that exports all node modules.
3. package.json file.

NOTE: Your custom node module must have above 3 items & you can link that custom node module to your system node environment through command npm link. You can also install a linked custom module in any other node module by running command npm link { custom-module-name }.

Create Lambda Layers

To create lambda layer for nodejs code, we need to creat a build for our code. At first make sure that you have linked custom node modules to your node environment using method described in above highlighted note.
Next, follow the below steps:

• Create a new folder on your machine ( recommended: no spaces in name).
• Navigate into that folder & create a new node project by running command npm init -y. Name parameters for your project as you like. A new file named "package.json" will be created.
• Now, install required public npm packages or link/install your custom node modules in current project using npm link { custom-module-name }. You will find a folder named node_modules.

• We have to make a build of node_modules folder. Nodejs environment on AWS lambda extracts node modules from a folder named nodejs which furthure contains node-modules folder having all modules.
  Paste following command in Scripts field of Package.json.
  

  "scripts": {
      "build": "npm install && mkdir -p nodejs && cp -r node_modules nodejs/ && zip -r  {file-name}.zip nodejs"
  }
  

  Give relative name to your build zip file, save it & run command npm run build. You will see a zip file which is basically build of your node modules.

• Now, go to layers section in your AWS Lambda console. Create a new layer, upload your build right there or attach it through a s3 link. (s3 links are recommended, if build file is greater than 13MB).

• Your lambda layer is created. Code in that layer is now COMMON to whole AWS account and resides at one place. Attach the layer to any lambda function in your account. That lambda function will be able to access the code. Thats it😀.

  For cross account sharing of lambda layers, visit official docs here.

Comments

[Craig 🏔️🏳️‍🌈]

Am I able to bundle all but one of my dependencies in with the lambda handler code using webpack but then put that one dependency in a layer? I'd prefer to only use a layer for a hefty dep that is unlikely to change and keep the rest bundled with the handler code.

[Afraz Khan]

Agreed! Common use cases for layers involve heavy dependencies that are shared across multiple Lambda functions.

[Colin]

In case anyone is wondering about this step: "index.js file that exports all node modules", here's what I did:

export * from "dependency-a";
export * from "dependency-b";

[Danilo Machado]

bro, you save my life! hahah

[Afraz Khan]

glad to hear this :)

[WeaponizedLego]

This uploads en entire node_modules which is fine for smaller projects, but when the project size is 300 Lambdas, with a lot of dependencies, it exceeds the 250mb limit even without development dependencies. Do you have any suggestions on if it's possible to utilise tree shading, and some sort of import method to only put the methods you use instead of entire libraries into the layer?

[Afraz Khan]

This looks possible only with your own custom packages, I don't know any technique to pick and choose the methods for 3rd party deps you are using in your code.

About the lambda deployment-package limit, I have two suggestions:

• Consider distributing the packages across multiple layers, especially if you're dealing with 300 lambdas. There's a good likelihood of breaking down the packages into multiple layers and using a single layer for a particular set of lambdas based on their requirements.
• Have you thought about utilizing Lambda Container images? They offer a deployment package capacity of up to 1GB. So, if you genuinely require a single package that surpasses the 250MB limit, you can spin-up containers to execute your lambda functions. here

[Gaurav Dubey]

Thanks for the info, if I need to use models for my database and export my modules containing the query functions, should I keep it in my node_modules folder or outside of it, inside the nodejs folder, making the structure look like this-
nodejs
-node_modules
-custom-module1
-custom-module2

[Afraz Khan]

You need to put your custom moduels into the node_modles folder. Nothing else should be there in nodejs folder like below
_ nodejs
__ node_modules
___ module0
___ module1
___ module2
.
.
.

[Nermin Skenderovic]

Are we supposed to replace file-name in the build command?

[Afraz Khan]

yes :)