AWS Lambda - Enhance the functionality with Lambda Layers

Back in Nov 20014, Lambda functions brought in a revolution in the world of cloud computing. That introduced us to serverless computing in the true sense. The concept of independent pieces of computation that are fast and easily affordable, provided a new paradigm.

Independent computing units were great. But soon, people realized that pieces of their code were really not independent. There was a lot that they needed to share. We need some common data structures, and utilities across the application. How do we provide them in Lambda functions?

That led to some duplication of code. People found innovative ways of working around this - by having a unified code in one place, that is configured to deploy into several different units. So the same code was deployed into different lambda functions. That did work to an extent. But then the deployment complexity blew up.

People thought of deploying everything for every lambda function. That works as well. But as the libraries and dependencies grew in size, we had to pay the price for it - in memory usage as well as the computation time.

Considering all these problems, AWS introduced the concept of Lambda Layers - in the reInvent 2018.

What is a Lambda Layer

In simple words, a Lambda Layer is kind of an external library that can be directly included, and used in the Lambda function.

The Lambda Layer is provided as a zipped file that can be uploaded via code or directly onto the web console. A lambda function that needs the code in there, should be configured as such to use the layer.

When such a lambda function runs, the contents of the layer are extracted into the /opt folder in the Lambda runtime environment. The layer need not be restricted to the language of the Lambda function. So long as it can be invoked and used by the code, it is just fine.

Let us now check out a simple example of creating and using Lambda Layers. We will develop a simple NodeJS based Lambda function, and configure it to use a custom layer.

• Creating the Lambda Layer
  • Using AWS Console
  • Command Line
• Using it in the Lambda Function
  • Using AWS Console
  • Command Line

Create a Lambda Layer

The Lambda layer is a chunk of organized code that can be used in the Lambda functions we develop. Or it can contain a set of modules that are not available in the Lambda environment. For the demo, let us create a simple JavaScript Layer - that contains custom code as well as the axios module for invoking http requests.

To do this, create a folder named nodejs. This folder name is sacrosanct if you are developing a JavaScript layer.

On the Linux prompt:

mkdir nodejs && cd nodejs

Then create an Node project using npm init. Since this is a test project, we do not care for the particular package details. So we provide the -y to accept defaults

npm init -y

Next, we install the moment.js module.

npm install --save moment

This generates a folder in there - node_modules. If you peep into it, you will see several folders and files that have landed in there.

Next, we create a small JavaScript code - custtom.js

moment=require("moment");
exports.now = () => { console.log(moment().format()) };

Add this into the node_modules folder. With all this in place, we zip the entire nodejs folder - where all this code was added.

And there.. we have created the Lambda Layer. Next, we have to push it into our AWS account.

We can create the lambda layer via the web console, or simple command line script. Doing it on the web console is simpler for newbies. But if you are serious about AWS, you have to break the barrier and get down to coding rather than configuring on UI. We will check out either of them here.

Using the AWS Web Console

To start with, open the AWS Lambda dashboard with your account. On the left side, you will see options to view the Dashboard, Applications, Functions and last in the list is Layers. Click on that.

That will open the layers page. Will show you something like this:

Now, click on the Create layer button

And it will open a menu like this

We give it a good name and upload the zip file. We also need to choose the compatible runtime. In this case, we can pick the available nodejs runtimes. And finally we click Create.

There, our lambda layer is created.

On the Command Line

Console is fine. But it is not always good. In real production deployments, we expect hundreds (if not thousands) of such artifacts to be deployed. We cannot go on creating them one by one. Apart from being tedious and time consuming, it is also error prone. Source code is fast and repeatable. So we write a script to configure our layer. It is just a few lines of code, if you have the AWS CLI installed on your machine, just invoke one line of code to create a new Lambda layer out of this zip file.

aws lambda publish-layer-version --layer-name learnlayer --compatible-runtime nodejs12.x --zip-file fileb://layer.zip

There, the lambda layer is published.

Note, that Lambda Layers are always published with a version number. And the layer cannot be changed after it is published. If we modify it, we have to publish a new version number.

This is very thoughtful. A layer is used in several lambda functions at a time. We do not want to surprise anyone when we modify the layer for a particular requirement. But, that is equally irritating when we actually want to modify the code for all the Lambda functions that use the layer.

There are ways to work around it. We will check it in a section below.

Using the Lambda Layer

Once we have the layer in place, we need to configure the Lambda function to use it. Once again, we have two options - to use the command line or the AWS Web Console.

Let us start with the web console.

Using the AWS Web Console

If you are reading this blog and reached upto this point, I am sure you know what is a Lambda function, and how to create it - at least on the web console. So I will skip those details. Once you have the lambda configured, you can add this layer into it

Click on the Layers in the Lambda configuration. That will show you the layer configuration, and a button to Add Layer. Click on that.

You will be taken to the page to choose the layer. You have options there, to choose the right layer from different sources. In this case, we will choose the Custom Layer. Because that is what we have. We see a dropdown - showing the layer we just created.

Choose the lambda layer you just created and click on Add. There, you have configured the Lambda Function to use the new layer.

On the Command Line

Command line makes this very simple. We just modify an existing lambda function to add the new layer to it.

aws lambda update-function-configuration --function-name TestLambda --layers arn:aws:lambda:us-east-1:869871132949:layer:learnlayer:1

Lambda Function

To use this layer, we can have a simple code in the Lambda Function

const moment = require("moment");
const custom = require("custom");

exports.handler = async (event, context) => {
    console.log(moment().format());
    custom.now();    

    const response = {
        statusCode: 200,
        body: JSON.stringify('Hello from Lambda!'),
    };
    return response;
};

The code uses the custom JavaScript code as well as the moment.js module that we had installed. Both work perfectly as expected and we get his in the logs

2020-09-10T19:20:03.267Z    b36bcc48-9b66-4437-82f8-4c4fe9459e2e    INFO    2020-09-10T19:20:03+00:00
2020-09-10T19:20:03.267Z    b36bcc48-9b66-4437-82f8-4c4fe9459e2e    INFO    2020-09-10T19:20:03+00:00

There! You just created a fully functional Lambda layer and used it in a Lambda Function. Of course, a real life Lambda Layer and a Lambda function would have a lot more functionality than this.

But the core concept remains the same. You can expand it with all your business logic.