AWS Lambda Layers

As part of my preparation towards AWS Certified Solutions Architect Professional (SAP-C01) certification exam, I am currently studying the details of AWS Lambda included in Serverless section of the exam blue print. Exam blue print or exam guide can be found here.

I want to share my learnings on Lambda layers as part of this blog. As you might be aware, AWS Lambda is a compute service that lets us to run our code without provisioning any virtual machines or compute infrastructure. It is a Function-as-a-Service offering from AWS. Lambda functions uses a runtime (example: Python) and runs in a runtime environment when it is invoked. As a service, you are billed for the duration that the functions runs.

When a Lambda function is created, the code is packaged as a deployment package into .zip file. As you add libraries and other dependencies - which is referred by the Lambda function, creating and uploading the deployment package can slow down the development and execution.

In November 2018, AWS introduced Lambda Layers. A Lambda layer is a .zip file archive that contains additional code, data, libraries, custom runtime and configuration files. The .zip file archive can be loaded to Lambda layer from an S3 bucket or from your local machine. Any layer which is used are actually extracted into /opt folder inside the runtime environment. Use of Lambda Layers promote code sharing and separation of responsibilities which enables faster development.

Lambda Layers offers following advantages over not using them in first place:

• Layers allow us to use new runtimes which are not currently supported by default with Lambda.
• Layers also allows libraries to be externalised which means contained in a separate package, enforcing separation of concerns if any, between dependencies and actual function code.
• Enables faster deployments, because code that needs to be packaged and uploaded is smaller.
• Layers can be reused by other Lambda functions within an AWS account and shared between AWS accounts or shared publicly with developer communities.

I also tested the example provided by AWS to see how layers work in real. Related AWS blog can be found here.

I created a Lambda function using the following piece of Python code with Python 3.8 as runtime.

import numpy as np
from scipy.spatial import ConvexHull

def lambda_handler(event, context):

    print("\nUsing NumPy\n")

    print("random matrix_a =")
    matrix_a = np.random.randint(10, size=(4, 4))
    print(matrix_a)

    print("random matrix_b =")
    matrix_b = np.random.randint(10, size=(4, 4))
    print(matrix_b)

    print("matrix_a * matrix_b = ")
    print(matrix_a.dot(matrix_b))
    print("\nUsing SciPy\n")

    num_points = 10
    print(num_points, "random points:")
    points = np.random.rand(num_points, 2)
    for i, point in enumerate(points):
        print(i, '->', point)

    hull = ConvexHull(points)
    print("The smallest convex set containing all",
        num_points, "points has", len(hull.simplices),
        "sides,\nconnecting points:")
    for simplex in hull.simplices:
        print(simplex[0], '<->', simplex[1])

Screenshot of the Lambda function:

After I deployed my Lambda function and tested the same with invoking it, I received the following error:

The error was because there was no module named NumPy. We haven't included their use in the Lambda function code. The default runtime we chose Python 3.8 does not include the NumPy module. To resolve the error, we either have to create a deployment package including this module and upload to Lambda OR use Lambda Layers.

To use Layers, I create a Layer from the corresponding section of the Lambda function:

AWS provides following 3 options to configure a layer. I choose to use AWS provided layers.

I selected the layer AWSLambda-Python38-ScyPy1x which provides the required module NumPy and also the SciPy module which can be used for advanced spatial algorithms.

Now, again I invoked the Lambda function. This time, the Lambda function was executed successfully by performing a matrix multiplication.

To conclude, I used a Lambda Layer to provide the required modules for the Lambda function. These modules were not uploaded manually as we used the externalized set of libraries provided by AWS.

Thanks for reading this blog. I wanted to write down my understanding of the Lambda Layers feature.

Please point out any mistakes and provide your feedback.