How to provision Lambda and Lambda Layer using CDK

Why using CDK?

CDK let you provision all your aws service using the programming language you use.

In this example, I will define a Lambda and Lambda Layers using CDK python.

First, make a new project

cdk init my-sample-app --language python

Then activate the virtual env

.env\Scripts\activate.bat

Set up the project structure

my-sample-app
|-lambda
|--boto3Folder
|--MyFirstLambdaFunction
|---app.py
|--All the lambda folder here
|-MySampleAppStack
|--my_sample_stack.py

What is Lambda Layer?
In my own words:
Lambda Layers is the place you install your library in cloud. Normally you just pip install some-library for your python project right? Lambda Layers is just do that, you install a library inside a local folder, then CDK help you package all file inside that folder and upload it to cloud, then you can use it elsewhere.

First install boto3 library inside lambda/boto3Folder

pip install --target=d:\yourLocalPath\my-sample-app\lambda\boto3Folder boto3

By this, you install boto3 library into my-sample-app/lambda/boto3Folder, then you should see some file and folder inside boto3Folder

Then inside my_sample_stack.py, define the Lambda Layer as follow:

from aws_cdk.aws_lambda_python import PythonFunction, PythonLayerVersion


class MySampleStack(core.Stack):

        def __init__(self, scope: core.Construct, id: str, **kwargs) -> None:
           super().__init__(scope, id, **kwargs)

           # All the aws resources define here

            # Here define a Lambda Layer 
            boto3_lambda_layer = PythonLayerVersion(
               self, 'Boto3LambdaLayer',
               entry='lambda/boto3Folder',
               compatible_runtimes=[lambda_.Runtime.PYTHON_3_8],
               description='Boto3 Library',
               layer_version_name='Whatever name you want'
           )

In this few lines of code, you defined a Lambda Layer, tell CDK where is the folder in entry, in our case is lambda/boto3Folder and other necessary information.

So now you have a Lambda layers, now we want to use this Lambda Layers, inside a Lambda function.

In MySampleStack above add this to define a Lambda Function:

my_first_lambda_function=PythonFunction(
         self, 'MyFirstLambda',
         entry='lambda/MyFirstLambdaFunction',
         index='app.py',
         runtime=lambda_.Runtime.PYTHON_3_8,
         layers=[boto3_lambda_layer], # here you use the lambda layer above 
         handler='MyFirstLambdaHandler',
         timeout=core.Duration.seconds(10)
)

This you define a Lambda function that will use the Lambda layers. In your MyFirstLambdaFunction/app.py you can use boto3 as usual.

Inside MyFirstLambdaFunction/app.py

import boto3

def MyFirstLambdaHandler(event, context):
    client = boto3.client('dynamodb')

    # do all the stuff with the client

cdk deploy

All the resources will start provision in the cloud.

Why using Lambda Layers?
1) Avoid install the same library over and over again.
Lets say you have 10 Lambda function all will using 2 same library, then you will need to install 10 separate times inside 10 Lambda. Use Lambda Layers, install 1 time, you it by 10 different Lambda.

2) Reduce size of your Lambda function.
If your Lambda function is more that 10MB, you wont able to edit it in Aws Lambda console.

Summary
In this article, you learned

• How to create a Lambda Layer and Lambda function
• How to use Lambda Layer inside a Lambda
• Why you need to use a Lambda Layer?

I hope you learned something. Have a good day.

In next part of series, I will talk about set up environment variable for a Lambda function using CDK.

Before you go, if you like this series or find this useful consider to buy me a coffee 😊🤞 for 5 USD or more.
I will prepare a GitHub repo for this whole tutorial series and arrange into separate commit for each part.
This will only available for my supporter cause I spent a lot of time to prepare this. Anyway, I appreciate you here. Have a good day.

Shout out to me on Twitter: @upupkenchoong

Comments

[Cam Cardosi]

Great stuff! My only recommendation is that you might want to demonstrate this with a library other than boto3, because the Python runtimes in AWS include boto3 by default, so if someone tries this and does something wrong, they may not realize it because boto3 will still be available for import.

docs.aws.amazon.com/lambda/latest/...

But I still appreciate the post!

[Ken Choong]

I see, no problem, later I will provide some example for few library other than Boto 3.

I appreciate your feedback. Thanks , I happy that I help you

[Aldwyn Cabarrubias]

Hi Ken. Great post! Just wondering though: would it be much more efficient to pair the lambda_python.PythonLayerVersion with lambda_.Function instead of lambda_python.PythonFunction? Because the PythonFunction creates a docker image with the Python libraries in it already and it's not basically using the layer. But with the Function and PythonLayerVersion combo, which makes more sense for no-redundancy bundling, it doesn't seem to work.

[Aldwyn Cabarrubias]

Welp it worked eventually :D

[Ken Choong]

I used lambda_.Function before, for some reason, lambda_.Function you will see some very weird problem when import the library from Lambda Layer into the Lambda function. Since then I just use PythonFunction, which let me don't even need to think about the bundling stuff.

[Ken Choong]

Great. Glad you like it

[JILLY]

I tried this, but getting error here.

boto3_lambda_layer = _alambda.PythonLayerVersion(

Does this require Docker to be installed?

jsii.errors.JavaScriptError:
Error: spawnSync docker ENOENT
at Object.spawnSync (node:internal/child_process:1124:20)
at spawnSync (node:child_process:876:24)
at dockerExec

[Eyal Shacham]

Thanks for the this article, Ken. I tried to add a lambda layer with pandas and numpy but cdk fails to deploy with the error of "Unzipped size must be smaller than 103178826 bytes". Is there a way to zip the layer before uploading and unzipping it after?

[Ken Choong]

I think nope. In case your lambda layer is bigger than the threshold, you should make it to 2 lambda layer, then your code can import from 2 lambda layers. It doesnt matter