DEV Community

Cover image for How to Import Pandas(library) in AWS Lambda Functions - AWS Lambda Layers
M. Abdullah Bin Aftab
M. Abdullah Bin Aftab

Posted on • Edited on

How to Import Pandas(library) in AWS Lambda Functions - AWS Lambda Layers

Imagine you need to run a Python script on the AWS Lambda function and you get this error πŸ”»

{
  "errorMessage": "Unable to import module 'lambda_function': No module named 'pandas',
  "errorType": "Runtime.ImportModuleError"
  ...
}
Enter fullscreen mode Exit fullscreen mode

Don't worry this is a common error and I am not going to make this long

How do you import Pandas in AWS Lambda Functions?

There are several ways but I am going to give you the easiest way to import pandas in AWS Lambda Function is to add Lambda Layer πŸ₯ͺ

What is AWS Lambda Layer?

It is a πŸ§€ cheese layer in Lambda Function containing additional code like libraries, dependencies, etc.

In Simple Words

AWS Lambda Layers are like building blocks for your functions. Imagine you need extra tools (like the Pandas library) to complete a project.

Instead of packing all those tools inside every single project (which wastes space and time), AWS allows you to create layers of tools (libraries, dependencies, or shared code).

These layers sit outside your main function but are always available when your function needs them.

In short, Lambda Layers help you:

Separate the main logic from the extra libraries to save space in your code. You can reuse libraries and code across multiple Lambda functions.

Easily update or manage your dependencies without changing your core function code.

Think of layers as an extra storage box attached to your Lambda function, holding everything your function needs to work smoothly. You can stack multiple layers on your function without cluttering your main code.

Steps to add Lambda Function Layer and Import Pandas

It takes only 3 steps to run Pandas in your Lambda Function successfully

Step 1 - Open Lambda Function through your AWS Management Console

As you can see we have an option Layers under the name of our Lambda Function, in my case, it's "import-pandas-function" and the Layers count is 0

Image description

Step 2- Add Script in your AWS Lambda Function

This step is further divided into two steps because we need to add a Python script that contains some Pandas code and write a test event in JSON to verify whether the code is running correctly.

2.1 - Add Python script - you can copy this code 👻

import json
import pandas as pd

def lambda_handler(event, context):
    data = event.get('data', [])

    df = pd.DataFrame(data)

    if not df.empty:
        mean_value = df['column_name'].mean()

        result = {
            "mean_value": mean_value,
            "data_shape": df.shape,
            "summary": df.describe().to_dict()
        }
    else:
        result = {
            "message": "Empty DataFrame"
        }

    # Return the response
    return {
        'statusCode': 200,
        'body': json.dumps(result)
    }

Enter fullscreen mode Exit fullscreen mode

2.2 - Add Test script in Json in the test tab - you can copy this code 👻

{
  "data": [
    {"column_name": 10, "other_column": "A"},
    {"column_name": 20, "other_column": "B"},
    {"column_name": 30, "other_column": "C"},
    {"column_name": 40, "other_column": "D"}
  ]
}

Enter fullscreen mode Exit fullscreen mode

Press the test button you probably got the πŸ”΄error:-
"errorMessage": "Unable to import module 'lambda_function': No module named 'pandas',
"errorType": "Runtime.ImportModuleError"
...

Step 3 - Add AWS Lambda Layer to Successfully run the Pandas in your Code

Scroll down to your Lambda Function, you probably can see the "Layers" separate section at the end of the page
Image description

3.1 - Click "Add a Layer"

After Clicking the "Add a Layer" you can see the page which has a couple of sections "Function runtime settings" and "Choose a layer"
Image description

3.2 - Click "AWS layers"

You can see three options in the "Choose a layer" section click the "AWS layers".
Image description

3.3 - Choose "AWS layers"

After selecting the AWS layers you can see the dropdown under "AWS layers".
Image description

3.4 - Choose "AWS layers" and "Version"

In a dropdown of "AWS layers" select -> AWSSDKPandas-Python312
Image description
In a dropdown of "Version" select -> 13(select the most one)
Image description
click the "Add" button
Image description

3.5 - Make sure the "Function Overview"

When your page is directed to the function overview you can see the layer is added below the function name "import-pandas-function"
Image description

Step 4 - Test the Function

You've successfully got the Response "statusCode": 200

{
  "statusCode": 200,
  "body": "{\"mean_value\": 25.0, \"data_shape\": [4, 2], \"summary\": {\"column_name\": {\"count\": 4.0, \"mean\": 25.0, \"std\": 12.909944487358056, \"min\": 10.0, \"25%\": 17.5, \"50%\": 25.0, \"75%\": 32.5, \"max\": 40.0}}}"
}

Enter fullscreen mode Exit fullscreen mode

Image description


Keep Coding 😎

Top comments (0)