Guide: AWS Lambda + Pillow for Complex Image Processing

In today’s digital landscape, images convey information, evoke emotions, and drive engagement. From social media platforms to e-commerce sites, businesses rely heavily on captivating images to secure their target audience’s attention. However, managing and processing these images can be challenging, demanding technical expertise and computational resources.

Imagine this scenario: A rapidly growing online marketplace is experiencing an influx of new products daily. The platform requires high-quality product images for display but struggles with size inconsistencies and varied file formats from different vendors. The company seeks an automated solution to resize uploaded images while maintaining quality resolutions to streamline the process and reduce manual intervention.

Enter AWS Lambda functions coupled with the powerful Python library — Pillow!

In this tutorial, we will walk you through integrating Pillow — a renowned image processing library — into your AWS Lambda function to create an optimized image resizing solution that addresses your complex problem effectively.

• *Note: This tutorial assumes a basic understanding of AWS Lambda functions and Python programming language.

Step 1: Preparing Your Python Environment with Shared Libraries (Pillow)

By installing the Pillow module into the specific folder ./python/lib/python3.10/site-packages/, we ensure that AWS Lambda can quickly locate and access the library during runtime. It also helps us maintain a clean and organized project structure, as AWS Lambda expects libraries to be in this folder for Python 3.10 runtime (check your runtime version).

Now that you understand why we target this specific folder, let’s proceed with creating our image resizing solution using AWS Lambda and Pillow:

1a. Open your terminal or command prompt and navigate to your desired project directory.

cd /path/to/your/project-folder

1b. Create a new folder named python in your project directory:

mkdir python

1c. Within the python folder, create another folder named lib. Inside it, create another one called python3.10. And finally, one last folder called site-packages inside the previous one:

mkdir -p python/lib/python3.10/site-packages/

1d. Now that you have created the appropriate directory structure for your Python libraries, install Pillow in the targeted location by executing this command:

pip3 install pillow -t ./python/lib/python3.10/site-packages/

This command installs the Pillow module into the specified path (./python/lib/python3.10/site-packages/) within your project’s directory structure.

In Step 4, you must zip that folder to create the Layer for your Lambda Function.

Step 2: Implementing Your Image Resizing Function

2a. In parallel to your “python” folder, create a new Python file named lambda_function.py:

touch lambda_function.py

2b. Open the lambda_function.py file using your favorite code editor and add the following code snippet:

import os
import boto3
from PIL import Image

def resize_image(image_path, resized_path):
    with Image.open(image_path) as image:
        image.thumbnail((800, 800))
        image.save(resized_path)

def lambda_handler(event, context):
    s3_client = boto3.client('s3')

    for record in event['Records']:
        bucket_name = record['s3']['bucket']['name']
        key_name = record['s3']['object']['key']

        download_path = '/tmp/{}'.format(os.path.basename(key_name))
        upload_path = '/tmp/resized-{}'.format(os.path.basename(key_name))

        s3_client.download_file(bucket_name, key_name, download_path)

        resize_image(download_path, upload_path)

        s3_client.upload_file(upload_path, bucket_name, 'resized-{}'.format(key_name))

This code defines an AWS Lambda handler that listens for new images uploaded to your desired S3 bucket. For each image detected, it will automatically download the file to the temporary Lambda environment storage ( /tmp ), resize it using Pillow’s thumbnail method (to a maximum width or height of 800 pixels ), and save the resized version locally /tmp, and re-upload it to the same S3 bucket with a ‘ resized- ‘ prefix.

Step 3: Creating Your AWS Lambda Layer for Pillow

Now that our Python function is ready, let’s package our Pillow library into an AWS Lambda layer.

Create a zip archive of your ./python/lib/python3.10/site-packages/ folder:

zip -r pillow_layer.zip python

This command compresses the contents of the whole python folder in which Pillow is located (./python/lib/python3.10/site-packages/) into a single pillow_layer.zip file.

Step 4: Deploying Your AWS Lambda Function and Layer

4a. Log in to your AWS Management Console, and navigate to the Lambda console.

4b. Click “Layers” from the left menu, then click “Create Layer”.

4c. Enter a name for your new layer (e.g., “ PillowLayer “), provide a description, upload pillow_layer.zip , choose Python 3.10 as the compatible runtime, and click “Create”.

4d. Next, navigate to the Functions tab and create a new Lambda function. Select Python 3.10 as the runtime environment.

Add an S3 trigger with appropriate access permissions to use your preferred bucket for image uploads.

4e.Add your newly created PillowLayer by clicking ‘Add Layer’ in the function designer pane.

Upload or write inline code for ‘ lambda_function.py ‘ when prompted during function creation and configure appropriate rights for accessing S3 resources via IAM role management.

Save settings when done.

Shared Library Object

The Pillow library requires a shared library to work correctly. You may face some issues like this one:

[ERROR] ImportError: /lib64/libz.so.1: version ZLIB_1.2.9' not found (required by /opt/python/lib/python3.10/site-packages/PIL/../Pillow.libs/libpng16-cb42a3c6.so.16.39.0) Traceback (most recent call last): File "/var/task/lambda_function.py", line 37, in lambda_handler render.render(output_file) File "/var/task/render.py", line 59, in render font = ImageFont.truetype("arial.ttf", int(obj['fontSize'])) File "/opt/python/lib/python3.10/site-packages/PIL/ImageFont.py", line 996, in truetype return freetype(font) File "/opt/python/lib/python3.10/site-packages/PIL/ImageFont.py", line 993, in freetype return FreeTypeFont(font, size, index, encoding, layout_engine) File "/opt/python/lib/python3.10/site-packages/PIL/ImageFont.py", line 220, in __init__ if core.HAVE_RAQM: File "/opt/python/lib/python3.10/site-packages/PIL/_util.py", line 19, in __getattr__ raise self.ex File "/opt/python/lib/python3.10/site-packages/PIL/ImageFont.py", line 58, in <module> from . import _imagingft as core [ERROR] ImportError: /lib64/libz.so.1: version ZLIB_1.2.9' not found (required by /opt/python/lib/python3.10/site-packages/PIL/../Pillow.libs/libpng16-cb42a3c6.so.16.39.0) Traceback (most recent call last): File "/var/task/lambda_function.py", line 37, in lambda_handler render.render(output_file) File "/var/task/render.py", line 59, in render font = ImageFont.truetype("arial.ttf", int(obj['fontSize'])) File "/opt/python/lib/python3.10/site-packages/PIL/ImageFont.py", line 996, in truetype return freetype(font) File "/opt/python/lib/python3.10/site-packages/PIL/ImageFont.py", line 993, in freetype return FreeTypeFont(font, size, index, encoding, layout_engine) File "/opt/python/lib/python3.10/site-packages/PIL/ImageFont.py", line 220, in init if core.HAVE_RAQM: File "/opt/python/lib/python3.10/site-packages/PIL/_util.py", line 19, in getattr raise self.ex File "/opt/python/lib/python3.10/site-packages/PIL/ImageFont.py", line 58, in <module> from . import _imagingft as core

The error: Missing Pillow dependency

This error occurs because Pillow has dependencies on specific shared libraries that are either unavailable or incompatible with the AWS Lambda environment. In this case, Pillow requires the libz.so.1 library with a specific version (ZLIB_1.2.9), which is missing or incompatible with AWS Lambda’s default environment.

To resolve this issue, we need to load the necessary shared libraries for Pillow and configure the environment variable so that our Python code can locate them properly.

Step 5A: Include Missing Shared Libraries

5a-1) Create a new “ lib ” folder in the root of your lambda function:

mkdir lib

5a-2) Download or compile the required Zlib library (libz.so.1) compatible with Amazon Linux 2 environments. After obtaining it, place it into your project’s newly created “lib” folder.

Add a new environment variable.

You must add a new variable called LD_LIBRARY_PATH to include /var/task/lib. It ensures that Python knows where to find any provided shared objects when checking for dependencies during the runtime execution of AWS Lambda functions.

Step 5: Testing Your Image Processing Solution

Finally, upload an image file into your specified S3 bucket as per the event trigger configuration set in the lambda.

AWS Lambda will automatically process and resize it using your implemented resizing function powered by Pillow!

You can verify its performance by checking that the resized image was successfully uploaded with a ‘resized-‘ prefix in its filename within the same S3 bucket.

That’s it! You’ve successfully leveraged AWS Lambda functions and Pillow’s capabilities to overcome complex image-processing challenges.

📣 Attention Aspiring Cloud Professionals! 📣

We have the perfect tool to supercharge your cloud knowledge journey – introducing our “AWS Learning Kit.” It’s FREE and designed to provide a comprehensive understanding of Amazon Web Services (AWS), the world’s leading cloud platform.

This is your chance to be part of the future. With more and more businesses migrating to the cloud daily, the demand for professionals navigating and optimizing these digital environments is soaring. That’s where AWS comes in. The skills you can gain from mastering AWS are essential to propel your career to the next level in this cloud-centric world.

AWS Learning Kit – VPC Mind Map

Our AWS Learning Kit is a treasure trove of resources, comprising 20 detailed Mind Maps that visually organize the complex world of AWS into an understandable and digestible format. But we didn’t stop there! The Kit also provides 260 carefully crafted Questions with in-depth Answers designed to test your knowledge and prepare you for real-world challenges.

Don’t let this opportunity slip away! Grab the AWS Learning Kit and kickstart your journey to becoming a cloud expert. There is no better time than NOW to invest in your future.

🚀 Download the FREE AWS Learning Kit today and take a giant leap toward a promising future in the Cloud. 🚀

Your future self will thank you.

Download AWS Learning Kit Here

Let’s make your Cloud dreams a reality together.

The post Guide: AWS Lambda + Pillow for Complex Image Processing appeared first on Bits Lovers.