Automating Image Thumbnails with AWS Lambda and S3: A Step-by-Step Guide

Introduction:

I recently completed an exciting project automating image thumbnail generation using AWS Lambda and Amazon S3. The best part? It's entirely serverless, meaning I didn’t have to worry about managing infrastructure—just focused on building the solution. This workflow is ideal for dynamically processing large-scale images without any overhead!

In this guide, I’ll walk you through the steps to build this system from scratch, sharing insights and challenges I faced along the way. Let's get started!

Step 1: Setting Up the S3 Buckets
To begin, we need two Amazon S3 buckets:
-Source Bucket: This is where users will upload the original images.
-Destination Bucket: Thumbnails will be stored here after Lambda processes the images.

Here’s a quick breakdown:

1. Go to the AWS Management Console and create the two S3 buckets.
2. Configure access permissions, ensuring the correct bucket policies are applied.

Figure1 Access S3 console.
Figure 1.1 create bucket.
Figure 2 Complete create bucket

Step 2: Configuring IAM Role for Lambda Execution

Security is crucial. We need a role that allows Lambda to:

• Access the S3 buckets for reading/writing images.
• Log events to CloudWatch for monitoring.

I used a managed policy (AmazonS3FullAccess) alongside custom permissions for logging in CloudWatch.

1. Create and attach the necessary policies for S3 and CloudWatch.

Figure 3 IAM dashboard
Figure 3.1 Create policy
Figure 3.1.1 Complete create policy
Figure 3.1.2 Complete create policy
Figure 3.1.3 Policy created

1. In the IAM Console, create a new role with basic Lambda execution permissions.

Figure 4 Create role

Figure 4.1 Choose service
Figure 4.1.1 Select lambda service

Figure 4.1.2 Add permission

Figure 4.1.3 Review
Figure 4.1.4 Complete configure role
Figure 4.1.5 Complete configure role
Figure 4.1.6 Role created

Step 3: Creating the Lambda Function

Now, let's move on to the Lambda function itself.

• Navigate to AWS Lambda, and click "Create Function."
• Select Python 3.9 as the runtime (you can use other languages, but I used Python for this project).

The function listens for an S3 event, processes the image, generates a thumbnail using the Pillow library, and uploads it to the second bucket.

figure 5 Lambda function console
Figure 5.1 Create lambda function
Figure 5.1.1 Complete create lambda
Figure 5.1.2 Add code to lambda function

Here’s a snippet of the code I used:

import boto3
import os
import sys
import uuid
from urllib.parse import unquote_plus
from PIL import Image

s3_client = boto3.client('s3')

def resize_image(image_path, resized_path):
    with Image.open(image_path) as image:
        image.thumbnail(tuple(x / 2 for x in image.size))
        image.save(resized_path)

def lambda_handler(event, context):
    for record in event['Records']:
        bucket = record['s3']['bucket']['name']
        key = unquote_plus(record['s3']['object']['key'])
        tmpkey = key.replace('/', '')
        download_path = '/tmp/{}{}'.format(uuid.uuid4(), tmpkey)
        upload_path = '/tmp/resized-{}'.format(tmpkey)
        s3_client.download_file(bucket, key, download_path)
        resize_image(download_path, upload_path)
        s3_client.upload_file(upload_path, '{}-resized'.format(bucket), 'resized-{}'.format(key))

Step 4: Adding an S3 Trigger

The Lambda function is triggered automatically whenever an image is uploaded to the source bucket.

• In the Lambda function's configuration, scroll down to "Add Trigger" and select S3.

Figure 6 Add trigger (s3 source bucket)
Figure 6.1 Complete add trigger
Figure 6.1.1 Function overview

• Configure it to trigger on PUT events, meaning any new image upload will start the function.

Figure 6.2 create test event

Here's example of test event

{
  "Records": [
    {
      "eventVersion": "2.0",
      "eventSource": "aws:s3",
      "awsRegion": "us-east-1",
      "eventTime": "1970-01-01T00:00:00.000Z",
      "eventName": "ObjectCreated:Put",
      "userIdentity": {
        "principalId": "xxxxxxxxxx"
      },
      "requestParameters": {
        "sourceIPAddress": "127.0.0.1"
      },
      "responseElements": {
        "x-amz-request-id": "EXAMPLE123456789",
        "x-amz-id-2": "EXAMPLE123/5678abcdefghijklambdaisawesome/mnopqrstuvwxyzABCDEFGH"
      },
      "s3": {
        "s3SchemaVersion": "1.0",
        "configurationId": "testConfigRule",
        "bucket": {
          "name": "photo-project1",
          "ownerIdentity": {
            "principalId": "xxxxxxxx"
          },
          "arn": "arn:aws:s3:::photo-project1"
        },
        "object": {
          "key": "cute-kitten-4k-im-1600x900.jpg",
          "size": 1024,
          "eTag": "0123456789abcdef0123456789abcdef",
          "sequencer": "0A1B2C3D4E5F678901"
        }
      }
    }
  ]
}

Step 5: Testing the Solution

Now, upload a sample image to the source S3 bucket. The Lambda function should:

1. Automatically get triggered.
2. Process the image and create a thumbnail.
3. Store the thumbnail in the destination bucket.

Figure 7 upload two photos in source bucket
Figure 7.1 Two photos after resizing
Figure 7.2 Photo resized
Figure 7.3 Photo resized

💡 Challenge:
I initially encountered issues with image formats and timeouts. Optimizing the image processing code to handle larger images and using the appropriate timeout settings resolved these.

Step 6: Monitoring with CloudWatch

Once the function is up and running, you can use CloudWatch to monitor the executions and logs. It’s essential for checking if the function ran successfully or troubleshooting issues like failures.

Conclusion:

This project demonstrated the power of serverless architecture and how easily AWS services like Lambda and S3 can handle dynamic tasks such as image processing at scale. By leveraging AWS’s scalability, this solution is efficient and cost-effective, especially for large-scale workloads. I also gained valuable insights into optimizing Lambda functions and managing event-driven architectures.

Key AWS Services:

• AWS Lambda
• Amazon S3
• IAM for security
• CloudWatch for monitoring

If you're working on automating processes in the cloud, AWS Lambda is a game-changer! Feel free to try this solution and let me know how it goes.