Building Your First Event-Driven Pipeline with S3 and Lambda

How to wire an S3 bucket to a Lambda function so that every file upload automatically triggers code without polling, without servers, and without a single cron job.

---

Up until this point in the series, our AWS setup has been passive. S3 holds files, CloudFront delivers them, Route 53 routes traffic. Everything waits to be asked.

This project changes that. We're going to make AWS react. Drop a file in an S3 bucket, and within milliseconds, a Lambda function wakes up, reads the event, and does something useful log the filename, write to CloudWatch, or fire off an email confirmation via SES.

This is event-driven architecture in its simplest and most teachable form. And once you understand it at this scale, you'll recognise the same pattern everywhere in notification systems, data pipelines, media processing workflows, and audit trails.

"A server waits. An event-driven function listens. There's a difference and it matters at scale."

---

What We're Building

A file lands in S3 → S3 fires an event → Lambda receives it → Lambda acts on it.

📤 You (upload a file)
    → 🪣  S3 Bucket (fires ObjectCreated event)
        → λ  Lambda Function (wakes up, processes event)
            → 📋  CloudWatch Logs (captures output)
            → ✉️  SES (sends email notification)

We'll implement all three response options so you understand each one:

Option	What it does
Log to CloudWatch	Print the filename and event details to CloudWatch Logs
Email via SES	Send a confirmation email when a file arrives
Both	Log and notify — the real-world pattern

---

Prerequisites

Before starting, make sure you have:

• An AWS account with access to S3, Lambda, IAM, CloudWatch, and optionally SES
• Basic familiarity with the AWS console this builds directly on the S3 knowledge from our two previously posted articles and here are the links; Link1 and Link2 ICYMI.
• If using the SES email option: a verified email address in SES would also be needed (covered in Step 4)

---

Step 1 — Create the S3 Bucket

If you already have an S3 bucket from our previous article, you can reuse it or create a fresh one for this exercise. Navigate to S3 → Create bucket.

• Give it a unique name, e.g. yourname-upload-trigger-demo
• Choose a region — note it down, Lambda must be in the same region
• Leave all other settings as default for now

⚠️ Keep bucket and Lambda in the same region: S3 event notifications trigger Lambda functions within the same region. If they're in different regions, the trigger won't work. Pick a region at the start and stick to it throughout this project.

---

Step 2 — Create the Lambda Function

Navigate to Lambda → Create function.

• Select Author from scratch
• Function name: s3-upload-handler
• Runtime: Python 3.12 (or latest available)
• Architecture: x86_64
• Execution role: select Create a new role with basic Lambda permissions

Click Create function. Lambda provisions your function and drops you into the inline code editor.

💡 What just happened with that IAM role? Lambda created an execution role automatically. This role grants your function permission to write logs to CloudWatch Logs. For the SES option, you'll need to extend this role manually covered in Step 4.

The event structure Lambda receives

Before writing code, it helps to understand what Lambda actually gets. When S3 fires an event, it sends a JSON payload that looks like this:

{
  "Records": [
    {
      "eventName": "ObjectCreated:Put",
      "s3": {
        "bucket": { "name": "yourname-upload-trigger-demo" },
        "object": {
          "key": "reports/q4-summary.pdf",
          "size": 204800
        }
      }
    }
  ]
}

Every upload becomes a record in this array. Your Lambda function digs into this structure to extract the bucket name, file key, and event type.

Option A — Log to CloudWatch

Paste this into the Lambda inline editor and click Deploy:

import json

def lambda_handler(event, context):
    for record in event['Records']:
        bucket = record['s3']['bucket']['name']
        key    = record['s3']['object']['key']
        size   = record['s3']['object']['size']

        print(f"✅ New upload detected")
        print(f"   Bucket : {bucket}")
        print(f"   File   : {key}")
        print(f"   Size   : {size} bytes")

    return {'statusCode': 200, 'body': 'Logged successfully'}

✅ In Lambda, every print() goes to CloudWatch. You don't need to import a logging library. Lambda automatically streams all stdout to CloudWatch Logs. Whatever you print() shows up there within seconds.

Option B — Send an email via SES

For this option, you'll need a verified sender address in SES first (covered in Step 4). Once that's done, use this function:

import json
import boto3

ses = boto3.client('ses', region_name='us-east-1')

SENDER    = "[e-mail]"   # must be SES-verified
RECIPIENT = "[e-mail]"   # can be same address

def lambda_handler(event, context):
    for record in event['Records']:
        bucket = record['s3']['bucket']['name']
        key    = record['s3']['object']['key']
        size   = record['s3']['object']['size']

        ses.send_email(
            Source=SENDER,
            Destination={'ToAddresses': [RECIPIENT]},
            Message={
                'Subject': {'Data': f"New file uploaded to {bucket}"},
                'Body': {'Text': {'Data':
                    f"File uploaded:\n\nBucket: {bucket}\nKey: {key}\nSize: {size} bytes"
                }}
            }
        )
        print(f"Email sent for upload: {key}")

    return {'statusCode': 200, 'body': 'Email sent'}

---

Step 3 — Add the S3 Trigger

Now we connect S3 to Lambda. In your Lambda function, click Add trigger in the function overview diagram at the top of the page.

• Trigger source: S3
• Bucket: select your bucket
• Event types: PUT — this fires on every new upload
• Optionally add a Prefix (e.g. uploads/) or Suffix (e.g. .pdf) to filter which uploads trigger the function
• Acknowledge the recursive invocation warning and click Add

⚠️ The recursive invocation warning — take it seriously. If your Lambda function writes files back to the same S3 bucket it's triggered from, you'll create an infinite loop: upload → trigger → Lambda writes file → trigger → Lambda writes file → ... Your bill will spike before you notice. Either write to a different bucket, use a prefix/suffix filter, or simply don't write back to the same bucket from this function.

🔍 What's happening under the hood: When you add the trigger in the Lambda console, AWS does two things automatically: it adds an S3 bucket notification configuration on the bucket, and it adds a resource-based policy to your Lambda function allowing S3 to invoke it. You can inspect both. The bucket notification config lives under S3 → Properties → Event notifications, and the Lambda resource policy lives under Lambda → Configuration → Permissions.

---

Step 4 — Grant Lambda the Right Permissions

The default Lambda execution role can write to CloudWatch Logs, which is enough for Option A. For SES email sending, you need to extend the role.

Add SES send permission to the Lambda role

Go to IAM → Roles, find the role Lambda created (it will be named something like s3-upload-handler-role-xxxxxxxx), and click Add permissions → Attach policies. Search for and attach AmazonSESFullAccess.

⚠️ AmazonSESFullAccess is broad — fine for learning, not for production. For a real system, create a custom IAM policy that grants only ses:SendEmail on the specific identities you need. The principle of least privilege applies here just as much as it did with the S3 bucket policy in our previous article.

Verify your sender address in SES

Navigate to SES → Verified identities → Create identity.

• Select Email address
• Enter the address you'll send from
• Click Create identity
• Check your inbox and click the verification link AWS sends

💡 SES sandbox mode: By default, new AWS accounts are in the SES sandbox. In sandbox mode, you can only send emails to verified addresses both the sender AND recipient must be verified. This is fine for testing. To send to anyone, you need to request production access from the SES console.

---

Step 5 — Test It End to End

Upload any file to your S3 bucket the console, CLI, or a drag-and-drop will all work.

# Upload via CLI
aws s3 cp ./test-document.pdf s3://yourname-upload-trigger-demo/

Check CloudWatch Logs

Navigate to CloudWatch → Log groups. Look for a log group named /aws/lambda/s3-upload-handler. Click the most recent log stream and you should see your print statements:

✅ New upload detected
   Bucket : yourname-upload-trigger-demo
   File   : test-document.pdf
   Size   : 204800 bytes

Test Lambda directly (before connecting S3)

You don't have to upload a real file to test your function. In the Lambda console, click Test and paste a sample event to simulate what S3 sends:

{
  "Records": [{
    "eventName": "ObjectCreated:Put",
    "s3": {
      "bucket": { "name": "yourname-upload-trigger-demo" },
      "object": { "key": "test-document.pdf", "size": 204800 }
    }
  }]
}

This lets you iterate on your function code quickly without uploading files every time.

---

Troubleshooting Common Issues

Lambda isn't firing on upload

Check the S3 bucket notification configuration under S3 → Properties → Event notifications. Confirm the event type is set to PUT (or All object create events) and the Lambda ARN is correct. Also confirm both resources are in the same region.

Lambda fires but CloudWatch shows no logs

The Lambda execution role likely doesn't have logs:CreateLogGroup, logs:CreateLogStream, and logs:PutLogEvents permissions. The default role created by Lambda should include these, but if you created a custom role, add the AWSLambdaBasicExecutionRole managed policy.

SES returns an error: Email address not verified

In sandbox mode, both the sender and recipient addresses must be verified in SES. Go to SES → Verified identities and confirm both addresses have a "Verified" status badge.

Function times out

The default Lambda timeout is 3 seconds. If you're making external calls (like SES), it can occasionally be slow. Raise the timeout to 10–15 seconds under Lambda → Configuration → General configuration.

---

What I Learned From This Project

Of all the projects in the Cloud Engineering Program, this was the one where cloud architecture stopped feeling abstract. Watching a file upload immediately trigger code execution with no server, no polling loop, no scheduler made the event-driven model click in a way that diagrams never had.

A few things that genuinely landed:

• Event-driven is fundamentally different from request-driven — your function doesn't wait for someone to call it. The infrastructure calls it for you, the moment something happens. That changes how you design systems entirely.
• IAM is everywhere, and it's intentional — Lambda needs permission to be invoked by S3. Lambda needs permission to send email via SES. Lambda needs permission to write logs. Every cross-service interaction requires an explicit grant. Once you internalise this, AWS architecture starts to feel less like bureaucracy and more like a coherent security model.
• print() is enough to start — CloudWatch captures everything. You don't need a logging framework, a log server, or a third-party observability tool to get started. The primitives are already there.
• The event payload is your contract — understanding the exact structure S3 sends to Lambda, and writing code that correctly parses it, is the real skill here. The rest is just AWS console navigation.

---

Quick Reference: The Full Setup

# Services and their roles

S3 bucket          → stores files, fires ObjectCreated events on upload
S3 event notif.    → routes PUT events to Lambda (configured on the bucket)
Lambda function    → Python handler, wakes up per event, processes Records[]
Lambda exec. role  → IAM role granting CloudWatch Logs write + SES send
CloudWatch Logs    → captures all print() output from Lambda automatically
SES                → sends transactional email (sandbox: verified addrs only)

# The trigger chain
Upload file to S3  → S3 fires event → Lambda invoked → logs + email sent

---

What's Next?

Now that you understand event-driven triggers, the natural next step is extending the Lambda function to do something more powerful like resizing images on upload, parsing CSV files and storing records in DynamoDB, or triggering a Step Functions workflow. The same S3 → Lambda pattern underpins all of these. Once you have the trigger working, the sky is the limit on what the function does.

I’m also excited to share that I’ve been able to secure a special discount, in partnership with Sanjeev Kumar’s team, for the DevOps & Cloud Job Placement / Mentorship Program.

For those who may not be familiar, Sanjeev Kumar brings over 20 years of hands-on experience across multiple domains and every phase of product delivery. He is known for his strong architectural mindset, with a deep focus on Automation, DevOps, Cloud, and Security.

Sanjeev has extensive expertise in technology assessment, working closely with senior leadership, architects, and diverse software delivery teams to build scalable and secure systems. Beyond industry practice, he is also an active educator, running a YouTube channel dedicated to helping professionals successfully transition into DevOps and Cloud careers.

This is a great opportunity for anyone looking to level up their DevOps/Cloud skills with real-world mentorship and career guidance.

Do refer below for the link with a dedicated discount automatically applied at checkout;

DevOps & Cloud Job Placement / Mentorship Program.

If you also found this interesting and would love to take the next steps in the application process with AltSchool Africa do use my referral link below;

Apply here or use this Code: W2jBG8 during the registration process and by so doing, you will be supporting me and also getting a discount!

Special Offer: By signing up through the link and using the code shared, you’ll receive a 10% discount!

Don’t miss out on this opportunity to transform your future and also save while doing it! Let’s grow together in the tech space. Also feel free to reach out if you need assistance or clarity regarding the program.

I’m Ikoh Sylva, a passionate cloud computing enthusiast with hands-on experience in AWS. I’m documenting my cloud journey here from a beginner’s perspective, aiming to inspire others along the way.

If you find my contents helpful, please like and follow my posts, and consider sharing this article with anyone starting their own cloud journey.

Let’s connect on social media. I’d love to engage and exchange ideas with you!

LinkedIn Facebook X

---