DEV Community

Cover image for AWS for Newbies — Episode 3: The Magic Helper Booth
surajrkhonde
surajrkhonde

Posted on

AWS for Newbies — Episode 3: The Magic Helper Booth

👦 Nephew: Uncle, when a file finishes uploading to S3... nothing happens to it after that. Shouldn't something make a thumbnail, or check the file is safe, or tell the user "hey, it's ready"?

👨‍🦳 Uncle: Great catch. And here's the fun part — we're not going to leave a server running all day just to check "did a new file show up yet? did a new file show up yet?" That would be like paying a guard to stand at an empty door all day, just in case someone knocks once a week.

👦 Nephew: So what do we do instead?

👨‍🦳 Uncle: We build a magic helper booth.


Part 1: The Magic Helper Booth

👨‍🦳 Uncle: Imagine a booth on the street. Nobody is inside it. It looks empty. But the moment someone presses a button on the booth, a helper pops up out of nowhere, does exactly one small job, hands it over, and then... poof. Gone again. No helper standing there getting bored and costing you money.

Nobody presses the button  →  booth is EMPTY, costs nothing
Someone presses the button →  helper POPS UP  →  does the job  →  vanishes again
Enter fullscreen mode Exit fullscreen mode

👦 Nephew: That sounds like magic. What's it actually called?

👨‍🦳 Uncle: AWS Lambda. And this whole idea — paying for a helper only while they're actually working, never while they're standing around — is called serverless.

👦 Nephew: Wait, "serverless" means there's no server?

👨‍🦳 Uncle: Sneaky trick question, and a common mix-up. There is still a real computer somewhere running your helper. You just never see it, never own it, never have to clean it or fix it. AWS handles all of that backstage. You only write down: "when the button gets pressed, do THIS."


Part 2: What Makes the Helper Pop Up?

👦 Nephew: What's "the button" in real life?

👨‍🦳 Uncle: Lots of different buttons! Here are the common ones:

The Button What Presses It
📁 A new file lands in S3 Someone uploads a picture (exactly your question!)
🌐 Someone visits a web address A person clicks something on your app
⏰ A clock strikes a certain time "Every night at 2 AM, do a cleanup"
📬 A message drops into a mailbox (queue) A background job needs doing

👦 Nephew: So for our thumbnail problem, the "button" is a picture landing in S3?

👨‍🦳 Uncle: Exactly. The moment a picture lands, S3 presses the button itself — nobody has to check or ask. Here's the helper's actual instructions, written in code:

exports.handler = async (event) => {
  console.log("Someone pressed my button! Here's what happened:", event);
  return { message: "Job done!" };
};
Enter fullscreen mode Exit fullscreen mode

👦 Nephew: What's event?

👨‍🦳 Uncle: It's a little note someone hands the helper the moment they pop up, explaining what just happened. If a picture triggered it, the note says which picture, in which bucket:

{
  "bucket": "my-app-uploads",
  "fileName": "images/dog-photo.jpg"
}
Enter fullscreen mode Exit fullscreen mode

👦 Nephew: Like a sticky note stuck to the helper's hand the instant they appear?

👨‍🦳 Uncle: Exactly that picture.


Part 3: Let's Actually Build the Thumbnail Helper

👨‍🦳 Uncle: Here's the helper's full job, step by step, in plain words first:

1. A picture lands in S3
2. Helper pops up, reads the sticky note (which picture?)
3. Helper downloads the picture
4. Helper shrinks it into a small thumbnail
5. Helper uploads the thumbnail back to S3
6. Helper vanishes
Enter fullscreen mode Exit fullscreen mode

Now in real code:

const { S3Client, GetObjectCommand, PutObjectCommand } = require("@aws-sdk/client-s3");
const sharp = require("sharp"); // a tool for resizing pictures

const s3 = new S3Client({ region: "ap-south-1" });

exports.handler = async (event) => {
  const bucket = event.Records[0].s3.bucket.name;
  const key = event.Records[0].s3.object.key;

  // Don't shrink a thumbnail that's already been shrunk!
  if (key.startsWith("thumbnails/")) return;

  // Step 3: download the picture
  const original = await s3.send(new GetObjectCommand({ Bucket: bucket, Key: key }));
  const buffer = Buffer.concat(await original.Body.toArray());

  // Step 4: shrink it
  const small = await sharp(buffer).resize(200, 200).toBuffer();

  // Step 5: upload the small version
  const thumbKey = key.replace("images/", "thumbnails/");
  await s3.send(new PutObjectCommand({ Bucket: bucket, Key: thumbKey, Body: small }));

  console.log("Thumbnail made:", thumbKey);
};
Enter fullscreen mode Exit fullscreen mode

👦 Nephew: And our normal backend server from Episode 2 never had to do any of this?

👨‍🦳 Uncle: Never touched it. Didn't even know it was happening. That's the whole magic — one small job, handled by a helper who only exists for the few seconds it takes.


Part 4: Sometimes the Helper Is Slow, Sometimes Fast — Why?

👦 Nephew: Is the helper always equally fast?

👨‍🦳 Uncle: No, and here's why. Imagine the helper booth is way out in a warehouse. The first time someone presses the button, the helper has to run all the way from the warehouse to the booth. That takes a moment. This is called a cold start.

🔔 Button pressed
   ↓
🏃 Helper runs all the way from the warehouse (SLOW — a "cold start")
   ↓
✅ Job done
Enter fullscreen mode Exit fullscreen mode

But! If someone presses the button again soon after, the helper is still standing right there, hasn't gone back to the warehouse yet. So they can do the job instantly. That's called a warm start.

🔔 Button pressed again, quickly
   ↓
⚡ Helper is STILL THERE — no running needed (FAST — a "warm start")
   ↓
✅ Job done
Enter fullscreen mode Exit fullscreen mode

👦 Nephew: So if nobody presses the button for a while, the helper goes back to the warehouse?

👨‍🦳 Uncle: Exactly — and then the next press is slow again. Cold, warm, cold, warm — depends on how busy the booth's been lately.


Part 5: The Helper Has NO Memory

👦 Nephew: Can the helper remember stuff from the last time they helped someone?

👨‍🦳 Uncle: Here's the golden rule: never assume they remember anything. Every time the helper pops up, treat it like they've never met you before in their life — total stranger, blank memory.

// ❌ WRONG — pretending the helper remembers a number from last time
let count = 0;
exports.handler = async () => {
  count++; // This SOMETIMES works if the same helper is still around...
  console.log(count); // ...but sometimes a totally different, fresh
                        // helper shows up and count is back to 0!
};
Enter fullscreen mode Exit fullscreen mode

👦 Nephew: That sounds like a sneaky bug — it works sometimes and breaks other times?

👨‍🦳 Uncle: Exactly the dangerous part. If you need the helper to remember something important, write it down somewhere permanent — like a notebook (a database) — not in their own head.

// ✅ RIGHT — write it in the permanent notebook (database), not the helper's memory
await db.query("UPDATE counters SET count = count + 1 WHERE id = 1");
Enter fullscreen mode Exit fullscreen mode

Part 6: The Helper Only Gets 15 Minutes

👨‍🦳 Uncle: Every helper has a strict rule: 15 minutes, maximum, no exceptions. If they're not done by then, they get yanked away instantly — mid-job, wherever they are.

Helper working... working... working...
⏰ 15:00 minutes hits
❌ POOF — gone, whether finished or not
Enter fullscreen mode Exit fullscreen mode

👦 Nephew: That sounds harsh! What if they were almost done?

👨‍🦳 Uncle: Doesn't matter — gone anyway. That's why you should only give this helper quick jobs — resizing a picture, sending one email, checking one file. Long jobs, like editing a giant 40-minute video, need a different kind of worker (a real, always-on server) instead.


Part 7: When the Helper Messes Up

👦 Nephew: What happens if the helper fails halfway?

👨‍🦳 Uncle: Depends on how they got called.

If someone's waiting right there for an answer (like a website waiting for a response): the helper just says "sorry, I failed," and that's it. Nobody automatically tries again — whoever's waiting has to decide to ask again themselves.

If nobody's waiting around (like our picture-upload trigger): AWS is kind — it automatically asks a new helper to try again. Up to 2 extra tries.

🔔 Button pressed
   ↓
Helper #1 tries... FAILS
   ↓
AWS: "Let's try again" → Helper #2 tries... FAILS
   ↓
AWS: "One more try" → Helper #3 tries... FAILS
   ↓
😢 All 3 failed. What now?
Enter fullscreen mode Exit fullscreen mode

👦 Nephew: What happens after all 3 fail?

👨‍🦳 Uncle: If you set it up properly, the failed job gets put in a special "lost and found" box (called a Dead Letter Queue) so a human can go look at it later. If you don't set this up... the job just quietly disappears forever. Nobody ever knows it failed.

👦 Nephew: That sounds really important to set up, then!

👨‍🦳 Uncle: Extremely important. Never let failures vanish silently.

The Sneaky "Do It Twice" Problem

👦 Nephew: Wait — if the helper already half-finished the job (like, already made the thumbnail) before failing, and then a NEW helper tries again from the start... don't we get the job done twice?

👨‍🦳 Uncle: You just found one of the trickiest bugs in this whole topic! Yes — exactly that can happen. The fix: always start the job by asking, "wait, have I actually already done this one?"

exports.handler = async (event) => {
  const key = event.Records[0].s3.object.key;

  // Check the notebook FIRST: did an earlier helper already finish this?
  const already = await db.query("SELECT 1 FROM done_jobs WHERE file_key = $1", [key]);
  if (already.rows.length > 0) {
    console.log("Already done this one — skip it!");
    return;
  }

  // ...do the actual work...

  // Write it down so the NEXT helper (if there is one) knows it's done
  await db.query("INSERT INTO done_jobs (file_key) VALUES ($1)", [key]);
};
Enter fullscreen mode Exit fullscreen mode

Part 8: How Do You Know What the Helper Did?

👦 Nephew: Since the helper vanishes, how do we know what happened, especially if something went wrong?

👨‍🦳 Uncle: Every single thing the helper says out loud (every console.log) gets automatically written down in a big logbook called CloudWatch Logs. You never have to set this up — it just happens.

exports.handler = async (event) => {
  console.log("Starting the job!");
  try {
    // ...do work...
  } catch (err) {
    console.log("Uh oh, something broke:", err.message);
    throw err; // IMPORTANT: still tell AWS it failed!
  }
};
Enter fullscreen mode Exit fullscreen mode

👦 Nephew: Why is throw err important there?

👨‍🦳 Uncle: If you just print the error and don't throw it again, AWS thinks everything went fine! No retry happens, nothing gets saved to the lost-and-found box — because as far as AWS can tell, the helper said "all good!" Always let AWS know when something truly broke.


Part 9: Does It Cost a Lot?

👨‍🦳 Uncle: Here's the best part. You pay for exactly how long the helper actually worked — down to the millisecond. Not one second more.

50,000 thumbnails made in a month
Each one takes less than 1 second

Total cost: usually under $1 for the whole month!
Enter fullscreen mode Exit fullscreen mode

👦 Nephew: And compared to keeping a real, always-on server running all day for this?

👨‍🦳 Uncle: That would cost around $7-15 a month minimum — even on the nights nobody uploads anything. For a job that only happens sometimes, the magic helper booth wins easily.

👦 Nephew: So when would a real server actually be cheaper?

👨‍🦳 Uncle: If the helper had to work constantly, all day, every day, nonstop — like thousands of times per second, all day long — then paying "per tiny job" adds up to more than just having your own regular worker there the whole time. The magic helper is best for jobs that happen sometimes, with quiet gaps in between.


Part 10: Only Let the Helper Touch What They're Allowed To

👨‍🦳 Uncle: Just like we did with EC2 and S3 in earlier episodes, this helper needs a permission slip — exactly what they're allowed to look at, nothing more.

{
  "Allow": ["read pictures from images/", "write thumbnails to thumbnails/"]
}
Enter fullscreen mode Exit fullscreen mode

👦 Nephew: So the helper can't go poking around in our other files, like documents or user data?

👨‍🦳 Uncle: Correct — they physically can't, even if the code somehow tried to. The permission slip simply doesn't allow it. Same safety rule as always: only give exactly what's needed, nothing extra.


Part 11: Lots of Buttons, Lots of Helpers, All at Once

👦 Nephew: What if 500 people upload pictures at the exact same second?

👨‍🦳 Uncle: Then 500 separate helpers pop up all at once, each one handling their own single picture, all at the same time. Nobody waits in line.

500 pictures uploaded at once
      ↓
500 helpers pop up together, each doing ONE picture
      ↓
All roughly done around the same moment
Enter fullscreen mode Exit fullscreen mode

👦 Nephew: Whoa — what if 10,000 helpers all try to talk to our one small notebook (database) at the same second? Wouldn't that overload it?

👨‍🦳 Uncle: Very sharp question — yes, that could genuinely crash your notebook. So you can set a rule: "only let 20 helpers work at once, no matter how many buttons get pressed." The rest just wait their turn calmly, instead of all crashing into the poor little notebook together.


Recap — In Simple Words

👨‍🦳 Uncle: Let's say it all back, simply:

  • Lambda is a magic helper who only shows up when a button is pressed, does one job, and disappears.
  • You only pay for the seconds they actually work — nothing while they're gone.
  • Cold start = helper has to run over first (slow). Warm start = they're already there (fast).
  • The helper remembers nothing between jobs — write anything important down in a real notebook (database).
  • The helper only gets 15 minutes max before getting yanked away, finished or not.
  • If a job fails and nobody's waiting for an answer, AWS tries 2 more times automatically — then, if you set it up, moves it to a "lost and found" box.
  • Because of retries, always check "did I already do this?" before doing the job again.
  • Everything the helper says gets written in a logbook automatically — but you must throw real errors so AWS knows something broke.
  • This is perfect for jobs that happen sometimes, not jobs that run constantly all day.
  • The helper only gets a permission slip for exactly what they need — nothing more.
  • Lots of buttons pressed at once = lots of helpers at once — you can put a limit on that if it might overwhelm something else.

Glossary — Simple Definitions

Word What It Means, Simply
Serverless You never see or manage the actual computer — just write the job
Lambda The magic helper who pops up, does a job, disappears
Trigger The "button" that makes the helper appear
Event The sticky note explaining what just happened
Cold start Helper has to run over first — a bit slow
Warm start Helper's already there — fast
Statelessness The helper forgets everything after each job
Timeout The maximum time (up to 15 min) before the helper gets pulled away
Retry AWS automatically asking a new helper to try again after a failure
Dead Letter Queue The "lost and found" box for jobs that failed every single try
Idempotency Making sure doing a job twice by accident doesn't cause a mess
CloudWatch Logs The automatic logbook of everything the helper said
Execution role The helper's permission slip — exactly what they can touch
Concurrency How many helpers can pop up and work at the same time

What's Next

👦 Nephew: Uncle, this makes a lot more sense now. Can we get back to the CDN thing you mentioned last time?

👨‍🦳 Uncle: Yes! A person in Chennai and a person in Delhi both want the same picture. Right now, both have to fetch it all the way from one bucket in Mumbai. Next time, we fix that — with something called CloudFront, which keeps little copies of your files scattered all around the world, so everyone gets it fast, no matter where they are.

👦 Nephew: Episode 4!

👨‍🦳 Uncle: Episode 4.

Top comments (0)