DEV Community: Jude Wakim

Building My S3 Security Scanner: A Solo Dev's Journey to Automate AWS Bucket Safety

Jude Wakim — Sun, 28 Sep 2025 17:33:59 +0000

Introduction: Why I Built This (And What You’re Getting Into)

Hey folks, Jude here — a DevSecOps engineer who spends way too much time wrangling AWS resources and not enough coffee breaks. If you’ve ever spun up an S3 bucket in a rush during a late-night coding sprint, only to realize it’s wide open to the internet the next day, this one’s for you.

I created the S3 Security Scanner to tackle that exact headache: a simple, serverless tool that scans your AWS S3 buckets for common misconfigurations (like public access or wildcard policies) and optionally fixes them automatically.

The goal? Give solo devs and small teams like mine a “set it and forget it” way to keep buckets secure without constant manual audits. It’s not a full enterprise suite — it’s an MVP I built in a few evenings to boost my skills and maybe sell on the AWS Marketplace someday.

Tech stack:

Python with Boto3 for the Lambda core,
CloudFormation for deployment, and
EventBridge for daily runs.

Purpose: Peace of mind in the cloud, one bucket at a time.

Let’s dive into how it came together.

The Build Process: From Idea to Running Code

I started with the basics: What pains me most about S3? Public buckets leaking data, right? So, the scanner checks four key areas — Public Access Block settings, ACL grants, bucket policies for wildcards, and default encryption — flagging risks with a simple severity score (none, low, medium, high).

The heart is a Lambda function. I sketched it out in pseudocode first: Parse an event for config (like excluded buckets), scan via Boto3 calls (list_buckets, get_bucket_acl, etc.), and output JSON risks. Then, for remediation, an optional step applies fixes like put_bucket_acl(ACL=’private’) or enabling Public Access Block — all with safety nets, like dry-run previews.

Judewakim / s3-misconfig

S3 Sentry: The Autonomous Data Security Officer

S3 Sentry is a high-fidelity, multi-tenant security orchestration platform designed to provide continuous oversight of AWS S3 storage environments. It acts as an automated "Data Security Officer," ensuring that organizational data remains private, encrypted, and compliant without manual intervention.

The Mission

In the modern cloud era, a single misconfigured S3 bucket can lead to catastrophic data exposure. S3 Sentry bridges the gap between complex AWS IAM policies and actionable security intelligence by providing a "zero-friction" onboarding experience and automated remediation scanning.

Core Architecture

S3 Sentry utilizes a Cross-Account Trust Handshake to scan client environments securely.

The Handshake: Uses a unique ExternalID protocol to prevent "Confused Deputy" attacks.
The Engine: Powered by an orchestrated Prowler CLI integration for industry-standard security checks.
The Single-Table Design: Leverages Amazon DynamoDB to manage thousands of tenants and millions of findings within a high-performance, scalable schema.

…

View on GitHub

Deployment was straightforward: Zipped the code, uploaded to my S3 bucket, and wrapped it in a CloudFormation template. Users deploy the stack, pick a mode (scanning only or auto-remediation), and EventBridge kicks it off daily. Total time? About a 5 hours of evenings, thanks to Claude Code for code gen and local testing with Grok.

Obstacles: The Bumps That Made It Better

Nothing’s smooth — first, I hit a OperationNotPageableError trying to paginate list_buckets (spoiler: S3 doesn’t need it; one call gets everything). Fixed by ditching the paginator. Permissions were of course an issue: AccessDenied on encryption checks until I added s3:GetEncryptionConfiguration to the IAM role. And exclusions? My test event had a string instead of a list — classic JSON gotcha. Each snag taught me to test locally first, then deploy iteratively.

Wrapping Up: What’s Next?

This project’s my love letter to secure-by-default cloud work — simple, effective, and fun to build. If you’re digging this, check out my next pieces: one on nailing the scanning logic, another on safe automation, and the final on CloudFormation magic.

Follow along on Medium or Dev.to for more DevSecOps adventures. What’s your biggest S3 headache? Drop a comment — let’s chat!

I regularly share hands-on cloud builds, automation tricks, and AWS-focused deep dives across the web:

🔗 LinkedIn — for cloud content, networking, and consulting
📖 Medium — where this and other walkthroughs live
👨‍💻 GitHub— for open-source tools and infra templates
🖥️ Dev.to — cross-posts and project write-ups

If you want me to help write the Terraform files or the user data script for your article too, just say the word!

How to Use Docker Compose and Python to Automate Your Jenkins Environment

Jude Wakim — Sat, 24 May 2025 16:18:47 +0000

If you’ve ever dipped your toes into DevOps, chances are you’ve heard about Docker, Jenkins, or automation at some point. In this post, I’ll walk you through a fun little project where I used Docker Compose and Python to automate the creation of a Jenkins environment from scratch. Whether you're just getting into CI/CD tools or looking to add more automation into your workflow, this post should give you some hands-on insight.

🐳 What is Docker and Why It’s a Game-Changer

Docker is basically your best friend when it comes to creating consistent environments across machines. It lets you package applications and all their dependencies into containers so they can run anywhere — your laptop, a server, a cloud instance — without the "but it works on my machine" headache.

It’s lightweight, fast, and makes spinning up complex environments surprisingly easy. Instead of installing Jenkins (or any other tool) directly on your machine, you just run a container with everything preconfigured. Simple, clean, and reversible.

📦 Enter Docker Compose

Docker Compose is like Docker’s orchestration tool for your local environment. It allows you to define multi-container applications in a single docker-compose.yml file and spin them all up with just one command. Perfect for projects that need more than one moving piece (like Jenkins and a custom volume or service).

It’s kind of like having a remote control for all your Docker containers — and trust me, once you start using it, you’ll wonder how you ever managed without it.

⚙️ Why Automation Matters

Manually repeating setup steps is a productivity killer. Automation helps reduce errors, saves time, and makes your projects way more maintainable. For this project, I automated:

Building a custom Jenkins image
Launching a Jenkins container
Mounting persistent data
Verifying that the setup worked

Using Python, I wrapped all those commands into a simple script so that I (or anyone else) could spin up the environment with one command. Clean and scalable.

🛠️ Step-by-Step: Building the Project

Let’s break down what I actually did to bring this to life.

1. Wrote the `docker-compose.yml`

This file defines the Jenkins container, ports, volume, and image. Here’s a simplified version:

version: "3.8"
services:
  jenkins:
    build: .
    container_name: my-custom-jenkins
    ports:
      - "8080:8080"
    volumes:
      - jenkins_data_custom:/var/jenkins_home

volumes:
  jenkins_data_custom:

This lets me build a Jenkins container from a custom image and persist the Jenkins home directory with a volume.

2. Created a Dockerfile

The Dockerfile defines the custom Jenkins image:

FROM jenkins/jenkins:lts

USER root

RUN apt-get update && apt-get install -y \
    sudo \
    curl \
    git \
    && rm -rf /var/lib/apt/lists/*

USER jenkins

Pretty straightforward. It gives Jenkins root privileges temporarily to install any required packages.

3. Wrote the Automation Script (automate.py)

This Python script uses the Docker SDK for Python to run all the container logic automatically:

import docker  # type: ignore
import time

client = docker.from_env()

IMAGE_NAME = "my-custom-jenkins"
VOLUME_NAME = "jenkins_data_custom"
CONTAINER_NAME = "my-custom-jenkins"
DOCKERFILE_PATH = "." 

# Step 2: Build the custom Jenkins image
print(f"Building image '{IMAGE_NAME}'...")
image, build_logs = client.images.build(path=DOCKERFILE_PATH, tag=IMAGE_NAME)
print("Image built successfully.")

# Step 3: Create volume
try:
    volume = client.volumes.get(VOLUME_NAME)
    print(f"Volume '{VOLUME_NAME}' already exists.")
except docker.errors.NotFound:
    volume = client.volumes.create(name=VOLUME_NAME)
    print(f"Volume '{VOLUME_NAME}' created.")

# Step 4: Run the container
print(f"Running container '{CONTAINER_NAME}'...")
try:
    container = client.containers.run(
        IMAGE_NAME,
        name=CONTAINER_NAME,
        ports={"8080/tcp": 8080},
        volumes={VOLUME_NAME: {'bind': '/var/jenkins_home', 'mode': 'rw'}},
        detach=True
    )
except docker.errors.APIError as e:
    print(f"Error: {e}")
    print("Trying to remove existing container and retry...")
    try:
        old_container = client.containers.get(CONTAINER_NAME)
        old_container.stop()
        old_container.remove()
        container = client.containers.run(
            IMAGE_NAME,
            name=CONTAINER_NAME,
            ports={"8080/tcp": 8080},
            volumes={VOLUME_NAME: {'bind': '/var/jenkins_home', 'mode': 'rw'}},
            detach=True
        )
    except Exception as e2:
        print(f"Failed to recover: {e2}")
        exit(1)

print("Container is starting. Waiting 15 seconds for Jenkins to initialize...")
time.sleep(15)  # Wait for Jenkins to generate password

# Step 5: Get the initial admin password
exec_log = container.exec_run("cat /var/jenkins_home/secrets/initialAdminPassword")
admin_password = exec_log.output.decode().strip()
print(f"\n  Jenkins Initial Admin Password:\n{admin_password}\n")

Just run this script, and your Jenkins container is up and running.

🧱 Common Roadblocks I Hit

No project is complete without a few bumps along the way. Here are a couple that tripped me up:

❌ mypy Not Recognized in PowerShell

Even after installing mypy, I kept seeing errors like:

mypy: The term 'mypy' is not recognized...

Turns out, PowerShell doesn’t run scripts from the current directory by default. The fix? Run it like this:

.\mypy

Or better yet, add the script location to your PATH.

🐍 ModuleNotFoundError: No module named 'docker'

Even though Docker was installed and working, Python couldn’t find the Docker SDK. I had to run:

pip install docker

Always make sure you’re using the correct Python environment (virtualenvs help!).

✅ Final Thoughts

If you’re learning DevOps or just want to get better with Docker and automation, I highly recommend trying something similar. Small projects like these sharpen your skills and make bigger tools like Jenkins feel a lot less intimidating.

Check out the full repo here:
👉 https://github.com/Judewakim/CD-with-Docker-and-Jenkins

And if you liked this article, feel free to drop a comment or share it with someone diving into DevOps!

🧭 Follow My Cloud Journey

I regularly share hands-on cloud builds, automation tricks, and AWS-focused deep dives across the web:

🔗 LinkedIn — for cloud content, networking, and consulting
📖 Medium — where this and other walkthroughs live
👨‍💻 GitHub — for open-source tools and infra templates
🖥️ Dev.to — cross-posts and project writeups
Until next time — secure your containers and keep building 🔐🐳⚙️

Automating My Docker Apache Server with Python

Jude Wakim — Thu, 15 May 2025 22:51:10 +0000

After building my first Docker project, I knew I couldn’t stop there. The Dockerfile worked perfectly, the image deployed, and Apache ran inside its own containerized environment. But there was still one essential piece missing:

🚀 Automation ✨

So I built it.

The Project (Recap + Link)

In my previous article, I walked through setting up a full Dockerized Apache server workflow—building a Docker image, pushing it to Docker Hub, and deploying it in a custom Docker network.

You can check out that full breakdown here:

👉 End-to-End Docker Apache Server: Build, Push, and Networked Deploy

Why Python for Automation?

Docker CLI is great, but typing in the same sequence of commands repeatedly isn’t efficient—especially when you’re building and deploying containers frequently.

Using Python (and the official Docker SDK for Python) allows us to:

Automate the build → push → deploy cycle
Control deployments via CLI flags (e.g. --build)
Manage Docker networks and containers programmatically
Practice Python in a real-world DevOps workflow

Let’s dive into the updated project and see how everything fits together.

The Dockerfile 🐳

The Dockerfile sets up an Ubuntu-based container with Apache pre-installed and ready to serve:

FROM ubuntu:20.04

ENV DEBIAN_FRONTEND=noninteractive

RUN apt-get update && \
    apt-get upgrade -y && \
    apt-get install -y apache2 && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

EXPOSE 80

CMD ["apachectl", "-D", "FOREGROUND"]

This file is responsible for building a clean, self-contained environment that runs Apache in the foreground—perfect for containerized deployment.

The deploy.py Script 🐍

This is the automation magic.

The Python script does the following:

✅ Builds the Docker image locally (if the --build flag is passed)
✅ Tags and pushes the image to Docker Hub
✅ Pulls the image (even if already local, for consistency)
✅ Creates a custom Docker network (if it doesn't exist)
✅ Deploys the container inside that network
✅ Automatically removes any conflicting containers

Here’s a snippet of the logic behind the --build flag:

if args.build:
    print(f"Building image '{image_name}'...")
    image, logs = client.images.build(path=".", tag=image_name)
    # Push to DockerHub
    dockerhub_tag = f"{dockerhub_username}/{image_name}"
    image.tag(dockerhub_tag)
    client.images.push(dockerhub_tag)
    print(f"Image pushed to {dockerhub_tag}")
else:
    # Pull from DockerHub
    client.images.pull(dockerhub_tag)

The full script handles everything from authentication to error catching and cleanup. You can test and run the automation with:

python3 deploy.py --build
Or just:

python3 deploy.py
To skip the build and pull the latest image from DockerHub.

Obstacles + Solutions 🧱

Docker SDK permissions – Needed to make sure Docker was properly installed and the Python SDK had access. Solved by ensuring Docker Desktop was running and permissions were configured.

Image caching – Docker caches builds, so when testing changes, the image might not reflect updates. Resolved by using the --build flag to force fresh builds.

Custom network errors – If a network already existed, the script would fail. Handled by checking for the network and reusing it if needed.

Local environment confusion – Mixing Docker CLI and Python SDK sometimes caused conflicts. Staying consistent with the SDK helped.

Final Thoughts 💭

This updated project now covers the full lifecycle of a Docker container—from image build to deployment—with no manual CLI steps required.

If you're learning Docker, automating your workflow with Python is a great way to cement your understanding while leveling up your DevOps toolkit.

You can check out the updated GitHub repo here:
👉 GitHub: judeWakim/apache-server-on-docker

Let me know if you try it or fork it—I'd love to see your improvements.

Happy coding! 🐳🚀

📬 Follow My Work

Want more DevOps and cloud projects?

📝 Medium

💼 LinkedIn

🐙 GitHub

💻 Dev.to

Like and comment if you liked this one — more hands-on AWS, Docker, and automation content coming soon. 🛠️💡

🚀 End-to-End Docker: Apache Server Build, Push, and Networked Deploy

Jude Wakim — Tue, 13 May 2025 21:06:41 +0000

So I finally sat down and got hands-on with Docker — and not just the hello world kind of hands-on. I mean building, customizing, pushing, and deploying an Apache web server inside a Docker container, running it on EC2, and even setting up a custom Docker network to simulate more complex environments.

It started off simple. Then got frustrating. Then got cool. Then very frustrating. Then… actually awesome.

If you’re ready to go from “I think I understand Docker…” to “I just built a full web server from scratch using Docker”, then this one’s for you.

Let’s dive in.

🐳 What Is Docker and Why Does It Matter?

Docker is a tool that allows developers to package applications and all their dependencies into lightweight, portable containers.

Why should you care? Because:

It works everywhere (local, staging, prod).
You avoid the classic “works on my machine” headache.
You can version, share, deploy, and replicate environments in seconds.
It’s the industry standard for modern DevOps and cloud-native apps.

Whether you're developing microservices, testing distributed systems, or just trying to keep your local machine clean, Docker is the go-to tool.

🧰 Prerequisites + What You'll Learn

✅ Prerequisites:

Basic Linux CLI experience
EC2 instance running Amazon Linux 2
Docker installed (we’ll cover that too)
Public-facing IP and security group rules set

💡 What You'll Learn:

How to install and use Docker on EC2
How to create and run a containerized Apache web server
How to commit + push your image to Docker Hub
How to create and attach Docker networks
How to troubleshoot common Docker issues

🔧 Foundational: Docker + Apache on EC2

First, we get Docker installed and running on our EC2 instance.

# 1. Update the package index
sudo yum update -y

# 2. Install Docker
sudo amazon-linux-extras install docker -y

# 3. Start the Docker service
sudo service docker start

Now let’s launch a base Ubuntu container and install Apache inside it:

# Run a Docker Ubuntu container (detached on port 80)
sudo docker run -dit -p 80:80 --name ubuntu-web ubuntu

# Go into the container
sudo docker exec -it ubuntu-web bash

# Update everything inside the container
apt-get update && apt-get upgrade -y

# Install Apache
apt-get install apache2 -y

# Start Apache
service apache2 start

Now check your EC2 public IP in your browser. If port 80 is open in your EC2 security group, you should see Apache’s default page! 🎉

🚀 Advanced: Commit, Push, and Pull Like a Pro

Let’s save this container as a reusable image, push it to Docker Hub, and redeploy it from scratch.

# Exit the container
exit

# Commit the container as an image
sudo docker commit ubuntu-web vvakim/ubuntu-webserver:v1

Before we push it, make sure Docker Hub is properly connected:


# Log in to Docker
docker login

⚠️ Troubleshooting: I had some wild issues here. Even though I got “Login Succeeded,” I wasn’t actually logged in — no username showed in docker info. Here's the fix:


# Add your user to the Docker group
sudo usermod -aG docker $USER
newgrp docker

# Confirm login
docker info | grep Username

Now push the image:


docker push vvakim/ubuntu-webserver:v1

Test it by removing the old container and pulling from Docker Hub:


docker stop ubuntu-web && docker rm ubuntu-web

# Run the pulled image
docker run -dit -p 8080:80 --name ubuntu-web-alt vvakim/ubuntu-webserver:v1

# Start Apache inside the container
docker exec -it ubuntu-web-alt bash
apachectl start

Check your EC2 public IP on port 8080. Apache is back up, this time from your own Docker image!

🧠 Complex: Networking & Multi-Container Readiness

Let’s simulate a more real-world use case — multiple containers on a shared Docker network.


# Create a custom Docker network
docker network create custom-net

# Run your container using the new network and a different port
docker run -dit --name ubuntu-web-net --network custom-net -p 8081:80 vvakim/ubuntu-webserver:v1

# Start Apache inside the new container
docker exec -it ubuntu-web-net bash
apachectl start

Now check the EC2 public IP on port 8081. You’ve got a second instance running side-by-side!

You can stop and remove the earlier container on 8080 if you want:


docker stop ubuntu-web-alt
docker rm ubuntu-web-alt

🧱 Roadblocks & Fixes

Let’s be real — it wasn’t smooth sailing the whole time. Here are a few issues I hit:

Apache “site not reached” error — Apache was installed but not running. Minimal Ubuntu images don’t auto-start services, so I used service apache2 start manually.
Docker login issues — Even with “Login Succeeded,” I wasn’t really logged in. Fixed it by adding my user to the Docker group and logging in again.
EC2 security group pain — Had to manually allow inbound TCP on ports 80, 8080, and 8081 for browser access.
Systemctl not working in container — systemctl isn’t supported in minimal Docker images. Stick to service or apachectl.

🎯 Conclusion

So what did we build?

An Apache web server inside Docker ✅
A versioned Docker image pushed to Docker Hub ✅
Networked containers running simultaneously ✅

This project helped me internalize how containers work, how Docker networks operate, and how real-world services get packaged and deployed. If you’re learning DevOps, Docker isn’t optional — it’s foundational.

📬 Follow My Work

Want more DevOps and cloud projects?

📝 Medium

💼 LinkedIn

🐙 GitHub

💻 Dev.to

Drop a follow if you liked this one — more hands-on AWS, Docker, and automation content coming soon. 🛠️💡

🎯 Auto Thumbnail Generator: Serverless Image Processing for YouTube Thumbnails

Jude Wakim — Fri, 25 Apr 2025 17:30:00 +0000

The program is a thumbnail generator available via a web browser with full follow-along

📦 Overview: What's this thing do?

This serverless web app lets you:

Upload an image (screenshot, still frame, etc.) via a simple web dashboard.
First Lambda, triggered via API Gateway using a POST request, generates a presigned POST policy and uploads the image to S3 with metadata.
Automatically trigger a second Lambda function using S3 event notifications.
Process the image using Pillow to create a properly sized thumbnail (1280x720).
Adds a text overlay using the metadata to create thumbnail text.
Store the processed image in a second S3 bucket.
Return a presigned URL to be downloaded.
Frontend polls periodically until the corresponding file is found in the processed bucket.
Second Lambda then generates and returns the presigned URL from the processed bucket.

🔧 Services: How's this thing do what it does?

Amazon S3 - store raw uploads and processed images.
AWS Lambda - perform image processing using Pillow.
Amazon API Gateway - support uploading and presigned URL generation from the web.
IAM - permission control.
Simple HTML Dashboard - static website for uploads and image preview.

Let's dive in and build this thing!

⚙️Build: How do you get the thing to do what it does?

1. Set Up Two S3 Buckets

Just create two S3 buckets. Name them:
something-uploads-related → stores original uploads
something-processed-related → stores finalized thumbnails

Then, on the something-uploads-related bucket, enable S3 event notifications to trigger the Lambda function that we are going to write next, on object creation.

2. Write Image Processing Lambda Function with Python & Pillow

The logic for the function should be as follows:

Get the image from the something-uploads-related bucket
Collect metadata about the image from the user input on the webpage
Resize it to YouTube's recommended 1280x720 resolution (or maintain aspect ratio + pad) using Pillow
Generate thumbnail text using the metadata and add text as an overlay
Save it to something-processed-related bucket

So the flow becomes something like this:
🡢The User Uploads an Image to S3: The user uploads an image to the S3 bucket via the web browser. The webpage will provide the user with the space to include thumbnail text for the overlay.
🡢The Lambda gets Triggered by Upload: Lambda processes the image file and stores it in another S3 bucket.

This is the basic code for the Lambda function:

from PIL import Image, ImageDraw, ImageFont, UnidentifiedImageError
import boto3
import os
import io

s3 = boto3.client('s3')

# Assume you added "Impact.ttf" font to your deployment package or layer
# Font path and size for text overlay is detailed in the Lambda layer
FONT_PATH = "/opt/Impact.ttf"  # If added via Lambda Layer
FONT_SIZE = 64

def lambda_handler(event, context):
    bucket = event['Records'][0]['s3']['bucket']['name']
    key    = event['Records'][0]['s3']['object']['key']

    try:
        # Get metadata (for custom text)
        head = s3.head_object(Bucket=bucket, Key=key)
        text = head.get("Metadata", {}).get("text", "Your Title Here")

        # Download image
        original_image = s3.get_object(Bucket=bucket, Key=key)['Body'].read()
        try:
            image = Image.open(io.BytesIO(original_image)).convert("RGB")
        except UnidentifiedImageError:
            return {
                'statusCode': 400,
                'body': 'The uploaded file is not a valid image.'
            }

    # Resize to 1280x720
    image.thumbnail((1280, 720))

    # Creates a black canvas that is always 1280xx720
    canvas = Image.new('RGB', (1280, 720), (0, 0, 0)) 
    # Paste the resized image on the center of the black canvas
    canvas.paste(image, ((1280 - image.width) // 2, (720 - image.height) // 2))

    # ImageDraw.Draw lets you draw shapes, lines, or text onto a Pillow image
    draw = ImageDraw.Draw(canvas)

    # Apply translucent black overlay for contrast 
    # (later will improve this logic to make it dynamic)
    overlay = Image.new('RGBA', canvas.size, (0, 0, 0, 100))
    canvas = Image.alpha_composite(canvas.convert('RGBA'), overlay)

    # Load font
    font = ImageFont.truetype(FONT_PATH, FONT_SIZE)

    # Text placement
    text_position = (50, 600)
    draw = ImageDraw.Draw(canvas)
    draw.text(text_position, text, font=font, fill='white')

    # Save final output
    output = io.BytesIO()
    canvas.convert("RGB").save(output, format='JPEG')
    output.seek(0)

    # Upload to processed bucket
    processed_key = f"processed-{key}"
    processed_bucket = 'something-processed-related'
    s3.upload_fileobj(output, processed_bucket, processed_key, ExtraArgs={'ContentType': 'image/jpeg'})

    # Generate presigned URL
    url = s3.generate_presigned_url('get_object', Params={
        'Bucket': processed_bucket,
        'Key': processed_key
    }, ExpiresIn=3600)

    return {
        'statusCode': 200,
        'body': url
    }

    except Exception as e:
        return {
            'statusCode': 500,
            'body': f'Error: {str(e)}'
        }

AWS Lambda does not have Pillow installed by default, so you should create a Lambda layer with Pillow pre-installed and attach it to your function.

Use a pre-existing layer for Pillow, like:

https://github.com/keithrozario/Klayers/tree/master/deployments/python3.12

Once the layer is created, just attach it to the function.

Then, add another Lambda layer for font styles for the thumbnail text. You can find many different fonts here:

https://fontsgeek.com/impactall-font

Once again, when the layer is created, just attach it to the function.

and…💥

3. Write the Lambda to Generate a Presigned POST URL

✅ Use Case Recap:

Browser sends the file and text to this Lambda
Lambda generates a presigned POST policy
Browser uploads directly to S3 using the POST form

This Lambda code could look something like this:

import json
import boto3
import os
from urllib.parse import parse_qs

s3 = boto3.client('s3')

UPLOAD_BUCKET = 'something-upload-related'
PROCESSED_BUCKET = 'something-process-related'

def lambda_handler(event, context):
    method = event.get('httpMethod')
    path = event.get('path')

    if method == 'POST' and path == '/generate-presigned-url':
        return generate_presigned_post(event)
    elif method == 'GET' and path == '/get-final-url':
        return get_processed_image_url(event)
    else:
        return {
            'statusCode': 404,
            'body': json.dumps({'error': 'Route not found'})
        }

def generate_presigned_post(event):
    try:
        body = json.loads(event['body'])
        filename = body.get('filename')
        text = body.get('text', '')  # Optional

        if not filename:
            return {
                'statusCode': 400,
                'body': json.dumps({'error': 'Filename is required'})
            }

        # Create a presigned POST to upload the file with metadata
        presigned_post = s3.generate_presigned_post(
            Bucket=UPLOAD_BUCKET,
            Key=filename,
            Fields={"x-amz-meta-text": text},
            Conditions=[
                {"x-amz-meta-text": text},
                ["starts-with", "$Content-Type", ""]
            ],
            ExpiresIn=300  # 5 minutes
        )

        return {
            'statusCode': 200,
            'body': json.dumps(presigned_post)
        }

    except Exception as e:
        return {
            'statusCode': 500,
            'body': json.dumps({'error': str(e)})
        }

def get_processed_image_url(event):
    try:
        params = event.get('queryStringParameters') or {}
        filename = params.get('filename')

        if not filename:
            return {
                'statusCode': 400,
                'body': json.dumps({'error': 'Missing filename'})
            }

        processed_key = f'processed-{filename}'

        # Check if the file exists
        try:
            s3.head_object(Bucket=PROCESSED_BUCKET, Key=processed_key)
        except s3.exceptions.ClientError as e:
            if e.response['Error']['Code'] == '404':
                return {
                    'statusCode': 404,
                    'body': json.dumps({'message': 'Thumbnail not ready yet'})
                }
            else:
                raise

        # File exists, generate download URL
        presigned_url = s3.generate_presigned_url('get_object', Params={
            'Bucket': PROCESSED_BUCKET,
            'Key': processed_key
        }, ExpiresIn=3600)

        return {
            'statusCode': 200,
            'body': json.dumps({'url': presigned_url})
        }

    except Exception as e:
        return {
            'statusCode': 500,
            'body': json.dumps({'error': str(e)})
        }

For the presigned POST URL Lambda to work, it needs an API Gateway hookup. Create a new HTTP API and add routes POST /generate-presigned-url and GET /get-final-url integrate it with the Lambda.

Then test it using the API endpoint and the command line using thecURL command.

4. Create the Web Dashboard

The end user should have a clean, easy dashboard to upload images and receive thumbnails. So we will create a static webpage using S3 that allows the user to:

Upload an image file with text for the thumbnail
🡢Make sure it is an acceptable file type.
🡢The text is passed as metadata.
Receive a thumbnail of the image for download
🡢A link is provided to view and download the image.
🡢The image is stored in an S3 bucket (with a lifecycle policy).

How?

Submit button triggers the Lambda using an API endpoint to generate a presigned URL to upload to S3
🡢API Gateway passes along the image file and thumbnail text stored as metadata
JavaScript on the webpage uploads an image file to the S3 bucket with the presigned URL
🡢Uploads to the S3 bucket trigger the Lambda function to process the image and store it in the processed images bucket
Submit button also triggers another Lambda using an API endpoint to poll the S3 bucket for processed images for the processed image, creates a presigned URL and returns it
🡢Same submit button (with a short delay) triggers another Lambda to search the processed images S3 bucket for an object with the proper filename
🡢Poll repeats periodically and when found, generates and returns a presigned URL back to the user for viewing or downloading

To do so, create another S3 bucket and enable static website hosting, set a bucket policy, and unblock all public access (we can go back and make this more secure later). Then upload an index.html file to the bucket. The JavaScript in my file looked something like this:

<script>
    document.getElementById('upload-form').addEventListener('submit', async (e) => {
      e.preventDefault();

      const text = document.getElementById('text').value;
      const filetype = document.getElementById('filetype').value;
      const fileInput = document.getElementById('image');
      const file = fileInput.files[0];

      if (!file) {
        alert("Please choose a file.");
        return;
      }

      // Step 1: Request a presigned URL
      const presignedRes = await fetch("[API-ENDPOINT]", {
        method: "POST",
        headers: {
          "Content-Type": "application/json"
        },
        body: JSON.stringify({
          text: text,
          filetype: filetype
        })
      });

      const { url, filename } = await presignedRes.json();

      // Step 2: Upload file to S3 using presigned URL
      const uploadRes = await fetch(url, {
        method: "PUT",
        headers: {
          "Content-Type": file.type
        },
        body: file
      });

      if (uploadRes.ok) {
        document.getElementById("result").innerHTML = `
          <strong>✅ Uploaded successfully!</strong><br>
          Processed thumbnail will appear shortly:<br>
          <code>${filename}</code>
        `;

        setTimeout(() => {
            checkForThumbnail(filename);
        }, 3000);
      } else {
        document.getElementById("result").innerHTML = "❌ Upload failed.";
      }
    });
    async function checkForThumbnail(filename, retries = 5) {
    const pollRes = await fetch(`[API-ENDPOINT]`);

    if (pollRes.ok) {
      const { downloadUrl } = await pollRes.json();
      document.getElementById("result").innerHTML += `
        <br><strong>✅ Thumbnail Ready!</strong><br>
        <a href="${downloadUrl}" target="_blank" download>Click here to view</a><br>

      `;
    } else if (retries > 0) {
      setTimeout(() => {
        checkForThumbnail(filename, retries - 1);
      }, 3000); // Retry after 3 seconds
    } else {
      document.getElementById("result").innerHTML += `<br>⏳ Thumbnail still processing. Try again shortly.`;
    }
  }
  </script>

5. Create the Third Lambda for Polling

At this point in the project, the webpage is up and communicating with the presigned URL generator Lambda. When you hit the "generate thumbnail" button, the API endpoint passes the image file and thumbnail text as metadata to the URL generation Lambda, which creates the presigned URL and returns it to the webpage. Then the webpage uploads the image file to the uploads bucket using the URL. When the image file gets uploaded to the bucket, the event notification triggers the thumbnail generator Lambda, which creates the thumbnail and stores it in the processed images bucket.

Now, what we have to set up here is how we are going to get the processed image back to the user. We are going to do this with another Lambda function for polling the processed images bucket for the new file and returning it back to the user. So, we will put another API endpoint on the "generate thumbnail" button, which waits a couple of seconds, then triggers the polling Lambda. The code for this Lambda is below.

import json
import boto3
import os

s3 = boto3.client('s3')

PROCESSED_BUCKET = 'something-processed-related' 

def lambda_handler(event, context):
    try:
        # If REST API: filename is in queryStringParameters
        # If HTTP API: filename may be in event['queryStringParameters']
        params = event.get("queryStringParameters") or event.get("querystring") or {}
        filename = params.get("filename")
        if not filename:
            return {
                'statusCode': 400,
                'body': json.dumps("Missing filename parameter.")
            }

        processed_key = f"processed-{filename}"

        # Try to get object metadata to confirm it exists
        s3.head_object(Bucket=PROCESSED_BUCKET, Key=processed_key)

        # If it exists, generate presigned URL
        url = s3.generate_presigned_url('get_object', Params={
            'Bucket': PROCESSED_BUCKET,
            'Key': processed_key
        }, ExpiresIn=3600)

        return {
            'statusCode': 200,
            'headers': {'Access-Control-Allow-Origin': '*'},  # CORS
            'body': json.dumps({'downloadUrl': url})
        }

    except s3.exceptions.ClientError as e:
        if e.response['Error']['Code'] == '404':
            return {
                'statusCode': 404,
                'headers': {'Access-Control-Allow-Origin': '*'},  # CORS
                'body': json.dumps("Thumbnail not ready yet.")
            }
        return {
            'statusCode': 500,
            'headers': {'Access-Control-Allow-Origin': '*'},  # CORS
            'body': json.dumps("Error checking file.")
        }

6. Security & IAM

Lambda needs s3:GetObject, s3:PutObject for both buckets, but not more.
Restrict the bucket policy and bucket access.
If using presigned POST or API Gateway, you'll control access via CORS but generating a CORS config template.

Issues: Why isn't this thing doing what it does ❓

1.🛡️CORS

Problem:
My frontend was politely trying to talk to my Lambda endpoints… but CORS was like "who are you again?"
Symptom:
Chrome Dev Tools screamed:
Access to fetch at '...' from origin '...' has been blocked by CORS policy.
Solution:
I added the right headers in every Lambda response - especially for 4XX and 5XX errors, which are often forgotten:

'headers': {
  'Access-Control-Allow-Origin': '*'
}

2.🔁Thumbnail "Still Processing" Forever

Problem:
Even after uploading, the frontend would just loop "Thumbnail still processing. Try again shortly."
Cause:
The Lambda that checks for the processed thumbnail was using head_object() to look for the file-but my IAM role didn't have s3:GetObject permission, which is sneakily required to do a HeadObject!
Fix:
I added this permission to the role:

{
  "Effect": "Allow",
  "Action": "s3:GetObject",
  "Resource": "arn:aws:s3:::your-processed-bucket-name/*"
}

3. ✔️❌Good URL, Bad Upload

Problem:
I'd get a valid presigned URL, but uploading the file would silently fail or return a confusing 403.
Diagnosis:
The Content-Type of the file must match the one used when generating the URL. A mismatch breaks the signature.
*Solution:
* I explicitly set the Content-Type during both the presigned URL generation in Lambda and the fetch() upload call on the frontend.

V2 Improvements: How to do what it does…better

Improve Logic - Thumbnails generally have text on top of the images and can be added in many different ways, depending on the image itself. Make the text overlay dynamic depending on the colors and contrast of the image itself.
Infrastructure as Code (IaC) - Make this whole project into one file in Terraform. Automation and easy source control.

🧪 Want to Try It Yourself?

You can view the full repo and clone it for your own AWS account here:

https://github.com/Judewakim/thumbnailgenerator

For more projects like this, check out my other Medium posts and check out my Github and LinkedIn.

🚀 How I Hosted a Static Website on Azure (in Under 15 Minutes)

Jude Wakim — Tue, 15 Apr 2025 00:31:04 +0000

Thinking of using Azure to host a simple static site? You’re in the right place.

In this short post, I’ll walk you through how I used Azure Blob Storage to host a static website in minutes — without overcomplicating it. No VMs, no servers, no chaos. Just clean cloud goodness.

☁️ First, Why Azure?

Azure is Microsoft’s cloud platform — same league as AWS and Google Cloud. What makes Azure nice?

Integrated with everything Microsoft
Great UI for beginners
You get $200 free with a trial account (aka: zero dollars to deploy your site)
And with Azure Blob Storage, you can turn a simple storage bucket into a static website with public access — perfect for portfolios, documentation, or proof-of-concept apps.

🛠️ Step-by-Step: Host a Static Website on Azure Blob Storage

Let’s get into it. No fluff — just what I did.

1. Create a Storage Account

Head to Azure Portal at portal.azure.com and sign in or up.
Create a new Storage Account.
Choose a region, name it something unique like 'myawesomestorage123'.

2. Create a Blob Container

Inside your new Storage Account, go to Containers.
Create a new container, name it web or website or something simple.
Set Public access level to Container (anonymous read access).

3. Upload Your Site

Upload your index.html file to the container.
If you’re fancy, add a error.html too.

4. Enable Static Website Hosting

Still in your storage account, find Static website under settings.
Turn it on, set index.html as the default.
Azure will give you a public endpoint like: 'https://.z13.web.core.windows.net'

5. Test It

Open that URL in incognito mode.
Boom. Site live. Cloud flex unlocked. 💪

⚠️ Common Issues & Fixes

❌ Blob Container Access Level is Greyed Out
When you try to set your Blob container to Container (anonymous read access) and it’s greyed out, don’t panic.

✅ Fix It:

Go to your Storage Account settings.
Find Configuration.
Flip the switch for Allow blob anonymous access to Enabled.
Save it.
Now go back and set the public access level — good to go.

❌ CDN Not Registered
I tried to add Azure Front Door to boost performance and HTTPS support. But if you’re on a Free Trial, you’ll probably hit this:

> Error: Microsoft.Cdn is not registered for the subscription

✅ How to Fix It (If You Have Access)
If you’re not on a Free Trial, here’s the fix using the Azure CLI:

az provider register --namespace Microsoft.Cdn

Done! Now, you can use Azure Front Door or Azure CDN. But if you’re on a trial account, just know it won’t work until you upgrade. Not a dealbreaker for simple sites.

🧹 Clean-Up: Don’t Forget to Shut It Down

If you’re done with your project and want to avoid unexpected charges (or just like a clean dashboard), here’s how to clean up:

Delete the Blob Container
Delete the Storage Account
Double-check Resource Group usage if you created one just for this

That’s it — you’re done.

🏁 Final Thoughts

Hosting a static site on Azure is easy, fast, and free (if you’re on a trial). You don’t need VMs or complicated infra to get a portfolio or simple app online.

And if you want more performance or HTTPS, Azure Front Door is there when you’re ready (and off the free trial 😅).

Judewakim (Jude Wakim) / Repositories · GitHub

Hi, I’m a DevOps Cloud Engineer. I use AWS ☁️, Python 🐍, Terraform🏗️, and Linux 🐧 to build things for fun. - Judewakim

github.com

Want to see more quick cloud wins like this? Follow me — I build fast, clean, cloud-native solutions for real-world projects. Check out my other Medium posts and check out my Github and LinkedIn.

Catch you in the cloud. ☁️

STOP USING THE AWS CONSOLE…use CloudFormation (IaC) instead

Jude Wakim — Fri, 21 Mar 2025 17:30:00 +0000

Scenario

Here’s a real-world scenario…imagine “Aurora Digital, an expanding online retailer specializing in luxury home goods, has decided to transition its digital operations to AWS to leverage cloud computing’s benefits. The company is experiencing significant growth and needs a solution that can handle increasing traffic and scale during major sales events without sacrificing performance or security.”

Now, your goal is to “help Aurora Digital implement a cloud-based infrastructure using AWS services, specifically focusing on scalability, security, cost-efficiency, and automation.” -Level Up In Tech, CDA03-CloudFormation

Objectives

Enhance Scalability: Automatically adjust computing resources to meet demand, ensuring the platform remains operational and responsive during peak traffic periods.

Increase Reliability: Improve website uptime and reduce the risk of failures associated with physical hardware and manual interventions.
Boost Security: Implement advanced security protocols and isolation through AWS to protect sensitive customer data and transactions.

Reduce Operational Costs: Lower overall infrastructure costs by optimizing resource usage and eliminating the need for upfront hardware investments.

Why use CloudFormation (IaC)

Automation: Reduce human error and free up developer time to focus on other tasks by automating the provisioning and management of a complex cloud environment.

Consistency: Ensure consistent environments are created every time, thereby eliminating the “it works on my machine” problem.
Version Control: Enhance collaboration among team members by allowing infrastructure to be version-controlled and reviewed as part of the application code.

Reusability: Reuse templates across the company or community, speeding up future deployments and ensuring best practices are followed.

Prerequisites

Be familiar with IaC
Be familiar with YAML
Be familiar with the AWS CLI
Have a text editor (I use VSC)

Create the Solution Architecture

So the first step when receiving an objective is designing the solution. Before you start building, you have to know what you are trying to build. To do that, it’s always best practice to create a diagram. I use Draw.io.

For this objective, we need to create a web server for Aurora Digital to host their website. We also need this web server to be automatically scalable in the event of spikes, secure to protect the company’s assets, and resilient in the event of disaster.

This architecture will have multiple EC2 instances, running Apache web server. It will have an auto scaling group to automatically scale the number of instances required to handle the traffic. This will make it scalable. The instances will be spread across three public subnets in three availability zones all within a VPC, making it resilient to any disasters in the area. We will use a load balancer to, guess what… balance the load, of traffic between the instances. This will help with the resilience because if one instance in one availability zone goes down, the auto scaling group will create another instance in another availability zone and if one instance is not performing well or getting overwhelmed with traffic, the load balancer will re-distribute the load to make sure we optimize the utilization of all the instances. Also, we will use security groups to restrict access to the web server only from the load balancer. There will be two security groups (the security group for the load balancer is not displayed in the diagram), one for the webservers that restricts inbound traffic to only allow HTTP traffic from the load balancer and another security group for the load balancer that allows HTTP traffic from anywhere. Doing this makes sure all traffic is routed through the same path, making it easier to control and secure.

Create the YAML Script

Now, that you have the diagram and you know what you are building, build it. Create the script using YAML or JSON ( I will use YAML).

Below is my complete YAML script that I created in Visual Studio Code (VSC).

#cda03-cloudformation 

AWSTemplateFormatVersion: "2010-09-09"
Description: The CloudFormation template in YAML for CDA03-CloudFormation use case.

Resources:
  #vpc
  CDA03VPC:
    Type: AWS::EC2::VPC
    Properties:  
      CidrBlock: 10.10.0.0/16
      EnableDnsSupport: true
      EnableDnsHostnames: true
      Tags:
        - Key: Name
          Value: CDA03VPC

  #subnet 1
  CDA03PublicSubnet1:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref CDA03VPC
      CidrBlock: 10.10.1.0/24
      MapPublicIpOnLaunch: true
      AvailabilityZone: us-east-1a
      Tags:
          - Key: Name
            Value: CDA03PublicSubnet1

  #subnet 2
  CDA03PublicSubnet2:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref CDA03VPC
      CidrBlock: 10.10.2.0/24
      MapPublicIpOnLaunch: true
      AvailabilityZone: us-east-1b
      Tags:
        - Key: Name
          Value: CDA03PublicSubnet2

  #subnet 3
  CDA03PublicSubnet3:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref CDA03VPC
      CidrBlock: 10.10.3.0/24
      MapPublicIpOnLaunch: true
      AvailabilityZone: us-east-1c
      Tags:
        - Key: Name
          Value: CDA03PublicSubnet3

  #igw
  CDA03InternetGateway:
    Type: AWS::EC2::InternetGateway
    Properties: 
      Tags: 
        - Key: Name
          Value: CDA03InternetGateway

  #igw attachment
  AttachGateway:
    Type: AWS::EC2::VPCGatewayAttachment
    Properties:
      VpcId: !Ref CDA03VPC
      InternetGatewayId: !Ref CDA03InternetGateway

  #route table
  CDA03RouteTable:
    Type: AWS::EC2::RouteTable
    Properties: 
      VpcId: !Ref CDA03VPC
      Tags: 
        - Key: Name
          Value: CDA03RouteTable

  #route to internet
  PublicRoute:
    Type: AWS::EC2::Route
    Properties: 
      RouteTableId: !Ref CDA03RouteTable
      DestinationCidrBlock: 0.0.0.0/0
      GatewayId: !Ref CDA03InternetGateway


  # Subnet Route Table Associations
  Subnet1RouteTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref CDA03PublicSubnet1
      RouteTableId: !Ref CDA03RouteTable

  Subnet2RouteTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref CDA03PublicSubnet2
      RouteTableId: !Ref CDA03RouteTable

  Subnet3RouteTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref CDA03PublicSubnet3
      RouteTableId: !Ref CDA03RouteTable

  #key pair
  CDA03KeyPair:
    Type: AWS::EC2::KeyPair
    Properties:  
      KeyName: cda03-keypair
      KeyType: rsa

  #launch template
  CDA03LaunchTemplate: 
    Type: AWS::EC2::LaunchTemplate
    Properties: 
      LaunchTemplateData:
        InstanceType: t2.micro
        KeyName: cda03-keypair
        ImageId: ami-0c02fb55956c7d316 # Amazon Linux 2 AMI ID (specific to region)
        SecurityGroupIds:
          - !Ref CDA03WebserverSecurityGroup
        UserData:
          Fn::Base64: |
            #!/bin/bash
            sudo yum update && sudo yum upgrade
            sudo yum install httpd -y
            sudo systemctl start httpd
            sudo systemctl enable httpd
            cd /usr/share/httpd/noindex/
            chown apache:apache /usr/share/httpd/noindex/index.html
            chmod 644 /usr/share/httpd/noindex/index.html
            echo "hello world, from $HOSTNAME" | sudo tee /usr/share/httpd/noindex/index.html > /dev/null
        TagSpecifications:
          - ResourceType: instance
            Tags:
              - Key: Name
                Value: CDA03-ApacheInstance
      LaunchTemplateName: CDA03LaunchTemplate

  #webserver sg
  CDA03WebserverSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Allow HTTP access only from the ALB
      VpcId: !Ref CDA03VPC
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 80
          ToPort: 80
          SourceSecurityGroupId: !Ref CDA03ALBSecurityGroup
      Tags:
        - Key: Name
          Value: CDA03WebserverSecurityGroup

  #asg
  CDA03ASG: 
    Type: AWS::AutoScaling::AutoScalingGroup
    Properties:
      LaunchTemplate:
        LaunchTemplateId: !Ref CDA03LaunchTemplate
        Version: !GetAtt  CDA03LaunchTemplate.LatestVersionNumber
      MinSize: 2
      DesiredCapacity: 2
      MaxSize: 5
      VPCZoneIdentifier:
        - !Ref CDA03PublicSubnet1
        - !Ref CDA03PublicSubnet2
        - !Ref CDA03PublicSubnet3
      TargetGroupARNs:
        - !Ref CDA03TargetGroup
      Tags: 
        - Key: Name
          Value: CDA03ASG
          PropagateAtLaunch: true

  # Target Group
  CDA03TargetGroup:
    Type: AWS::ElasticLoadBalancingV2::TargetGroup
    Properties:
      Name: CDA03TargetGroup
      Protocol: HTTP
      Port: 80
      VpcId: !Ref CDA03VPC
      TargetType: instance
      HealthCheckEnabled: true
      HealthCheckPath: /
      HealthCheckIntervalSeconds: 30
      HealthCheckTimeoutSeconds: 5
      HealthyThresholdCount: 3
      UnhealthyThresholdCount: 2
      Matcher:
        HttpCode: 200

  #alb
  CDA03LoadBalancer:
    Type: AWS::ElasticLoadBalancingV2::LoadBalancer
    Properties: 
      Name: CDA03ApplicationLoadBalancer
      Scheme: internet-facing
      Subnets:
        - !Ref CDA03PublicSubnet1
        - !Ref CDA03PublicSubnet2
        - !Ref CDA03PublicSubnet3
      SecurityGroups: 
        - !Ref CDA03ALBSecurityGroup
      Tags:
        - Key: Name
          Value: CDA03LoadBalancer

  #alb sg
  CDA03ALBSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties: 
      GroupDescription: Allow HTTP traffic from anywhere to ALB
      VpcId: !Ref CDA03VPC
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 80
          ToPort: 80
          CidrIp: 0.0.0.0/0
      Tags:
        - Key: Name
          Value: CDA03ALBSecurityGroup

  #alb listener
  CDA03ALBListener:
    Type: AWS::ElasticLoadBalancingV2::Listener
    Properties: 
      DefaultActions: 
        - Type: forward
          TargetGroupArn: !Ref CDA03TargetGroup
      LoadBalancerArn: !Ref CDA03LoadBalancer
      Port: 80
      Protocol: HTTP

#outputs (dns url)
Outputs:
  ALBDNSName:
    Description: "DNS name of the Application Load Balancer"
    Value: !GetAtt CDA03LoadBalancer.DNSName
    Export:
      Name: "ALBDNSName"

Troubleshoot

Once I created the script, I used the AWS CLI to validate the code. In the terminal within VSC, I ran this command, aws cloudformation validate-template --template-body file://cda03-cloudformation-code.yml to see if the code was all correct and ready to be deployed.

This was my response:

“An error occurred (ValidationError) when calling the ValidateTemplate operation: Invalid template resource property ‘ALBDNSName’”

There was an error with the output. I wanted the CloudFormation stack to output the DNS name of the load balancer so I don’t have to look around for it. After a little troubleshooting, I realized I had the outputs block within the resource block. The outputs block is not a resource and is supposed to be on par with the resource block. So after a minor adjustment, I saved the changes to the file and ran the validation again.

Success

Now that the code is validated, and confirmed working. We can create the CloudFormation stack. You will need to run this command aws cloudformation create-stack --stack-name CDA03Stack --template-body file://cda03-cloudformation-code.yml and kick back and relax while CloudFormation creates everything. Wait a couple of minutes for all the services to finish provisioning.

If you try to query the stack for the load balancer DNS name before it is ready you will see a result of “none”, which just means it’s not done provisioning, so wait and try again.

Go to that address to confirm the web server is up and running. Make sure you use HTTP not HTTPS, because that is what we specified in the YAML code. Refresh the page a couple of times to verify that the load balancer is balancing between different instances.

Lastly, let’s confirm that you can only access the web server via the load balancer and not via the instance’s IP addresses directly. To find the instance’s IP addresses from the AWS CLI, use this command aws ec2 describe-instances --region us-east-1 to list all the instances in the us-east-1 region. The public IP address should be listed there, copy it and paste into a browser (using HTTP) to confirm that you cannot access the web server.

Congratulations!
If you followed along and reached this point, you should now see why IaC is better to use than manually creating everything in the console. You created a web server that is scalable, resilient, and secure, and you now can automate its creation process.

To delete the stack from the AWS CLI, use this command aws cloudformation delete-stack --stack-name CDA03Stack. You should not get any response from the terminal but wait a couple minutes until the entire stack gets de-provisioned.

Originally published on Medium

Find me in Linkedin

For similar projects, check out my Github

Master Python Automation: Extract and Display File Info Like a Pro

Jude Wakim — Fri, 21 Mar 2025 17:03:00 +0000

Scenario

“Your company needs to learn about the files located on various machines. You have been asked to build a script that extracts information such as the name and size about the files in the current working directory and stores it in a list of dictionaries.” -LUIT, Python2

Automation scripting is a game-changer and the LUIT-Python2 GitHub repository provides a robust solution. This repository offers a Python-based tool that simplifies tasks like network provisioning, application deployment, and system configuration.

Why Automate Infrastructure Management?

Efficiency: By automating repetitive tasks, you save time and reduce the risk of human error.

Consistency: Each infrastructure deployment is identical, ensuring that “it works on my machine” never becomes an issue.

Scalability: Automated scripts can scale effortlessly, provisioning multiple environments with just a few commands.

Maintainability: Once an automated script is written, it can be reused, updated, and version-controlled.

Getting Started

Before diving into the repository, make sure you have Python 2.7 (or compatible) installed on your machine. You will also need a basic understanding of Linux-based systems, as many of the scripts are tailored for that environment.

Clone the Repository: Start by cloning the LUIT-Python2 repository from GitHub:
git clone https://github.com/Judewakim/LUIT-Python2.git

Install Dependencies: The repository uses a set of Python packages to handle various tasks. You can install them using pip:
cd LUIT-Python2 pip install -r requirements.txt

Writing Python

To break down the essential components of the data_extraction.py file, lets start with the imports. This code requires to import information about the operating system and the datetime.

import os
import time

Next, we begin defining the first function of the code. This function will collect all the information that we will need. Remember, the goal of this code is to “build a script that extracts information such as the name and size about the files in the current working directory” so we will collect all the information about the working directory (or whatever directory is specified) and later we will present this information similar to the ls -al command in the Linux command line. That function will look like this:

def get_files_info(path='.'):  # Function that collects file details, defaulting to current directory
    files_info = []  # List to store file details

    for root, _, files in os.walk(path):  # Recursively traverse directories and files
        for filename in files:
            file_path = os.path.join(root, filename)  # Construct the full file path
            file_stat = os.stat(file_path)  # Get file details/statistics

            files_info.append({  # Store file details in a dictionary
                'name': filename,  # File name
                'path': file_path,  # Full file path
                'size_bytes': file_stat.st_size,  # File size in bytes
                'last_modified': time.ctime(file_stat.st_mtime),  # Last modified time
                'permissions': oct(file_stat.st_mode)[-3:],  # File permissions in octal format (last 3 digits)
            })

    return files_info  # Return the list of file details

Now, we have a function that collects the file name, file path, file size, last modification time, and file permissions of each file in the path. The next thing to do is create another function that will display all this collected data in the way we want it. That second function will look like this:

def print_ll_view(files_info):  # Function to print file details in 'll' view format
    for file in files_info:  # Loop through the list of file dictionaries
        print(f"{file['permissions']} {file['size_bytes']} {file['last_modified']} {file['path']}")  # Print file details

Lastly, we will call this functions and add a bit of user interaction to make the program more streamlined for the user. That last bit of code is set to only run when the Python is run directly and cannot be called from another file. This is what that looks like:

if __name__ == "__main__":  # Run the script only if executed directly
    path = input("Enter the directory path (press Enter to use current directory): ") or "."  # Prompt user for path, default to current directory
    files_data = get_files_info(path)  # Get file details for the specified path
    print("\nLinux CLI 'll' View:")  # Print header
    print_ll_view(files_data)  # Display file details

Running the Program

The program is created and ready to be used. Navigate to the location where the program is stored. If you cloned it from my Github repository the location should be LUIT-Python2 . Once there, you can run the program use the command python .\data_extraction.py.

At this point, you have a Python program that will display the files from whatever location you specify and will display those files in Linux ls format for easy readability. This program can be repeated with whatever file path you need.

You can modify this program to fit your company’s needs by cloning it locally or forking it on Github.

This entire project is available on GitHub

Originally published on Medium

Find me on Linkedin

DEV Community: Jude Wakim

Building My S3 Security Scanner: A Solo Dev's Journey to Automate AWS Bucket Safety

Introduction: Why I Built This (And What You’re Getting Into)

The Build Process: From Idea to Running Code

Judewakim / s3-misconfig

S3 Sentry: The Autonomous Data Security Officer

The Mission

Core Architecture

Obstacles: The Bumps That Made It Better

Wrapping Up: What’s Next?

How to Use Docker Compose and Python to Automate Your Jenkins Environment

🐳 What is Docker and Why It’s a Game-Changer

📦 Enter Docker Compose

⚙️ Why Automation Matters

🛠️ Step-by-Step: Building the Project

1. Wrote the docker-compose.yml

2. Created a Dockerfile

3. Wrote the Automation Script (automate.py)

🧱 Common Roadblocks I Hit

❌ mypy Not Recognized in PowerShell

🐍 ModuleNotFoundError: No module named 'docker'

✅ Final Thoughts

🧭 Follow My Cloud Journey

Automating My Docker Apache Server with Python

The Project (Recap + Link)

Why Python for Automation?

The Dockerfile 🐳

The deploy.py Script 🐍

Obstacles + Solutions 🧱

Final Thoughts 💭

📬 Follow My Work

🚀 End-to-End Docker: Apache Server Build, Push, and Networked Deploy

🐳 What Is Docker and Why Does It Matter?

🧰 Prerequisites + What You'll Learn

✅ Prerequisites:

💡 What You'll Learn:

🔧 Foundational: Docker + Apache on EC2

🚀 Advanced: Commit, Push, and Pull Like a Pro

🧠 Complex: Networking & Multi-Container Readiness

🧱 Roadblocks & Fixes

🎯 Conclusion

📬 Follow My Work

🎯 Auto Thumbnail Generator: Serverless Image Processing for YouTube Thumbnails

📦 Overview: What's this thing do?

🔧 Services: How's this thing do what it does?

⚙️Build: How do you get the thing to do what it does?

1. Set Up Two S3 Buckets

2. Write Image Processing Lambda Function with Python & Pillow

3. Write the Lambda to Generate a Presigned POST URL

4. Create the Web Dashboard

5. Create the Third Lambda for Polling

6. Security & IAM

Issues: Why isn't this thing doing what it does ❓

1.🛡️CORS

2.🔁Thumbnail "Still Processing" Forever

3. ✔️❌Good URL, Bad Upload

V2 Improvements: How to do what it does…better

🧪 Want to Try It Yourself?

🚀 How I Hosted a Static Website on Azure (in Under 15 Minutes)

☁️ First, Why Azure?

🛠️ Step-by-Step: Host a Static Website on Azure Blob Storage

⚠️ Common Issues & Fixes

🧹 Clean-Up: Don’t Forget to Shut It Down

🏁 Final Thoughts

Judewakim (Jude Wakim) / Repositories · GitHub

STOP USING THE AWS CONSOLE…use CloudFormation (IaC) instead

Scenario

Objectives

Why use CloudFormation (IaC)

Prerequisites

Create the Solution Architecture

Create the YAML Script

Troubleshoot

Success

Master Python Automation: Extract and Display File Info Like a Pro

Scenario

Why Automate Infrastructure Management?

Getting Started

Writing Python

Running the Program

1. Wrote the `docker-compose.yml`