DEV Community: Reet

Get started with deploying web apps on AWS

Reet — Tue, 19 Aug 2025 09:27:57 +0000

A serverless To-Do Web Application on AWS

This document outlines the steps to build and deploy a simple, serverless to-do application. The project uses a suite of AWS services to create a fully functional web app without managing any servers. The frontend has a minimalist, brutalist design and interacts with a serverless backend to manage tasks.

This project is a great introduction to serverless architecture, demonstrating how to connect a static frontend to a dynamic, event-driven backend in the cloud.

Architecture

The application is built on a classic serverless pattern:

Frontend Hosting: An Amazon S3 bucket is configured to host the static web files (HTML, CSS, JavaScript).
API Layer: Amazon API Gateway provides a RESTful API endpoint that the frontend can call. It acts as the front door to our backend logic.
Backend Logic: An AWS Lambda function, written in Python, contains all the business logic for creating, reading, updating, and deleting tasks.
Database: Amazon DynamoDB, a NoSQL database, stores the tasks in a table.

The flow is straightforward: The user interacts with the website hosted on S3. The JavaScript on the site makes secure calls to API Gateway, which triggers the Lambda function. The Lambda function then executes the required logic and manipulates the data in the DynamoDB table.

Setup Instructions

Follow these steps carefully to deploy the application in your own AWS account.

Step 1: Create the DynamoDB Table

First, we need a database to store our tasks.

Navigate to the Amazon DynamoDB console.
Click on Create table.
Set the Table name to ToDo_Tasks. This name must match exactly what is in the Lambda function code.
For the Partition key, enter taskId and ensure its type is set to String.
Leave all other settings as their default values and click Create table.

Step 2: Create the IAM Role

Our backend function will need permission to access the database. We will create an IAM role for this purpose.

Navigate to the IAM console.
Select Roles from the side menu and click Create role.
For the Trusted entity type, select AWS service.
For the Use case, choose Lambda, then click Next.
On the "Add permissions" page, search for and attach the following two policies:
- AWSLambdaBasicExecutionRole (This allows the function to write logs).
- AmazonDynamoDBFullAccess (This grants full access to DynamoDB).
Click Next.
Set the Role name to TasksLambdaRole and click Create role.

Step 3: Create the Lambda Function

This is the core of our backend, containing the application logic.

Navigate to the AWS Lambda console.
Click Create function.
Select Author from scratch.
Set the Function name to tasksManager.
For the Runtime, select a recent Python version (e.g., Python 3.9).
Expand the Change default execution role section.
Select Use an existing role and choose the TasksLambdaRole you created in the previous step.
Click Create function.
Once the function is created, go to the Code source editor and replace the default code with the full Python script for the application.
Click the Deploy button to save your code.

Step 4: Configure the API Gateway

Next, we will create a public endpoint for our frontend to communicate with the Lambda function.

Navigate to the Amazon API Gateway console.
Find the REST API option and click Build.
Select New API and give it an API name, such as TasksAPI.
Click Create API.
Create the Resources and Methods:
- Select the root (/) resource, click Actions, and choose Create Resource. Name it tasks.
- With the new /tasks resource selected, click Actions and Create Method. Choose GET from the dropdown. For the integration, select Lambda Function, check the Use Lambda Proxy integration box, and select your tasksManager function. Save the method.
- Repeat the process to create a POST method on the /tasks resource, linking it to the same Lambda function.
- Now, select the /tasks resource again, click Actions, and Create Resource. For the Resource Path, enter {taskId}. The curly braces are critical as they define a path parameter.
- On the new /tasks/{taskId} resource, create PUT and DELETE methods, linking both to the same tasksManager Lambda function with proxy integration.
Enable CORS (Cross-Origin Resource Sharing):
- Select the /tasks resource, click Actions, and select Enable CORS.
- Confirm the settings in the dialog box and click the Enable CORS... button. This will automatically create an OPTIONS method to handle preflight requests from the browser.
Deploy the API:
- Click Actions and select Deploy API.
- Choose [New Stage] for the deployment stage and enter a name, such as prod.
- Click Deploy.
- After deployment, copy the Invoke URL. You will need this for the frontend configuration.

Step 5: Prepare the Frontend Files

On your local machine, create the three frontend files: index.html, style.css, and script.js.

Crucially, open the script.js file and find the API_ENDPOINT constant. Paste the Invoke URL you copied from API Gateway as its value.

// Example from script.js
const API_ENDPOINT = 'https://xxxxxxxxx.execute-api.your-region.amazonaws.com/prod/tasks';

Step 6: Host the Website on S3

Finally, we will upload our frontend files to S3 and make them publicly accessible.

Navigate to the Amazon S3 console.
Click Create bucket.
Provide a globally unique bucket name.
Uncheck the "Block all public access" box and acknowledge the warning.
Click Create bucket.
Open your new bucket and go to the Properties tab.
Scroll down to Static website hosting and click Edit. Enable it, and set the Index document to index.html. Save your changes.

Go to the Permissions tab and edit the Bucket policy. Paste the following JSON, replacing YOUR_BUCKET_NAME with your actual bucket name.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "PublicReadGetObject",
            "Effect": "Allow",
            "Principal": "*",
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::YOUR_BUCKET_NAME/*"
        }
    ]
}

Save the policy.
Go to the Objects tab and upload your index.html, style.css, and script.js files.

Accessing Your Application

To view your live application, return to the Properties tab of your S3 bucket. Scroll down to the Static website hosting section and click on the Bucket website endpoint URL. Your serverless to-do application is now deployed and ready to use.

Creating Lambda functions and APIs in AWS and integrating them- Beginners Guide

Reet — Thu, 01 May 2025 04:37:28 +0000

AWS Lambda lets you run code without provisioning or managing servers, whereas
Amazon API Gateway is a fully managed service that makes it easy for developers to create, publish, maintain, monitor, and secure APIs.

In this beginners guide we will be creating a Lambda function, we will then create an API and integrate the API to the Lambda function (to invoke the lambda function through API calls). We will do all of this while being in the free tier.

1. Creating and testing the lambda function

• In AWS Console, search and open 'Lambda'

• Click on `Create Function`

• In the new page, select `Author from scratch`

• Give a name to the function

In 'Runtime' you can give a programming language of your choice.
Select the x86 instruction set architecture. Unless you are writing very low level stuff you can select arm64 too.

• Hover into the `Code` tab

Here, you can write the code which you want the function to perform.

After, every change, we use Deploy to make the change effective.

Test is pretty self-explanatory, its used to test the code.

Your code may show different behaviour in Test and API Calls since Tests are not invoking HTTP Requests.

Paste the below code in the code editor

import json

def lambda_handler(event, context):

    # Parse JSON body
    try:
        body = json.loads(event.get("body", "{}"))
    except json.JSONDecodeError:
        return {
            "statusCode": 400,
            "body": json.dumps({"error": "Invalid JSON body"})
        }
    x = body.get("message", "no-message")

    return {
        "statusCode": 200,
        "message": x
    }

lambda_handler - loads the event body, and context object of the execution
event is the request body passed by the API gateway in a json format

body = json.loads(event.get("body", "{}")) - it first checks for the body key in the event, if present, it returns the string present in it, which the json.loads() function converts into a (key,value) dictionary, if theres no body key present, it loads an empty dictionary.

x = body.get("message", "no-message") - assign the value stored by the message key to x, if there's no message key, assign "no message" to x

At last we return the message recieved by the API with a status code.

• Testing the code with `Test Events`

Test events can be used to create various tests for our code.

Click on + Create new test event.
A new pane in the code appears.

Give a name to the test (for now, we keep the tests private).
Paste a valid test case like the case below -

{
  "body": "{\"message\": \"Hello from client\"}"
}

Click on Save and then the play button next to the test event in the sidebar to perform tests.
A more comprehensive way to handle tests is available in the Test tab next to Code.

2. Creating REST API and integrating it with Lambda function

• Search and open `API Gateway`

• Scroll down to `REST API` and select `Build`

• Give a name to your API, scroll down and select `Create API`

• Creating a Method

Every API handles its tasks using different HTTP Methods. Most popularly used htttp methods are GET, POST, PUT, PATCH, and DELETE, these methods can perform CRUD operations on the server.

Therefore, we need to create Methods which our API can handle.

In the newly created API, select Create method.

In Method type section, select POST from the drop down menu.

Select the Integration type as Lambda function.

Under the Lambda function section, click on the search bar and select the lambda function created to integrate with the API.

• Testing the API Method

In the method created, go to Test tab

In the request body paste the following HTTP request:

{
  "body": "{\"message\": \"Hello from client\"}"
}

and, click Test.

You should see a response body like this

• Creating `Stages` and Deploying the API

Stages are environments where you can deploy your API. Stages create a referecne to different lifecycles of the API, e.g. dev, prod, beta, etc.

In the sidebar, select Stages

In the stages section, select Create stage, give a name to the stage, e.g. prod, test or dev, and select *new stage* for stage deployement. The select Create stage.

In the sidebar, select Resources and click on Deploy API

Your API is now live!

• Sending requests to the API

In the Stages section, copy the Invoke url. This is the url created for a stage, to send requests.

Open ReqBin, and paste the invoke url and request body, select POST as request method type and Send the request.

You should see a response like this

Sending requests using curl --

Paste the below command in the command line (curl should be installed) --

curl -X POST "<your-invoke-url>" -d '{"body": "{\"message\": \"Hello from Command Line\"}"}'

You should see a reponse like this from AWS --

{"statusCode": 200, "message": "Hello from Command Line"}%

Quick & easy way to clean and preprocess image dataset for Machine Learning

Reet — Sat, 26 Apr 2025 07:37:06 +0000

Go to GitHub - Skin melanoma classification using DL Techniques for complete source code implementation

1. Dataset Description:

Name: Melanoma Skin Cancer Dataset of 10000 Images
Desc: This dataset contains two classes of melanoma cancer, malignant and benign.

Total size: 103.28mb

Total files: 10.6k

Dataset files:

type	train	test
benign	5000	500
malignant	4605	500

URL: https://doi.org/10.34740/kaggle/dsv/3376422

2. Dataset Cleaning and Processing

2.1 Data Cleaning:

Data cleaning aims to remove erroneous data from the dataset. We remove images that don't follow a strict set of rules (e.g. have entropy <8.0 and can be decoded correctly).
Cleaning involves below checks to be performed on images:

1. file jpeg signature check using magic number
2. file size thresold
3. image dimesion (must be 300x300)
        3.1 aspect ratio (must be 1)
4. Decoding validity (can the image be loaded)
5. Entropy check:
       For 24-bit RGB images, where each channel is 8-bit, 
       entropy can theoretically go up to: Max Entropy = 24.0 (8 bits × 3 channels),
       where R, G & B are the 3 channels,
       generally, images with entropy 12-15 should be inspected and 
       those having entropy >15 should be discarded

Checks discarded:
1. EXIF data check - no image had exif data & it wasn't required for this use case
2. Uniform color check - gave faulty results for almost all images
3. Progressive Loading Test (JPEG) - not needed for this case
4. Extension mismatch - the images had '.jpg' extension & all the files already had     'jpeg' signatures

2.1.1 File jpeg signature check using magic number

File Signature (Magic Number):
Check if the header bytes match the expected format (e.g., JPEG: FF D8, PNG: 89 50 4E 47).

sig_map = { 'jpeg': [b'\xFF\xD8\xFF'] }
if any(header.startswith(s) for s in sig_map.get(fmt, [])):
        print(f"Magic: OK ({fmt})")
        return 1

2.1.2 File size thresold

Unusually small or excessively large files might indicate corruption.

min_size=1000
max_size=100000000
size = os.path.getsize(path)
if size < min_size:
    print(f"Size: Too small ({size} B)")
    return 0
elif size > max_size:
    print(f"Size: Too large ({size} B)")
    return 0
else:
    print(f"Size: OK ({size} B)")
    return 1

2.1.3 Image dimension

For our case we identify the images that aren't 300x300.
Images not having size 300x300 are then checked if their aspect ratio is 1 or not, they can be then treated accordingly.

w, h = img.size
if w == 300 and h == 300:
    return 1
else:
    if w/h == 1:
        return 0
    else:
        return -1

2.1.4 Decoding validity

Attempt decoding using a reliable image library (e.g., OpenCV, Pillow). Any exceptions indicate corruption.

try:
    img = Image.open(path)
    img.verify()         # structural check
    img = Image.open(path)
    img.load()           # pixel data check
    print("Decode: OK")
    return [img, 1]
except Exception as e:
    print(f"Decode: ERR ({e})")
    return [None, 0]

2.1.5 Entropy check

Checks for very high detail or possibly random noise. Extremely low or high entropy may suggest compression issues or data corruption.

For 24-bit RGB Images (max 8 bits of entropy in each of R G B channels) —

Entropy	Interpretation	Action
< 7.0	Very low detail, possibly blank or flat	🔴 Discard or review
7.0 – 12.0	Normal range for real-world RGB images	🟢 Accept
12.0 – 15.0	High detail or noise	🟡 Inspect if unusual
> 15.0	Highly random or corrupted (e.g., noise)	🔴 Suspicious, review
> 24.0	Impossible for 8-bit RGB	❌ Discard (bug or corruption)

⚠️After visual inspection of training images of the dataset chosen—

Entropy of range 5.5 - 12.0 turned out fine.

hist = img.histogram()
total = sum(hist)
probs = [h/total for h in hist if h > 0]
entropy = -sum(p * math.log2(p) for p in probs)
# 7.0 – 12.0 is the acceptable entropy for colored 24-bit RGB
# but on visual inspection of training images, entropy range of
# 5.5 - 12.0 is also fine
if entropy < 5.5 or entropy > 12.0:
    return 0
return 1

2.2 Data Cleaning Verdict:

All benign & malignant images in train and test were NOT Erroneous

All images returned the [1, 1, 1, 1, 1] array. Where:
- the array returned represent [valid_magic_number, valid_file_size, valid_image_structure_and_pixel_data, valid_dimensions, valid_entropy]

Verdict of benign training images —

PATH: dataset/train/benign/
Print verdict of each image?[y/n]: n
Erroneous images:  0
Done.

Verdict of malignant training images —

PATH: dataset/train/malignant/
Print verdict of each image?[y/n]: n
Erroneous images:  0
Done.

2.3 Data Preprocessing

To transform the data into a clean, consistent, and standardized format suitable for model training.
Enhances data quality, reduce noise, and ensure uniformity across the dataset, thereby improving model performance and training efficiency.
Below are the 4 methods used for data preprocessing

1. image resizing (spatial normalization) (optional: all images are 300x300)
2. min-max normalization (pixel scaling) to [0, 1]
3. data augmentation
4. zero-centered normalization (standardization)

Library used - Tensorflow

2.3.1 Image resizing (spatial normalization)

This resizes the image to a fixed width & height.
This was needed because:

ML models need a fixed-sized input tensors
Prevents dimension mismatch errors during training

image = tf.image.resize(image, target_size)

Athough, for the dataset used, this was optional since all images were already 300x300.

2.3.2 Normalize to 0, 1

Converts pixel values from the typical 0–255 range to a [0, 1] float range.
[0,1] normalization was needed because:

Makes convergence faster

image = tf.cast(image, tf.float32) / 255.0

2.3.3 Data augmentation

Process of artificially increasing the size and diversity of your training dataset by applying random, label-preserving transformations to input images.
Applies random transformations such as flipping, brightness, and contrast adjustments.
Data augmentation was needed because:

It increases the diversity of training data without needing more labeled samples.
Helps models learn invariant features (e.g., object orientation doesn’t matter).
Reduces overfitting.

image = tf.image.random_flip_left_right(image)
image = tf.image.random_brightness(image, max_delta=0.1)
image = tf.image.random_contrast(image, 0.9, 1.1)

2.3.4 Zero-centered Normalization (or, Standardization)

Centers pixel values around zero with unit variance ((x - mean) / std).
Transforms the pixel distribution to have mean = 0 and std = 1 for each channel.

Zero-centred normalization was needed because:

Ensures input distribution is balanced, which improves training stability.
Faster convergence
More stable gradient descent

image = tf.image.per_image_standardization(image)

⚠️We have performed normalization 2 times.

Zero-centered normalization (or, Standardization) becomes neccessary after performing [0,1] Normalization:

The values are first scaled from [0-255] to 0-1 then we center them around a reference point (standardization) so the machine handles all inputs symmetrically.

If we skip scaling, standardization might be skewed due to large initial values.
If we skip standardization, optimization may be suboptimal — some channels may dominate learning.

2.4 Building image classification dataset pipeline after preprocessing

def build_tf_dataset_pipeline(dataset_directory, batch_size=32):
    dataset = tf.keras.utils.image_dataset_from_directory(
        directory=dataset_directory,
        labels='inferred',    # Infers labels from subfolder names
        label_mode='int',     # Labels are marked as integer indices
        batch_size=batch_size,# Return individual (img, label) pairs for custom pipeline
        image_size=(300,300), # Optional: initially resizes (we’ll override this later)
        shuffle=True
    )

    dataset = dataset.map(resize_image)
    dataset = dataset.map(normalize_image)
    dataset = dataset.map(augment_image)
    dataset = dataset.map(standardize_image)
    return dataset

Returns: A dataset of image tensors.
The returned value is an object of the class tensorflow.python.data.ops.map_op._MapDataset

tf.keras.utils.image_dataset_from_directory() arguments:

directory - specify the directory of the dataset to be preprocessed, in our case, its, /dataset/train/

batch_size - defines how many images to process at once

label_mode - set to int, defines that the labels will be marked like 0,1,2,3,....
e.g.

    (directory) dataset/train/
                             ├── benign      # label 0
                             ├── malignant   # label 1

labels='inferred' - infer the labels from the name of the subfolders

image_size - sets the image size

shuffle - randomizes the dataset order (buffer size = 1000), prevents learning sequence bias

The image is transformed in memory inside the pipeline — it's just passed along to the next map() stage.
Each stage of the pipeline (like .map(normalize_image), .map(augment_image)) creates a new transient image tensor for that step.

The returned tensorflow.python.data.ops.map_op._MapDataset object contains image tensors with labels, this object can now be passed for model training.