DEV Community: Randika Madhushan Perera

Building a Scalable Pet Rehoming Platform with AWS Serverless Architecture

Randika Madhushan Perera — Fri, 09 Jan 2026 09:49:08 +0000

Introduction

PawReHome is a modern, full-featured pet rehoming platform that connects loving families with pets in need. Build with a serverless-first approach, the platform leverages AWS managed services to provide a scalable, cost-effective and secure solution for animal rescue organizations and induvial users.

This article explores the AWS architecture behind the PawReHome, how several AWS services work together to create a robust, role based access control, real-time data management.

Project Overview

PawReHome is a application fully integrated with AWS backend services. The platform supports:

User authentication and authorization with role-based access control
Multi-tenant organization management for rescue groups
Pet listing management with approval workflows
Image upload for pet photos
Adoption applications and inquiry systems
Platform administration for content moderation
Featured pets curation for homepage visibility

AWS Architecture Overview

This utilizes a serverless architecture that eliminates server management overhead and provides automatic scaling.

Complete AWS services stack:

AWS Amplify: Frontend hosting, CI/CD and deployment
Amazon Cognito: User authentication and authorization
AWS Lambda: Serverless backend functions
Amazon API Gateway: RESTFUL API management and routing
DynamoDB: NoSQL database for all application data
S3: Object storage for pet images
IAM: Identity and access management

Architecture Flow

High-Level Architecture Diagram

Deep Dive: AWS Services Implementation

1. AWS Amplify - Frontend Hosting & CI/CD

Purpose: As you can see Amplify hosts the Next.js frontend application and provides continues deployment capabilities.

Implementation Details:

The amplify.yml configuration defines the build process
Automatic builds triggered on Git commits
Environment variables managed through Amplify Console
Global CDN ensures low latency worldwide
Built-in SSL certificate provisioning via AWS Certificate Manager

Benefits of the Amplify

Zero server management, you do not need to worry about.
Automatic scaling when traffic spikes.
Preview deployment.
Rollback capabilities when you have an issue.

2. Cognito - Authentication & Authorization

Cognito provides secure user access and authentication across your application. This has role-based access control (RBAC) thorough Cognito groups.

Let me explain the Authorization Flow:

User logs in via Cognito
Cognito issues JWT token with user claims
Token includes cognito:groups claim with user's groups
API Gateway validates token
Lambda functions check group membership for authorization
DynamoDB queries are filtered based on user permissions

3. Lambda - Serverless Backend Functions

AWS Lambda functions handle all the business logic, data processing and API responses without managing servers. How relief :)

Example Lambda function (pet listing):

const { DynamoDBClient } = require("@aws-sdk/client-dynamodb");
const { DynamoDBDocumentClient, PutCommand, ScanCommand } = require("@aws-sdk/lib-dynamodb");

const docClient = DynamoDBDocumentClient.from(new DynamoDBClient({}));
const TABLE_NAME = "PawReHome-Pets";

exports.handler = async (event) => {
  // Handle pet rehoming submission
  if (event.httpMethod === "POST") {
    const pet = {
      id: uuidv4(),
      ...JSON.parse(event.body),
      status: "pending",
      createdAt: new Date().toISOString()
    };

    await docClient.send(new PutCommand({
      TableName: TABLE_NAME,
      Item: pet
    }));

    return { statusCode: 201, body: JSON.stringify(pet) };
  }

  // Get approved pets
  if (event.httpMethod === "GET") {
    const result = await docClient.send(new ScanCommand({
      TableName: TABLE_NAME,
      FilterExpression: "#status = :status",
      ExpressionAttributeNames: { "#status": "status" },
      ExpressionAttributeValues: { ":status": "approved" }
    }));

    return { statusCode: 200, body: JSON.stringify(result.Items) };
  }
};

4. Amazon API Gateway - RESTful API Management

API gateway servers as the front door for all backend API requests, providing routing, authentication and rate limiting. (Do not forget about the rate limit, your pocket will empty 😉 and will have some concerns. 🛡️)

API Gateway Configuration:

REST API endpoint
CORS enabled for frontend cross-origin requests
Cognito Authorizer for automatic JWT validation
Lambda Proxy Integration for seamless function invocation
Stage Variables for environment management

5. DynamoDB - NoSQL Database

DynamoDB stores all application data with single-digit millisecond latency and automatic scaling.

Table Design as below:

PawReHome-Pets

Partition Key: id (String)
Attributes: name, species, breed, age, size, gender, description, 
           images[], userId, organizationId, status, featured, 
           createdAt, updatedAt
GSIs:
  - UserIdIndex (userId) - Query user's pets
  - OrgIdIndex (organizationId) - Query organization's pets

PawReHome-Organizations

Partition Key: id (String)
Attributes: name, slug, email, phone, description, logo, 
           status, verified, createdAt, reviewedBy, reviewedAt
GSIs:
  - SlugIndex (slug) - Query by URL slug

PawReHome-OrgMembers

Partition Key: organizationId (String)
Sort Key: userId (String)
Attributes: role (admin/member), joinedAt, invitedBy
GSIs:
  - UserIdIndex (userId) - Query user's organizations

PawReHome-Applications

Partition Key: id (String)
Attributes: petId, userId, status, applicantInfo, 
           submittedAt, reviewedAt
GSIs:
  - UserIdIndex (userId) - Query user's applications
  - PetIdIndex (petId) - Query pet's applications

PawReHome-Questions

Partition Key: id (String)
Attributes: petId, userId, status, applicantInfo, 
           submittedAt, reviewedAt
GSIs:
  - UserIdIndex (userId) - Query user's applications
  - PetIdIndex (petId) - Query pet's applications

PawReHome-Favorites

Partition Key: userId (String)
Sort Key: petId (String)
Attributes: addedAt

6. S3 - Image Storage

S3 uses to stores pet photos with public read access for displaying images on the website.

to be continued.....

Made with ❤️ for pets everywhere

Building a Full-Stack E-Commerce Platform with AWS

Randika Madhushan Perera — Sat, 27 Dec 2025 10:59:22 +0000

A Technical Deep-Dive into Modern Serverless Architecture

Introduction

I recently build an e-commerce website that handles custom gifts orders with image uploads, payment processing, order tracking, and admin dashboard and all running on a completely serverless architecture. This blog walks through the complete technical decisions, architecture patterns and lessons learned.

The Tech Stack that I used

Layer: Frontend Framework
Technology: Next.js 15
Why I choose it: Server components, built in API routs

Layer: Database
Technology: Amazon DynamoDB
Why I choose it: Serverless, pay-per-request, automatic scaling

Layer: File Storage
Technology: S3
Why I choose it: Reliable, cheap, presigned URLs for secure access

Layer: Authentication
Technology: Cognito
Why I choose it: Managed auth, MFA support, integrates with Amplify

Layer: Email Service
Technology: Amazon SES
Why I choose it: Transactional emails, high deliverability, low cost

Layer: Hosting
Technology: Amazon Amplify Gen 2
Why I choose it: Git based deployment, automates SSL, CDN included.

Architecture Overview

The Customer Journey

Step by Step:

Landing Page - Customer learns about the product and clicks "Order Now"
Customization - Customer writes their custom message and optionally uploads a photo
Address Entry - Customer enters recipient shipping address and their billing details
Payment - Redirected to Payment Gateway
Success - After payment, customer sees their order ID and tracking link
Email - Customer receives confirmation email with order details
Tracking - Customer can check order status anytime using their order ID

Payment Flow (The Tricky Part)

Image Upload Architecture

Why I Used Presigned URLs?

Instead of making the S3 bucket public, I use presigned URLs that:

Expire after 1 hour
Are generated on-demand
Keep the bucket private and secure
Work with CloudFront CDN

Email Notification System

Email Features

HTML templates with incline CSS for email client compatibility
Multiple admin recipients
Reply-to headers, so admin can respond directly to customers
Order details formatted for easy reading and fulfillment

Deployment Pipeline

The entire deployment is automated,

Push the code to GitHub
Amplify detects the change
Builds the Next.js application
Injects environment variables
Deploys to CloudFront CDN
Site is live in 305 minutes

Deep Dive: AWS Services Powering This Application

AWS Services Architecture

1. AWS Amplify Gen 2

What is it?

AWS Amplify Gen 2 is a complete development platform that handles hosting, CI/CD, and backend infrastructure for web applications. It's the second generation of Amplify.

Why I chose it?

Git based deployments: Push to GitHub --> Automatically deployed into AWS
Built in CI/CD: No Jenkins, No GitHub Actions
Free SSL certificates: HTTPS enabled automatically
Global CDN: CloudFront is included
Custom domains: Easy DNS configurations
Environment variables: Secure secrets management
Preview Deployment: Test PRs before merging

How It's Used in This Project

2. Amazon DynamoDB

What is it?

DynamoDB is a fully managed NoSQL database service that provides single-digit millisecond performance at any scale. It's serverless, no provisioning, no patching or management required.

Why I chose it?

Serverless, Auto-scaling
Pay-per request
Single digit ms latency
AWS-native integration

3. Amazon S3

What is it?

S3 is object storage built for storing and retrieving any amount of data. It's the backbone of AWS storage.

Why I chose it?

Unlimited storage
Presigned URLs
Cheap
Direct SDK access

S3 Features Used

Private Bucket: Images not publicly accessible
Presigned URLs: Temporary access links (1 hour)
Server-side Upload: API route handles upload, not browser
Content-Type: Proper MIME types for images

Security Configuration

Bucket Policy: Private by default
CORS: Enabled for upload endpoints
Presigned URLs: Time-limited access
IAM: Scoped permissions for put/get only

4. Amazon Cognito

What is it?

Amazon Cognito provides authentication, authorization and user management for web and mobile apps. Users can sign in directly or through third party identity providers.

Why I chose it?

AWS-native
MFA support

How It's Used

5. Amazon SES (Simple Email Service)

What is it?

Amazon SES is a cloud based email service for sending transactional, marketing and notification emails.

How It's Used

6. Amazon CloudFront (via Amplify)

What is it?

CloudFront is a content delivery network that delivers content with low latency from 400+ edge locations worldwide.

How It's Used

Terraform-CodeGen0: A Terraform Code Generator

Randika Madhushan Perera — Sun, 12 Jan 2025 20:58:48 +0000

This is a submission for the GitHub Copilot Challenge : New Beginnings

What I Built

When we are deploying an application, especially in a cloud environment, we often need to manage the various infrastructure components, such as VPC, Servers, Databases, CDN, etc. Doing this manually is somewhat time-consuming and challenging. To overcome this, we can use terraform.

Terraform allows the development of the architecture that deploys the application in a cloud environment.

But writing the terraform is also somewhat time-consuming we need to create those codes manually, plus we need to know the terraform.

Using this Terraform-CodeGen0 you can easily generate the terraform codes. All you need to input the VPC name, Subnets, Route Tabels, IGW, and NAT details into this application. This Terraform-CodeGen0 provides your terraform code.

Demo

Repo

RandiakM / Terraform-CodeGen0

GitHub Copilot 1-Day Build Challenge: This is a Terraform Code generation application for VPC

VPC Terraform Generator

Introduction

The VPC Terraform Generator is a tool designed to generate Terraform code for AWS infrastructure components such as VPCs, Subnets, Route Tables, Internet Gateways (IGWs), and NAT Gateways. This project uses a microservices architecture and is hosted on Netlify, with MongoDB as the database.

How to Use

The VPC Terraform Generator allows users to generate Terraform code for various AWS components through a web interface. Users can input the necessary details for each component, and the application will generate the corresponding Terraform code.

Installation Steps

Clone the repository:

git clone https://github.com/your-repo/vpc-terraform-generator.git
cd vpc-terraform-generator

Install backend dependencies:

cd api
npm install

Install frontend dependencies:

cd ../frontend
npm install

Set up environment variables: Create a .env file in the api directory with the following content:

MONGODB_URI=your_mongodb_connection_string

Start the backend server:

cd ../api
npm run dev

Start the frontend server:

cd ../frontend
npm start

Netlify Hosting Steps

Create a…

View on GitHub

Copilot Experience

So for the initial start, I have used the below prompt to get an idea of this project.

Build a Terraform code generator for AWS infrastructure components (VPCs, Subnets, Route Tables, IGWs, NAT, etc.) using a microservices architecture. The application will be hosted on Netlify, and MongoDB will be used as the database.

Starting project structure:

Terraform Code Generation:

Ask to implement the utility functions to generate Terraform code-based user inputs for each resource type.

Edits:

Auto Complete:

For this code auto-completion, I provide the description as a comment and start to create the function, then Copilot starts suggesting the code sections that are related to my task.

Documentation Creation:

Used the below prompt:

@workspace create a README with the below details:

1. Introduction 
2. How to use 
3. Installation Steps 
4. Netlify Hosting Steps 
5. User guide

Conclusion

Through this challenge and the journey, Copilot was an AI pair programmer who helped me build this application. From the start and even from the ending of this project Copilot was really helped me.

Copilot reviewed the codes and provide alternative solutions to overcome some issues when developing this application.

With the help of Copilot I was able to create this amazing application within just 24 hours, Wihtout copilot help this may take long.

Configuring AWS WAF, CloudFront, and S3 Bucket for Secure Access

Randika Madhushan Perera — Fri, 03 Jan 2025 08:49:10 +0000

How to Set Up AWS WAF, CloudFront and an S3 Bucket to serve content securely and Restrict content Access to certain originating sources only. This guide looks at some issues, for instance, 403 Forbidden, in order to troubleshoot the problem.

1. Objective

The purpose of this assignment is to deploy AWS WAF, CloudFront and S3 bucket:

Restrict Access to certain origins, for example, https://dev.test.com and https://int.test.com.
When the CloudFront distribution is accessed, the index.html page should be served by default.
To prevent unauthorized access and to correct common errors.

2. You will require.

And AWS S3 bucket called, sample-bucket, where the files for the static website are stored.

An AWS CloudFront distribution created with OAC linked to the S3 bucket.

3. Configuration Steps

Step 1: Make Sure S3 Bucket Policy has below

Open the S3 Console.
Select your bucket (e.g., sample-bucket).
Check the following bucket policy to allow access from CloudFront:

{
    "Version": "2008-10-17",
    "Id": "PolicyForCloudFrontPrivateContent",
    "Statement": [
        {
            "Sid": "AllowCloudFrontServicePrincipal",
            "Effect": "Allow",
            "Principal": {
                "Service": "cloudfront.amazonaws.com"
            },
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::sample-bucket/*",
            "Condition": {
                "StringEquals": {
                    "AWS:SourceArn": "arn:aws:cloudfront::AWS_ACC_ID:distribution/CLOUDFRONT_ID"
                }
            }
        }
    ]
}

Step 2: Configure CORS for the S3 Bucket

In the S3 Console, go to the Permissions tab.
Edit the CORS configuration and set the following:

[
    {
        "AllowedHeaders": ["*"],
        "AllowedMethods": ["GET", "HEAD", "PUT", "POST", "DELETE"],
        "AllowedOrigins": [
            "https://dev.test.com",
            "https://int.test.com"
        ]
    }
]

Replace AllowedOrigins with your allowed domains.
Restrict AllowedHeaders to necessary headers for production.

Step 3: Configure AWS WAF

Create WAF Rules:

Open the AWS WAF Console and select your Web ACL.
Add the following rules:

Rule 1: Allow-Traffic

Description: Allows requests from specific origins.
Rule Definition:

{
    "Name": "Allow-Traffic",
    "Priority": 1,
    "Statement": {
        "OrStatement": {
            "Statements": [
                {
                    "ByteMatchStatement": {
                        "SearchString": "https://dev.test.com",
                        "FieldToMatch": {
                            "SingleHeader": {
                                "Name": "origin"
                            }
                        },
                        "TextTransformations": [
                            {
                                "Priority": 0,
                                "Type": "NONE"
                            }
                        ],
                        "PositionalConstraint": "EXACTLY"
                    }
                },
                {
                    "ByteMatchStatement": {
                        "SearchString": "https://int.test.com",
                        "FieldToMatch": {
                            "SingleHeader": {
                                "Name": "origin"
                            }
                        },
                        "TextTransformations": [
                            {
                                "Priority": 0,
                                "Type": "NONE"
                            }
                        ],
                        "PositionalConstraint": "EXACTLY"
                    }
                }
            ]
        }
    },
    "Action": {
        "Allow": {}
    },
    "VisibilityConfig": {
        "SampledRequestsEnabled": true,
        "CloudWatchMetricsEnabled": true,
        "MetricName": "Allow-Traffic"
    }
}

Rule 2: Block-All-Other-Traffic

Description: Blocks all requests that don’t match the allowed origins.
Rule Definition:

{
    "Name": "Block-All-Other-Traffic",
    "Priority": 2,
    "Statement": {
        "NotStatement": {
            "Statement": {
                "OrStatement": {
                    "Statements": [
                        {
                            "ByteMatchStatement": {
                                "SearchString": "https://dev.test.com",
                                "FieldToMatch": {
                                    "SingleHeader": {
                                        "Name": "origin"
                                    }
                                },
                                "TextTransformations": [
                                    {
                                        "Priority": 0,
                                        "Type": "NONE"
                                    }
                                ],
                                "PositionalConstraint": "EXACTLY"
                            }
                        },
                        {
                            "ByteMatchStatement": {
                                "SearchString": "https://int.test.com",
                                "FieldToMatch": {
                                    "SingleHeader": {
                                        "Name": "origin"
                                    }
                                },
                                "TextTransformations": [
                                    {
                                        "Priority": 0,
                                        "Type": "NONE"
                                    }
                                ],
                                "PositionalConstraint": "EXACTLY"
                            }
                        }
                    ]
                }
            }
        }
    },
    "Action": {
        "Block": {}
    },
    "VisibilityConfig": {
        "SampledRequestsEnabled": true,
        "CloudWatchMetricsEnabled": true,
        "MetricName": "Block-All-Other-Traffic"
    }
}

4. Testing and Verification

Test Allowed Origins

Run the following command:

curl -X GET https://cloudfront_domain_name \
     -H "Origin: https://dev.test.com" \
     -v

Expected Result: HTTP 200 OK.

Test Disallowed Origins

Run the following command:

curl -X GET https://cloudfront_domain_name \
     -H "Origin: https://unauthorized.com" \
     -v

Expected Result: HTTP 403 Forbidden.

Check WAF Logs

Enable logging for the Web ACL in the AWS WAF Console.
Monitor logs in CloudWatch to verify rule matches.

Full Stack Application Hosting in AWS

Randika Madhushan Perera — Mon, 30 Dec 2024 16:09:30 +0000

Todo Application Documentation

Introduction
Architecture Overview
Backend Setup
Frontend Setup
AWS Configuration
Deployment
Troubleshooting
Maintenance and Updates

1. Introduction

This document provides a comprehensive guide for setting up, deploying, and maintaining a full-stack Todo application using Node.js, React, Express, and MongoDB. The application is deployed on AWS, utilizing services such as Lambda, API Gateway, S3, and CloudFront.

2. Architecture Overview

Backend: Node.js with Express, deployed as an AWS Lambda function
Frontend: React, built with Vite, hosted on S3 and served via CloudFront
Database: MongoDB Atlas
API: AWS API Gateway
Authentication: (To be implemented)

3. Backend Setup

3.1 Lambda Function

Create a new file named index.js:

const mongoose = require('mongoose');

let cachedDb = null;

async function connectToDatabase() {
  if (cachedDb) {
    return cachedDb;
  }

  const connection = await mongoose.connect(process.env.MONGODB_URI, {
    useNewUrlParser: true,
    useUnifiedTopology: true,
  });

  cachedDb = connection;
  return connection;
}

const todoSchema = new mongoose.Schema({
  title: { type: String, required: true },
  completed: { type: Boolean, default: false },
  createdAt: { type: Date, default: Date.now },
  updatedAt: { type: Date, default: Date.now },
});

const Todo = mongoose.model('Todo', todoSchema);

exports.handler = async (event) => {
  const corsHeaders = {
    'Access-Control-Allow-Origin': 'https://your-cloudfront-domain.cloudfront.net',
    'Access-Control-Allow-Headers': 'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token',
    'Access-Control-Allow-Methods': 'GET,POST,PUT,DELETE,OPTIONS',
    'Access-Control-Allow-Credentials': 'true'
  };

  if (event.httpMethod === 'OPTIONS') {
    return {
      statusCode: 200,
      headers: corsHeaders,
      body: JSON.stringify({ message: 'CORS preflight request successful' })
    };
  }

  try {
    await connectToDatabase();

    const { httpMethod, resource, pathParameters, body } = event;

    switch (`\${httpMethod} \${resource}`) {
      case 'GET /todos':
        const todos = await Todo.find().sort({ createdAt: -1 });
        return { 
          statusCode: 200, 
          headers: { ...corsHeaders, 'Content-Type': 'application/json' },
          body: JSON.stringify(todos) 
        };

      case 'POST /todos':
        const newTodo = new Todo(JSON.parse(body));
        const savedTodo = await newTodo.save();
        return { 
          statusCode: 201, 
          headers: { ...corsHeaders, 'Content-Type': 'application/json' },
          body: JSON.stringify(savedTodo) 
        };

      case 'PUT /todos/{id}':
        const updatedTodo = await Todo.findByIdAndUpdate(
          pathParameters.id,
          { ...JSON.parse(body), updatedAt: Date.now() },
          { new: true }
        );
        if (!updatedTodo) {
          return { 
            statusCode: 404, 
            headers: { ...corsHeaders, 'Content-Type': 'application/json' },
            body: JSON.stringify({ message: 'Todo not found' }) 
          };
        }
        return { 
          statusCode: 200, 
          headers: { ...corsHeaders, 'Content-Type': 'application/json' },
          body: JSON.stringify(updatedTodo) 
        };

      case 'DELETE /todos/{id}':
        const deletedTodo = await Todo.findByIdAndDelete(pathParameters.id);
        if (!deletedTodo) {
          return { 
            statusCode: 404, 
            headers: { ...corsHeaders, 'Content-Type': 'application/json' },
            body: JSON.stringify({ message: 'Todo not found' }) 
          };
        }
        return { 
          statusCode: 200, 
          headers: { ...corsHeaders, 'Content-Type': 'application/json' },
          body: JSON.stringify({ message: 'Todo deleted successfully' }) 
        };

      default:
        return { 
          statusCode: 400, 
          headers: { ...corsHeaders, 'Content-Type': 'application/json' },
          body: JSON.stringify({ message: 'Invalid request' }) 
        };
    }
  } catch (error) {
    console.error('Error:', error);
    return { 
      statusCode: 500, 
      headers: { ...corsHeaders, 'Content-Type': 'application/json' },
      body: JSON.stringify({ message: 'Internal server error' }) 
    };
  }
};

3.2 Package Configuration

Create a package.json file:

{
  "name": "todo-api-lambda",
  "version": "1.0.0",
  "description": "Todo API Lambda Function",
  "main": "index.js",
  "dependencies": {
    "mongoose": "^6.0.0"
  }
}

3.3 Deployment Package

Install dependencies: npm install
Create a ZIP file: zip -r function.zip index.js node_modules

4. Frontend Setup

4.1 Create React App

Create a new Vite project: npm create vite@latest client -- --template react
Navigate to the project directory: cd client
Install dependencies: npm install

4.2 API Service

Create a file named src/services/todoService.js:

import axios from 'axios';

const API_URL = 'https://your-api-gateway-url.execute-api.us-west-2.amazonaws.com/prod/todos';

const api = axios.create({
  baseURL: API_URL,
  withCredentials: true
});

export const getTodos = async () => {
  const response = await api.get('');
  return response.data;
};

export const createTodo = async (title) => {
  const response = await api.post('', { title });
  return response.data;
};

export const updateTodo = async (id, updates) => {
  const response = await api.put(`/\${id}`, updates);
  return response.data;
};

export const deleteTodo = async (id) => {
  await api.delete(`/\${id}`);
};

4.3 Main App Component

Update src/App.jsx:

import React, { useState, useEffect } from 'react';
import { getTodos, createTodo, updateTodo, deleteTodo } from './services/todoService';
import './App.css';

function App() {
  const [todos, setTodos] = useState([]);
  const [newTodo, setNewTodo] = useState('');

  useEffect(() => {
    fetchTodos();
  }, []);

  const fetchTodos = async () => {
    const fetchedTodos = await getTodos();
    setTodos(fetchedTodos);
  };

  const handleCreateTodo = async (e) => {
    e.preventDefault();
    if (newTodo.trim()) {
      const createdTodo = await createTodo(newTodo);
      setTodos([createdTodo, ...todos]);
      setNewTodo('');
    }
  };

  const handleUpdateTodo = async (id, updates) => {
    const updatedTodo = await updateTodo(id, updates);
    setTodos(todos.map(todo => todo._id === id ? updatedTodo : todo));
  };

  const handleDeleteTodo = async (id) => {
    await deleteTodo(id);
    setTodos(todos.filter(todo => todo._id !== id));
  };

  return (
    <div className="App">
      <h1>Todo App</h1>
      <form onSubmit={handleCreateTodo}>
        <input
          type="text"
          value={newTodo}
          onChange={(e) => setNewTodo(e.target.value)}
          placeholder="Add a new todo"
        />
        <button type="submit">Add Todo</button>
      </form>
      <ul>
        {todos.map(todo => (
          <li key={todo._id}>
            <input
              type="checkbox"
              checked={todo.completed}
              onChange={() => handleUpdateTodo(todo._id, { completed: !todo.completed })}
            />
            <span style={{ textDecoration: todo.completed ? 'line-through' : 'none' }}>
              {todo.title}
            </span>
            <button onClick={() => handleDeleteTodo(todo._id)}>Delete</button>
          </li>
        ))}
      </ul>
    </div>
  );
}

export default App;

5. AWS Configuration

5.1 Lambda Function

Create a new Lambda function in the AWS Console.
Upload the ZIP file created in step 3.3.
Set the handler to index.handler.
Add environment variable: MONGODB_URI with your MongoDB connection string.

5.2 API Gateway

Create a new API in API Gateway.
Create resources and methods for /todos and /todos/{id}.
Integrate each method with your Lambda function.
Enable CORS for each method.
Deploy the API to a new stage (e.g., "prod").

5.3 S3 and CloudFront

Create an S3 bucket for hosting the React app.
Create a CloudFront distribution with the S3 bucket as the origin.
Set up Origin Access Control (OAC) for secure access to the S3 bucket.

6. Deployment

6.1 Backend Deployment

Update the Lambda function code:

zip -r function.zip index.js node_modules
aws lambda update-function-code --function-name YourFunctionName --zip-file fileb://function.zip

6.2 Frontend Deployment

Build the React app: npm run build
Upload to S3:

   aws s3 sync build/ s3://your-bucket-name --delete

Invalidate CloudFront cache:

   aws cloudfront create-invalidation --distribution-id YourDistributionID --paths "/*"

7. Troubleshooting

7.1 CORS Issues

If encountering CORS errors:

Ensure CORS is enabled in API Gateway for all methods.
Verify CORS headers in the Lambda function response.
Check that the Access-Control-Allow-Origin header matches your CloudFront domain.

7.2 API Gateway 5XX Errors

Check Lambda function logs in CloudWatch.
Verify that the Lambda function has the correct permissions to access other AWS services.

7.3 MongoDB Connection Issues

Ensure the MONGODB_URI environment variable is set correctly in Lambda.
Verify that the Lambda function has network access to MongoDB Atlas (may require VPC configuration).

8. Maintenance and Updates

8.1 Updating the Backend

Make changes to the Lambda function code.
Redeploy using the steps in section 6.1.

8.2 Updating the Frontend

Make changes to the React application.
Rebuild and redeploy using the steps in section 6.2.

8.3 Monitoring

Use CloudWatch to monitor Lambda function performance and errors.
Set up CloudWatch Alarms for critical metrics.

8.4 Scaling

Adjust Lambda function memory and timeout settings as needed.
Consider implementing caching at the API Gateway level for frequently accessed data.

This documentation provides a comprehensive guide for setting up, deploying, and maintaining your Todo application. Remember to keep your dependencies updated and regularly review AWS best practices for potential improvements to your architecture.

Docker Hands-On Part 06

Randika Madhushan Perera — Wed, 14 Feb 2024 12:51:54 +0000

06. Docker Container Networking

Introduction

The process of containerizing applications represents a significant leap forward in how we deploy and manage software. The Flask framework, popular for its simplicity and flexibility for web application development, is an excellent candidate for Dockerization. This transformative journey from a centralized server setup to a containerized deployment offers numerous advantages, including streamlined deployment processes, improved scalability, and enhanced isolation for development and production environments.

The Journey to Dockerize a Flask Application

The task at hand involves moving a Flask application, aptly named "Notes App," into a Docker container. This application, designed for employees to store notes on current projects, will benefit from the portability and efficiency that containers provide.

Step-by-Step Guide to Containerization

01. Initial Configuration:

Begin by writing the necessary configuration code, including a Dockerfile and .dockerignore file, to build a clean and efficient Docker image for the Notes App.

Step 01: Create Dokcerignore file

$ vim .dockerignore

add below

# Exclude files
.dockeringnore
Dockerfile
.gitignore
Pipfile.lock
migrations/

Create Dockerfile and add below

# version 01
FROM python:3
ENV PYBASE /pybase
ENV PYTHONUSERBASE $PYBASE
ENV PATH $PYABASE/bin:$PATH
#install pipenv
RUN pip install pipenv

WORKDIR /tmp
COPY Pipfile .
RUN pipenv lock
RUN PIP_USER=1 PIP_IGNORE_INSTALLED=1 pipenv install -d --system --ignore-pipfile

COPY . /app/notes
WORKDIR /app/notes
EXPOSE 80
CMD ["flask", "run", "--port=80", "--host=0.0.0.0"]

Building the First Image:

With the Dockerfile in place, build the initial image version of the Notes App. This step includes setting up the database within a container derived from the first image, ensuring that the application's data layer is ready for use.

Step 02: Build the image and container

$ docker build -t notesapp:0.1 .

Step 03: Create and run a container for postgres:12.1-alpine

$ docker pull postgres:12.1-alpine

$ docker run -d --name notesdb postgres:12.1-alpine

Step 04: Create a network and attach it to the bridge network

$ docker network create -d bridge notes

Step 05: List the docker networks

$ docker network ls

NETWORK ID     NAME      DRIVER    SCOPE
df7f5f15c7ab   bridge    bridge    local
7405f25b96f6   host      host      local
ee4c71834f71   none      null      local
14219a8cf079   notes     bridge    local

Step 06: Run a container using the notesapp image and mount the migrations directory

$ docker run --rm -it --network notes -v /home/ec2-user/notes/migrations:/app/notes/migrations notesapp:0.1 bash

Step 07:

# flask db migrate

Running the Application:

Deploy the Notes App container to verify its functionality. During this phase, monitoring the logs for any issues, such as the application running in debug mode, is crucial. Debug mode is not suitable for production due to potential security and performance implications.
Enhancing for Production:

Address the debug mode issue by switching the Flask application to run on a more robust WSGI (Web Server Gateway Interface) server, such as Gunicorn. This change involves updating the environment variables, modifying the pip file to include Gunicorn, and adjusting the Dockerfile to execute Gunicorn instead of Flask directly.

Docker Hands-On Part 05

Randika Madhushan Perera — Wed, 14 Feb 2024 12:51:28 +0000

05. Building Container Images Using Dockerfiles

Introduction

The evolution of container technology seeks to address and simplify the deployment process, making it more efficient and less prone to errors. Manual image creation, while possible, contradicts this objective by introducing unrepeatable processes fraught with potential mistakes. Dockerfiles represent Docker's solution to this challenge, enabling the automation of image creation through a process that is both reproducible and version-controllable.

The Dockerfile Advantage

A Dockerfile is essentially a script composed of various commands, each instructing Docker how to build a specific image. This method allows developers to define their images as code, significantly reducing the manual labor involved in image creation and ensuring that the process can be easily repeated and integrated into continuous integration/continuous deployment (CI/CD) pipelines.

Building an Image for a Web Server

In this tutorial, we'll demonstrate how to utilize a Dockerfile to build a container image designed to serve a company's website. The process will be divided into three key versions, each improving upon the last:

Version 1: Updating the Base Image

Start by creating a Dockerfile that updates the base image using the package manager. While updating the base image is generally not required for every Dockerfile, it serves as an illustrative first step in this tutorial.

Version 2: Removing the Default Apache Welcome Page

The second version of the Dockerfile will modify the image to remove the default Apache welcome page. This step is crucial for preparing the image to host custom web content.

Version 3: Adding Website Data to the Image

The final version of the Dockerfile will incorporate the actual website data into the image. This ensures that the website is served directly from the container upon startup, providing a seamless deployment experience.

Deploying the Final Image

Once the Dockerfile for version 3 is complete, the next step involves building the image and running a container from it. This process will demonstrate the web server in action, serving the company's website from the containerized environment.

Getting Started

To begin, create a Dockerfile in a directory dedicated to this project. The Dockerfile will evolve through each version, culminating in a container image that is ready for deployment. This method not only streamlines the development and deployment process but also aligns with best practices for managing application infrastructure as code.

Step 01: Create a Docker file

$ cd content-widget-factory-inc

$ vim Dokcerfile

add the below codes to it.

#base image httpd
FROM httpd:latest   
# Run the app update
RUN apt update -y && apt upgrade -y && apt autoremove -y && apt clean && rm -rf /var/lib/apt/lists*

Step 02: Build the image

$ docker build -t widgetfactory:0.1 .

Now you can see

$ docker images

REPOSITORY              TAG       IMAGE ID       CREATED          SIZE
widgetfactory           0.1       b1f9112e68d5   22 seconds ago   180MB

Step 03: Set variables to examine the image's size and layers

$ export showLayers='{{ range .RootFS.Layers }}{{ println . }}{{end}}'
$ export showSize='{{ .Size }}'

Step 04: Show the widgetfactory image's size

$ docker inspect -f "$showSize" widgetfactory:0.1

Step 05: We are going to change Dockerfile

Add the below two lines

WORKDIR /usr/local/apache2/htdocs COPY ./web .

#base image httpd
FROM httpd:latest   
# Run the app update
RUN apt update -y && apt upgrade -y && apt autoremove -y && apt clean && rm -rf /var/lib/apt/lists*
WORKDIR /usr/local/apache2/htdocs
COPY ./web .

$ docker build -t widgetfactory:0.2 .

Step 06: Get the widgetfactory:0.2 container

$ docker run --rm -it widgetfactory:0.3 bash

$ ls -l

Step 07: Run the container.

$ docker run -d --name web1 -p 80:80 widgetfactory:0.3

Step 08: Let's test

$ wget localhost

--2024-02-14 10:31:37--  http://localhost/
Resolving localhost (localhost)... 127.0.0.1
Connecting to localhost (localhost)|127.0.0.1|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 3276 (3.2K) [text/html]
Saving to: ‘index.html’

index.html                                          100%[=================================================================================================================>]   3.20K  --.-KB/s    in 0s

2024-02-14 10:31:37 (376 MB/s) - ‘index.html’ saved [3276/3276]

Docker Hands-On Part 04

Randika Madhushan Perera — Fri, 09 Feb 2024 17:15:47 +0000

04. Storing Container Data In Docker Volumes

Introduction

In this lesson, you are going to learn how to store your container data in Docker Volumes. You can store config files, static shared files, or any stateful information that you need.

Here we are going to create 3 docker volumes.

Step-by-Step Guide Storing Container Data In Docker Volumes

Step 01: Run the docker volume command to see if having existing docker volumes.

$ docker volume ls

DRIVER    VOLUME NAME

Step 02: Now pull the image for Postgres image for setting up the docker volumes. Here I'm using Postgres 12.1. Postgres needs volumes in the background for the data.

$ docker pull postgres:12.1

$ docker run -d --name db1 postgres:12.1

$ docker run -d --name db2 postgres:12.1

Step 03: Now run the docker volume command to see the volumes

$ docker volume ls

DRIVER    VOLUME NAME
local     1ae4ee6124f0bee423bec887931d9d80aec330a50b5e4edbaf4f151a19c2f23a
local     3dd9a005bb83fc892e2098237fa2bb3ad26407b82bb28096b64a327f26f4eb89

Step 04: It's very difficult to identify the volumes that using above. So we can run the below command.

$ docker inspect db1 -f '{{ json .Mounts }}' | python -m json.tool

[
    {
        "Type": "volume",
        "Name": "3dd9a005bb83fc892e2098237fa2bb3ad26407b82bb28096b64a327f26f4eb89",
        "Source": "/var/lib/docker/volumes/3dd9a005bb83fc892e2098237fa2bb3ad26407b82bb28096b64a327f26f4eb89/_data",
        "Destination": "/var/lib/postgresql/data",
        "Driver": "local",
        "Mode": "",
        "RW": true,
        "Propagation": ""
    }
]

Create the Docker Volumes

Step 01: We can create docker volumes using the create command.

$ docker volume create website

DRIVER    VOLUME NAME
local     1ae4ee6124f0bee423bec887931d9d80aec330a50b5e4edbaf4f151a19c2f23a
local            e534041b91b8b520b79a9f42c4f97fa260cae5e25a11517ce2eed0f92faefad2
local     website

Step 02: Put data in the docker volumes. So we are going to put our web app codes under the website volume

$  sudo cp -r /home/ec2-user/content-widget-factory-inc/web/* /var/lib/docker/volumes/website/_data/

To see the files

$ sudo ls -l /var/lib/docker/volumes/website/_data
total 16
drwxr-xr-x. 2 root root   76 Feb  9 16:51 img
-rw-r--r--. 1 root root 3276 Feb  9 16:51 index.html
-rw-r--r--. 1 root root 2910 Feb  9 16:51 quote.html
-rw-r--r--. 1 root root 2611 Feb  9 16:51 support.html
-rw-r--r--. 1 root root 2645 Feb  9 16:51 widgets.html

Remove Unused Docker Volumes

Run the docker volume prune command.

$ docker volume prune

WARNING! This will remove anonymous local volumes not used by at least one container.
Are you sure you want to continue? [y/N] y
Deleted Volumes:
1ae4ee6124f0bee423bec887931d9d80aec330a50b5e4edbaf4f151a19c2f23a
b61fc8243670de085a3fb120fceec864893ec097838e8be67c05b2e47daaef99
107c5e2f43fb50271133c7df2868716f2f4080e65dcc5cf2367581be6425149e

Total reclaimed space: 0B

Backup and Restore data from Docker Volumes

To do this we need to switch to the root.

# docker ps -a
CONTAINER ID   IMAGE           COMMAND                  CREATED        STATUS        PORTS                                   NAMES
77fc2b4639f4   postgres:12.1   "docker-entrypoint.s…"   4 hours ago    Up 4 hours    5432/tcp                                db2
dfa30582e464   postgres:12.1   "docker-entrypoint.s…"   4 hours ago    Up 4 hours    5432/tcp                                db1
53aace921e70   httpd:latest    "httpd-foreground"       12 hours ago   Up 12 hours   0.0.0.0:8080->80/tcp, :::8080->80/tcp   httpd


# docker volume ls
DRIVER    VOLUME NAME
local     3dd9a005bb83fc892e2098237fa2bb3ad26407b82bb28096b64a327f26f4eb89
local     e534041b91b8b520b79a9f42c4f97fa260cae5e25a11517ce2eed0f92faefad2
local     website

We need to inspect the volume.

# docker volume inspect website
[
    {
        "CreatedAt": "2024-02-09T16:48:21Z",
        "Driver": "local",
        "Labels": null,
        "Mountpoint": "/var/lib/docker/volumes/website/_data",
        "Name": "website",
        "Options": null,
        "Scope": "local"
    }
]

To get the backup we need to follow the below instructions. We are storing the backup under/tmp/ dir.

# tar czf /tmp/website_$(date +%Y-%m-%d-%H%M).tgz -C /var/lib/docker/volumes/website/_data .

# ls -l /tmp/website_2024-02-09-1708.tgz
-rw-r--r--. 1 root root 11385 Feb  9 17:08 /tmp/website_2024-02-09-1708.tgz

Now we see how we are doing this using non-privileged users.

$ docker run -it --rm -v website:/website -v /tmp:/backup bash tar czf /backup/website_$(date +%Y-%m-%d-%H%M).tgz -C /website .

$ ls -l /tmp/website_*
-rw-r--r--. 1 root root 11385 Feb  9 17:08 /tmp/website_2024-02-09-1708.tgz
-rw-r--r--. 1 root root 11375 Feb  9 17:13 /tmp/website_2024-02-09-1713.tgz

Docker Hands-On Part 03

Randika Madhushan Perera — Fri, 09 Feb 2024 09:17:31 +0000

03. Handcrafting a Container Image

Introduction

This approach to creating a container image, emphasizes the practicality of incorporating data directly into the image rather than manually mounting it each time a container is started. This method is particularly useful for web deployment, where efficiency and customization are key. The process outlined involves starting with a pre-built HTTPD image and modifying it to suit specific needs.

Step-by-Step Guide to Creating a Handcrafted Container Image

01. Selecting a Base Image: The journey begins with selecting HTTPD as the base image, a popular choice for serving web content.

02. Adding Tools and Data: To customize the container, tools are added to fetch website data from a repository, such as GitHub. This step ensures that the necessary content is directly embedded within the container.

03. Creating the Initial Image: After customizing the template container with the website's data, an image is created from this state, labeled Widget Factory v1.

04. Optimizing the Image: Inspection of the initial image reveals it to be larger than necessary due to residual installation artifacts. By removing these, a cleaner and more efficient image, Web App v2, is produced.

05. Deployment and Testing: Multiple containers are spawned from the optimized v2 image to test and view the final product in action.

The Process Visualized

The creation process visualizes the transition from a generic HTTPD latest version image to a specialized Widget Factory image. The initial modifications result in the v1 image, which is further refined to create the optimized v2 image. This final version is then used to run multiple containers, showcasing the website deployment.

Step 01: Create another container called "webtemplate" using httpd.

$ docker run --name webtemplate -d httpd:latest

$ docker ps -a
CONTAINER ID   IMAGE          COMMAND              CREATED         STATUS         PORTS                                   NAMES
4dadb7c6ca70   httpd:latest   "httpd-foreground"   5 seconds ago   Up 4 seconds   80/tcp                                  webtemplate

Step 02: Get an internal terminal of the webtemplate container.

$ docker exec -it webtemplate bash

Step 03: Update the container and install git.

# apt update && apt install -y git

Step 04: Clone the repo from the GitHub.

Repo Link: https://github.com/RandiakM/content-widget-factory-inc.git

# git clone https://github.com/RandiakM/content-widget-factory-inc.git /tmp/widget-factory-inc

# ls -l /tmp/widget-factory-inc

Step 05: Now we need to set up the htdocs folder. Remove the default apache page and copy our web pages.

root@4dadb7c6ca70:/usr/local/apache2# pwd
/usr/local/apache2

# rm -rf htdocs/index.html

# cp -r /tmp/widget-factory-inc/web/* htdocs

# ls -l htdocs

Step 06: Create an image from the container, we need the container ID.

$ docker ps -a
CONTAINER ID   IMAGE          COMMAND              CREATED             STATUS             PORTS                                   NAMES
4dadb7c6ca70   httpd:latest   "httpd-foreground"   About an hour ago   Up About an hour   80/tcp                                  webtemplate

$ docker commit 4dadb7c6ca70 example/widgetfactory:v1

$ docker images
[ec2-user@ip-172-16-1-10 ~]$ docker images
REPOSITORY              TAG       IMAGE ID       CREATED          SIZE
example/widgetfactory   v1        1676623db0fc   11 seconds ago   277MB
httpd                   latest    59bcd61b45fd   3 weeks ago      167MB
hello-world             latest    d2c94e258dcb   9 months ago     13.3kB

You can see our example/widgetfactory size is large. All this coming from the extra build tools. We need to clean this up.

Step 07: Go to the inside of the container, and need to remove git, web app codes, and remove the cache.

$ docker exec -it webtemplate bash

# rm -rf /tmp/widget-factory-inc

# apt remove git -y && apt autoremove -y && apt clean

# exit

Step 08: Now we need to run docker commit again.

$ docker commit 4dadb7c6ca70 example/widgetfactory:v2

$ docker images
REPOSITORY              TAG       IMAGE ID       CREATED         SIZE
example/widgetfactory   v2        01e2bd60f515   7 seconds ago   187MB
example/widgetfactory   v1        1676623db0fc   7 minutes ago   277MB
httpd                   latest    59bcd61b45fd   3 weeks ago     167MB

Step 09: Remove v1 image and run the v2. Stop the webtemplate.

$ docker rmi example/widgetfactory:v1

$ docker run -d --name web1 -p 8081:80  example/widgetfactory:v2

$ docker ps -a
CONTAINER ID   IMAGE                      COMMAND              CREATED             STATUS             PORTS                                   NAMES
adaf110c7088   example/widgetfactory:v2   "httpd-foreground"   4 seconds ago       Up 3 seconds       0.0.0.0:8081->80/tcp, :::8081->80/tcp   web1
4dadb7c6ca70   httpd:latest               "httpd-foreground"   About an hour ago   Up About an hour   80/tcp                                  webtemplate
53aace921e70   httpd:latest               "httpd-foreground"   4 hours ago         Up 4 hours         0.0.0.0:8080->80/tcp, :::8080->80/tcp   httpd


$ docker stop webtemplate

Step 10: Now in your browser navigate to the public_ip:8081

Docker Hands-On Part 02

Randika Madhushan Perera — Fri, 09 Feb 2024 03:25:39 +0000

02. Working With Prebuilt Docker Images

Overview

Let's say you have been assigned a task to migrate a website from a server to a container. So how do we get a container? The easy way is to create a container from a prebuilt image. Since we are hosting a website we will need a web server.

This lab has 5 goals;

Explore Docker Hub: We have to find the web server images we want to try.
Get and View httpd: Retrieve the httpd image from Docker Hub, run a container from the image, and view the results.
Run the Website in httpd: Get the website content, add it to an httpd container, and view the webpage.
Get and View Nginx: Retrieve the nginx image from Docker Hub, run a container from the image, and view the results.
Run the website in Nginx: Add the website content to an nginx container and view the webpage.

When we hit the server from our browser, the web page will load. We want that same thing to happen with the containers. We'll load the website into an httpd container and then when we hit it, it will get back to the same website. Lastly, we'll host the site using nginx just like we did with httpd.

1. Explore Docker Hub

Navigate to the Docker Hub and find the httpd container (latest).

Step 01: Verify the docker service is running by using the below command.

$ docker ps -a
CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES

Step 02: We can start by pulling the httpd docker from the Docker Hub.

$ docker pull httpd

Step 03: To see the docker images run the below command.

$ docker images
REPOSITORY    TAG       IMAGE ID       CREATED        SIZE
httpd         latest    59bcd61b45fd   2 weeks ago    167MB

Step 04: Now we need to start our container

the port will be 8080 on the server and port 80 will be connected in to the container. (-p 8080:80)

-d --> detach mode

$ docker run --name httpd -p 8080:80 -d httpd:latest
016b3196078f73a0ea062024b4b6b82430a81aad1b02fe8329fa78997931fc

$ docker ps -a
CONTAINER ID   IMAGE          COMMAND              CREATED          STATUS          PORTS                                   NAMES
016b3196078f   httpd:latest   "httpd-foreground"   44 seconds ago   Up 43 seconds   0.0.0.0:8080->80/tcp, :::8080->80/tcp   httpd

Step 05: Now we can test it if the httpd is running using a web browser.

Enter your server's public IP in the web browser. You can see it.

Step 06: Pulling the content.

GitHub Repo: Link

We are using the git clone command to clone the repo. If you haven't installed git on your server, you need to install.

$ git clone https://github.com/RandiakM/content-widget-factory-inc.git

$ cd content-widget-factory-inc

Step 07: Now we need to mount this web app with our httpd docker. First, we need to stop the httpd docker.

$ docker stop httpd
httpd

$ docker ps -a
CONTAINER ID   IMAGE          COMMAND              CREATED      STATUS                     PORTS     NAMES
016b3196078f   httpd:latest   "httpd-foreground"   2 days ago   Exited (0) 4 seconds ago             httpd

So if we are trying to mount this with the same name it will give an error, to avoid this we can remove the container for httpd.

$ docker rm httpd
httpd

[ec2-user@ip-172-16-1-10 web]$ docker ps -a
CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES

Step 08: Mount the web with the container.

$ docker run --name httpd -p 8080:80 -v /home/ec2-user/content-widget-factory-inc/web:/usr/local/apache2/htdocs:ro -d httpd:latest
f4eca276afa82dcc50424ad4a3d0ebcb344c946f6797aebfaaada4c2b432e54f


$ docker ps -a
CONTAINER ID   IMAGE          COMMAND              CREATED          STATUS          PORTS                                   NAMES
f4eca276afa8   httpd:latest   "httpd-foreground"   12 seconds ago   Up 11 seconds   0.0.0.0:8080->80/tcp, :::8080->80/tcp   httpd

Step 09: Now you can navigate "http://public_ip:8080/" to see the web app.

Docker Hands-On Part 01

Randika Madhushan Perera — Sun, 04 Feb 2024 09:42:29 +0000

01. Initializing the Docker Environment

Installing Docker CE

Installing Required Packages: If you are using Essential packages for Docker CE in CentOS, such as utils, device-mapper-persistent-data, and lvm2, are installed using the yum package manager.

I'm using the Linux EC2 2023 server.

To install Docker on Amazon Linux 2 or Amazon Linux 2023

1.Update the installed packages and package cache on your instance.

$ sudo yum update -y

2.Install the most recent Docker Community Edition package.

For Amazon Linux 2, run the following:

$ sudo amazon-linux-extras install docker

For Amazon Linux 2023, run the following:

$ sudo yum install -y docker

3.Start and enable the Docker service

$ systemctl enable --now docker.service
$ systemctl ststus docker.service

Add the ec2-user to the docker group so that you can run Docker commands without using sudo.

$ sudo usermod -a -G docker ec2-user

Pick up the new docker group permissions by logging out and logging back in again. To do this, close your current SSH terminal window and reconnect to your instance in a new one. Your new SSH session should have the appropriate docker group permissions.

Verify that the ec2-user can run Docker commands without using sudo.

$ docker ps -a

Learning Docker - Storage Driver

Randika Madhushan Perera — Sun, 04 Feb 2024 08:40:31 +0000

03. Selecting a Storage Driver

Overview

In this lesson, we delve into the concept of storage drivers within Docker, a pivotal component in managing the temporary internal storage of a container's writable layer. Docker's storage drivers facilitate a pluggable framework that allows for diverse management strategies of the container's internal storage, especially when the container's software writes data to disk without using mounted volumes or external storage.

The Role of Storage Drivers

Pluggable Framework: Docker employs a pluggable framework for its storage drivers, offering flexibility in how container storage is managed. This adaptability is crucial for supporting multiple operating systems and environments, as certain storage methods are only viable in specific settings.

Selection Criteria: The choice of a storage driver depends on the operating system, environment, and specific use cases. Factors such as performance efficiency for read/write operations and compatibility with the underlying OS influence this decision.

Key Storage Drivers

Overlay2: Overlay2 is a file-based storage driver and is the default option for both Ubuntu and CentOS (version 8 and newer). It is efficient for most general use cases, particularly where reading operations predominate.

Devicemapper: In contrast, Devicemapper offers block storage and is the default for CentOS 7 and earlier versions. It tends to perform better in scenarios with heavy write operations to the container layers.

Managing Storage Drivers

Checking the Current Driver: The docker info command can be used to identify the current storage driver in use. For example, CentOS 7 defaults to Devicemapper.

Changing Storage Drivers: Users can override the default storage driver by either passing a flag to the Docker daemon or by specifying the desired driver in the daemon.json configuration file. The latter method is recommended for its standardization across different environments.

Run the below command to see the docker-storage option

docker info

$ docker info
Client:
 Version:    24.0.5
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.0.0+unknown
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx

Server:
 Containers: 1
  Running: 0
  Paused: 0
  Stopped: 1
 Images: 1
 Server Version: 24.0.5
Storage Driver: overlay2
  Backing Filesystem: xfs

In my case, I have Storage Driver as overlay2.

Practical Application

I used the AWS Ec2 Linux 2023 server, showing how to check the current storage driver and switch to a different one if necessary.
The process involves editing the Docker daemon's configuration file (daemon.json) to explicitly set the storage driver to devicemapper or overlay2 highlighting the standardized approach for modifying Docker daemon settings.

$ vim /etc/docker/daemon.json

In daemon.json file you need to put below.

{
  "storage-driver": "overlay2"
}

$ systemctl restart docker.service

Conclusion

Understanding and effectively managing Docker's storage drivers is essential for optimizing container performance and ensuring compatibility with various environments. This lesson provides a foundational knowledge of Docker storage drivers, setting the stage for more advanced container management and operation practices in subsequent lessons.

DEV Community: Randika Madhushan Perera

Building a Scalable Pet Rehoming Platform with AWS Serverless Architecture

Introduction

Project Overview

AWS Architecture Overview

Architecture Flow

High-Level Architecture Diagram

Deep Dive: AWS Services Implementation

Building a Full-Stack E-Commerce Platform with AWS

A Technical Deep-Dive into Modern Serverless Architecture

Introduction

The Tech Stack that I used

Architecture Overview

The Customer Journey

Payment Flow (The Tricky Part)

Image Upload Architecture

Why I Used Presigned URLs?

Email Notification System

Deployment Pipeline

Deep Dive: AWS Services Powering This Application

AWS Services Architecture

Terraform-CodeGen0: A Terraform Code Generator

What I Built

Demo

Repo

RandiakM / Terraform-CodeGen0

GitHub Copilot 1-Day Build Challenge: This is a Terraform Code generation application for VPC

VPC Terraform Generator

Introduction

How to Use

Installation Steps

Netlify Hosting Steps

Copilot Experience

Conclusion

Configuring AWS WAF, CloudFront, and S3 Bucket for Secure Access

Full Stack Application Hosting in AWS

Todo Application Documentation

Table of Contents

1. Introduction

2. Architecture Overview

3. Backend Setup

3.1 Lambda Function

3.2 Package Configuration

3.3 Deployment Package

4. Frontend Setup

4.1 Create React App

4.2 API Service

4.3 Main App Component

5. AWS Configuration

5.1 Lambda Function

5.2 API Gateway

5.3 S3 and CloudFront

6. Deployment

6.1 Backend Deployment

6.2 Frontend Deployment

7. Troubleshooting

7.1 CORS Issues

7.2 API Gateway 5XX Errors

7.3 MongoDB Connection Issues

8. Maintenance and Updates

8.1 Updating the Backend

8.2 Updating the Frontend

8.3 Monitoring

8.4 Scaling

Docker Hands-On Part 06

06. Docker Container Networking

Docker Hands-On Part 05

05. Building Container Images Using Dockerfiles

Docker Hands-On Part 04

04. Storing Container Data In Docker Volumes

Docker Hands-On Part 03

03. Handcrafting a Container Image

Docker Hands-On Part 02

02. Working With Prebuilt Docker Images

Docker Hands-On Part 01

01. Initializing the Docker Environment

Learning Docker - Storage Driver

03. Selecting a Storage Driver