DEV Community: Ngozi Opara

AWS Serverless Guide: Securing IoT Data Ingestion with API Gateway, Lambda, and DynamoDB

Ngozi Opara — Wed, 31 Dec 2025 16:28:39 +0000

When I set out to build my gas emissions monitoring system, I faced a choice. I could build the system the "textbook" way using AWS IoT Core, MQTT protocols, and device shadows. Or, I could build it the "Ship It" way using AWS API Gateway, Lambda, and Amplify.

The textbook approach is robust. However, I had strict time constraints and only one gas sensing device to configure. I chose the "Ship It" way.

In this guide, I will show you how to build a serverless architecture to monitor IoT devices. We will skip the complex protocols and create a frontend dashboard using standard web tools.

Prerequisites:

A configured gas sensing unit: This guide assumes your ESP32 controller is ready to retrieve data.
An AWS Account: You will need complete CLI and console access.
Basic Knowledge: You should understand APIs and how serverless functions work.

Architecture Diagram

Lane 1: The Machine Lane

First, we need to create the engine that acts as the brain of our application. To reduce maintenance and keep costs low, we will use AWS Lambda. This function will trigger a write operation to our NoSQL database in DynamoDB when it receives data from our API. It will also send this data to our frontend application.

STEP 1: Creating the Database

On the right side of the dashboard, click Create table. You will see a table creation interface. DynamoDB is schemaless, but you must specify a primary key. Enter the following details:

Partition Key: device_id (String). This groups your data by device.
Sort Key: timestamp (String). This orders your data by time.

Even though DynamoDB handles unstructured data, defining these keys makes it cheaper to query your table later. This guide from the AWS documentation explains how choosing the right keys can save you money on query costs.

Leave all other settings as default and click “Create table.” Take note of the region your table is in because you will need it later. After a few minutes, your table will appear with an active status:

Step 2: Create a Lambda Function

AWS Lambda is responsible for the ingestion and read logic of your architecture. It allows us to send and query data without managing servers. To create a function, search for “Lambda” in the search bar and open it in a new tab.

You will be redirected to the Functions page. Click on “Create function” in the top-right corner of that page.

You have the option of authoring your function, using a Blueprint or using a container image. We will keep things simple and stick to writing our function from scratch. Keep “Author from scratch” selected.

The next thing we have to do is fill in the basic information like you see below:

When naming your Lambda functions, it is important that you use a name that accurately describes the function of your code. In this guide, we are going to be using Node.js as our preferred language. However, AWS Lambda supports many other programming languages.

For our server, we will be sticking to the standard x86_64 AMD architecture. This is the safest standard option. If you are curious about the difference between arm64 and x86, this article by AWS explains the use cases perfectly.

Leave the durable execution setting unchecked and click “Create function.” You should now see your function running in the Functions tab.

Step 3: Write the Code to Run the Function

Enter the console of your function and navigate to the “Code” tab. Replace the existing code with the following block:

import { DynamoDBClient } from "@aws-sdk/client-dynamodb";
import { DynamoDBDocumentClient, ScanCommand, PutCommand } from "@aws-sdk/lib-dynamodb";

const client = new DynamoDBClient({ region: "eu-north-1" });
const docClient = DynamoDBDocumentClient.from(client);

export const handler = async (event) => {
    const tableName = "VehicleEmissionData";

    // 1. Identify what kind of request this is (GET or POST)
    const method = event.httpMethod || (event.requestContext && event.requestContext.http ? event.requestContext.http.method : null);

    console.log("Incoming Request Method:", method);

    const headers = {
        "Access-Control-Allow-Origin": "*",
        "Access-Control-Allow-Headers": "Content-Type",
        "Access-Control-Allow-Methods": "GET, POST, OPTIONS"
    };

    try {
        // SCENARIO 1: WRITING DATA (POST) - For ESP32
        if (method === 'POST') {
            if (!event.body) {
                throw new Error("No body found in the request");
            }
            const body = typeof event.body === 'string' ? JSON.parse(event.body) : event.body;

            console.log("Received Data to Write:", body);
            const newItem = {
                device_id: body.device_id || "UNKNOWN_DEVICE",
                timestamp: new Date().toISOString(), // Server-side timestamp is safer
                CO: Number(body.CO),
                CO2: Number(body.CO2),
                location: body.location || "Unknown"
            };

            const command = new PutCommand({
                TableName: tableName,
                Item: newItem
            });

            await docClient.send(command);

            return {
                statusCode: 200,
                headers: headers,
                body: JSON.stringify({ message: "Data saved successfully!", item: newItem })
            };
        }

        // SCENARIO 2: READING DATA (GET) - For React App
        else {
            const command = new ScanCommand({ TableName: tableName });
            const data = await docClient.send(command);

            return {
                statusCode: 200,
                headers: headers,
                body: JSON.stringify(data.Items)
            };
        }

    } catch (err) {
        console.error("Error:", err);
        return {
            statusCode: 500,
            headers: headers,
            body: JSON.stringify({ error: err.message })
        };
    }
};

In this block of code, we use a simple if/else statement to control the architectural flow. This acts like a traffic cop, sending POST requests one way and GET requests another.

When we get a POST request, we read the data and add our own timestamp. We do this because the clocks on small devices are often wrong. This guide on IoT time synchronization explains why you should not trust them. Once the data is ready, we save it to the database.

The else block handles GET requests from our frontend dashboard. Here, we grab every record in the table and send it back as a list. It is a simple way to dump all your data straight to the user.

Step 4: Grant Write Permissions to Lambda

For our sensor data to be stored in our database, we need to give Lambda function permission to write to and read from DynamoDB.

To change the permission settings on your function, Click on the “Configuration” tab in your Lambda function. Select “Permissions” on the left navigation bar.

Click the role name under "Execution role". This redirects you to the AWS IAM console. Click the “Add permissions” dropdown and select “Attach policies.”

Search AmazonDynamoDBFullAccess in the new tab that opens up and check the box next to the policy name. Click “Add permissions”. Now your Lambda function has full access to DynamoDB.

Step 5: Creating the API with API Gateway

APIs allow different software to communicate and exchange data. Here, the APIs we create will enable our Lambda function to retrieve information from the IoT device and send data to the web user.

To begin, Search for API Gateway in the console. In the API Gateway interface, click “Create an API” to begin:

After clicking that, you will see a list of API types that AWS offers. Click “Build” on the HTTP API as that is what we will be using here.

Next name your API and leave the end point as IPV4. As good practice, your API name should be descriptive of what you will use it for.

In the Integrations section, click on “Add integration” and Select Lambda. Choose your function (EmissionDataProcessor) from the list as you see below:

Now, we need to add Routes to our API to tell our API what functions it will be carrying out.

Click on Next and you will see the Configure routes interface. Click “Add routes” and you will be directed to a drop down with different route options. Configure your routes this way:

The GET route is the web users’ access to the sensor data from a frontend dashboard. The POST route will allow us to write to our database. The Resource Path is essentially the street address or name for a specific set of actions within your API.

It tells the API Gateway what resource the client wants to interact with. For example, in the path /EmissionDataProcessor, the resource is the sensor data itself. Click “Next” to review and create your API.

To connect your IoT device, you need the Invoke URL. You can find this URL on the main API page under the "Stages" section. Copy this URL and add it to your Arduino code.

Lane 1: The Human Lane (Velocity via Amplify)

The end goal of this project is to create a human-readable dashboard available to users via the web. This dashboard will read data live from our database and display it beautifully with the help of our API.

To create and deploy this dashboard, we will use this React app in this GitHub repository. Clone this repository to your local machine. I recommend using a Linux shell with the AWS CLI configured. This article by AWS teaches how to securely configure the AWS CLI.

Understanding the Project Layout

Before we deploy, it helps to understand what we are building. Here is the layout of our application:

emissions-dashboard/
├── public/
│   ├── index.html
│   └── favicon.ico
├── src/
│   ├── components/
│   │   ├── Header.js
│   │   ├── SummaryCards.js
│   │   ├── Charts.js
│   │   ├── ForecastCard.js
│   │   └── DataTable.js
│   ├── services/
│   │   └── dataService.js         # DynamoDB connection logic
│   ├── utils/
│   │   ├── statusCalculator.js    # Emission status logic
│   │   └── constants.js           # Thresholds, colors, etc.
│   ├── styles/
│   │   └── styles.js              # Shared styles
│   ├── App.js                     # Main component
│   ├── index.js
│   └── aws-exports.js             # Auto-generated by Amplify
├── amplify/
│   └── (Amplify configuration files)
├── .env.local                     # Local environment variables
├── .gitignore
├── package.json
└── README.md

Two files here are critical. dataService.js handles the connection to DynamoDB, while statusCalculator.js determines if the emission levels are safe.

After cloning this application, check that it runs with the npm start command and you should see this:

Step 6: Deploy with AWS Amplify

Now we need to get this application online. We will use the AWS Amplify CLI to host our frontend. This tool does more than just host files. It also distributes your site across a global content delivery network (CDN) called Amazon CloudFront, ensuring your dashboard loads instantly for users anywhere in the world.

To begin, open your terminal and install the Amplify tool globally. Follow this Amplify documentation to install and configure the Amplify CLI on your local machine.

Once it is installed, initialize the project in your terminal:

amplify init

The CLI will ask you a series of questions to understand your project. You can answer them as follows:


? Enter a name for the project: emissionsdashboard
? Initialize the project with the above configuration? No
? Enter a name for the environment: dev
? Choose your default editor: Visual Studio Code (or your preference)
? Choose the type of app that you're building: javascript
? What javascript framework are you using: react
? Source Directory Path: src
? Distribution Directory Path: build
? Build Command: npm run build
? Start Command: npm start
? Select the authentication method you want to use: AWS profile
? Please choose the profile you want to use: default

After that, you should see:

✔ Successfully created initial AWS cloud resources for deployments.
✔ Initialized provider successfully.

Next, we tell Amplify how we want to host the site. Run the following command:

amplify add hosting

Select Hosting with Amplify Console (Managed hosting with custom domains, Continuous Deployment) and choose Manual deployment for now.

Finally, push your application to the cloud:

amplify publish

This command runs your build script, uploads the files to an S3 bucket, and deploys them to the content delivery network. After a few minutes, the terminal will display your live URL.

After waiting 2-5 minutes, you should see:

✔ Deployment complete!
https://dev.xxxxxxxx.amplifyapp.com

View your project in the Amplify Console with the command:

amplify console

Your live application needs to know where to find your API. In local development, we used a .env file, but in the cloud, we must set this manually. Open your application and click on "Environment variables” in the side menu.

Click on “Manage variables” and add the new variable in this format:

For the value, paste your API Gateway Invoke URL. The variable name must match exactly what is in your code. This ensures that when Amplify builds your app in the cloud, it knows exactly where to send your API requests.

Step 7: Set Up Continuous Deployment With Git

Manually running amplify publish every time you change a line of code is tedious. It is also error-prone. In a professional environment, we use Continuous Deployment (CD). This means that every time you save changes to your code repository, AWS automatically detects the update, builds the new version, and deploys it for you.

This guide on CI/CD best practices explains why automating your deployment pipeline is the single best thing you can do for your team's velocity.

First, initialize a Git repository in your project folder:

git init
git add .
git commit -m "<commit message>"

Next, create a new repository on GitHub. Then, link your local project to GitHub and push your code:

git remote add origin https://github.com/yourusername/emissions-dashboard.git
git branch -M main
git push -u origin main

Step 8: Connect GitHub to Amplify Console

Now we will tell AWS to listen to that GitHub repository.

Go back to the AWS Amplify Console.
Find your app (emissionsdashboard) and open it.
Navigate to the “Hosting environments” tab and click “Connect branch.”
Select GitHub as your source code provider and authorize AWS to access your account.

Amplify usually detects the settings automatically, but you should double-check the Build settings section. Ensure the Build output directory is set to build. If it is set to /, your deployment will fail because Amplify won't know which folder contains your website files.

Click “Save and deploy.” AWS will now trigger a build. You can watch the progress bars as it provisions a server, downloads your code, builds the React app, and deploys it. Once the circles turn green, your automated pipeline is live.

How I Would Optimize This for Production

I built this system to work fast, not to last forever. The "Ship It" approach is perfect for prototypes or hackathons, but if I were tasked with scaling this for a paying client, I would make three major changes.

Closing the Open Door

In the Lambda code, I set the CORS header to *. This is the digital equivalent of leaving your front door wide open. It means any website on the internet can send requests to my API. In a real production environment, I would lock this down to only allow requests from my specific Amplify domain.

Checking ID Cards

Right now, my API is public. Anyone with the URL can send fake data to my database or read my sensor history. To secure this, I would implement AWS IAM authentication or API Keys. This ensures that only trusted devices and users can interact with the system. For an extra layer of defense, I might even add a Web Application Firewall (WAF) to block bots and malicious traffic, but checking IDs comes first.

Escaping the "Scan" Trap

I used ScanCommand to read my data. This works fine for a few hundred records, but it reads every single item in the database every time the dashboard loads. If this sensor ran for a year, that table would grow to millions of records. A Scan operation on a table that size would be incredibly slow and expensive. This breakdown of DynamoDB pricing models highlights why I would switch to QueryCommand and use proper indexes for a long-term solution.

For now, though, the dashboard is live. I built something from scratch that actually works, and that is the most important step in engineering.

How to Deploy Microservices App With EKS (And Automate the Full Pipeline with AWS CloudFormation & GitHub Actions)

Ngozi Opara — Sun, 30 Nov 2025 22:29:06 +0000

In production environments for SaaS platforms, manual management is a recipe for disaster. DevOps and Cloud engineering teams do not just build systems and the automation that manages the systems.

In this tutorial, you will build the cloud infrastructure for an online retail store using a microservice system. This retail store is split into five independent key players:

UI: The frontend store.
Catalog: The product inventory.
Cart: The user's shopping session.
Orders: The processing logic.
Checkout: The payment gateway interface.

You will not just click buttons. You will define the network, the cluster, and the pipelines as code using CloudFormation and GitHub Actions

Architecture Diagram

The Cost of the Cloud: Pricing & Calculator

Before you write a single line of code, you need to estimate the cost. Cost optimization is one of the pillars of the AWS Well-Architected Framework, and ignoring it early is one of the most common mistakes engineers make.

You can use the AWS Pricing Calculator to estimate the monthly run cost of this project.

1. The Visible Costs

Open the calculator and search for Amazon EKS. The control plane alone runs about $0.10 per hour, which comes out to roughly $73/month just for the cluster itself.

Next, search for Amazon EC2.
You will need compute power for your microservices. For a small production cluster, you might select t3.medium instances. If you run 2 worker nodes, that is roughly $60.00/month (depending on the region).

2. The Hidden Costs (The "Gotchas")

This is where many engineers get surprised. The network is not free.

NAT Gateways are one of the bigger gotchas. In this architecture, your private nodes need to reach the internet to pull updates and images.

NAT Gateways make that possible, but you pay for every hour they exist, plus roughly $0.045 per GB of data that passes through them. If your app is pulling large images constantly, that fee compounds fast.

Load Balancers (ALB/NLB): They are another line item to watch. Exposing your UI to the public requires an Application Load Balancer. You pay an hourly rate plus a fee for LCU-hours, which scales with traffic.
Cross-AZ data transfer: It is very easy to miss this entirely. EKS places a network interface on your nodes for every pod. The interface itself is usually free, but if Service A in Zone 1 calls Service B in Zone 2, you're paying data transfer fees on every one of those requests.

Prerequisites

Basic knowledge of networking on AWS: for a refresher on the core concepts of a Virtual Private Cloud (VPC) , review the official Amazon VPC documentation.
Beginner understanding of containerization and container orchestration: The Kubernetes Basics tutorial is a solid starting point if you're new to orchestration.
GitHub Account: You will need this for your code and automation pipelines.

Phase 1: The Network Foundation

Your first task is to build the land where your city will live. That land is the Virtual Private Cloud (VPC) and you will create a CloudFormation template named infrastructure.yaml.

This architecture is robust. It consists of:

2 Public Subnets: One in each Availability Zone (AZ) for the Load Balancers and NAT Gateways.
4 Private Subnets:
- 2 for your Worker Nodes (where the apps live).
- 2 for your Databases (RDS).
Gateways: An Internet Gateway for public traffic and 2 NAT Gateways for private outbound traffic.
Route Tables: Three separate tables to control traffic flow.

Defining the VPC and Subnets

Add this code to `CustomVPC.yaml`:

AWSTemplateFormatVersion: '2010-09-09'
Description: >
  This template provisions the CORE requirements for Project Bedrock:
  A VPC with public/private subnets, gateways, route tables,
  essential EKS IAM roles, and necessary Security Groups for the EKS cluster stack.

Parameters:
  VpcCIDR:
    Description: The CIDR block for the VPC (e.g., 10.0.0.0/16).
    Type: String
    Default: 10.0.0.0/16

  SubnetCIDR:
    Description: The CIDR block for the VPC (e.g., 10.0.0.0/16).
    Type: String
    Default: 10.0.0.0/24

  InstanceType: 
    Description: The EC2 instance type (e.g., t2.micro). 
    Type: String 
    Default: t3.micro 
    AllowedValues: 
      - t3.micro 
      - t3.small 
      - t3.medium 
    ConstraintDescription: Must be a valid EC2 instance type.

Resources:
  MyVPC:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: !Ref VpcCIDR
      EnableDnsSupport: 'true'
      EnableDnsHostnames: 'true'
      Tags:
        - Key: Name
          Value: !Sub "${AWS::StackName}-VPC"

# Attaching Internet Gateways
  InternetGateway:
    Type: AWS::EC2::InternetGateway
    Properties:
      Tags:
        - Key: Name
          Value: !Sub "${AWS::StackName}-IGW"

  AttachGateway:
    Type: AWS::EC2::VPCGatewayAttachment
    Properties:
      VpcId: !Ref MyVPC
      InternetGatewayId: !Ref InternetGateway

# Public Subnets
  PublicSubnetA:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref MyVPC
      CidrBlock: 10.0.1.0/24
      MapPublicIpOnLaunch: 'true'
      AvailabilityZone: !Select [ 0, !GetAZs '' ]
      Tags:
        - Key: Name
          Value: !Sub "${AWS::StackName}-PublicSubnetA"
        # Tag for ALB discovery
        - Key: kubernetes.io/role/elb
          Value: '1'
        - Key: kubernetes.io/cluster/${AWS::StackName}-EKSCluster
          Value: shared

  PublicSubnetB:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref MyVPC
      CidrBlock: 10.0.2.0/24
      MapPublicIpOnLaunch: 'true'
      AvailabilityZone: !Select [ 1, !GetAZs '' ]
      Tags:
        - Key: Name
          Value: !Sub "${AWS::StackName}-PublicSubnetB"
        - Key: kubernetes.io/role/elb
          Value: '1'
        - Key: kubernetes.io/cluster/${AWS::StackName}-EKSCluster
          Value: shared

# Private Subnets 
  PrivateSubnetA1:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref MyVPC
      CidrBlock: 10.0.10.0/24
      MapPublicIpOnLaunch: 'false'
      AvailabilityZone: !Select [ 0, !GetAZs '' ]
      Tags:
        - Key: Name
          Value: !Sub "${AWS::StackName}-PrivateSubnetA1"
        # Tag for internal ALB discovery
        - Key: kubernetes.io/role/internal-elb
          Value: '1'
        - Key: kubernetes.io/cluster/${AWS::StackName}-EKSCluster
          Value: shared

  PrivateSubnetA2:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref MyVPC
      CidrBlock: 10.0.11.0/24
      MapPublicIpOnLaunch: 'false'
      AvailabilityZone: !Select [ 0, !GetAZs '' ]
      Tags:
        - Key: Name
          Value: !Sub "${AWS::StackName}-PrivateSubnetA2"
        - Key: kubernetes.io/role/internal-elb
          Value: '1'
        - Key: kubernetes.io/cluster/${AWS::StackName}-EKSCluster
          Value: shared

  PrivateSubnetB1:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref MyVPC
      CidrBlock: 10.0.20.0/24
      MapPublicIpOnLaunch: 'false'
      AvailabilityZone: !Select [ 1, !GetAZs '' ]
      Tags:
        - Key: Name
          Value: !Sub "${AWS::StackName}-PrivateSubnetB1"
        - Key: kubernetes.io/role/internal-elb
          Value: '1'
        - Key: kubernetes.io/cluster/${AWS::StackName}-EKSCluster
          Value: shared

  PrivateSubnetB2:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref MyVPC
      CidrBlock: 10.0.21.0/24
      MapPublicIpOnLaunch: 'false'
      AvailabilityZone: !Select [ 1, !GetAZs '' ]
      Tags:
        - Key: Name
          Value: !Sub "${AWS::StackName}-PrivateSubnetB2"
        - Key: kubernetes.io/role/internal-elb
          Value: '1'
        - Key: kubernetes.io/cluster/${AWS::StackName}-EKSCluster
          Value: shared

2. Gateways and Routing

Now you add the networking logic. Your private subnets need a way to reach the internet to pull updates and container images, but you don't want them publicly exposed. NAT Gateways solve that and route tables then tell each subnet exactly where to send its traffic.

Add this to CustomVPC.yaml:

# NAT GATEWAYS

# Elastic IPs
  MyNATGateway1EIP:
    Type: AWS::EC2::EIP
    Properties:
      Domain: VPC
      Tags:
        - Key: Name
          Value: !Sub "${AWS::StackName}-NATGateway1-EIP"

  MyNATGateway2EIP:
    Type: AWS::EC2::EIP
    Properties:
      Domain: VPC
      Tags:
        - Key: Name
          Value: !Sub "${AWS::StackName}-NATGateway2-EIP"

# NATGATEway Actual Resources
  MyNATGateway1:
    Type: AWS::EC2::NatGateway
    Properties:
      AllocationId: !GetAtt MyNATGateway1EIP.AllocationId
      SubnetId: !Ref PublicSubnetA
      Tags:
        - Key: Name
          Value: !Sub "${AWS::StackName}-NATGateway1"

  MyNATGateway2:
    Type: AWS::EC2::NatGateway
    Properties:
      AllocationId: !GetAtt MyNATGateway2EIP.AllocationId
      SubnetId: !Ref PublicSubnetB
      Tags:
        - Key: Name
          Value: !Sub "${AWS::StackName}-NATGateway2"

# Route Table
  PublicRouteTable:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref MyVPC
      Tags:
        - Key: Name
          Value: PublicRouteTable

  PublicRoute:
    Type: AWS::EC2::Route
    Properties:
      RouteTableId: !Ref PublicRouteTable
      DestinationCidrBlock: 0.0.0.0/0
      GatewayId: !Ref InternetGateway

  PublicSubnetARouteTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref PublicSubnetA
      RouteTableId: !Ref PublicRouteTable

  PublicSubnetBRouteTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref PublicSubnetB
      RouteTableId: !Ref PublicRouteTable

  PrivateRouteTableA:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref MyVPC
      Tags:
        - Key: Name
          Value: PrivateRouteTableA

  PrivateRouteA:
    Type: AWS::EC2::Route
    Properties:
      DestinationCidrBlock: 0.0.0.0/0
      NatGatewayId: !Ref MyNATGateway1
      RouteTableId: !Ref PrivateRouteTableA

  PrivateSubnetA1RouteTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      RouteTableId: !Ref PrivateRouteTableA
      SubnetId: !Ref PrivateSubnetA1

  PrivateSubnetA2RouteTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      RouteTableId: !Ref PrivateRouteTableA
      SubnetId: !Ref PrivateSubnetA2

  PrivateRouteTableB:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref MyVPC
      Tags:
        - Key: Name
          Value: PrivateRouteTableB

  PrivateRouteB:
    Type: AWS::EC2::Route
    Properties:
      DestinationCidrBlock: 0.0.0.0/0
      NatGatewayId: !Ref MyNATGateway2
      RouteTableId: !Ref PrivateRouteTableB

  PrivateAppSubnetBRouteTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      RouteTableId: !Ref PrivateRouteTableB
      SubnetId: !Ref PrivateSubnetB1

  PrivateDBSubnetBRouteTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      RouteTableId: !Ref PrivateRouteTableB
      SubnetId: !Ref PrivateSubnetB2

3. Security Groups and Roles

Security Groups and IAM Roles act as the firewalls that protect your application from unauthorized access. You are also spinning up two EC2 instances to act as bastion hosts or test servers.

Add this to CustomVPC.yaml:

# ALB Security Group     
  MyALBSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Allow HTTP, HTTPS, and SSH traffic
      VpcId: !Ref MyVPC
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 80
          ToPort: 80
          CidrIp: 0.0.0.0/0 # Allows HTTP traffic
        - IpProtocol: tcp
          FromPort: 443
          ToPort: 443
          CidrIp: 0.0.0.0/0 # Allows HTTPS traffic
      Tags:
        - Key: Name
          Value: MyALBSecurityGroup

# SSH Security Group for Maintenance
  MySSHSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Allow SSH traffic
      VpcId: !Ref MyVPC
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 22
          ToPort: 22
          CidrIp: 0.0.0.0/0
      Tags:
        - Key: Name
          Value: MySSHSecurityGroup

# Server Security Group
  MyServerSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Allow HTTP and HTTPS traffic
      VpcId: !Ref MyVPC
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 22
          ToPort: 22
          SourceSecurityGroupID: !Ref MySSHSecurityGroup
        - IpProtocol: tcp
          FromPort: 80
          ToPort: 80
          SourceSecurityGroupID: !Ref MyALBSecurityGroup
        - IpProtocol: tcp
          FromPort: 443
          ToPort: 443
          SourceSecurityGroupID: !Ref MyALBSecurityGroup
      Tags:
        - Key: Name
          Value: MySecurityGroup

# My EC2 instance
  MyEC2InstanceA:
    Type: AWS::EC2::Instance
    Properties:
      InstanceType: !Ref InstanceType
      ImageId: "ami-0a716d3f3b16d290c" # Replace with the desired AMI
      SubnetId: !Ref PublicSubnetA
      KeyName: "InnovateMart" # Replace with name of your key pair
      SecurityGroupIds:
      - !Ref MyServerSecurityGroup
      - !Ref MySSHSecurityGroup
      Tags:
        - Key: Name
          Value: MyInstance

  MyEC2InstanceB:
    Type: AWS::EC2::Instance
    Properties:
      InstanceType: !Ref InstanceType
      ImageId: "ami-0a716d3f3b16d290c" # Replace with the desired AMI
      SubnetId: !Ref PublicSubnetB
      KeyName: "InnovateMart"
      SecurityGroupIds:
      - !Ref MyServerSecurityGroup
      - !Ref MySSHSecurityGroup
      Tags:
        - Key: Name
          Value: MyInstance

# EKS Cluster Role
  EKSClusterRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: !Sub "${AWS::StackName}-EKSClusterRole"
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              Service: eks.amazonaws.com
            Action: sts:AssumeRole
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/AmazonEKSClusterPolicy

# EKS Node Role
  EKSNodeRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: !Sub "${AWS::StackName}-EKSNodeRole"
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              Service: ec2.amazonaws.com
            Action: sts:AssumeRole
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy
        - arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly
        - arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy

4. Outputs

The last section of this template is the exports block. In phase two, you will be creating another template and Output is how it finds the VPC and Subnets we just created.

Add this to CustomVPC.yaml:

Outputs:
  VpcId:
    Description: The ID of the VPC
    Value: !Ref MyVPC
    Export:
      Name: !Sub ${AWS::StackName}-MyVPC

  PublicSubnetIDs:
    Description: A comma-separated list of public subnet IDs
    Value: !Join
      - ','
      - - !Ref PublicSubnetA
        - !Ref PublicSubnetB
    Export:
      Name: !Sub "${AWS::StackName}-PublicSubnetIDs"

  PrivateSubnetIDs:
    Description: A comma-separated list of private subnet IDs for EKS nodes
    Value: !Join
      - ','
      - - !Ref PrivateSubnetA1
        - !Ref PrivateSubnetA2
        - !Ref PrivateSubnetB1
        - !Ref PrivateSubnetB2
    Export:
      Name: !Sub "${AWS::StackName}-PrivateSubnetIDs"

  MyDBSecurityGroup:
    Description: The name of the DB Security Group cluster
    Value: !Ref MyServerSecurityGroup
    Export:
      Name: !Sub "${AWS::StackName}-MyDBSecurityGroup"

  MyALBSecurityGroup:
    Description: The name of the ALB Security Group cluster
    Value: !Ref MyALBSecurityGroup
    Export:
      Name: !Sub "${AWS::StackName}-MyALBSecurityGroup"

  MySSHSecurityGroup:
    Description: The name of the ALB Security Group cluster
    Value: !Ref MySSHSecurityGroup
    Export:
      Name: !Sub "${AWS::StackName}-MySSHSecurityGroup"

  MyServerSecurityGroup:
    Description: The name of the ALB Security Group cluster
    Value: !Ref MyServerSecurityGroup
    Export:
      Name: !Sub "${AWS::StackName}-MyServerSecurityGroup"

  EKSClusterRoleArn:
    Description: The ARN of the IAM role for the EKS cluster
    Value: !GetAtt EKSClusterRole.Arn
    Export:
      Name: !Sub "${AWS::StackName}-EKSClusterRoleArn"

  EKSNodeRoleArn:
    Description: The ARN of the IAM role for the EKS nodes
    Value: !GetAtt EKSNodeRole.Arn
    Export:
      Name: !Sub "${AWS::StackName}-EKSNodeRoleArn"

To build this foundation, run the following command. We will name the stack bedrock-vpc.

aws cloudformation create-stack \
  --stack-name bedrock-vpc \
  --template-body file://CustomVPC.yaml \
  --capabilities CAPABILITY_NAMED_IAM

Wait for this stack to reach CREATE_COMPLETE status before moving to Phase 2.

Phase 2: The Cluster and Nodes (The House)

Now that the land is prepped, you can build the house. Create a file named EKSCluster.yaml.

This template uses the Fn::ImportValue function. It looks for the exports from your bedrock-vpc stack to know where to place the servers. It creates the Control Plane (Brain), the Node Group (Workers), and a Read-Only Developer User.

File: `EKSCluster.yaml`

AWSTemplateFormatVersion: '2010-09-09'
Description: >
  This template provisions the EKS Cluster and a Managed Node Group
  by importing resources from the VPC stack.

Parameters:
  VPCStackName:
    Type: String
    Description: >
      The name of the CloudFormation stack that created the VPC

  ClusterVersion:
    Type: String
    Default: '1.29'
    Description: The Kubernetes version for the EKS cluster.

  InstanceType: 
    Description: The EC2 instance type for the nodes (e.g., t3.medium). 
    Type: String 
    Default: t3.medium

Resources:
  # 1. EKS Cluster (The "Brain")

  EKSCluster:
    Type: AWS::EKS::Cluster
    Properties:
      Name: !Sub "${VPCStackName}-EKSCluster"
      Version: !Ref ClusterVersion

      # This is where we import the VPC info
      RoleArn: !ImportValue
        Fn::Sub: "${VPCStackName}-EKSClusterRoleArn"
      ResourcesVpcConfig:
        SubnetIds:
          - !Select [ 0, !Split [ ',', !ImportValue { Fn::Sub: "${VPCStackName}-PublicSubnetIDs" } ] ]
          - !Select [ 1, !Split [ ',', !ImportValue { Fn::Sub: "${VPCStackName}-PublicSubnetIDs" } ] ]
          - !Select [ 0, !Split [ ',', !ImportValue { Fn::Sub: "${VPCStackName}-PrivateSubnetIDs" } ] ]
          - !Select [ 1, !Split [ ',', !ImportValue { Fn::Sub: "${VPCStackName}-PrivateSubnetIDs" } ] ]
          - !Select [ 2, !Split [ ',', !ImportValue { Fn::Sub: "${VPCStackName}-PrivateSubnetIDs" } ] ]
          - !Select [ 3, !Split [ ',', !ImportValue { Fn::Sub: "${VPCStackName}-PrivateSubnetIDs" } ] ]

        # This tells EKS which SGs to use for its own networking
        SecurityGroupIds:
          - !ImportValue
            Fn::Sub: "${VPCStackName}-MyServerSecurityGroup"
        EndpointPublicAccess: true
        EndpointPrivateAccess: false

  # 2. EKS Node Group (The "Workers")

  EKSNodeGroup:
    Type: AWS::EKS::Nodegroup
    Properties:
      ClusterName: !Ref EKSCluster
      NodegroupName: !Sub "${VPCStackName}-NodeGroup"
      InstanceTypes:
        - !Ref InstanceType
      NodeRole: !ImportValue
        Fn::Sub: "${VPCStackName}-EKSNodeRoleArn"
      # Workers live in your private subnets
      Subnets:
        - !Select [ 0, !Split [ ',', !ImportValue { Fn::Sub: "${VPCStackName}-PrivateSubnetIDs" } ] ]
        - !Select [ 1, !Split [ ',', !ImportValue { Fn::Sub: "${VPCStackName}-PrivateSubnetIDs" } ] ]
        - !Select [ 2, !Split [ ',', !ImportValue { Fn::Sub: "${VPCStackName}-PrivateSubnetIDs" } ] ]
        - !Select [ 3, !Split [ ',', !ImportValue { Fn::Sub: "${VPCStackName}-PrivateSubnetIDs" } ] ]
      ScalingConfig:
        MinSize: 2
        DesiredSize: 2
        MaxSize: 4
      RemoteAccess:
        Ec2SshKey: "InnovateMart"
        # This SG must allow SSH access
        SourceSecurityGroups:
          - !ImportValue
            Fn::Sub: "${VPCStackName}-MySSHSecurityGroup"

  ReadOnlyDevUser:
    Type: AWS::IAM::User
    Properties:
      UserName: dev-readonly-user

  ReadOnlyDevUserKey:
    Type: AWS::IAM::AccessKey
    Properties:
      UserName: !Ref ReadOnlyDevUser

Outputs:
  EKSClusterName:
    Description: The name of the EKS cluster
    Value: !Ref EKSCluster
    Export:
      Name: !Sub "${AWS::StackName}-EKSClusterName"

  EKSClusterEndpoint:
    Description: The endpoint for the EKS cluster
    Value: !GetAtt EKSCluster.Endpoint
    Export:
      Name: !Sub "${AWS::StackName}-EKSClusterEndpoint"

  ReadOnlyDevUserARN:
    Description: "ARN of the read-only developer user"
    Value: !GetAtt ReadOnlyDevUser.Arn

  ReadOnlyDevUserAccessKey:
    Description: "Access key for the read-only user"
    Value: !Ref ReadOnlyDevUserKey

  ReadOnlyDevUserSecretKey:
    Description: "Secret key for the read-only user"
    Value: !GetAtt ReadOnlyDevUserKey.SecretAccessKey

Deploy the stack:

aws cloudformation create-stack \
  --stack-name project-bedrock \
  --template-body file://infrastructure.yaml \
  --capabilities CAPABILITY_NAMED_IAM

Phase 3: The Handshake

Your cluster is alive, but your local machine doesn't know how to talk to it yet. You need to establish a secure handshake with a 'kubeconfig' file and the 'kubectl' command.

Run the update command to generate the config file:

aws eks update-kubeconfig \
  --region us-east-1 \
  --name RetailStoreCluster

Test the connection. If you see the internal IP addresses of your Kubernetes services, the handshake is complete.

kubectl get svc

If you see the internal IP addresses of your Kubernetes services come back, you're connected. The handshake is done.

Phase 4: The Database

Your microservices need somewhere to persist data. In Phase 1 you created two private database subnets, one in each Availability Zone, and tagged them for internal use.

Now you actually put something in them. You will provision a PostgreSQL RDS instance that sits in those subnets, completely isolated from the public internet, and only reachable from your worker nodes.

Create a new file named RDS.yaml.

1. The DB Subnet Group and Security Group

Before RDS can launch, it needs to know which subnets it's allowed to live in. A DB subnet group is just a named collection of subnets that RDS picks from when it places your instance.

You also need a dedicated security group that only allows PostgreSQL traffic on port 5432, and only from the worker node security group. Nothing else gets in.

AWSTemplateFormatVersion: '2010-09-09'
Description: >
  This template provisions a PostgreSQL RDS instance
  inside the private DB subnets created by the VPC stack.

Parameters:
  VPCStackName:
    Type: String
    Description: The name of the CloudFormation stack that created the VPC.

  DBUsername:
    Type: String
    Default: retailadmin
    Description: The master username for the database.

  DBPassword:
    Type: String
    NoEcho: true
    Description: The master password for the database.

  DBInstanceClass:
    Type: String
    Default: db.t3.medium
    Description: The RDS instance type.

Resources:

  DBSubnetGroup:
    Type: AWS::RDS::DBSubnetGroup
    Properties:
      DBSubnetGroupDescription: Subnet group for the retail store RDS instance
      SubnetIds:
        - !Select [ 2, !Split [ ',', !ImportValue { Fn::Sub: "${VPCStackName}-PrivateSubnetIDs" } ] ]
        - !Select [ 3, !Split [ ',', !ImportValue { Fn::Sub: "${VPCStackName}-PrivateSubnetIDs" } ] ]
      Tags:
        - Key: Name
          Value: !Sub "${AWS::StackName}-DBSubnetGroup"

  DBSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Allow PostgreSQL access from worker nodes only
      VpcId: !ImportValue
        Fn::Sub: "${VPCStackName}-MyVPC"
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 5432
          ToPort: 5432
          SourceSecurityGroupId: !ImportValue
            Fn::Sub: "${VPCStackName}-MyServerSecurityGroup"
      Tags:
        - Key: Name
          Value: !Sub "${AWS::StackName}-DBSecurityGroup"

2. The RDS Instance

Now add the actual database resource. You are enabling Multi-AZ so RDS automatically maintains a standby replica in the second Availability Zone. If the primary goes down, RDS fails over to the standby without you doing anything.

Add this to RDS.yaml:

  RetailDB:
    Type: AWS::RDS::DBInstance
    Properties:
      DBInstanceIdentifier: !Sub "${AWS::StackName}-retaildb"
      DBInstanceClass: !Ref DBInstanceClass
      Engine: postgres
      EngineVersion: '15.4'
      MasterUsername: !Ref DBUsername
      MasterUserPassword: !Ref DBPassword
      AllocatedStorage: '20'
      StorageType: gp3
      MultiAZ: true
      DBSubnetGroupName: !Ref DBSubnetGroup
      VPCSecurityGroups:
        - !Ref DBSecurityGroup
      PubliclyAccessible: false
      DeletionProtection: false
      Tags:
        - Key: Name
          Value: !Sub "${AWS::StackName}-RetailDB"

Outputs:
  DBEndpoint:
    Description: The connection endpoint for the RDS instance
    Value: !GetAtt RetailDB.Endpoint.Address
    Export:
      Name: !Sub "${AWS::StackName}-DBEndpoint"

  DBPort:
    Description: The port the database is listening on
    Value: !GetAtt RetailDB.Endpoint.Port
    Export:
      Name: !Sub "${AWS::StackName}-DBPort"

Deploy the database stack:

aws cloudformation create-stack \
  --stack-name project-bedrock-rds \
  --template-body file://RDS.yaml \
  --capabilities CAPABILITY_NAMED_IAM \
  --parameters \
    ParameterKey=VPCStackName,ParameterValue=bedrock-vpc \
    ParameterKey=DBUsername,ParameterValue=retailadmin \
    ParameterKey=DBPassword,ParameterValue=YOUR_SECURE_PASSWORD

RDS takes a few minutes to provision, especially with Multi-AZ enabled. Wait for CREATE_COMPLETE before moving on.

3. Storing the Credentials as a Kubernetes Secret

Your pods need the database connection string, but you never want credentials hardcoded into your Helm values or your container images.

A Kubernetes Secret keeps them out of your codebase and injects them into the pod at runtime as environment variables.

Once your RDS stack is complete, grab the endpoint from the outputs:

aws cloudformation describe-stacks \
  --stack-name project-bedrock-rds \
  --query "Stacks[0].Outputs"

Then create the secret in your cluster:

kubectl create secret generic retail-db-credentials \
  --from-literal=host=YOUR_RDS_ENDPOINT \
  --from-literal=port=5432 \
  --from-literal=username=retailadmin \
  --from-literal=password=YOUR_SECURE_PASSWORD \
  --from-literal=dbname=retailstore

Your microservices can now reference these values as environment variables in their pod specs without the actual credentials ever appearing in your Git repository.

Phase 5: Deploying the Application

Now you need the actual retail store code. Rather than building one from scratch, you'll use the official AWS sample retail application.

Clone the Repository:

You will use the official AWS sample retail app. Run this command in your working directory:

git clone https://github.com/aws-containers/retail-store-sample-app.git
cd retail-store-sample-app

Deploy with Helm:

We will use Helm to install the microservices defined in this repository.

helm install retail-app ./helm

Phase 6: Developer Access (RBAC)

Back in Phase 2, you created an IAM user called dev-readonly-user. That user exists in AWS, but Kubernetes doesn't know anything about it yet. You need to wire the two together using RBAC.

Start by creating a k8s directory inside the cloned repo:

mkdir k8s

Create the RBAC Files:

Inside that folder, you will create a Role and a RoleBinding. The Role defines what permissions exist, and the RoleBinding attaches that Role to the IAM user.

Together, they give a developer a way to observe what's running in the cluster without being able to change anything.

Phase 7: Automation (CI/CD)

The final piece is setting up GitHub Actions to watch this repository.

Create the file:

.github/workflows/deploy.yml in your project root.

name: Deploy to EKS
on:
  push:
    branches: [ "main" ]

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v3

      - name: Configure AWS Credentials
        uses: aws-actions/configure-aws-credentials@v2
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: us-east-1

      - name: Update Kubeconfig
        run: aws eks update-kubeconfig --name bedrock-vpc-EKSCluster

      - name: Deploy
        run: helm upgrade --install retail-app ./helm

Every push to main now triggers a deploy. The pipeline authenticates to AWS using secrets you store in your GitHub repository settings, updates the kubeconfig, and runs a Helm upgrade. If the release already exists, it upgrades it and if it does not, it creates it.

Phase 8: Exposing the Store to the Internet (Ingress)

Your application is deployed and running, but it's completely locked inside the VPC. No customer can reach it. To fix that, you need an Ingress resource and the controller that knows how to act on it. This is where your ingress.yaml file comes in.

Install the Controller:

Kubernetes does not do anything with an Ingress object on its own. You need to install the AWS Load Balancer Controller into your cluster first You need to install the AWS Load Balancer Controller into your cluster.

This controller listens for Ingress creation requests and automatically creates the corresponding AWS Application Load Balancer (ALB).

Create the Ingress File:

Place your ingress.yaml file inside the k8s folder created in Phase 5.

Example command to verify file placement:

ls k8s/ingress.yaml

Apply the Configuration:

Unlike the Helm deployment, this file is applied manually (or added to your CI/CD pipeline).

kubectl apply -f k8s/ingress.yaml

Once you apply the configuration, the Controller detects the new Ingress resource and provisions an AWS Application Load Balancer in the public subnets created in Phase 1.

You can then point your domain (e.g., store.example.com) to this Load Balancer using Route 53.

Common Issues You Might Encounter and How to Fix Them

CloudFormation stacks fail. Pods don't start. The ALB never shows up. None of this is unusual, it is just part of working with distributed infrastructure. Most failures in this architecture follow a short list of patterns, and knowing where to look cuts your debug time significantly.

The most common stack failure is a CREATE_FAILED status on the EKS cluster, usually because the VPC stack hadn't fully settled before Phase 2 ran. When this happens, open the CloudFormation events tab in the AWS console and find the first red entry, not the last one. The AWS CloudFormation troubleshooting guide breaks down the most common status codes and what they mean in plain terms.

If your pods are stuck in Pending after the Helm install, the node group is usually where the problem lives. Run kubectl describe pod <pod-name> and read the events section at the bottom, where you will likely see a message about insufficient CPU or memory. That means your t3.medium nodes are undersized, or your desired node count is too low.

You can increase DesiredSize in the node group config and update the stack. If kubectl get nodes returns nothing at all, the node IAM role is probably missing a policy, which stops the nodes from joining the cluster entirely. The EKS worker node troubleshooting docs cover this exact scenario and are worth keeping open during your first deployment.

The last thing that catches people off guard is the ALB not provisioning after you apply ingress.yaml. This almost always comes down to two things: the AWS Load Balancer Controller isn't running, or the subnet tags from Phase 1 are missing.

Run kubectl get pods -n kube-system to confirm the controller pod is healthy, then check that your public subnets carry the kubernetes.io/role/elb: "1" tag and your private subnets carry kubernetes.io/role/internal-elb: "1". Without those tags the controller has no idea where to place the load balancer, and it will silently do nothing.

DEV Community: Ngozi Opara

AWS Serverless Guide: Securing IoT Data Ingestion with API Gateway, Lambda, and DynamoDB

Prerequisites:

Architecture Diagram

Lane 1: The Machine Lane

STEP 1: Creating the Database

Step 2: Create a Lambda Function

Step 3: Write the Code to Run the Function

Step 4: Grant Write Permissions to Lambda

Step 5: Creating the API with API Gateway

Lane 1: The Human Lane (Velocity via Amplify)

Understanding the Project Layout

Step 6: Deploy with AWS Amplify

Step 7: Set Up Continuous Deployment With Git

Step 8: Connect GitHub to Amplify Console

How I Would Optimize This for Production

How to Deploy Microservices App With EKS (And Automate the Full Pipeline with AWS CloudFormation & GitHub Actions)

Architecture Diagram

The Cost of the Cloud: Pricing & Calculator

1. The Visible Costs

2. The Hidden Costs (The "Gotchas")

Prerequisites

Phase 1: The Network Foundation

Defining the VPC and Subnets

Add this code to CustomVPC.yaml:

2. Gateways and Routing

3. Security Groups and Roles

4. Outputs

File: EKSCluster.yaml

Deploy the stack:

Phase 3: The Handshake

Phase 4: The Database

1. The DB Subnet Group and Security Group

2. The RDS Instance

3. Storing the Credentials as a Kubernetes Secret

Phase 5: Deploying the Application

Clone the Repository:

Deploy with Helm:

Phase 6: Developer Access (RBAC)

Create the RBAC Files:

Phase 7: Automation (CI/CD)

Phase 8: Exposing the Store to the Internet (Ingress)

Install the Controller:

Create the Ingress File:

Example command to verify file placement:

Apply the Configuration:

Common Issues You Might Encounter and How to Fix Them

Add this code to `CustomVPC.yaml`:

File: `EKSCluster.yaml`