DEV Community: David Hyppolite

How to Deploy a FastAPI App to Azure with Docker, ACR, and GitHub Actions

David Hyppolite — Sat, 26 Apr 2025 05:43:29 +0000

Shipping a containerized app to the cloud doesn’t have to be complicated.

In this blog post, I’ll walk you through deploying a Dockerized application to Azure step-by-step from pushing your image to Azure Container Registry (ACR) to automating your deployments with GitHub Actions.

Whether you're building your first cloud native app or just need a quick blueprint for setting up Azure CI/CD, this walkthrough has you covered with screenshots, code snippets, and tips to avoid common pitfalls.

Prerequisites

Before starting, ensure you have the Azure CLI installed and you're logged in. Follow the installation guidehere

A GitHub repository containing your Dockerfile & FastAPI code.

Step 1. Create an Azure Container Registry (ACR)

We'll:

Create a Container Registry.
Push our Docker image to ACR
Take note of the login server, username, and password

Create Container Registry

In the Azure Portal, click Create a resource > Container Registry.

Use these settings.

Subscription: Your Azure subscription
Resource group: Create or use an existing one
Registry name: Give your container a unique name
Location: Choose your region
Pricing plan: Basic

Skip Networking, Encryption, and Tags.

Click Create after "Validation passed" appears.

Push Your Image

Your ACR deployment is now created and completed we have to push our image store in local machine (VSCode) to the remote container.

Make sure you are in the folder/path that contains the image

Once deployed:

Click Go to resource.

Scroll down and select Pushing a container image

You'll see instructions like these (note: Azure provides generic hello-world example commands that we'll need to modify):

To push your FastAPI application instead of the hello-world example, use these commands modify the example commands to fit your project:


# Login to your Azure Container Registry
az acr login --name <acr-name>

# Build your FastAPI Docker image (if not already built)
docker build -t <acr-name> .

# Tag your local image with your ACR address
docker tag fastapidemo myfastapidemo.azurecr.io/fastapidemo:latest

# Push your tagged image to Azure Container Registry
docker push myfastapidemo.azurecr.io/fastapidemo:latest

Best practice: use explicit version tags (v1.0, v2025‑04‑23, etc.).

You should see an output similar to mines.

Verify your image was pushed successfully with:

az acr repository list --name <acr-name> --output table

You can also see it from the Azure portal.

Step 2. Create an APP Service & Web App

In this Section you will:

Create an App Service Plan (Linux)
Create a Web App for Containers
Link it to ACR

An AppService is an HTTP-based service for hosting web applications, REST APIs, and mobile back ends.

A WebApp is an AppService that focuses on hosting web applications.

In Azure Portal, search App Services > Create > Web App

Basics

Every Field with an (* = required) must be filled

Use these Settings:
Resource Group: Select your resource group
Name: Create a Unique name
Publish(*): Container
Operating System(*): Linux
Region: Select your region
Pricing plan: Free F1 (to keep cost low)

Example.

Hit Next and skip for the Database page

Configure the container

Use the settings:

Image Source: Azure Container Registry
Registry: Select the ACR registry
Authentication: Managed identity
Image: fastapidemo (your image name)
Tag: latest (your image tag)

For the tag its best practice to avoid generic tags such as :latest instead use 1.O I only use latest for demonstration purposes

Hit Next

Networking

Enable public access: On

Hit Next

Monitor + secure

Enable Application Insights: No

In production its best to turn on to track and logs your apps data
Enable Defender for App Service: leave blank

Hit Next and Skip the Tags page

Review + Create -> Create

You should now see your basic web app all you have to do is verify your configurations now click Create

If you receive any failed deployments error switch to a different region and redeploy

Access & Verify Web App

On success, select Go to resource and open the Default Domain to confirm the app loads.

If you set everything up right you should see your containers page.

If you have any issues on the resource page under Deployment Center > View logs you will see where your deployment failed.

Step 3. Set Up GitHub Actions CI/CD

We will:

Enable Deployment Center
Generate a workflow file
Add GitHub secrets

Enable deployment Center

In your Web App portal on the left side select Deployment > Deployment Center

If you see the red "SCM basic authentication is disabled for your app. Click here to go to your configuration settings to enable." it needs to be enabled.

Click it and under Platform settings toggle SCM Basic Auth Publishing: On

Save the changes above and continue to update the app.

Connect to GitHub

Source: Github Actions

Authorize GitHub, select repo + branch once completed fill in your repos details.

The rest of your Registry settings should auto‑populate (note the managed identity it shows).

In a new tab access your container registry.

Azure Portal > your ACR > Settings > Identity > User Assigned.

Add the user assigned managed identity you have listed in Deployment Center.

Now head on back to your deployment center

Hit the Save button up above

Our build will fail we will resolve this issue in the next.

✏️ Fix the generated workflow

Head over to your GitHub repo — you'll see a new folder named .github/workflows.

This was automatically created by Azure Deployment Center when you enabled CI/CD.

Open the .yml file inside (e.g., azure-webapps.yml) — and click the ✏️ pencil icon to edit it directly in GitHub.

The code is a general-purpose template — we’ll customize it to match our actual Azure setup.

Replace the Docker login block with Azure‑based auth:

🐳 Original Build & Login Block:

You'll see something like this:

  jobs:
  build:
    runs-on: 'ubuntu-latest'

    steps:
    - uses: actions/checkout@v2

    - name: Set up Docker Buildx
      uses: docker/setup-buildx-action@v2

    - name: Log in to registry
      uses: docker/login-action@v2
      with:
        registry: https://myfastapidemo.azurecr.io/
        username: ${{ secrets.AzureAppService_ContainerUsername_dac00c4d6aa44559aceb4c95969103fb }}
        password: ${{ secrets.AzureAppService_ContainerPassword_acd4866e025148088786c136581277b5 }}

    - name: Build and push container image to registry
      uses: docker/build-push-action@v3
      with:
        push: true
        tags: myfastapidemo.azurecr.io/${{ secrets.AzureAppService_ContainerUsername_dac00c4d6aa44559aceb4c95969103fb }}/fastapidemo:${{ github.sha }}
        file: ./Dockerfile

This login method assumes credentials for Docker Hub or a public registry — but we’re using Azure Container Registry (ACR) with Azure credentials.

✅ Replace It With:

jobs:
  build:
    runs-on: 'ubuntu-latest'

    steps:
    - uses: actions/checkout@v2

    - name: Set up Docker Buildx
      uses: docker/setup-buildx-action@v2

    - name: Azure Login
      uses: azure/login@v1
      with:
        creds: ${{ secrets.AZURE_CREDENTIALS }}

    - name: 'Build and push image'
      uses: azure/docker-login@v1
      with:
        login-server: ${{ secrets.REGISTRY_LOGIN_SERVER }}
        username: ${{ secrets.REGISTRY_USERNAME }}
        password: ${{ secrets.REGISTRY_PASSWORD }}
    - run: |
        docker build . -t ${{ secrets.REGISTRY_LOGIN_SERVER }}/<youracr>:${{ github.sha }}
        docker push ${{ secrets.REGISTRY_LOGIN_SERVER }}//<youracr>:${{ github.sha }}

This uses the Azure CLI to securely authenticate to ACR using a service principal instead of Docker username/password.

Add GitHub secrets

In this section we will be setting the AZURE_CREDENTIALS secret.

Open your terminal and run the following commands

This command shows all your active resource groups look for you resource group related to your ACR/webapp. Copy the resource group "id".

az group list

Create a service principal:

az ad sp create-for-rbac --scopes <resource‑group‑id> --role Contributor --sdk-auth

example

az ad sp create-for-rbac \
  --scopes /subscriptions/88888839283023/resourceGroups/FastApI \
  --role Contributor \
  --sdk-auth

Your output will look similar to:

{
  "clientId": "xxxx6ddc-xxxx-xxxx-xxx-ef78a99dxxxx",
  "clientSecret": "xxxx79dc-xxxx-xxxx-xxxx-aaaaaec5xxxx",
  "subscriptionId": "aaaa0a0a-bb1b-cc2c-dd3d-eeeeee4e4e4e",
  "tenantId": "aaaabbbb-0000-cccc-1111-dddd2222eeee",
  "activeDirectoryEndpointUrl": "https://login.microsoftonline.com",
  "resourceManagerEndpointUrl": "https://management.azure.com/",
  "activeDirectoryGraphResourceId": "https://graph.windows.net/",
  "sqlManagementEndpointUrl": "https://management.core.windows.net:8443/",
  "galleryEndpointUrl": "https://gallery.azure.com/",
  "managementEndpointUrl": "https://management.core.windows.net/"
}

Save the JSON output because it's used in a later step.

Take note of the clientId, which you need to update the service principal in the next section.

Now we need to update the service principal to allow ACR to push and pull

Lets get our registry name.

az acr list --query "[].{Name:name}" --output table

Get the resource ID of your container registry. Replace and .

registryId=$(az acr show --name <registry-name> --resource-group <resource-group-name> --query id --output tsv)

Assign AcrPush to the service principal, which gives push and pull access to the registry. Substitute the client ID of your service principal:

az role assignment create --assignee <ClientId> --scope $registryId --role AcrPush

Save the credentials to your GitHub repo, go to your repository Settings under Security > Secrets and variables > Actions > New repository secret.

Add the following secrets:

AZURE_CREDENTIALS paste full JSON including "{}"

For enterprise-grade setups, consider using Workload Identity Federation or certificate-based service principals to avoid storing long-term secrets.

Test the Pipeline

Lets make a change to our code to see if any changes made in our code will build and deploy and show on our Azure app service.

Any changes to the repo will automatically trigger our GitHub action to Build and deploy to Azure.

For me I will be making the changes to my fastapi code from {"Welcome": "To Azure"} to {"Novice": "To Azure"} and commit the code.

After you have made the changes wait for your actions to deploy than check back to you app service domain.

As you can see the change have been reflected Automatically.

Cleanup resources

Now that we have deployed everything we need to cleanup the resources to not accrue charges.

Delete the resource group to stop charges:

az group delete --name ExampleResourceGroup -y

Replace ExampleResourceGroup with your resource group name you can locate your resource groups using

az group list

Or remove it via Azure Portal.

Search for Resource groups in the azure portal and locate the resource group you create and select it and just hit Delete resource group

Conclusion

If you followed along, you now have:

A Docker image safely stored in Azure Container Registry (ACR)
A FastAPI web app running live on Azure App Service
An end‑to‑end CI/CD pipeline powered by GitHub Actions that redeploys on every push

Deploying Grafana on an AWS EC2 Instance(2025)

David Hyppolite — Wed, 05 Mar 2025 00:51:54 +0000

This blog post walks you through setting up Grafana on an AWS EC2 instance to visualize real-time performance metrics and run stress tests, all while being secured by using Instance Connect to bypass the need for SSH.

Problem

Cloud infrastructure issues often go undetected until they cause downtime costing businesses thousands per minute and putting immense pressure on engineering teams.

Solution

This tutorial demonstrates how to implement proactive monitoring with Grafana on AWS EC2. You’ll learn to deploy an EC2 instance, attach the necessary IAM role, integrate CloudWatch, and configure Grafana to visualize performance metrics in real time.

Why This Matters

Mastering this setup enables you to:

Configure IAM roles effectively.
Deploy and secure EC2 instances using Instance Connect.
Integrate AWS CloudWatch for comprehensive monitoring.
Create dynamic Grafana dashboards for real-time performance visualization.

What You'll Learn

By following this guide, you will achieve a complete AWS monitoring setup that not only demonstrates your infrastructure deployment skills but also provides visual proof of your ability to monitor and manage real-time performance metrics effectively.

Tools Used

AWS Console: For managing EC2 and IAM roles.
Grafana: For visualization, dashboard creation, and alerting.
Amazon Linux 2: As the operating system for the EC2 instance.
CloudWatch: For monitoring EC2 metrics.
Stress: Command-line tool to simulate load on the instance.
Instance Connect: For connecting to the EC2 instance without SSH.

Step 1 Create your IAM role

In the AWS Console, navigate to Roles and click Create Role.

Select Trusted Entity: AWS service

Service Use Case: EC2

Click Next.

Attach the permission policy: AmazonGrafanaCloudWatchAccess.

Click Next.

Provide a unique name for your role.

Click Create Role.

Step 2 Launch your instance

Launch and install Grafana on an EC2 instance.

In the AWS Console, search for EC2 and click Launch Instance.

Configure your instance with these settings:

Name: Choose a unique name
AMI: Amazon Linux 2
Instance type: t2.micro
Key pair (login): Proceed without a key pair.

Network settings

Firewall (security groups): Create a new security group.
Allow SSH traffic from: com.amazonaws.us-east-1.ec2-instance-connect (replace us-east-1 with your region)
Allow HTTP traffic from the internet: Checked

replace us-east-1 with your region This will allow us to instance connect.

Advanced details

IAM instance profile: Attach the IAM role you created earlier. This is crucial for granting the EC2 instance the necessary permissions to access CloudWatch via Grafana.

User Data

Paste the following script to install and start Grafana:

#!/usr/bin/env bash

# Install Grafana
sudo yum install -y https://dl.grafana.com/enterprise/release/grafana-enterprise-11.5.2-1.x86_64.rpm

# Enable/Start Grafana
sudo systemctl enable grafana-server
sudo systemctl start grafana-server

Click Launch instance to create your EC2 instance.

Connect to your instance using Instance Connect (instead of SSH).

To verify the installation run:


sudo systemctl status grafana-server

You should see Grafana running, similar to the provided screenshot.

Step 3 Connect to Grafana UI

Gain Access to the security group of your EC2 instance.

Edit the inbound rules of your instance’s security group

Type: Custom TCP
Port range: 3000
Source: Custom
CIDR Block: 0.0.0.0/0

Access Grafana by copying your instance's public IP and appending port 3000 (e.g., 54.164.96.183:3000).

You should see a web page like this.

email: admin
password: admin

You can skip the password change prompt if desired.

Step 4 Configure a data source

Grafana needs to be provided a data source to fetch and display data.

In Grafana, navigate to the Data Sources:

Select CloudWatch

Use these settings:

Authentication Provider: AWS SDK default
Default Region: us-east-1
Namespaces of Custom Metrics ec2-monitoring

Click Save & test. You should see a confirmation message indicating that the queries to the CloudWatch metrics and logs APIs were successful.

Build a Dashboard

On the success page, click Build a dashboard > Add visualization

Select your CloudWatch data source.

At the moment "No data" will be shown.

Delete the default query.

Add a new query with these parameters:

Namespace: AWS/EC2
Metric name: CPUUtilization
Statistic: Average

add dimension
Dimensions: InstanceId (select your Grafana instance's ID)

Click Run Queries to view your data.

Congratultions! You should now see your first queried Data from your EC2 instance.

Step 5 Stress test the EC2

We can stress test our EC2 to see check the results on Grafana.

Reconnect to your EC2 instance using Instance Connect if necessary.

Install the stress testing tool:


sudo amazon-linux-extras install epel -y

sudo yum install stress -y

Run the stress test:


stress --cpu 4 --timeout 120s

After the test completes, check your Grafana dashboard to see the updated metrics.

Conclusion

By following these steps, you've successfully deployed Grafana on an EC2 instance, attached the necessary IAM role, and set up monitoring for CPU utilization using CloudWatch. This guide demonstrates a straightforward approach to achieving real-time monitoring with AWS and Grafana, providing essential insights for system performance analysis and stress testing.

Happy monitoring!

Securely Deploy a EKS Cluster in a Private AWS VPC with Terraform & AWS SSM (No Bastion Hosts or SSH Keys Required)

David Hyppolite — Fri, 14 Feb 2025 19:30:52 +0000

In this guide, I demonstrate how to deploy a production-grade Amazon EKS cluster within a private AWS VPC using Terraform and AWS Systems Manager (SSM). This approach enhances security by eliminating the need for SSH keys and bastion hosts, showcasing expertise in infrastructure as code, cloud security, and Kubernetes orchestration.

What You'll Learn

Deploy a production-grade EKS cluster in a private VPC
Implement secure access using AWS Systems Manager
Automate infrastructure with Terraform
Configure networking and security best practices
Deploy a sample FastAPI application

Prerequisites
Introduction
- Why This Approach?
- Critical Access Configuration
Infrastructure Setup and Configuration
- Step 1: GitHub Repository
- Step 2: IAM Role Configuration
- Step 3: Create ECR Repository
- Step 4: Infrastructure Deployment
Deployment and Access Configuration
- Step 6: Creating the EC2 Instance
- Kubernetes Manifests and Deployment
Resource Cleanup
Optional: Implement CI/CD with GitHub Actions

Technologies & Skills

AWS EKS
AWS Systems Manager (SSM)
Terraform
FastAPI
Amazon ECR
Infrastructure as Code (IaC)
Cloud Security
Kubernetes Orchestration

Prerequisites

Basic understanding of AWS networking concepts (VPCs, subnets, routing)
AWS Account with appropriate permissions
AWS CLI installed and configured
kubectl installed
Terraform installed
Docker installed

Introduction

Deploying an EKS cluster in a private VPC ensures that your Kubernetes API server is not exposed to the public internet, enhancing security. However, this setup introduces challenges in managing access and automating infrastructure provisioning. By leveraging Terraform and AWS SSM, you can automate the entire deployment process, manage IAM roles effectively, and securely access your cluster without exposing it publicly.

Why This Approach?

Enhanced Security: Deploy EKS clusters in private subnets without exposing SSH ports
Simplified Access: Use IAM roles instead of managing SSH keys
Cost Effective: Eliminate the need for dedicated bastion hosts
Automation Ready: Infrastructure as Code with Terraform
Production Grade: Suitable for enterprise environments

Critical Access Configuration ⚠️

Important: Two key points about access management:

The IAM role EC2RoleForSSM_EKS (or similar) must exist and have the necessary permissions for both SSM and EKS interactions. Without this, you will be unable to access the cluster from within the private VPC.
When an EKS cluster is created, only the user/role that creates the cluster (e.g., Terraform) automatically receives system:master permissions. All other users or roles requiring cluster access must be explicitly added to the cluster's RBAC configuration.

Missing either of these configurations will prevent access to resources in the private VPC, whether through SSM or via the CLI.

Key Takeaways

Mastered deploying EKS in a secure, private environment
Implemented infrastructure automation with Terraform
Enhanced security using AWS Systems Manager (SSM)
Demonstrated proficiency in Kubernetes and cloud networking

Infrastructure Setup and Configuration

Step 1: GitHub Repository

Start by organizing your project repository to manage your FastAPI application, Terraform configurations, and deployment scripts.

You can use your own repository or follow along with mine:

Fork this repository: Github

Repo Contains:


Repository Structure:
├── main.py        # FastApi application
├── Dockerfile        # Docker
├── terraform/         # terraform module
├── fastapi-deploy.yaml           # service scripts
├── fastapi-service.yaml           # service scripts
└── requirements.txt

Step 2: IAM Role Configuration

First, we'll create the necessary IAM role for Systems Manager access:

In the AWS console search up Roles > Create role

Trusted Entity: EC2 > EC2 use case.

Click Next

Add Permissions:

AmazonSSMManagedInstanceCore
AmazonEKSWorkerNodePolicy
AmazonEKSServicePolicy
AmazonEKSClusterPolicy

Click Next

Provide a name for the role.

EC2RoleForSSM_EKS

Review your settings then click Create role.

Step 3: Create ECR repository

To store container image for the FastAPI application, we'll be using Amazon Elastic Container Registry(ECR). While the infrastructure for VPC and EKS will be automated with Terraform, we'll create the ECR repository manually via the AWS console.

In the AWS Console search ECR > Create

Set the repository name: fastapi-app
Leave the default settings
click Create

View Repository Commands:

After creation select the repo
Click View push commands to get the commands to build and push your Docker image to ECR. Follow the commands.
Note: These command are what you use to build and push your local Docker image to your ECR container to hold.

The ECR repo stores the Docker image, which will later be pulled by the EKS cluster during deployment.

Once your image is pushed let's move on to setting up our VPC and EKS using Terraform.

Step 4: Infrastructure Deployment

Automate the creation of a private VPC, EKS cluster, and node groups using Terraform.

The Terraform configuration creates three key modules:

VPC Module: Provisions a private VPC with public and private subnets.
Security Group Module: Configures security groups for ALB and EKS communication.
EKS Module: Deploys the EKS cluster and worker nodes in private subnets.

Below are snippets for the complete Terraform configuration. The full code can be found in the GitHub repository: Repo

Set up the VPC

First, we'll set up our VPC with public and private subnets. The public subnets will host our NAT Gateway, while private subnets will contain our EKS nodes.

Terraform Configuration:

data "aws_availability_zones" "available" {
  state = "available"
}

# Create vpc 
resource "aws_vpc" "main" {
    cidr_block = var.vpc_cidr
    enable_dns_support = true
    enable_dns_hostnames = true

    tags = {
      Name = "${var.project_name}-vpc"
    }

}

# Create public subnets
resource "aws_subnet" "public" {
  count = length(var.azs)      
  vpc_id     = aws_vpc.main.id
  cidr_block = "10.0.${count.index + 1}.0/24"
  availability_zone = var.azs[count.index]

  map_public_ip_on_launch = true

  tags = {
    Name = "${var.project_name}-public-${count.index + 1}"
  }
}

# internet gateway for public subnets
resource "aws_internet_gateway" "main" {
    vpc_id = aws_vpc.main.id

    tags = {
        Name = "${var.project_name}-igw"}
}

# route table for public subnets
resource "aws_route_table" "public" {
  vpc_id = aws_vpc.main.id

  route {
    cidr_block = "0.0.0.0/0"
    gateway_id = aws_internet_gateway.main.id
  }

  tags = {
    Name = "${var.project_name}-public-rt"}
}

# 
resource "aws_route_table_association" "public" {
  count = length(var.azs)  
  subnet_id      = aws_subnet.public[count.index].id
  route_table_id = aws_route_table.public.id
}

Private subnets


# create private subnets
resource "aws_subnet" "private" {
  count = length(var.azs)      
  vpc_id     = aws_vpc.main.id
  cidr_block = "10.0.${count.index + 10}.0/24"
  availability_zone = var.azs[count.index]

  tags = {
    Name = "${var.project_name}-private-${count.index + 1}"
  }
}

resource "aws_eip" "nat" {
  domain = "vpc"

  tags = {
    Name = "${var.project_name}-nat-eip"
    }
}

resource "aws_nat_gateway" "main" {
  allocation_id = aws_eip.nat.id
  subnet_id = aws_subnet.public[0].id

  tags = {
    Name = "${var.project_name}-ngw"
    }

    depends_on = [ aws_internet_gateway.main ]
}

# route table for private subnets
resource "aws_route_table" "private" {
  vpc_id = aws_vpc.main.id

  route {
    cidr_block = "0.0.0.0/0"
    nat_gateway_id = aws_nat_gateway.main.id
  }

  tags = {
    Name = "${var.project_name}-private-rt"}
}

resource "aws_route_table_association" "private" {
  count = length(var.azs)  
  subnet_id      = aws_subnet.private[count.index].id
  route_table_id = aws_route_table.private.id
}

Note: The NAT Gateway and Internet Gateway ensure proper routing between private subnets and external services.

Security Group Configuration

These security groups control access to our EKS cluster and load balancer:

ALB Security Group:

Allows inbound traffic on port 80 from the internet.
Permits unrestricted outbound traffic.

EKS Security Group:

Allows inbound traffic from the ALB for internal communication.

Terraform Configuration:


# ALB Security group
resource "aws_security_group" "alb" {
  name = "${var.project_name}-alb-sg"
  vpc_id = var.vpc_id

  tags = {
    Name = "${var.project_name}-eks-alb"
  }
}

# EKS Secuirty group
resource "aws_security_group" "eks" {
  name = "${var.project_name}-eks-sg"
  vpc_id = var.vpc_id

  tags = {
    Name = "${var.project_name}-eks-sg"
  }
}

# Allow ALB to communicate with EKS

resource "aws_security_group_rule" "allow_alb" {
  type = "ingress"
  security_group_id = aws_security_group.eks.id
  source_security_group_id = aws_security_group.alb.id
  from_port         = 80
  protocol       = "tcp"
  to_port           = 80

}

# Allow EKS to outbound traffic
resource "aws_security_group_rule" "allow_eks_egress" {
  type = "egress"
  security_group_id = aws_security_group.eks.id
  from_port         = 0
  protocol       = "-1"
  to_port           = 0
  cidr_blocks = [ "0.0.0.0/0" ]
}

Tip: Always define strict ingress and egress rules to maintain security.

EKS Cluster Configuration

The EKS cluster is deployed in private subnets, and the ALB will forward traffic to the worker nodes. We let AWS manage the security up for cluster and node group creation this reduces the need for manual intervention.

Terraform Configuration:

# Create the EKS Cluster
resource "aws_eks_cluster" "main" {
  name     = "fastAPI-cluster"
  role_arn = aws_iam_role.cluster.arn
  version  = "1.31"

  vpc_config {
    endpoint_private_access = true
    endpoint_public_access  = false
    subnet_ids              = var.private_subnet_ids
  }

  depends_on = [
    aws_iam_role_policy_attachment.cluster_AmazonEKSClusterPolicy,
  ]
}

# Create EKS Node Group
resource "aws_eks_node_group" "main_node" {
  cluster_name    = aws_eks_cluster.main.name
  node_group_name = "fastAPI-node"
  node_role_arn   = aws_iam_role.node_group.arn
  subnet_ids      = var.private_subnet_ids

  scaling_config {
    desired_size = 3
    max_size     = 6
    min_size     = 3
  }

  remote_access {
    source_security_group_ids = [ var.eks_sg_id ]
  }

  instance_types = [ "t2.micro" ]

  depends_on = [
    aws_iam_role_policy_attachment.node_AmazonEKSWorkerNodePolicy,
    aws_iam_role_policy_attachment.node_AmazonEKS_CNI_Policy,
aws_iam_role_policy_attachment.node_AmazonEC2ContainerRegistryReadOnly,
  ]

EKS Access Entries and Policy Associations

  data "aws_iam_role" "console_role" {
  name = "EC2RoleForSSM_EKS" # Replace for with your roles name
}

  # Map IAM Role to Kubernetes Group using EKS Access Entry
  resource "aws_eks_access_entry" "additional_access" {
  cluster_name      = aws_eks_cluster.main.name
  principal_arn     = data.aws_iam_role.console_role.arn
  kubernetes_groups = ["eks:admin"] 
  type              = "STANDARD"
}

# # Associate Access Policy with Access Entry
resource "aws_eks_access_policy_association" "AdminPolicy" {
  cluster_name  = aws_eks_cluster.main.name
  policy_arn    = "arn:aws:eks::aws:cluster-access-policy/AmazonEKSAdminPolicy"
  principal_arn = data.aws_iam_role.console_role.arn

  access_scope {
    type       = "cluster"
  }
}

}

In the EKS access control replace your name with your create role name

Applying Terraform Configuration

Run the following commands to deploy your infrastructure:


terraform init

terraform validate

terraform plan

terraform apply -auto-approve

Deployment and Access Configuration

⚠️ Prerequisites Check: Before proceeding, verify that the IAM role 'EC2RoleForSSM_EKS' or similar exists and has the correct permissions for both SSM and EKS. Remember that only the cluster creator has default access - all other users/roles need explicit RBAC configuration. Missing these steps will prevent cluster access.

Step 6. Creating the EC2 Instance for Cluster Access

We'll create an EC2 instance in our private subnet that will serve as our secure access point to the EKS cluster.

In the AWS console search up EC2 > launch Instance

Launch EC2 Instance

Name: Give a unique name
AMI: Amazon Linux 2023
Instance Type: t2.micro
Key pair (login): Proceed without a key pair.
Network settings: Click edit

VPC: Select your custom vpc managed by terraform
Subnet: Select your private subnet managed by terraform
Auto-assign public IP: disabled
Firewall (security groups): Create security group(make a note of name)

Remove all inbound rules to ensure the instance is accessible only via SSM.

Under Advanced details:

IAM instance profile: EC2RoleForSSM_EKS

User Data Script

Include this script in the EC2 instance user data:


#!/usr/bin/env bash

yum -y upgrade
yum -y update 

# install kubectl to interact with the private EKS cluster.
curl -O https://s3.us-west-2.amazonaws.com/amazon-eks/1.31.2/2024-11-15/bin/linux/amd64/kubectl

chmod +x ./kubectl

sudo mv ./kubectl /usr/local/bin/kubectl

Click Launch instance

Select your new instance.

Select the Security tab.

Make a Note of the Security Group ID

Allow EC2 access to control plane security group

In the AWS console search VPC > Security Groups

Search and select the control plane security group "eks-cluster-sg-fastAPI-cluster-XXXXXX".

look for a description "EKS created security group applied to ENI that is attached to EKS Control Plane master nodes, as well as any managed workloads."

Click Edit inbound rules

Add a new rule:

Type: HTTPS
Protocol: TCP
Port Range: 443
Source: Select "Custom" and enter the Security Group ID of your EC2 instance (e.g., sg-xxxxxx).

Click Save rules to apply the changes.

Connecting via Systems Manager

In the AWS Console Search "Systems Manager"

Under Node Tools Click Session Manager > Start Session

Select the Instance you created.

Click Start session.

sudo su

To gain root access

Verify Kubectl is installed:


kubectl version --client

Configure kubectl to connect to the EKS cluster:

aws eks --region <region> update-kubeconfig --name <cluster-name>

I used:

aws eks --region us-east-1 update-kubeconfig --name fastAPI-cluster

Kubernetes Manifests and Deployment

Understanding the Manifests

Our application deployment requires two Kubernetes manifests:

A Deployment manifest to manage our FastAPI application pods
A Service manifest to expose our application through a load balancer

Deployment Manifest

The Deployment manifest (fastapi-deploy.yaml) defines how our application should run:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: fastapi-deployment
  labels:
    app: fastapi
spec:
  replicas: 3
  selector:
    matchLabels:
      app: fastapi
  template:
    metadata:
      labels:
        app: fastapi
    spec:
      containers:
      - name: fastapi
        image: ACCOUNT_ID.dkr.ecr.REGION.amazonaws.com/fastapi-app:latest
        ports:
        - containerPort: 80

Key components:

replicas: 3: Maintains three running instances of our application
app: fastapi: Label used to identify our application pods
image: References our container in ECR (replace with your ECR URI)

Example ECR URI format:

123456789012.dkr.ecr.us-east-1.amazonaws.com/fastapi-app:latest

Service Manifest

The Service manifest (fastapi-service.yaml) configures how to access our application:

apiVersion: v1
kind: Service
metadata:
  name: fastapi-service
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-scheme: "internet-facing"
spec:
  type: LoadBalancer
  selector:
    app: fastapi
  ports:
    - port: 80
      targetPort: 80
      protocol: TCP

Key components:

aws-load-balancer-scheme: "internet-facing": Creates an public facing load balancer
type: LoadBalancer: Provisions an AWS Load Balancer
selector: app: fastapi: Connects to pods with matching labels

Deploying the Manifests

1. Create the deployment file

nano fastapi-deploy.yaml

Paste the deployment manifest content.

2. Create the service file

nano fastapi-service.yaml

Paste the service manifest content.

3. Apply the manifests

kubectl apply -f fastapi-deploy.yaml
kubectl apply -f fastapi-service.yaml

4. Verify the deployment

# Check pods status
kubectl get pods

# Check service and load balancer
kubectl get services

Expected outputs:

# Pods output
NAME                                  READY   STATUS    RESTARTS   
fastapi-deployment-xxxxxxxxxx-xxxx   1/1     Running   0          
fastapi-deployment-xxxxxxxxxx-xxxx   1/1     Running   0          
fastapi-deployment-xxxxxxxxxx-xxxx   1/1     Running   0          

# Services output
NAME              TYPE           CLUSTER-IP      EXTERNAL-IP                                    
fastapi-service   LoadBalancer   XXX.XXX.XXX.XX   internal-xxxxx.us-east-1.elb.amazonaws.com

Troubleshooting Tips

If pods aren't starting, check the logs: kubectl logs <pod-name>
For service issues: kubectl describe service fastapi-service
To verify ECR access: kubectl describe pod <pod-name>

Access the application using the IP

Resource Cleanup

Clean Kubernetes Resources:

kubectl delete --all deployments
kubectl delete --all services
kubectl delete --all pods

AWS Console Cleanup:

Remove EC2 inbound rule from Kubernetes control plane security group
Terminate EC2 instance
Delete ECR repository if no longer needed

Infrastructure Cleanup:

terraform destroy -auto-approve

If you run into terraform resources not being deleted. Delete through console.

📝 Conclusion

Deploying an EKS cluster within a private VPC enhances security by restricting API access. By leveraging Terraform and AWS Systems Manager, you can automate the provisioning process, manage IAM roles effectively, and securely access your cluster without exposing it to the public internet. This approach not only simplifies infrastructure management but also adheres to best practices for security and scalability in cloud environments.

Best Practices Implemented:

Private VPC deployment for enhanced security
IAM role-based access control
Systems Manager for secure instance access
Infrastructure as Code for consistency
Containerized application deployment

🟢(Optional) Implement CI/CD with GitHub Actions

To further streamline your deployment process, integrating a CI/CD pipeline can automate the building and deployment of your application. While this blog focuses on setting up the EKS cluster and deploying the application manually, you can enhance your workflow by implementing CI/CD using GitHub Actions check out my blog post here that covers thatBuild a Secure CI/CD Pipeline for Amazon EKS Using GitHub Actions and AWS OIDC.

Build a Secure CI/CD Pipeline for Amazon EKS Using GitHub Actions and AWS OIDC

David Hyppolite — Wed, 29 Jan 2025 01:37:19 +0000

In this guide, I’ll show you how to build a secure AWS EKS(Kubernetes) CI/CD pipeline for your FastAPI app complete with GitHub Actions, Docker, and OpenID Connect (OIDC) all while following AWS security best practices like least-privilege IAM policies. We won’t obsess over every detail of FastAPI or Docker, but we’ll cover enough to get your application running on EKS with confidence.

Note: The primary focus of this blog is on CI/CD processes and AWS configuration, so we won’t dive too deeply into FastAPI or Docker fundamentals.

Prerequisites
Step 1: GitHub Repository
Step 2: Create an Amazon ECR Repository
Step 3: Create an Amazon EKS Cluster with Eksctl
Step 4: Set Up a GitHub Actions Workflow for EKS Deployment
Step 5: Github Actions Workflow
Clean Up Resources
Conclusion

Technologies & Skills

AWS EKS (Kubernetes on AWS)
FastAPI
Amazon ECR
Cloud Security & IAM
Kubernetes Orchestration

Prerequisites

AWS Account: Admin permissions for creating EKS clusters, IAM roles, and ECR repositories.
AWS CLI installed and configured: Install Guide
eksctl installed: Installation Guide
Docker installed: Get Started with Docker

Introduction

Managing Kubernetes on AWS via EKS is a great approach, but configuring secure access for CI/CD can be tricky. With OIDC, GitHub Actions can assume roles in AWS without storing secret keys, so no static credentials, Let’s walk through setting up an EKS cluster, creating an ECR repo, and configuring GitHub Actions to deploy our FastAPI service to Kubernetes automatically.

This guide walks through:

Setting up an EKS cluster with eksctl
Creating an ECR repository for your Docker images
Configuring OIDC so GitHub Actions can assume an IAM role
Deploying the FastAPI application to EKS

Key Takeaways

Security-First CI/CD: By using OIDC, there’s no need to embed AWS credentials in GitHub.
Least-Privilege Access: You’ll attach minimal IAM permissions, restricted to your ECR repo or EKS cluster.
Easy Automation: GitHub Actions automatically builds, pushes, and deploys whenever you push code to your main branch.

Step-by-Step Guide

Step 1: GitHub Repository

Create (or use) a repository that holds your FastAPI app and Kubernetes manifests:


Repository Structure:
├── main.py        # FastApi application
├── Dockerfile        # Docker
├── fastapi-deploy.yaml           # deploy scripts
├── fastapi-service.yaml           # service scripts
└── requirements.txt

Keep your code organized. The .yaml files will be applied to EKS to deploy the FastAPI service.

Step 2: Create an Amazon ECR Repository

Amazon ECR is used to store your Docker images.

Go to AWS Console > Search ECR > Create Repository
Name it (e.g., fastapi-app) or anything you prefer
Leave defaults, then Create

Set the repository name: fastapi-app
Leave the default settings
click Create

View Repository Commands:

After creation select the repo
Click View push commands on your new repo to see the exact Docker build/push steps.

Retrieve the ECR ARN for use in IAM policies:

aws ecr describe-repositories --region us-east-1

This is important if you want to create inline IAM policies restricting actions to this specific repository.

Step 3: Create an Amazon EKS Cluster with Eksctl

Make sure you have the AWS CLI and Eksctl installed on your device to create the EKS cluster.
Using eksctl, create the EKS cluster:

eksctl create cluster --name cluster-name \
                      --region <YOUR_REGION> \
                      --node-type instance-type \
                      --nodes-min 2 \
                      --nodes-max 6

I used.

eksctl create cluster --name fastapi-demo \
                      --region us-east-1 \
                      --node-type t3.medium \
                      --nodes-min 2 \
                      --nodes-max 6

This will take several minutes. Once finished, your kubeconfig will be updated so you can run kubectl commands against the new cluster.

Verify the cluster:

kubectl get nodes

Step 4: Set Up a GitHub Actions Workflow for EKS Deployment

To avoid storing permanent AWS credentials in GitHub, we’ll configure OpenID Connect (OIDC) so GitHub Actions can assume roles in AWS. Guthub Action in AWS

If you have previously created a Identity provider for GitHub skip this step if not create one:

Add a New OIDC Provider

Go to the AWS Console > IAM > Identity Providers > Add provider.

Provider type: OpenID Connect
Provider Url: https://token.actions.githubusercontent.com
Audience: sts.amazonaws.com

Next Add provider.

Assign a role

Copy the Identity provider role arn.

Select the newly your newly created provider on the Identity providers screen

Click Assign a role

Role options: Create a new role

Trusted entity type: Custom trust policy

Click next

GitHub organization , the repository named , and the branch named . Update the Federated ARN with the GitHub IdP ARN that you copied previously.

Replace ,, and with your AWS account ID, GitHub repository owner, repository name, and branch name.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "arn:aws:iam::<AWS_ACCOUNT_ID>:oidc-provider/token.actions.githubusercontent.com"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "token.actions.githubusercontent.com:aud": "sts.amazonaws.com"
        },
        "StringLike": {
          "token.actions.githubusercontent.com:sub": "repo:<OWNER>/<REPO>:ref:refs/heads/<BRANCH>"
        }
      }
    }
  ]
}

Than click Next

Attach the policies below:

AmazonEKSClusterPolicy
AmazonEKSWorkerNodePolicy

Than click Next

Name the role, e.g. GitHubActionsEKSECRRole

Next click Create Role

Access the newly created role.

Add Inline Policy for ECR:

Add Permissions > Create Inline policy > Use the JSON editor

Copy the code below to restrict to you repository ARN, as shown below.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "ecr:GetAuthorizationToken"
      ],
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "ecr:BatchCheckLayerAvailability",
        "ecr:GetDownloadUrlForLayer",
        "ecr:GetRepositoryPolicy",
        "ecr:DescribeRepositories",
        "ecr:ListImages",
        "ecr:DescribeImages",
        "ecr:BatchGetImage",
        "ecr:GetLifecyclePolicy",
        "ecr:GetLifecyclePolicyPreview",
        "ecr:ListTagsForResource",
        "ecr:DescribeImageScanFindings",
        "ecr:InitiateLayerUpload",
        "ecr:UploadLayerPart",
        "ecr:CompleteLayerUpload",
        "ecr:PutImage"
      ],
      "Resource": "arn:aws:ecr:<AWS-REGION>:<ACCOUNT-ID>:repository/<REPO-NAME>"
    }
  ]
}

Give your role policy a name; GitHubActionsEKSECRRole

Click Next

Instead of editing aws-auth ConfigMap, you can give this role direct EKS access:

Run the following code in your CLI.

Replace cluster-name with the name of your cluster.

"aws eks create-access-entry --cluster-name <cluster-name> --principal-arn arn:aws:iam::<AWS_ACCOUNT_ID>:role/GitHubActionsEKSECRRole --type STANDARD"

Then associate the policy that allows cluster admin actions:

aws eks associate-access-policy --cluster-name <cluster-name> --principal-arn arn:aws:iam::<AWS_ACCOUNT_ID>:role/GitHubActionsEKSECRRole \
  --access-scope type=cluster --policy-arn arn:aws:eks::aws:cluster-access-policy/AmazonEKSClusterAdminPolicy

You can verify your entry:

aws eks list-access-entries --cluster-name fastapi-demo

Step 5: Github actions Workflow

Finally, set up your GitHub Actions workflow to build, push, and deploy automatically.

In your GitHub repo, navigate to Actions > New Workflow.

Copy the code below and replace with your information. Anything that is full capitilized and begins with a "$" is sensitive information that will not be easily shown in your code.

name: Deploy to AWS

on:
  workflow_dispatch:  
  push:
    branches:
      - main

jobs:
  deploy:
    runs-on: ubuntu-latest

    permissions:
      id-token: write
      contents: read

    steps:
      - name: Checkout code
        uses: actions/checkout@v4.2.2

      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: ${{ secrets.AWS_ROLE_ARN }}
          role-session-name: GitHubActionsSession
          aws-region: ${{ secrets.AWS_REGION }}

      - name: Login to Amazon ECR
        id: login-ecr
        uses: aws-actions/amazon-ecr-login@v2

      - name: Build, tag, and push docker image to Amazon ECR
        env:
          REGISTRY: ${{ steps.login-ecr.outputs.registry }}
          REPOSITORY: fastapi-app # your ECR repository name
          IMAGE_TAG: ${{ github.sha }}
        run: |
          docker build -t $REGISTRY/$REPOSITORY:$IMAGE_TAG .
          docker push $REGISTRY/$REPOSITORY:$IMAGE_TAG

      - name: Update kube config
        run: aws eks update-kubeconfig --name ${{ secrets.EKS_CLUSTER_NAME }} --region ${{ secrets.AWS_REGION }}

      - name: Deploy to EKS
        env:
          ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}        
          IMAGE_TAG: ${{ github.sha }}
          EKS_CLUSTER_NAME: ${{ secrets.EKS_CLUSTER_NAME }}
        run: |
          sed -i.bak "s|DOCKER_IMAGE|$ECR_REGISTRY/$REPOSITORY:$IMAGE_TAG|g" fastapi-deploy.yaml
          kubectl apply -f fastapi-deploy.yaml
          kubectl apply -f fastapi-service.yaml

Understanding Our GitHub Actions Workflow for AWS Deployment

Let's walk through our CI/CD pipeline that automates deploying applications to AWS using GitHub Actions. This workflow handles everything from building Docker images to deploying on EKS, all while maintaining security best practices.

Workflow Triggers and Permissions

name: Deploy to AWS
on:
  workflow_dispatch:  
  push:
    branches:
      - main

Our pipeline springs into action in two scenarios:

When code is pushed to the main branch
Manually through workflow_dispatch (useful for testing or one-off deployments)

The workflow needs specific permissions to interact with AWS securely:

permissions:
  id-token: write
  contents: read

These permissions enable OIDC authentication with AWS while keeping access to our GitHub repository read-only - following the principle of least privilege.

Secure AWS Authentication

- name: Configure AWS credentials
  uses: aws-actions/configure-aws-credentials@v4
  with:
    role-to-assume: ${{ secrets.AWS_ROLE_ARN }}
    role-session-name: GitHubActionsSession
    aws-region: ${{ secrets.AWS_REGION }}

This step establishes secure communication with AWS using OIDC. Instead of storing long-lived credentials, we assume an IAM role temporarily. This role grants just enough permissions to:

Push images to ECR
Deploy to our EKS cluster

Container Image Management

- name: Login to Amazon ECR
  id: login-ecr
  uses: aws-actions/amazon-ecr-login@v2

- name: Build, tag, and push docker image to Amazon ECR
  env:
    REGISTRY: ${{ steps.login-ecr.outputs.registry }}
    REPOSITORY: fastapi-app
    IMAGE_TAG: ${{ github.sha }}
  run: |
    docker build -t $REGISTRY/$REPOSITORY:$IMAGE_TAG .
    docker push $REGISTRY/$REPOSITORY:$IMAGE_TAG

Here's where we handle our Docker image:

First, we authenticate with Amazon ECR
Then we build the image using our Dockerfile
We tag it with the git commit SHA for version tracking
Finally, we push it to our ECR repository

EKS Deployment

- name: Update kube config
  run: aws eks update-kubeconfig --name ${{ secrets.EKS_CLUSTER_NAME }} --region ${{ secrets.AWS_REGION }}

- name: Deploy to EKS
  env:
    ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}        
    IMAGE_TAG: ${{ github.sha }}
    EKS_CLUSTER_NAME: ${{ secrets.EKS_CLUSTER_NAME }}
  run: |
    sed -i.bak "s|DOCKER_IMAGE|$ECR_REGISTRY/$REPOSITORY:$IMAGE_TAG|g" fastapi-deploy.yaml
    kubectl apply -f fastapi-deploy.yaml
    kubectl apply -f fastapi-service.yaml

The final stage deploys our application to EKS:

We configure kubectl to communicate with our EKS cluster
Update our Kubernetes manifests with the new image details
Apply both the deployment and service configurations

Security Considerations

All sensitive values are stored as GitHub secrets
OIDC provides secure, temporary AWS credentials
The workflow uses specific, versioned actions to prevent supply chain attacks
Each step follows the principle of least privilege

Remember to replace the placeholder values (anything starting with $) with your specific information when setting up this workflow.n

Github Secrets

Store your AWS account details as secrets:
It is good practice to not hardcode sensitive information but we still need to use it to save sensitive infornation we will be using GitHub secrets.

Go to your repo > Settings → Secrets & Variables > Actions to add them.
You're going to create a New Repository Secret.

Your entries will look like this.

Add entries for:

AWS_REGION

AWS_ROLE_ARN # Role created

EKS_CLUSTER_NAME

Push Code and Access Cluster

You can monitor the progress of your CI/CD pipeline by visiting the “GitHub Actions” tab in your repository. Here, you’ll see the status of each step in your workflow.

Here is what a successful deployment looks like.

IN the AWS console got to EC2 > Load balancer

Copy the DNS name and go to port 80.

Clean up resources

When you’re done, it’s best to delete unneeded resources to avoid costs:

eksctl delete cluster --name=fastapi-demo

Also, remove or delete the ECR repository if you no longer need it.

Conclusion

Congratulations! You’ve created a fully automated pipeline that deploys a FastAPI app to EKS, all without embedding long-lived AWS credentials in GitHub. By pairing OIDC with carefully scoped IAM policies, you maintain strong security practices while enjoying a streamlined deployment process.

Understanding Kubernetes Manifests: A Beginner’s Guide to Deployments and Services

David Hyppolite — Mon, 13 Jan 2025 22:21:35 +0000

When working with Kubernetes, you will have to use manifests to configure your workloads. I wrote this guide to help break down the two most basic k8s YAML files that you'll need. Kubernetes manifests serve as instruction manuals that define resources like Pods, Services, and Deployments, written in YAML format.

Tip: YAML is strict with indentation—use an online validator to avoid syntax errors.

Github with full examples here.

Key Kubernetes Manifests

Two common manifests in a Kubernetes cluster are:

Deployment: Manages Pods and ensures high availability.
Service: Exposes applications and manages networking.

Parts of a Kubernetes Configuration

There are 3 parts to a Kubernetes configuration:

Metadata: Where you define the name and labels for your resource
Spec (Specification): The actual configuration of what you want, specific to each resource type
Status: Automatically generated by Kubernetes to reflect the current state of the resource.

Metadata

Metadata includes essential information about the resource, such as its name and labels. Labels are key-value pairs that help categorize and select Kubernetes resources. They enable flexible organization, such as grouping resources by environment (e.g., env: production), tier (e.g., tier: backend), or other custom criteria.

metadata:
  name: nginx-deployment  # Name of the Deployment
  labels:
    app: nginx           # Labels to identify the application

Spec (Specification)

The spec section defines the desired state of the resource. For example, in a Deployment, the spec includes the number of replicas, the selector for matching Pods, and the template for Pod configuration.

spec:
  replicas: 2              # Number of pod replicas for high availability
  selector:
    matchLabels:
      app: nginx           # Must match the labels in the template
  template:
    metadata:
      labels:
        app: nginx         # Labels applied to the pods
    spec:
      containers:
      - name: nginx         # Name of the container
        image: nginx:1.14.2 # Replace with the URL of your container image
        ports:
        - containerPort: 80 # Port the container listens on

Status

The status section is automatically generated by Kubernetes. It reflects the current state of the resource, such as the number of available replicas in a Deployment. Kubernetes continuously monitors the actual state against the desired state and takes action to reconcile any differences, leveraging its self-healing capabilities.

For example, in our deployment manifest we specified we wanted 2 replicas:

spec:
  replicas: 2

When the deployment is created, Kubernetes will continuously check the status to make sure we have 2 replicas running at all times. If only 1 replica(actual state) is running, it's not at the desired state.

Full YAML Example

Deployment:

A Deployment manages a set of identical Pods, ensuring that the specified number of replicas are running at all times. This ensures high availability and load distribution across your application.

apiVersion: apps/v1
kind: Deployment # Create a deployment resource
metadata:
  name: nginx-deployment # Name of the Deployment
  labels:
    app: nginx # Labels to identify the application
spec:
  replicas: 2 # Number of pod replicas for high availability
  selector:
    matchLabels:
      app: nginx # Must match the labels in the template
  template:
    metadata:
      labels:
        app: nginx # Labels applied to the pods
    spec:
      containers:
      - name: nginx # Name of the container
        image: nginx:1.14.2 # Replace with the URL of your container image
        ports:
        - containerPort: 80 # Port the container listens on

Service:

A Service exposes your application to the network, managing how traffic is directed to the Pods. By setting the Service type to LoadBalancer, Kubernetes provisions an external load balancer (if supported by your cloud provider) to route traffic to your Service, allowing users to access your application via a public IP address.

apiVersion: v1
kind: Service # Create a service resource
metadata:
  name: nginx-service # Name of the service
spec:
  type: LoadBalancer # Exposes the service via a cloud provider's load balancer
  selector:
    app: nginx # Links this service to Pods with the label app: fastapi
  ports:
    - port: 80 # Exposed port for external traffic
      targetPort: 80 # Port on the container to route traffic to
      protocol: TCP # Communication protocol

Resource Declaration (First Two Lines)

The first two line in a Kubernetes yaml manifest is where you declare to Kubernetes the resource type you want to create:

Service

apiVersion: v1
kind: Service # Create a service resource

Deployment

apiVersion: apps/v1
kind: Deployment # Create a deployment resource

Connecting Services to deployments

Service and deployments are connected through selectors and labels. This connection ensures that traffic directed to the Service is properly routed to the appropriate Pods managed by the Deployment.

In the deployment file:

metadata:
  name: nginx-deployment 
  labels:
    app: nginx # Labels to identify the application

spec:
  replicas: 2 
  selector:
    matchLabels:
      app: nginx # Must match the labels in the template

  template:
    metadata:
      labels:
        app: nginx # Labels applied to the pods

Service File

labels:
    app: nginx # Labels to identify the application

How It Works

Labels: In the Deployment file, the metadata.labels and template.metadata.labels sections assign the label app: nginx to the Deployment and its Pods. Labels are key-value pairs that help categorize and select Kubernetes resources.
Selectors: In the Deployment's spec.selector.matchLabels and the Service's spec.selector, the label app: nginx is used to match and connect the Service to the appropriate Pods.
Traffic Routing: The Service uses the selector to identify which Pods to send traffic to. When external traffic arrives at the Service's port, it is forwarded to the targetPort on the matched Pods.

Port Definitions

  ports:
    - port: 80 # Exposed port for external traffic
      targetPort: 80 # Port on the container to route traffic to
      protocol: TCP # Communication protocol

Think of it this way:

External traffic comes to the Service on 'port'
The Service forwards that traffic to 'targetPort' on your pods
Your container needs to be listening on 'targetPort'

These ports must align properly:

The Service's "targetPort" must match the "containerPort" in your Deployment
The Service's "port" can be different, but is commonly set to the same number for simplicity

Enhancing Clarity with Detailed Explanations

Metadata

Metadata provides essential information about your Kubernetes resources. Proper labeling is crucial for organizing and managing resources efficiently. Labels allow you to group resources based on various criteria, such as environment (env: production), application tier (tier: backend), or custom categories (app: nginx).

Spec (Specification)

The spec section is the core of your manifest, defining the desired state of the resource. For Deployments, key spec fields include:

Replicas: Specifies the number of Pod instances to run. For example, replicas: 2 ensures that two Pods are always running, providing high availability.

replicas: 2

Selector: Determines how the Deployment identifies which Pods to manage. It must match the labels defined in the Pod template.

selector:
  matchLabels:
    app: nginx

Template: Defines the Pod's metadata and specifications, including labels, containers, and ports.

template:
  metadata:
    labels:
      app: nginx
  spec:
    containers:
    - name: nginx
      image: nginx:1.14.2
      ports:
      - containerPort: 80

Status

The status section is managed by Kubernetes and provides real-time information about the resource's current state. It helps Kubernetes track whether the actual state matches the desired state defined in the spec. If discrepancies are detected, Kubernetes takes corrective actions, such as restarting Pods to maintain the desired number of replicas.

Example of Self-Healing:

If a Pod crashes or becomes unhealthy, Kubernetes will detect that the actual number of running Pods does not match the desired replicas specified in the Deployment. It will then automatically create a new Pod to replace the failed one, ensuring that the application remains available.

Understanding the Big Picture

Now that we've broken down each component of Kubernetes manifests, you can see how they work together to create a robust and self-healing application infrastructure. The combination of proper metadata, well-defined specs, and Kubernetes' built-in status monitoring ensures your applications run reliably and efficiently.

Final Thoughts

After reading this guide, you should have a solid foundation for working with Kubernetes manifests. You now understand how to read, structure, and use both Deployment and Service configurations to manage and deploy applications within your Kubernetes cluster. The examples and explanations provided here will serve as a practical reference as you begin your Kubernetes journey.

Ready to dive deeper? Check out Kubernetes Official Documentation for more in-depth information and advanced configurations.

Secure EC2 Access Without SSH Using AWS Systems Manager

David Hyppolite — Wed, 08 Jan 2025 14:54:12 +0000

Need to access your EC2 instances securely without managing SSH keys? AWS Systems Manager (SSM) Session Manager provides a secure, auditable solution that might be exactly what you're looking for. Let's explore how it works.

While SSH is secure, managing SSH access at scale across multiple VPCs and regions creates significant operational complexity. Organizations often struggle with key management, rotation policies, and maintaining bastion hosts - all of which increase both security risks and operational overhead.

The Challenge

Consider a common scenario: You're managing EC2 instances across multiple VPCs and regions, with strict security policies prohibiting open SSH ports. Traditional approaches might require maintaining multiple bastion hosts - even a single t2.micro bastion host running 24/7 adds unnecessary cost and complexity. SSM eliminates this overhead while providing secure, auditable access to your instances.

Enter Systems Manager

AWS Systems Manager (SSM) Session Manager offers a compelling alternative, especially when working with cloud-native architectures. For teams using AWS-supported AMIs - including Amazon Linux 2023, Ubuntu, Windows Server, and many others - the SSM agent comes pre-installed, eliminating additional setup steps, making it a truly managed service that adds an extra layer of security without the complexity of key management.

Benefits

When working with cloud-native technologies, teams frequently spin up instances to test services and scripts in private subnets. While traditional solutions might rely on bastion hosts or VPC endpoints, SSM provides direct access to these private instances through IAM roles - simplifying your architecture while maintaining strict security controls.

Business Value

From a business perspective, SSM offers several key advantages:

Comprehensive Audit Capabilities: Through session logging and integration with CloudWatch, every action is recorded for compliance and troubleshooting.
Session History Tracking: Easily track who accessed which instance and when.
Flexible Log Storage: Option to send session logs to Amazon S3 for long-term storage or stream them to CloudWatch Logs for real-time monitoring.
Enhanced Security: Eliminates the need for open inbound ports, reducing the attack surface and leveraging IAM for access control.

IAM Role

IAM Instance Profile

An IAM instance profile is a container for an IAM role that you can assign to an EC2 instance. This allows the instance to securely interact with other AWS services without the need to manage credentials manually.

Creating an IAM Role for SSM Access

In the AWS console search up Roles > Create role

Trusted Entity: EC2 > EC2 use case.

Click Next

Add Permissions: Attach the AmazonSSMManagedInstanceCore policy to grant necessary permissions for SSM.

Click Next

Name the role.

Provide a name for the role, such as DemoEC2RoleForSSM.

Review your settings then click Create role.

Create a EC2 instance

In the AWS console search up EC2 > launch Instance

Name: Give a unique name
AMI: Amazon Linux 2023
Instance Type: t2.micro
Key pair (login): Proceed without a key pair.

Network settings

Auto-assign public IP: Disable.
Firewall (security groups): Create a new security group.
Allow SSH traffic from: Uncheck to ensure SSH is not open.

For Custom VPC with Private Subnets

VPC: Select your custom vpc
Subnet: Choose your private subnet
Auto-assign public IP: Disabled.
Firewall (security groups): Remove the security group; SSM will handle access.

Advanced details

IAM instance profile: Select DemoEC2RoleForSSM.

Click Launch instance to create your EC2 instance.

Access EC2 via SSM

In the AWS Console Search "Systems Manager"

Start a Session:

Under Node Tools Click on Session Manager.
Click Start Session

Select your Instance click Start Session.

You will now be logged into your EC2 instance without needing SSH.

Terminal interface within Session Manager.

Conclusion

By leveraging AWS Systems Manager Session Manager, you can achieve secure EC2 access without the hassles of SSH key management. This AWS Systems Manager tutorial demonstrates a streamlined, cost-effective approach to managing your EC2 instances across multiple VPCs and regions. Embrace this method to bypass SSH with AWS SSM, enhancing both your security posture and operational efficiency.

EC2 CI/CD Pipeline using AWS CodePipeline, CodeBuild, CodeDeploy

David Hyppolite — Mon, 30 Dec 2024 17:05:09 +0000

Set up a fully automated CI/CD pipeline using AWS CodePipeline, CodeBuild, CodeDeploy, EC2, and GitHub. This guide will help you create a zero-touch solution that automates code updates, ensures reliable deployments, and eliminates the need for manual SSH sessions. By the end, you'll have a robust and efficient deployment process that enhances both reliability and speed.

Quick Links

Overview & Benefits
Step 1: GitHub Repository Setup
Step 2: IAM Roles
Step 3: EC2 Configuration
Step 4: CodeDeploy Setup
Step 5: CodePipeline Integration
Conclusion

Technologies Used:

GitHub: Stores the application’s source code.
AWS CodePipeline: Detects changes in the repository, triggers builds, and initiates deployments.
AWS CodeBuild: Builds your code into production-ready artifacts.
AWS CodeDeploy: Automates the deployment of your builds onto EC2 instances.
Amazon EC2: Hosts your web application.
Amazon S3 (Optional): Stores build artifacts
IAM: Manages secure permissions for the pipeline and services to interact.

Prerequisites

AWS Account with administrative access
GitHub account
Basic understanding of AWS services
Familiarity with React/Vite applications

Business Value

Key Benefits

Improved Reliability: No more manual SSH deployments. CodeDeploy ensures consistency in every deployment.
Faster Iteration: Push code and let the pipeline test and deploy automatically, speeding up feedback loops.
Rollback Capability: If something goes wrong, CodeDeploy can roll back to a previous stable version quickly, minimizing downtime.

Problem Statement

Your team currently deploys updates by SSHing into a live EC2 instance—leading to human error, missed steps, and risky rollbacks. This tutorial fixes that by detecting GitHub changes automatically, deploying them via CodeDeploy, and using a repeatable, testable process.

Problem Solved

Traditional SSH-based deployments lead to:

Human errors during manual deployments
Inconsistent deployment steps
Difficult rollbacks
Security vulnerabilities

This solution implements industry best practices for automated, secure, and reliable deployments.

Ready to eliminate manual deployments? Let's build your pipeline.

Implementation Guide

Step 1: GitHub Repository

You can use your own repo or follow along with mine:

Fork this repository:

Github Repo: Github EC2

Repo Contains:


Repository Structure:
├── my-react-app/        # Sample React application
├── buildspec.yml        # CodeBuild instructions
├── appspec.yml         # CodeDeploy configuration
└── scripts/           # Deployment scripts

Step 2: IAM Roles

Important: If you attach a new role to an existing EC2 instance, reboot the instance so it recognizes the updated permissions.

EC2 Role

In the AWS Console search Roles -> Create role.

Trusted Entity: EC2 > EC2 use case.

Click Next

Add Permissions: Attach the AmazonEC2RoleforAWSCodeDeploy policy.

Click Next

Provide a name for the role.

EC2CodeDeployRole

Review your settings then click "Create role".

CodeDeploy Role

Click Create role.

Trusted Entity: EC2 > EC2 use case.

Click Next

Add Permissions: Attach AWSCodeDeployRole.

Click Next

Provide a name for the role.

CodeDeployRole

Review your settings then click "Create role".

Edit the trusted policy using:

Search for the newly created CodeDeployRole and select.

Choose the Trust relationships tab.

Choose Edit trust policy.

Copy and paste this trust policy.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "",
            "Effect": "Allow",
            "Principal": {
                "Service": [
                    "codedeploy.amazonaws.com"
                ]
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

Click "Update policy".

Step 3: EC2 Configuration

In the AWS console search up EC2 > Launch Instance.

Launch an Instance

Keep default settings.

Name and tags: Name your instance.
Application and OS Images: Ubuntu
Instance type: t2.micro
Key pair (login): Create or use an existing key pair.

Network settings: Use or create a security group allowing SSH (port 22) from your IP and HTTP (port 80) from the internet..

Under Advanced details

IAM instance profile: EC2CodeDeployRole

User data: Install the CodeDeploy agent on launch. Adjust region endpoints if you’re not using us-east-1:

Use this link to get your region to fetch your appropriate code-deploy region identifier

Edit this line in the script "aws-codedeploy-us-east-1.s3.us-east-1." with your appropriate region and identifier.

Copy and paste the code below.

#!/bin/bash

# Update system
apt update -y

# Install CodeDeploy Agent
apt install ruby-full -y
cd /home/ubuntu
wget https://aws-codedeploy-us-east-1.s3.us-east-1.amazonaws.com/latest/install
chmod +x ./install
./install auto

systemctl enable codedeploy-agent
systemctl start codedeploy-agent

Click Launch Instance and wait until it’s initialized..

Step 4: CodeDeploy Setup

We are setting up CodeDeploy first because it makes it easier to use and reference for later in the CodePipeline Section.

In the AWS search and select CodeDeploy > Create Application.

Application name: vite-react-deploy
Compute platform EC2/on-premise

Click Create application.

Click Create deployment group

Create deployment group

![deployemnt-groutp(https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7tr2q3qqmwan6610vcew.png)

Deployment group name: vite-react-group
Service role: CodeDeployRole (Select the service role we created earlier)

Deployment type: In-place

Select "Amazon Instances".
Key: Name
Value: Select your Instance

Leave the default settings for Agent configuration with AWS Systems Manager and Deployment settings.

Load balancer

Enable load balancing: Uncheck selection

Click Create deployment group

Review changes than Click Create deployment

Now on the creating a deployment page we can leave this section empty(exit page) and just move on to the next section because CodePipeline will handle the deployment for.

AppSpec Basics

version: 0.0
os: linux
files:
  - source: /my-react-app/dist
    destination: /var/www/html/
hooks:
  BeforeInstall:
    - location: scripts/before_install.sh
      timeout: 300
      runas: root
  AfterInstall:
    - location: scripts/after_install.sh
      timeout: 300
      runas: root
  ApplicationStart:
    - location: scripts/start_application.sh
      timeout: 300
      runas: root

AppSpec Configuration Explained

source: /my-react-app/dist is where CodeBuild places the compiled files.
destination: /var/www/html/ is where Nginx serves static files.
hooks: are scripts you define to run before/after install and on app start.

Step 5 CodePipeline Integration

In the AWS console search CodePipeline > Create Pipeline

This option allows us to build a custom pipeline to deploy from GitHub.

Click Next.

Name the Pipeline: Choose a meaningful name.
Service Role: Select New service role to allow AWS to create a role automatically.

Advanced settings: Use the default location to allow AWS to create an S3 bucket for CodePipeline artifacts, which will maintain deployment history.

Source

I have previously connected to GitHub so I will be using that connection.

For New Connections:

Click Connect to GitHub.

Name your connection and click Connect to Github.

Click Authorize AWS Connector for Github to grant AWS access to your GitHub repository.

A basic GitHub user connection suffices for our deployment pipeline.
If working with multiple organizations, select the specific user or organization repositories you want to use.
Click Connect.

Configure Source Stage

Source Provider: GitHub (via GitHub App)
Connection: Your GitHub connection
Repository Name: Select your repository
Default Branch: main
Output Artifact Format: Default settings
Webhook Events: Trigger the pipeline on push and pull request events.
Click Next.

Build Stage

Here is where we will be the vite app using the buildspec.yml. so we will be creating a CodeBuild project to reference our file in the repo.

Click Create project

Keep the default settings.

Name your project.

A new service role will be create for us through AWS.

Buildspec

Build specifications: Use a buildspec file

Here is where your buildspec.yml in the repo will be referenced.

version: 0.2

phases:
  install:
    runtime-versions:
      nodejs: 20
    commands:
        - cd ./my-react-app
        - npm install

  build:
    commands:
        - npm run build

artifacts:
  files:
    - 'my-react-app/dist/**/*'
    - 'appspec.yml'
    - 'scripts/**/*'
  discard-paths: no

(This builds your Vite app and includes the dist, appspec.yml, and scripts folder in the output artifacts.)

BuildSpec Configuration Explained

cd ./my-react-app will move to the react app folder
npm install will install all our node dependencies
npm build will build our production static vite app into the dist folder
The './my-react-app/dist' line is to ensure all files in the build output are copied.
'**/*' will include all build files in the root folder
The appspec.yml and scripts/**/* are explicitly listed to ensure they're included

Click Continue to CodePipeline

Build provider: Other build providers (AWS CodeBuild)
Build Type: Single Type
Region: Select your Region
Input artifacts: Keep defaults

Click Next

Deploy Stage

Here is how we will deploy to our EC2 instance. We will be using the CodeDeploy application we created earlier.

Deploy provider: CodeDeploy
Region: Select your region
Input artifacts: Keep defaults
Application name: vite-react-deploy
Deployment group: vite-react-group

Click Next

Review your settings than click "Create pipeline"

Your Pipeline will now run an each stage we worked on will get built.

Now head on over to your instance an get the public address our site should now be deployed on the EC2 Instance.

Conclusion

By following these steps, you’ve implemented a fully automated CI/CD pipeline using CodePipeline as the orchestrator an AWS Codebuild and CodeDeploy, EC2, and GitHub. This setup transforms your deployment process from fragile and manual to consistent, secure, and fast. You’ve minimized human error, introduced a clear rollback mechanism, and set the stage for more advanced enhancements—like adding integration tests, canary deployments, or blue/green strategies.

This is a crucial step toward embracing modern DevOps practices and ensures any team can deliver value to users with speed and confidence.

Automating Website Deployments with AWS CodePipeline and S3 No Upload

David Hyppolite — Fri, 20 Dec 2024 14:13:25 +0000

A step-by-step implementation of automated deployments using AWS CodePipeline, S3 static website hosting, and GitHub integration. This project demonstrates CI/CD pipeline creation, IAM security configuration, and version-controlled deployments - replacing traditional FTP uploads with a zero-touch solution.

Quick Links

Business Benefits
The Problem
The Solution: AWS CodePipeline
Implementation Guide
Making Changes and Verifying Deployment

Initial Architecture

Business Value

Business Benefits

Zero Manual Intervention: Automates the deployment process.
Complete Deployment History: Maintains a detailed record of all deployments.
Quick Rollback Capability: Easily revert to previous versions if needed.
Secure Access Control: Uses IAM roles to manage permissions securely.
Reduced Human Error: Minimizes mistakes associated with manual uploads.

The Problem

Your retail company currently maintains its static website content on-premises, relying on manual FTP uploads for updates. This manual approach often results in incomplete or inconsistent deployments, making it difficult to maintain a reliable version history or roll back changes when issues arise. With holiday promotions driving sudden traffic spikes, the site frequently struggles under the burden of rushed, ad-hoc updates—leading to downtime, frustrated customers, and missed revenue opportunities.

By introducing a fully automated CI/CD pipeline, you can eliminate the need for FTP uploads, ensure a consistent deployment history, and improve your site’s resiliency, especially during critical high-traffic periods.

Common Issues with FTP Deployments

Incomplete Uploads: Can break the website if files aren't fully uploaded.
Lack of Deployment History: No records of what was deployed and when.
Change Tracking: Difficult to identify who made specific changes.
No Rollback Capability: Unable to revert to previous versions easily.
Security Vulnerabilities: FTP lacks robust security measures, exposing the site to potential threats.

The Solution: AWS CodePipeline

Key Takeaways

Automated Deployments: Eliminate the risks associated with FTP.
Version Control: Provides a comprehensive deployment history.
IAM Roles: Ensure secure access and permissions management.
S3 Static Hosting: Enables reliable and scalable content delivery.
CodePipeline Automation: Streamlines the entire deployment process from source to deployment.

While many tutorials jump straight to CloudFront and custom domains, let's focus on the fundamental problem: automating deployments reliably.

Instead of manually uploading files to S3, this guide demonstrates a true zero-touch process. We'll start with an empty S3 bucket and let CodePipeline handle the entire deployment lifecycle.

Implementation Guide

Step 1: GitHub Repository Setup

Create your HTML file by running the following command in your repository:

touch index.html

Copy and paste the provided HTML code:

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Codepipeline Demo</title>
</head>
<body>
    <H1>Deploy via AWS Codepipeline</H1>
    <p>Deployment version: <span>1</span></p>
</body>
</html>

Push your code to your repo.

Step 2: S3 Configuration

First, create an S3 bucket with static website hosting:

In the AWS Console Search "S3".

Click the "Create Bucket" button.

Bucket Name: myawsbucket-demov1 (ensure the name is unique).

Keep Default settings except:

Uncheck the box: Block All Public Access.
Acknowledge the previous selection to Enable Public Access.
Bucket Versioning: Enable.

Enable Versioning (we are simulating a company its important to have versioning enabled for accidental deletes and to track history of changes)

Accept the remaining defaults and create the bucket.

Enable Static Website Hosting

After creating your bucket, select it and navigate to the Properties tab. Scroll down to Static website hosting and click Edit.

Static website hosting: enable
Hosting type: Host a static website
Index document: index.html
Save Changes

We Have created a bucket but we don't have the necessary permissions to access the items in the bucket.

Configure Bucket Policy

Navigate to the Permissions tab and scroll to Bucket Policy.

Click Edit and add the following policy to grant public read access to your objects:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "PublicReadGetObject",
            "Effect": "Allow",
            "Principal": "*",
            "Action": [
                "s3:GetObject"
            ],
            "**Resource**": "arn:aws:s3:::my-bucket/*"
        }
    ]
}

Adapt the Resource to match your bucket name, then save changes.

Step 3: CodePipeline Setup

In the AWS Console, search for and select CodePipeline.

Click Create Pipeline.

This option allows us to build a custom pipeline to deploy from GitHub to s3.

Click Next.

Pipeline Settings

Name the Pipeline: Choose a meaningful name.
Service Role: Select New service role to allow AWS to create a role automatically.

Advanced settings: Use the default location to allow AWS to create an S3 bucket for CodePipeline artifacts, which will maintain deployment history.

Add Source Stage

Click Connect to GitHub.

Name your connection and click Connect to Github.

Click Authorize AWS Connector for Github to grant AWS access to your GitHub repository.

A basic GitHub user connection suffices for this static website deployment pipeline.
If working with multiple organizations, select the specific user or organization repositories you want to use.
Click Connect.

Configure Source Stage

Source Provider: GitHub (via GitHub App)
Connection: Your GitHub connection
Repository Name: Select your repository
Default Branch: main
Output Artifact Format: Default settings
Webhook Events: Trigger the pipeline on push and pull request events.
Click Next.

Skip Build and Test Stages

Since you're deploying HTML files, you can skip the build stage.
Optionally, skip the test stage.

Deploy stage

Deploy Provider: Amazon S3
Region: Your chosen AWS region
Bucket: Select your static hosted S3 bucket
S3 Object Key: Leave blank
Extract file before deploy: Checked
Click Next.

Review and Create Pipeline

Review all settings and click Create Pipeline.

Step4: Testing and Verification

Pipeline automatically starts

The pipeline creation process should complete quickly.

As you can see, we've created a zero-touch solution from GitHub to S3, where files are automatically pulled from the repository without manual uploads.

With versioning enabled, you can track deployments:

Making Changes and Verifying Deployment

Make a change in your repository with a commit message like "Update index.html" to update the deployment version to 2.

CodePipeline will automatically detect the changes:

View the updated site reflecting the changes:

Conclusion

By following this guide, you've successfully set up an automated, secure, and efficient deployment pipeline using AWS CodePipeline, S3 static website hosting, and GitHub. This zero-touch solution not only streamlines your deployment process but also enhances security, provides version control, and ensures high availability during peak traffic periods.

Zero-Touch AWS Deployments: Building a Modern CI/CD Pipeline with GitHub Actions and CloudFront

David Hyppolite — Mon, 16 Dec 2024 14:35:00 +0000

Tech Stack Deep Dive

CI/CD & DevOps: GitHub Actions for automation, self-hosted runners for deployment
Containerization: Docker for consistent backend deployments
Cloud Infrastructure: AWS S3, CloudFront, EC2, ACM
Nginx: Reverse proxy
Database: MongoDB Atlas for reliable data persistence
Security & Auth: ACM for SSL, OIDC for AWS Role Assumption
Frontend: React, Vite
Backend: Python

GitHub
Github Actions Workflow

The Wake-Up Call

It started with a simple need: I wanted to know exactly how much money I needed to set aside before payday to cover my next two weeks. Simple enough, right? I built Wisewallet to solve this problem, but little did I know this straightforward application would become my crash course in modern cloud architecture and DevOps practices.

Picture this: It's 10 PM, and I'm staring at my terminal, hands slightly shaking as I rsync changes. My EC2 had to be rebooted instance, taking my SQLite database with it into the digital void. All that data, gone. Again. My monolithic setup - a React frontend, Python backend, SQLite database, and Nginx, all crammed onto a single EC2 instance - was starting to feel like a house of cards.

The Breaking Point

Every deployment was a small adventure in anxiety. I'd SSH into my server, fingers crossed, hoping my rsync command wouldn't miss any crucial files or add files that shouldn't belong:

rsync -avz --exclude 'node_modules' --exclude '.git' --exclude '.env' \
  --exclude 'venv' --exclude '__pycache__' --exclude 'db.sqlite3' \
  -e "ssh -i ~/.ssh/id_rsa" \
  . ubuntu@ec2-24-127-53:~/app

Then came the ritual of commands:

sudo apt update
sudo apt upgrade
sudo systemctl stop frontend.service
sudo systemctl stop backend-container.service
cd ~/app/frontend
npm install
npm run build

Each step was another chance for something to go wrong. And things did go wrong. Often. The EC2 instance would run out of storage during npm install. The build would timeout. The database would vanish with instance restarts. It was like playing DevOps roulette, and I was losing sleep over it. The app was not usable - it was a burden for months, constantly having to add the information over again.

The Rumination Phase

I spent weeks turning the problem over in my mind. The data loss issue haunted me the most. I kept thinking, "There has to be a better way to handle persistence." That's when I started researching MongoDB Atlas. The idea of a managed database service was appealing - no more data loss anxiety, automatic backups, and scaling without the headache. But I wrestled with the decision - was I overcomplicating things? Was I just adding unnecessary complexity?

Then there was the frontend deployment issue. Every time I needed to update the UI, I had to run npm run build on the EC2 instance. It was slow, resource-intensive, and frankly, felt wrong. The lightbulb moment came during a particularly frustrating deployment: "Why am I building static files on a server? These could live anywhere!"

The Architecture Evolution

Current CI/CD pipeline

Breaking Free from the Monolith

The first major decision was moving to MongoDB Atlas. It wasn't just about preventing data loss - it was about peace of mind. Knowing my data would survive any EC2 mishaps was worth the migration effort. The process taught me about data modeling, replication, and the true value of managed services. It took some time to change my code over from SQL to NoSQL but it was worth it.

For the frontend, S3 was an obvious choice, but the real game-changer was CloudFront. I remember the exact moment the decision clicked. I was comparing costs between running an Application Load Balancer and using CloudFront and saw how the configurations meant I could use it for the backend too!

This led to an interesting architecture: two CloudFront distributions - one for the S3-hosted frontend and another for the EC2 backend. It was unconventional, perhaps, but it solved multiple problems:

HTTPS everywhere without managing Nginx certificates
Caching at the edge
Cost-effective compared to an ALB for my use case

The CI/CD Epiphany

The manual deployment process still bothered me. I wanted zero SSH access needed for routine deployments, and I wanted to essentially mimic the exact steps I did manually but automated without errors or forgetting anything. The solution came in layers:

First, containerize the backend with Docker:

jobs:
  dockerize-backend:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4.1.7

      - name: Setup Docker Buildx
        uses: docker/setup-buildx-action@v3.4.0

      - name: Login to Docker Hub
        uses: docker/login-action@v3.2.0
        with:
          username: ${{ secrets.DOCKER_USERNAME }}
          password: ${{ secrets.DOCKER_PASSWORD }}

      - name: Build and push Docker image
        uses: docker/build-push-action@v6.4.1
        with:
          context: ./Backend/
          push: true
          tags: ${{ secrets.DOCKER_USERNAME }}/wisewallet-backend:latest

The key was sending the built image to DockerHub to be held until needed.

But how to get the container onto EC2 without SSH? This stumped me for days. The breakthrough came when I discovered self-hosted GitHub Actions runners. I could install a runner on my EC2 instance, waiting to pull and run the latest Docker image. It was elegant in its simplicity:

  deploy:
    needs: dockerize-backend
    name: aws-ec2
    runs-on: self-hosted  
    steps:
      - name: Pull image from Docker Hub
        run: docker pull ${{ secrets.DOCKER_USERNAME }}/wisewallet-backend:latest

      - name: Delete old Container
        run: docker rm -f ${{ secrets.DOCKER_USERNAME }}/wisewallet-backend:latest

      - name: Run docker container
        run: sudo systemctl restart backend.service

Then came the systemd services - one to keep the runner alive, another to manage the container lifecycle. No more manual intervention needed.

Installing a Self-Hosted Runner

The process is straightforward:

Go to repo under actions tab
Under management < runners
Hit new Runners < new self-hosted runner
Runners are versatile for Linux, Windows, macOS

Here's the runner service configuration:

[Unit]
Description=Github actions runner
After=network.target

[Service]
User=ubuntu
WorkingDirectory=/home/ubuntu/actions-runner/
ExecStart=/home/ubuntu/actions-runner/run.sh
Restart=always
RestartSec=3

[Install]
WantedBy=multi-user.target

And the backend service:

[Unit]
Description=Docker for Backend Service
Requires=docker.service
After=network.target docker.service

[Service]
Type=simple
User=ubuntu
WorkingDirectory=/home/ubuntu/
ExecStartPre=/usr/bin/docker pull image/backend:latest
ExecStart=/usr/bin/docker run --rm --name backend -p 8000:8000 --env-file /etc/app.env image/backend:latest uvicorn main:app --host 0.0.0.0
ExecStop=/usr/bin/docker stop backend
Restart=always
EnvironmentFile=/etc/app.env

[Install]
WantedBy=multi-user.target

Building the Static Files

No more manual npm run build, npm installs, or timeouts:

  build-frontend:
    runs-on: ubuntu-latest
    permissions:
      id-token: write   
      contents: read

    env:
      AWS_REGION: ${{ secrets.AWS_REGION }}

    steps:
      - name: Checkout code
        uses: actions/checkout@v4.1.7

      - name: Set up Node.js
        uses: actions/setup-node@v4.0.3
        with:
          node-version: 20.12.1

      - name: Cache Node.js modules
        uses: actions/cache@v4
        with:
          path: Frontend/node_modules
          key: ${{ runner.os }}-node-${{ hashFiles('**/Frontend/package-lock.json') }}
          restore-keys: |
            ${{ runner.os }}-node-

      - name: Install dependencies
        run: |
          cd ./Frontend
          npm install

      - name: Build the frontend
        env:
          VITE_APP_API_URL: ${{ secrets.VITE_APP_API_URL }}
        run: |
          cd ./Frontend
          npm run build

The Security Evolution

The final piece was security. Storing AWS credentials in GitHub secrets felt wrong. That's when I discovered AWS OpenID Connect. The ability to generate temporary credentials just when needed was exactly what I was looking for. It was a perfect example of the principle of least privilege in action. It allowed me to get my static images to S3 and invalidate my CloudFront cache instantly to apply changes - the only stored credentials are fairly simple:

      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: ${{ secrets.AWS_ROLE_ARN }}
          role-session-name: GitHubActionsSession
          aws-region: ${{ secrets.AWS_REGION }}

      - name: Deploy to S3
        run: |
          aws s3 sync ./Frontend/dist/ s3://${{ secrets.AWS_S3_BUCKET }} --region ${{ secrets.AWS_REGION }} --delete

      - name: Invalidate CloudFront cache
        run: |
          aws cloudfront create-invalidation \
            --distribution-id ${{ secrets.CLOUDFRONT_DISTRIBUTION_ID }} \
            --paths "/index.html"

The Results

The transformation has been dramatic. What used to be a stress-inducing manual process is now a simple git push, but with a twist - the deploy is now manual so I can send features once a month. The architecture is distributed but not overly complex. Each component does one thing well:

MongoDB Atlas handles data persistence
S3 and CloudFront manage static content delivery
Docker provides consistent backend deployments
GitHub Actions orchestrates everything

More importantly, I sleep better at night. No more 10 PM panic sessions. No more data loss anxiety. No more deployment roulette.

Looking Forward

This journey taught me that good architecture isn't about using the latest tech - it's about solving real problems. Each decision was driven by a specific pain point, not just a desire to use cool technology.

Looking ahead, I'm exploring:

Container orchestration with ECS (though for my scale, the current setup works well)
Integration tests in the CI/CD pipeline
Potentially going serverless with Lambda

But the biggest lesson? Sometimes the best solutions come from living with the pain long enough to truly understand the problem. Every sleepless night, every failed deployment, every data loss incident - they all contributed to the solution I have today.

Want to discuss cloud architecture, DevOps practices, or share your own journey? Connect with me on LinkedIn. I'm always eager to learn from others' experiences!

How to Fix Next.js CloudFront Permission Denied: Complete Guide to Static Export URL Issues

David Hyppolite — Fri, 13 Dec 2024 17:25:05 +0000

Technical Overview

This post demonstrates my expertise in:

Cloud Infrastructure: AWS (CloudFront, S3, Lambda) infrastructure management with production debugging
Modern Web Development: Next.js static site optimization and deployment
DevOps Practices: Infrastructure as Code with Terraform, CI/CD pipeline automation
Problem Solving: Complex debugging of URL routing and permissions in distributed systems
Cloud Security: IAM roles, bucket policies, and CloudFront function implementation
Monitoring & Logging: Setting up ETL pipeline for CloudFront logs using Lambda

Tech Stack: Next.js, AWS (CloudFront, S3, Lambda), Terraform, Python, JavaScript, CI/CD

The Engineering Journey
The Initial Setup
The Problem
Setting Up CloudFront Logging
- Configuring CloudFront Access
Setting Up Amazon Lambda
- Adding S3 Permissions
Creating the CloudFront Function
- Deploying the Function
Key Takeaways & Troubleshooting Lessons
- What Went Wrong
- Troubleshooting Approach
- Technical Lessons
- Prevention for Others

The Engineering Journey

Being an engineer, you're always learning things on the fly, and when you're hosting your own services, you're always running into unexpected bugs.

I hope this post helps somebody out there. Typically, issues like these are due to mismatched URL paths between what CloudFront expects and what it's getting back from your S3 bucket.

It's harder to know what to expect in production, especially in situations like these where you can't locally simulate the architecture to address edge cases.

The problem

<Error>
  <Code>AccessDenied</Code>
  <Message>Access Denied</Message>
  <RequestId>3W0PC46EVXQ</RequestId>
  <HostId>kz2tzcKKQpl+s0W8fpk+glo4QGR1p4V91ky9WRLTmHK8AgloIvPKsA1/80=</HostId>
</Error>

Here's the tricky part: when users navigate through the site using links, everything works perfectly because the links generated by Next.js include the correct file extension (e.g., /about.html). This makes the problem harder to diagnose because it only surfaces when users try to access the pages directly or refresh the page.

The Initial Setup

I was originally building my Next.js files as a static site on S3 using this code in next.config.ts and using trailingSlash to export all files as index.html:

import type { NextConfig } from "next";

const nextConfig: NextConfig = {
  output: 'export',
  trailingSlash: true
};

export default nextConfig;

Everything was perfect - I got the best of both worlds, but I wasn't aware of the limitations. CloudFront added an extra layer of complexity to URLs.

I figure this out later but this is crucial to fix the issue, you must open next.config.ts and make sure that trailingSlash is removed or set to false:

import type { NextConfig } from "next";

const nextConfig: NextConfig = {
  output: 'export',
  trailingSlash: false,
};

export default nextConfig;

I haved already saved you time.

Now when I use output: 'export' with trailingSlash: false, my files are being built with the default /about.html format. This allows Next.js to export each blog post in its own directory containing an index.html (so it will output /about/index.html).

From my experience with React/Vite I created a custom error response to for HTTP error Code 403 to customize error response200 and send to "index.html" or "/" but that just creates and infinite loop back to the homepage.

Setting Up CloudFront Logging

I wanted to understand deeply what was going on, so let's create logs to see what CloudFront is doing.

First, I created an S3 bucket to add CloudFront logs, using the default settings that S3 provides.

Configuring CloudFront Access

Go to your current CloudFront distribution and navigate to the Logging tab

We're going to associate our S3 bucket in the Standard log destination

Hit "Add" and select "Amazon S3"

Select the "Destination S3 bucket" to be your newly created log bucket

Access your site, give it some time for logs to generate, and you should see new logs being created in this format:

"distribution ID".YYYY-MM-DD-HH.unique-ID.gz

"optional prefix"/"distribution ID".YYYY-MM-DD-HH.unique-ID.gz

We cannot access the logs directly as they are zipped. This is where Lambda comes in.

Setting Up Amazon Lambda

Why use Amazon lambda?

Why use Amazon Lambda? For this moment, Lambda serves 3 purposes:

Extract: Getting the gzipped logs from CloudFront/S3
Transform: Unzipping/decompressing the files
Load: Sending to CloudWatch

Let's create a lambda function with basic configurations.

Hit create function.

We are using a blueprint to save us time.

Start with the "Get S3 Object" blueprint using Python runtime

Name your function.

Create a new role with basic Lambda permissions (this will only include permissions for writing logs to CloudWatch but lacks permissions for accessing S3)

Select your bucket you created logs
- Under event types, use only "Put" since logs are a put event
- Make sure to check the "Recursive invocation" box

Now hit create function.

Update the Lambda function with this code:

import json
import urllib.parse
import boto3
import gzip
import io

print('Loading function')
s3 = boto3.client('s3')

def lambda_handler(event, context):
    try:
        bucket = event['Records'][0]['s3']['bucket']['name']
        key = urllib.parse.unquote_plus(event['Records'][0]['s3']['object']['key'], encoding='utf-8')

        response = s3.get_object(Bucket=bucket, Key=key)
        compressed = response['Body'].read()

        with gzip.GzipFile(fileobj=io.BytesIO(compressed)) as gz:
            content = gz.read().decode('utf-8')
            print(content) 

        return {
            "statusCode": 200,
            "body": content
        }

    except Exception as e:
        print(f"Error: {str(e)}")
        raise

The only addition to the original code given is this will allow us to unzip the log files in the s3 bucket.

Adding S3 Permissions

Now we need to add permissions for our Lambda role to access the S3 bucket:

In Lambda, go to Configuration > Permissions

Select the role name under "Execution Role"

Here is the role with current basic policy.

Now go to "Add permissions" than "Create Inline Policy".

Add an inline policy with these permissions:
- GetObject: to retrieve log files
- ListBucket: to select the Log bucket

Restrict access to your specific log bucket ARN

Once your done hit "Next"

Name the policy. Now your lambda function will have access to log bucket.

Now go to your site and click and refreshed it to get some permission denied errors to generate some logs.

First set of logs. Hit "Search all log streams".

We can now see the access logs are being successfully decompressed.

You want to go through your logs to get the exact errors and paths affected.

Creating the CloudFront Function

After analyzing the logs, we can create a CloudFront function to rewrite the URLs properly:

1.Go to Cloudfront and on the left Side look for functions

Give your function a name and description of what it does. Than hit "Create function".

Create a new function with this our your code:

function handler(event) {
    var request = event.request;
    var uri = request.uri;

    if (uri === "/blog" || uri === "/blog/") {
        request.uri = "/blog.html";
        return request;
    }

    if (!uri.includes('.')) {
        uri += '.html';
    }

    request.uri = uri;
    return request;
}

Now save your changes.

Deploying the Function

Publish the function

Add an association to your CloudFront Distribution:
- Event Type: "Viewer Request"
- Behavior: Use the affected path (e.g., /blog*)

Note: I changed my behavior path to "/blog*" - this ensures that both /blog and /blog/* are handled by the same cache behavior and CloudFront Function.

Wait for your Distribution to redeploy, then invalidate your cache. Open your site in a new private window to test.

This solution worked for my case. If it doesn't work for you, go through your set of logs to see what the errors/paths are and alter your function accordingly - that's what I did.

Key Learnings and Root Cause

Understanding URL Path Behavior

Static site generators like Next.js create HTML files for each route
S3 expects full file paths (e.g., /about.html) while users access clean URLs (e.g., /about)
CloudFront and S3 handle URL paths differently in production vs local development
The URL mismatch causes permission denied errors only on direct access or page refreshes

Key Technical Insights

Next.js Configuration Impact:
- trailingSlash setting significantly affects URL structure
- Static export behavior needs special handling with CloudFront
- Default configurations might work locally but fail in production
Production Debugging Strategy:
- Set up proper logging infrastructure first
- Use Lambda to process CloudFront logs effectively
- Monitor actual user access patterns
- Test both direct access and navigation scenarios
CloudFront Function Benefits:
- Handles URL rewrites at the edge
- Provides clean URLs for users while maintaining proper S3 access
- Cost-effective solution for URL path manipulation
- Allows flexible routing rules based on your needs

Prevention Tips for Others

Start with trailingSlash: false in Next.js when using CloudFront
Set up comprehensive logging before deploying to production
Test all URL access patterns:
- Direct URL access
- Navigation through links
- Page refreshes
- Nested routes
Document your CloudFront function behavior for future reference

Remember: What seems like a simple permission issue might actually be a complex interaction between your static site generator, CDN, and storage service. Always validate your assumptions with proper logging and testing in production.

Building a Cloud-Native Blog Platform with Terraform

David Hyppolite — Wed, 11 Dec 2024 12:05:30 +0000

Why Build Another Blog Platform?
Architectural Decisions
- Core Infrastructure Components
- Infrastructure Management
The Real Challenges
- Cross-Account Issues
- CI/CD Pipeline Setup
Key Learnings
Looking Forward

Code Examples and Documentation

Next.js + AWS (CloudFront, S3, Route53, ACM) + Terraform + GitHub Actions CI/CD

Github Code

The complete Terraform configuration, GitHub Actions workflows, and detailed setup instructions are available in the repository. Feel free to use this as a reference for your own projects or to suggest improvements.

Remember: Sometimes the longest path teaches you the most valuable lessons.

Why Build Another Blog Platform?

"Just use WordPress," they said. "You could have it up in minutes."

As an AWS Solutions Architect, I knew I could spin up a simple blog in the console within an hour. But here's the thing - I've already built production systems maintaining 99.9% uptime. I've already clicked through the console countless times. This project wasn't about the fastest path to a blog; it was about pushing my limits.

I needed a platform that would:

Force me to translate my console knowledge into code
Challenge my understanding of AWS service interactions
Create a real-world testing ground for complex cloud architectures
Serve as documentation for other engineers facing similar challenges

The real value? The gap between knowing how to do something in the console and implementing it in Terraform. That's where the learning happens. That's where engineers grow.

Architectural Decisions

Core Infrastructure Components

I designed the architecture with several key requirements in mind:

High Availability: Using CloudFront for edge distribution
Security: Implementing proper IAM roles and OIDC federation
Scalability: S3 for static content and versioning
Automation: Complete CI/CD pipeline with GitHub Actions

Here's how each piece fits together:

Content Delivery and Security

CloudFront Distribution: Handles HTTPS termination, caching, and low-latency delivery
ACM (AWS Certificate Manager): Manages SSL certificates for secure communication
Origin Access Identity (OAI): Restricts S3 access exclusively to CloudFront
S3 Bucket: Hosts static files with versioning enabled

Infrastructure Management

Private S3 Backend: Maintains Terraform state files securely
IAM Roles: Enables long-term automation with least privilege
OIDC Web Identity: Secures GitHub Actions' AWS access

This setup not only adheres to AWS’s six pillars of well-architected frameworks but also taught me the intricacies of networking and IaC.

The Real Challenges: A Sequential Journey

What looks like a straightforward architecture on paper turned into a complex troubleshooting journey. Here's how each challenge led to the next:

1. The Hosted Zone Puzzle

Initially, Terraform couldn't access the hosted zone by name. While AWS CLI could find it, Terraform kept failing. This was my first hint that something deeper was wrong, though I didn't realize it yet.

# Initial attempt that failed`
resource "zone_name" "primary" {
  name = var.domain_name
}

# Solution: Use zone ID directly`
data "zone_id" "primary" {
  zone_id = var.zone_id
}

2. The Multi-Account Maze

The hosted zone issue unveiled a complex problem: I was unknowingly trying to access resources across multiple AWS accounts without proper cross-account permissions. Here's how it manifested:

First sign: Permission errors when Terraform tried accessing resources such as hosted zone
Second sign: User not found in the expected account
Final revelation: S3 bucket 403 errors indicating cross-account access attempts

After numerous attempts at fixing permissions, with fatigue setting in and countless Chrome tabs open, I stepped away from the computer. Sometimes the best debugging tool is a fresh perspective. When I returned, I methodically verified each environment variable, discovering they weren't properly configured.

Initially, I set environment variables for my default account:

# Checking current identity
aws sts get-caller-identity

# Setting up environment variables
export AWS_ACCESS_KEY_ID="new_access_key"
export AWS_SECRET_ACCESS_KEY="new_secret_key"
export AWS_DEFAULT_REGION="us-east-1"

# Profile
export AWS_PROFILE=default

But this created conflicts when trying to switch between accounts. The real issue? The Terraform role lacked cross-account access permissions.

3. Cross-Account Resource Management

The situation became clear:

Hosted zone: In main account but inaccessible due to resource in backup account
S3 bucket: In backup account but main role lacked cross-account access
Terraform: Were able to create resources becuase I gave permission from backup account but not main account.

With a fresh mind,I saw the clear picture. I took a systematic approach:

Resolution required systematic cleanup:

Profile Configuration: I configured a profile specifically for the backup account, which I had initially removed. By setting export AWS_PROFILE=backup, I aligned the Terraform operations with the correct account.
Environment Variable Adjustment: I unset the previous environment variables that were causing conflicts and reconfigured the AWS CLI to ensure that operations were correctly authenticated and executed under the appropriate account.
Verification and Cleanup: I ran aws sts get-caller-identity to verify that the CLI was now using the correct account. Subsequently, I used terraform init to reinitialize Terraform and terraform destroy --auto-approve to remove mistakenly created resources in backup.

The resolution of these issues marked a significant turning point in the project. Not only did I manage to streamline the AWS architecture by ensuring resources were correctly aligned with their respective accounts, but I also reinforced best practices in AWS management and Terraform usage. This process underscored the importance of vigilant account management and served as a valuable lesson in maintaining clarity and organization in cloud environments.

4. CI/CD Pipeline Optimization

The final challenge emerged in the CI/CD pipeline. While the YAML was syntactically correct, Terraform operations were hanging at Terraform Plan. The root cause? Environment variables needed to be passed to each Terraform command individually:

- name: Terraform Init
  run: |
    cd ./terraform-config
    terraform init
  env:
    TF_VAR_domain_name: ${{ secrets.DOMAIN_NAME }}
    TF_VAR_aws_region: ${{ secrets.AWS_REGION}}
    TF_VAR_zone_id: ${{ secrets.ZONE_ID }}

- name: Terraform
  id: plan
  run: |
    cd ./terraform-config
    terraform plan -out=tfplan

- name: Terraform Apply
  id: apply
  run: |
    cd ./terraform-config
    terraform apply -auto-approve tfplan

Working solution

- name: Terraform Init
  run: |
    cd ./terraform-config
    terraform init
  env:
    TF_VAR_domain_name: ${{ secrets.DOMAIN_NAME }}
    TF_VAR_aws_region: ${{ secrets.AWS_REGION}}
    TF_VAR_zone_id: ${{ secrets.ZONE_ID }}

- name: Terraform
  id: plan
  run: |
    cd ./terraform-config
    terraform plan -out=tfplan
  env:
    TF_VAR_domain_name: ${{ secrets.DOMAIN_NAME }}
    TF_VAR_aws_region: ${{ secrets.AWS_REGION}}
    TF_VAR_zone_id: ${{ secrets.ZONE_ID }}

- name: Terraform Apply
  id: apply
  run: |
    cd ./terraform-config
    terraform apply -auto-approve tfplan 
  env:
    TF_VAR_domain_name: ${{ secrets.DOMAIN_NAME }}
    TF_VAR_aws_region: ${{ secrets.AWS_REGION}}
    TF_VAR_zone_id: ${{ secrets.ZONE_ID }}

This challenge took me two hours to solve, highlighting the importance of understanding the nuances of Terraform commands within CI/CD pipelines and reinforcing the need for meticulous configuration to ensure smooth automation.

Key Learnings

Account Management is Critical

- Always verify which account you're operating in
- Set up proper profile management early
- Use `aws sts get-caller-identity` frequently

Infrastructure as Code Requires Different Thinking

- What works in the console might need different approaches in Terraform
- Resource relationships need explicit definition
- State management is crucial

Authentication Flow Understanding

- OIDC federation setup requires careful configuration
- Environment variables affect different tools differently
- Multiple authentication methods need careful management

Best Practices Emerged

- Always verify account context before operations


Implement clear naming conventions
Maintain separation of concerns between accounts
Document environment variable requirements

Embracing the Hard Path

One year ago, I made a choice: stop settling for the easy path. I had already built multiple AWS projects. Already maintained Linux servers. Already earned my AWS Solutions Architect certification. But Terraform? That was my next mountain to climb.

The console is comfortable. It's visual. It's immediate. But code? Code demands precision. Every resource relationship must be explicit. Every permission must be defined. Every interaction must be planned.

This project forced me to think differently about infrastructure:

No more clicking through options
No more relying on AWS's default settings
No more visual confirmation of changes

Instead, I had to:

Define every resource relationship in code
Understand every service interaction deeply
Plan for automation from the start
Think in terms of state management and drift detection

Was it harder? Absolutely.
Did it take longer? Without question.
Was it worth it? Every frustrating minute.

Looking Forward

This platform isn't just a blog—it's a testament to the complexity and learning opportunities in modern cloud architecture. Every challenge forced me to dig deeper into AWS services, Terraform behavior, and Cloud practices.

The final architecture follows AWS Well-Architected Framework principles:

Security: Proper IAM roles, OIDC federation, and access controls
Reliability: CloudFront distribution and S3 versioning
Performance Efficiency: Edge caching and optimization
Cost Optimization: Practically Free
Operational Excellence: Fully automated deployments

Resolving the Lambda S3 Trigger Error

David Hyppolite — Tue, 10 Dec 2024 12:00:24 +0000

While setting up a Lambda function to process S3 logs, I encountered an error that wasn't immediately obvious how to resolve:

Your Lambda function "[function-name]" was successfully created, but an error occurred when creating the trigger: Configuration is ambiguously defined. Cannot have overlapping suffixes in two rules if the prefixes are overlapping for the same event type.

The error appeared when trying to set up the trigger through the AWS Console. My use case was straightforward - I needed to process logs as they arrived in an S3 bucket. The Lambda function was created successfully, but the S3 trigger configuration kept failing.

What Didn't Work

Initially, I tried:

Deleting and recreating the trigger through the AWS Console
Double-checking the event types and prefix/suffix settings
Verifying IAM permissions were correct

None of these steps resolved the issue, and the error message wasn't particularly helpful in pointing to the real problem.

Finding the Actual Issue

The breakthrough came when I decided to inspect the bucket's configuration using the AWS CLI:

aws s3api get-bucket-notification-configuration --bucket "my-bucket"

This revealed something interesting - a lingering configuration that wasn't visible in the AWS Console:

$ aws s3api get-bucket-notification-configuration --bucket "my-bucket"
{
    "LambdaFunctionConfigurations": [
        {
            "Id": "23410cc7-d44d-4d8a-b50d-5c41dcf80ec9",
            "LambdaFunctionArn": "arn:aws:lambda:us-east-1:637423581329:function:ingest_s3logs",
            "Events": [
                "s3:ObjectCreated:*"
            ]
        }
    ]
}

The Solution

With this discovery, the fix was simple:

aws s3api put-bucket-notification-configuration --bucket "my-bucket" --notification-configuration '{}'

This command cleared out the hidden configuration, allowing me to successfully create the new trigger for the log processing function.

Lessons Learned

The AWS Console doesn't always show the full picture of your resource configurations
When dealing with S3 event notifications, the CLI can reveal hidden settings
Sometimes starting with a clean slate is the quickest path to resolution

DEV Community: David Hyppolite

How to Deploy a FastAPI App to Azure with Docker, ACR, and GitHub Actions

Prerequisites

Step 1. Create an Azure Container Registry (ACR)

Create Container Registry

Push Your Image

Step 2. Create an APP Service & Web App

Basics

Configure the container

Networking

Monitor + secure

Review + Create -> Create

Access & Verify Web App

Step 3. Set Up GitHub Actions CI/CD

Enable deployment Center

Connect to GitHub

✏️ Fix the generated workflow

🐳 Original Build & Login Block:

Add GitHub secrets

Save the credentials to your GitHub repo, go to your repository Settings under Security > Secrets and variables > Actions > New repository secret.

Test the Pipeline

Cleanup resources

Conclusion

Deploying Grafana on an AWS EC2 Instance(2025)

Problem

Solution

Why This Matters

What You'll Learn

Tools Used

Step 1 Create your IAM role

Step 2 Launch your instance

Network settings

Advanced details

User Data

Step 3 Connect to Grafana UI

Step 4 Configure a data source

Build a Dashboard

Step 5 Stress test the EC2

Conclusion

Securely Deploy a EKS Cluster in a Private AWS VPC with Terraform & AWS SSM (No Bastion Hosts or SSH Keys Required)

What You'll Learn

Table of Contents

Technologies & Skills

Prerequisites

Introduction

Why This Approach?

Critical Access Configuration ⚠️

Key Takeaways

Infrastructure Setup and Configuration

Step 1: GitHub Repository

Step 2: IAM Role Configuration

Step 3: Create ECR repository

Step 4: Infrastructure Deployment

Set up the VPC

Private subnets

Security Group Configuration

EKS Cluster Configuration

EKS Access Entries and Policy Associations

Applying Terraform Configuration

Deployment and Access Configuration

Step 6. Creating the EC2 Instance for Cluster Access

Launch EC2 Instance

User Data Script

Allow EC2 access to control plane security group

Connecting via Systems Manager

Kubernetes Manifests and Deployment

Understanding the Manifests

Deployment Manifest

Service Manifest

Deploying the Manifests

1. Create the deployment file

2. Create the service file

3. Apply the manifests

4. Verify the deployment

Troubleshooting Tips

Resource Cleanup

📝 Conclusion

🟢(Optional) Implement CI/CD with GitHub Actions

Build a Secure CI/CD Pipeline for Amazon EKS Using GitHub Actions and AWS OIDC

Table of Contents