DEV Community: Tandap Noel Bansikah

Kyverno Without the Noise: Practical Kubernetes Policies for Security & Best Practices

Tandap Noel Bansikah — Fri, 23 Jan 2026 17:12:41 +0000

How to introduce guardrails in Kubernetes using native YAML — without slowing teams down.

Introduction

Kubernetes gives teams a lot of power. Too much power, sometimes. One privileged: true or missing resource limit can quietly turn into a security incident or a production outage.

That's where Kyverno comes in.

This article is not a full Kyverno manual. Instead, it's a guided introduction: why Kyverno exists, how it fits into Kubernetes, and a few concrete security-focused examples to get you started. If it clicks, you can go deeper using the official docs.

What is Kyverno — and why should you care?

Kyverno is a Kubernetes-native policy engine. It lets you define rules for your cluster using plain YAML, the same format you already use for Deployments, Services, and Ingresses.

No new language
No external policy service
No complex tooling

If Kubernetes understands YAML, Kyverno understands your policy.

Why Kubernetes needs policy enforcement

Kubernetes validates syntax, not intent.

That means Kubernetes will happily accept:

Pods running as root
Containers without memory limits
Images pulled from untrusted registries
Inconsistent labels and annotations

In real clusters — especially multi-team ones — these are not edge cases. They're normal.

Policies turn expectations into guarantees.

Where Kyverno fits in the Kubernetes lifecycle

Kyverno runs as an admission controller. Every time someone tries to create, update, or delete a resource, Kyverno gets a chance to inspect it.

It can:

Validate it (allow or deny)
Mutate it (fix or enhance it)
Generate other resources
Audit it (report without blocking)

Admission flow (simplified)

This happens before anything is stored in etcd.

Policies as YAML — and why that matters

Most policy engines introduce a new DSL. Kyverno doesn't.

A Kyverno policy looks like a Kubernetes resource:

apiVersion
kind
metadata
spec

That means:

Developers can read policies
Platform teams can review them
GitOps tools can manage them
Policies feel like part of the cluster — not an add-on

The main types of Kyverno policies (quick tour)

Kyverno supports multiple policy types for different use cases:

Core Policy Types

ClusterPolicy - Cluster-wide policies (used in our examples)

Validate, mutate, generate resources
Verify image signatures and attestations

Policy - Namespace-scoped policies

Same capabilities as ClusterPolicy but limited to specific namespaces

Specialized Policy Types (v1.14+)

ValidatingPolicy - Pure validation

Validate Kubernetes resources or JSON payloads
Lightweight alternative to full ClusterPolicy

ImageValidatingPolicy - Image security

Verify container image signatures and attestations
Focused on supply chain security

MutatingPolicy (v1.15+) - Resource modification

Mutate new or existing resources
Dedicated mutation capabilities

GeneratingPolicy (v1.15+) - Resource creation

Create or clone resources based on flexible triggers
Advanced resource generation

Utility Types

CleanupPolicy - Resource cleanup

Delete matching resources based on a schedule
Automated cluster maintenance

DeletingPolicy (v1.15+) - Scheduled deletion

Delete matching resources based on a schedule
Enhanced cleanup capabilities

CEL Libraries - Extended functions

Extended CEL functions for complex policy logic
Advanced policy features

For this guide, we'll focus on ClusterPolicy - the most versatile option that covers validation, mutation, and generation in one resource type.

Prerequisites

You'll need a Kubernetes cluster to follow along. Choose one:

Minikube: minikube start
KodeKloud Playground: kodecloud kubernetes playground
Kind, k3s, or any Kubernetes cluster

Installation

Install Kyverno

Multiple installation options:

Option 1: Direct YAML (recommended for testing)

kubectl create -f https://github.com/kyverno/kyverno/releases/latest/download/install.yaml

Option 2: Helm

helm repo add kyverno https://kyverno.github.io/kyverno/
helm install kyverno kyverno/kyverno -n kyverno --create-namespace

Option 3: Kustomize

kubectl apply -k https://github.com/kyverno/kyverno/config/default

Get the Demo Repository

git clone https://github.com/bansikah22/kyverno-demo.git
cd kyverno-demo

Demo time: 3 practical Kyverno scenarios

All examples below are ready-to-run and intentionally minimal.

Scenario 1: Block privileged containers

Running privileged containers is one of the fastest ways to break your security model.

Policy: deny privileged Pods

apiVersion: kyverno.io/v1        # Kyverno API version
kind: ClusterPolicy              # Policy applies cluster-wide
metadata:
  name: disallow-privileged-containers
spec:
  validationFailureAction: Enforce # Block resources that fail validation
  rules:
    - name: no-privileged
      match:
        resources:
          kinds:
            - Pod                # Apply this rule to Pod resources
      validate:
        message: "Privileged containers are not allowed"
        pattern:
          spec:
            containers:
              - securityContext:
                  privileged: false # Require privileged to be false

Result: the Pod never reaches the cluster.

Scenario 2: Enforce resource requests and limits

Missing resource limits are a classic reliability problem.

Policy: require CPU & memory limits

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: require-resource-limits
spec:
  validationFailureAction: Enforce # Block pods without limits
  rules:
    - name: check-limits
      match:
        resources:
          kinds:
            - Pod                # Target Pod resources
      validate:
        message: "CPU and memory limits are required"
        pattern:
          spec:
            containers:
              - resources:
                  limits:
                    cpu: "?*"      # Require any CPU limit value
                    memory: "?*"   # Require any memory limit value

This turns best practices into cluster rules, not documentation.

Scenario 3: Let Kyverno fix things automatically

Blocking is good. Helping is better.

Policy: inject a default securityContext

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: add-security-context
spec:
  rules:
    - name: add-non-root
      match:
        resources:
          kinds:
            - Pod                # Apply to Pod resources
      mutate:                    # Modify the resource
        patchStrategicMerge:     # Merge this config into the Pod
          spec:
            securityContext:
              runAsNonRoot: true # Automatically add non-root security

Developers don't change anything — the cluster becomes safer by default.

Policy Scope: ClusterPolicy vs Policy

Kyverno offers two policy scopes:

ClusterPolicy (used in our examples):

Applies cluster-wide to all namespaces
Perfect for organization-wide security standards
Requires cluster-admin permissions

Policy (namespace-scoped):

Applies only to resources in a specific namespace
Ideal for team-specific rules
Can be managed by namespace admins

apiVersion: kyverno.io/v1
kind: Policy                    # Namespace-scoped policy
metadata:
  name: team-specific-limits
  namespace: development        # Only applies to 'development' namespace
spec:
  # Same rules as ClusterPolicy

Choose based on your governance model.

Audit vs Enforce: adopting Kyverno safely

Kyverno doesn't force you to go all-in.

You can start in Audit mode:

Policies report violations
No workloads are blocked

Then move to Enforce when teams are ready.

This makes Kyverno realistic for existing clusters.

When Kyverno shines — and when it doesn't

Kyverno is excellent when:

You want Kubernetes-native policies
You value readability and YAML
You enforce platform and security standards

Kyverno is not:

A runtime security tool
A replacement for vulnerability scanners
A general-purpose policy engine outside Kubernetes

Knowing this avoids misuse.

Where to go next

If Kyverno makes sense so far:

Explore more policy examples in the official policy library
Look into image verification and supply-chain security
Try combining Kyverno with GitOps workflows

Kyverno doesn't try to do everything — it does policy enforcement the Kubernetes way.

And that's exactly why it works.

Hands-on Testing

Now let's test everything together:

1. Apply the policies

kubectl apply -f policies/

2. Test policy enforcement

Try the bad pod (should be rejected):

kubectl apply -f workloads/bad-pod.yaml
# Expected: Error - privileged containers not allowed

Deploy the good pod (should succeed):

kubectl apply -f workloads/good-pod.yaml
# Expected: Pod created successfully

Deploy production workload:

kubectl apply -f workloads/kubesrv-deployment.yaml
# Expected: Deployment and service created

3. Verify the results

# Check pods
kubectl get pods

# Test the service (use port 8080 to avoid permission issues)
kubectl port-forward svc/kubesrv-service 8080:8080

# In another terminal, test with curl
curl http://localhost:8080

## or visit browser with http://localhost:8080 and you will se

4. Check policy reports

# View policy violations
kubectl get cpol
kubectl describe cpol disallow-privileged-containers

Conclusion

Kyverno brings policy enforcement to Kubernetes without the complexity. By using familiar YAML syntax and integrating natively with the Kubernetes API, it makes security and governance accessible to every team member.

The key takeaways:

Start small: Begin with a few critical policies in Audit mode
Use ClusterPolicy for organization-wide standards, Policy for team-specific rules
Leverage mutation to help teams rather than just block them
Combine with GitOps for policy-as-code workflows

Kyverno doesn't try to solve every problem — it solves policy enforcement the Kubernetes way. And that focus is exactly what makes it powerful.

References

Ready to implement Kyverno in your cluster? Start with the demo repository and gradually expand your policy coverage. Remember: the best policy is one that helps teams succeed while keeping the cluster secure.

Terraform State Management with Amazon S3

Tandap Noel Bansikah — Wed, 07 Jan 2026 07:02:37 +0000

In our previous article the AWS EKS series, i mentioned that we will later have an article about managing terraform state. Managing Terraform state correctly is one of the most important skills for anyone using Terraform in real-world AWS environments.

In this article, we'll explore how to store your Terraform state in an Amazon S3 remote backend, why this matters, essential best practices (including state locking), and a hands-on example you can follow.

This guide uses official Terraform documentation as the source of truth.

Why Terraform State Matters

Terraform uses a state file (terraform.tfstate) to track what infrastructure it manages. This file contains important mappings between your configuration and real resources — so Terraform can make safe and predictable changes in future runs.

By default, Terraform stores state locally, which works for experimentation but is unsafe for collaboration, automation, and production use.

Why Use a Remote Backend like S3?

Using a remote backend like Amazon S3 provides:

Centralized state accessible by all team members
Collaboration support
Encryption and access control via IAM
Versioning for recovery
Support for state locking to avoid conflicts

Terraform's official documentation confirms that S3 is a supported backend type that can store state and support locking — and it's one of the most popular and reliable backends used in AWS environments.

State Locking — Preventing Conflicts

Terraform needs to prevent two people from writing changes to the same state file at the same time — otherwise state could become corrupted.

Modern Locking (S3 Native)

Terraform now supports native state locking in S3 through a lockfile mechanism. This means Terraform will create a .tflock file in the bucket alongside your state file, preventing concurrent runs.

This is done using the optional argument use_lockfile in your backend config.

terraform {
  backend "s3" {
    bucket       = "my-terraform-state-bucket2z"
    key          = "envs/prod/terraform.tfstate"
    region       = "us-east-1"
    use_lockfile = true
  }
}

use_lockfile = true enables S3-native locking
S3 creates a .tflock file during a Terraform run
Other processes must wait until the lock is released

This removes the need for a separate DynamoDB table for state locking — an approach that is officially deprecated and will be removed in future Terraform releases.

Note: DynamoDB locking is still supported for backward compatibility, but Terraform now prefers S3 native locking for new configurations.

Prerequisites

Before we begin, make sure you have:

AWS CLI installed and configured

aws configure

Terraform installed

terraform version

Permissions to create S3 buckets and apply Terraform changes

Step 1 — Create an S3 Bucket for Terraform State

Terraform cannot create the backend bucket for you, so you must create it first.

aws s3api create-bucket \
  --bucket my-terraform-state-bucket2z \
  --region us-east-1

Replace my-terraform-state-bucket2z with a globally unique name.

Step 2 — Enable Bucket Versioning (Best Practice)

Versioning helps you recover from accidental deletions or corruption.

aws s3api put-bucket-versioning \
  --bucket my-terraform-state-bucket2z \
  --versioning-configuration Status=Enabled

Step 3 — Enable Encryption on the Bucket

Encrypting state at rest is essential for security:

aws s3api put-bucket-encryption \
  --bucket my-terraform-state-bucket2z \
  --server-side-encryption-configuration '{
    "Rules": [{
      "ApplyServerSideEncryptionByDefault": {
        "SSEAlgorithm": "AES256"
      }
    }]
  }'

Troubleshooting: AccessDenied Error

If you encounter an AccessDenied error when running the encryption command:

An error occurred (AccessDenied) when calling the PutBucketEncryption operation: Access Denied

Common causes and fixes:

Insufficient IAM Permissions — Your IAM user/role needs s3:PutEncryptionConfiguration permission
Bucket doesn't exist — Verify the bucket name is correct:

   aws s3api head-bucket --bucket my-terraform-state-bucket2z

Wrong AWS credentials — Verify your identity:

   aws sts get-caller-identity

Step 4 — Configure Terraform Backend

In your Terraform directory, define the backend like this:

terraform {
  backend "s3" {
    bucket       = "my-terraform-state-bucket2z"
    key          = "demo/terraform.tfstate"
    region       = "us-east-1"
    use_lockfile = true
  }
}

Option	Description
`bucket`	The S3 bucket you created
`key`	Path to store the state file
`use_lockfile`	Enables native S3 state locking

Demo: Using S3 Remote State

1. Create a Terraform Project

mkdir terraform-s3-state-demo
cd terraform-s3-state-demo

2. Add Terraform Code

Create a file main.tf:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }

  backend "s3" {
    bucket       = "my-terraform-state-bucket2z"
    key          = "demo/terraform.tfstate"
    region       = "us-east-1"
    encrypt      = true
    use_lockfile = true
  }
}

provider "aws" {
  region = "us-east-1"
}

resource "aws_s3_bucket" "example" {
  bucket = "tf-s3-backend-demo-example"
}

3. Initialize the Backend

terraform init

Terraform will initialize and automatically configure the backend.

4. Apply the Changes

terraform apply

Your state file is now stored in S3, and state locking is enabled.

Useful Terraform Backend Commands

Here are some helpful Terraform commands for managing remote state:

Migrate Local State to Remote

terraform init -migrate-state

Reconfigure Backend

terraform init -reconfigure

List State Resources

terraform state list

Inspect a Resource

terraform state show aws_s3_bucket.example

Additional Best Practices

Store State Remotely

Use S3 or Terraform Cloud — never store state locally in team environments.

Don't Edit or Delete State Manually

Manual changes corrupt state.

Isolate State by Environment

Use different keys for different environments:

envs/dev/terraform.tfstate
envs/staging/terraform.tfstate
envs/prod/terraform.tfstate

Regular Backups

Enable S3 versioning so you can roll back state accidentally overwritten or deleted.

Lock State on All Writes

Enable use_lockfile = true. It ensures only one Terraform process can modify state at a time, protecting against conflicts and corruption.

Cleanup: Removing Demo Resources Safely

After completing the demo, it's important to clean up resources to avoid unnecessary AWS costs and to keep your environment tidy.

Important: Always destroy Terraform-managed infrastructure before deleting the remote state backend.

Step 1: Destroy Terraform Resources

From your Terraform project directory, run:

terraform destroy

This command will:

Read the state file from S3
Identify all managed resources
Safely delete them from AWS

Confirm when prompted.

At this point:

All demo infrastructure is removed
The Terraform state file still exists in S3 (this is expected)

Step 2: Verify State File in S3 (Optional)

You can confirm the state file still exists:

aws s3 ls s3://my-terraform-state-bucket2z/demo/

Terraform keeps the state file so you have:

A record of managed infrastructure
Historical versions (if versioning is enabled)

Step 3: Delete the Terraform State File (Optional)

If this was only a demo and you no longer need the state file, you may delete it after destroying resources.

aws s3 rm s3://my-terraform-state-bucket2z/demo/terraform.tfstate

Warning: Never delete a state file while infrastructure still exists — this will orphan resources.

Step 4: (Optional) Disable or Remove Lock Files

If you used S3 native locking, Terraform automatically removes the .tflock file after operations complete.

You normally do not need to manually remove lock files.

Step 5: Delete the S3 Backend Bucket (Optional)

If the S3 bucket was created only for this demo, and you no longer need it:

Empty the Bucket:

aws s3 rm s3://my-terraform-state-bucket2z --recursive

Delete the Bucket:

aws s3api delete-bucket \
  --bucket my-terraform-state-bucket2z \
  --region us-east-1

Cleanup Best Practices

Always run terraform destroy first
Never delete state files before destroying resources
Keep state buckets for long-term projects
Enable versioning for recovery
Restrict access using IAM

Summary

Terraform state is foundational to infrastructure automation.

Storing it in a remote S3 backend with native state locking enabled:

Ensures collaboration
Prevents concurrent modifications
Allows controlled recovery
Supports CI/CD workflows

S3 remote state with native locking is now a best practice recommended by Terraform — and removes the need for a DynamoDB lock table in most cases.

References

Final Thoughts

Terraform state cleanup is just as important as provisioning.

A safe cleanup process ensures:

No orphaned resources
No unexpected AWS charges
Clean environments for future work

When using remote state: Terraform is always the source of truth — let it manage the lifecycle from start to finish.

EKS vs ECS: A Comprehensive Comparison

Tandap Noel Bansikah — Mon, 05 Jan 2026 22:06:18 +0000

What Problem Are We Solving?

Imagine you're running a restaurant. You have multiple chefs (containers) cooking different dishes (applications). Now, who decides:

Which chef works on which dish?
What happens if a chef gets sick mid-service?
How many chefs do you need during the lunch rush vs. a quiet Tuesday afternoon?

In the software world, containers are like those chefs—small, independent workers running your applications. And just like a restaurant needs a manager to coordinate everything, containers need a container orchestrator.

What is Container Orchestration?

Container orchestration is simply automated container management. Think of it as an intelligent manager that:

Task	What It Means
Deployment	Deciding where each container should run
Scaling	Adding more containers when traffic spikes, removing them when it's quiet
Self-healing	Automatically restarting containers that crash
Load balancing	Spreading incoming traffic evenly across containers

Without orchestration, you'd have to manually start, stop, monitor, and manage potentially hundreds of containers. That's not practical—or fun.

Meet the Two AWS Solutions

AWS offers two main container orchestration services:

ECS (Elastic Container Service) — AWS's own solution
EKS (Elastic Kubernetes Service) — Managed Kubernetes on AWS

Let's break down each one.

Amazon ECS: The Simple Choice

What is ECS?

ECS is AWS's homegrown container orchestration service. It's designed to be straightforward and works seamlessly with other AWS services.

Think of ECS as hiring a property management company that only works with buildings in one city (AWS). They know that city inside and out, making everything smooth—but you can't use them if you ever move to another city.

How Does ECS Work?

ECS has two main parts:

Control Plane — The "brain" that decides what runs where (AWS manages this for you—for free!)
Compute — The actual servers that run your containers

Where Do Your Containers Actually Run?

You have two options:

Option 1: EC2 Launch Type

You provide your own virtual servers (EC2 instances), and ECS places containers on them.

Pros:

More control over the underlying servers
Can be cheaper for steady, predictable workloads

Cons:

You manage the servers (updates, security patches, capacity)

Option 2: Fargate Launch Type

AWS handles everything. You just say "run this container" and AWS figures out the rest.

Pros:

Zero server management
Pay only for what you use, down to the second
Perfect for variable workloads

Cons:

Can be more expensive for always-on workloads
Less control over the underlying infrastructure

Amazon EKS: The Kubernetes Option

What is EKS?

EKS is Kubernetes managed by AWS. Kubernetes (often called "K8s") is the industry-standard container orchestrator, originally created by Google.

Think of EKS as hiring an internationally certified property management company. They follow global standards, so if you ever move to another city (another cloud provider), your processes stay the same.

Why Does Kubernetes Exist?

Running Kubernetes yourself is hard. You'd need to:

Set up and secure master nodes (the control plane)
Install and configure multiple complex components
Handle upgrades, backups, and high availability
Keep everything secure and patched

EKS takes care of all the hard parts. AWS manages the control plane, and you just focus on running your applications.

Where Do Your Containers Run on EKS?

Just like ECS, you need compute resources. EKS gives you three options:

Option 1: Self-Managed Nodes

You create and manage your own EC2 instances, installing all the necessary Kubernetes components yourself.

Best for: Teams needing maximum customization or specific compliance requirements.

Option 2: Managed Node Groups (Recommended)

AWS creates and manages EC2 instances for you. Updates and patches happen automatically.

Best for: Most production workloads. You get convenience without giving up too much control.

Option 3: Fargate

Completely serverless. AWS handles everything—you don't even think about servers.

Best for: Variable workloads or teams that want zero infrastructure management.

ECS vs EKS: The Honest Comparison

Let's compare these services across what actually matters:

Learning Curve

ECS	EKS
Easier — Simple concepts, AWS-native terminology	Harder — Must learn Kubernetes concepts plus AWS specifics
Can be productive in days	May take weeks to become comfortable

Bottom line: If you're new to containers, ECS will get you running faster.

Cost

ECS	EKS
Control plane is free	Control plane is paid (per cluster per hour)
Pay only for compute (EC2 or Fargate)	Pay for control plane + compute

Note: For current EKS pricing, refer to the official AWS EKS Pricing page. Pricing varies based on Kubernetes version support tier (standard vs. extended).

Bottom line: ECS is cheaper, especially for smaller deployments.

Vendor Lock-in

ECS	EKS
AWS-only — your knowledge and configurations don't transfer	Kubernetes is universal — skills transfer to any cloud
If AWS changes ECS, you adapt	Kubernetes is open-source with a stable API

Bottom line: EKS gives you more flexibility and portability.

Ecosystem and Tooling

ECS	EKS
Limited to AWS tools	Massive ecosystem: Helm, ArgoCD, Istio, Prometheus, and hundreds more
Smaller community	Huge global community with endless resources

Bottom line: EKS has a richer ecosystem and more community support.

Operational Complexity

ECS	EKS
Simpler to operate day-to-day	More moving parts to understand and manage
Less configuration needed	More powerful, but more configuration required

Bottom line: ECS is easier to operate; EKS offers more power at the cost of complexity.

Making Your Decision: A Simple Framework

Ask yourself these questions:

Choose ECS if:

You're new to containers and want to start quickly
Your team is small and doesn't have Kubernetes experience
You're fully committed to AWS with no plans to use other clouds
You want the simplest possible setup
Cost optimization is a top priority

Choose EKS if:

Your team already knows Kubernetes
You might use multiple clouds in the future (AWS, Azure, GCP)
You need advanced features like service mesh, custom controllers, or complex networking
You want to use popular K8s tools (Helm, ArgoCD, etc.)
You're running large-scale, complex microservices

Real-World Scenarios

Scenario 1: Startup Building Their First App

Situation: A 5-person startup launching their first web application.

Recommendation: ECS with Fargate

Why? They need to move fast, have limited DevOps experience, and shouldn't spend time managing infrastructure. ECS is simpler, and Fargate means zero server management.

Scenario 2: Enterprise Migrating from On-Premises

Situation: A large company moving their existing Kubernetes workloads from on-premises data centers to AWS.

Recommendation: EKS with Managed Node Groups

Why? They already have Kubernetes expertise and existing configurations. EKS lets them reuse their knowledge and tooling with minimal changes.

Scenario 3: Company with Multi-Cloud Strategy

Situation: A company that uses AWS primarily but also deploys to Azure for specific clients.

Recommendation: EKS

Why? Kubernetes skills and configurations work across clouds. They can use similar practices on AWS (EKS) and Azure (AKS).

Quick Reference Cheat Sheet

Question	ECS	EKS
How hard is it to learn?	Easy	Moderate to Hard
What does the control plane cost?	Free	Paid (per cluster per hour)
Can I use it on other clouds?	No	Yes (Kubernetes is universal)
How big is the community?	Small	Massive
Best for beginners?	Yes	Not ideal
Best for Kubernetes users?	No	Yes
Serverless option?	Fargate	Fargate

Final Thoughts

There's no universally "better" choice between ECS and EKS—only what's better for your situation.

ECS is like an automatic car: easy to learn, gets you where you need to go, and requires less attention. Perfect for most drivers.

EKS is like a manual car: more control, more powerful in the right hands, but requires more skill to operate well. Preferred by enthusiasts and professionals.

Start with what matches your team's skills and your project's needs. You can always evolve your choice as your requirements grow.

References

Amazon EKS Pricing - Official AWS pricing page for EKS
Amazon ECS Pricing - Official AWS pricing page for ECS
AWS Fargate Pricing - Official AWS pricing page for Fargate
Amazon EKS Documentation - Official EKS documentation
Amazon ECS Documentation - Official ECS documentation
Kubernetes Documentation - Official Kubernetes documentation

Deploying Applications on Amazon ECS: A Practical Guide

Tandap Noel Bansikah — Fri, 19 Dec 2025 08:47:58 +0000

Deploying Applications on Amazon ECS: A Practical Guide

Introduction

In this article, I will walk you through deploying a containerized application on Amazon ECS (Elastic Container Service) using AWS Fargate. We will use a real full-stack application called CloudNotes that has already been developed and dockerized.

Our focus will be entirely on ECS concepts, the deployment process, and understanding the Terraform infrastructure code. By the end of this article, you will have a clear understanding of how ECS works and how to deploy your own containerized applications on AWS.

Repository: github.com/bansikah22/cloudnotes

What Is Amazon ECS?

Amazon ECS (Elastic Container Service) is a fully managed container orchestration service provided by AWS. It manages how your Docker containers are defined, deployed, scaled, and maintained.

ECS handles the complexity of running containers at scale while integrating deeply with other AWS services like IAM, CloudWatch, VPC, and Application Load Balancers.

ECS Core Concepts

Understanding ECS requires grasping its main objects and how they relate to each other:

Cluster

An ECS Cluster is a logical grouping of tasks or services. It serves as the foundation where your container workloads run. You can think of it as a namespace that organizes your services.

Task Definition

A Task Definition is a blueprint that describes how containers should be deployed. It is unique to ECS and defines:

How much CPU and memory a container will use
The container image to pull
Port mappings
Environment variables
Volumes and other runtime settings

Essentially, a Task Definition contains the same configuration you would normally place in a docker-compose.yml file.

Important Note: Port mapping in ECS (especially with awsvpc network mode on Fargate) must map the same port to the same port (e.g., 3000:3000). Mappings like 3000:3001 will not work because ECS does not allow arbitrary host-to-container remapping in this mode.

Task

A Task is an instance of a Task Definition. It represents a running copy of your application. The Task contains one or more running containers defined by the Task Definition.

Service

An ECS Service ensures that a specified number of Tasks are running at all times. It provides:

Automatic restart of tasks if they crash or exit
Rescheduling of tasks on healthy instances if an EC2 instance fails (in EC2 mode)
Integration with load balancers to distribute traffic across tasks

ECS Load Balancers

An Application Load Balancer (ALB) can be assigned to an ECS Service. It routes external traffic to tasks through a Target Group. The ECS service automatically registers and deregisters tasks to the Target Group as they start and stop.

Launch Types: Fargate vs EC2

ECS offers two launch types:

Launch Type	Description	Best For
Fargate	Serverless - AWS manages the underlying infrastructure	Most workloads, simpler operations
EC2	You manage the EC2 instances that host your containers	GPU workloads, specific instance requirements

For this guide, we use Fargate because it eliminates server management, you pay only for resources used, and it handles infrastructure scaling automatically.

The CloudNotes Application

The application we are deploying is a full-stack ToDo Notes application. It has already been developed and dockerized, so our focus remains on the ECS deployment.

Tech Stack:

Frontend: React, Vite, TypeScript, served via Nginx
Backend: Node.js, Express, TypeScript
Infrastructure: Terraform, AWS ECS Fargate

The Docker images are available on Docker Hub:

bansikah/cloudnotes-frontend:latest
bansikah/cloudnotes-backend:latest

Getting Started

To follow along with this guide, clone the repository and explore the infrastructure code:

git clone https://github.com/bansikah22/cloudnotes.git
cd cloudnotes

Prerequisites

Before deploying, ensure you have:

AWS CLI installed and configured with appropriate credentials (aws configure)
Terraform v1.0+ installed
Docker installed (for local testing if needed)

Understanding the Terraform Infrastructure

The infrastructure code is located in the infrastructure/ directory. Let me walk you through the key components.

VPC Configuration

We use the terraform-aws-modules VPC module to create a network with public and private subnets across two availability zones:

module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "~> 5.0"

  name = "cloudnotes-vpc"
  cidr = "10.0.0.0/16"

  azs             = slice(data.aws_availability_zones.available.names, 0, 2)
  public_subnets  = ["10.0.1.0/24", "10.0.2.0/24"]
  private_subnets = ["10.0.101.0/24", "10.0.102.0/24"]

  enable_nat_gateway = true
  single_nat_gateway = true
}

The ALB sits in public subnets while ECS tasks run in private subnets, accessing the internet through the NAT Gateway.

ECS Cluster

The cluster is simply a logical grouping:

resource "aws_ecs_cluster" "cloudnotes" {
  name = "cloudnotes-cluster"
}

Task Definitions

Here is the backend task definition showing how we define container configurations:

resource "aws_ecs_task_definition" "backend" {
  family                   = "cloudnotes-backend"
  requires_compatibilities = ["FARGATE"]
  network_mode             = "awsvpc"
  cpu                      = var.backend_cpu
  memory                   = var.backend_memory
  execution_role_arn       = aws_iam_role.ecs_task_execution_role.arn
  task_role_arn            = aws_iam_role.ecs_task_role.arn
  container_definitions    = jsonencode(local.backend_container)
}

The container definition includes the image, port mappings, environment variables, and logging configuration:

backend_container = [
  {
    name      = "backend"
    image     = var.backend_image
    essential = true
    portMappings = [
      {
        containerPort = 5000
        hostPort      = 5000
        protocol      = "tcp"
      }
    ]
    environment = [
      {
        name  = "NODE_ENV"
        value = "production"
      }
    ]
    logConfiguration = {
      logDriver = "awslogs"
      options = {
        "awslogs-group"         = aws_cloudwatch_log_group.backend.name
        "awslogs-region"        = var.aws_region
        "awslogs-stream-prefix" = "backend"
      }
    }
  }
]

ECS Services

The service ensures our tasks are always running and connects them to the load balancer:

resource "aws_ecs_service" "backend" {
  name            = "cloudnotes-backend"
  cluster         = aws_ecs_cluster.cloudnotes.id
  task_definition = aws_ecs_task_definition.backend.arn
  desired_count   = var.desired_count
  launch_type     = "FARGATE"

  network_configuration {
    subnets         = module.vpc.private_subnets
    security_groups = [aws_security_group.backend.id]
    assign_public_ip = false
  }

  load_balancer {
    target_group_arn = aws_lb_target_group.backend.arn
    container_name   = "backend"
    container_port   = 5000
  }

  health_check_grace_period_seconds = 60

  depends_on = [aws_lb_listener_rule.api_path]
}

Application Load Balancer with Path-Based Routing

The ALB distributes traffic between frontend and backend services using path-based routing:

resource "aws_lb" "cloudnotes" {
  name               = "cloudnotes-alb"
  internal           = false
  load_balancer_type = "application"
  security_groups    = [aws_security_group.alb.id]
  subnets            = module.vpc.public_subnets
}

resource "aws_lb_listener" "http" {
  load_balancer_arn = aws_lb.cloudnotes.arn
  port              = "80"
  protocol          = "HTTP"

  default_action {
    type             = "forward"
    target_group_arn = aws_lb_target_group.frontend.arn
  }
}

resource "aws_lb_listener_rule" "api_path" {
  listener_arn = aws_lb_listener.http.arn
  priority     = 100

  action {
    type             = "forward"
    target_group_arn = aws_lb_target_group.backend.arn
  }

  condition {
    path_pattern {
      values = ["/api/*"]
    }
  }
}

This routes all /api/* requests to the backend service and everything else to the frontend.

Security Groups

The security groups control network access:

ALB security group allows inbound HTTP/HTTPS from the internet
Frontend security group allows traffic only from the ALB on port 80
Backend security group allows traffic only from the ALB on port 5000

resource "aws_security_group" "backend" {
  name        = "cloudnotes-backend-sg"
  description = "Allow only ALB to access backend"
  vpc_id      = module.vpc.vpc_id

  ingress {
    from_port       = 5000
    to_port         = 5000
    protocol        = "tcp"
    security_groups = [aws_security_group.alb.id]
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

IAM Roles

Two IAM roles are created:

Task Execution Role: Allows ECS to pull images and write logs to CloudWatch
Task Role: Grants permissions to the running application itself

Deploying the Infrastructure

Navigate to the infrastructure directory and run:

cd infrastructure

# Initialize Terraform
terraform init

# Validate the configuration
terraform validate

# Review the execution plan
terraform plan

# Deploy the infrastructure
terraform apply

Type yes when prompted to confirm the deployment.

Observing the Deployed Application

Once the deployment completes, Terraform outputs the ALB DNS name. You can access your application using this URL.

ECS Cluster

The cluster dashboard shows your services and their status:

Running Services

Both frontend and backend services are running with their desired task counts:

Task Definition

The task definition view shows container configurations, resource allocations, and logging settings:

The Running Application

Access the application via the ALB DNS name to see CloudNotes in action:

Architecture

The architecture follows a standard pattern for containerized applications on AWS:

Users access the application through the Application Load Balancer
The ALB sits in public subnets and routes traffic based on URL paths
Frontend requests go to the frontend ECS service
API requests (/api/*) go to the backend ECS service
Both services run as Fargate tasks in private subnets
Tasks pull images from Docker Hub and send logs to CloudWatch
A NAT Gateway enables outbound internet access for the private subnets

Clean Up

To avoid ongoing AWS charges, destroy the infrastructure when you are done:

terraform destroy --auto-approve

Summary

In this article, we covered:

What Amazon ECS is and its core components (Cluster, Service, Task Definition, Task)
How traffic flows from users through the ALB to ECS services
The Terraform code that provisions the entire infrastructure
How to deploy and observe a running application on ECS Fargate

ECS provides a straightforward way to run containers on AWS without managing the underlying infrastructure. With Fargate, you can focus entirely on your application while AWS handles server provisioning, scaling, and maintenance.

What is Next

In upcoming articles, we will explore:

ECS vs EKS: A detailed comparison to help you choose the right service

Resources

Repository: github.com/bansikah22/cloudnotes
AWS ECS Documentation: docs.aws.amazon.com/ecs
Terraform AWS Provider: registry.terraform.io/providers/hashicorp/aws

If you have questions or feedback, feel free to drop a comment below. If you found this helpful, consider following for more AWS and DevOps content.

Amazon EKS Series - Part 5: GitOps with Argo CD on EKS

Tandap Noel Bansikah — Thu, 18 Dec 2025 14:54:00 +0000

Introduction

Welcome to the final part of the Amazon EKS at Scale series!

In Part 4, we deployed applications manually using kubectl and explored Helm for package management. While these approaches work, they have limitations:

Manual processes are error-prone
No audit trail of who deployed what and when
Drift can occur between what's in Git and what's running
Rollbacks require remembering previous configurations

In this article, we'll solve these problems by implementing GitOps with Argo CD. You'll learn how to:

Understand GitOps principles and benefits
Install and configure Argo CD on EKS
Deploy a real application (CloudNotes) using GitOps
Enable automated sync and self-healing
Observe deployments through the Argo CD UI

Prerequisites:

EKS cluster from Part 3 running
kubectl configured and working
Basic understanding of Kubernetes Deployments and Services

# Verify cluster connection
kubectl get nodes

What Is GitOps?

GitOps is an operational framework that uses Git as the single source of truth for declarative infrastructure and applications.

The Core Idea

Instead of running commands to deploy applications:

# Traditional approach
kubectl apply -f deployment.yaml

With GitOps, you:

Commit your desired state to Git
An agent (like Argo CD) watches the repository
Automatically applies changes to the cluster

GitOps Principles

Principle	Description
Declarative	The entire system is described declaratively in Git
Versioned	All changes are tracked with Git history
Automated	Approved changes are applied automatically
Self-healing	The system corrects drift from the desired state

Desired State vs Actual State

GitOps continuously compares:

Desired State: What's defined in your Git repository
Actual State: What's currently running in the cluster

If they differ, GitOps reconciles by applying the desired state.

┌─────────────────┐         ┌─────────────────┐
│   Git Repo      │         │   Kubernetes    │
│  (Desired State)│◄───────►│  (Actual State) │
└─────────────────┘  Sync   └─────────────────┘
         │                           │
         └───────── Argo CD ─────────┘
              (Reconciliation)

Why GitOps Matters for EKS

GitOps is particularly powerful for Amazon EKS environments:

1. Auditability

Every deployment is a Git commit:

Who made the change?
When was it made?
What exactly changed?
Why? (commit message)

This is invaluable for compliance and debugging.

2. Easy Rollbacks

Made a bad deployment? Simply revert the Git commit:

git revert HEAD
git push

Argo CD automatically rolls back the cluster.

3. Team Collaboration

Use pull requests for deployment reviews
Enforce approval workflows
No need to share cluster credentials widely

4. Reduced Human Error

No more typos in kubectl commands
No forgotten flags or wrong contexts
Consistent deployments every time

5. Disaster Recovery

Your entire cluster state is in Git. If you lose the cluster:

Create a new EKS cluster
Install Argo CD
Point to your Git repos
Everything redeploys automatically

Introducing Argo CD

Argo CD is a declarative, GitOps continuous delivery tool for Kubernetes.

How Argo CD Works

Runs inside your cluster as a set of Kubernetes controllers
Watches Git repositories for changes
Compares desired state (Git) with actual state (cluster)
Syncs the cluster to match Git (manually or automatically)
Reports status through UI, CLI, and API

Argo CD vs kubectl/Helm

Aspect	kubectl/Helm	Argo CD
Deployment trigger	Manual command	Git commit
State tracking	None	Continuous
Drift detection	Manual	Automatic
Rollback	Manual	Git revert
Audit trail	External logging	Git history
UI	None (CLI only)	Web dashboard

Argo CD Architecture (Simplified)

┌──────────────────────────────────────────────────────┐
│                    Argo CD                           │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐  │
│  │ API Server  │  │ Repo Server │  │ Application │  │
│  │  (UI/CLI)   │  │ (Git sync)  │  │ Controller  │  │
│  └─────────────┘  └─────────────┘  └─────────────┘  │
└──────────────────────────────────────────────────────┘
         │                  │                │
         ▼                  ▼                ▼
    ┌─────────┐       ┌──────────┐    ┌───────────────┐
    │ Web UI  │       │   Git    │    │  Kubernetes   │
    │   CLI   │       │  Repos   │    │    Cluster    │
    └─────────┘       └──────────┘    └───────────────┘

API Server: Provides the web UI and CLI interface
Repo Server: Clones and caches Git repositories
Application Controller: Monitors applications and syncs state

Installing Argo CD on EKS

Let's install Argo CD on our EKS cluster.

Step 1: Create Namespace

kubectl create namespace argocd

Step 2: Install Argo CD

We'll use the official installation manifests:

kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

This installs:

Argo CD API Server
Argo CD Repo Server
Argo CD Application Controller
Redis (for caching)
Dex (for authentication)

Step 3: Wait for Pods to be Ready

kubectl get pods -n argocd --watch

Wait until all pods show Running status (1-2 minutes):

NAME                                               READY   STATUS    RESTARTS   AGE
argocd-application-controller-0                    1/1     Running   0          60s
argocd-applicationset-controller-5f7b9d6b4-xxxxx   1/1     Running   0          60s
argocd-dex-server-6dcf4f8d4b-xxxxx                 1/1     Running   0          60s
argocd-notifications-controller-5c8d7b4c6d-xxxxx   1/1     Running   0          60s
argocd-redis-6976fc7dfc-xxxxx                      1/1     Running   0          60s
argocd-repo-server-7b8d7c4f5-xxxxx                 1/1     Running   0          60s
argocd-server-5f8d7c4f5b-xxxxx                     1/1     Running   0          60s

Press Ctrl+C to stop watching.

Step 4: Expose the Argo CD Server

For this tutorial, we'll use port-forwarding. In production, you'd use an Ingress or LoadBalancer.

kubectl port-forward svc/argocd-server -n argocd 8080:443

Keep this terminal open. The Argo CD UI is now available at: https://localhost:8080

Step 5: Get the Initial Admin Password

Argo CD generates a random password for the admin user:

argocd admin initial-password -n argocd

Or using kubectl:

kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d
echo

Save this password - you'll need it to log in.

Step 6: Access the UI

Open https://localhost:8080 in your browser
Accept the self-signed certificate warning
Log in with:
- Username: admin
- Password: (from Step 5)

You should see the Argo CD dashboard - currently empty since we haven't deployed any applications yet.

Step 7: Install Argo CD CLI (Optional but Recommended)

# Linux
curl -sSL -o argocd-linux-amd64 https://github.com/argoproj/argo-cd/releases/latest/download/argocd-linux-amd64
sudo install -m 555 argocd-linux-amd64 /usr/local/bin/argocd
rm argocd-linux-amd64

# macOS
brew install argocd

# Verify installation
argocd version

argocd login localhost:8080 --username admin --password <your-password> --insecure

The CloudNotes Application

For this tutorial, we'll deploy CloudNotes - a full-stack ToDo Notes application.

Application Overview

Repository: github.com/bansikah22/cloudnotes
Frontend: React + Vite + TypeScript (served by Nginx)
Backend: Node.js + Express + TypeScript
Images: Available on Docker Hub

Why CloudNotes?

Real application - Not just a demo
Multiple services - Frontend and Backend
Already containerized - Docker images ready
Demonstrates GitOps - Multiple manifests to sync

Preparing Kubernetes Manifests for GitOps

For GitOps, we need Kubernetes manifests stored in a Git repository. We'll create them in our EKS series repository.

Recommended Structure

argocd/
├── README.md
├── applications/
│   └── cloudnotes.yaml          # Argo CD Application resource
└── cloudnotes/
    ├── namespace.yaml           # Kubernetes namespace
    ├── nginx-configmap.yaml     # Nginx config for API proxy
    ├── backend-deployment.yaml  # Backend deployment
    ├── backend-service.yaml     # Backend service (ClusterIP)
    ├── frontend-deployment.yaml # Frontend deployment
    └── frontend-service.yaml    # Frontend service (LoadBalancer)

This separation keeps:

Argo CD configuration in applications/
Application manifests in cloudnotes/

Writing the Kubernetes Manifests

Let's create the manifests for CloudNotes.

Namespace

Create argocd/cloudnotes/namespace.yaml:

apiVersion: v1
kind: Namespace
metadata:
  name: cloudnotes
  labels:
    app: cloudnotes
    managed-by: argocd

Backend Deployment

Create argocd/cloudnotes/backend-deployment.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: cloudnotes-backend
  namespace: cloudnotes
  labels:
    app: cloudnotes
    component: backend
spec:
  replicas: 2
  selector:
    matchLabels:
      app: cloudnotes
      component: backend
  template:
    metadata:
      labels:
        app: cloudnotes
        component: backend
    spec:
      containers:
      - name: backend
        image: bansikah/cloudnotes-backend:latest
        ports:
        - containerPort: 5000
        env:
        - name: NODE_ENV
          value: "production"
        - name: PORT
          value: "5000"
        resources:
          requests:
            cpu: "100m"
            memory: "128Mi"
          limits:
            cpu: "500m"
            memory: "512Mi"
        readinessProbe:
          httpGet:
            path: /api/health
            port: 5000
          initialDelaySeconds: 10
          periodSeconds: 5
        livenessProbe:
          httpGet:
            path: /api/health
            port: 5000
          initialDelaySeconds: 15
          periodSeconds: 10

Backend Service

Create argocd/cloudnotes/backend-service.yaml:

apiVersion: v1
kind: Service
metadata:
  name: cloudnotes-backend
  namespace: cloudnotes
  labels:
    app: cloudnotes
    component: backend
spec:
  type: ClusterIP
  selector:
    app: cloudnotes
    component: backend
  ports:
  - protocol: TCP
    port: 5000
    targetPort: 5000

Note: The backend uses ClusterIP (internal only). The frontend's Nginx will proxy API requests to this service.

Nginx ConfigMap (API Proxy)

The frontend is a static React app served by Nginx. When your browser makes API calls to /api/*, we need Nginx to proxy those requests to the backend service.

Create argocd/cloudnotes/nginx-configmap.yaml:

apiVersion: v1
kind: ConfigMap
metadata:
  name: nginx-config
  namespace: cloudnotes
  labels:
    app: cloudnotes
    component: frontend
data:
  default.conf: |
    server {
        listen 80;

        # Serve static frontend files
        location / {
            root /usr/share/nginx/html;
            index index.html index.htm;
            try_files $uri $uri/ /index.html;
        }

        # Proxy API requests to backend service
        location /api/ {
            proxy_pass http://cloudnotes-backend:5000/api/;
            proxy_http_version 1.1;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
        }
    }

Why is this needed?

The frontend runs in your browser (external)
The backend is ClusterIP (internal only)
Without the proxy, API calls from the browser would fail
Nginx acts as a reverse proxy, forwarding /api/* requests to the backend

Frontend Deployment

Create argocd/cloudnotes/frontend-deployment.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: cloudnotes-frontend
  namespace: cloudnotes
  labels:
    app: cloudnotes
    component: frontend
spec:
  replicas: 2
  selector:
    matchLabels:
      app: cloudnotes
      component: frontend
  template:
    metadata:
      labels:
        app: cloudnotes
        component: frontend
    spec:
      containers:
      - name: frontend
        image: bansikah/cloudnotes-frontend:latest
        ports:
        - containerPort: 80
        resources:
          requests:
            cpu: "50m"
            memory: "64Mi"
          limits:
            cpu: "200m"
            memory: "256Mi"
        readinessProbe:
          httpGet:
            path: /
            port: 80
          initialDelaySeconds: 5
          periodSeconds: 5
        livenessProbe:
          httpGet:
            path: /
            port: 80
          initialDelaySeconds: 10
          periodSeconds: 10
        volumeMounts:
        - name: nginx-config
          mountPath: /etc/nginx/http.d/default.conf
          subPath: default.conf
      volumes:
      - name: nginx-config
        configMap:
          name: nginx-config

Important: The volumeMounts section mounts our custom Nginx config, enabling the API proxy to the backend.

Frontend Service

Create argocd/cloudnotes/frontend-service.yaml:

apiVersion: v1
kind: Service
metadata:
  name: cloudnotes-frontend
  namespace: cloudnotes
  labels:
    app: cloudnotes
    component: frontend
spec:
  type: LoadBalancer
  selector:
    app: cloudnotes
    component: frontend
  ports:
  - protocol: TCP
    port: 80
    targetPort: 80

Creating an Argo CD Application

Now we create the Argo CD Application resource that tells Argo CD what to deploy and where.

Understanding the Application CRD

An Argo CD Application is a Custom Resource that defines:

Source: Where to get the manifests (Git repo, path, branch)
Destination: Where to deploy (cluster, namespace)
Sync Policy: How to handle updates (manual/automatic)

The Application Manifest

Create argocd/applications/cloudnotes.yaml:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: cloudnotes
  namespace: argocd
  finalizers:
    - resources-finalizer.argocd.argoproj.io
spec:
  project: default

  source:
    repoURL: https://github.com/bansikah22/eks-at-scale.git
    targetRevision: master
    path: argocd/cloudnotes

  destination:
    server: https://kubernetes.default.svc
    namespace: cloudnotes

  syncPolicy:
    automated:
      prune: true
      selfHeal: true
      allowEmpty: false
    syncOptions:
      - CreateNamespace=true
      - PrunePropagationPolicy=foreground
      - PruneLast=true
    retry:
      limit: 5
      backoff:
        duration: 5s
        factor: 2
        maxDuration: 3m

Breaking Down the Application

Field	Purpose
`metadata.name`	Name of the application in Argo CD
`metadata.namespace`	Must be `argocd`
`spec.project`	Argo CD project (use `default` for now)
`source.repoURL`	Git repository URL
`source.targetRevision`	Branch, tag, or commit to track
`source.path`	Path to manifests within the repo
`destination.server`	Target cluster (in-cluster uses this URL)
`destination.namespace`	Target namespace for resources
`syncPolicy.automated`	Enable automatic sync
`syncPolicy.automated.prune`	Delete resources removed from Git
`syncPolicy.automated.selfHeal`	Revert manual changes to cluster

Deploying via GitOps

Now let's see GitOps in action!

Step 1: Commit and Push the Manifests

First, ensure all your manifests are committed to Git:

cd /path/to/eks-at-scale
git add argocd/
git commit -m "Add CloudNotes GitOps manifests"
git push origin master

Step 2: Apply the Argo CD Application

kubectl apply -f argocd/applications/cloudnotes.yaml

Output:

application.argoproj.io/cloudnotes created

Step 3: Watch Argo CD Sync

Check the application status:

kubectl get applications -n argocd

Output:

NAME         SYNC STATUS   HEALTH STATUS
cloudnotes   Synced        Healthy

Or use the Argo CD CLI:

argocd app get cloudnotes

Step 4: Observe in the UI

Open the Argo CD UI (https://localhost:8080) and you'll see:

The cloudnotes application card
Sync Status: Synced (green)
Health Status: Healthy (green heart)

Click on the application to see:

All Kubernetes resources created
Resource relationships (Deployment -> ReplicaSet -> Pods)
Real-time sync status

Step 5: Verify the Deployment

Check that CloudNotes is running:

# Check pods
kubectl get pods -n cloudnotes

# Check services
kubectl get svc -n cloudnotes

Expected output:

NAME                                   READY   STATUS    RESTARTS   AGE
cloudnotes-backend-xxxxx-xxxxx         1/1     Running   0          2m
cloudnotes-backend-xxxxx-xxxxx         1/1     Running   0          2m
cloudnotes-frontend-xxxxx-xxxxx        1/1     Running   0          2m
cloudnotes-frontend-xxxxx-xxxxx        1/1     Running   0          2m

Step 6: Access the Application

Get the LoadBalancer URL:

kubectl get svc cloudnotes-frontend -n cloudnotes --watch

Once you have the EXTERNAL-IP:

LB_URL=$(kubectl get svc cloudnotes-frontend -n cloudnotes -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')
echo "CloudNotes URL: http://$LB_URL"

Open the URL in your browser to see the CloudNotes application!

Automated Sync and Self-Healing

This is where GitOps truly shines.

Automated Sync in Action

Let's make a change to our manifests and watch Argo CD automatically sync.

1. Update the replica count:

Edit argocd/cloudnotes/frontend-deployment.yaml:

spec:
  replicas: 3  # Changed from 2 to 3

2. Commit and push:

git add argocd/cloudnotes/frontend-deployment.yaml
git commit -m "Scale frontend to 3 replicas"
git push origin master

3. Watch Argo CD sync:

Within 3 minutes (default sync interval), Argo CD will:

Detect the Git change
Compare with cluster state
Apply the new configuration

kubectl get pods -n cloudnotes -l component=frontend --watch

You'll see a third frontend pod being created automatically!

and on ArgoCD dashboard you will see

Self-Healing in Action

Self-healing reverts unauthorized changes made directly to the cluster.

1. Manually delete a pod:

kubectl delete pod -n cloudnotes -l component=frontend --wait=false

2. Watch Argo CD restore it:

kubectl get pods -n cloudnotes -l component=frontend --watch

Within seconds, Argo CD detects the drift and recreates the pod to match the desired state (3 replicas).

3. Try scaling manually:

kubectl scale deployment cloudnotes-frontend -n cloudnotes --replicas=1

Watch as Argo CD reverts this change back to 3 replicas (as defined in Git).

Why This Matters

Accidental deletions are automatically recovered
Unauthorized changes are reverted
Cluster state always matches Git
No manual intervention required

Observing the Deployment

Argo CD UI Features

The Argo CD web interface provides:

Application View:

Visual resource tree showing all Kubernetes objects
Health status for each resource
Sync status (in sync, out of sync, unknown)
Recent sync history

Resource Details:

Click any resource to see its YAML
View events and logs
See relationships between resources

Sync Operations:

Manual sync button
Sync with prune option
Rollback to previous versions

Useful CLI Commands

# List all applications
argocd app list

# Get detailed application info
argocd app get cloudnotes

# View application history
argocd app history cloudnotes

# Manually trigger sync
argocd app sync cloudnotes

# View application logs
argocd app logs cloudnotes

# Rollback to previous version
argocd app rollback cloudnotes 1

Checking Health Status

# Quick status check
kubectl get applications -n argocd

# Detailed health
argocd app get cloudnotes --show-operation

Health statuses:

Healthy: All resources are healthy
Progressing: Resources are being updated
Degraded: Some resources have issues
Suspended: Application is paused
Missing: Resources don't exist yet

GitOps Workflow Summary

Here's the complete GitOps workflow we've implemented:

Developer                    Git                      Argo CD                  EKS Cluster
    │                         │                          │                          │
    │  1. Edit manifests      │                          │                          │
    │─────────────────────────►                          │                          │
    │                         │                          │                          │
    │  2. git commit & push   │                          │                          │
    │─────────────────────────►                          │                          │
    │                         │                          │                          │
    │                         │  3. Detect changes       │                          │
    │                         │◄─────────────────────────│                          │
    │                         │                          │                          │
    │                         │                          │  4. Apply manifests      │
    │                         │                          │─────────────────────────►│
    │                         │                          │                          │
    │                         │                          │  5. Report status        │
    │                         │                          │◄─────────────────────────│
    │                         │                          │                          │
    │  6. View in UI/CLI      │                          │                          │
    │◄─────────────────────────────────────────────────────                         │

Clean Up

When you're done experimenting:

Delete the Argo CD Application

# This will delete all CloudNotes resources
argocd app delete cloudnotes --cascade

# Or using kubectl
kubectl delete application cloudnotes -n argocd

Uninstall Argo CD (Optional)

kubectl delete -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml
kubectl delete namespace argocd

Delete the EKS Cluster

If you're done with the entire series:

cd eks-terraform
terraform destroy --auto-approve

Summary

In this article, we covered:

GitOps Fundamentals - Git as the single source of truth
Why GitOps Matters - Auditability, rollbacks, collaboration, disaster recovery
Argo CD - A powerful GitOps tool for Kubernetes
Installation - Setting up Argo CD on EKS
Application Deployment - Deploying CloudNotes via GitOps
Automated Sync - Changes in Git automatically applied
Self-Healing - Drift detection and automatic correction

Key Takeaways

Git is the source of truth - All desired state lives in version control
Argo CD automates deployments - No more manual kubectl apply
Changes are auditable - Every deployment is a Git commit
Self-healing ensures consistency - The cluster always matches Git
Rollbacks are simple - Just revert the Git commit
Team collaboration improves - Use PRs for deployment reviews

Series Conclusion

Congratulations! You've completed the Amazon EKS at Scale series!

Over five articles, you've learned:

Part	Topic	Key Skills
1	Introduction to EKS	Understanding managed Kubernetes, worker nodes
2	EKS Architecture	Control plane, networking, IAM integration
3	Provisioning with Terraform	Infrastructure as Code, VPC, EKS modules
4	Deploying Applications	Deployments, Services, LoadBalancers
5	GitOps with Argo CD	Declarative deployments, automated sync, self-healing

You now have a complete, production-ready workflow:

Infrastructure (Terraform) → Cluster (EKS) → Applications (GitOps/Argo CD)

What's Next?

Here are some areas to explore further:

Secrets Management - HashiCorp Vault, AWS Secrets Manager, Sealed Secrets
CI/CD Integration - GitHub Actions, Jenkins, GitLab CI with Argo CD
Multi-Cluster - Argo CD managing multiple EKS clusters
Advanced Argo CD - ApplicationSets, App of Apps pattern
Monitoring - Prometheus, Grafana, CloudWatch Container Insights
Service Mesh - Istio, AWS App Mesh for advanced networking

Resources

Thank You!

Thank you for following along with this series!

If you found it helpful:

Star the GitHub repository
Share with others learning Kubernetes
Drop your questions and feedback in the comments

Happy deploying!

This concludes the Amazon EKS at Scale series. Stay tuned for more DevOps and cloud-native content!

Amazon EKS Series - Part 4: Deploying Applications on Amazon EKS

Tandap Noel Bansikah — Wed, 17 Dec 2025 15:50:54 +0000

Introduction

Welcome back to the Amazon EKS at Scale series!

In Part 3, we provisioned a production-ready EKS cluster using Terraform and community modules. Now it's time to put that cluster to work!

In this article, you'll learn how to:

Deploy a simple application using Kubernetes
Understand Deployments and Pods
Expose applications with Services
Use AWS Load Balancers with EKS
Get started with Helm for package management

Prerequisites: Ensure your EKS cluster from Part 3 is running and kubectl is configured.

# Verify cluster connection
kubectl get nodes

You should see your worker nodes in Ready status.

Choosing a Sample Application

When learning Kubernetes, it's best to start with simple applications. This lets you focus on Kubernetes concepts rather than application complexity.

Why Simple Applications?

Fewer moving parts — Easier to debug
Quick feedback — Fast deployment cycles
Focus on fundamentals — Learn the platform, not the app

Our Choice: NGINX

We'll deploy NGINX — a lightweight web server that:

Starts quickly
Has a small image size
Returns a simple webpage (easy to verify)
Is widely used in examples and documentation

Later, we'll also explore deploying with Helm using a different application.

Understanding Kubernetes Deployments

Before we deploy anything, let's understand what a Deployment is.

What is a Deployment?

A Deployment is a Kubernetes resource that:

Declares the desired state of your application
Manages Pods (the smallest deployable units)
Handles updates with rolling deployments
Ensures availability by maintaining replica counts

Think of it as telling Kubernetes: "I want 3 copies of this application running at all times." Kubernetes then makes it happen and keeps it that way.

What is a Pod?

A Pod is the smallest unit in Kubernetes:

Contains one or more containers
Shares networking and storage
Has a unique IP address within the cluster
Is ephemeral (can be replaced at any time)

Important: You rarely create Pods directly. Instead, you create Deployments that manage Pods for you.

Deployment YAML Structure

Here's what a minimal Deployment looks like:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.25
        ports:
        - containerPort: 80

Breaking Down the YAML

Field	Purpose
`apiVersion`	API version for this resource type
`kind`	Type of resource (Deployment)
`metadata.name`	Name of the Deployment
`spec.replicas`	Number of Pod copies to run
`spec.selector`	How the Deployment finds its Pods
`spec.template`	Template for creating Pods
`containers`	List of containers in each Pod
`image`	Docker image to use
`containerPort`	Port the container listens on

New to kubernetes? you can read my Kubernetes for beginners post.

Deploying Our First Application

Let's deploy NGINX to our EKS cluster.

Step 1: Create the Deployment

Create a file named nginx-deployment.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.25
        ports:
        - containerPort: 80
        resources:
          requests:
            cpu: "100m"
            memory: "128Mi"
          limits:
            cpu: "250m"
            memory: "256Mi"

Note: We've added resource requests and limits. This is a best practice that helps Kubernetes schedule Pods efficiently and prevents any single Pod from consuming too many resources.

Step 2: Apply the Deployment

kubectl apply -f nginx-deployment.yaml

Output:

deployment.apps/nginx-deployment created

Step 3: Verify the Deployment

Check the Deployment status:

kubectl get deployments

Output:

NAME               READY   UP-TO-DATE   AVAILABLE   AGE
nginx-deployment   3/3     3            3           30s

Check the Pods:

kubectl get pods

Output:

NAME                                READY   STATUS    RESTARTS   AGE
nginx-deployment-6d6565499c-2xjkl   1/1     Running   0          45s
nginx-deployment-6d6565499c-8mzft   1/1     Running   0          45s
nginx-deployment-6d6565499c-vwxyz   1/1     Running   0          45s

All 3 replicas are running!

Step 4: View More Details

For detailed information about a Pod:

kubectl describe pod <pod-name>

To see which nodes the Pods are running on:

kubectl get pods -o wide

Understanding Kubernetes Services

Our NGINX Pods are running, but how do we access them?

The Problem

Pods have IP addresses, but they're internal to the cluster
Pods are ephemeral — they can be replaced at any time
Each Pod has a different IP address

If you try to access a Pod directly by IP, what happens when that Pod dies and a new one takes its place with a different IP?

The Solution: Services

A Service provides:

Stable endpoint — A single IP/DNS name that doesn't change
Load balancing — Distributes traffic across all matching Pods
Service discovery — Other apps can find your service by name

Service Types

Kubernetes offers several Service types:

Type	Description	Use Case
`ClusterIP`	Internal IP only (default)	Internal communication between services
`NodePort`	Exposes on each node's IP at a static port	Development, testing
`LoadBalancer`	Provisions an external load balancer	Production traffic from internet

ClusterIP (Default)

Only accessible within the cluster
Great for internal services (databases, caches, etc.)
Other Pods can access via service-name.namespace.svc.cluster.local

NodePort

Exposes the service on each node's IP at a specific port (30000-32767)
Accessible from outside the cluster via <NodeIP>:<NodePort>
Not recommended for production (no load balancing across nodes)

LoadBalancer

Best for production workloads on AWS
Automatically provisions an AWS Elastic Load Balancer (ELB)
Provides a public DNS name
Handles SSL termination (with proper configuration)

Exposing Our Application with LoadBalancer

Let's create a LoadBalancer Service to expose our NGINX deployment to the internet.

Step 1: Create the Service

Create a file named nginx-service.yaml:

apiVersion: v1
kind: Service
metadata:
  name: nginx-service
  labels:
    app: nginx
spec:
  type: LoadBalancer
  selector:
    app: nginx
  ports:
  - protocol: TCP
    port: 80
    targetPort: 80

Understanding the Service YAML

Field	Purpose
`type: LoadBalancer`	Tells Kubernetes to provision an external load balancer
`selector.app: nginx`	Routes traffic to Pods with label `app: nginx`
`port: 80`	Port the Service listens on
`targetPort: 80`	Port on the Pods to forward traffic to

Step 2: Apply the Service

kubectl apply -f nginx-service.yaml

Output:

service/nginx-service created

Step 3: Watch the Service Status

kubectl get service nginx-service --watch

Initially, you'll see <pending> for the external IP:

NAME            TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
nginx-service   LoadBalancer   172.20.10.123   <pending>     80:31234/TCP   10s

After 1-2 minutes, AWS provisions the load balancer:

NAME            TYPE           CLUSTER-IP      EXTERNAL-IP                                                              PORT(S)        AGE
nginx-service   LoadBalancer   172.20.10.123   a1b2c3d4e5f6g7h8i9.us-east-1.elb.amazonaws.com   80:31234/TCP   90s

Press Ctrl+C to stop watching.

What Happened Behind the Scenes?

When you created the LoadBalancer Service:

Kubernetes detected the Service type and called the AWS cloud controller
AWS provisioned a Classic Load Balancer (or NLB with annotations)
The ELB was configured to forward traffic to your worker nodes
kube-proxy on each node routes traffic to the correct Pods

This integration is one of the powerful benefits of running Kubernetes on AWS!

Verifying the Application

Get the Load Balancer URL

kubectl get service nginx-service -o jsonpath='{.status.loadBalancer.ingress[0].hostname}'

Or simply:

kubectl get svc nginx-service

Access via Browser

Open the EXTERNAL-IP URL in your browser:

http://a1b2c3d4e5f6g7h8i9.us-east-1.elb.amazonaws.com

You should see the default NGINX welcome page!

Access via curl

# Store the LoadBalancer URL in a variable and curl it
LB_URL=$(kubectl get svc nginx-service -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')
curl http://$LB_URL

Output:

<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
...
</html>

Check Pod Logs

View the access logs from NGINX:

kubectl logs -l app=nginx --tail=10

You'll see HTTP request logs from your browser/curl requests.

Scaling the Application

One of Kubernetes' strengths is easy scaling.

Scale Up

kubectl scale deployment nginx-deployment --replicas=5

Verify:

kubectl get pods

You now have 5 NGINX Pods!

Scale Down

kubectl scale deployment nginx-deployment --replicas=2

Kubernetes will gracefully terminate 3 Pods.

Autoscaling (Preview)

For automatic scaling based on CPU/memory, you can use the Horizontal Pod Autoscaler (HPA). We'll cover this in a future article.

Introduction to Helm

So far, we've been writing YAML files manually. This works, but what happens when:

You need to deploy to multiple environments?
You want to share your deployment with others?
Applications require dozens of YAML files?

What is Helm?

Helm is the package manager for Kubernetes. Think of it like:

apt for Ubuntu
brew for macOS
npm for Node.js

Key Concepts

Concept	Description
Chart	A package containing all Kubernetes manifests for an application
Release	A deployed instance of a chart
Repository	A collection of charts (like a package registry)
Values	Configuration options to customize a chart

Why Use Helm?

Templating — Use variables instead of hardcoded values
Reusability — Package once, deploy many times
Versioning — Track and rollback releases
Community — Thousands of pre-built charts available
Simplicity — Deploy complex apps with one command

Helm vs Raw YAML

Aspect	Raw YAML	Helm
Learning curve	Lower	Slightly higher
Flexibility	Full control	Template-based
Reusability	Copy-paste	Charts and values
Complex apps	Many files to manage	Single command
Environment config	Manual changes	Values files

Installing Helm

Install Helm CLI

# Linux
curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash

# macOS
brew install helm

# Verify installation
helm version

Helm Basics

# Add a chart repository (example with prometheus-community)
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts

# Update repository information
helm repo update

# Search for charts
helm search repo prometheus

Note: Many popular Helm chart repositories like Bitnami have moved to enterprise licensing models. When choosing charts, prefer community-maintained repositories or official project charts. Always check the license and image sources before using a chart in production.

Deploying Apache (Manual Approach)

While Helm is powerful, sometimes the simplest approach is best. Let's deploy Apache using the official Docker Hub image with manual manifests - similar to how we deployed NGINX.

Why Manual Deployment?

No licensing concerns - Uses official Docker Hub images
Full control - You understand exactly what's deployed
Simpler debugging - Fewer abstractions to troubleshoot
Learning value - Reinforces Kubernetes concepts

Step 1: Create the Apache Deployment

Create a file named apache-deployment.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: apache-deployment
  labels:
    app: apache
spec:
  replicas: 3
  selector:
    matchLabels:
      app: apache
  template:
    metadata:
      labels:
        app: apache
    spec:
      containers:
      - name: apache
        image: httpd:2.4
        ports:
        - containerPort: 80
        resources:
          requests:
            cpu: "100m"
            memory: "128Mi"
          limits:
            cpu: "250m"
            memory: "256Mi"

The httpd:2.4 image is the official Apache HTTP Server image from Docker Hub.

Step 2: Create the Apache Service

Create a file named apache-service.yaml:

apiVersion: v1
kind: Service
metadata:
  name: apache-service
  labels:
    app: apache
spec:
  type: LoadBalancer
  selector:
    app: apache
  ports:
  - protocol: TCP
    port: 80
    targetPort: 80

Step 3: Deploy Apache

kubectl apply -f apache-deployment.yaml -f apache-service.yaml

Output:

deployment.apps/apache-deployment created
service/apache-service created

Step 4: Verify the Deployment

Check the pods:

kubectl get pods -l app=apache

Output:

NAME                               READY   STATUS    RESTARTS   AGE
apache-deployment-6c57ff8d-cdfzr   1/1     Running   0          75s
apache-deployment-6c57ff8d-nbdjq   1/1     Running   0          75s
apache-deployment-6c57ff8d-xqv5f   1/1     Running   0          75s

Step 5: Get the LoadBalancer URL

kubectl get svc apache-service --watch

Wait for the EXTERNAL-IP to be assigned (1-2 minutes):

NAME             TYPE           CLUSTER-IP      EXTERNAL-IP                                                              PORT(S)        AGE
apache-service   LoadBalancer   172.20.86.183   a4108c17eb1b441978cb345f8622d540-727057055.us-east-1.elb.amazonaws.com   80:31783/TCP   60s

Step 6: Test the Application

Note: AWS ELB DNS propagation can take 2-5 minutes. If you get a DNS resolution error, wait a moment and try again.

LB_URL=$(kubectl get svc apache-service -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')
curl http://$LB_URL

Output:

<html><body><h1>It works!</h1></body></html>

or YOu can check on the browser :

You now have two applications running on your EKS cluster - NGINX and Apache!

When to Use Helm

Helm shines for:

Complex applications with many interdependent resources
Community charts from trusted sources (check licenses!)
Environment-specific configurations using values files
Release management with rollback capabilities

For simple deployments like web servers, manual manifests are often clearer and easier to maintain.

Example: Using Helm with Community Charts

Here's how you might use Helm with a community-maintained chart:

# Add a repository
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts

# Install with custom values
helm install my-prometheus prometheus-community/prometheus \
  --set server.persistentVolume.enabled=false

# List releases
helm list

# Upgrade a release
helm upgrade my-prometheus prometheus-community/prometheus --set server.replicas=2

# View history
helm history my-prometheus

# Rollback if needed
helm rollback my-prometheus 1

# Uninstall
helm uninstall my-prometheus

Project Structure

For reference, here's the complete file structure for this article:

k8s-manifests/
├── README.md                    # Usage instructions
├── nginx-deployment.yaml        # NGINX Deployment manifest
├── nginx-service.yaml           # NGINX LoadBalancer Service 
├── apache-deployment.yaml       # Apache Deployment manifest
└── apache-service.yaml          # Apache LoadBalancer Service

All files are available in the series repository.

Clean Up

Let's clean up the resources we created:

# Navigate to the k8s-manifests directory
cd k8s-manifests

# Delete NGINX deployment and service
kubectl delete -f nginx-service.yaml
kubectl delete -f nginx-deployment.yaml

# Delete Apache deployment and service
kubectl delete -f apache-service.yaml
kubectl delete -f apache-deployment.yaml

Verify everything is removed:

kubectl get all

Summary

In this article, we covered:

Kubernetes Deployments — Declare and manage your application Pods
Pods — The smallest deployable units in Kubernetes
Services — Provide stable endpoints and load balancing
LoadBalancer Services — Automatically provision AWS ELBs for external access
Scaling — Easily scale applications up or down
Helm — Package manager for Kubernetes that simplifies deployments

Key Takeaways

Deployments manage Pods and ensure your desired state is maintained
Services provide stable networking for ephemeral Pods
LoadBalancer Services integrate with AWS to provision ELBs automatically
Helm simplifies deploying complex applications with a single command
Always use resource requests and limits for production workloads
Labels are crucial for connecting Deployments and Services

Any doubts in the code you can always refer to the github repo Manifest files

What's Next?

In Part 5, we'll introduce GitOps and manage our deployments using Argo CD. You'll learn:

What GitOps is and why it matters
Setting up Argo CD on EKS
Deploying applications from Git repositories
Automated sync and self-healing deployments

Stay tuned!

Resources

Found this article helpful? Follow along with the series and drop your questions in the comments!

Amazon EKS Series - Part 3: Provisioning an EKS Cluster with Terraform

Tandap Noel Bansikah — Wed, 17 Dec 2025 14:33:53 +0000

Introduction

Welcome back to the Amazon EKS at Scale series!

In Part 2, we explored the architecture of Amazon EKS — the control plane, worker nodes, networking, and IAM integration. Now it's time to put that knowledge into practice.

In this article, you'll learn how to:

Set up a Terraform project for EKS
Use community modules for VPC and EKS
Provision a production-ready EKS cluster
Verify your cluster with kubectl

What we're building: A fully functional EKS cluster with a VPC, private subnets, and managed node groups — all defined as code.

Why Terraform for EKS?

While eksctl is great for quick setups (as we saw in Part 1), production environments demand more control and repeatability. That's where Terraform shines.

Benefits of Infrastructure as Code (IaC)

Reproducibility — Create identical clusters across dev, staging, and production environments
Version Control — Track all infrastructure changes in Git with full history
Team Collaboration — Review infrastructure changes through pull requests
Automation — Integrate with CI/CD pipelines for automated deployments
Documentation — Your code serves as living documentation of your infrastructure

Why Community Modules?

We'll use the official Terraform AWS modules:

terraform-aws-modules/vpc/aws
terraform-aws-modules/eks/aws

These modules are:

Production-tested — Used by thousands of organizations
Actively maintained — Regular updates and security patches
Well-documented — Comprehensive examples and documentation
Interview-friendly — Widely recognized in the industry

Tools & Prerequisites

Before we begin, ensure you have the following installed and configured.

Required Tools

Tool	Version	Purpose
AWS CLI	v2.x	AWS authentication
Terraform	>= 1.0	Infrastructure provisioning
kubectl	Latest	Kubernetes cluster interaction

Installing the Tools

AWS CLI

# Linux
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install

# macOS
brew install awscli

# Verify installation
aws --version

Terraform

# Linux (Ubuntu/Debian)
sudo apt-get update && sudo apt-get install -y gnupg software-properties-common
wget -O- https://apt.releases.hashicorp.com/gpg | gpg --dearmor | sudo tee /usr/share/keyrings/hashicorp-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list
sudo apt update && sudo apt install terraform

# macOS
brew tap hashicorp/tap
brew install hashicorp/tap/terraform

# Verify installation
terraform --version

kubectl

# Linux
curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
chmod +x kubectl
sudo mv kubectl /usr/local/bin/

# macOS
brew install kubectl

# Verify installation
kubectl version --client

AWS Credentials

Configure your AWS credentials using one of these methods:

Option 1: AWS Configure (Recommended for beginners)

aws configure

You'll be prompted for:

AWS Access Key ID
AWS Secret Access Key
Default region (e.g., us-east-1)
Default output format (e.g., json)

Option 2: Environment Variables

export AWS_ACCESS_KEY_ID="your-access-key"
export AWS_SECRET_ACCESS_KEY="your-secret-key"
export AWS_DEFAULT_REGION="us-east-1"

Verify AWS Configuration

# Verify AWS CLI is configured
aws sts get-caller-identity

You should see your AWS account ID and user/role information:

{
    "UserId": "AIDAXXXXXXXXXXXXXXXXX",
    "Account": "123456789012",
    "Arn": "arn:aws:iam::123456789012:user/your-username"
}

IAM Permissions Required

Your AWS user/role needs permissions to create:

VPC and networking resources (subnets, NAT gateway, etc.)
EKS clusters and node groups
IAM roles and policies
EC2 instances

For learning purposes, you can use the AdministratorAccess policy. For production, follow the principle of least privilege.

Project Structure

A clean project structure makes your Terraform code maintainable. Here's what we'll create or you can visit this repository to see the code EKS terraform code:

eks-terraform/
├── main.tf          # Main resources (VPC & EKS modules)
├── providers.tf     # Provider configuration
├── versions.tf      # Terraform and provider versions
├── variables.tf     # Input variables
└── outputs.tf       # Output values

Why This Structure?

versions.tf — Pin Terraform and provider versions for consistency
providers.tf — Configure AWS provider settings
variables.tf — Define configurable inputs (region, cluster name, etc.)
main.tf — The core infrastructure definitions
outputs.tf — Expose useful values after deployment

This separation keeps your code organized and easy to navigate.

Step 1: Terraform Versions

First, let's pin our Terraform and provider versions. This ensures consistent behavior across different machines and CI/CD pipelines.

versions.tf

terraform {
  required_version = ">= 1.0"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

Why version pinning matters:

Prevents unexpected breaking changes
Ensures team members use compatible versions
Makes builds reproducible over time

Step 2: Provider Configuration

Configure the AWS provider with the region variable.

providers.tf

provider "aws" {
  region = var.aws_region
}

Simple and clean — the region comes from our variables file.

Step 3: Input Variables

Define the configurable parameters for your infrastructure.

variables.tf

variable "aws_region" {
  description = "AWS region to deploy resources"
  type        = string
  default     = "us-east-1"
}

variable "cluster_name" {
  description = "Name of the EKS cluster"
  type        = string
  default     = "eks-cluster"
}

variable "cluster_version" {
  description = "Kubernetes version for the EKS cluster"
  type        = string
  default     = "1.31"
}

variable "vpc_cidr" {
  description = "CIDR block for VPC"
  type        = string
  default     = "10.0.0.0/16"
}

variable "environment" {
  description = "Environment name (e.g., dev, staging, prod)"
  type        = string
  default     = "dev"
}

Key variables explained:

aws_region — Where your cluster will be deployed
cluster_name — Identifier for your EKS cluster
cluster_version — Kubernetes version (use a recent stable version)
vpc_cidr — IP address range for your VPC (10.0.0.0/16 gives you 65,536 IPs)
environment — Useful for tagging and distinguishing environments

Step 4: Main Configuration (VPC & EKS)

This is where the magic happens. We'll use community modules to create our VPC and EKS cluster.

main.tf

# -----------------------------------------------------------------------------
# Data Sources
# -----------------------------------------------------------------------------
data "aws_availability_zones" "available" {
  filter {
    name   = "opt-in-status"
    values = ["opt-in-not-required"]
  }
}

# -----------------------------------------------------------------------------
# Local Variables
# -----------------------------------------------------------------------------
locals {
  azs = slice(data.aws_availability_zones.available.names, 0, 3)

  tags = {
    Environment = var.environment
    Terraform   = "true"
    Project     = "eks-at-scale"
  }
}

# -----------------------------------------------------------------------------
# VPC Module
# -----------------------------------------------------------------------------
module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "~> 5.0"

  name = "${var.cluster_name}-vpc"
  cidr = var.vpc_cidr

  azs             = local.azs
  private_subnets = [for k, v in local.azs : cidrsubnet(var.vpc_cidr, 4, k)]
  public_subnets  = [for k, v in local.azs : cidrsubnet(var.vpc_cidr, 8, k + 48)]

  enable_nat_gateway   = true
  single_nat_gateway   = true
  enable_dns_hostnames = true

  # Tags required for EKS
  public_subnet_tags = {
    "kubernetes.io/role/elb" = 1
  }

  private_subnet_tags = {
    "kubernetes.io/role/internal-elb" = 1
  }

  tags = local.tags
}

# -----------------------------------------------------------------------------
# EKS Module
# -----------------------------------------------------------------------------
module "eks" {
  source  = "terraform-aws-modules/eks/aws"
  version = "~> 20.0"

  cluster_name    = var.cluster_name
  cluster_version = var.cluster_version

  cluster_endpoint_public_access = true

  # Cluster Add-ons
  cluster_addons = {
    coredns                = {}
    eks-pod-identity-agent = {}
    kube-proxy             = {}
    vpc-cni                = {}
  }

  vpc_id     = module.vpc.vpc_id
  subnet_ids = module.vpc.private_subnets

  # EKS Managed Node Group(s)
  eks_managed_node_groups = {
    default = {
      ami_type       = "AL2023_x86_64_STANDARD"
      instance_types = ["t3.medium"]

      min_size     = 2
      max_size     = 4
      desired_size = 2
    }
  }

  # Cluster access entry
  enable_cluster_creator_admin_permissions = true

  tags = local.tags
}

Breaking Down the Configuration

Data Sources

data "aws_availability_zones" "available" { ... }

Fetches available AZs in the region, filtering out any that require opt-in.

Local Variables

locals {
  azs = slice(data.aws_availability_zones.available.names, 0, 3)
  ...
}

Selects the first 3 availability zones
Defines common tags for all resources

VPC Module

The VPC module creates:

VPC with the specified CIDR block
Public subnets — For load balancers and NAT gateways
Private subnets — For worker nodes (more secure)
NAT Gateway — Allows private subnets to access the internet
Required tags — EKS uses these tags to discover subnets for load balancers

Subnet tagging is critical:

kubernetes.io/role/elb = 1 — Public subnets for internet-facing load balancers
kubernetes.io/role/internal-elb = 1 — Private subnets for internal load balancers

EKS Module

The EKS module creates:

EKS Cluster — The managed control plane
Cluster Add-ons — Essential components (CoreDNS, VPC CNI, kube-proxy)
Managed Node Group — Worker nodes with auto-scaling

Node group configuration:

ami_type: AL2023_x86_64_STANDARD — Amazon Linux 2023 (latest EKS-optimized AMI)
instance_types: ["t3.medium"] — 2 vCPUs, 4GB RAM (good for learning)
min_size: 2, max_size: 4 — Auto-scaling boundaries
desired_size: 2 — Initial number of nodes

Note on Instance Types:

For learning and testing purposes, it is possible to use small instance types such as t2.micro. However, due to limited CPU and memory, this setup is not recommended for running real workloads. For a more stable experience, instance types like t3.small or t3.medium are preferred.

Step 5: Outputs

Define useful values to display after deployment.

outputs.tf

output "cluster_name" {
  description = "Name of the EKS cluster"
  value       = module.eks.cluster_name
}

output "cluster_endpoint" {
  description = "Endpoint for EKS control plane"
  value       = module.eks.cluster_endpoint
}

output "cluster_version" {
  description = "Kubernetes version of the cluster"
  value       = module.eks.cluster_version
}

output "cluster_security_group_id" {
  description = "Security group ID attached to the EKS cluster"
  value       = module.eks.cluster_security_group_id
}

output "region" {
  description = "AWS region"
  value       = var.aws_region
}

output "vpc_id" {
  description = "VPC ID"
  value       = module.vpc.vpc_id
}

output "configure_kubectl" {
  description = "Command to configure kubectl"
  value       = "aws eks update-kubeconfig --region ${var.aws_region} --name ${module.eks.cluster_name}"
}

The configure_kubectl output is especially helpful — it gives you the exact command to run after deployment.

Step 6: Deploy the Infrastructure

Now let's provision our EKS cluster!

Initialize Terraform

cd eks-terraform
terraform init

This downloads the required providers and modules. You'll see output like:

Initializing provider plugins...
- Finding hashicorp/aws versions matching "~> 5.0"...
- Installing hashicorp/aws v5.x.x...

Terraform has been successfully initialized!

Review the Plan

terraform validate

terraform plan

This shows what Terraform will create without making any changes. Review the output carefully — you should see resources for:

VPC and subnets
Internet Gateway and NAT Gateway
Route tables
EKS cluster
Managed node group
Security groups

Apply the Configuration

terraform apply

Type yes when prompted to confirm.

Note: This process takes approximately 10-15 minutes. EKS cluster creation is the longest step.

You'll see progress output as resources are created:

module.vpc.aws_vpc.this[0]: Creating...
module.vpc.aws_vpc.this[0]: Creation complete after 2s
...
module.eks.aws_eks_cluster.this[0]: Creating...
module.eks.aws_eks_cluster.this[0]: Still creating... [5m0s elapsed]
module.eks.aws_eks_cluster.this[0]: Creation complete after 9m32s
...
Apply complete! Resources: 65 added, 0 changed, 0 destroyed.

You can also login to your aws account and you will see your newly created cluster

Step 7: Configure kubectl & Verify

Update kubeconfig

After successful deployment, configure kubectl to connect to your cluster:

aws eks update-kubeconfig --region us-east-1 --name eks-cluster

Or use the output from Terraform:

$(terraform output -raw configure_kubectl)

You should see:

Added new context arn:aws:eks:us-east-1:ACCOUNT_ID:cluster/eks-cluster to /home/user/.kube/config

Verify the Cluster

Check that your nodes are ready:

kubectl get nodes

Expected output:

NAME                             STATUS   ROLES    AGE   VERSION
ip-10-0-1-xxx.ec2.internal      Ready    <none>   5m    v1.31.x
ip-10-0-2-xxx.ec2.internal      Ready    <none>   5m    v1.31.x

Check cluster information:

kubectl cluster-info

Verify system pods are running:

kubectl get pods -n kube-system

You should see CoreDNS, kube-proxy, and VPC CNI pods running.

Step 8: Clean Up (Important!)

To avoid unnecessary AWS charges, destroy the resources when you're done:

terraform destroy

Type yes to confirm. This will remove all resources created by Terraform.

Warning: This is irreversible and will delete your entire cluster and VPC.

Best Practices for IaC

Here are key best practices to follow as you work with Terraform:

Use community modules — Don't reinvent the wheel; leverage tested modules
Pin versions — Lock Terraform, provider, and module versions
Keep state remote — Use S3 + DynamoDB for team collaboration (we'll cover this later)
Separate environments — Use workspaces or separate directories for dev/staging/prod
Use variables — Make your code reusable across environments
Tag everything — Tags help with cost tracking and resource management
Review plans carefully — Always check terraform plan before applying
Use .gitignore — Never commit .terraform/ or *.tfstate files

Summary

In this article, we:

Set up a clean Terraform project structure
Used community modules for VPC and EKS
Provisioned a production-ready EKS cluster
Configured kubectl and verified the cluster
Learned IaC best practices

You now have a repeatable, version-controlled way to create EKS clusters!

What's Next?

In Part 4, we'll deploy applications to our EKS cluster and explore:

Deploying a sample application
Understanding Kubernetes Deployments and Services
Exposing applications with Load Balancers
Introduction to Helm for package management

Stay tuned!

Resources

Code Repository

All code from this article is available in the eks-terraform/ directory of the series repository.

Found this article helpful? Follow along with the series and drop a comment with your questions!

Amazon EKS Series - Part 2: EKS Architecture and Core Components

Tandap Noel Bansikah — Wed, 17 Dec 2025 13:40:55 +0000

Introduction

Welcome back to the Amazon EKS at Scale series!

In Part 1, we covered the fundamentals of Amazon EKS — what it is, why to use it, and the different ways to manage worker nodes. We also created our first EKS cluster using eksctl.

In this article, we'll take a deeper look at:

The high-level architecture of Amazon EKS
Control plane components (AWS-managed)
Worker nodes (customer-managed)
Networking fundamentals
IAM and authentication

Understanding this architecture is essential before diving into production deployments.

High-Level View of Amazon EKS

Amazon EKS is a managed Kubernetes service that runs the Kubernetes control plane for you. This means AWS handles the complex, undifferentiated heavy lifting of running Kubernetes, while you focus on deploying and managing your applications.

What AWS Manages vs What You Manage

Component	Managed By
Control Plane (API Server, etcd, Scheduler, Controllers)	AWS
Control Plane High Availability	AWS
Control Plane Security Patches	AWS
Worker Nodes	You (or AWS with Managed Node Groups/Fargate)
Application Deployments	You
Pod Networking	You (with AWS VPC CNI)
IAM Roles and Policies	You

This shared responsibility model allows you to leverage AWS's expertise in running highly available infrastructure while maintaining control over your workloads.

EKS Control Plane (AWS-Managed)

The control plane is the brain of your Kubernetes cluster. In EKS, AWS fully manages this component, running it across multiple Availability Zones for high availability.

Core Control Plane Components

1. Kubernetes API Server (`kube-apiserver`)

The API server is the front door to your Kubernetes cluster:

Exposes the Kubernetes API over HTTPS
Validates and processes all API requests (from kubectl, controllers, and other components)
Acts as the gateway for all cluster operations — creating pods, services, deployments, etc.
Authenticates requests using AWS IAM (via the AWS IAM Authenticator)

When you run kubectl get pods, your request goes to the API server, which retrieves the information from etcd and returns it to you.

2. etcd

etcd is a distributed key-value store that serves as Kubernetes' database:

Stores all cluster state and configuration data
Holds information about pods, services, secrets, ConfigMaps, and more
Provides strong consistency guarantees
In EKS, AWS manages etcd replication across multiple Availability Zones

You never interact with etcd directly — all access goes through the API server.

3. Scheduler (`kube-scheduler`)

The scheduler is responsible for placing pods on nodes:

Watches for newly created pods that have no node assigned
Evaluates resource requirements (CPU, memory, storage)
Considers constraints like node selectors, taints, tolerations, and affinity rules
Selects the most suitable node and binds the pod to it

The scheduler ensures efficient resource utilization across your cluster.

4. Controller Manager (`kube-controller-manager`)

The controller manager runs control loops that regulate cluster state:

Node Controller — Monitors node health and responds when nodes go down
Replication Controller — Ensures the correct number of pod replicas are running
Endpoints Controller — Populates endpoint objects (joins Services and Pods)
Service Account Controller — Creates default service accounts for new namespaces

Each controller watches the current state and works to move it toward the desired state.

Key Characteristics of EKS Control Plane

Runs in an AWS-managed VPC — Isolated from your account and other customers
Highly available — At least two API server instances and three etcd nodes across multiple AZs
Automatically scaled — AWS scales control plane resources based on cluster size
No direct access — You cannot SSH into control plane nodes; you interact only via the API
Automatic updates — AWS handles patching and security updates

Worker Nodes (Customer-Managed)

Worker nodes are the compute resources where your applications actually run. Unlike the control plane, you are responsible for provisioning and managing worker nodes (unless using Fargate).

Worker Node Components

Each worker node runs several Kubernetes components:

1. kubelet

The kubelet is the primary node agent:

Registers the node with the Kubernetes API server
Watches for pods scheduled to its node
Ensures containers are running and healthy
Reports node and pod status back to the control plane
Executes liveness and readiness probes

The kubelet communicates with the container runtime to manage container lifecycle.

2. kube-proxy

kube-proxy handles networking on each node:

Maintains network rules for pod-to-pod communication
Implements Kubernetes Services (ClusterIP, NodePort, LoadBalancer)
Uses iptables or IPVS to route traffic to the correct pods
Enables service discovery within the cluster

3. Container Runtime

The container runtime executes containers:

EKS uses containerd as the default runtime (Docker support was deprecated in Kubernetes 1.24)
Pulls container images from registries (like Amazon ECR)
Creates and manages container processes
Handles container isolation using Linux namespaces and cgroups

Worker Node Options in EKS

Option	Node Management	Scaling	Best For
Self-Managed Nodes	You manage everything	Manual or custom	Full control, custom AMIs
Managed Node Groups	AWS manages provisioning and updates	Auto Scaling Groups	Most production workloads
AWS Fargate	AWS manages everything	Automatic per-pod	Serverless, variable workloads

Networking in EKS

Networking is a critical aspect of any Kubernetes deployment. EKS integrates deeply with AWS networking services.

VPC (Virtual Private Cloud)

Your EKS cluster runs inside a VPC — an isolated virtual network in AWS:

Provides network isolation and security
You define the IP address range (CIDR block)
Contains subnets, route tables, and internet gateways
EKS requires a VPC with subnets in at least two Availability Zones

Subnets

Subnets divide your VPC into smaller network segments:

Public Subnets

Have a route to an Internet Gateway
Resources can have public IP addresses
Used for load balancers and bastion hosts
NAT Gateways are placed here for private subnet internet access

Private Subnets

No direct route to the internet
Access the internet via NAT Gateway (for pulling images, etc.)
Recommended for worker nodes — provides better security
Pods and nodes are not directly accessible from the internet

AWS VPC CNI (Container Network Interface)

EKS uses the Amazon VPC CNI plugin for pod networking:

Each pod gets a real IP address from your VPC CIDR range
Pods can communicate directly with other AWS services (RDS, ElastiCache, etc.)
No overlay network — native VPC networking performance
Supports security groups for pods (with specific configurations)

How it works:

The CNI plugin attaches Elastic Network Interfaces (ENIs) to worker nodes
Each ENI can have multiple secondary IP addresses
These IPs are assigned to pods running on the node
Pod-to-pod traffic uses native VPC routing

Important consideration: The number of pods per node is limited by the number of ENIs and IPs the instance type supports.

Simplified Network Flow

┌─────────────────────────────────────────────────────────────┐
│                         AWS Cloud                           │
│  ┌───────────────────────────────────────────────────────┐  │
│  │                      Your VPC                          │  │
│  │  ┌─────────────────┐    ┌─────────────────┐           │  │
│  │  │  Public Subnet  │    │  Public Subnet  │           │  │
│  │  │   (AZ-1)        │    │   (AZ-2)        │           │  │
│  │  │  Load Balancer  │    │  NAT Gateway    │           │  │
│  │  └─────────────────┘    └─────────────────┘           │  │
│  │  ┌─────────────────┐    ┌─────────────────┐           │  │
│  │  │ Private Subnet  │    │ Private Subnet  │           │  │
│  │  │   (AZ-1)        │    │   (AZ-2)        │           │  │
│  │  │  Worker Nodes   │    │  Worker Nodes   │           │  │
│  │  │  (Pods)         │    │  (Pods)         │           │  │
│  │  └─────────────────┘    └─────────────────┘           │  │
│  └───────────────────────────────────────────────────────┘  │
│                              │                              │
│  ┌───────────────────────────┴───────────────────────────┐  │
│  │           EKS Control Plane (AWS-Managed)             │  │
│  │     API Server  │  etcd  │  Scheduler  │  Controllers │  │
│  └───────────────────────────────────────────────────────┘  │
└─────────────────────────────────────────────────────────────┘

IAM and Authentication

EKS uses AWS IAM for authentication, providing a secure and familiar way to manage access to your cluster.

How Authentication Works

User runs kubectl command — e.g., kubectl get pods
AWS IAM Authenticator — kubectl uses the AWS CLI to get a token
Token sent to API Server — The token is included in the request
EKS validates the token — Confirms the IAM identity
Kubernetes RBAC — Determines what the user can do

IAM vs Kubernetes RBAC

These are two separate but complementary systems:

Aspect	AWS IAM	Kubernetes RBAC
Purpose	Who can access the cluster	What they can do inside
Scope	AWS account level	Kubernetes cluster level
Managed by	AWS IAM policies	Kubernetes Role/ClusterRole
Example	"User X can call EKS APIs"	"User X can list pods in namespace Y"

The aws-auth ConfigMap

The aws-auth ConfigMap maps IAM identities to Kubernetes users and groups:

apiVersion: v1
kind: ConfigMap
metadata:
  name: aws-auth
  namespace: kube-system
data:
  mapRoles: |
    - rolearn: arn:aws:iam::123456789012:role/NodeInstanceRole
      username: system:node:{{EC2PrivateDNSName}}
      groups:
        - system:bootstrappers
        - system:nodes
  mapUsers: |
    - userarn: arn:aws:iam::123456789012:user/admin
      username: admin
      groups:
        - system:masters

Key points:

Worker nodes use IAM roles mapped in aws-auth to join the cluster
You can map IAM users and roles to Kubernetes groups
The system:masters group has full cluster admin access
EKS also supports EKS access entries (a newer, simpler alternative to aws-auth)

Best Practices for IAM in EKS

Use IAM roles, not users — Roles are more secure and support temporary credentials
Follow least privilege — Grant only the permissions needed
Use IRSA (IAM Roles for Service Accounts) — Allow pods to assume IAM roles for AWS API access
Audit access regularly — Review who has access to your cluster

Summary

In this article, we explored the architecture of Amazon EKS:

EKS is a managed Kubernetes service — AWS handles the control plane, you manage worker nodes and applications
Control plane components — API Server, etcd, Scheduler, and Controller Manager work together to manage cluster state
Worker nodes — Run kubelet, kube-proxy, and container runtime to execute your workloads
Networking — EKS uses VPC networking with the AWS VPC CNI plugin, giving pods real VPC IP addresses
IAM integration — Authentication uses AWS IAM, while authorization uses Kubernetes RBAC

Key Takeaways

The EKS control plane runs in an AWS-managed VPC, highly available across multiple AZs
You never access control plane nodes directly — only through the Kubernetes API
Worker nodes run in your VPC and are your responsibility (unless using Fargate)
Pods get real VPC IP addresses, enabling direct communication with AWS services
IAM handles authentication (who you are), RBAC handles authorization (what you can do)
The aws-auth ConfigMap bridges IAM identities to Kubernetes users

What's Next?

In Part 3, we'll get hands-on and provision an Amazon EKS cluster using Terraform and community modules. You'll learn:

Setting up Terraform for EKS
Using the terraform-aws-eks module
Configuring VPC, subnets, and node groups
Best practices for infrastructure as code

Stay tuned!

Resources

Found this article helpful? Follow along with the series and share your thoughts in the comments!

Amazon EKS Series - Part 1: Introduction to EKS

Tandap Noel Bansikah — Wed, 17 Dec 2025 10:40:15 +0000

Introduction

Welcome to the first part of the Amazon EKS at Scale series! In this series, we'll explore Amazon Elastic Kubernetes Service (EKS) from the ground up, covering everything from basic concepts to advanced cluster management.

In this article, we'll cover:

What is Amazon EKS?
Why use EKS?
Understanding Worker Node options
Creating and connecting to an EKS cluster

What is Amazon EKS?

Amazon Elastic Kubernetes Service (EKS) is a managed Kubernetes service provided by AWS. It simplifies running Kubernetes by handling the complex parts of cluster management for you.

What AWS Manages for You

With EKS, AWS takes care of:

Provisioning and maintaining master nodes — AWS automatically sets up and maintains the master nodes that run the Kubernetes control plane, eliminating the need for you to manage this critical infrastructure.
Installing control plane processes:
- API Server — The front-end for the Kubernetes control plane. It handles all REST requests for modifications to pods, services, and other resources.
- Scheduler — Watches for newly created pods with no assigned node and selects a node for them to run on based on resource requirements.
- etcd — A consistent and highly-available key-value store used as Kubernetes' backing store for all cluster data.
Scaling and backups of the cluster — AWS automatically handles scaling the control plane and maintains backups of your cluster state, ensuring high availability.
Patching and updating control plane components — Security patches and version updates are managed by AWS, keeping your control plane secure and up-to-date.

This means you can focus on deploying your applications rather than managing Kubernetes infrastructure.

Why Use EKS?

There are several compelling reasons to choose EKS:

Simplified Operations — Running and scaling Kubernetes on your own requires significant expertise. You need to manage etcd clusters, configure API servers, and handle upgrades carefully. EKS removes this complexity by providing a production-ready control plane out of the box.
Enhanced Security — Kubernetes security involves network policies, RBAC, secrets management, and more. EKS integrates natively with AWS IAM for authentication, making it easier to implement least-privilege access. Your control plane runs in an AWS-managed VPC, isolated from other customers.
AWS Integration — EKS works seamlessly with services like:
- IAM for authentication and authorization
- VPC for network isolation
- CloudWatch for logging and monitoring
- ELB for load balancing
- ECR for container image storage

This tight integration simplifies building production-grade applications.

Worker Nodes: Your Options

While EKS manages the control plane, you are responsible for the worker nodes (where your applications actually run). There are three ways to set up worker nodes:

1. Self-Managed Nodes

With self-managed nodes, you have complete control but also full responsibility:

Manual EC2 provisioning — You select and launch EC2 instances yourself, choosing instance types, AMIs, and configurations.
Install Kubernetes components — All worker processes must be installed manually:
- kubelet — The primary node agent that ensures containers are running in pods
- kube-proxy — Maintains network rules for pod communication
- Container runtime — Software that runs containers (e.g., containerd, Docker)
Ongoing maintenance — Security patches, OS updates, and Kubernetes version upgrades are your responsibility.
Node registration — You must configure nodes to register with the EKS control plane.

Best for: Teams needing full control over node configuration, custom AMIs, or specific compliance requirements.

2. Managed Node Groups

Managed node groups let AWS handle the heavy lifting while you retain some control:

Automated provisioning — AWS creates and configures EC2 instances for you using EKS-optimized AMIs that come pre-configured with the necessary Kubernetes components.
Simplified lifecycle management — Create, update, or terminate nodes with a single API call. AWS handles rolling updates gracefully.
Auto Scaling integration — Each node group is backed by an Auto Scaling group, allowing automatic scaling based on demand.
Managed updates — AWS can update nodes to new AMI versions while respecting pod disruption budgets.

Best for: Most production workloads where you want a balance of control and convenience.

3. AWS Fargate

Fargate provides a fully serverless approach to running containers:

No node management — You don't provision, configure, or scale EC2 instances. AWS handles all infrastructure.
On-demand compute — Fargate spins up compute resources only when pods are scheduled, and removes them when pods terminate.
Right-sized resources — Based on your pod's CPU and memory requirements, Fargate automatically provisions appropriately sized compute.
Pay-per-use pricing — You only pay for the vCPU and memory your pods actually consume, billed per second.

Best for: Variable or unpredictable workloads, cost optimization, and teams wanting zero infrastructure management.

Creating an EKS Cluster

Prerequisites

To create an EKS cluster, you need to configure several components:

Cluster configuration:
- Cluster name — A unique identifier for your cluster within the region
- Kubernetes version — The version of Kubernetes to run (EKS supports multiple versions)
IAM role for the cluster — An IAM role that grants EKS permissions to:
- Provision nodes — Create and manage EC2 instances for worker nodes
- Access storage — Interact with EBS volumes and other storage services
- Manage secrets — Access AWS Secrets Manager or Systems Manager Parameter Store
Networking:
- VPC and Subnets — The network where your cluster will run. Subnets should span multiple Availability Zones for high availability.
- Security groups — Firewall rules controlling traffic to and from your cluster components

Creating Worker Nodes

After creating the cluster, set up your node group:

Create a Node Group (a group of nodes for your Kubernetes environment)
Select instance type
Define min/max number of nodes
Specify which EKS cluster to connect to

Methods to Create an EKS Cluster

There are three primary ways to create an EKS cluster:

1. AWS Console

The web-based approach - more steps involved but provides a visual interface.

2. eksctl CLI

A command-line tool that simplifies cluster creation:

eksctl create cluster

This single command provisions:

Security groups
VPC
Subnets
Node groups
And more...

3. Infrastructure as Code (IaC)

Using tools like:

Terraform
Pulumi
AWS CloudFormation

Best for: Reproducible, version-controlled infrastructure.

Hands-On Demo: Creating Your First EKS Cluster

Step 1: Install eksctl

Follow the official AWS documentation to install eksctl OR you can use these steps on this github profile to install both kubectl and eksctl, these steps works for Linux, MacOS and Windows Link to github repo

Step 2: Ensure AWS Authentication

Make sure you can awsctl installed so that you can correctly setup your credentials

Step 3: Explore eksctl Commands

# Get help
eksctl --help

# Explore create options
eksctl create --help

# Explore cluster creation options
eksctl create cluster --help

Step 4: Create the Cluster

eksctl create cluster \
  -n cluster1 \
  --nodegroup-name ng1 \
  --region us-east-1 \
  --node-type t2.micro \
  --nodes 2

Breakdown of Each Option

eksctl create cluster

Creates a new Amazon EKS cluster and provisions the required AWS infrastructure.
-n cluster1

Sets the name of the EKS cluster to cluster1.
--nodegroup-name ng1

Defines the name of the managed node group as ng1.
--region us-east-1

Specifies the AWS region where the EKS cluster will be created.
--node-type t2.micro

Chooses the EC2 instance type for worker nodes in the node group.
--nodes 2

Sets the desired number of worker nodes in the node group to 2.

What This Command Creates

An Amazon EKS control plane managed by AWS
A managed EC2 node group named ng1
Two EC2 worker nodes using the t2.micro instance type
Required networking resources (VPC, subnets, and security groups) if they don’t already exist

Wait for the provisioning to complete.

It takes quite some time because it needs to create like the whole infrastructure for your, the Subnets, Node groups and so ... when it is done you will have something like this

Step 5: Verify Created Resources

After creation, check the following in AWS Console:

VPC
Subnets
EKS Services
Compute > Node Group
EC2 Instances

First you can search EKS and you will see that our cluster has ben created

and if you look further you will see the version of kubernetes we have

We also have our Node group created with our nodes(EC2) you can see

And if you check the instances you will see that our instances are up and running ...

And you can also see the security group and so

Connecting to Your EKS Cluster

Verify kubectl Configuration

You can verify that your kubeconfig file has been updated.

The kubeconfig file tells kubectl which cluster to connect to and
how to authenticate when communicating with the Kubernetes API server.

kubectl config view

Check Your Nodes

You can also get your worker nodes directly from the terminal, and you can clearly see that using the tool eksctl all the cluster creation and worker nodes connection is automatically done for you

kubectl get nodes

If you see your worker nodes listed, your connection is successful!

Cleaning Up

To avoid unnecessary charges, delete your cluster when done:

eksctl delete cluster -n cluster1

Summary

In this first part of the series, we covered:

What EKS is - A managed Kubernetes service where AWS handles the control plane
Why use EKS - Simplified operations, enhanced security, and AWS integration
Worker node options - Self-managed, Managed Node Groups, and Fargate
Cluster creation methods - Console, eksctl, and IaC
Basic cluster operations - Creating, connecting, and deleting clusters

What's Next?

In Part 2, we'll dive deeper into:

EKS architecture and components
How to Setup an EKS cluster using terraform.

Stay tuned!

Resources

Did you find this article helpful? Follow along with the series and drop a comment with your questions!

Achieving Least Privilege in Docker

Tandap Noel Bansikah — Sat, 08 Nov 2025 11:29:35 +0000

Ensuring containers run with only the permissions they truly need is one of the most effective defenses against compromise. The Docker documentation emphasises least privilege as a core security practice—limit what a container can do, lower the blast radius if it is exploited, and make abuse easier to detect.¹ This guide walks through key controls Docker exposes, and finishes with a repeatable hands-on exercise you can run locally.

1. What “Least Privilege” Means for Containers

At its heart, least privilege is the idea that workloads only need:

the minimum filesystem access required to operate
a non-root identity, so the container cannot trivially take over the host
only the Linux capabilities that map to the tasks it performs
explicit resource limits to contain runaway processes

Docker exposes controls for every one of these dimensions through both the Dockerfile (build-time choices) and docker run flags (runtime policy).

2. Build-Time Controls (Dockerfile)

The official guidelines highlight three quick wins:¹

Use a minimal base image (e.g., distroless or Alpine) so the container inherits fewer default binaries and attack surface.
Create and switch to a non-root user with only the filesystem paths it needs.
Scope file ownership and permissions so write access is limited.

# Dockerfile
FROM python:3.12-slim

# Create a system user and group
RUN addgroup --system app && adduser --system --ingroup app app

# Copy application code and drop ownership to the app user
WORKDIR /opt/service
COPY --chown=app:app requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY --chown=app:app . .

# Run as the non-root user
USER app

CMD ["python", "app.py"]

Key points:

The USER directive ensures all subsequent layers (and the running process) inherit the non-root identity.
--chown on COPY prevents root-owned artefacts, which could otherwise allow privilege escalation if the runtime tried to write to those files.

3. Runtime Controls (`docker run`)

Even with a hardened image, Docker defaults to granting capabilities beyond what most applications need. Official docs recommend dropping all capabilities by default, then re-adding only the ones the workload relies on.²

Here are the runtime flags we will use in the practical section:

Flag	Why it matters
`--user`	Overrides the container’s user ID at runtime. Using the same non-root ID as the Dockerfile keeps both layers consistent.
`--cap-drop ALL` and `--cap-add`	Remove every capability, then add back a handful—for example `NET_BIND_SERVICE` for binding to privileged ports.
`--read-only`	Mount the container filesystem as read-only; pair with writable volumes for state that must persist.
`--tmpfs`	Provide ephemeral writable space (e.g., `/tmp`) without broad write access elsewhere.
`--security-opt no-new-privileges`	Blocks setuid binaries and other mechanisms from elevating privilege inside the container.
`--memory`, `--cpus`, `--pids-limit`	Resource governance—if the process is compromised it cannot starve the host.

4. Hands-On: Locking Down a Simple Web Service

The following example runs a tiny Flask service with progressively tighter permissions. You can adapt it to your own applications by swapping out the base image and command.

4.1 Prepare the demo

cat <<'EOF' > app.py
from flask import Flask

app = Flask(__name__)

@app.route("/")
def hello():
    return "Least privilege demo!"

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=5000)
EOF

cat <<'EOF' > requirements.txt
flask==3.0.2
EOF

cat <<'EOF' > Dockerfile
FROM python:3.12-slim

RUN addgroup --system app && adduser --system --ingroup app app
WORKDIR /opt/service
COPY --chown=app:app requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY --chown=app:app app.py .

USER app
CMD ["python", "app.py"]
EOF

docker build -t least-privilege-demo:latest .

4.2 Run with default privileges (baseline)

docker run --rm -p 5000:5000 least-privilege-demo:latest

This works, but the container still has many capabilities and a writable root filesystem.

4.3 Re-run with least privilege

docker run --rm -d \
  --name least-priv \
  --user app \
  --read-only \
  --tmpfs /tmp \
  --cap-drop ALL \
  --cap-add NET_BIND_SERVICE \
  --security-opt no-new-privileges \
  --memory 256m \
  --cpus 0.5 \
  least-privilege-demo:latest

What changed:

The filesystem is now read-only, except for the tmpfs mount at /tmp.
Capabilities are stripped down to one—the app can bind to port 5000 (above 1024), so technically it would work even without NET_BIND_SERVICE, but this illustrates how to add back the minimum set.
no-new-privileges ensures setuid binaries cannot escalate.
Resource limits prevent denial-of-service side effects.

Verify the container is healthy:

curl http://localhost:5000
docker exec least-priv id
docker exec least-priv cat /proc/self/status | grep CapEff
docker exec least-priv touch /tmp/test-file && docker exec least-priv ls -l /tmp/test-file

Expected output:

$ curl http://localhost:5000
Least privilege demo!

$ docker exec least-priv id
uid=100(app) gid=101(app) groups=101(app)

$ docker exec least-priv cat /proc/self/status | grep CapEff
CapEff: 0000000000000000

$ docker exec least-priv ls -l /tmp/test-file
-rw-r--r-- 1 app app 0 Nov  8 08:04 /tmp/test-file

The zeroed-out CapEff confirms all capabilities are dropped. Trying to write anywhere other than /tmp should fail:

docker exec least-priv touch /opt/service/newfile
# touch: cannot touch '/opt/service/newfile': Read-only file system

4.4 Clean up

docker rm -f least-priv
rm -f Dockerfile app.py requirements.txt

5. Additional Hardening Ideas

Rootless Docker: Run the Docker daemon itself without root privileges.³
Dedicated AppArmor/SELinux Profiles: Limit system calls and file access even further.
Read-only Secrets: Mount configuration through Docker secrets or Kubernetes ConfigMaps and avoid baking secrets into images.
Supply chain controls: Pin base image digests and verify them before promotion.

6. Key Takeaways

Least privilege is a principle; Docker gives you the tooling to apply it.
Small, non-root images eliminate entire classes of privilege escalation.
Runtime flags let you defend in depth—capabilities, filesystem access, resource limits.
The pattern mirrors Kubernetes security contexts, so what you practice locally carries over to orchestrated environments.

With these habits, containers remain focused on the work they should do—and little else.

Docker Docs — Apply security best practices to Dockerfiles and Runtime privilege & Linux capabilities. ↩
Docker Docs — Linux capabilities in practice. ↩
Docker Docs — Run the Docker daemon as a non-root user (Rootless mode). ↩

Building MCP Servers: Understanding Transport Layers and Core Components

Tandap Noel Bansikah — Tue, 16 Sep 2025 21:20:26 +0000

In my previous article, we explored what the Model Context Protocol (MCP) is and why it's revolutionizing how AI models interact with external systems. Now that we understand MCP's potential, let's dive deep into the implementation details—how to actually build MCP servers that are robust, scalable, and secure.

Building effective MCP servers requires mastering two key areas: the transport mechanisms that enable communication between clients and servers, and the core components (tools, resources, and prompts) that define your server's functionality. Let's explore each in detail.

Transport Mechanisms: How MCP Connects

Think of transport mechanisms as the "delivery methods" for messages between AI models and your MCP server. Just like choosing how to send a package—hand delivery for urgent local items, postal service for standard shipping, or express courier for time-sensitive deliveries—MCP offers different transport options optimized for specific scenarios.

Imagine you're running a restaurant (your MCP server) and customers (AI models) want to place orders. You could take orders through:

Direct conversation (STDIO) - fastest, but only one customer at a time
Phone calls (HTTP) - can handle multiple customers, works remotely
Live updates on a screen (SSE) - great for showing order status in real-time

The transport layer handles all the communication logistics—message formatting, delivery confirmation, error handling—so you can focus on cooking (your server's business logic) rather than managing the ordering system.

STDIO Transport: Direct Process Communication

STDIO transport is like having a private, dedicated phone line between the AI model and your server. When Claude Desktop wants to use your MCP server, it's like hiring a personal assistant (spawning your server process) who sits right next to them and communicates through simple notes passed back and forth.

Think of it as the difference between shouting across a crowded room versus having a quiet, one-on-one conversation. There's no network noise, no interference from other conversations, and no delay—just direct, immediate communication. This is why STDIO is perfect for desktop applications where speed and reliability matter most.

// Server setup with STDIO transport
import { Server } from '@modelcontextprotocol/sdk/server/index.js';
import { StdioServerTransport } from '@modelcontextprotocol/sdk/server/stdio.js';

const server = new Server({
  name: 'my-mcp-server',
  version: '1.0.0',
});

const transport = new StdioServerTransport();
await server.connect(transport);

Technical Details:

Protocol: JSON-RPC over stdin/stdout streams
Process Model: Client spawns server as child process
Latency: Ultra-low (~1ms) - no network overhead
Concurrency: Single client per server instance
Connection: Automatic cleanup when client terminates

When to Use STDIO:

Building tools for Claude Desktop or similar local AI apps
Creating development utilities that need fast response times
When you want simple deployment (just an executable file)
For personal productivity tools on your local machine

Example Scenario: You're building a code analysis tool for developers. Using STDIO transport means Claude Desktop can spawn your server instantly when needed, analyze code with minimal delay, and automatically clean up when done.

HTTP Transport: Web-Scale Communication

HTTP transport is like turning your MCP server into a popular restaurant that takes phone orders. Instead of having one dedicated assistant (STDIO), you now have a phone system that can handle multiple customers calling simultaneously from anywhere in the world.

Just like a restaurant can serve customers who call from different locations, HTTP transport allows multiple AI clients to connect to your server over the internet. You can set up security (like requiring a password to place orders), handle busy periods with load balancing (multiple phone lines), and even serve customers in different time zones. The trade-off is slightly slower service due to the "phone call overhead," but you gain the ability to serve many more customers.

// HTTP server implementation
import { Server } from '@modelcontextprotocol/sdk/server/index.js';
import { HttpServerTransport } from '@modelcontextprotocol/sdk/server/http.js';

const server = new Server({
  name: 'web-mcp-server',
  version: '1.0.0',
});

const transport = new HttpServerTransport({
  port: 3000,
  host: 'localhost'
});

await server.connect(transport);

Technical Details:

Protocol: JSON-RPC over HTTP/HTTPS
Connection Model: Stateless request-response
Latency: Network-dependent (typically 10-100ms+ depending on distance)
Concurrency: Handles multiple simultaneous clients
Scaling: Can use load balancers, reverse proxies, clustering

When to Use HTTP:

Remote Access: Server and client are on different machines
Web Integration: Embedding MCP functionality in web applications
Team Sharing: Multiple developers need access to the same server
Cloud Deployment: Running servers on AWS, Google Cloud, etc.
Production Systems: Need monitoring, logging, and enterprise features

Example Scenario: Your team needs a shared database query tool. Deploy it as an HTTP MCP server on your cloud infrastructure, and multiple team members can access it remotely through their AI clients. Add authentication headers to control access and use HTTPS for secure communication.

Server-Sent Events (SSE): Real-Time Streaming

SSE transport is like having a live sports commentator giving play-by-play updates. Instead of the AI having to repeatedly ask "Are you done yet?" (like checking order status every few minutes), your server can actively broadcast updates as things happen.

Imagine you're processing a large dataset—with SSE, you can send updates like "10% complete... 25% complete... found interesting pattern... 75% complete..." This keeps the AI informed and engaged, rather than leaving it waiting in silence. It's perfect for long-running tasks where you want to show progress, like a pizza tracker showing when your order is being prepared, baked, and out for delivery.

// SSE transport for real-time updates
import { SseServerTransport } from '@modelcontextprotocol/sdk/server/sse.js';

const transport = new SseServerTransport({
  endpoint: '/events',
  port: 3000
});

// Stream real-time updates
transport.sendEvent({
  type: 'progress',
  data: { completed: 75, total: 100 }
});

Technical Details:

Protocol: Server-Sent Events over HTTP
Connection Model: Long-lived, server-to-client streaming
Latency: Near real-time (typically <100ms for event delivery)
Concurrency: Multiple clients can subscribe to event streams
Reliability: Built-in reconnection and event ID tracking

When to Use SSE:

Long Operations: File processing, data analysis, model training
Live Updates: Real-time dashboards, monitoring systems
Progress Tracking: Show completion status for multi-step workflows
Event Notifications: Alert clients about system changes or updates

Example Scenario: You're building an AI-powered data processing pipeline. Use SSE to stream progress updates as files are processed, allowing the AI client to provide real-time feedback to users and make decisions based on intermediate results.

Transport Selection Guide

Transport	Latency	Scalability	Deployment	Security	Best For
STDIO	~1ms	Single client	Executable file	Process isolation	Local tools, Claude Desktop
HTTP	10-100ms+	High (load balancing)	Web server + networking	HTTPS + auth headers	Remote access, web apps
SSE	<100ms	Medium (connection limits)	Web server + event handling	HTTP security + validation	Real-time updates, monitoring

Decision Framework

Choose STDIO when:

Your server will run on the same machine as the AI client
You need the fastest possible response times
You want simple deployment (single executable)
Building for Claude Desktop or similar local applications

Choose HTTP when:

You need remote access (server and client on different machines)
Multiple clients need to use the same server simultaneously
You're integrating with web applications or cloud services
You need enterprise features like load balancing and monitoring

Choose SSE when:

Your operations take significant time and need progress updates
You're building real-time monitoring or dashboard systems
You need to push notifications or alerts to clients
Your workflow involves streaming data or continuous updates

Core Components: The Building Blocks

Think of MCP servers like a Swiss Army knife for AI models. Just as a Swiss Army knife has different tools for different tasks, MCP servers have three types of components that give AI models different capabilities. Understanding these components is like learning what each tool in the knife does and when to use it.

Tools: Executable Actions

Tools are like the power tools in a workshop—they're what allow the AI to actually do things in the real world. Just as a carpenter has different tools for different jobs (saw for cutting, drill for holes, sander for smoothing), your MCP server provides different tools for different actions.

Think of tools as the AI's "hands"—they extend the AI's capabilities beyond just thinking and talking. Want to search through files? There's a tool for that. Need to send an email? Another tool. Process an image? Yet another tool.

The magic happens when you design tools like a good craftsperson organizes their workshop: each tool has a clear purpose, is reliable when used correctly, and comes with clear instructions (documentation) so even a novice can use it safely and effectively.

// Tool definition with comprehensive validation
server.setRequestHandler(ListToolsRequestSchema, async () => ({
  tools: [
    {
      name: 'search_files',
      description: 'Search for files using glob patterns',
      inputSchema: {
        type: 'object',
        properties: {
          pattern: {
            type: 'string',
            description: 'Glob pattern (e.g., "*.ts", "**/*.json")',
            pattern: '^[^/\\\\:*?"<>|]+$' // Security: prevent path traversal
          },
          maxResults: {
            type: 'number',
            minimum: 1,
            maximum: 100,
            default: 50
          }
        },
        required: ['pattern']
      }
    }
  ]
}));

// Tool execution with error handling
server.setRequestHandler(CallToolRequestSchema, async (request) => {
  const { name, arguments: args } = request.params;

  try {
    switch (name) {
      case 'search_files':
        return await handleSearchFiles(validateArgs(args));
      default:
        throw new Error(`Unknown tool: ${name}`);
    }
  } catch (error) {
    return {
      content: [{
        type: 'text',
        text: `Error: ${error.message}`
      }],
      isError: true
    };
  }
});

Tool Design Principles:

1 Single Responsibility: Each tool should do one thing well

✅ Good: search_files - searches for files using patterns
❌ Bad: file_manager - searches, creates, deletes, and modifies files

2 Input Validation: Always validate and sanitize inputs

   // Validate file patterns to prevent security issues
   if (!/^[a-zA-Z0-9*.\-_/]+$/.test(pattern)) {
     throw new Error('Invalid pattern: contains unsafe characters');
   }

3 Error Handling: Provide clear, actionable error messages

   // Instead of: "Error occurred"
   // Use: "File not found: /path/to/file.txt. Check the path and permissions."

4 Rich Documentation: Help AI models understand when and how to use tools

   description: 'Search for files using glob patterns. Use * for wildcards, ** for recursive search. Example: "**/*.js" finds all JavaScript files recursively.'

Technical Implementation Details:

Tools use JSON Schema for input validation, providing type safety and automatic validation
The inputSchema property defines exactly what parameters the tool accepts
Return values should always include a content array with structured responses
Use the isError flag to indicate when operations fail

Resources: Accessible Data

Resources are like a well-organized library that the AI can browse and read from. Just as a library has books, magazines, DVDs, and digital resources all organized with a clear cataloging system (like the Dewey Decimal System), MCP resources use URI patterns to organize different types of information.

Think of resources as the AI's "reference materials." When a student needs to write a research paper, they don't just need tools (pen, computer, printer)—they need access to information (books, articles, databases). Similarly, AI models need both tools to take actions AND resources to understand context and make informed decisions.

The URI system works like library call numbers: file://documents/readme.md is like saying "go to the Documents section, find the README file," while api://users/123/profile means "check the User Records section, look up person #123's profile." This consistent addressing system helps the AI quickly find exactly what it needs.

// Resource listing with metadata
server.setRequestHandler(ListResourcesRequestSchema, async () => ({
  resources: [
    {
      uri: 'file://documents/readme.md',
      name: 'Project README',
      description: 'Main project documentation',
      mimeType: 'text/markdown'
    },
    {
      uri: 'api://users/profile',
      name: 'User Profile',
      description: 'Current user profile data',
      mimeType: 'application/json'
    }
  ]
}));

// Resource reading with content negotiation
server.setRequestHandler(ReadResourceRequestSchema, async (request) => {
  const { uri } = request.params;

  if (uri.startsWith('file://')) {
    const filePath = uri.slice(7);
    const content = await fs.readFile(filePath, 'utf8');

    return {
      contents: [{
        uri,
        mimeType: mime.lookup(filePath) || 'text/plain',
        text: content
      }]
    };
  }

  throw new Error(`Unsupported resource URI: ${uri}`);
});

Resource Design Patterns:

1 URI Schemes: Create consistent, hierarchical patterns

   // File system resources
   'file://documents/readme.md'
   'file://logs/2024/january/app.log'

   // API resources  
   'api://users/123/profile'
   'api://orders/456/items'

   // Database resources
   'db://customers/active'
   'db://products/category/electronics'

2 MIME Types: Enable proper content handling

   // Let AI clients know what type of data they're getting
   mimeType: 'application/json'     // Structured data
   mimeType: 'text/markdown'        // Documentation
   mimeType: 'text/csv'            // Tabular data
   mimeType: 'image/png'           // Binary content

3 Caching Strategy: Implement efficient updates

   // Use ETags to avoid re-reading unchanged resources
   const etag = generateETag(content);
   if (request.headers['if-none-match'] === etag) {
     return { status: 304 }; // Not modified
   }

4 Security Considerations: Always validate access

   // Prevent path traversal attacks
   const safePath = path.resolve(SAFE_ROOT, requestedPath);
   if (!safePath.startsWith(SAFE_ROOT)) {
     throw new Error('Access denied: path outside allowed directory');
   }

Technical Implementation:

Resources are read-only from the AI client's perspective
Use the ListResourcesRequestSchema to advertise available resources
Implement ReadResourceRequestSchema to serve resource content
Support content negotiation through MIME types for optimal AI processing

Prompts: Conversation Templates

Prompts are like having a collection of conversation scripts or templates that help guide interactions. Think of them as the "Mad Libs" of AI conversations—you create a template with blanks to fill in, and the AI can use these templates to start productive conversations.

Imagine you're a manager who frequently needs to conduct different types of meetings. Instead of improvising each time, you might have templates: "Performance Review Template," "Project Kickoff Template," "Problem-Solving Template." Each template has a structure and key questions, but you fill in the specific details for each situation.

MCP prompts work the same way—they provide proven conversation patterns that AI models can use as starting points, customized with specific context for each situation.

Technical Architecture:

Prompts use template variables for dynamic content generation
They return complete conversation contexts with role-based messages
Parameters enable customization without changing the core template
The AI client invokes prompts to establish structured conversation flows

When to Use Prompts:

Standardizing Workflows: Create consistent approaches to common tasks
Domain Expertise: Embed specialized knowledge into conversation starters
Complex Processes: Guide AI through multi-step procedures
Quality Control: Ensure AI interactions follow best practices

// Prompt definition with variables
server.setRequestHandler(ListPromptsRequestSchema, async () => ({
  prompts: [
    {
      name: 'code_review',
      description: 'Generate code review comments',
      arguments: [
        {
          name: 'language',
          description: 'Programming language',
          required: true
        },
        {
          name: 'focus_areas',
          description: 'Areas to focus on (security, performance, etc.)',
          required: false
        }
      ]
    }
  ]
}));

// Prompt rendering with context injection
server.setRequestHandler(GetPromptRequestSchema, async (request) => {
  const { name, arguments: args } = request.params;

  if (name === 'code_review') {
    const language = args?.language || 'javascript';
    const focusAreas = args?.focus_areas || 'general best practices';

    return {
      description: `Code review for ${language}`,
      messages: [
        {
          role: 'user',
          content: {
            type: 'text',
            text: `Please review this ${language} code focusing on ${focusAreas}. 
                   Provide specific, actionable feedback with examples.`
          }
        }
      ]
    };
  }

  throw new Error(`Unknown prompt: ${name}`);
});

Integration Patterns: Making Components Work Together

The real power of MCP comes from how these components work together, like instruments in an orchestra. Each component has its role, but the magic happens when they're coordinated to create something greater than the sum of their parts.

Tool-Resource Synergy

Think of the relationship between tools and resources like a detective and evidence. The detective (AI) uses tools (investigation methods) to gather and analyze resources (evidence, witness statements, documents). Sometimes using a tool creates new evidence that becomes a resource for further investigation.

Tools and resources work together to create powerful workflows:

// Tool that creates resources dynamically
async function handleAnalyzeCode(args: { filePath: string }) {
  // Tool action: analyze the code
  const analysis = await analyzeCodeFile(args.filePath);

  // Create a resource with results
  const resourceUri = `analysis://reports/${Date.now()}`;
  resourceCache.set(resourceUri, {
    content: JSON.stringify(analysis, null, 2),
    mimeType: 'application/json',
    timestamp: new Date()
  });

  return {
    content: [{
      type: 'text',
      text: `Analysis complete. View results at: ${resourceUri}`
    }]
  };
}

Real-World Example:
Imagine building a code review assistant. The AI uses the analyze_code tool to examine a file, which creates an analysis resource. Then it uses the read_resource capability to access the detailed analysis, and finally uses a generate_report tool to create a formatted review document. Each step builds on the previous one, creating a powerful workflow.

Prompt-Driven Workflows

Prompts act like a GPS for complex tasks—they provide step-by-step directions to help the AI navigate from problem to solution. Just as GPS breaks down a long journey into manageable turns ("In 500 feet, turn left, then continue straight for 2 miles"), prompts break down complex workflows into clear, actionable steps.

Prompts can guide AI models through complex multi-step processes:

// Multi-step workflow prompt
const workflowPrompt = {
  name: 'debug_issue',
  arguments: [
    { name: 'error_message', required: true },
    { name: 'component', required: false }
  ],
  messages: [
    {
      role: 'system',
      content: {
        type: 'text',
        text: 'You are a debugging expert. Follow systematic troubleshooting procedures.'
      }
    },
    {
      role: 'user',
      content: {
        type: 'text',
        text: `Debug this ${component || 'system'} issue: "${error_message}"

               Follow this systematic approach:
               1. Use search_files to find relevant code files
               2. Use read_file to examine the problematic code
               3. Use get_logs to check for related errors
               4. Use check_dependencies to verify system state
               5. Provide a diagnosis with specific fix recommendations`
      }
    }
  ]
};

Practical Application:
When a developer encounters a bug, they can invoke the debug_issue prompt with the error message. The AI follows the structured workflow, systematically using different tools to investigate, and provides a comprehensive diagnosis. This ensures consistent, thorough debugging every time.

Performance and Security: Building for the Real World

Building an MCP server is like constructing a building—you need both a solid foundation (security) and efficient systems (performance) to handle real-world usage. Just as architects consider both structural integrity and energy efficiency, MCP developers must balance security and performance.

Performance Optimization

Performance optimization is like organizing your kitchen for efficiency. Just as a chef keeps frequently used ingredients within easy reach and pre-prepares common components, your MCP server should cache expensive operations and optimize for common use patterns.

// Implement caching for expensive operations
const cache = new Map<string, { data: any; timestamp: number }>();

async function cachedOperation(key: string, operation: () => Promise<any>) {
  const cached = cache.get(key);
  const now = Date.now();

  if (cached && (now - cached.timestamp) < 300000) { // 5 min cache
    return cached.data;
  }

  const data = await operation();
  cache.set(key, { data, timestamp: now });
  return data;
}

Security Best Practices

Security in MCP servers is like being a careful bouncer at an exclusive club. You need to check IDs (validate inputs), ensure people don't wander into restricted areas (prevent path traversal), and maintain order while still providing good service to legitimate guests.

// Input validation and sanitization
function validateFilePath(path: string): string {
  // Prevent path traversal
  const normalized = path.normalize(path);
  if (normalized.includes('..') || normalized.startsWith('/')) {
    throw new Error('Invalid path: path traversal detected');
  }

  // Ensure path is within allowed directory
  const resolved = path.resolve(ALLOWED_ROOT, normalized);
  if (!resolved.startsWith(ALLOWED_ROOT)) {
    throw new Error('Access denied: path outside allowed directory');
  }

  return resolved;
}

Putting It All Together: A Practical Example

Let's walk through building a complete MCP server for a development team's needs:

Scenario: Your team needs an AI assistant that can help with code reviews, manage project documentation, and monitor system health.

Transport Decision:

Use HTTP transport because multiple developers need access
Deploy on your internal cloud infrastructure
Enable HTTPS with team authentication

Component Design:

1 Tools (Actions the AI can perform):

   // Code analysis
   'analyze_code' - examines code quality and security
   'run_tests' - executes test suites
   'deploy_staging' - deploys to staging environment

   // Documentation
   'update_docs' - modifies project documentation
   'generate_changelog' - creates release notes

   // Monitoring  
   'check_system_health' - monitors server status
   'get_error_logs' - retrieves recent error logs

2 Resources (Information the AI can access):

   // Code and documentation
   'file://src/**/*.ts' - source code files
   'file://docs/**/*.md' - documentation files

   // System data
   'api://monitoring/metrics' - performance metrics
   'api://logs/errors/recent' - recent error logs
   'db://deployments/history' - deployment history

3 Prompts (Structured workflows):

   'code_review_checklist' - systematic code review process
   'incident_response' - structured incident handling
   'release_preparation' - pre-release validation steps

Result: Developers can ask the AI to "review this pull request" (uses code_review_checklist prompt), "check if the system is healthy" (uses monitoring tools and resources), or "prepare for release" (follows release_preparation workflow).

Conclusion

Building effective MCP servers is like designing a well-equipped workshop—you need the right tools for the job, organized materials you can easily find, and proven procedures for complex tasks.

Key Takeaways:

Transport choice matters: STDIO for local speed, HTTP for remote access, SSE for real-time updates
Design for clarity: Each tool should have a single, clear purpose
Organize systematically: Use consistent URI patterns for resources
Guide with prompts: Provide structured workflows for complex tasks
Secure by default: Always validate inputs and control access
Optimize for real use: Cache expensive operations and monitor performance

Start simple with a few essential tools and resources, then expand based on actual usage patterns. The best MCP servers solve real problems elegantly rather than trying to do everything at once.

Visit this repository to learn more about MCP servers

Kubernetes Security Context: A Comprehensive Guide

Tandap Noel Bansikah — Thu, 11 Sep 2025 19:39:51 +0000

Introduction
What is Security Context?
Security Context Fields
Pod-level vs Container-level Security Context
Practical Examples
Security Context Constraints
Best Practices
Troubleshooting
References

Introduction

Security is a critical aspect of container orchestration, and Kubernetes provides robust mechanisms to enforce security policies at the pod and container level. One of the most important features for container security is the Security Context, which allows you to define privilege and access control settings for pods and containers.

This article provides a comprehensive overview of Kubernetes Security Context, covering both theoretical concepts and practical implementations based on official Kubernetes documentation.

What is Security Context?

A Security Context defines privilege and access control settings for a Pod or Container. It includes settings such as:

User ID (UID) and Group ID (GID) for running processes
Linux capabilities
SELinux options
AppArmor profiles
Seccomp profiles
File system permissions
Privilege escalation controls

Security contexts help enforce the principle of least privilege by ensuring containers run with only the minimum permissions necessary to function correctly.

Security Context Fields

Core Security Context Fields

Based on the official Kubernetes documentation, here are the key fields available in a Security Context:

User and Group Settings

runAsUser: Specifies the user ID to run the container processes
runAsGroup: Specifies the primary group ID for container processes
runAsNonRoot: Indicates that the container must run as a non-root user
fsGroup: Defines a file system group ID for volume ownership

Privilege Controls

privileged: Determines if the container runs in privileged mode
allowPrivilegeEscalation: Controls whether a process can gain more privileges
readOnlyRootFilesystem: Mounts the container's root filesystem as read-only

Linux Security Modules

seLinuxOptions: SELinux security context settings
appArmorProfile: AppArmor profile configuration
seccompProfile: Seccomp profile settings

Capabilities

capabilities: Linux capabilities to add or drop

Pod-level vs Container-level Security Context

Security Context can be specified at two levels:

Pod-level Security Context

Applied to all containers in the pod (unless overridden at container level):

apiVersion: v1
kind: Pod
metadata:
  name: security-context-demo
spec:
  securityContext:
    runAsUser: 1000
    runAsGroup: 3000
    fsGroup: 2000
  containers:
  - name: sec-ctx-demo
    image: busybox:1.28
    command: [ "sh", "-c", "sleep 1h" ]

Container-level Security Context

Applied to specific containers and overrides pod-level settings:

apiVersion: v1
kind: Pod
metadata:
  name: security-context-demo-2
spec:
  containers:
  - name: sec-ctx-demo
    image: busybox:1.28
    command: [ "sh", "-c", "sleep 1h" ]
    securityContext:
      runAsUser: 2000
      allowPrivilegeEscalation: false

Practical Examples

Let's explore various Security Context configurations with practical examples.

Example 1: Basic User and Group Configuration

apiVersion: v1
kind: Pod
metadata:
  name: security-context-demo-1
spec:
  securityContext:
    runAsUser: 1000
    runAsGroup: 3000
    fsGroup: 2000
  containers:
  - name: sec-ctx-vol
    image: busybox:1.28
    command: [ "sh", "-c", "sleep 1h" ]
    volumeMounts:
    - name: sec-ctx-vol
      mountPath: /data/demo
  volumes:
  - name: sec-ctx-vol
    emptyDir: {}

What this does:

Runs the container process as user ID 1000
Sets the primary group ID to 3000
Sets the file system group to 2000 (volumes will be owned by this group)

Example 2: Non-Root User Enforcement

apiVersion: v1
kind: Pod
metadata:
  name: security-context-demo-2
spec:
  securityContext:
    runAsNonRoot: true
    runAsUser: 1000
  containers:
  - name: sec-ctx-demo
    image: gcr.io/google-samples/node-hello:1.0
    securityContext:
      allowPrivilegeEscalation: false
      readOnlyRootFilesystem: true

What this does:

Ensures the container cannot run as root
Prevents privilege escalation
Mounts the root filesystem as read-only

Example 3: Capabilities Management

apiVersion: v1
kind: Pod
metadata:
  name: security-context-demo-3
spec:
  containers:
  - name: sec-ctx-demo
    image: gcr.io/google-samples/node-hello:1.0
    securityContext:
      capabilities:
        add: ["NET_ADMIN", "SYS_TIME"]
        drop: ["ALL"]

What this does:

Drops all default capabilities
Adds only NET_ADMIN and SYS_TIME capabilities

Example 4: SELinux Configuration

apiVersion: v1
kind: Pod
metadata:
  name: security-context-demo-4
spec:
  securityContext:
    seLinuxOptions:
      level: "s0:c123,c456"
  containers:
  - name: sec-ctx-demo
    image: gcr.io/google-samples/node-hello:1.0

Example 5: Seccomp Profile

apiVersion: v1
kind: Pod
metadata:
  name: security-context-demo-5
spec:
  securityContext:
    seccompProfile:
      type: RuntimeDefault
  containers:
  - name: sec-ctx-demo
    image: gcr.io/google-samples/node-hello:1.0

Example 6: Complete Security Hardening

apiVersion: v1
kind: Pod
metadata:
  name: hardened-pod
spec:
  securityContext:
    runAsNonRoot: true
    runAsUser: 10001
    runAsGroup: 10001
    fsGroup: 10001
    seccompProfile:
      type: RuntimeDefault
  containers:
  - name: hardened-container
    image: gcr.io/google-samples/node-hello:1.0
    securityContext:
      allowPrivilegeEscalation: false
      readOnlyRootFilesystem: true
      capabilities:
        drop:
        - ALL
    resources:
      limits:
        memory: "128Mi"
        cpu: "100m"
      requests:
        memory: "64Mi"
        cpu: "50m"
    volumeMounts:
    - name: tmp-volume
      mountPath: /tmp
  volumes:
  - name: tmp-volume
    emptyDir: {}

Security Context Constraints

Pod Security Standards

Kubernetes provides Pod Security Standards that work with Security Context:

Privileged: Unrestricted policy
Baseline: Minimally restrictive policy
Restricted: Heavily restricted policy

Example Pod Security Policy (Deprecated in favor of Pod Security Standards)

apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: restricted
spec:
  privileged: false
  allowPrivilegeEscalation: false
  requiredDropCapabilities:
    - ALL
  volumes:
    - 'configMap'
    - 'emptyDir'
    - 'projected'
    - 'secret'
    - 'downwardAPI'
    - 'persistentVolumeClaim'
  runAsUser:
    rule: 'MustRunAsNonRoot'
  seLinux:
    rule: 'RunAsAny'
  fsGroup:
    rule: 'RunAsAny'

Best Practices

1. Always Run as Non-Root

securityContext:
  runAsNonRoot: true
  runAsUser: 10001

2. Drop All Capabilities by Default

securityContext:
  capabilities:
    drop:
    - ALL

3. Use Read-Only Root Filesystem

securityContext:
  readOnlyRootFilesystem: true

4. Disable Privilege Escalation

securityContext:
  allowPrivilegeEscalation: false

5. Use Seccomp Profiles

securityContext:
  seccompProfile:
    type: RuntimeDefault

6. Set Resource Limits

resources:
  limits:
    memory: "128Mi"
    cpu: "100m"
  requests:
    memory: "64Mi"
    cpu: "50m"

Troubleshooting

Common Issues and Solutions

1. Permission Denied Errors

Problem: Container fails to start due to permission issues
Solution: Check runAsUser, runAsGroup, and fsGroup settings

kubectl logs <pod-name>
kubectl describe pod <pod-name>

2. Capability Issues

Problem: Application requires specific capabilities
Solution: Add required capabilities while dropping unnecessary ones

securityContext:
  capabilities:
    add: ["NET_BIND_SERVICE"]
    drop: ["ALL"]

3. File System Access Issues

Problem: Cannot write to mounted volumes
Solution: Set appropriate fsGroup

securityContext:
  fsGroup: 2000

Debugging Commands

# Check pod security context
kubectl get pod <pod-name> -o yaml | grep -A 10 securityContext

# Exec into pod to check user/group
kubectl exec -it <pod-name> -- id

# Check file permissions
kubectl exec -it <pod-name> -- ls -la /path/to/volume

Testing Security Context

Create a Test Pod

kubectl apply -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
  name: security-test
spec:
  securityContext:
    runAsUser: 1000
    runAsGroup: 3000
    fsGroup: 2000
  containers:
  - name: test-container
    image: busybox:1.28
    command: ["sleep", "3600"]
    securityContext:
      allowPrivilegeEscalation: false
      readOnlyRootFilesystem: true
      capabilities:
        drop:
        - ALL
EOF

Verify Security Settings

# Check user and group
kubectl exec security-test -- id

#output
uid=1000 gid=3000 groups=2000,3000

# Try to escalate privileges (should fail)
kubectl exec security-test -- su -

#output
su: must be suid to work properly
command terminated with exit code 1

# Check capabilities
kubectl exec security-test -- cat /proc/1/status | grep Cap

#output
CapInh: 0000000000000000
CapPrm: 0000000000000000
CapEff: 0000000000000000
CapBnd: 0000000000000000
CapAmb: 0000000000000000

Conclusion

Security Context is a fundamental component of Kubernetes security that provides granular control over how pods and containers run in your cluster. By properly configuring Security Context, you can enhance your cluster's security posture and follow the principle of least privilege.

Key takeaways: start with restrictive settings, test thoroughly, and combine Security Context with other Kubernetes security measures for comprehensive protection. Security is an ongoing process - regularly review and update your configurations as your applications and threat landscape evolve.

This article is based on Kubernetes official documentation and best practices for container security. Always refer to the latest Kubernetes documentation for the most up-to-date information.