DEV Community: Prithiviraj R

EKS-node-viewer: See Your EKS Costs in Real-Time (No Dashboard Needed)

Prithiviraj R — Sat, 21 Feb 2026 12:47:28 +0000

🔥 Meet eks-node-viewer

A tiny CLI tool from AWS Labs. Built originally for Karpenter demos. Now open source. And honestly, underrated.

One command:

eks-node-viewer --resources cpu,memory

You instantly see:

Every node in your cluster
CPU + Memory usage (based on pod requests)
Instance type (t3.medium, c6a.2xlarge...)
On-Demand vs Spot
Cost per hour AND per month
Pods per node

Real-time. In your terminal. No dashboard hopping. No Grafana. No console tab-switching.

🎯 When I Use It

I use eks-node-viewer when:

✅ Debugging uneven pod scheduling across nodes
✅ Checking if Karpenter consolidation is actually working
✅ Quick cost sanity check before a prod deploy
✅ Spotting nodes that are packed too tight (or wasting capacity)
✅ Answering "why is my bill so high?" in 30 seconds

If you run multiple clusters (or even one busy prod cluster), this is a ridiculously fast way to understand what's happening.

🚀 Let's Build a Demo

I'll show you exactly how to use eks-node-viewer with a real EKS cluster.

What we'll do:

Create an EKS cluster (1 node)
Install eks-node-viewer
Deploy workloads
Watch costs in real-time
See Karpenter-style scaling (optional)

Step 1: Create EKS Cluster

First, we need an EKS cluster. I'm using eksctl because it's the fastest way.

Create Cluster Config

cat > eks-node-viewer-cluster.yaml << 'EOF'
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig

metadata:
  name: eks-node-viewer-demo
  region: us-east-1
  version: "1.31"

managedNodeGroups:
  - name: demo-nodes
    instanceType: t3.medium
    minSize: 1
    maxSize: 1
    desiredCapacity: 1
    volumeSize: 20
    labels:
      role: demo
    tags:
      Environment: demo
      Tool: eks-node-viewer
EOF

Create the Cluster

eksctl create cluster -f eks-node-viewer-cluster.yaml

While it's creating, let's install eks-node-viewer.

Step 2: Install eks-node-viewer

Option 1: Using Go (Recommended)

go install github.com/awslabs/eks-node-viewer/cmd/eks-node-viewer@latest

Option 2: Using Homebrew (Mac)

brew tap aws/tap
brew install eks-node-viewer

Option 3: Download Binary

# Download latest release
curl -LO https://github.com/awslabs/eks-node-viewer/releases/latest/download/eks-node-viewer_$(uname -s)_$(uname -m).tar.gz

# Extract
tar -xzf eks-node-viewer_*.tar.gz

# Move to PATH
sudo mv eks-node-viewer /usr/local/bin/
sudo chmod +x /usr/local/bin/eks-node-viewer

Verify Installation

eks-node-viewer --version
# Output: eks-node-viewer version v0.7.4

Step 3: Connect to Cluster

Once your cluster is ready, update your kubeconfig:

# List clusters
aws eks list-clusters --region us-east-1

# Update kubeconfig
aws eks update-kubeconfig --name eks-node-viewer-demo --region us-east-1

# Verify connection
kubectl get nodes

Output:

NAME                             STATUS   ROLES    AGE   VERSION
ip-192-168-13-156.ec2.internal   Ready    <none>   2m    v1.31.14-eks-70ce843

✅ Cluster is ready!

Step 4: Run eks-node-viewer (Baseline)

Now for the magic. Run eks-node-viewer:

# Set AWS region (important!)
export AWS_REGION=us-east-1

# Run eks-node-viewer
eks-node-viewer --resources cpu,memory

What you'll see:

1 nodes (550m/1930m) 28.5% cpu    ███████████░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ $0.042/hour | $30.368/month
        540Mi/3376720Ki  16.4% memory ███████░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░
6 pods (0 pending 6 running 6 bound)

Let's break this down:

1 nodes - You have 1 node (t3.medium)
550m/1930m - Using 550 millicores out of 1930m available (28.5%)
540Mi/3376720Ki - Using 540MB out of 3.3GB RAM (16.4%)
$0.042/hour | $30.368/month - This is costing you $30/month
6 pods - System pods (coredns, kube-proxy, aws-node, metrics-server)

📸 Screenshot this! This is your baseline.

Step 5: Deploy Nginx (Watch Costs Update)

Keep eks-node-viewer running in Terminal 1.

Open a NEW terminal (Terminal 2) and deploy nginx:

# Set region
export AWS_REGION=us-east-1

# Deploy nginx with 5 replicas
kubectl create deployment nginx --image=nginx --replicas=5

# Check deployment
kubectl get deployments
kubectl get pods

Output:

NAME    READY   UP-TO-DATE   AVAILABLE   AGE
nginx   5/5     5            5           30s

Switch back to Terminal 1 (eks-node-viewer).

You'll see it update in real-time:

1 nodes (750m/1930m) 38.9% cpu    ████████████████░░░░░░░░░░░░░░░░░░░░░░░░ $0.042/hour | $30.368/month
        840Mi/3376720Ki  25.5% memory ███████████░░░░░░░░░░░░░░░░░░░░░░░░░░░
11 pods (0 pending 11 running 11 bound)

What changed:

CPU usage: 28.5% → 38.9%
Memory usage: 16.4% → 25.5%
Pods: 6 → 11 (added 5 nginx pods)
Cost: Still $30/month (same node)

📸 Screenshot this!

Step 6: Deploy CPU-Intensive App

Let's stress the CPU and watch it happen live:

# Deploy stress test (Terminal 2)
kubectl create deployment stress --image=polinux/stress \
  --replicas=3 -- stress --cpu 1 --timeout 3600s

# Check pods
kubectl get pods | grep stress

In eks-node-viewer (Terminal 1):

1 nodes (1550m/1930m) 80.3% cpu   ████████████████████████████████░░░░░░░░ $0.042/hour | $30.368/month
        1140Mi/3376720Ki  34.6% memory ██████████████░░░░░░░░░░░░░░░░░░░░░░░░
14 pods (0 pending 14 running 14 bound)

What changed:

CPU usage: 38.9% → 80.3% 🔥
Memory usage: 25.5% → 34.6%
Pods: 11 → 14 (added 3 stress pods)
Cost: Still $30/month (same node, but now heavily utilized)

📸 Screenshot this!

This is the key insight: You can see your node is now 80% utilized. If you add more workloads, you'll need another node (and another $30/month).

Step 7: Scale Up (Watch Node Get Packed)

Let's push it further:

# Scale nginx to 20 replicas (Terminal 2)
kubectl scale deployment nginx --replicas=20

# Watch pods
kubectl get pods | grep nginx | wc -l

In eks-node-viewer:

1 nodes (1850m/1930m) 95.9% cpu   ████████████████████████████████████████ $0.042/hour | $30.368/month
        1640Mi/3376720Ki  49.8% memory ████████████████████░░░░░░░░░░░░░░░░░░
29 pods (0 pending 29 running 29 bound)

What changed:

CPU usage: 80.3% → 95.9% 🚨
Memory usage: 34.6% → 49.8%
Pods: 14 → 29 (added 15 more nginx pods)
Node is almost maxed out!

📸 Screenshot this!

Real-world insight: If you were using Karpenter, it would see this 95% CPU usage and provision a new node. You'd see a second line appear in eks-node-viewer with another $30/month cost.

Step 8: Check Pod Distribution

In eks-node-viewer, you can see exactly how pods are distributed:

# Run with pods resource
eks-node-viewer --resources cpu,memory,pods

Output:

1 nodes (1850m/1930m) 95.9% cpu    ████████████████████████████████████████ $0.042/hour | $30.368/month
        1640Mi/3376720Ki  49.8% memory ████████████████████░░░░░░░░░░░░░░░░░░
        29/110 pods       26.4% pods   ███████████░░░░░░░░░░░░░░░░░░░░░░░░░░░
29 pods (0 pending 29 running 29 bound)

New info:

29/110 pods - You have 29 pods running, max capacity is 110 pods per node
26.4% pods - You're using 26% of pod capacity

This is useful for understanding if you're hitting pod limits (which can happen before CPU/memory limits).

📸 Screenshot this!

Step 9: Cleanup (Watch Resources Free Up)

Let's clean up and watch the resources free up in real-time:

# Delete deployments (Terminal 2)
kubectl delete deployment nginx stress

# Watch pods terminating
kubectl get pods

In eks-node-viewer:

1 nodes (550m/1930m) 28.5% cpu    ███████████░░░░░░░░░░░░░░░░░░░░░░░░░░░░ $0.042/hour | $30.368/month
        540Mi/3376720Ki  16.4% memory ███████░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░
6 pods (0 pending 6 running 6 bound)

Back to baseline!

CPU: 95.9% → 28.5%
Memory: 49.8% → 16.4%
Pods: 29 → 6
Cost: Still $30/month (node still exists)

📸 Screenshot this!

🎯 Real-World Use Cases

Use Case 1: "Why is my bill so high?"

Before eks-node-viewer:

Open AWS Console
Go to EC2 → Instances
Filter by cluster
Manually check instance types
Look up pricing
Calculate total
Check utilization in Grafana
Realize you have 10 nodes at 20% utilization 😱

With eks-node-viewer:

eks-node-viewer --resources cpu,memory

Output:

10 nodes (2000m/19300m) 10.4% cpu   ████░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ $0.416/hour | $303.68/month
         5400Mi/33767200Ki 16.4% memory ███████░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░
60 pods (0 pending 60 running 60 bound)

Instant insight: You're paying $303/month for 10 nodes that are only 10% utilized. You could consolidate to 2-3 nodes and save $200/month.

Time saved: 15 minutes → 30 seconds

Use Case 2: "Pods are pending"

Scenario: You deploy a new app and pods are stuck in Pending state.

kubectl get pods
# NAME                    READY   STATUS    RESTARTS   AGE
# my-app-7d8f9c5b-xk2lp   0/1     Pending   0          2m

Check eks-node-viewer:

3 nodes (5700m/5790m) 98.4% cpu   ████████████████████████████████████████ $0.126/hour | $91.10/month
        11200Mi/10130160Ki 113.2% memory ████████████████████████████████████████
85 pods (5 pending 80 running 80 bound)

Instant insight: Nodes are at 98% CPU and 113% memory (overcommitted). You need more nodes.

Solution: Scale up or let Karpenter add nodes.

Use Case 3: "Is Karpenter working?"

Scenario: You installed Karpenter but you're not sure if it's actually consolidating nodes.

Terminal 1: Run eks-node-viewer

eks-node-viewer --resources cpu,memory

Terminal 2: Scale down workload

kubectl scale deployment my-app --replicas=2

Watch eks-node-viewer:

After 30 seconds (Karpenter's default consolidation delay)
You should see nodes disappear
Cost should decrease

If nodes DON'T disappear: Karpenter isn't working. Check your NodePool configuration.

Use Case 4: "On-Demand vs Spot Cost Comparison"

Filter by capacity type:

# Show only On-Demand nodes
eks-node-viewer --node-selector karpenter.sh/capacity-type=on-demand

# Show only Spot nodes
eks-node-viewer --node-selector karpenter.sh/capacity-type=spot

Example output:

On-Demand:

5 nodes (5000m/9650m) 51.8% cpu   ████████████████████░░░░░░░░░░░░░░░░░░░ $0.208/hour | $151.84/month

Spot:

5 nodes (5000m/9650m) 51.8% cpu   ████████████████████░░░░░░░░░░░░░░░░░░ $0.062/hour | $45.26/month

Instant insight: Same workload, same utilization, but Spot is 70% cheaper ($151 vs $45).

🧹 Cleanup

When you're done with the demo:

# Delete deployments
kubectl delete deployment nginx stress

# Delete cluster
eksctl delete cluster --name eks-node-viewer-demo --region us-east-1

# Verify deletion
aws eks list-clusters --region us-east-1

🎯 Key Takeaways

What eks-node-viewer Shows You

✅ Real-time cost - Per hour and per month

✅ Instance types - t3.medium, c6a.2xlarge, etc.

✅ Capacity type - On-Demand vs Spot

✅ Resource usage - CPU, Memory, Pods

✅ Utilization bars - Visual representation

✅ Pod distribution - How many pods per node

When to Use It

✅ Cost visibility - Quick cost checks

✅ Capacity planning - Before deploying new apps

✅ Debugging - Why are pods pending?

✅ Karpenter monitoring - Is consolidation working?

✅ Optimization - Finding underutilized nodes

Why It's Better Than Alternatives

✅ No setup - Just install and run

✅ No dashboard - Works in terminal

✅ Real-time - Updates as things change

✅ Cost-aware - Shows pricing automatically

✅ Fast - Answer questions in seconds

🚀 Try It Yourself

5-minute quick start:

# 1. Install
go install github.com/awslabs/eks-node-viewer/cmd/eks-node-viewer@latest

# 2. Set region
export AWS_REGION=us-east-1

# 3. Run
eks-node-viewer --resources cpu,memory

# 4. Deploy something
kubectl create deployment nginx --image=nginx --replicas=10

# 5. Watch it update!

That's it. You'll get it immediately.

📚 Resources

GitHub: https://github.com/awslabs/eks-node-viewer
Karpenter Docs: https://karpenter.sh
EKS Best Practices: https://aws.github.io/aws-eks-best-practices/
AWS Pricing: https://aws.amazon.com/ec2/pricing/

💬 Final Thoughts

eks-node-viewer is one of those tools that seems simple but becomes indispensable once you start using it.

Before: Checking costs meant opening 5 tabs, clicking through dashboards, and doing mental math.

After: One command. Instant visibility. Make decisions in seconds.

If you run EKS (especially with Karpenter), try this once. You'll understand why it's underrated.

Have you used eks-node-viewer? What's your favorite use case? Let me know in the comments! 👇

AWS #EKS #Kubernetes #Karpenter #DevOps #CloudCost #CostOptimization #eks-node-viewer

Happy Learning
Prithiviraj Rengarajan
DevOps Engineer

Building a Multi-Agent AI System with Amazon Bedrock and Agent Squad

Prithiviraj R — Sat, 21 Feb 2026 04:27:03 +0000

Modern AI applications are no longer single, monolithic chatbots. As use cases expand across technology, healthcare, finance, travel, security, and operations, a single general-purpose model quickly becomes inefficient, expensive, and hard to control.

This is where multi-agent AI systems come in.

A multi-agent system divides responsibility across specialized AI agents, each designed to handle a specific domain. Instead of asking one model to “know everything,” the system intelligently decides:

Which agent is best suited for this question?
Which model should respond?
How do we preserve conversation context across agents?

In this tutorial, we implement this architecture using:

AWS Agent Squad for orchestration
Amazon Bedrock for foundation models
Streamlit for a clean, interactive UI

The result is a scalable, cost-efficient, and production-friendly AI assistant.

🧠 What Is AWS Agent Squad?

Agent Squad is an open-source orchestration framework from AWS that helps you build collaborative, multi-agent AI systems.

At a high level, Agent Squad provides:

A central orchestrator to manage agents
Automatic intent-based routing
Shared conversation memory
Clean abstractions for adding new agents or tools

You define:

What each agent specializes in
Which foundation model it uses
How requests are routed

Agent Squad handles the rest.

🧩 Core Concepts You Should Understand

Before diving deeper, it’s important to understand the three core building blocks.

1️⃣ Orchestrator (The Brain)

The AgentSquad orchestrator is the control plane of the system.

Its responsibilities include:

Receiving user input
Maintaining session and conversation context
Comparing the query against agent descriptions
Selecting the most relevant agent
Forwarding the request and returning the response

Think of it as an AI traffic controller.

2️⃣ Specialized Agents (Domain Experts)

Each agent is configured with:

A name
A clear domain description
A foundation model from Amazon Bedrock

Examples:

Tech Agent → AWS, cloud, software engineering
Health Agent → fitness, wellness, nutrition
Travel Agent → destinations, itineraries, tourism

The descriptions are critical—they are what the classifier uses to decide who should answer.

3️⃣ Classifier Model (The Router)

Agent Squad uses a lightweight classifier model (Claude Haiku in this tutorial) to analyze:

The user’s question
All agent descriptions
Previous conversation context

It does not generate the final answer.
Its only job is to determine which agent is best suited.

This keeps costs low and routing fast.

🔄 How the System Works (End-to-End Flow)

Let’s walk through a complete request lifecycle.

Step 1: User Submits a Query

The user types a question into the Streamlit chat UI, such as:

“What is AWS Lambda?”

Step 2: Request Reaches the Orchestrator

The Streamlit frontend sends the input to the Agent Squad orchestrator, along with:

User ID
Session ID
Conversation history

Step 3: Intelligent Agent Selection

The orchestrator invokes the classifier model (Claude Haiku), providing:

The user query
Agent descriptions
Context from prior messages

The classifier evaluates semantic relevance and selects the Tech Agent.

Step 4: Response Generation

The selected agent (using Claude Sonnet) generates a high-quality, domain-specific response.

Because the agent is specialized:

Answers are more accurate
Prompts are shorter
Token usage is optimized

Step 5: Response Returned to the UI

The response is:

Displayed to the user
Tagged with the agent name
Stored in session history for context continuity

Demo:

💡 Why This Architecture Is Powerful

This pattern offers several key advantages:

✅ Specialization Over Generalization

Each agent is optimized for a domain, improving accuracy and relevance.

✅ Lower Costs

Only the selected agent generates the response.
The classifier uses a cheaper, faster model.

✅ Easy Extensibility

Adding a new domain is as simple as registering another agent.

✅ Production Readiness

Clear separation of concerns:

UI
Routing
Reasoning
Model invocation

✅ Enterprise-Friendly

This architecture aligns well with:

Security boundaries
Compliance requirements
Observability and monitoring

🏭 Real-World Use Cases

This multi-agent pattern is ideal for:

Customer support platforms
Internal developer portals
AI-powered knowledge bases
Multi-domain enterprise assistants
Workflow automation systems
DevOps and cloud operations copilots

Reference:
Github: https://github.com/awslabs/agent-squad

Happy Learning
Prithiviraj Rengarajan
DevOps Engineer

From Ingress to Gateway API: Building a Production-Ready Kubernetes Gateway on AWS EKS

Prithiviraj R — Sat, 31 Jan 2026 09:12:27 +0000

🎯 Introduction

Kubernetes Ingress is being deprecated! With Ingress NGINX scheduled for decommissioning in March 2026, it's time to migrate to the Gateway API - the official successor that offers better role separation, portability, and expressiveness.

In this comprehensive guide, I'll walk you through deploying a production-ready Gateway API setup on AWS EKS using Envoy Gateway, demonstrating the modern approach to Kubernetes traffic management.

📋 Table of Contents

Why Gateway API?
Architecture Overview
Prerequisites
Understanding Gateway API Components
Step-by-Step Implementation
Testing and Verification
Troubleshooting
Cleanup
Conclusion

🚀 Why Gateway API?

The Problem with Ingress

Traditional Kubernetes Ingress has several limitations:

Monolithic design: Single resource controls everything
Vendor-specific annotations: Not portable across providers
Limited expressiveness: Can't handle advanced routing scenarios
No role separation: Cluster admins and developers share the same resource
Being deprecated: Ingress NGINX decommissioning in March 2026

Gateway API Benefits

Gateway API solves these problems with:

✅ Role-based model: Clear separation between infrastructure, cluster admin, and developer concerns

✅ Portable & consistent: Standard API across all providers (AWS, GCP, Azure, on-prem)

✅ More expressive: Header-based routing, query parameters, traffic splitting, request/response modification

✅ Future-proof: Official Kubernetes successor, graduated to GA (v1.0.0)

✅ Better security: Fine-grained RBAC and policy attachment

🏗️ Architecture Overview

Ingress vs Gateway API Flow

Traditional Ingress:

Client → External LB (Ingress Controller) → Ingress Resource → Service → Pods

Modern Gateway API:

Client Request
    ↓
External Load Balancer (NLB)
    ↓
Gateway Controller Pod (Envoy)
    ↓
GatewayClass (Infra Provider)
    ↓
Gateway (Cluster Admin)
    ↓
HTTPRoute (Developers)
    ↓
Service → Pods

Role Separation

Role	Responsibility	Resources
Infrastructure Provider	Provides Gateway implementation	GatewayClass
Cluster Administrator	Configures listeners, TLS, load balancers	Gateway
Application Developer	Defines routing rules for their apps	HTTPRoute, TCPRoute, etc.

📦 Prerequisites

Before starting, ensure you have:

AWS Account with appropriate permissions
AWS CLI configured (aws configure)
kubectl (v1.28+)
eksctl (v0.150+)
Helm (v3.0+)
Basic understanding of Kubernetes concepts

Install Required Tools

# Install eksctl (macOS)
brew tap weaveworks/tap
brew install weaveworks/tap/eksctl

# Install kubectl
brew install kubectl

# Install Helm
brew install helm

# Verify installations
eksctl version
kubectl version --client
helm version

🧩 Understanding Gateway API Components

1. GatewayClass

What it is: Defines the controller implementation (like a StorageClass for storage)

Who manages it: Infrastructure provider (platform team)

Purpose: Specifies which Gateway controller will handle Gateway resources

apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
  name: eg
spec:
  controllerName: gateway.envoyproxy.io/gatewayclass-controller
  parametersRef:
    group: gateway.envoyproxy.io
    kind: EnvoyProxy
    name: internet-facing-proxy
    namespace: envoy-gateway-system

Key Points:

One GatewayClass per controller type
Can have multiple GatewayClasses (e.g., internal, external, premium)
Immutable once created

2. Gateway

What it is: Defines the load balancer configuration and listeners

Who manages it: Cluster administrator

Purpose: Configures how traffic enters the cluster (ports, protocols, TLS)

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: demo-gateway
spec:
  gatewayClassName: eg
  listeners:
  - name: http
    protocol: HTTP
    port: 80
  - name: https
    protocol: HTTPS
    port: 443
    tls:
      certificateRefs:
      - name: my-cert

Key Points:

Creates the actual load balancer (NLB in AWS)
Defines listeners (HTTP, HTTPS, TCP, UDP)
Manages TLS certificates
Can have multiple listeners on different ports

3. HTTPRoute

What it is: Defines routing rules for HTTP traffic

Who manages it: Application developers

Purpose: Routes traffic from Gateway to backend services

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: backend-route
spec:
  parentRefs:
  - name: demo-gateway
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /api
    backendRefs:
    - name: backend-service
      port: 80

Key Points:

Attached to a Gateway via parentRefs
Supports path, header, query parameter matching
Can split traffic between multiple backends
Supports request/response modification

4. EnvoyProxy (Envoy Gateway Specific)

What it is: Configuration for Envoy Gateway's data plane

Who manages it: Infrastructure provider

Purpose: Customizes how Envoy Gateway creates load balancers

apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
  name: internet-facing-proxy
  namespace: envoy-gateway-system
spec:
  provider:
    type: Kubernetes
    kubernetes:
      envoyService:
        type: LoadBalancer
        annotations:
          service.beta.kubernetes.io/aws-load-balancer-scheme: "internet-facing"

Key Points:

Envoy Gateway specific (not part of standard Gateway API)
Controls load balancer type (internal vs internet-facing)
Configures resource limits, replicas, etc.

🛠️ Step-by-Step Implementation

Step 1: Create EKS Cluster

Create a new EKS cluster with Kubernetes 1.31:

eksctl create cluster \
  --name eks-gateway-demo \
  --region us-east-1 \
  --version 1.31 \
  --nodegroup-name gateway-nodes \
  --node-type t3.medium \
  --nodes 2 \
  --nodes-min 2 \
  --nodes-max 4 \
  --managed

What this does:

Creates a new EKS cluster named eks-gateway-demo
Uses Kubernetes version 1.31 (latest stable)
Creates 2 t3.medium nodes in a managed node group
Sets up VPC, subnets, security groups automatically

Time: ~15-20 minutes

Verify:

kubectl get nodes
# Should show 2 nodes in Ready state

Step 2: Install Gateway API CRDs

Install the Gateway API Custom Resource Definitions:

kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.0.0/standard-install.yaml

What this installs:

GatewayClass CRD
Gateway CRD
HTTPRoute CRD
ReferenceGrant CRD
TCPRoute, UDPRoute, TLSRoute CRDs

Verify:

kubectl get crd | grep gateway
# Should show gateway-related CRDs

Step 3: Install Envoy Gateway

Install Envoy Gateway as the Gateway controller:

kubectl apply -f https://github.com/envoyproxy/gateway/releases/download/v1.0.0/install.yaml

What this installs:

Envoy Gateway controller deployment
Required RBAC (ServiceAccount, ClusterRole, ClusterRoleBinding)
Envoy Gateway configuration
Webhook configurations

Wait for it to be ready:

kubectl wait --timeout=2m -n envoy-gateway-system \
  deployment/envoy-gateway --for=condition=Available

Verify:

kubectl get pods -n envoy-gateway-system
# Should show envoy-gateway pod running

Step 4: Configure Internet-Facing Load Balancer

Create EnvoyProxy configuration for internet-facing NLB:

kubectl apply -f - << 'EOF'
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
  name: internet-facing-proxy
  namespace: envoy-gateway-system
spec:
  provider:
    type: Kubernetes
    kubernetes:
      envoyService:
        type: LoadBalancer
        annotations:
          service.beta.kubernetes.io/aws-load-balancer-scheme: "internet-facing"
EOF

Why this is needed:

By default, Envoy Gateway creates internal load balancers
This configuration makes the NLB internet-facing
AWS-specific annotation controls the load balancer scheme

Step 5: Create GatewayClass

Create a GatewayClass that references the EnvoyProxy configuration:

kubectl apply -f - << 'EOF'
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
  name: eg
spec:
  controllerName: gateway.envoyproxy.io/gatewayclass-controller
  parametersRef:
    group: gateway.envoyproxy.io
    kind: EnvoyProxy
    name: internet-facing-proxy
    namespace: envoy-gateway-system
EOF

Verify:

kubectl get gatewayclass
# Should show 'eg' with ACCEPTED=True

Step 6: Deploy Sample Applications

Deploy backend and frontend applications:

# Backend Application
kubectl apply -f - << 'EOF'
apiVersion: apps/v1
kind: Deployment
metadata:
  name: backend
spec:
  replicas: 2
  selector:
    matchLabels:
      app: backend
  template:
    metadata:
      labels:
        app: backend
    spec:
      containers:
      - name: backend
        image: hashicorp/http-echo
        args:
        - "-text=Backend API Response"
        ports:
        - containerPort: 5678
---
apiVersion: v1
kind: Service
metadata:
  name: backend-service
spec:
  selector:
    app: backend
  ports:
  - port: 80
    targetPort: 5678
EOF

# Frontend Application
kubectl apply -f - << 'EOF'
apiVersion: apps/v1
kind: Deployment
metadata:
  name: frontend
spec:
  replicas: 2
  selector:
    matchLabels:
      app: frontend
  template:
    metadata:
      labels:
        app: frontend
    spec:
      containers:
      - name: frontend
        image: hashicorp/http-echo
        args:
        - "-text=Frontend UI Response"
        ports:
        - containerPort: 5678
---
apiVersion: v1
kind: Service
metadata:
  name: frontend-service
spec:
  selector:
    app: frontend
  ports:
  - port: 80
    targetPort: 5678
EOF

Verify:

kubectl get pods
kubectl get svc
# Should show 4 pods (2 backend + 2 frontend) and 2 services

Step 7: Create Gateway

Create the Gateway resource that provisions the NLB:

kubectl apply -f - << 'EOF'
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: demo-gateway
spec:
  gatewayClassName: eg
  listeners:
  - name: http
    protocol: HTTP
    port: 80
EOF

What happens:

Envoy Gateway controller sees the new Gateway
Creates an Envoy proxy deployment
Creates a LoadBalancer Service
AWS provisions an internet-facing NLB
NLB targets are registered (takes 2-3 minutes)

Monitor Gateway creation:

kubectl get gateway demo-gateway -w
# Wait for ADDRESS to be populated and PROGRAMMED=True

Get the NLB DNS:

kubectl get gateway demo-gateway -o jsonpath='{.status.addresses[0].value}'

Step 8: Create HTTPRoutes

Create routing rules for backend and frontend:

# Backend Route
kubectl apply -f - << 'EOF'
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: backend-route
spec:
  parentRefs:
  - name: demo-gateway
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /api
    backendRefs:
    - name: backend-service
      port: 80
EOF

# Frontend Route
kubectl apply -f - << 'EOF'
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: frontend-route
spec:
  parentRefs:
  - name: demo-gateway
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /
    backendRefs:
    - name: frontend-service
      port: 80
EOF

How HTTPRoute works:

parentRefs: Attaches the route to a Gateway
matches: Defines conditions for routing (path, headers, query params)
backendRefs: Specifies which Service to route to

Path matching order:

More specific paths are matched first
/api matches before /
This is why backend-route works correctly

Verify:

kubectl get httproute
kubectl describe httproute backend-route
# Should show ACCEPTED=True and ResolvedRefs=True

🧪 Testing and Verification

Wait for NLB to be Ready

# Check NLB status in AWS
aws elbv2 describe-load-balancers --region us-east-1 \
  --query 'LoadBalancers[?contains(LoadBalancerName, `envoy`)].{Name:LoadBalancerName,State:State.Code,Scheme:Scheme}' \
  --output table

# Check target health
TG_ARN=$(aws elbv2 describe-target-groups --region us-east-1 \
  --query 'TargetGroups[?contains(LoadBalancerArns[0], `envoy`)].TargetGroupArn' \
  --output text | head -1)

aws elbv2 describe-target-health --region us-east-1 \
  --target-group-arn "$TG_ARN" \
  --query 'TargetHealthDescriptions[*].{Target:Target.Id,State:TargetHealth.State}' \
  --output table

Wait until:

NLB State = active
At least one target State = healthy

Test the Endpoints

# Get NLB DNS
NLB_DNS=$(kubectl get gateway demo-gateway -o jsonpath='{.status.addresses[0].value}')

# Test backend
curl http://$NLB_DNS/api
# Expected: Backend API Response

# Test frontend
curl http://$NLB_DNS/
# Expected: Frontend UI Response

Verify Gateway API Resources

# Check all Gateway API resources
kubectl get gatewayclass,gateway,httproute

# Check Gateway details
kubectl describe gateway demo-gateway

# Check HTTPRoute details
kubectl describe httproute backend-route

# Check Envoy Gateway logs
kubectl logs -n envoy-gateway-system deployment/envoy-gateway --tail=50

# Check Envoy Proxy logs
ENVOY_POD=$(kubectl get pods -n envoy-gateway-system -l app.kubernetes.io/component=proxy -o jsonpath='{.items[0].metadata.name}')
kubectl logs -n envoy-gateway-system $ENVOY_POD -c envoy --tail=50

🔍 How It Works: Deep Dive

Request Flow

Let's trace a request from client to pod:

1. Client sends: GET http://nlb-dns.amazonaws.com/api

2. DNS resolves to NLB IP addresses

3. NLB forwards to Envoy proxy pod (NodePort 31735)

4. Envoy proxy receives request on port 10080

5. Envoy matches request against HTTPRoutes:
   - Checks backend-route: path=/api ✓ MATCH
   - Routes to backend-service:80

6. backend-service (ClusterIP) load balances to backend pods

7. Backend pod responds: "Backend API Response"

8. Response flows back through Envoy → NLB → Client

Gateway Controller Reconciliation Loop

Envoy Gateway continuously watches for changes:

1. Watch Gateway API resources (Gateway, HTTPRoute, etc.)

2. On change detected:
   a. Validate resource
   b. Translate to Envoy configuration (xDS)
   c. Update Envoy proxy pods
   d. Update Gateway/HTTPRoute status

3. Watch Envoy proxy pods:
   a. Ensure desired replicas running
   b. Monitor health
   c. Update Gateway status

4. Watch LoadBalancer Service:
   a. Get external IP/DNS
   b. Update Gateway status with address

Envoy Proxy Configuration

Envoy Gateway translates Gateway API resources to Envoy xDS configuration:

Gateway → Listener

Gateway:
  listeners:
  - name: http
    port: 80
    protocol: HTTP

Translates to Envoy Listener:
  name: default/demo-gateway/http
  address: 0.0.0.0:10080
  filter_chains:
  - filters:
    - name: envoy.filters.network.http_connection_manager

HTTPRoute → Route Configuration

HTTPRoute:
  matches:
  - path:
      type: PathPrefix
      value: /api
  backendRefs:
  - name: backend-service
    port: 80

Translates to Envoy Route:
  match:
    prefix: /api
  route:
    cluster: httproute/default/backend-route/rule/0

Service → Cluster & Endpoints

Service: backend-service
  selector: app=backend
  port: 80

Translates to Envoy Cluster:
  name: httproute/default/backend-route/rule/0
  type: EDS
  endpoints:
  - 192.168.15.16:5678  (backend pod 1)
  - 192.168.38.106:5678 (backend pod 2)

Load Balancing

NLB Level:

Distributes traffic across Envoy proxy pods
Uses NodePort (31735 in our case)
Health checks on /healthz endpoint

Envoy Level:

Distributes traffic across backend pods
Default: Round-robin
Supports weighted, least-request, ring-hash, etc.

Service Level:

Kubernetes Service provides ClusterIP
kube-proxy manages iptables/IPVS rules
Distributes to healthy endpoints only

🐛 Troubleshooting

Issue 1: Gateway Not Getting Address

Symptoms:

kubectl get gateway demo-gateway
# ADDRESS column is empty, PROGRAMMED=False

Causes:

Envoy Gateway controller not running
LoadBalancer service not created
AWS quota limits reached

Solutions:

# Check controller
kubectl get pods -n envoy-gateway-system

# Check controller logs
kubectl logs -n envoy-gateway-system deployment/envoy-gateway --tail=50

# Check for LoadBalancer service
kubectl get svc -n envoy-gateway-system

# Restart controller if needed
kubectl rollout restart deployment/envoy-gateway -n envoy-gateway-system

Issue 2: HTTPRoute Not Accepted

Symptoms:

kubectl describe httproute backend-route
# Conditions show Accepted=False

Causes:

Gateway doesn't exist
parentRefs incorrect
Backend service doesn't exist

Solutions:

# Verify Gateway exists
kubectl get gateway demo-gateway

# Check parentRefs match Gateway name
kubectl get httproute backend-route -o yaml | grep -A 5 parentRefs

# Verify backend service exists
kubectl get svc backend-service

Issue 3: 404 Not Found

Symptoms:

curl http://$NLB_DNS/api
# Returns 404

Causes:

HTTPRoute path doesn't match
HTTPRoute not attached to Gateway
Backend service selector wrong

Solutions:

# Check HTTPRoute status
kubectl describe httproute backend-route

# Test backend service directly
kubectl run test-curl --image=curlimages/curl --rm -it --restart=Never -- \
  curl backend-service/api

# Check service endpoints
kubectl get endpoints backend-service

Issue 4: Connection Timeout

Symptoms:

curl http://$NLB_DNS/api
# Hangs and times out

Causes:

NLB targets unhealthy
Security group blocking traffic
Network ACLs blocking traffic

Solutions:

# Check target health
TG_ARN=$(aws elbv2 describe-target-groups --region us-east-1 \
  --query 'TargetGroups[?contains(LoadBalancerArns[0], `envoy`)].TargetGroupArn' \
  --output text | head -1)

aws elbv2 describe-target-health --region us-east-1 --target-group-arn "$TG_ARN"

# Check security groups
aws ec2 describe-security-groups --region us-east-1 \
  --filters "Name=tag:kubernetes.io/cluster/eks-gateway-demo,Values=owned"

# Check if NLB is internet-facing
aws elbv2 describe-load-balancers --region us-east-1 \
  --query 'LoadBalancers[?contains(LoadBalancerName, `envoy`)].Scheme'

🧹 Cleanup

To avoid AWS charges, clean up all resources:

# Delete HTTPRoutes
kubectl delete httproute backend-route frontend-route

# Delete Gateway (this deletes the NLB)
kubectl delete gateway demo-gateway

# Delete GatewayClass
kubectl delete gatewayclass eg

# Delete EnvoyProxy configuration
kubectl delete envoyproxy internet-facing-proxy -n envoy-gateway-system

# Uninstall Envoy Gateway
kubectl delete -f https://github.com/envoyproxy/gateway/releases/download/v1.0.0/install.yaml

# Delete Gateway API CRDs
kubectl delete -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.0.0/standard-install.yaml

# Delete applications
kubectl delete deployment backend frontend
kubectl delete service backend-service frontend-service

# Delete EKS cluster (this deletes everything)
eksctl delete cluster --name eks-gateway-demo --region us-east-1

Verify cleanup:

# Check no NLBs remain
aws elbv2 describe-load-balancers --region us-east-1 \
  --query 'LoadBalancers[?contains(LoadBalancerName, `envoy`)]'

# Check cluster deleted
aws eks list-clusters --region us-east-1

📊 Comparison: Ingress vs Gateway API

Feature	Ingress	Gateway API
Role Separation	❌ Single resource	✅ GatewayClass, Gateway, HTTPRoute
Portability	❌ Vendor annotations	✅ Standard API
Expressiveness	❌ Basic path/host	✅ Headers, query params, weights
Protocol Support	❌ HTTP/HTTPS only	✅ HTTP, HTTPS, TCP, UDP, gRPC
Traffic Splitting	❌ Limited	✅ Native support
Request Modification	❌ Controller-specific	✅ Standard filters
Multi-tenancy	❌ Difficult	✅ Built-in with ReferenceGrant
Status	⚠️ Deprecated (March 2026)	✅ GA (v1.0.0)

🚀 Production Readiness Checklist

Before going to production:

[ ] High Availability
- [ ] Multiple Gateway replicas
- [ ] Multi-AZ deployment
- [ ] Pod disruption budgets
[ ] Security
- [ ] TLS enabled on all listeners
- [ ] RBAC configured
- [ ] Network policies in place
- [ ] Secrets encrypted at rest
[ ] Monitoring
- [ ] Prometheus metrics scraped
- [ ] Alerts configured
- [ ] Dashboards created
- [ ] Logging aggregated
[ ] Performance
- [ ] Load testing completed
- [ ] Resource limits set
- [ ] Autoscaling configured
- [ ] Connection limits tuned
[ ] Disaster Recovery
- [ ] Backup strategy defined
- [ ] Restore procedure tested
- [ ] Runbooks documented
- [ ] On-call rotation established

📸 Screenshots

🎯 Conclusion

Gateway API represents a significant evolution in Kubernetes networking:

✅ Better separation of concerns between infrastructure, cluster admins, and developers

✅ Portable and consistent across all cloud providers and on-premises

✅ More expressive with advanced routing capabilities

✅ Future-proof as the official successor to Ingress

With Ingress NGINX being decommissioned in March 2026, now is the perfect time to migrate to Gateway API. Envoy Gateway provides a production-ready implementation that's easy to deploy and manage.

Key Takeaways

Gateway API is not just a replacement - it's a complete redesign with better architecture
Role separation makes it easier to manage in large organizations
Envoy Gateway is a solid choice for AWS EKS deployments
Migration from Ingress is straightforward with proper planning
Advanced features like traffic splitting and header routing are built-in

Next Steps

Explore other Gateway controllers (Istio, Kong, Traefik)
Implement advanced routing scenarios
Add observability with Prometheus and Grafana
Integrate with service mesh for mTLS
Automate deployments with GitOps (ArgoCD, Flux)

Resources

Happy Learning
Prithiviraj Rengarajan
DevOps Engineer

Top 15 Container Security Best Practices

Prithiviraj R — Sun, 25 Jan 2026 19:40:33 +0000

*AWS-Specific Implementation (EKS, ECR, ECS) *

1. Use Minimal Base Images (AWS)

EKS / ECS

Use minimal images when building containers deployed to EKS or ECS.

Recommended

Distroless
Alpine
AWS-provided minimal images (for Lambda container workloads)

Why AWS cares

Smaller images = faster pulls from ECR
Reduced blast radius in shared nodes (EKS)

Example

FROM public.ecr.aws/distroless/base-debian12
COPY app /app
CMD ["/app"]

2. Scan Images for Vulnerabilities (Amazon ECR)

Amazon ECR Enhanced Scanning

Amazon ECR provides native vulnerability scanning using Inspector.

Enable scanning

aws ecr put-registry-scanning-configuration \
  --scan-type ENHANCED \
  --rules '[{"scanFrequency":"CONTINUOUS_SCAN"}]'

What you get

CVE detection
Severity ratings
Integration with AWS Security Hub

3. Sign and Verify Images (EKS + ECR)

AWS + Cosign

Use Cosign with ECR and enforce verification via Kubernetes admission controllers.

Sign image

cosign sign \
  --key awskms:///alias/ecr-signing-key \
  123456789012.dkr.ecr.us-east-1.amazonaws.com/app:latest

Verify in EKS

Use Kyverno or Gatekeeper to block unsigned images

Benefit
Protects against supply chain attacks

4. Run Containers as Non-Root (EKS & ECS)

EKS

securityContext:
  runAsNonRoot: true
  runAsUser: 1000

ECS Task Definition

"user": "1000"

AWS impact

Prevents privilege escalation on worker nodes
Aligns with Pod Security Standards (restricted)

5. Implement Resource Limits (EKS & ECS)

EKS

resources:
  requests:
    cpu: "250m"
    memory: "256Mi"
  limits:
    cpu: "500m"
    memory: "512Mi"

ECS

"cpu": 512,
"memory": 1024

Why this matters in AWS

Prevents noisy-neighbor issues
Improves cluster stability
Reduces autoscaling anomalies

6. Use Read-Only File Systems (EKS)

securityContext:
  readOnlyRootFilesystem: true

AWS Security Benefit

Stops malware persistence
Blocks runtime tampering
Reduces blast radius in shared EKS nodes

7. Never Store Secrets in Images (AWS Secrets Manager)

EKS

Use Secrets Manager + IRSA

env:
  - name: DB_PASSWORD
    valueFrom:
      secretKeyRef:
        name: db-secret
        key: password

ECS

"secrets": [
  {
    "name": "DB_PASSWORD",
    "valueFrom": "arn:aws:secretsmanager:..."
  }
]

Why AWS recommends this

Central rotation
IAM-based access
Audit logging via CloudTrail

8. Enable Runtime Security Profiles (EKS)

Seccomp Example

securityContext:
  seccompProfile:
    type: RuntimeDefault

Advanced

Combine with Falco on EKS
Stream alerts to CloudWatch or SIEM

9. Use Multi-Stage Builds (ECR Optimization)

AWS Advantage

Faster ECR pulls
Lower storage costs
Fewer vulnerabilities detected

FROM node:22 AS build
RUN npm run build

FROM public.ecr.aws/distroless/nodejs22
COPY --from=build /app /app
CMD ["app.js"]

10. Network Segmentation (EKS + VPC)

Kubernetes Network Policy

policyTypes:
- Ingress
- Egress

AWS Layer

Security Groups for Pods (EKS)
Private subnets
VPC flow logs

Outcome
Prevents lateral movement inside the cluster.

11. Use Trusted Registries (ECR)

Best Practice

Disable public registry pulls
Allow only ECR

EKS Admission Policy

Block images not from *.amazonaws.com

Why
Public images are a major malware vector.

12. Drop Linux Capabilities (EKS & ECS)

EKS

capabilities:
  drop:
    - ALL

ECS

"linuxParameters": {
  "capabilities": {
    "drop": ["ALL"]
  }
}

AWS Security Gain

Reduces kernel attack surface
Limits container breakout attempts

13. Logging & Monitoring (AWS Native)

EKS

CloudWatch Container Insights
Fluent Bit
Falco → CloudWatch Logs

ECS

awslogs driver
FireLens for advanced routing

Security Use Case

Detect suspicious syscalls
Audit container behavior

14. Image Lifecycle Management (ECR)

ECR Lifecycle Policy

{
  "rules": [
    {
      "rulePriority": 1,
      "description": "Expire untagged images after 30 days",
      "selection": {
        "tagStatus": "untagged",
        "countType": "sinceImagePushed",
        "countUnit": "days",
        "countNumber": 30
      },
      "action": { "type": "expire" }
    }
  ]
}

Benefits

Reduces attack surface
Controls storage cost
Forces fresh rebuilds

15. Prevent Container Escape (EKS & ECS)

EKS

No privileged pods
No hostPath mounts
Enforce Pod Security Admission (restricted)

ECS

"privileged": false

AWS Security Impact
Container escape = EC2 compromise
These controls block the most dangerous attack vector.

AWS Security Alignment

These 15 practices directly map to:

AWS Well-Architected Framework – Security Pillar
EKS Best Practices Guide
Zero Trust container principles
CIS Kubernetes Benchmarks

Final AWS-Focused Takeaway

If you are running containers on EKS, ECS, or ECR, these controls are not optional hardening—they are baseline production requirements. When applied together, they significantly reduce:

Misconfiguration risk
Supply chain attacks
Runtime compromises
Cloud security incidents

Happy learning
Prithiviraj Rengarajan
DevOps Engineer

One small Docker decision can have an outsized impact on your AWS workloads.

Prithiviraj R — Sun, 18 Jan 2026 00:25:32 +0000

Most teams don’t realize how much their Node.js base image affects build time, storage costs, and runtime performance—until they compare them side by side.

📦 node:22 → 1.12 GB
📦 bitnami/node:22 → 974 MB

Lean alternatives:
📦 node:22-slim → 220 MB
📦 node:22-alpine → 155 MB
📦 Chainguard Node → 145 MB
📦 Distroless Node → 141 MB

That’s nearly 1 GB of overhead eliminated by a simple base image change.

Why this matters on AWS:

Faster CI/CD pipelines (CodeBuild, GitHub Actions, Jenkins)

Lower ECR storage and data transfer costs

Smaller attack surface → fewer CVEs to track

Faster cold starts on EKS and ECS

Better developer feedback loops

The best part: you can unlock these improvements without touching application code—just choose the right base image.

Happy learning
Prithiviraj Rengarajan
DevOps Engineer

Building a Smart Inbox: AI-Powered Message Routing on AWS EKS

Prithiviraj R — Sat, 17 Jan 2026 06:06:20 +0000

Real-time sentiment analysis and intelligent message prioritization using Kubernetes, AWS Bedrock, and React

Introduction

In this comprehensive guide, I'll walk you through building a production-ready, AI-powered message routing system using AWS services. This project demonstrates how to leverage modern cloud-native technologies to create an intelligent application that automatically analyzes message sentiment and routes them accordingly.

What We Built:

A sentiment analysis API using AWS Bedrock (Claude 3 Haiku)
Kubernetes deployment on Amazon EKS
React frontend with real-time updates
Persistent storage with DynamoDB
Container registry with Amazon ECR

Tech Stack:

Backend: Python, FastAPI
AI/ML: AWS Bedrock (Claude 3 Haiku)
Container Orchestration: Amazon EKS (Kubernetes)
Database: Amazon DynamoDB
Container Registry: Amazon ECR
Frontend: React, Vite
Infrastructure: AWS (EKS, EC2, VPC, Load Balancer)

Architecture Overview
Why These AWS Services?
Setting Up Amazon EKS
Building the Backend API
Containerization with Docker
Amazon ECR Setup
DynamoDB Configuration
Kubernetes Deployment
Frontend Development
Testing & Demo

Architecture Overview

Our Smart Inbox follows a modern microservices architecture:

┌─────────────┐
│   Browser   │
│  (React UI) │
└──────┬──────┘
       │ HTTP
       ▼
┌─────────────────┐
│ Network Load    │
│   Balancer      │
└────────┬────────┘
         │
         ▼
┌─────────────────────────┐
│   Kubernetes Service    │
│   (smart-inbox-service) │
└────────┬────────────────┘
         │
    ┌────┴────┐
    ▼         ▼
┌────────┐ ┌────────┐
│ Pod 1  │ │ Pod 2  │  (2 replicas for HA)
│FastAPI │ │FastAPI │
└───┬────┘ └───┬────┘
    │          │
    └────┬─────┘
         ▼
    ┌─────────────┐
    │ AWS Bedrock │
    │  (Claude)   │
    └─────────────┘
         │
         ▼
    ┌─────────────┐
    │  DynamoDB   │
    │  (Storage)  │
    └─────────────┘

Data Flow:

User submits message via React frontend
Request hits Network Load Balancer
Load Balancer routes to one of the Kubernetes pods
FastAPI application processes the request
AWS Bedrock analyzes sentiment using Claude AI
Results stored in DynamoDB
Response sent back to frontend
Dashboard updates in real-time

Why These AWS Services?

1. Amazon EKS (Elastic Kubernetes Service)

What is it?
Amazon EKS is a managed Kubernetes service that makes it easy to run Kubernetes on AWS without needing to install and operate your own Kubernetes control plane.

Why we chose it:

✅ Managed Control Plane: AWS handles Kubernetes master nodes, updates, and patches
✅ High Availability: Multi-AZ deployment by default
✅ Scalability: Auto-scaling for both pods and nodes
✅ Security: Integrated with AWS IAM for authentication
✅ Production-Ready: Used by thousands of companies worldwide
✅ Cost-Effective: Pay only for worker nodes, control plane is managed

Our Use Case:
We use EKS to orchestrate our containerized FastAPI application, ensuring high availability with 2 pod replicas and automatic failover.

Key Concepts:

Cluster: The Kubernetes control plane and worker nodes
Nodes: EC2 instances that run your containers
Pods: Smallest deployable units containing one or more containers
Service: Exposes pods to network traffic
Deployment: Manages pod replicas and updates

2. Amazon EC2 (Worker Nodes)

What is it?
EC2 instances serve as worker nodes in our EKS cluster, running the actual containerized applications.

Why we chose it:

✅ Flexibility: Choose instance types based on workload
✅ Performance: Dedicated compute resources
✅ Control: Full control over node configuration
✅ Cost Options: On-demand, Reserved, or Spot instances

Our Configuration:

Instance Type: t3.medium (2 vCPU, 4GB RAM)
Count: 2 nodes for high availability
Auto-scaling: Min 2, Max 4, Desired 2

Why t3.medium?

Balanced compute and memory for our API workload
Burstable performance for handling traffic spikes
Cost-effective for development and small production workloads

3. Amazon DynamoDB

What is it?
DynamoDB is a fully managed NoSQL database service that provides fast and predictable performance with seamless scalability.

Why we chose it:

✅ Serverless: No servers to manage
✅ Performance: Single-digit millisecond latency
✅ Scalability: Automatically scales up/down
✅ Pay-per-request: Only pay for what you use
✅ High Availability: Multi-AZ replication by default
✅ No Schema: Flexible data model

Our Use Case:
We store analyzed messages with their sentiment scores, timestamps, and metadata.

Table Schema:

Primary Key: message_id (String)

Attributes:
- message_id: Unique identifier
- text: Original message content
- sender: Email of sender
- category: Message category (general, support, feedback, complaint)
- sentiment: AI-detected sentiment (positive, negative, neutral)
- confidence: Confidence score (0.0 - 1.0)
- timestamp: When message was analyzed
- priority: Routing priority (NORMAL, HIGH)

Why NoSQL over SQL?

No complex relationships needed
Flexible schema for future attributes
Better performance for simple key-value lookups
Easier to scale horizontally

4. Amazon ECR (Elastic Container Registry)

What is it?
ECR is a fully managed Docker container registry that makes it easy to store, manage, and deploy Docker container images.

Why we chose it:

✅ Integration: Native integration with EKS
✅ Security: Images encrypted at rest and in transit
✅ Scanning: Automatic vulnerability scanning
✅ IAM Integration: Fine-grained access control
✅ High Availability: Replicated across multiple AZs
✅ No Limits: Unlimited repositories and images

Our Use Case:
We store our FastAPI application Docker image, which EKS pulls to run in pods.

Image Details:

Repository: smart-inbox
Base Image: python:3.11-slim
Size: ~200MB
Platform: linux/amd64 (for EC2 compatibility)

Why ECR over Docker Hub?

Faster pulls from within AWS
Better security with IAM
No rate limiting
Integrated with AWS services

5. AWS Bedrock (Claude 3 Haiku)

What is it?
AWS Bedrock is a fully managed service that offers foundation models from leading AI companies through a single API.

Why we chose it:

✅ Managed Service: No infrastructure to manage
✅ Multiple Models: Access to Claude, Llama, Titan, etc.
✅ Pay-per-use: Only pay for API calls
✅ Low Latency: Fast inference times
✅ Security: Data not used for training
✅ Compliance: SOC, HIPAA, GDPR compliant

Why Claude 3 Haiku?

Fast: Optimized for speed (sub-second responses)
Cost-Effective: $0.25 per 1K input tokens
Accurate: High accuracy for sentiment analysis
Context: 200K token context window
Reliable: Consistent output format

Our Prompt:

prompt = f"""Analyze the sentiment of this message. Return ONLY valid JSON.

Message: "{text}"

Respond with this exact format:
{{"sentiment": "positive", "confidence": 0.95}}

Sentiment must be: positive, neutral, or negative
Confidence must be: 0.0 to 1.0"""

Alternative Approaches:

AWS Comprehend (simpler but less flexible)
SageMaker (more control but more complex)
Third-party APIs (vendor lock-in concerns)

Setting Up Amazon EKS

Step 1: Create IAM Role for EKS Cluster

EKS needs permissions to manage AWS resources on your behalf.

# Create trust policy
aws iam create-role \
  --role-name smart-inbox-eks-cluster-role \
  --assume-role-policy-document '{
    "Version": "2012-10-17",
    "Statement": [{
      "Effect": "Allow",
      "Principal": {"Service": "eks.amazonaws.com"},
      "Action": "sts:AssumeRole"
    }]
  }'

# Attach required policy
aws iam attach-role-policy \
  --role-name smart-inbox-eks-cluster-role \
  --policy-arn arn:aws:iam::aws:policy/AmazonEKSClusterPolicy

What this does:

Creates an IAM role that EKS can assume
Grants permissions to manage networking, load balancers, and EC2 instances

Step 2: Create EKS Cluster

aws eks create-cluster \
  --name smart-inbox-cluster \
  --region us-east-1 \
  --role-arn arn:aws:iam::<ACCOUNT_ID>:role/smart-inbox-eks-cluster-role \
  --resources-vpc-config subnetIds=<SUBNET_1>,<SUBNET_2>,<SUBNET_3> \
  --kubernetes-version 1.31

Parameters Explained:

--name: Cluster identifier
--region: AWS region (us-east-1 for lowest latency to Bedrock)
--role-arn: IAM role created in Step 1
--resources-vpc-config: Network configuration
- Requires at least 2 subnets in different AZs
- Subnets must have internet gateway for public access
--kubernetes-version: Latest stable version

What happens:

AWS provisions Kubernetes control plane (2-3 master nodes)
Sets up etcd database for cluster state
Configures API server, scheduler, and controller manager
Takes 10-15 minutes to complete

Verification:

aws eks describe-cluster \
  --name smart-inbox-cluster \
  --region us-east-1 \
  --query 'cluster.status'

Expected output: "ACTIVE"

Step 3: Create Node Group

Worker nodes run your actual application containers.

Create IAM Role for Nodes:

# Create role
aws iam create-role \
  --role-name smart-inbox-eks-node-role \
  --assume-role-policy-document '{
    "Version": "2012-10-17",
    "Statement": [{
      "Effect": "Allow",
      "Principal": {"Service": "ec2.amazonaws.com"},
      "Action": "sts:AssumeRole"
    }]
  }'

# Attach required policies
aws iam attach-role-policy \
  --role-name smart-inbox-eks-node-role \
  --policy-arn arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy

aws iam attach-role-policy \
  --role-name smart-inbox-eks-node-role \
  --policy-arn arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy

aws iam attach-role-policy \
  --role-name smart-inbox-eks-node-role \
  --policy-arn arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly

Create Node Group:

aws eks create-nodegroup \
  --cluster-name smart-inbox-cluster \
  --nodegroup-name smart-inbox-nodes \
  --region us-east-1 \
  --subnets <SUBNET_1> <SUBNET_2> <SUBNET_3> \
  --node-role arn:aws:iam::<ACCOUNT_ID>:role/smart-inbox-eks-node-role \
  --scaling-config minSize=2,maxSize=4,desiredSize=2 \
  --instance-types t3.medium \
  --disk-size 20

Configuration Explained:

minSize=2: Minimum nodes for high availability
maxSize=4: Maximum nodes for scaling
desiredSize=2: Initial node count
instance-types=t3.medium: 2 vCPU, 4GB RAM
disk-size=20: 20GB EBS volume per node

What happens:

AWS launches EC2 instances in your subnets
Installs kubelet, kube-proxy, and container runtime
Joins nodes to the EKS cluster
Takes 5-10 minutes

Step 4: Configure kubectl

Connect your local kubectl to the EKS cluster:

aws eks update-kubeconfig \
  --name smart-inbox-cluster \
  --region us-east-1

Verify connection:

kubectl get nodes

Expected output:

NAME                          STATUS   ROLES    AGE   VERSION
ip-172-31-x-x.ec2.internal    Ready    <none>   5m    v1.31.0
ip-172-31-y-y.ec2.internal    Ready    <none>   5m    v1.31.0

Building the Backend API

Our backend is a FastAPI application that handles sentiment analysis requests.

Project Structure

backend/
├── app/
│   ├── main.py          # FastAPI application
│   ├── sentiment.py     # Bedrock integration
│   ├── models.py        # Pydantic models
│   ├── database.py      # DynamoDB operations
│   └── requirements.txt # Python dependencies
└── Dockerfile           # Container definition

Core Components

1. FastAPI Application (main.py)

FastAPI is a modern, fast web framework for building APIs with Python.

Why FastAPI?

✅ Fast performance (comparable to Node.js)
✅ Automatic API documentation (Swagger UI)
✅ Type hints and validation
✅ Async support
✅ Easy to learn and use

Key Endpoints:

POST /api/analyze - Analyze message sentiment
GET /api/messages - Retrieve recent messages
GET /api/stats - Get sentiment statistics
GET /health - Health check for load balancer

2. Bedrock Integration (sentiment.py)

Handles communication with AWS Bedrock API.

Key Features:

Structured prompts for consistent output
JSON parsing with error handling
Fallback to neutral sentiment on errors
Regex extraction for robust JSON parsing

3. Data Models (models.py)

Pydantic models ensure data validation.

Benefits:

Automatic request validation
Type safety
Clear API contracts
Automatic documentation

4. Database Layer (database.py)

Abstracts DynamoDB operations.

Key Functions:

save_message() - Store analyzed message
get_recent_messages() - Retrieve messages
Decimal conversion for DynamoDB compatibility

Part 2: Containerization, Deployment & Frontend

Containerization with Docker

Why Docker?

Docker containers package your application with all its dependencies, ensuring it runs consistently across different environments.

Benefits:

✅ Consistency: "Works on my machine" → "Works everywhere"
✅ Isolation: Each container is isolated
✅ Portability: Run anywhere Docker runs
✅ Efficiency: Lightweight compared to VMs
✅ Scalability: Easy to replicate

Our Dockerfile

FROM python:3.11-slim
WORKDIR /app
COPY app/requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY app/ .
EXPOSE 8000
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

Line-by-Line Explanation:

FROM python:3.11-slim
- Base image: Python 3.11 on Debian
- "slim" variant: Smaller size (~150MB vs ~900MB)
- Official Python image from Docker Hub
WORKDIR /app
- Sets working directory inside container
- All subsequent commands run from /app
COPY app/requirements.txt .
- Copy dependencies file first
- Enables Docker layer caching
- Rebuilds only if requirements change
RUN pip install --no-cache-dir -r requirements.txt
- Install Python packages
- --no-cache-dir: Reduces image size
- Runs during build, not runtime
COPY app/ .
- Copy application code
- Done after pip install for better caching
EXPOSE 8000
- Documents which port the app uses
- Doesn't actually publish the port
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]
- Command to run when container starts
- Uvicorn: ASGI server for FastAPI
- --host 0.0.0.0: Listen on all interfaces
- --port 8000: Application port

Building for Production

Solution: Multi-platform build

docker buildx build \
  --platform linux/amd64 \
  -t smart-inbox:latest .

What this does:

buildx: Docker's build extension
--platform linux/amd64: Target architecture
-t smart-inbox:latest: Tag the image
.: Build context (current directory)

Image Size Optimization:

Base image: python:3.11-slim (150MB)
Dependencies: ~50MB
Application code: <1MB
Total: ~200MB

Best Practices Applied:

✅ Use slim base images
✅ Multi-stage builds (if needed)
✅ Layer caching optimization
✅ .dockerignore file
✅ Non-root user (production)
✅ Health checks

Amazon ECR Setup

Creating the Repository

aws ecr create-repository \
  --repository-name smart-inbox \
  --region us-east-1 \
  --image-scanning-configuration scanOnPush=true

Features Enabled:

scanOnPush=true: Automatic vulnerability scanning
Encryption at rest (AES-256)
Encryption in transit (TLS)

Pushing Images to ECR

Step 1: Authenticate Docker to ECR

aws ecr get-login-password --region us-east-1 | \
  docker login --username AWS --password-stdin \
  <ACCOUNT_ID>.dkr.ecr.us-east-1.amazonaws.com

Step 2: Tag Image

docker tag smart-inbox:latest \
  <ACCOUNT_ID>.dkr.ecr.us-east-1.amazonaws.com/smart-inbox:latest

Step 3: Push Image

docker push \
  <ACCOUNT_ID>.dkr.ecr.us-east-1.amazonaws.com/smart-inbox:latest

What happens:

Docker compresses image layers
Uploads to ECR
ECR scans for vulnerabilities
Image available for EKS to pull

Image URI Format:

<account-id>.dkr.ecr.<region>.amazonaws.com/<repository>:<tag>

DynamoDB Configuration

Creating the Table

aws dynamodb create-table \
  --table-name smart-inbox-messages \
  --attribute-definitions AttributeName=message_id,AttributeType=S \
  --key-schema AttributeName=message_id,KeyType=HASH \
  --billing-mode PAY_PER_REQUEST \
  --region us-east-1

Configuration Explained:

Attribute Definitions:

message_id: String type (S)
Only key attributes need to be defined
Other attributes are schema-less

Key Schema:

HASH: Partition key (required)
No sort key (RANGE) needed for our use case

Billing Mode:

PAY_PER_REQUEST: On-demand pricing
Alternative: PROVISIONED (fixed capacity)
Better for unpredictable workloads

Data Model Design

Item Structure:

{
  "message_id": "uuid-string",
  "text": "Message content",
  "sender": "email@example.com",
  "category": "general",
  "sentiment": "positive",
  "confidence": 0.95,
  "timestamp": "2026-01-17T10:00:00Z",
  "priority": "NORMAL"
}

Design Decisions:

UUID as Partition Key
- Ensures even distribution across partitions
- Prevents hot partitions
- Globally unique
No Sort Key
- Simple access pattern (get by ID)
- Scan for recent messages (acceptable for low volume)
- Could add timestamp as sort key for better queries
Denormalized Data
- All data in one item
- No joins needed
- Optimized for read performance

Handling Floats:
DynamoDB doesn't support float types natively.

Solution:

from decimal import Decimal

# Before saving
message_data['confidence'] = Decimal(str(confidence))

# After reading
confidence = float(item['confidence'])

Kubernetes Deployment

IAM Roles for Service Accounts (IRSA)

Pods need AWS permissions to access Bedrock and DynamoDB.

Why IRSA?

✅ No hardcoded credentials
✅ Automatic credential rotation
✅ Fine-grained permissions per pod
✅ Follows AWS best practices

Setup Steps:

1. Get OIDC Provider ID

aws eks describe-cluster \
  --name smart-inbox-cluster \
  --query 'cluster.identity.oidc.issuer' \
  --output text

2. Create Trust Policy

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Principal": {
      "Federated": "arn:aws:iam::<ACCOUNT_ID>:oidc-provider/oidc.eks.us-east-1.amazonaws.com/id/<OIDC_ID>"
    },
    "Action": "sts:AssumeRoleWithWebIdentity",
    "Condition": {
      "StringEquals": {
        "oidc.eks.us-east-1.amazonaws.com/id/<OIDC_ID>:sub": 
          "system:serviceaccount:smart-inbox:smart-inbox-sa"
      }
    }
  }]
}

3. Create IAM Role

aws iam create-role \
  --role-name smart-inbox-pod-role \
  --assume-role-policy-document file://trust-policy.json

aws iam attach-role-policy \
  --role-name smart-inbox-pod-role \
  --policy-arn arn:aws:iam::aws:policy/AmazonDynamoDBFullAccess

aws iam attach-role-policy \
  --role-name smart-inbox-pod-role \
  --policy-arn arn:aws:iam::aws:policy/AmazonBedrockFullAccess

Kubernetes Manifests

1. Namespace (namespace.yaml)

apiVersion: v1
kind: Namespace
metadata:
  name: smart-inbox

Why namespaces?

Logical isolation
Resource quotas
Access control
Organization

2. Service Account (serviceaccount.yaml)

apiVersion: v1
kind: ServiceAccount
metadata:
  name: smart-inbox-sa
  namespace: smart-inbox
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::<ACCOUNT_ID>:role/smart-inbox-pod-role

Key annotation:

Links Kubernetes SA to AWS IAM role
Enables IRSA

3. Deployment (deployment.yaml)

apiVersion: apps/v1
kind: Deployment
metadata:
  name: smart-inbox-api
  namespace: smart-inbox
spec:
  replicas: 2
  selector:
    matchLabels:
      app: smart-inbox
  template:
    metadata:
      labels:
        app: smart-inbox
    spec:
      serviceAccountName: smart-inbox-sa
      containers:
      - name: api
        image: <ACCOUNT_ID>.dkr.ecr.us-east-1.amazonaws.com/smart-inbox:latest
        ports:
        - containerPort: 8000
        env:
        - name: DYNAMODB_TABLE
          value: smart-inbox-messages
        - name: AWS_DEFAULT_REGION
          value: us-east-1
        resources:
          requests:
            memory: "256Mi"
            cpu: "250m"
          limits:
            memory: "512Mi"
            cpu: "500m"
        livenessProbe:
          httpGet:
            path: /health
            port: 8000
          initialDelaySeconds: 30
          periodSeconds: 10
        readinessProbe:
          httpGet:
            path: /health
            port: 8000
          initialDelaySeconds: 5
          periodSeconds: 5

Configuration Explained:

Replicas:

replicas: 2: Two pod instances
High availability
Load distribution
Zero-downtime updates

Resources:

requests: Guaranteed resources
limits: Maximum resources
Prevents resource starvation

Probes:

livenessProbe: Restart if unhealthy
readinessProbe: Remove from service if not ready
Ensures traffic only to healthy pods

4. Service (service.yaml)

apiVersion: v1
kind: Service
metadata:
  name: smart-inbox-service
  namespace: smart-inbox
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
    service.beta.kubernetes.io/aws-load-balancer-scheme: "internet-facing"
spec:
  type: LoadBalancer
  selector:
    app: smart-inbox
  ports:
  - name: http
    port: 80
    targetPort: 8000
    protocol: TCP

Service Types:

ClusterIP: Internal only (default)
NodePort: Exposes on node IP
LoadBalancer: Creates AWS NLB (our choice)

Why Network Load Balancer?

✅ Layer 4 (TCP) load balancing
✅ Ultra-low latency
✅ Handles millions of requests/sec
✅ Static IP addresses
✅ TLS termination support

Port Mapping:

port: 80: External port (HTTP)
targetPort: 8000: Container port
Traffic: 80 → 8000

Deploying to Kubernetes

# Apply manifests
kubectl apply -f kubernetes/namespace.yaml
kubectl apply -f kubernetes/serviceaccount.yaml
kubectl apply -f kubernetes/deployment.yaml
kubectl apply -f kubernetes/service.yaml

# Verify deployment
kubectl get pods -n smart-inbox
kubectl get svc -n smart-inbox

# Check logs
kubectl logs -f -l app=smart-inbox -n smart-inbox

Deployment Process:

Kubernetes pulls image from ECR
Creates 2 pod replicas
Assigns IAM role via IRSA
Runs health checks
Provisions Network Load Balancer
Routes traffic to healthy pods

Load Balancer Provisioning:

Takes 2-3 minutes
Creates in multiple AZs
Assigns DNS name
Configures health checks

Frontend Development

Technology Stack

React + Vite

Why React?

✅ Component-based architecture
✅ Virtual DOM for performance
✅ Large ecosystem
✅ Easy to learn

Why Vite?

✅ Lightning-fast HMR (Hot Module Replacement)
✅ Optimized builds
✅ Modern tooling
✅ Better DX than Create React App

Project Structure

frontend/
├── src/
│   ├── App.jsx              # Main component
│   ├── App.css              # Styles
│   ├── components/
│   │   ├── MessageForm.jsx  # Message submission
│   │   ├── Dashboard.jsx    # Messages list
│   │   └── SentimentCard.jsx # Individual message
│   └── api/
│       └── client.js        # API calls
├── index.html
├── package.json
└── vite.config.js

Key Features

1. Real-Time Statistics

const [stats, setStats] = useState({
  total: 0,
  positive: 0,
  negative: 0,
  neutral: 0
})

useEffect(() => {
  fetchData()
  const interval = setInterval(fetchData, 5000) // Poll every 5s
  return () => clearInterval(interval)
}, [])

Auto-refresh every 5 seconds:

Polls /api/stats endpoint
Updates dashboard without page reload
Clean up on component unmount

2. Message Submission

const handleSubmit = async (messageData) => {
  setLoading(true)
  try {
    await analyzeMessage(messageData)
    await fetchData() // Refresh messages
  } catch (error) {
    alert('Error analyzing message')
  } finally {
    setLoading(false)
  }
}

User experience:

Loading state during analysis
Error handling with user feedback
Automatic refresh after submission

3. Color-Coded Sentiment Cards

const sentimentColors = {
  positive: 'bg-green-100 border-green-500',
  negative: 'bg-red-100 border-red-500',
  neutral: 'bg-gray-100 border-gray-500'
}

Visual feedback:

🟢 Green: Positive messages
🔴 Red: Negative messages (with URGENT badge)
⚪ Gray: Neutral messages

4. API Client

const API_URL = import.meta.env.VITE_API_URL

const api = axios.create({
  baseURL: API_URL,
  headers: { 'Content-Type': 'application/json' }
})

export const analyzeMessage = async (message) => {
  const response = await api.post('/api/analyze', message)
  return response.data
}

Environment variables:

Development: http://localhost:8000
Production: Load Balancer URL
Configured in .env file

Styling Approach

CSS-in-JS vs CSS Modules vs Plain CSS

We chose Plain CSS for simplicity:

✅ No build complexity
✅ Easy to understand
✅ Good performance
✅ Sufficient for our needs

Design System:

Purple gradient header
Card-based layout
Responsive grid
Smooth transitions
Mobile-friendly

Building a Smart Inbox: AI-Powered Message Routing on AWS EKS

Part 3: Testing & Deployment

Testing & Demo

Testing Strategy

1. Unit Testing (Backend)

# Test sentiment analysis
def test_analyze_sentiment():
    result = analyze_sentiment("I love this!")
    assert result['sentiment'] == 'positive'
    assert result['confidence'] > 0.7

# Test database operations
def test_save_message():
    message = {
        'message_id': 'test-123',
        'text': 'Test message',
        'sentiment': 'positive'
    }
    save_message(message)
    # Verify in DynamoDB

2. Integration Testing

# Test API endpoints
curl http://<API_URL>/health
# Expected: {"status":"healthy"}

curl -X POST http://<API_URL>/api/analyze \
  -H "Content-Type: application/json" \
  -d '{"text":"Great product!","sender":"test@test.com","category":"feedback"}'
# Expected: Full analysis response with sentiment

3. Load Testing

# Using Apache Bench
ab -n 1000 -c 10 http://<API_URL>/health

# Using k6
k6 run --vus 10 --duration 30s load-test.js

Demo Scenarios

Scenario 1: Positive Customer Feedback

Message: "I absolutely love this product! The quality is amazing and 
         delivery was super fast. Your customer service team was 
         incredibly helpful. Thank you so much!"
Sender: happy.customer@example.com
Category: Feedback

Expected Result:
✅ Sentiment: POSITIVE
✅ Confidence: 95%+
✅ Priority: NORMAL
✅ Card Color: Green
✅ No urgent badge

Scenario 2: Angry Customer Complaint

Message: "This is completely unacceptable! I've been waiting for 3 weeks 
         with no response. The product arrived damaged and I want a full 
         refund immediately. Worst experience ever!"
Sender: angry.customer@example.com
Category: Complaint

Expected Result:
✅ Sentiment: NEGATIVE
✅ Confidence: 90%+
✅ Priority: HIGH
✅ Card Color: Red
✅ URGENT badge displayed

Scenario 3: Neutral Inquiry

Message: "Hi, I would like to know more about your pricing plans and 
         available features. Can you please send me the detailed 
         information? Also, what are the payment options?"
Sender: inquiry@example.com
Category: General

Expected Result:
✅ Sentiment: NEUTRAL
✅ Confidence: 80%+
✅ Priority: NORMAL
✅ Card Color: Gray
✅ No urgent badge

Conclusion

What We Built

A production-ready, AI-powered message routing system that:

✅ Analyzes sentiment in real-time using AWS Bedrock
✅ Runs on Kubernetes for high availability
✅ Scales automatically based on demand
✅ Provides beautiful, responsive UI
✅ Handles thousands of messages per day

Key Takeaways

Cloud-Native Architecture Works
- Kubernetes provides excellent orchestration
- Managed services reduce operational burden
- Serverless components lower costs
AI Integration is Straightforward
- AWS Bedrock makes AI accessible
- No ML expertise required
- Pay-per-use pricing is economical
Modern Tools Improve Productivity
- FastAPI for rapid API development
- React + Vite for fast frontend
- Docker for consistent deployments
AWS Services Integrate Well
- EKS, ECR, DynamoDB work seamlessly
- IAM provides unified security
- CloudWatch for centralized monitoring

Skills Demonstrated

✅ Kubernetes orchestration
✅ Docker containerization
✅ AWS cloud architecture
✅ API development (FastAPI)
✅ Frontend development (React)
✅ AI/ML integration (Bedrock)
✅ Database design (DynamoDB)
✅ DevOps practices
✅ Security best practices

Resources

Documentation:

Published: January 17, 2026

Author: Prithiviraj Rengarajan
Tags: AWS, EKS, Kubernetes, AI, Bedrock, DynamoDB, FastAPI, React, Cloud Architecture

🚀 AWS Introduces Regional NAT Gateway: Simplifying Outbound Connectivity

Prithiviraj R — Wed, 31 Dec 2025 10:57:34 +0000

AWS has introduced Regional NAT Gateway, a significant improvement to how outbound internet access is handled for workloads running in private subnets. This change meaningfully reduces architectural complexity while improving availability and security.

Let’s start with the basics.

What is a NAT Gateway?

A NAT Gateway allows resources in private subnets to access the internet for outbound traffic (for example, OS updates, package downloads, external APIs) without allowing inbound internet traffic.

Traditionally, NAT Gateways were zonal resources, which came with architectural and operational overhead.

🔹 Traditional (Zonal) NAT Gateway Model

In the classic design:

One NAT Gateway per Availability Zone

NAT Gateways deployed in public subnets

AZ-specific route tables pointing to the local NAT Gateway

More AZs = more NAT Gateways + more routing complexity

Challenges:

Increased cost due to multiple NAT Gateways

Operational overhead managing routes per AZ

Public subnet dependency

Risk of misconfiguration

IP exhaustion concerns at scale

Because of these factors, many teams tried to avoid NAT Gateways where possible.

🔹 New: Regional NAT Gateway

The Regional NAT Gateway fundamentally changes this model.

Key Characteristics:

Created at the VPC level

No public subnets required

Automatically scales across all Availability Zones

A single NAT Gateway serves private subnets in all AZs

AWS manages high availability internally

This eliminates the need to think about NAT placement per AZ.

✅ Benefits

Simpler Architecture

Fewer NAT Gateways

Fewer route tables

Cleaner VPC design

Improved Security

No public subnets required

Reduced blast radius and misconfiguration risk

Built-in High Availability

No need to deploy or manage NATs per AZ

Automatically adapts as workloads scale across AZs

Lower Operational Overhead

Less maintenance

Fewer moving parts

Easier to reason about networking

📊 Architectural Impact

In AWS reference diagrams, you’ll notice that Regional NAT Gateways completely remove public subnets from the design. NAT now operates cleanly at the VPC level, which results in a more elegant and maintainable architecture.

Final Thoughts

Historically, many teams avoided NAT Gateways due to:

Cost

Maintenance complexity

IP exhaustion risks

With Regional NAT Gateway, most of these concerns are significantly reduced. You get:

Simpler architecture

Built-in high availability

Less operational burden

For modern AWS environments, this is a change well worth evaluating and adopting.

Happy Learning

Prithiviraj Rengarajan

Deploying a Professional Kubernetes Web UI Dashboard (kubectl-webui)

Prithiviraj R — Fri, 05 Dec 2025 09:07:42 +0000

A practical, step‑by‑step guide to containerizing, publishing, and running a lightweight Kubernetes Web UI on EKS using ECR, eksctl and standard Kubernetes manifests.

Why this guide?

Working with Kubernetes purely from the CLI is powerful, but sometimes you need a small, secure web UI to help teams quickly inspect resources, tail logs, and run quick in‑pod commands without giving full kubectl access. This blog post walks through building kubectl-webui—a production‑aware Flask UI packaged as a container, pushed to ECR, and deployed to an AWS EKS cluster with RBAC, sample apps for testing, and a short troubleshooting section.

Prerequisites

Make sure you have the following installed and configured:

AWS CLI (aws --version) and credentials (run aws configure).
Docker (docker --version).
kubectl (kubectl version --client).
eksctl (eksctl version).

Verify AWS identity with:

aws sts get-caller-identity

Project overview

Repository structure (important files):

kubectl-webui/
├── app.py # Flask application
├── Dockerfile # Docker build configuration
├── requirements.txt # Python dependencies
├── templates/ # HTML templates
├── static/ # CSS / assets
├── scripts/ # helper shell scripts
├── nginx.conf
└── supervisord.conf

Build the Docker image

Dockerfile highlights:

Base image: python:3.9-slim.
Installs kubectl and AWS CLI inside the image for cluster interactions.
Uses supervisord + nginx to serve the app alongside process supervision.
Adds an unprivileged app user and copies application files.

Build command (AMD64 for EKS node compatibility):

docker build --platform linux/amd64 -t kubectl-webui-professional:amd64 .

Verify the image exists:

docker images | grep kubectl-webui

Push image to Amazon ECR

Create the repository, login, tag and push:

ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
REGION="us-west-2"
REPO_NAME="kubectl-webui"


aws ecr create-repository --repository-name $REPO_NAME --region $REGION
aws ecr get-login-password --region $REGION | docker login --username AWS --password-stdin $ACCOUNT_ID.dkr.ecr.$REGION.amazonaws.com


docker tag kubectl-webui-professional:amd64 $ACCOUNT_ID.dkr.ecr.$REGION.amazonaws.com/$REPO_NAME:professional


docker push $ACCOUNT_ID.dkr.ecr.$REGION.amazonaws.com/$REPO_NAME:professional

Confirm the image with:

aws ecr describe-images --repository-name $REPO_NAME --region $REGION

Create an EKS cluster

Create a managed cluster for testing or production readiness:

eksctl create cluster \
--name kubectl-webui-cluster \
--region us-west-2 \
--nodegroup-name kubectl-webui-nodes \
--node-type t3.medium \
--nodes 2 --nodes-min 1 --nodes-max 3 --managed


aws eks update-kubeconfig --region us-west-2 --name kubectl-webui-cluster
kubectl get nodes

Deploy the Web UI and RBAC manifests

A single YAML (kubectl-webui-k8s.yaml) contains Deployment, Service (LoadBalancer), ServiceAccount, ClusterRole and ClusterRoleBinding. Key points:

Deployment references the ECR image URI (replace ACCOUNT_ID and REGION).
Container listens on 5000 (Flask) and is exposed through an ELB on port 80.
RBAC is scoped to read resources (get, list, watch) for pods, services, nodes, deployments and related objects.

Apply the manifests and watch rollout:

kubectl apply -f kubectl-webui-k8s.yaml
kubectl rollout status deployment/kubectl-webui
kubectl get svc kubectl-webui-service

Note: the LoadBalancer hostname takes a couple of minutes to provision.

Deploy sample applications (for testing)

sample-apps.yaml includes simple test workloads:

nginx-app + ClusterIP service
redis-app + ClusterIP service
busybox-app (sleeping pod) to exec into
log-collector DaemonSet (fluentd)

kubectl apply -f sample-apps.yaml

kubectl get all

Testing the dashboard

Get LoadBalancer hostname:

kubectl get svc kubectl-webui-service -o jsonpath='{.status.loadBalancer.ingress[0].hostname}'

Open the returned URL in your browser (http). Example placeholder used in the repo is an ELB hostname.
Try the UI features:

List pods, services, deployments
Describe and view pod logs
Exec into the busybox pod to run commands
View events and basic resource usage

Troubleshooting (common patterns)

ImagePullBackOff

Ensure the image exists in ECR and the image URI used in the Deployment is correct.
Use aws ecr describe-images and kubectl get deployment -o yaml | grep image to verify.

RBAC permission errors

Confirm the ServiceAccount exists: kubectl get sa kubectl-webui-sa.
Test permissions from the service account context:

kubectl auth can-i list pods --as=system:serviceaccount:default:kubectl-webui-sa

LoadBalancer stuck in Pending

Confirm cluster nodes are in public subnets with proper tagging or your cloud provider setup allows ELB provisioning.
Check kubectl describe svc kubectl-webui-service and your cloud console for ELB status.

Pod fails to start

Inspect logs: kubectl logs -l app=kubectl-webui.
Describe the pod for events: kubectl describe pod -l app=kubectl-webui.

Architecture (high level)

Developer browser → ELB (LoadBalancer) → kubectl-webui Pod → Kubernetes API Server → cluster workloads (nginx, redis, busybox)

This pattern keeps the UI stateless (single replica acceptable for small teams) and relies on EKS IAM + Kubernetes RBAC to control access.

Security & Production Notes

Do not give this UI cluster‑wide admin rights. Keep RBAC read‑only or narrowly scoped.
If exposing publicly, place the service behind an HTTPS ALB with authentication (OIDC, Cognito or a reverse proxy with auth).
Use image scanning and automated CI to build/push images, and use immutable tags (e.g. :sha-<commit>) rather than :latest.

Conclusion

The kubectl‑webui project demonstrates how a compact, secure web UI can simplify everyday cluster operations while preserving Kubernetes best practices. By containerizing the app, publishing images to ECR, and deploying on EKS with scoped RBAC, you get a reproducible, auditable workflow that teams can trust. For production use, tighten access with HTTPS and authentication, use private networking where possible, and adopt CI/CD with immutable image tags.

Happy learning

Prithiviraj Rengarajan
DevOps Engineer

Monitoring Demo: Setting Up Prometheus and Grafana on Amazon EKS

Prithiviraj R — Fri, 28 Nov 2025 11:16:13 +0000

Observability is essential in any Kubernetes environment. To monitor workloads, analyze performance, and gain visibility into cluster health, I set up Prometheus and Grafana inside my Amazon EKS cluster. This demo walks through the exact steps I followed, including the real service and pod names generated during deployment.

1. Create a Dedicated Namespace

To keep monitoring components organized:

kubectl create namespace monitoring

2. Add Helm Repositories

Prometheus and Grafana are installed using the official Helm charts.

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo add grafana https://grafana.github.io/helm-charts
helm repo update

3. Install the Kube-Prometheus Stack

The kube-prometheus-stack chart bundles everything needed for monitoring:

Prometheus
Grafana
Alertmanager
Node Exporter
Kube-State-Metrics
Prometheus Operator

Installation:

helm install kube-prom-stack prometheus-community/kube-prometheus-stack \
  --namespace monitoring

Once deployed, the following services were created in my cluster:

grafana-nodeport
prometheus-grafana
prometheus-kube-prometheus-prometheus
prometheus-kube-prometheus-operator
prometheus-kube-state-metrics
prometheus-prometheus-node-exporter
alertmanager-prometheus-kube-prometheus-alertmanager

All components came up successfully under the monitoring namespace.

4. Verify the Monitoring Pods

To ensure the monitoring stack was healthy:

kubectl get pods -n monitoring

Key components running in my demo:

prometheus-grafana-79f589d988-r74cb
prometheus-prometheus-kube-prometheus-prometheus-0
prometheus-kube-prometheus-operator-6cd4f88f57-x5mx2
prometheus-kube-state-metrics-56f99dc587-rtj22
prometheus-prometheus-node-exporter-jhnnx
alertmanager-prometheus-kube-prometheus-alertmanager-0

5. Accessing Grafana

The Helm chart created a NodePort service named grafana-nodeport, exposed on port 30080.

Option 1 — Port Forward (Local Access)

kubectl port-forward -n monitoring svc/grafana-nodeport 3000:80

Then open:

http://localhost:3000

Option 2 — NodePort (Direct Access)

If your EKS worker nodes are publicly accessible:

http://<worker-node-public-ip>:30080

Grafana Credentials

Retrieve the admin password:

kubectl get secret -n monitoring kube-prom-stack-grafana \
  -o jsonpath="{.data.admin-password}" | base64 --decode

6. Accessing Prometheus

To access the Prometheus UI:

kubectl port-forward -n monitoring \
svc/prometheus-kube-prometheus-prometheus 9090:9090

From here, you can:

Check scrape targets
Explore metrics
View alerting rules
Validate ServiceMonitor configurations

Prometheus will automatically discover and scrape the service.

7. Built-In Dashboards in Grafana

Grafana includes preloaded dashboards for:

Nodes
Pods
Workloads
Kubernetes API server
Node exporter
Kube-state-metrics

These dashboards begin populating immediately once Prometheus collects metrics.

Conclusion

This setup delivers a complete observability stack for Amazon EKS.
With Prometheus collecting metrics and Grafana providing rich visualizations, you gain:

Full visibility into cluster health
Metrics for application performance
Alerts for failures and anomalies
A foundation for long-term monitoring

Happy Learning
Prithiviraj Rengarajan
DevOps Engineer

Amazon EKS adds native support for the AWS Secrets Store CSI Driver Provider

Prithiviraj R — Sat, 22 Nov 2025 12:42:37 +0000

AWS has released a new EKS Add-on that makes it easier to securely fetch secrets from AWS Secrets Manager and SSM Parameter Store and mount them directly into Kubernetes pods — without custom plugins or manual setups.

🔐 Why this matters

Centralized & secure secrets management
Seamless integration with EKS Add-ons
Mount secrets as files inside pods
Reduced operational overhead

Available in all AWS Regions & GovCloud

Great update for teams running workloads on EKS and looking to streamline secure secret delivery across clusters.

AWS #EKS #Kubernetes #DevOps #CloudComputing #SecretsManager #SSM #AWSUpdate 🚀

Reference:
https://aws.amazon.com/about-aws/whats-new/2025/11/amazon-eks-add-ons-aws-secrets-store-csi-driver-provider/

https://github.com/aws/secrets-store-csi-driver-provider-aws

Happy Learning
Prithiviraj Rengarajan
DevOps Engineer

Kubernetes Troubleshooting with K8sGPT + Amazon Bedrock

Prithiviraj R — Fri, 21 Nov 2025 11:37:05 +0000

Introduction
Troubleshooting Kubernetes issues often requires switching between logs, events, and documentation. K8sGPT simplifies this by using AI to analyze cluster problems and generate human-readable explanations and solutions.
In this summary, I’ll walk through how I used K8sGPT with Amazon Bedrock to diagnose issues inside an EKS cluster and how it improved the entire troubleshooting experience.

What is K8sGPT?
K8sGPT is an open-source CLI and operator that scans Kubernetes resources and uses AI models to:

Detect misconfigurations
Explain errors in simple language
Provide actionable fixes
Recommend best practices

Why Integrate with Amazon Bedrock?
Amazon Bedrock enhances K8sGPT by offering:

Multiple LLM choices (Nova, Claude, Llama)
Secure, enterprise-ready identity (no API keys)
Low-latency inference from AWS regions
Cost savings using lightweight models like Nova Lite Setup Overview For the demo, I used: EKS cluster

Broken and misconfigured workloads

K8sGPT with Amazon Bedrock Nova Lite model

1. Create the EKS Cluster

I used a simple eksctl configuration to provision a small EKS cluster.

2. Install K8sGPT

Download the K8sGPT CLI and place it in your system’s PATH.
The installation is straightforward — just download the binary, extract it, and move it into /usr/local/bin/.

3. Configure K8sGPT to Use Bedrock

K8sGPT supports Bedrock natively.
You configure it by adding:

Backend provider → amazonbedrock
Bedrock region → us-east-1
Model → amazon.nova-lite-v1:0

Then set Bedrock as the default provider.

4. Create Problematic Kubernetes Resources

To test K8sGPT, I intentionally deployed workloads with issues:

Examples of injected failures

Broken Image Pod → invalid image registry
Resource Heavy Pod → impossible CPU/memory requests
StatefulSet with Wrong StorageClass → PVC creation failure

These represent issues commonly faced in real Kubernetes environments.

5. Run K8sGPT Analysis

Once workloads are deployed:
k8sgpt analyze

k8sgpt analyze --filter Pod --namespace k8sgpt-demo --explain | head -20

K8sGPT quickly identifies issues and generates fixes using AI.

Example Output (Summarized)

Image Pull Error Issue: Back-off pulling image Fix: Correct the registry/image tag
Insufficient Resources Issue: Pods cannot schedule Fix: Adjust CPU/memory or scale nodegroup
Invalid StorageClass Issue: PVCs stuck in Pending Fix: Update storage class or create a valid one

These explanations are far more readable compared to raw Kubernetes error messages.

Essential commands

1. Essential Test Commands:

Basic Analysis (No AI)

k8sgpt analyze

2. AI-Powered Analysis (Uses Bedrock)
k8sgpt analyze --explain

3. Specific Namespace

k8sgpt analyze --explain --namespace k8sgpt-demo

4.Specific Resource Type

k8sgpt analyze --explain --filter Pod

k8sgpt analyze --explain --filter Deployment

5. Multiple Filters

k8sgpt analyze --explain --filter Pod,Deployment,Service

6. Check Configuration

k8sgpt auth list

k8sgpt filters list

7. Verbose Output

k8sgpt analyze --explain --verbose

Benefits Observed

1. Intelligent Issue Detection

K8sGPT identified issues across pods, StatefulSets, services, and jobs.

2. Human-Friendly Explanations

It rewrites cryptic Kubernetes errors into simple sentences.

3. Actionable Remediation Steps

Includes:

kubectl commands
Configuration fixes
Architecture recommendations

4. Speed

It analyzed more than a dozen issues in seconds.

Amazon Bedrock Model Options

Model	Speed	Quality	Cost	Best Use Case
Nova Lite	Fastest	Great	Cheapest	Day-to-day troubleshooting
Nova Pro	High	Excellent	Moderate	Complex multi-resource analysis
Claude 3 Haiku	Fast	High	Good	Detailed explanations

Best recommendation: Start with Nova Lite.

Fixing Issues (Summary)

Fix Image Pull Failure

Delete the broken pod and recreate it with a valid image.

Fix Misconfigured Deployment

Patch Deployment or update chart values.

Fix Storage Issues

Correct the StorageClass or create a valid one.

Production Considerations

Security

Use IAM Roles
Avoid embedding credentials
Enable CloudTrail auditing

Cost

Monitor Bedrock usage
Use model tiering based on workload

Integrations

Prometheus + Grafana
Alerting systems
CI/CD pre-deployment checks

In-Cluster Operator Deployment

You can deploy K8sGPT operator using Helm and set Bedrock as the backend for ongoing cluster analysis.

Before vs After K8sGPT

Before

Manual debugging
Slow MTTR
High cognitive load
Heavy dependency on senior engineers

After

Fast AI-driven diagnostics
Consistent solutions
Junior engineers troubleshoot confidently
Reduced MTTR significantly

Conclusion

K8sGPT combined with Amazon Bedrock is a powerful way to modernize Kubernetes troubleshooting. It minimizes time spent on debugging, improves team efficiency, and brings clarity to complex cluster issues.

If you're managing EKS environments at scale, this integration provides:

Faster resolutions
Clearer insights
Better operational consistency

The future of Kubernetes troubleshooting is AI-driven, and tools like K8sGPT make that future easy to adopt.

Quick Start Checklist

Configure AWS credentials
Install K8sGPT
Add Amazon Bedrock as AI backend
Deploy workloads
Run k8sgpt analyze
Apply the recommended fixes

Happy Learning
Prithiviraj Rengarajan
DevOps Engineer

The Smart Way to Automate AWS Cost Audits Across Multiple Regions

Prithiviraj R — Fri, 17 Oct 2025 15:33:44 +0000

🌍 Why AWS Cost Audits Matter

Running workloads across multiple AWS regions gives you flexibility — but it also hides a silent cost problem: idle resources.
An EC2 instance left running, an unattached EBS volume, or an unused Elastic IP can quietly drain your monthly budget.

That’s where the AWS Cost Audit Tool comes in.
It’s a lightweight Dockerized app that scans all AWS regions, finds unused resources, and shows your potential monthly savings — all through a simple web dashboard.

I recently ran it across 17 regions, and the results were eye-opening.

⚙️ What the Tool Does

✅ Scans all AWS regions
✅ Detects idle EC2 instances, EBS volumes, and Elastic IPs
✅ Calculates potential monthly savings
✅ Displays real-time progress in a web UI

No need for complex APIs or third-party dashboards — just Docker, AWS credentials, and your browser.

🚀 Quick Start Guide

1️⃣ Prerequisites

Docker installed
AWS credentials configured
Any web browser
clone the github URL "https://github.com/prithiviraj123/aws-cost-audit-v1"

2️⃣ Configure AWS Credentials

Create a .env file in your project folder:

AWS_ACCESS_KEY_ID=your-access-key-here
AWS_SECRET_ACCESS_KEY=your-secret-key-here
AWS_DEFAULT_REGION=us-east-1

3️⃣ Deploy the Application

chmod +x deploy-simple.sh
./deploy-simple.sh

Then open: 👉 http://localhost:8080

🧭 How to Use It

Step 1: Test AWS Connection

Click “Test AWS Connection”.
If everything’s configured correctly, you’ll see:

✅ Connected to AWS Account: [your-account-id]

Step 2: Run Full Audit

Click “Start Full Audit.”
The app will scan all regions — watch the progress bar and logs update live.
Takes around 5–15 minutes, depending on the number of resources.

Step 3: Review Results

When complete, you’ll see:

Total Idle Resources
Estimated Monthly Savings
Detailed Recommendations per resource

You can also export results to Excel for record-keeping or reporting.

If it says “Your AWS account is optimized!” — that’s a good thing 🎉

Excel Report

💡 What Gets Scanned

Resource Type	Criteria	Recommended Action	Estimated Savings
EC2 Instances	CPU usage < 10% (24h)	Stop or downsize	$8–200+/month
EBS Volumes	Unattached (available)	Delete	$0.08–0.125 / GB / month
Elastic IPs	Not associated	Release	$3.65/month per IP

This gives you visibility and actionable insights without manual digging.

📊 Real Results

After scanning 17 AWS regions, the tool revealed:

Multiple idle EC2 instances
Dozens of unused EBS volumes
A few unassociated Elastic IPs

That translated to immediate cost savings and a more optimized environment.

🧠 Final Thoughts

The AWS Cost Audit Tool proves that powerful automation doesn’t have to be complicated.
In just a few commands, you can audit your entire AWS footprint, visualize idle resources, and cut waste.

No third-party costs, no manual digging — just DevOps simplicity at its best.

💻 Simple setup. Real savings. Zero guesswork.
🚀 Run your own audit today and watch your AWS bill shrink!

Happy Learning

By Prithiviraj Rengarajan | DevOps & Cloud Engineer