DEV Community: Oluwafemi Lawal

Go vs Python on AWS Lambda: a practical Terraform shootout (with Function URLs)

Oluwafemi Lawal — Thu, 18 Dec 2025 23:52:37 +0000

If you've been living in AWS for a while, you've probably seen a pattern:

Python becomes the "default Lambda language" because it's quick to write and easy to ship.
Go shows up when someone says, "This Lambda is basically a tiny compute worker… why is it slow and expensive?"

In this post, we'll build the same Lambda twice (Go and Python), deploy both with Terraform, put them behind Lambda Function URLs, and run a simple benchmark that makes Go look unfair, on purpose.

Use case: Telemetry burst aggregation

A client sends an array of events (endpoint, status code, duration). The function returns:

total events

unique users

status code counts

p95 duration (requires sorting a big slice/list)

top endpoints by hit count

This is a very "Lambda-ish" job: bursty, CPU-heavy, and latency-sensitive.

Heads-up: Go's managed runtime is deprecated

AWS deprecated the go1.x managed runtime. Go is still supported on Lambda, but you run it through an OS-only runtime such as provided.al2023 (custom runtime) and ship a bootstrap executable in your zip.

Docs:

Lambda runtimes: https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtimes.html
Go on Lambda: https://docs.aws.amazon.com/lambda/latest/dg/lambda-golang.html
Custom runtime entrypoint: https://docs.aws.amazon.com/lambda/latest/dg/runtimes-custom.html

This matters because it slightly changes packaging (but you still write normal Go with aws-lambda-go).

What you'll build

Project available at https://github.com/Femi-lawal/go_v_py_lambda

Two endpoints:

telemetry-go → provided.al2023 → compiled bootstrap
telemetry-py → python3.12 → app.handler

Both exposed through Function URLs so you can curl them without API Gateway.

⚠️ For a real system, do not use AuthType = NONE in production. Use AWS_IAM or put CloudFront/WAF in front.

Function URL docs: https://docs.aws.amazon.com/lambda/latest/dg/urls-configuration.html

Why Go outshines Python in this specific setup

Go advantages (for this use case)

Faster cold starts (compiled binary, minimal runtime overhead)
CPU-bound speed (JSON decode + sorting + maps)
Lower memory pressure (often fewer "surprise" allocations)
Easy to parallelize parsing/aggregation with goroutines when it makes sense

Python advantages (in general)

Fast iteration and fewer steps to "get something running"
Huge ecosystem (data parsing, scientific libs, AWS tooling)
Great when your Lambda is mostly orchestration (glue code) or IO-bound work

The trade-off

Go is fantastic when your Lambda is:

hot path
CPU-heavy
cost-sensitive at scale

Python is fantastic when your Lambda is:

mostly IO (calling APIs, DynamoDB, S3, etc.)
rapidly changing
maintained by teams who live in Python day-to-day

Project layout

Here's a small repo layout you can copy:

.
├── infra
│   ├── main.tf
│   ├── versions.tf
│   └── outputs.tf
├── lambda-go
│   ├── go.mod
│   ├── go.sum          # auto-generated by go mod tidy
│   └── main.go
├── lambda-py
│   └── app.py
└── scripts
    ├── gen_payload.py
    └── bench.sh

The Lambda code (same logic, two languages)

1) Python: `lambda-py/app.py`

import json
import time

def handler(event, context):
    start = time.time()

    body = event.get("body") or ""
    if event.get("isBase64Encoded"):
        import base64
        body = base64.b64decode(body)

    if isinstance(body, (bytes, bytearray)):
        body = body.decode("utf-8")

    try:
        events_ = json.loads(body) if body else []
        if not isinstance(events_, list):
            raise ValueError("Expected a JSON array")
    except Exception as e:
        return {
            "statusCode": 400,
            "headers": {"content-type": "application/json"},
            "body": json.dumps({"error": "invalid request", "detail": str(e)}),
        }

    counts = {}
    status_counts = {}
    unique_users = set()
    durations = []

    for ev in events_:
        if not isinstance(ev, dict):
            continue

        endpoint = ev.get("endpoint", "unknown")
        counts[endpoint] = counts.get(endpoint, 0) + 1

        status = str(ev.get("status", "unknown"))
        status_counts[status] = status_counts.get(status, 0) + 1

        uid = ev.get("user_id")
        if uid is not None:
            unique_users.add(str(uid))

        d = ev.get("duration_ms", 0)
        try:
            durations.append(int(d))
        except Exception:
            durations.append(0)

    durations.sort()
    p95 = durations[int(0.95 * (len(durations) - 1))] if durations else 0

    top_endpoints = sorted(counts.items(), key=lambda kv: kv[1], reverse=True)[:5]

    compute_ms = int((time.time() - start) * 1000)

    return {
        "statusCode": 200,
        "headers": {"content-type": "application/json"},
        "body": json.dumps(
            {
                "total_events": len(events_),
                "unique_users": len(unique_users),
                "p95_duration_ms": p95,
                "top_endpoints": top_endpoints,
                "status_counts": status_counts,
                "compute_ms": compute_ms,
                "language": "python",
            }
        ),
    }

2) Go: `lambda-go/main.go`

Go on provided.al2023 expects your deployment zip to contain an executable named bootstrap at the root.

package main

import (
    "context"
    "encoding/json"
    "sort"
    "time"

    "github.com/aws/aws-lambda-go/events"
    "github.com/aws/aws-lambda-go/lambda"
)

type TelemetryEvent struct {
    Endpoint   string `json:"endpoint"`
    Status     int    `json:"status"`
    DurationMS int    `json:"duration_ms"`
    UserID     string `json:"user_id"`
}

type Response struct {
    TotalEvents     int               `json:"total_events"`
    UniqueUsers     int               `json:"unique_users"`
    P95DurationMS   int               `json:"p95_duration_ms"`
    TopEndpoints    [][2]interface{}  `json:"top_endpoints"`
    StatusCounts    map[string]int    `json:"status_counts"`
    ComputeMS       int64             `json:"compute_ms"`
    Language        string            `json:"language"`
    Error           string            `json:"error,omitempty"`
    ErrorDetail     string            `json:"detail,omitempty"`
}

func handler(ctx context.Context, req events.LambdaFunctionURLRequest) (events.LambdaFunctionURLResponse, error) {
    start := time.Now()

    var items []TelemetryEvent
    if req.Body != "" {
        if err := json.Unmarshal([]byte(req.Body), &items); err != nil {
            out, _ := json.Marshal(Response{Error: "invalid request", ErrorDetail: err.Error(), Language: "go"})
            return events.LambdaFunctionURLResponse{
                StatusCode: 400,
                Headers:    map[string]string{"content-type": "application/json"},
                Body:       string(out),
            }, nil
        }
    }

    counts := make(map[string]int, 64)
    statusCounts := make(map[string]int, 16)
    unique := make(map[string]struct{}, 128)
    durations := make([]int, 0, len(items))

    for _, ev := range items {
        ep := ev.Endpoint
        if ep == "" {
            ep = "unknown"
        }
        counts[ep]++

        statusCounts[itoa(ev.Status)]++

        if ev.UserID != "" {
            unique[ev.UserID] = struct{}{}
        }

        durations = append(durations, ev.DurationMS)
    }

    sort.Ints(durations)
    p95 := 0
    if len(durations) > 0 {
        p95 = durations[int(0.95*float64(len(durations)-1))]
    }

    // Top 5 endpoints
    type kv struct {
        k string
        v int
    }
    tmp := make([]kv, 0, len(counts))
    for k, v := range counts {
        tmp = append(tmp, kv{k: k, v: v})
    }
    sort.Slice(tmp, func(i, j int) bool { return tmp[i].v > tmp[j].v })
    topN := 5
    if len(tmp) < topN {
        topN = len(tmp)
    }
    top := make([][2]interface{}, 0, topN)
    for i := 0; i < topN; i++ {
        top = append(top, [2]interface{}{tmp[i].k, tmp[i].v})
    }

    out, _ := json.Marshal(Response{
        TotalEvents:   len(items),
        UniqueUsers:   len(unique),
        P95DurationMS: p95,
        TopEndpoints:  top,
        StatusCounts:  statusCounts,
        ComputeMS:     time.Since(start).Milliseconds(),
        Language:      "go",
    })

    return events.LambdaFunctionURLResponse{
        StatusCode: 200,
        Headers:    map[string]string{"content-type": "application/json"},
        Body:       string(out),
    }, nil
}

// tiny int->string helper (avoids fmt.Sprintf)
func itoa(n int) string {
    if n == 0 {
        return "0"
    }
    neg := false
    if n < 0 {
        neg = true
        n = -n
    }
    var buf [12]byte
    i := len(buf)
    for n > 0 {
        i--
        buf[i] = byte('0' + n%10)
        n /= 10
    }
    if neg {
        i--
        buf[i] = '-'
    }
    return string(buf[i:])
}

func main() {
    lambda.Start(handler)
}

lambda-go/go.mod:

module telemetry-go

go 1.22

require github.com/aws/aws-lambda-go v1.48.0

Terraform: deploy both functions the same way

`infra/versions.tf`

terraform {
  required_version = ">= 1.5.0"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      # Needs support for provided.al2023 and modern runtimes.
      version = ">= 5.26.0"
    }
  }
}

provider "aws" {
  region = var.aws_region
}

`infra/main.tf`

variable "aws_region" {
  type    = string
  default = "us-east-1"
}

locals {
  project = "go-vs-python-lambda"
}

data "aws_caller_identity" "current" {}

resource "aws_iam_role" "lambda_exec" {
  name = "${local.project}-exec"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Effect = "Allow"
      Principal = { Service = "lambda.amazonaws.com" }
      Action = "sts:AssumeRole"
    }]
  })
}

resource "aws_iam_role_policy_attachment" "basic" {
  role       = aws_iam_role.lambda_exec.name
  policy_arn = "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
}

# --------------------
# Packaging
# --------------------

# Python zip: just zip the folder
data "archive_file" "py_zip" {
  type        = "zip"
  source_dir  = "${path.module}/../lambda-py"
  output_path = "${path.module}/dist/lambda-py.zip"
}

# Go zip: expects dist/bootstrap already built
data "archive_file" "go_zip" {
  type        = "zip"
  source_file = "${path.module}/dist/bootstrap"
  output_path = "${path.module}/dist/lambda-go.zip"
}

# --------------------
# Lambdas
# --------------------

resource "aws_lambda_function" "py" {
  function_name = "${local.project}-py"
  role          = aws_iam_role.lambda_exec.arn

  filename         = data.archive_file.py_zip.output_path
  source_code_hash = data.archive_file.py_zip.output_base64sha256

  runtime = "python3.12"
  handler = "app.handler"

  timeout      = 10
  memory_size  = 512
  architectures = ["arm64"]
}

resource "aws_lambda_function" "go" {
  function_name = "${local.project}-go"
  role          = aws_iam_role.lambda_exec.arn

  filename         = data.archive_file.go_zip.output_path
  source_code_hash = data.archive_file.go_zip.output_base64sha256

  runtime = "provided.al2023"
  handler = "bootstrap"

  timeout      = 10
  memory_size  = 512
  architectures = ["arm64"]
}

# --------------------
# Function URLs (public for demo)
# --------------------

resource "aws_lambda_function_url" "py" {
  function_name      = aws_lambda_function.py.function_name
  authorization_type = "NONE"
}

resource "aws_lambda_function_url" "go" {
  function_name      = aws_lambda_function.go.function_name
  authorization_type = "NONE"
}

# As of late 2024, Function URLs may require BOTH InvokeFunctionUrl and InvokeFunction permissions
# depending on your account settings. We grant both for compatibility.
resource "aws_lambda_permission" "py_url" {
  statement_id           = "AllowPublicInvokeUrlPy"
  action                 = "lambda:InvokeFunctionUrl"
  function_name          = aws_lambda_function.py.function_name
  principal              = "*"
  function_url_auth_type = "NONE"
}

resource "aws_lambda_permission" "py_invoke" {
  statement_id  = "AllowPublicInvokePy"
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.py.function_name
  principal     = "*"
}

resource "aws_lambda_permission" "go_url" {
  statement_id           = "AllowPublicInvokeUrlGo"
  action                 = "lambda:InvokeFunctionUrl"
  function_name          = aws_lambda_function.go.function_name
  principal              = "*"
  function_url_auth_type = "NONE"
}

resource "aws_lambda_permission" "go_invoke" {
  statement_id  = "AllowPublicInvokeGo"
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.go.function_name
  principal     = "*"
}

`infra/outputs.tf`

output "python_url" {
  value = aws_lambda_function_url.py.function_url
}

output "go_url" {
  value = aws_lambda_function_url.go.function_url
}

Build & deploy

1) Build the Go `bootstrap` for arm64

From the repo root:

mkdir -p infra/dist

cd lambda-go
GOOS=linux GOARCH=arm64 CGO_ENABLED=0 go build -ldflags "-s -w" -o ../infra/dist/bootstrap .
cd ..

chmod +x infra/dist/bootstrap

2) Terraform apply

cd infra
terraform init
terraform apply

Terraform will output both URLs.

Generate a payload and hit both endpoints

`scripts/gen_payload.py`

import json
import random
import string

def rand_user():
    return "u_" + "".join(random.choices(string.ascii_lowercase + string.digits, k=8))

def main(n=20000):
    endpoints = ["/login", "/search", "/checkout", "/profile", "/feed", "/ping"]
    out = []
    for _ in range(n):
        out.append({
            "endpoint": random.choice(endpoints),
            "status": random.choice([200, 200, 200, 201, 400, 401, 403, 500]),
            "duration_ms": int(random.expovariate(1/120)) + random.randint(0, 30),
            "user_id": rand_user(),
        })
    print(json.dumps(out))

if __name__ == "__main__":
    main()

Run it:

python3 scripts/gen_payload.py > payload.json

Invoke:

PY_URL="$(cd infra && terraform output -raw python_url)"
GO_URL="$(cd infra && terraform output -raw go_url)"

curl -s -X POST "$PY_URL" -H "content-type: application/json" --data-binary @payload.json | jq
curl -s -X POST "$GO_URL" -H "content-type: application/json" --data-binary @payload.json | jq

Quick benchmark (side-by-side)

Option A: `hey`

# Install hey (if you don't have it)
# macOS: brew install hey
# Linux:  https://github.com/rakyll/hey

hey -n 200 -c 20 -m POST -H "content-type: application/json" -D payload.json "$PY_URL"
hey -n 200 -c 20 -m POST -H "content-type: application/json" -D payload.json "$GO_URL"

Option B: a tiny `bench.sh`

scripts/bench.sh:

#!/usr/bin/env bash
set -euo pipefail

PY_URL="$(cd infra && terraform output -raw python_url)"
GO_URL="$(cd infra && terraform output -raw go_url)"

echo "Python URL: $PY_URL"
echo "Go URL:     $GO_URL"

for url in "$PY_URL" "$GO_URL"; do
  echo
  echo "== Benchmarking: $url"
  hey -n 200 -c 20 -m POST -H "content-type: application/json" -D payload.json "$url"
done

What to look for in the results

When you benchmark:

p95 / p99 latency: Go often wins when the handler is CPU-heavy.
cold start: run once after no traffic for a while and compare first-hit latency.
cost: Lambda charges for GB-seconds. If Go finishes faster at the same memory size, your bill can drop.

Also look at CloudWatch logs and metrics:

Duration
Max memory used
Init duration (cold start)

Actual Benchmark Results

We ran a comprehensive benchmark with the following configuration:

Memory: 512 MB (both functions)
Architecture: ARM64 (Graviton2)
Region: us-east-1
Iterations per test: 30
Payload sizes: 100, 1,000, 5,000, 10,000, and 20,000 events

Latency Summary (all times in milliseconds)

Events	Payload Size	Language	Cold Start	Avg Latency	P50	P95	P99
100	8 KB	Python	272	70	66	89	98
100	8 KB	Go	136	41	39	54	55
1,000	82 KB	Python	68	72	70	82	107
1,000	82 KB	Go	46	53	52	68	74
5,000	411 KB	Python	124	124	121	151	160
5,000	411 KB	Go	135	108	102	130	140
10,000	822 KB	Python	220	205	202	236	263
10,000	822 KB	Go	190	197	199	221	261
20,000	1.6 MB	Python	422	473	460	573	595
20,000	1.6 MB	Go	357	359	359	391	396

Go vs Python Speedup Factor

Payload	Avg Latency Speedup	P95 Latency Speedup	Cold Start Speedup
100 events (8 KB)	1.70x faster	1.66x faster	2.00x faster
1,000 events (82 KB)	1.35x faster	1.21x faster	1.47x faster
5,000 events (411 KB)	1.15x faster	1.16x faster	0.92x
10,000 events (822 KB)	1.04x faster	1.07x faster	1.16x faster
20,000 events (1.6 MB)	1.32x faster	1.46x faster	1.18x faster

Key Observations

Cold Starts: Go's cold start is up to 2x faster for small payloads. At 100 events, Go cold starts at ~136ms vs Python's ~272ms. This matters for bursty, event-driven workloads.
Small Payloads Win Big: For small to medium payloads (100-1000 events), Go's advantage is most pronounced (1.35x-1.70x faster). This is where the JSON parsing and initial setup overhead dominates.
Large Payloads Converge: At 10,000+ events, the speedup narrows (1.04x-1.32x) because the actual computation time dominates over runtime overhead. However, Go maintains a consistent advantage in tail latencies (P95/P99).
Predictable Tail Latencies: Go's P95 to P99 spread is tighter. For 20,000 events:
- Python: P95=573ms, P99=595ms (22ms spread)
- Go: P95=391ms, P99=396ms (5ms spread)

This predictability is critical for SLO compliance.

Internal Compute Time: At larger payloads, Python's internal compute time (compute_ms from the response) is actually slightly lower than Go's for sorting operations (20K events: Python=117ms, Go=140ms). This is because Python's timsort is highly optimized C code, while Go's sort.Ints is pure Go. However, the compute_ms metric only measures post-JSON-parse processing. Go's faster JSON deserialization (which dominates at scale) isn't captured in this number. The total end-to-end latency still favors Go due to faster JSON parsing and lower runtime overhead.

Cost Implications

With Lambda ARM64 (Graviton2) pricing at $0.0000133334 per GB-second (20% cheaper than x86):

Scenario	Python (avg)	Go (avg)	Monthly Savings (1M invocations)
100 events	70ms	41ms	$0.20
1,000 events	72ms	53ms	$0.13
20,000 events	473ms	359ms	$0.76

Calculation: (Python_ms - Go_ms) / 1000 × 0.5 GB × $0.0000133334 × 1,000,000 invocations

At scale (100M invocations/month with 20,000-event payloads), that's approximately $76/month savings just by choosing Go, plus the inherent 20% ARM64 discount on top of x86 pricing.

When I'd choose Go vs Python for Lambda

I reach for Go when:

this function is on the hot path (lots of traffic)
it does non-trivial compute (parsing, compression, crypto, transforms)
I care about predictable latency at scale
I want small deployment packages and simple dependencies

I reach for Python when:

the function mostly orchestrates AWS services
the logic changes often (product iteration)
I need a library ecosystem advantage
it's a glue script with real business value and low perf pressure

Final notes and improvements

If you want to make this "production-ish":

flip Function URLs to AWS_IAM auth (or front with CloudFront/WAF)
set log retention (so you don't pay forever)
add structured logging + tracing
add provisioned concurrency if you need consistent latency under cold starts

Additional Benchmark Scenarios to Try

1. Memory Configuration Comparison

Try running the same benchmarks at different memory levels:

# In main.tf, change memory_size to test:
memory_size = 256   # Minimum viable
memory_size = 512   # Balanced (our test)
memory_size = 1024  # 2x CPU
memory_size = 1769  # Full vCPU
memory_size = 3008  # 2 vCPUs

Expected behavior: Go benefits less from extra memory since it's already efficient. Python may see bigger gains from more CPU (Lambda CPU scales with memory).

2. Concurrent Request Simulation

Test how each handles concurrent bursts:

# Using hey for concurrency testing
hey -n 500 -c 50 -m POST -H "content-type: application/json" -D payload.json "$GO_URL"
hey -n 500 -c 50 -m POST -H "content-type: application/json" -D payload.json "$PY_URL"

Watch for:

Error rates under load
Latency distribution (P50 vs P99 gap)
Throttling behavior

3. I/O-Bound Workload Comparison

Modify the Lambda to include DynamoDB or S3 calls:

# In app.py - add I/O operation
import boto3
dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('telemetry-events')

# Write aggregated result to DynamoDB
table.put_item(Item={'id': str(uuid.uuid4()), 'result': result})

Expected result: The Go vs Python gap narrows significantly when I/O dominates. For pure I/O workloads, Python's simplicity often wins.

4. Package Size Impact

Compare deployment package sizes:

Language	Package Size	Contains
Go	6.3 MB	Single static binary
Python	1.2 KB	Just `app.py`

But with dependencies:

Language	Package Size	Contains
Go	6.3 MB	Still just the binary
Python + pandas + numpy	50+ MB	Requires layers or container

5. Provisioned Concurrency Test

To eliminate cold starts entirely:

resource "aws_lambda_provisioned_concurrency_config" "go" {
  function_name                     = aws_lambda_function.go.function_name
  provisioned_concurrent_executions = 10
  qualifier                         = aws_lambda_function.go.version
}

This adds ~$0.000004463 per GB-second for provisioned capacity.

Real-World Use Cases Where These Results Matter

✅ Good fit for Go Lambda:

API Gateway backends - Low latency matters for user-facing APIs
Kinesis/SQS processors - High throughput event processing
Real-time data aggregation - Like our telemetry example
Image/video thumbnail generation - CPU-bound transforms
JWT validation layers - Crypto operations benefit from Go

✅ Good fit for Python Lambda:

ML inference with SageMaker - Rich SDK ecosystem
Data pipeline orchestration - Step Functions triggers
S3 event handlers - boto3 makes this trivial
Slack/Discord bots - Rapid iteration matters more
One-off automation scripts - Time-to-deployment wins

Cleanup

Don't forget to tear down your infrastructure when done:

cd infra
terraform destroy -auto-approve

Reproducibility

To reproduce these benchmarks yourself:

# 1. Clone and enter the project
git clone https://github.com/Femi-lawal/go_v_py_lambda
cd go_v_py_lambda

# 2. Build the Go binary for ARM64 Lambda
cd lambda-go
go mod tidy
GOOS=linux GOARCH=arm64 CGO_ENABLED=0 go build -ldflags "-s -w" -o ../infra/dist/bootstrap .
cd ..

# 3. Deploy with Terraform
cd infra
terraform init
terraform apply -auto-approve

# 4. Generate test payload
python3 scripts/gen_payload.py 20000 > payload.json

# 5. Run benchmarks (requires 'hey' - install via: go install github.com/rakyll/hey@latest)
PY_URL="$(terraform output -raw python_url)"
GO_URL="$(terraform output -raw go_url)"

hey -n 100 -c 10 -m POST -H "content-type: application/json" -D ../payload.json "$PY_URL"
hey -n 100 -c 10 -m POST -H "content-type: application/json" -D ../payload.json "$GO_URL"

# 6. Cleanup when done
terraform destroy -auto-approve

Summary

Our benchmarks confirm the conventional wisdom with hard numbers:

Metric	Go Advantage
Cold Start (small payload)	2x faster
Warm Latency (small payload)	1.7x faster
Warm Latency (large payload)	1.3x faster
P95 Tail Latency	1.2x-1.7x tighter
Monthly Cost Savings (ARM64)	~$76 at 100M requests

The takeaway: If your Lambda is on the hot path, does CPU work, and you care about cost or latency SLOs, Go is worth the learning curve. If you're gluing AWS services together or iterating rapidly, Python's ergonomics win.

Happy building

Navigating AWS EKS with Terraform: Configuring Karpenter for Just-in-Time Node Provisioning

Oluwafemi Lawal — Thu, 20 Nov 2025 23:08:41 +0000

In this article, we will integrate Karpenter, which will enable our cluster to provision nodes dynamically based on actual workload requirements. Unlike Cluster Autoscaler which scales existing node groups, Karpenter provisions exactly the right-sized nodes on demand. We will need to make some adjustments to our EKS module.

Prerequisites

Ensure you have:

An active AWS account
Terraform installed and configured
Helm installed
kubectl

Setup Overview

In the previous part of the series, we installed and configured the Kubernetes Cluster Autoscaler (CA) on an EKS cluster provisioned with Terraform. That article showed how to let Kubernetes resize existing node groups when Pods can't be scheduled.

Karpenter takes a different approach; it's an open-source autoscaling solution from AWS that goes beyond resizing existing node groups. Instead, Karpenter talks directly to AWS and launches the right EC2 instances on demand to fit the Pods that are actually pending.

Key Architectural Difference: Self-Managed Nodes

Important: Nodes provisioned by Karpenter will appear as "Self-managed" in the AWS EKS console, while Cluster Autoscaler nodes show as part of "Managed Node Groups". This is by design:

Cluster Autoscaler: Works within existing Auto Scaling Groups (ASGs) or EKS Managed Node Groups. It scales by adjusting the desired capacity of predefined node groups.
Karpenter: Provisions nodes as individual EC2 instances directly using the AWS API, bypassing ASGs and node groups entirely. This gives Karpenter the flexibility to dynamically select instance types, launch nodes faster, and optimize costs.

Both approaches result in fully functional nodes in your cluster - the "self-managed" designation simply indicates that Karpenter manages the instance lifecycle independently. This architecture enables Karpenter's key advantages: faster provisioning (30-60 seconds vs 2-5 minutes), dynamic instance type selection, and more efficient resource utilization.

For more details, see:

Prepare Your EKS Cluster

Ensure your EKS cluster is ready. If needed, refer back to our guide on setting up an EKS cluster using Terraform

Modifications

To support Karpenter's auto-discovery feature, we need to update the tags on our EKS resources. Unlike Cluster Autoscaler which used k8s.io/cluster-autoscaler/* tags, Karpenter uses karpenter.sh/discovery tags to identify the subnets and security groups it can use when provisioning nodes.

Update EKS Module to Support New Parameters

First, we need to extend our EKS module to support the parameters required for Karpenter. Add these variables to modules/aws/eks/v1/variables.tf:

variable "eks_cluster_version" {
  description = "Kubernetes version for the EKS cluster"
  type        = string
  default     = null
}

variable "additional_role_mappings" {
  description = "Additional IAM role mappings for aws-auth ConfigMap"
  type = list(object({
    rolearn  = string
    username = string
    groups   = list(string)
  }))
  default = []
}

variable "additional_launch_template_tags" {
  description = "Additional tags to add to the launch template"
  type        = map(string)
  default     = {}
}

Update modules/aws/eks/v1/main.tf to use these variables:

Add version to the EKS cluster:

resource "aws_eks_cluster" "main" {
  name                      = var.cluster_name
  role_arn                  = aws_iam_role.eks_cluster_role.arn
  enabled_cluster_log_types = var.enabled_cluster_log_types
  version                   = var.eks_cluster_version  # Add this line
  # ... rest of configuration
}

Update the launch template tags to merge additional tags:

resource "aws_launch_template" "eks_node_group" {
  # ... existing configuration ...

  tags = merge(
    {
      "Name"                                      = "${var.cluster_name}-eks-node-group"
      "kubernetes.io/cluster/${var.cluster_name}" = "owned"
    },
    var.additional_launch_template_tags
  )
}

Update the aws-auth ConfigMap to include additional role mappings:

resource "kubernetes_config_map" "aws_auth" {
  metadata {
    name      = "aws-auth"
    namespace = "kube-system"
  }

  data = {
    mapRoles = yamlencode(concat(
      [
        {
          rolearn  = aws_iam_role.eks_admins_role.arn
          username = aws_iam_role.eks_admins_role.name
          groups   = ["system:masters"]
        },
        {
          rolearn  = aws_iam_role.node_role.arn
          username = "system:node:{{EC2PrivateDNSName}}"
          groups   = ["system:bootstrappers", "system:nodes"]
        }
      ],
      var.additional_role_mappings
    ))
    # ... rest of configuration
  }
}

Update the nodes security group to include additional tags:

resource "aws_security_group" "eks_nodes_sg" {
  name        = "${var.cluster_name}-eks-nodes-sg"
  description = "Security group for all nodes in the cluster"
  vpc_id      = var.vpc_id

  tags = merge(
    {
      Name                                        = "${var.cluster_name}-eks-nodes-sg"
      "kubernetes.io/cluster/${var.cluster_name}" = "owned"
    },
    var.additional_launch_template_tags
  )
}

Add new outputs to modules/aws/eks/v1/outputs.tf:

output "oidc_provider_url" {
  description = "The OIDC URL of the Cluster"
  value       = aws_iam_openid_connect_provider.eks.url
}

output "node_instance_role_arn" {
  description = "The ARN of the node instance role"
  value       = aws_iam_role.node_role.arn
}

Update VPC and EKS Configuration

Critical: Karpenter needs to discover subnets and security groups using tags. You must add the karpenter.sh/discovery tag to private subnets only (not public subnets) and to the node security group.

Why private subnets only? Karpenter-provisioned nodes need outbound internet access to download container images and communicate with the EKS control plane. Private subnets with NAT gateway provide this access. If you tag public subnets, Karpenter might launch nodes there without public IPs, causing them to fail to join the cluster.

Update vpc.tf to add Karpenter discovery tags:

module "vpc" {
  source = "./modules/aws/vpc/v1"

  vpc_name             = "${var.cluster_name}-vpc"
  cidr_block           = "10.0.0.0/16"
  nat_gateway          = true
  enable_dns_support   = true
  enable_dns_hostnames = true

  public_subnet_count  = 3
  private_subnet_count = 3
  public_subnet_tags = {
    "kubernetes.io/cluster/${var.cluster_name}" = "shared"
    "kubernetes.io/role/elb"                    = "1"
    # DO NOT add karpenter.sh/discovery tag to public subnets
  }

  private_subnet_tags = {
    "kubernetes.io/cluster/${var.cluster_name}" = "shared"
    "kubernetes.io/role/internal-elb"           = "1"
    "karpenter.sh/discovery"                    = var.cluster_name  # Only tag private subnets
  }
}

Now update eks.tf to configure the EKS cluster with Karpenter support:

module "eks" {
  source = "./modules/aws/eks/v1"

  region              = var.region
  cluster_name        = var.cluster_name
  private_subnets     = module.vpc.private_subnets
  public_subnets      = module.vpc.public_subnets
  vpc_id              = module.vpc.vpc_id
  eks_cluster_version = "1.31"

  managed_node_groups = {
    demo_group = {
      name           = "demo-node-group"
      desired_size   = 2
      min_size       = 1
      max_size       = 3
      instance_types = ["t3a.small"]
    }
  }

  additional_role_mappings = [
    {
      rolearn  = aws_iam_role.karpenter_controller_role.arn
      username = "system:serviceaccount:karpenter:karpenter"
      groups   = ["system:masters"]
    }
  ]

  additional_launch_template_tags = {
    "karpenter.sh/discovery" = var.cluster_name
  }
}

Note on Hybrid Architecture: This configuration uses a best practice approach where:

The managed node group (2 small nodes) hosts critical system components (CoreDNS, kube-proxy, etc.)
Karpenter dynamically provisions additional nodes for application workloads

This hybrid model provides stability for system pods while giving you Karpenter's flexibility and cost optimization for your applications.

Create IAM Role for Karpenter

The Karpenter controller needs specific permissions to interact with EC2 instances and AWS services. We'll create an IAM role that uses IRSA (IAM Roles for Service Accounts) to allow the Karpenter pods to assume this role securely.

Create a new file called karpenter.tf at the root of your repository:

############################################################################################################
### KARPENTER
############################################################################################################

# IAM Role for Karpenter Controller
data "aws_iam_policy_document" "karpenter_assume_role_policy" {
  statement {
    actions = ["sts:AssumeRoleWithWebIdentity"]
    effect  = "Allow"

    condition {
      test     = "StringEquals"
      variable = "${replace(module.eks.oidc_provider_url, "https://", "")}:sub"
      values   = ["system:serviceaccount:karpenter:karpenter"]
    }

    principals {
      identifiers = [module.eks.oidc_provider_arn]
      type        = "Federated"
    }
  }
}

resource "aws_iam_role" "karpenter_controller_role" {
  name               = "${var.cluster_name}-karpenter-controller"
  assume_role_policy = data.aws_iam_policy_document.karpenter_assume_role_policy.json
}

# Karpenter Controller Policy
data "aws_iam_policy_document" "karpenter_controller_policy" {
  statement {
    sid    = "AllowScopedEC2InstanceAccessActions"
    effect = "Allow"
    actions = [
      "ec2:RunInstances",
      "ec2:CreateFleet"
    ]
    resources = [
      "arn:aws:ec2:${var.region}::image/*",
      "arn:aws:ec2:${var.region}::snapshot/*",
      "arn:aws:ec2:${var.region}:*:security-group/*",
      "arn:aws:ec2:${var.region}:*:subnet/*",
      "arn:aws:ec2:${var.region}:*:launch-template/*"
    ]
  }

  statement {
    sid    = "AllowScopedEC2InstanceActionsWithTags"
    effect = "Allow"
    actions = [
      "ec2:RunInstances",
      "ec2:CreateFleet",
      "ec2:CreateLaunchTemplate"
    ]
    resources = [
      "arn:aws:ec2:${var.region}:*:fleet/*",
      "arn:aws:ec2:${var.region}:*:instance/*",
      "arn:aws:ec2:${var.region}:*:volume/*",
      "arn:aws:ec2:${var.region}:*:network-interface/*",
      "arn:aws:ec2:${var.region}:*:launch-template/*",
      "arn:aws:ec2:${var.region}:*:spot-instances-request/*"
    ]
  }

  statement {
    sid    = "AllowScopedResourceCreationTagging"
    effect = "Allow"
    actions = [
      "ec2:CreateTags"
    ]
    resources = [
      "arn:aws:ec2:${var.region}:*:fleet/*",
      "arn:aws:ec2:${var.region}:*:instance/*",
      "arn:aws:ec2:${var.region}:*:volume/*",
      "arn:aws:ec2:${var.region}:*:network-interface/*",
      "arn:aws:ec2:${var.region}:*:launch-template/*",
      "arn:aws:ec2:${var.region}:*:spot-instances-request/*"
    ]
  }

  statement {
    sid    = "AllowScopedResourceTagging"
    effect = "Allow"
    actions = [
      "ec2:CreateTags"
    ]
    resources = [
      "arn:aws:ec2:${var.region}:*:instance/*"
    ]
  }

  statement {
    sid    = "AllowScopedDeletion"
    effect = "Allow"
    actions = [
      "ec2:TerminateInstances",
      "ec2:DeleteLaunchTemplate"
    ]
    resources = [
      "arn:aws:ec2:${var.region}:*:instance/*",
      "arn:aws:ec2:${var.region}:*:launch-template/*"
    ]
  }

  statement {
    sid    = "AllowRegionalReadActions"
    effect = "Allow"
    actions = [
      "ec2:DescribeAvailabilityZones",
      "ec2:DescribeImages",
      "ec2:DescribeInstances",
      "ec2:DescribeInstanceTypeOfferings",
      "ec2:DescribeInstanceTypes",
      "ec2:DescribeLaunchTemplates",
      "ec2:DescribeSecurityGroups",
      "ec2:DescribeSpotPriceHistory",
      "ec2:DescribeSubnets"
    ]
    resources = ["*"]
  }

  statement {
    sid    = "AllowSSMReadActions"
    effect = "Allow"
    actions = [
      "ssm:GetParameter"
    ]
    resources = [
      "arn:aws:ssm:${var.region}::parameter/aws/service/*"
    ]
  }

  statement {
    sid    = "AllowPricingReadActions"
    effect = "Allow"
    actions = [
      "pricing:GetProducts"
    ]
    resources = ["*"]
  }

  statement {
    sid    = "AllowInterruptionQueueActions"
    effect = "Allow"
    actions = [
      "sqs:DeleteMessage",
      "sqs:GetQueueUrl",
      "sqs:GetQueueAttributes",
      "sqs:ReceiveMessage"
    ]
    resources = ["arn:aws:sqs:${var.region}:*:${var.cluster_name}"]
  }

  statement {
    sid    = "AllowPassNodeIAMRole"
    effect = "Allow"
    actions = [
      "iam:PassRole"
    ]
    resources = [
      module.eks.node_instance_role_arn
    ]
  }

  statement {
    sid    = "AllowScopedInstanceProfileCreationActions"
    effect = "Allow"
    actions = [
      "iam:CreateInstanceProfile"
    ]
    resources = [
      "arn:aws:iam::*:instance-profile/*"
    ]
  }

  statement {
    sid    = "AllowScopedInstanceProfileTagActions"
    effect = "Allow"
    actions = [
      "iam:TagInstanceProfile"
    ]
    resources = [
      "arn:aws:iam::*:instance-profile/*"
    ]
  }

  statement {
    sid    = "AllowScopedInstanceProfileActions"
    effect = "Allow"
    actions = [
      "iam:AddRoleToInstanceProfile",
      "iam:RemoveRoleFromInstanceProfile",
      "iam:DeleteInstanceProfile"
    ]
    resources = [
      "arn:aws:iam::*:instance-profile/*"
    ]
  }

  statement {
    sid    = "AllowInstanceProfileReadActions"
    effect = "Allow"
    actions = [
      "iam:GetInstanceProfile"
    ]
    resources = [
      "arn:aws:iam::*:instance-profile/*"
    ]
  }

  statement {
    sid    = "AllowAPIServerEndpointDiscovery"
    effect = "Allow"
    actions = [
      "eks:DescribeCluster"
    ]
    resources = [
      "arn:aws:eks:${var.region}:*:cluster/${var.cluster_name}"
    ]
  }
}

resource "aws_iam_role_policy" "karpenter_controller_policy" {
  name   = "${var.cluster_name}-karpenter-controller-policy"
  role   = aws_iam_role.karpenter_controller_role.id
  policy = data.aws_iam_policy_document.karpenter_controller_policy.json
}

The Karpenter controller policy gives Karpenter the permissions it needs to:

Launch and terminate EC2 instances
Create and manage launch templates
Describe EC2 resources (subnets, security groups, instance types, etc.)
Manage instance profiles
Tag resources appropriately

We also pass the role mapping to the config map in the EKS module (shown in the eks.tf above), which allows the Karpenter service account to assume this IAM role.

We also need to update our Terraform outputs in ouput.tf to expose values needed by our deployment scripts:

output "karpenter_controller_role_arn" {
  description = "The ARN of the Karpenter controller IAM role"
  value       = aws_iam_role.karpenter_controller_role.arn
}

output "aws_region" {
  description = "The AWS region where the EKS cluster is deployed"
  value       = var.region
}

output "cluster_name" {
  description = "The name of the EKS cluster"
  value       = var.cluster_name
}

output "cluster_endpoint" {
  description = "The endpoint for the EKS cluster"
  value       = module.eks.cluster_endpoint
}

output "node_instance_role_arn" {
  description = "The ARN of the node instance role"
  value       = module.eks.node_instance_role_arn
}

We can then apply our changes with terraform:

terraform apply

We can confirm our cluster is up and running in the AWS console.

Install Karpenter using Helm

We have created the role mapping that the karpenter service account will use, but we still need to create the actual service account. The manifest to create that will look like this:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: karpenter
  namespace: karpenter
  annotations:
    eks.amazonaws.com/role-arn: "arn:aws:iam::<ACCOUNT_ID>:role/<IAM_ROLE_NAME>"

For simplicity (or just because I'm lazy), I have created scripts to help with the rest of the article. They make use of terraform outputs to avoid having to put placeholders or keep track of which command to run first.

The script scripts/deploy_karpenter.sh updates our kubectl context to that of the cluster, creates the service account for Karpenter with the IRSA annotation, and deploys Karpenter using helm:

#!/bin/bash
set -euxo pipefail

# Retrieve Terraform outputs
REGION=$(terraform output -raw aws_region)
CLUSTER_NAME=$(terraform output -raw cluster_name)
KARPENTER_ROLE_ARN=$(terraform output -raw karpenter_controller_role_arn)
CLUSTER_ENDPOINT=$(terraform output -raw cluster_endpoint)
NODE_INSTANCE_ROLE_ARN=$(terraform output -raw node_instance_role_arn)

# Extract the role name from the ARN
NODE_INSTANCE_ROLE_NAME=$(echo $NODE_INSTANCE_ROLE_ARN | awk -F'/' '{print $NF}')

# Update kubeconfig
aws eks update-kubeconfig --region $REGION --name $CLUSTER_NAME

# Create the karpenter namespace
kubectl create namespace karpenter || true

# Create the service account for Karpenter
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: ServiceAccount
metadata:
  name: karpenter
  namespace: karpenter
  annotations:
    eks.amazonaws.com/role-arn: $KARPENTER_ROLE_ARN
EOF

# Install Karpenter using Helm
export KARPENTER_VERSION="0.16.3"

helm repo add karpenter https://charts.karpenter.sh/ || true
helm repo update

helm upgrade --install karpenter karpenter/karpenter \
  --namespace karpenter \
  --create-namespace \
  --version ${KARPENTER_VERSION} \
  --set serviceAccount.create=false \
  --set serviceAccount.name=karpenter \
  --set clusterName=${CLUSTER_NAME} \
  --set clusterEndpoint=${CLUSTER_ENDPOINT} \
  --set aws.defaultInstanceProfile=${NODE_INSTANCE_ROLE_NAME} \
  --set controller.resources.requests.cpu=500m \
  --set controller.resources.requests.memory=512Mi \
  --set controller.resources.limits.cpu=1 \
  --set controller.resources.limits.memory=1Gi \
  --set replicas=1 \
  --wait || true

# Wait for Karpenter to be ready
kubectl wait --for=condition=available --timeout=300s deployment/karpenter -n karpenter || true

# Create EC2NodeClass
cat <<EOF | kubectl apply -f -
apiVersion: karpenter.k8s.aws/v1alpha1
kind: AWSNodeTemplate
metadata:
  name: default
spec:
  amiFamily: AL2
  instanceProfile: ${NODE_INSTANCE_ROLE_NAME}
  subnetSelector:
    karpenter.sh/discovery: ${CLUSTER_NAME}
  securityGroupSelector:
    karpenter.sh/discovery: ${CLUSTER_NAME}
  tags:
    karpenter.sh/discovery: ${CLUSTER_NAME}
EOF

# Create NodePool
kubectl apply -f kubernetes/karpenter/nodepool.yaml

# Deploy test deployment (starts at 0 replicas)
kubectl apply -f kubernetes/karpenter/test-deployment.yaml

echo "Karpenter installation completed successfully!"
echo ""
echo "To verify Karpenter is running:"
echo "kubectl get pods -n karpenter"
echo ""
echo "To view Karpenter logs:"
echo "kubectl logs -f -n karpenter -l app.kubernetes.io/name=karpenter"
echo ""
echo "To test autoscaling, run:"
echo "./scripts/test_karpenter.sh"

The script also creates the Karpenter AWSNodeTemplate (which tells Karpenter how to configure EC2 instances) and applies the Provisioner configuration from kubernetes/karpenter/nodepool.yaml:

apiVersion: karpenter.sh/v1alpha5
kind: Provisioner
metadata:
  name: default
spec:
  requirements:
    - key: kubernetes.io/arch
      operator: In
      values: ["amd64"]
    - key: kubernetes.io/os
      operator: In
      values: ["linux"]
    - key: karpenter.sh/capacity-type
      operator: In
      values: ["on-demand"]
    - key: karpenter.k8s.aws/instance-category
      operator: In
      values: ["t"]
    - key: karpenter.k8s.aws/instance-generation
      operator: Gt
      values: ["2"]
  providerRef:
    name: default
  limits:
    resources:
      cpu: 10
  ttlSecondsAfterEmpty: 30
  ttlSecondsUntilExpired: 2592000 # 30 days

This Provisioner configuration:

Only provisions amd64 Linux nodes
Uses on-demand instances (you can add "spot" for cost savings)
Restricts to t-family instances (t3, t3a, etc.) generation 3 or higher
Sets a CPU limit of 10 vCPUs total across all Karpenter-provisioned nodes
Automatically deprovisions nodes 30 seconds after they become empty
Expires nodes after 30 days for security patching

Run the deployment script:

chmod +x scripts/deploy_karpenter.sh
./scripts/deploy_karpenter.sh

Verify Karpenter is running:

kubectl get pods -n karpenter

You will notice that we only have two nodes at the moment.

Deploy Applications To Trigger Scale Up Operation

Next, we will deploy a test workload to the cluster. The test deployment requests resources that our two small EC2 instances cannot fully accommodate. Karpenter will come to the rescue and provision an additional node to make our cluster healthy again.

The test deployment kubernetes/karpenter/test-deployment.yaml looks like this:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: inflate
  namespace: default
spec:
  replicas: 0
  selector:
    matchLabels:
      app: inflate
  template:
    metadata:
      labels:
        app: inflate
    spec:
      terminationGracePeriodSeconds: 0
      containers:
        - name: inflate
          image: public.ecr.aws/eks-distro/kubernetes/pause:3.7
          resources:
            requests:
              cpu: "1"
              memory: "1.5Gi"

The script scripts/test_karpenter.sh scales this deployment from 0 to 5 replicas:

#!/bin/bash
set -euxo pipefail

echo "Scaling up the inflate deployment to trigger Karpenter provisioning..."

# Scale up the deployment to trigger node provisioning
kubectl scale deployment inflate --replicas=5 -n default

echo ""
echo "Deployment scaled to 5 replicas. Monitoring cluster events..."

Run the test script:

chmod +x scripts/test_karpenter.sh
./scripts/test_karpenter.sh

Watch Karpenter logs to see it making provisioning decisions:

kubectl logs -f -n karpenter -l app.kubernetes.io/name=karpenter -c controller

You'll see logs like:

INFO    controller.provisioner  found provisionable pod(s)
INFO    controller.provisioner  computed new node(s) to fit pod(s)
INFO    controller.provisioner  created node with 1 pods requesting cpu=1000m, memory=1536Mi

You will notice that Karpenter did its job and added an extra node to our cluster. The deployments were able to succeed.

Important Note: In the AWS EKS console, you'll see the new node listed as "Self-managed" under the Compute tab, while your original 2 nodes show as part of the "Managed Node Group". This is expected! Karpenter provisions nodes as individual EC2 instances directly (not through node groups), which is why they appear as self-managed. You can verify these are Karpenter nodes by checking their labels:

kubectl get nodes --show-labels | grep karpenter

You'll see labels like karpenter.sh/capacity-type=on-demand, karpenter.k8s.aws/instance-category=t, etc.

If you check the events of your cluster, you will be able to find the scale up events.

Scale Down Operation

Now we will scale down the test deployment and watch to see if Karpenter automatically removes the node.

The cleanup script scripts/cleanup_karpenter_test.sh scales the deployment back to 0:

#!/bin/bash
set -euxo pipefail

echo "Scaling down the inflate deployment..."

# Scale down the deployment
kubectl scale deployment inflate --replicas=0 -n default

echo ""
echo "Deployment scaled to 0 replicas."
echo "Watch nodes with: kubectl get nodes -w"
echo "Karpenter should remove the empty node in about 30 seconds (ttlSecondsAfterEmpty)"

Run the cleanup script:

chmod +x scripts/cleanup_karpenter_test.sh
./scripts/cleanup_karpenter_test.sh

Monitor the nodes:

kubectl get nodes -w

After about 30 seconds (the ttlSecondsAfterEmpty setting in our Provisioner), Karpenter will:

Cordon the empty node
Drain any remaining system pods
Terminate the EC2 instance

You'll see this in the Karpenter logs:

INFO    controller.deprovisioning   deprovisioning via emptiness delete
INFO    controller.deprovisioning   cordoned node
INFO    controller.deprovisioning   deleted node

Karpenter scales down the cluster back to two nodes much faster than Cluster Autoscaler (30-60 seconds vs 10+ minutes).

Understanding Karpenter's Advantages

Through this exercise, you've seen several advantages of Karpenter over Cluster Autoscaler:

Faster Provisioning

Karpenter provisions nodes in 30-60 seconds because it launches instances directly rather than scaling Auto Scaling Groups.

No Pre-defined Node Groups

You don't need to define multiple node groups for different instance types. Karpenter chooses the best fit automatically based on pending pod requirements.

Better Bin Packing

Karpenter looks at all pending pods together and provisions nodes that maximize utilization.

Faster Scale Down

Karpenter can deprovision nodes in 30-60 seconds vs 10+ minutes for Cluster Autoscaler.

Consolidation

Karpenter actively consolidates nodes to reduce costs, even moving pods to smaller instances when possible.

Spot Instance Support

Adding spot instances is as simple as adding "spot" to the karpenter.sh/capacity-type values in your Provisioner.

Comparison: Cluster Autoscaler vs Karpenter

Feature	Cluster Autoscaler	Karpenter
Provisioning Time	2-5 minutes	30-60 seconds
Node Management	Auto Scaling Groups / Managed Node Groups	Individual EC2 instances (self-managed)
Instance Selection	Predefined in node group	Dynamic based on pod requirements
Scale Down Time	10+ minutes	30-60 seconds
Spot Support	Via ASG configuration	Native, simple toggle
Bin Packing	Limited (per node group)	Advanced (across all options)
Consolidation	No	Yes (active rebalancing)
Configuration	Via ASG/node group settings	Via Kubernetes CRDs
Console Display	Shows as "Managed Node Group"	Shows as "Self-managed"

The complete code used in this article can be found on this branch of the repository

Troubleshooting Common Issues

Issue: Nodes Stuck in NotReady State

Symptom: Karpenter provisions a node, but it never becomes Ready. Kubelet never starts.

Cause: The node was likely launched in a public subnet without a public IP, preventing it from reaching the EKS API server.

Solution: Ensure only private subnets are tagged with karpenter.sh/discovery. Remove this tag from public subnets:

public_subnet_tags = {
  "kubernetes.io/cluster/${var.cluster_name}" = "shared"
  "kubernetes.io/role/elb"                    = "1"
  # DO NOT add karpenter.sh/discovery tag here
}

private_subnet_tags = {
  "kubernetes.io/cluster/${var.cluster_name}" = "shared"
  "kubernetes.io/role/internal-elb"           = "1"
  "karpenter.sh/discovery"                    = var.cluster_name  # Only here
}

After fixing the tags, run terraform apply, then delete any stuck nodes and let Karpenter recreate them.

Issue: Capacity-Block Warnings in Logs

Symptom: Karpenter logs show errors like:

ERROR   controller      Reconciler error    {"commit": "...", "reconciler": "capacity-block", ...}

Cause: This is a harmless warning in Karpenter v0.16.3. The controller tries to query newer GPU instance types (P5, P6, Trn2) that didn't exist when this version was released.

Impact: None - these warnings don't affect Karpenter's functionality for standard instance types.

Solution: Either ignore the warnings or filter them out:

kubectl logs -n karpenter -l app.kubernetes.io/name=karpenter -f | grep -v "capacity-block"

To permanently suppress, you could upgrade to a newer Karpenter version, but test thoroughly as newer versions may introduce breaking changes.

Issue: Why Are My Nodes "Self-Managed" in AWS Console?

Symptom: New nodes appear as "Self-managed" in the EKS console instead of being part of a managed node group.

Explanation: This is expected behavior for Karpenter. Unlike Cluster Autoscaler which scales existing node groups, Karpenter provisions individual EC2 instances directly using the AWS API. These appear as "self-managed" because they're not part of an EKS Managed Node Group or Auto Scaling Group.

Verification: Check that the nodes have Karpenter labels:

kubectl get nodes --show-labels | grep karpenter

You should see labels like karpenter.sh/capacity-type=on-demand, karpenter.k8s.aws/instance-category=t, etc.

Reference: See Karpenter documentation for architectural details.

Conclusion

Your EKS cluster now has a functioning Karpenter setup that is adept at provisioning node resources in response to workload changes. Karpenter provides faster scaling, smarter instance selection, and better cost optimization compared to traditional Cluster Autoscaler.

Don't forget to clean up any resources you are not using, it can get quite expensive.

To clean up Karpenter resources:

chmod +x scripts/delete_all.sh
./scripts/delete_all.sh

If you want to destroy all infrastructure:

terraform destroy

Important: Karpenter-provisioned nodes should be cleaned up automatically when you delete the Provisioner, but you may want to verify in the AWS console that no orphaned instances remain.

For more advanced configurations and best practices, refer to:

Navigating AWS EKS with Terraform: Implementing Cluster Auoscaler in your EKS Cluster

Oluwafemi Lawal — Tue, 14 Jan 2025 01:34:17 +0000

In this article, we will integrate Cluster Autoscaler, which will enable our cluster to adjust the number of nodes based on demand. We will need to make some adjustments to our EKS module.

Prerequisites

Ensure you have:

An active AWS account
Terraform installed and configured
Helm installed
kubectl

Setup Overview

Prepare Your EKS Cluster

Ensure your EKS cluster is ready. If needed, refer back to our guide on setting up an EKS cluster using Terraform

Modifications

To enhance flexibility and customization, we've introduced new variables that streamline the addition of role and user mappings to the Kubernetes aws-auth ConfigMap. Additionally, to support Cluster Autoscaler's auto-discovery feature, it is essential to include specific tags in the launch configuration. We have also incorporated a variable to manage the Kubernetes version for the EKS cluster, allowing for more controlled upgrades and maintenance.

variable "additional_role_mappings" {
  description = "Additional role mappings for aws-auth ConfigMap"
  type = list(object({
    rolearn  = string
    username = string
    groups   = list(string)
  }))
  default = []
}

variable "additional_user_mappings" {
  description = "Additional user mappings for aws-auth ConfigMap"
  type = list(object({
    userarn  = string
    username = string
    groups   = list(string)
  }))
  default = []
}

variable "eks_cluster_version" {
  description = "The desired Kubernetes version for the EKS cluster"
  type        = string
  default     = "1.29"
}

variable "additional_launch_template_tags" {
  description = "Additional tags to apply to the launch template"
  type = map(string)
  default = {}
}

Create IAM Role for Cluster Autoscaler

The autoscaler needs specific permissions to interact with EC2 instances and AWS services.
We will use recommended permissions from the documentation

############################################################################################################
### AUTOSCALING
############################################################################################################
data "aws_iam_policy_document" "cluster_autoscaler_assume_role_policy" {
  statement {
    actions = ["sts:AssumeRoleWithWebIdentity"]
    effect  = "Allow"

    condition {
      test     = "StringEquals"
      variable = "${replace(module.eks.oidc_provider_url, "https://", "")}:sub"
      values   = ["system:serviceaccount:kube-system:cluster-autoscaler"]
    }

    principals {
      identifiers = ["${module.eks.oidc_provider_arn}"]
      type        = "Federated"
    }
  }
}

# IAM Role for Cluster Autoscaler
resource "aws_iam_role" "cluster_autoscaler_role" {
  name               = "${var.cluster_name}-cluster-autoscaler"
  assume_role_policy = data.aws_iam_policy_document.cluster_autoscaler_assume_role_policy.json
}

# Custom policy for Cluster Autoscaler
data "aws_iam_policy_document" "cluster_autoscaler_policy" {
  statement {
    effect = "Allow"
    actions = [
      "autoscaling:DescribeAutoScalingGroups",
      "autoscaling:DescribeAutoScalingInstances",
      "autoscaling:DescribeLaunchConfigurations",
      "autoscaling:DescribeScalingActivities",
      "ec2:DescribeImages",
      "ec2:DescribeInstanceTypes",
      "ec2:DescribeLaunchTemplateVersions",
      "ec2:GetInstanceTypesFromInstanceRequirements",
      "eks:DescribeNodegroup"
    ]
    resources = ["*"]
  }

  statement {
    effect = "Allow"
    actions = [
      "autoscaling:SetDesiredCapacity",
      "autoscaling:TerminateInstanceInAutoScalingGroup"
    ]
    resources = ["*"]
  }
}

resource "aws_iam_role_policy" "cluster_autoscaler_policy" {
  name   = "${var.cluster_name}-cluster-autoscaler-policy"
  role   = aws_iam_role.cluster_autoscaler_role.id
  policy = data.aws_iam_policy_document.cluster_autoscaler_policy.json
}

We can then pass the role mapping to the config map in the module, as well as the tags needed for auto-discovery

module "eks" {
  source = "./modules/aws/eks/v1"

  region          = var.region
  cluster_name    = var.cluster_name
  private_subnets = module.vpc.private_subnets
  public_subnets  = module.vpc.public_subnets
  vpc_id          = module.vpc.vpc_id

  managed_node_groups = {
    demo_group = {
      name           = "demo-node-group"
      desired_size   = 2
      min_size       = 1
      max_size       = 3
      instance_types = ["t3a.small"]
    }
  }

  additional_role_mappings = [
    {
      rolearn  = aws_iam_role.cluster_autoscaler_role.arn
      username = "system:serviceaccount:kube-system:cluster-autoscaler"
      groups   = ["system:masters"]
    }
  ]

  additional_launch_template_tags = {
      "k8s.io/cluster-autoscaler/${var.cluster_name}" = "owned"
      "k8s.io/cluster-autoscaler/enabled"             = "true"    
  }
}

We can then apply our changes with terraform

terraform apply

We can confirm our cluster is up and running in the AWS console.

Install Cluster Autoscaler using Helm

We have created the role mapping that the cluster-autoscaler service account will use, but we still need to create the actual service account, the manifest to create that will look like this

apiVersion: v1
kind: ServiceAccount
metadata:
  name: cluster-autoscaler
  namespace: kube-system
  annotations:
    eks.amazonaws.com/role-arn: "arn:aws:iam::<ACCOUNT_ID>:role/<IAM_ROLE_NAME>"

For simplicity (or just because I'm lazy), I have created four scripts to help with the rest of the article. They make use of terraform outputs to avoid me having to put placeholders or keep track of which command I have to run first.

An explanation of what is happening is that our kubectl context is updated to that of the cluster, the service account for cluster autoscaler is created, and cluster autoscaler is deployed using helm.

#!/bin/bash
set -euxo pipefail

# Retrieve Terraform outputs
REGION=$(terraform output -raw aws_region)
CLUSTER_NAME=$(terraform output -raw cluster_name)
CLUSTER_AUTOSCALER_ROLE_ARN=$(terraform output -raw cluster_autoscaler_role_arn)

# Update kubeconfig
aws eks update-kubeconfig --region $REGION --name $CLUSTER_NAME

# Create the service account for cluster autoscaler
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: ServiceAccount
metadata:
  name: cluster-autoscaler
  namespace: kube-system
  annotations:
    eks.amazonaws.com/role-arn: $CLUSTER_AUTOSCALER_ROLE_ARN
EOF

# Deploy the cluster autoscaler Helm chart
helm repo add autoscaler https://kubernetes.github.io/autoscaler || true
helm repo update

helm upgrade --install cluster-autoscaler autoscaler/cluster-autoscaler \
  --namespace kube-system \
  --set autoDiscovery.clusterName=$CLUSTER_NAME \
  --set rbac.serviceAccount.create=false \
  --set rbac.serviceAccount.name=cluster-autoscaler \
  --set awsRegion=$REGION \
  --set extraArgs.balance-similar-node-groups=true \
  --set extraArgs.skip-nodes-with-system-pods=false \
  --set extraArgs.expander=least-waste || true

I also deployed kubernetes dashboard just so we can have an application with a user interface we can interact with running on the cluster, and to confirm what we are seeing on the AWS dashboard regarding our cluster. This where the kube proxy add on comes in handy.

# Deploy the Kubernetes Dashboard
kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.5.1/aio/deploy/recommended.yaml || true

# Create the admin-user service account and cluster role binding
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: ServiceAccount
metadata:
  name: admin-user
  namespace: kubernetes-dashboard
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: admin-user
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: admin-user
  namespace: kubernetes-dashboard
EOF

# Get the token for the admin-user
kubectl -n kubernetes-dashboard create token admin-user || true

# Print the Kubernetes Dashboard URL
echo "Kubernetes Dashboard URL: http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/"

# Start a kubectl proxy to access the Kubernetes Dashboard
kubectl proxy --port=8001

We can see the token and a link to view the dashboard via kube proxy

You can follow the link in the command and enter the token to access the dashboard and keep track of events. You will notice that we only have two nodes at the moment.

This matches what we see in the AWS console.

Deploy Applications To Trigger Scale Up Operation

Next, we will deploy Prometheus and Grafana to the cluster, but because of how little CPU and Memory the EC2 type we selected has, the deployments will fail to reach a healthy state. Cluster autoscale will come to the rescue and deploy an additional node to make our cluster healthy again.

I am also using a script to accomplish the deployments with helm

#!/bin/bash
set -euxo pipefail


# Deploy the metrics server Helm chart
helm repo add metrics-server https://kubernetes-sigs.github.io/metrics-server || true
helm repo update

helm upgrade --install metrics-server metrics-server/metrics-server \
  --namespace kube-system \
  --set args[0]="--kubelet-insecure-tls" \
  --set args[1]="--kubelet-preferred-address-types=InternalIP" || true

# Deploy Prometheus and Grafana using the kube-prometheus-stack Helm chart
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts || true
helm repo add grafana https://grafana.github.io/helm-charts || true
helm repo update

helm upgrade --install prometheus prometheus-community/kube-prometheus-stack \
  --namespace monitoring \
  --create-namespace \
  --set grafana.enabled=true \
  --set grafana.adminPassword='admin' \
  --set prometheus.prometheusSpec.serviceMonitorSelectorNilUsesHelmValues=false \
  --set defaultRules.create=true \
  --set alertmanager.enabled=false || true

# Ensure Prometheus is scraping metrics from the cluster autoscaler and metrics server
cat <<EOF | kubectl apply -f -
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: cluster-autoscaler
  namespace: kube-system
  labels:
    release: prometheus
spec:
  selector:
    matchLabels:
      app: cluster-autoscaler
  endpoints:
  - port: http
    interval: 30s
EOF

cat <<EOF | kubectl apply -f -
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: metrics-server
  namespace: kube-system
  labels:
    release: prometheus
spec:
  selector:
    matchLabels:
      k8s-app: metrics-server
  endpoints:
  - port: https
    interval: 30s
    tlsConfig:
      insecureSkipVerify: true
EOF

You will notice that cluster autoscaler did it's job and added an extra node to our cluster, the deployments were able to succeed after their initial failure

If you check the events of your cluster, you will be able to find the scale up events.

Scale Down Operation

Now we will delete the Prometheus and Grafana applications and watch to see if cluster autoscaler scales down the cluster.
Once again I am using a script for the deletes

#!/bin/bash
set -euxo pipefail

# Delete the metrics server Helm release
helm uninstall metrics-server --namespace kube-system || true

# Delete Prometheus and Grafana Helm release
helm uninstall prometheus --namespace monitoring || true

# Delete the ServiceMonitor for cluster autoscaler
kubectl delete servicemonitor cluster-autoscaler --namespace kube-system --ignore-not-found=true

# Delete the ServiceMonitor for metrics server
kubectl delete servicemonitor metrics-server --namespace kube-system --ignore-not-found=true

# Delete the monitoring namespace
kubectl delete namespace monitoring --ignore-not-found=true

Cluster Autoscaler might scale down our cluster back to two Nodes immediately, or it might modify our desired number of nodes in our node group to require three nodes because of how weak the EC2 instance types we selected are, which is what happened in this case.

After some minutes of no new activity, cluster autoscaler scales down the cluster back to two Nodes.

It also changes the desired size of our node group back to two nodes.

The complete code used in this article can be found on this branch of the repository

Conclusion

Your EKS cluster now has a functioning Cluster Autoscaler that is adept at scaling node resources in response to workload changes.
Don't forget to clean up any resources you are not using, it can get quite expensive.

For advanced configurations and best practices, refer to the Cluster Autoscaler documentation on AWS.

Navigating AWS EKS with Terraform: Understanding EKS Cluster Configuration

Oluwafemi Lawal — Tue, 12 Mar 2024 22:03:15 +0000

In our journey through the AWS EKS ecosystem, we have laid a solid foundation with VPC networking and security groups. Now we will discuss the Terraform resources required to implement a working cluster.

The entire project will look like this by the end:



.
├── eks.tf
├── modules
│   └── aws
│       ├── eks
│       │    ├── main.tf
│       │    ├── outputs.tf
│       │    ├── versions.tf
│       │    └── variables.tf
│       └── vpc
│            ├── main.tf
│            ├── outputs.tf
│            ├── versions.tf
│            └── variables.tf
├── versions.tf
├── variables.tf
└── vpc.tf

Cluster Resource

The Terraform configuration creates an Amazon EKS cluster with enhanced security and logging features:

Cluster Configuration:
- name: Sets the cluster's name to a variable, allowing for customizable deployments.
- role_arn: Specifies the IAM role that EKS will assume to create AWS resources for the cluster.
- enabled_cluster_log_types: Enables logging for audit, API, and authenticator logs, enhancing security and compliance.
VPC Configuration:
- Integrates the cluster with specified private and public subnets, allowing workloads to be placed accordingly for both security and accessibility.
- Associates the cluster with a specific security group to control inbound and outbound traffic.
Encryption Configuration:
- Utilizes a KMS key for encrypting Kubernetes secrets, safeguarding sensitive information.
- Specifies that the encryption applies to Kubernetes secrets, ensuring that secret data stored in the cluster is encrypted at rest.
Dependencies:
- Ensures the cluster is created after the necessary IAM role and policy attachments are in place, maintaining the deployment order and security posture.



# Fetch current AWS account details
data "aws_caller_identity" "current" {}

############################################################################################################
### EKS CLUSTER
############################################################################################################
resource "aws_eks_cluster" "main" {
  name                      = var.cluster_name
  role_arn                  = aws_iam_role.eks_cluster_role.arn
  enabled_cluster_log_types = var.enabled_cluster_log_types

  vpc_config {
    subnet_ids             = concat(var.private_subnets, var.public_subnets)
    security_group_ids     = [aws_security_group.eks_cluster_sg.id]
    endpoint_public_access = true

  }

  encryption_config {
    provider {
      key_arn = aws_kms_key.eks_encryption.arn
    }
    resources = ["secrets"]
  }

  depends_on = [
    aws_iam_role_policy_attachment.eks_cluster_policy,
  ]
}

The Kubernetes provider configuration is necessary for Terraform to interact with your Amazon EKS cluster's Kubernetes API



data "aws_eks_cluster_auth" "main" {
  name = var.cluster_name
}

provider "kubernetes" {
  host                   = aws_eks_cluster.main.endpoint
  cluster_ca_certificate = base64decode(aws_eks_cluster.main.certificate_authority[0].data)
  token                  = data.aws_eks_cluster_auth.main.token
}

OIDC for EKS: Secure Access Management

Integrating OIDC with your EKS cluster enhances security by facilitating identity federation between AWS and Kubernetes. This configuration allows for secure role-based access control, which is crucial for managing access to your cluster.



############################################################################################################
### OIDC CONFIGURATION
############################################################################################################

data "tls_certificate" "eks" {
  url = aws_eks_cluster.main.identity[0].oidc[0].issuer
}

resource "aws_iam_openid_connect_provider" "eks" {
  client_id_list  = ["sts.amazonaws.com"]
  thumbprint_list = [data.tls_certificate.eks.certificates[0].sha1_fingerprint]
  url             = aws_eks_cluster.main.identity[0].oidc[0].issuer
}

Secrets Encryption

In the EKS cluster resource above, this KMS key is referenced. It is needed to encrypt Kubernetes secrets in the cluster.



############################################################################################################
### KMS KEY
############################################################################################################
resource "aws_kms_key" "eks_encryption" {
  description         = "KMS key for EKS cluster encryption"
  policy              = data.aws_iam_policy_document.kms_key_policy.json
  enable_key_rotation = true
}

# alias
resource "aws_kms_alias" "eks_encryption" {
  name          = "alias/eks/${var.cluster_name}"
  target_key_id = aws_kms_key.eks_encryption.id
}

data "aws_iam_policy_document" "kms_key_policy" {
  statement {
    sid = "Key Administrators"
    actions = [
      "kms:Create*",
      "kms:Describe*",
      "kms:Enable*",
      "kms:List*",
      "kms:Put*",
      "kms:Update*",
      "kms:Revoke*",
      "kms:Disable*",
      "kms:Get*",
      "kms:Delete*",
      "kms:ScheduleKeyDeletion",
      "kms:CancelKeyDeletion",
      "kms:Decrypt",
      "kms:DescribeKey",
      "kms:Encrypt",
      "kms:ReEncrypt*",
      "kms:GenerateDataKey*",
      "kms:TagResource"
    ]
    principals {
      type = "AWS"
      identifiers = [
        "arn:aws:iam::${data.aws_caller_identity.current.account_id}:root",
        data.aws_caller_identity.current.arn
      ]
    }
    resources = ["*"]
  }

  statement {
    actions = [
      "kms:Encrypt",
      "kms:Decrypt",
      "kms:ReEncrypt*",
      "kms:GenerateDataKey*",
      "kms:DescribeKey"
    ]
    principals {
      type        = "Service"
      identifiers = ["eks.amazonaws.com"]
    }
    resources = ["*"]
  }
}


resource "aws_iam_policy" "cluster_encryption" {
  name        = "${var.cluster_name}-encryption-policy"
  description = "IAM policy for EKS cluster encryption"
  policy      = data.aws_iam_policy_document.cluster_encryption.json
}

data "aws_iam_policy_document" "cluster_encryption" {
  statement {
    actions = [
      "kms:Encrypt",
      "kms:Decrypt",
      "kms:ListGrants",
      "kms:DescribeKey"
    ]
    resources = [aws_kms_key.eks_encryption.arn]
  }
}

# Granting the EKS Cluster role the ability to use the KMS key
resource "aws_iam_role_policy_attachment" "cluster_encryption" {
  policy_arn = aws_iam_policy.cluster_encryption.arn
  role       = aws_iam_role.eks_cluster_role.name
}

Node Groups: Scaling and Optimization with Amazon EKS

Node Groups in Amazon EKS allow the management of EC2 instances as Kubernetes nodes. These are critical for running your applications and can be customized to suit specific workload needs through AWS Managed Node Groups, Self-Managed Node Groups, and AWS Fargate.

AWS Managed Node Groups

Managed Node Groups simplify node management by automating provisioning, lifecycle management, and scaling. They offer:

Automated Updates: Automatic application of security patches and OS updates.
Customization: Via Launch Templates for AMIs, instance types, etc.
Scalability: Directly adjustable scaling configurations, Kubernetes labels, and AWS tags.

Self-Managed Node Groups

Offering full control over configuration, Self-Managed Node Groups require manual scaling and updates but allow for:

Custom AMIs: For specific compliance or policy requirements.
Manual Management: Users handle software and OS updates, leveraging AWS CloudFormation for operations.

Serverless Option: AWS Fargate with Amazon EKS

AWS Fargate abstracts server management, allowing specifications for CPU and RAM without considering instance types. This offers a simplified, serverless option for running containers in EKS.

Comparison Table

Criteria	Managed Node Groups	Self-Managed Nodes	AWS Fargate
Deployment to AWS Outposts	No	Yes	No
Windows Container Support	Yes	Yes (but with at least one compulsory Linux node)	No
Linux Container Support	Yes	Yes	Yes
GPU Workloads	Yes (Amazon Linux)	Yes (Amazon Linux)	No
Custom AMI Deployment	Yes (via Launch Template)	Yes	No
Operating System Maintenance	Automated by AWS	Manual	N/A
SSH Access	Yes	Yes	No (No node OS)
Kubernetes DaemonSets	Yes	Yes	No
Pricing	EC2 instance cost	EC2 instance cost	Fargate pricing per Pod
Update Node AMI	Automated notification & one-click update in EKS console for EKS AMI; manual for custom AMI	Can not be done from EKS console. Manual process using external tools	N/A
Update Node Kubernetes Version	Automated notification & one-click update in EKS console for EKS AMI; manual for custom AMI	Can not be done from EKS console. Manual process using external tools	N/A
Use Amazon EBS with Pods	Yes	Yes	No
Use Amazon EFS with Pods	Yes	Yes	Yes
Use Amazon FSx for Lustre with Pods	Yes	Yes	No

AWS Fargate for EKS provides a higher level of abstraction, managing more server aspects for you. It eliminates the need to select instance types, focusing solely on the required CPU and RAM for your workloads.

Our Managed Node Groups Configuration

For our EKS module, we will use the managed node group Terraform resource.
Our terraform resources below creates a series of node groups for an EKS cluster, where each group's configuration is based on user-defined variables. These node groups are essential for running your Kubernetes workloads on AWS, providing the compute capacity as EC2 instances.

Specifies the EKS cluster to which these node groups belong.
Defines the size (min, max, desired) of each node group, allowing for scalability.
Associates a launch template to dictate the configuration of EC2 instances within the node group.
Allows selection of instance types, AMI types, and capacity types (e.g., On-Demand or Spot Instances) for flexibility and cost optimization.
Includes a mechanism to force updates to node groups when the launch template changes.



############################################################################################################
### MANAGED NODE GROUPS
############################################################################################################
resource "aws_eks_node_group" "main" {
  for_each = var.managed_node_groups

  cluster_name    = aws_eks_cluster.main.name
  node_group_name = each.value.name
  node_role_arn   = aws_iam_role.node_role.arn
  subnet_ids      = var.private_subnets

  scaling_config {
    desired_size = each.value.desired_size
    max_size     = each.value.max_size
    min_size     = each.value.min_size
  }

  launch_template {
    id      = aws_launch_template.eks_node_group.id
    version = "$Default"
  }

  instance_types       = each.value.instance_types
  ami_type             = var.default_ami_type
  capacity_type        = var.default_capacity_type
  force_update_version = true
}

Launch Template

This defines a blueprint for the EC2 instances that will be launched as part of the node groups. This template encapsulates various settings and configurations for the instances.

Configures instance networking, including security groups for controlling access to/from the instances.
Sets instance metadata options to enhance security, like enforcing IMDSv2 for metadata access.
Details the block device mappings for instance storage, including the root EBS volume size, type, and deletion policy.
Automatically tags instances for easier management, cost tracking, and integration with Kubernetes.
Ensures that the launch template is created with a lifecycle policy that avoids conflicts and dangling resources.

This combined setup allows for automated management of the underlying infrastructure for Kubernetes workloads, leveraging EKS-managed node groups and EC2 launch templates for fine-grained control over instance properties and scaling behaviours.



############################################################################################################
### LAUNCH TEMPLATE
############################################################################################################  
resource "aws_launch_template" "eks_node_group" {
  name_prefix = "${var.cluster_name}-eks-node-group-lt"
  description = "Launch template for ${var.cluster_name} EKS node group"

  vpc_security_group_ids = [aws_security_group.eks_nodes_sg.id]

  metadata_options {
    http_endpoint               = "enabled"
    http_tokens                 = "required"
    http_put_response_hop_limit = 2
    instance_metadata_tags      = "enabled"
  }

  block_device_mappings {
    device_name = "/dev/xvda" # Adjusted to the common root device name for Linux AMIs

    ebs {
      volume_size           = 20    # Disk size specified here
      volume_type           = "gp3" # Example volume type, adjust as necessary
      delete_on_termination = true
    }
  }

  tags = {
    "Name"                                      = "${var.cluster_name}-eks-node-group"
    "kubernetes.io/cluster/${var.cluster_name}" = "owned"
  }

  lifecycle {
    create_before_destroy = true
  }
}

EKS Add-ons: Enhancing Cluster Capabilities

With the aws_eks_addon resource we can pass a list of addons to install. The defaults for this project will be vpc-cni, kube-proxy, coredns and aws-ebs-csi-driver

EKS Add-ons Explanation

vpc-cni: The Amazon VPC CNI (Container Network Interface) plugin allows Kubernetes pods to have the same IP address inside the pod as they do on the VPC network. This plugin is responsible for providing high-performance networking for Amazon EKS clusters, enabling native VPC networking for Kubernetes pods.
kube-proxy: Manages network rules on each node. This allows network communication to your pods from network sessions inside or outside of your cluster. It makes sure that each node has the latest network rules for communicating with other nodes in the cluster.
coredns: A flexible, extensible DNS server that can serve as the Kubernetes cluster DNS. It translates human-readable hostnames like www.example.com into IP addresses. CoreDNS is used for service discovery within the cluster, allowing pods to find each other and services to scale up and down.
aws-ebs-csi-driver: The Amazon EBS CSI (Container Storage Interface) Driver provides a way to configure and manage Amazon EBS volumes. It allows you to use Amazon EBS volume as persistent storage for stateful applications in EKS. This driver supports dynamic provisioning, snapshots, and resizing of volumes.



############################################################################################################
# PLUGINS
############################################################################################################
data "aws_eks_addon_version" "main" {
  for_each = toset(var.cluster_addons)

  addon_name         = each.key
  kubernetes_version = aws_eks_cluster.main.version
}

resource "aws_eks_addon" "main" {
  for_each = toset(var.cluster_addons)

  cluster_name                = aws_eks_cluster.main.name
  addon_name                  = each.key
  addon_version               = data.aws_eks_addon_version.main[each.key].version
  resolve_conflicts_on_create = "OVERWRITE"
  resolve_conflicts_on_update = "OVERWRITE"
  depends_on = [
    aws_eks_node_group.main
  ]
}

IAM Roles and Policies: Securing EKS

IAM Roles for EKS

Securing your EKS clusters involves creating and associating specific IAM roles and policies, ensuring least privilege access to AWS resources. This step is fundamental in protecting your cluster's interactions with other AWS services.

EKS Cluster Role and Policies:
- A role (eks_cluster_role) is created for the EKS cluster to interact with other AWS services.
- The role trusts the eks.amazonaws.com service to assume the role.
- Attached policies:
  - CloudWatchFullAccess: Grants full access to CloudWatch, allowing the cluster to log and monitor.
  - AmazonEKSClusterPolicy: Provides permissions that Amazon EKS requires to manage clusters.
  - AmazonEKSVPCResourceController: Allows the cluster to manage VPC resources for the cluster.
Node Group Role and Policies:
- An instance profile (eks_node) is created for EKS node groups, associating them with a role (node_role) that nodes assume for AWS service interaction.
- The role trusts the ec2.amazonaws.com service to assume the role.
- Attached policies:
  - AmazonEKSWorkerNodePolicy: Grants nodes in the node group the permissions required to operate within an EKS cluster.
  - AmazonEKS_CNI_Policy: Allows nodes to manage network resources, which is necessary for the Amazon VPC CNI plugin to operate.
  - AmazonEBSCSIDriverPolicy: Allows nodes to manage EBS volumes, enabling dynamic volume provisioning.
  - AmazonEC2ContainerRegistryReadOnly: Provides read-only access to AWS Container Registry, allowing nodes to pull container images.
VPC CNI Plugin Role:
- A specific role (vpc_cni_role) is created for the VPC CNI plugin to operate within the EKS cluster.
- The role trusts federated access via the cluster's OIDC provider, specifically for the aws-node Kubernetes service account within the kube-system namespace.
- Attached policy:
  - AmazonEKS_CNI_Policy: Grants the necessary permissions for the VPC CNI plugin to manage AWS network resources.



############################################################################################################
### IAM ROLES
############################################################################################################
# EKS Cluster role
resource "aws_iam_role" "eks_cluster_role" {
  name               = "${var.cluster_name}-eks-cluster-role"
  assume_role_policy = data.aws_iam_policy_document.eks_assume_role_policy.json
}

data "aws_iam_policy_document" "eks_assume_role_policy" {
  statement {
    actions = ["sts:AssumeRole"]

    principals {
      type        = "Service"
      identifiers = ["eks.amazonaws.com"]
    }
  }
}

# EKS Cluster Policies
resource "aws_iam_role_policy_attachment" "eks_cloudwatch_policy" {
  policy_arn = "arn:aws:iam::aws:policy/CloudWatchFullAccess"
  role       = aws_iam_role.eks_cluster_role.name
}

resource "aws_iam_role_policy_attachment" "eks_cluster_policy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSClusterPolicy"
  role       = aws_iam_role.eks_cluster_role.name
}

resource "aws_iam_role_policy_attachment" "eks_vpc_resource_controller_policy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSVPCResourceController"
  role       = aws_iam_role.eks_cluster_role.name
}

# Managed Node Group role
resource "aws_iam_instance_profile" "eks_node" {
  name = "${var.cluster_name}-node-role"
  role = aws_iam_role.node_role.name
}

resource "aws_iam_role" "node_role" {
  name               = "${var.cluster_name}-node-role"
  assume_role_policy = data.aws_iam_policy_document.assume_role_policy.json
}

data "aws_iam_policy_document" "assume_role_policy" {
  statement {
    actions = [
      "sts:AssumeRole"
    ]
    principals {
      type        = "Service"
      identifiers = ["ec2.amazonaws.com"]
    }
  }
}

# Node Group Policies
resource "aws_iam_role_policy_attachment" "eks_worker_node_policy" {
  role       = aws_iam_role.node_role.name
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy"
}

resource "aws_iam_role_policy_attachment" "eks_cni_policy" {
  role       = aws_iam_role.node_role.name
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy"
}

resource "aws_iam_role_policy_attachment" "eks_ebs_csi_policy" {
  role       = aws_iam_role.node_role.name
  policy_arn = "arn:aws:iam::aws:policy/service-role/AmazonEBSCSIDriverPolicy"
}

resource "aws_iam_role_policy_attachment" "eks_registry_policy" {
  role       = aws_iam_role.node_role.name
  policy_arn = "arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly"
}

# VPC CNI Plugin Role
data "aws_iam_policy_document" "vpc_cni_assume_role_policy" {
  statement {
    actions = ["sts:AssumeRoleWithWebIdentity"]
    effect  = "Allow"

    condition {
      test     = "StringEquals"
      variable = "${replace(aws_iam_openid_connect_provider.eks.url, "https://", "")}:sub"
      values   = ["system:serviceaccount:kube-system:aws-node"]
    }

    principals {
      identifiers = [aws_iam_openid_connect_provider.eks.arn]
      type        = "Federated"
    }
  }
}

resource "aws_iam_role" "vpc_cni_role" {
  assume_role_policy = data.aws_iam_policy_document.vpc_cni_assume_role_policy.json
  name               = "${var.cluster_name}-vpc-cni-role"
}

resource "aws_iam_role_policy_attachment" "vpc_cni_policy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy"
  role       = aws_iam_role.vpc_cni_role.name
}

Cluster Access and Role Based Access Control

When an Amazon EKS cluster is created, the AWS identity (user or role) that creates the cluster is automatically granted system:masters permissions, providing full administrative access to the cluster. This ensures immediate operational control without additional configuration steps.

However, in collaborative environments or larger organizations, more than one individual or service may need administrative access to manage the cluster effectively. To extend administrative privileges securely, additional IAM roles can be created and configured with permissions similar to the cluster creator.

This Terraform configuration defines an IAM role (aws_iam_role.eks_admins_role) designed for EKS Administrators, granting them similar administrative capabilities over the EKS cluster. It uses an IAM policy document (data.aws_iam_policy_document.eks_admins_assume_role_policy_doc) to specify that the role can be assumed by specified AWS principals. This approach adheres to the principle of least privilege, allowing for controlled access to the EKS cluster based on organizational requirements.

It's best to also add the node role with the username system:node:{{EC2PrivateDNSName}} in groups "system:bootstrappers", "system:nodes", though it is actually added to the ConfigMap automatically, it will get removed by your next terraform apply command if not defined explicitly.

Lastly, the kubernetes_config_map.aws_auth resource updates the aws-auth ConfigMap in the EKS cluster. This action registers the role with the cluster, mapping it to system:masters, thus granting it administrative access similar to the cluster creator. This setup ensures that designated administrators can manage the cluster without using the credentials of the original creator, enhancing security and operational flexibility.



############################################################################################################
### CLUSTER ROLE BASE ACCESS CONTROL
############################################################################################################
# Define IAM Role for EKS Administrators
resource "aws_iam_role" "eks_admins_role" {
  name = "${var.cluster_name}-eks-admins-role"

  assume_role_policy = data.aws_iam_policy_document.eks_admins_assume_role_policy_doc.json
}

# IAM Policy Document for assuming the eks-admins role
data "aws_iam_policy_document" "eks_admins_assume_role_policy_doc" {
  statement {
    actions = ["sts:AssumeRole"]
    principals {
      type        = "AWS"
      identifiers = ["arn:aws:iam::${data.aws_caller_identity.current.account_id}:root"]
    }
    effect = "Allow"
  }
}

# Define IAM Policy for administrative actions on EKS
data "aws_iam_policy_document" "eks_admin_policy_doc" {
  statement {
    actions   = ["eks:*", "ec2:Describe*", "iam:ListRoles", "iam:ListRolePolicies", "iam:GetRole"]
    resources = ["*"]
  }
}

# Create IAM Policy based on the above document
resource "aws_iam_policy" "eks_admin_policy" {
  name   = "${var.cluster_name}-eks-admin-policy"
  policy = data.aws_iam_policy_document.eks_admin_policy_doc.json
}

# Attach IAM Policy to the EKS Administrators Role
resource "aws_iam_role_policy_attachment" "eks_admin_role_policy_attach" {
  role       = aws_iam_role.eks_admins_role.name
  policy_arn = aws_iam_policy.eks_admin_policy.arn
}

# Update the aws-auth ConfigMap to include the IAM group
resource "kubernetes_config_map" "aws_auth" {
  metadata {
    name      = "aws-auth"
    namespace = "kube-system"
  }

  data = {
    mapRoles = yamlencode([
      {
        rolearn  = aws_iam_role.eks_admins_role.arn
        username = aws_iam_role.eks_admins_role.name
        groups   = ["system:masters"]
      },
      {
        rolearn  = aws_iam_role.node_role.arn
        username = "system:node:{{EC2PrivateDNSName}}"
        groups   = ["system:bootstrappers", "system:nodes"]
      }
    ])
    mapUsers = yamlencode([
      {
        userarn  = data.aws_caller_identity.current.arn
        username = split("/", data.aws_caller_identity.current.arn)[1]
        groups   = ["system:masters"]
      }
    ])
  }

}

Conclusion: Mastering EKS Management

By delving into advanced configuration options, securing your clusters with IAM, and employing effective logging, monitoring, and scaling strategies, you can create Kubernetes clusters that are not only robust and scalable but also optimized for cost, performance, and security. The journey to mastering AWS EKS with Terraform is ongoing, and each step forward enhances your ability to manage complex cloud-native ecosystems effectively.

The complete project can be found here:
Sample Implementation GitHub Repo

Navigating AWS EKS with Terraform: Understanding VPC Essentials for EKS Cluster Management

Oluwafemi Lawal — Mon, 04 Mar 2024 22:11:09 +0000

AWS EKS (Elastic Kubernetes Service) offers a lighthouse for those navigating the complex waters of cloud computing, providing a managed Kubernetes service that simplifies running containerized applications. While EKS alleviates much of the heavy lifting associated with setting up a Kubernetes cluster, a deeper understanding of its components and requirements can significantly enhance your cloud infrastructure's efficiency and security.

Prerequisites

Embarking on this journey requires:

A grasp of AWS services and Kubernetes fundamentals
The AWS CLI, configured with your credentials
Terraform, to define infrastructure as code

Project directory structure:

.
├── modules
│   └── aws
│       └── vpc
│            ├── main.tf
│            ├── outputs.tf
│            ├── variables.tf
│            └── versions.tf
├── variables.tf
├── versions.tf
└── vpc.tf

versions.tf file content:

terraform {
  required_version = "1.5.1"

  required_providers {
    aws = {
      version = "5.20.0"
      source  = "hashicorp/aws"
    }
  }

  backend "s3" {
    bucket         = ""
    key            = ""
    region         = ""
    dynamodb_table = ""
    encrypt        = true
  }
}

provider "aws" {
  region = var.region
  default_tags {
    tags = var.global_tags
  }
}

You can remove the S3 backend configuration if you want to store state locally, otherwise you should ensure you have created an S3 bucket and DynamoDB table to use as values for the backend.

variables.tf file:

variable "region" {
  type        = string
  default     = "eu-west-1"
  description = "Target AWS region"
}

variable "cluster_name" {
  type        = string
  default     = "demo-cluster"
  description = "Name of the EKS cluster"
}

variable "aws_account_number" {
  type        = number
  description = "AWS account number used for deployment."
}

variable "global_tags" {
  type = map(string)
  default = {
    "ManagedBy"   = "Terraform"
    "Environment" = "dev"
  }
}

Mastering VPC Networking for AWS EKS Clusters

When deploying a Kubernetes cluster in AWS using the Elastic Kubernetes Service (EKS), understanding the underlying Virtual Private Cloud (VPC) networking is crucial. It not only ensures secure communication within your cluster but also affects your cluster's interaction with the outside world, including how services are exposed and how resources are accessed.

The Importance of VPC Networking in EKS

VPC networking forms the backbone of your EKS cluster, dictating everything from pod communication to external access via load balancers. Here’s why getting your VPC setup right is critical:

Security: Properly configured VPCs contain and protect your cluster, allowing only authorized access.
Connectivity: Your VPC setup determines how your cluster communicates with other AWS services, the internet, and on-premises networks.
Service Discovery & Load Balancing: Integrating with AWS services like Elastic Load Balancers (ELB) requires specific VPC configurations for seamless operation.

Architecting Your VPC: Public and Private Subnets

A well-architected VPC for EKS typically involves both public and private subnets:

Private Subnets are used for your worker nodes. This ensures that your workloads run securely, isolated from direct access to and from the internet. Private subnets connect to the internet through a NAT Gateway, allowing outbound traffic (e.g., for pulling container images) without exposing the nodes to inbound traffic.
Public Subnets are utilized for resources that need to be accessible from the internet, like ELBs that route external traffic to your services.

Understanding NAT Gateways and Route Tables in Your VPC

NAT Gateway

A NAT Gateway in your VPC enables instances in a private subnet to initiate outbound traffic to the internet (for updates, downloading software, etc.) without allowing inbound traffic from the internet. This is crucial for the security and integrity of your Kubernetes nodes, ensuring they have access to necessary resources while maintaining a strong security posture.

Route Tables

Route tables in AWS VPC define rules, known as routes, which determine where network traffic from your subnets or gateways is directed. In the context of EKS:

Public Route Table: Directs traffic from the public subnet to the internet gateway, allowing resources in the public subnet (like ELBs) to be accessible from the internet.
Private Route Table: Uses the NAT Gateway for routing outbound traffic from private subnets, ensuring that worker nodes can access the internet for essential tasks while remaining unreachable directly from the internet.

Importance of Tags

Tags in AWS serve as identifiers for your resources, enabling you to organize, track, and manage your infrastructure components efficiently. For EKS, tagging subnets correctly is crucial:

Kubernetes.io/cluster/<cluster-name>: This tag, with the value of either shared or owned, is essential for the Kubernetes cluster to identify which VPC resources it can manage and utilize for deploying services like Load Balancers.
kubernetes.io/role/elb: This tag, set to 1, identifies subnets that should be considered for hosting internet-facing ELBs. By tagging a subnet with kubernetes.io/role/elb, you're explicitly allowing Kubernetes to provision external load balancers in these subnets, facilitating access from the internet to services running in your cluster.
kubernetes.io/role/internal-elb: Similarly, this tag, set to 1, designates subnets for hosting internal ELBs. Internal load balancers are used to route traffic within your VPC, offering a method to expose services within your cluster to other components or services in your VPC without exposing them to the internet.

Why Tagging Matters

Tagging subnets with these roles guides the EKS control plane and the AWS Cloud Provider implementation in Kubernetes when automatically creating ELBs for services of type LoadBalancer. Without these tags:

Kubernetes may not be able to correctly provision an ELB for your service, leading to deployment issues or accessibility problems.
You could manually manage load balancer provisioning and association, which increases operational overhead and complexity.

Our completed VPC module will look something like this:

versions.tf:

terraform {
  required_version = ">= 1.5.1"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = ">= 5.20.0"
    }
  }
}

variables.tf:

variable "nat_gateway" {
  type        = bool
  default     = false
  description = "A boolean flag to deploy NAT Gateway."
}

variable "vpc_name" {
  type        = string
  nullable    = false
  description = "Name of the VPC."
}

variable "cidr_block" {
  type        = string
  default     = "10.0.0.0/16"
  description = "The IPv4 CIDR block for the VPC."

  validation {
    condition     = can(cidrnetmask(var.cidr_block))
    error_message = "Must be a valid IPv4 CIDR block address."
  }
}

variable "enable_dns_support" {
  type        = bool
  default     = true
  description = "A boolean flag to enable/disable DNS support in the VPC."
}

variable "enable_dns_hostnames" {
  type        = bool
  default     = false
  description = "A boolean flag to enable/disable DNS hostnames in the VPC."
}

variable "default_tags" {
  type        = map(string)
  default     = {}
  description = "A map of tags to add to all resources."
}

variable "public_subnet_count" {
  type        = number
  default     = 3
  description = "Number of Public subnets."
}

variable "public_subnet_additional_bits" {
  type        = number
  default     = 4
  description = "Number of additional bits with which to extend the prefix."
}

variable "public_subnet_tags" {
  type        = map(string)
  default     = {}
  description = "A map of tags to add to all public subnets."
}

variable "private_subnet_count" {
  type        = number
  default     = 3
  description = "Number of Private subnets."
}

variable "private_subnet_additional_bits" {
  type        = number
  default     = 4
  description = "Number of additional bits with which to extend the prefix."
}

variable "private_subnet_tags" {
  type        = map(string)
  default     = {}
  description = "A map of tags to add to all private subnets."
}

main.tf:

data "aws_availability_zones" "available" {}


############################################################################################################
### VPC 
############################################################################################################
resource "aws_vpc" "main" {

  cidr_block           = var.cidr_block
  enable_dns_support   = var.enable_dns_support
  enable_dns_hostnames = var.enable_dns_hostnames

  tags = merge(
    var.default_tags,
    {
      Name = var.vpc_name
    }
  )
}

resource "aws_default_security_group" "main" {
  vpc_id = aws_vpc.main.id
}

############################################################################################################
### SUBNETS 
############################################################################################################
## Public subnets
resource "aws_subnet" "public" {
  count = var.public_subnet_count

  vpc_id            = aws_vpc.main.id
  cidr_block        = cidrsubnet(var.cidr_block, var.public_subnet_additional_bits, count.index)
  availability_zone = data.aws_availability_zones.available.names[count.index]

  tags = merge(
    var.default_tags, var.public_subnet_tags, {
      Name = "${var.vpc_name}-public-subnet-${count.index + 1}"
  })
}

## Private Subnets
resource "aws_subnet" "private" {
  count = var.private_subnet_count

  vpc_id            = aws_vpc.main.id
  cidr_block        = cidrsubnet(var.cidr_block, var.private_subnet_additional_bits, count.index + var.public_subnet_count)
  availability_zone = data.aws_availability_zones.available.names[count.index]

  tags = merge(
    var.default_tags, var.private_subnet_tags, {
      Name = "${var.vpc_name}-private-subnet-${count.index + 1}"
  })
}


############################################################################################################
### INTERNET GATEWAY
############################################################################################################
resource "aws_internet_gateway" "main" {
  vpc_id = aws_vpc.main.id

  tags = merge(
    var.default_tags, {
      Name = "${var.vpc_name}-internetgateway"
  })
}


############################################################################################################
### NAT GATEWAY 
############################################################################################################
resource "aws_eip" "nat_gateway" {
  count = var.nat_gateway ? 1 : 0

  domain = "vpc"
}

resource "aws_nat_gateway" "main" {
  count = var.nat_gateway ? 1 : 0

  allocation_id = aws_eip.nat_gateway[0].id
  subnet_id     = aws_subnet.public[0].id

  tags = merge(
    var.default_tags, {
      Name = "${var.vpc_name}-natgateway-default"
  })

  depends_on = [
    aws_internet_gateway.main
  ]
}


############################################################################################################
### ROUTE TABLES 
############################################################################################################
# Public Route table
resource "aws_route_table" "public" {
  vpc_id = aws_vpc.main.id

  tags = merge(
    var.default_tags, {
      Name = "${var.vpc_name}-routetable-public"
  })
}

## Public Route Table rules
resource "aws_route" "public" {
  route_table_id         = aws_route_table.public.id
  gateway_id             = aws_internet_gateway.main.id
  destination_cidr_block = "0.0.0.0/0"
}

## Public Route table associations
resource "aws_route_table_association" "public" {
  count = length(aws_subnet.public)

  subnet_id      = aws_subnet.public[count.index].id
  route_table_id = aws_route_table.public.id
}

# Private Route table
resource "aws_route_table" "private" {
  vpc_id = aws_vpc.main.id

  tags = merge(
    var.default_tags, {
      Name = "${var.vpc_name}-routetable-private"
  })
}

## Private Route Table rules
resource "aws_route" "private" {
  route_table_id         = aws_route_table.private.id
  nat_gateway_id         = var.nat_gateway ? aws_nat_gateway.main[0].id : null
  destination_cidr_block = "0.0.0.0/0"
}

## Private Route table associations
resource "aws_route_table_association" "private" {
  count = length(aws_subnet.private)

  subnet_id      = aws_subnet.private[count.index].id
  route_table_id = aws_route_table.private.id
}

outputs.tf

output "vpc_id" {
  value = aws_vpc.main.id
}

output "public_subnets" {
  value = [for subnet in aws_subnet.public : subnet.id]
}

output "private_subnets" {
  value = [for subnet in aws_subnet.private : subnet.id]
}

output "aws_internet_gateway" {
  value = aws_internet_gateway.main
}

output "aws_route_table_public" {
  description = "The ID of the public route table"
  value       = aws_route_table.public.id
}

output "aws_route_table_private" {
  description = "The ID of the private route table"
  value       = aws_route_table.private.id
}


output "nat_gateway_ipv4_address" {
  value = var.nat_gateway ? aws_eip.nat_gateway[0].public_ip : null
}

The usage of the module will look like this:

vpc.tf:

module "vpc" {
  source = "./modules/aws/vpc/v1"

  vpc_name             = "${var.cluster_name}-vpc"
  cidr_block           = "10.0.0.0/16"
  nat_gateway          = true
  enable_dns_support   = true
  enable_dns_hostnames = true

  public_subnet_count  = 3
  private_subnet_count = 3
  public_subnet_tags = {
    "kubernetes.io/cluster/${var.cluster_name}" = "shared"
    "kubernetes.io/role/elb"                    = "1"
  }

  private_subnet_tags = {
    "kubernetes.io/cluster/${var.cluster_name}" = "shared"
    "kubernetes.io/role/internal-elb"           = "1"
  }
}

Securing AWS EKS Networking with Security Groups

In the realm of AWS EKS, securing the network traffic to and from your Kubernetes cluster is paramount. Security groups serve as the first line of defence, defining the rules that allow or deny network traffic to your EKS cluster and worker nodes. This article explores the role of security groups in EKS and outlines the necessary rules for secure communication within your cluster.

Security Groups: The Gatekeepers

EKS Cluster Security Group

The EKS Cluster Security Group acts as a shield for the control plane, governing the traffic to the Kubernetes API server, which is hosted by AWS. It's critical for enabling secure communication between the worker nodes and the control plane.

resource "aws_security_group" "eks_cluster_sg" {
  name        = "${var.cluster_name}-eks-cluster-sg"
  description = "Security group for EKS cluster control plane communication with worker nodes"
  vpc_id      = module.vpc.vpc_id
  tags = {
    Name = "${var.cluster_name}-eks-cluster-sg"
  }
}

Key Rules:

Ingress from Worker Nodes: Allows inbound traffic on port 443 from worker nodes to the control plane, facilitating Kubernetes API calls.

resource "aws_security_group_rule" "eks_cluster_ingress_nodes" {
  type                     = "ingress"
  from_port                = 443
  to_port                  = 443
  protocol                 = "tcp"
  security_group_id        = aws_security_group.eks_cluster_sg.id
  source_security_group_id = aws_security_group.eks_nodes_sg.id
  description              = "Allow inbound traffic from the worker nodes on the Kubernetes API endpoint port"
}

Egress to Kublet: Permits the control plane to initiate traffic to the kubelet running on each worker node, crucial for log collection and commands execution.

resource "aws_security_group_rule" "eks_cluster_egress_kublet" {
  type                     = "egress"
  from_port                = 10250
  to_port                  = 10250
  protocol                 = "tcp"
  security_group_id        = aws_security_group.eks_cluster_sg.id
  source_security_group_id = aws_security_group.eks_nodes_sg.id
  description              = "Allow control plane to node egress for kubelet"
}

Worker Nodes Security Group

Worker Nodes Security Group safeguards your worker nodes, which run your applications. It controls both inbound and outbound traffic to ensure only legitimate and secure communication occurs.

resource "aws_security_group" "eks_nodes_sg" {
  name        = "${var.cluster_name}-eks-nodes-sg"
  description = "Security group for all nodes in the cluster"
  vpc_id      = module.vpc.vpc_id
  tags = {
    Name                                        = "${var.cluster_name}-eks-nodes-sg"
    "kubernetes.io/cluster/${var.cluster_name}" = "owned"
  }
}

Key Rules:

Control to Worker Node: Even though we gave the Cluster security group an egress rule to the Worker Nodes on the kubelet port 10250, the Worker Nodes security group still has actually to allow that traffic.

resource "aws_security_group_rule" "worker_node_ingress_kublet" {
  type                     = "ingress"
  from_port                = 10250
  to_port                  = 10250
  protocol                 = "tcp"
  security_group_id        = aws_security_group.eks_nodes_sg.id
  source_security_group_id = aws_security_group.eks_cluster_sg.id
  description              = "Allow control plane to node ingress for kubelet"
}

Node-to-Node Communication: Allows nodes to communicate among themselves, which is essential for distributed systems like Kubernetes.

resource "aws_security_group_rule" "worker_node_to_worker_node_ingress_ephemeral" {
  type              = "ingress"
  from_port         = 1025
  to_port           = 65535
  protocol          = "tcp"
  self              = true
  security_group_id = aws_security_group.eks_nodes_sg.id
  description       = "Allow workers nodes to communicate with each other on ephemeral ports"
}

Egress to the Internet: Enables nodes to initiate outbound connections to the internet via the NAT Gateway, vital for pulling container images or reaching external services. This also covers any other extra egress rules that would be needed, such as being able to communicate to the control plane on port 443.

resource "aws_security_group_rule" "worker_node_egress_internet" {
  type              = "egress"
  from_port         = 0
  to_port           = 0
  protocol          = "-1"
  cidr_blocks       = ["0.0.0.0/0"]
  security_group_id = aws_security_group.eks_nodes_sg.id
  description       = "Allow outbound internet access"
}

CoreDNS Rules:
When deploying a Kubernetes cluster on AWS EKS, managing DNS resolution and traffic flow between pods is crucial for the stability and performance of your applications. CoreDNS plays a vital role in this ecosystem, serving as the cluster DNS service that enables DNS-based service discovery in Kubernetes. Its integration into an EKS cluster facilitates seamless communication between service endpoints within and outside your cluster. It also has dynamic DNS updates with pod state changes.

resource "aws_security_group_rule" "worker_node_to_worker_node_ingress_coredns_tcp" {
  type              = "ingress"
  from_port         = 53
  to_port           = 53
  protocol          = "tcp"
  security_group_id = aws_security_group.eks_nodes_sg.id
  self              = true
  description       = "Allow workers nodes to communicate with each other for coredns TCP"
}

resource "aws_security_group_rule" "worker_node_to_worker_node_ingress_coredns_udp" {
  type              = "ingress"
  from_port         = 53
  to_port           = 53
  protocol          = "udp"
  security_group_id = aws_security_group.eks_nodes_sg.id
  self              = true
  description       = "Allow workers nodes to communicate with each other for coredns UDP"
}

You can then deploy your project:

$ terraform init
$ terraform plan
$ terraform apply

The output will look something like this:

Our deployed VPC:

Conclusion

As we conclude our exploration of VPC essentials for EKS cluster management with Terraform, it's clear that a well-architected network foundation is pivotal for the successful deployment and operation of Kubernetes clusters on AWS. Through the thoughtful configuration of NAT Gateways, Route Tables, and appropriate tagging, we ensure our EKS clusters are both secure and efficient, positioned to leverage the best of AWS infrastructure.

Security groups are essential for crafting a secure environment for your AWS EKS cluster. By meticulously defining ingress and egress rules, you ensure that your cluster communicates securely within its components and the outside world. Implementing these security measures lays the foundation for a robust and secure Kubernetes ecosystem on AWS.

Karpenter vs. Cluster Autoscaler in EKS: A Comparative Guide

Oluwafemi Lawal — Sat, 10 Feb 2024 09:02:38 +0000

When it comes to managing applications in Kubernetes, it is crucial to balance resource availability without over-provisioning. Autoscaling is the solution to this dilemma, which allows resources to be dynamically adjusted based on the workload requirements. In this regard, let's delve into the intricacies of autoscaling in Kubernetes, with a particular focus on two significant players, namely Cluster Autoscaler and Karpenter, in the context of Amazon EKS.

What is Autoscaling?

Autoscaling refers to the automatic adjustment of computing resources in a server farm, which is usually a cloud environment, to match the current demand. It's similar to having a smart thermostat for your cloud infrastructure that increases the computing power during peak times and reduces it when the demand lessens.

How it Works in Non-EKS Kubernetes Clusters

In vanilla Kubernetes clusters, autoscaling can happen at different levels:

Horizontal Pod Autoscaler (HPA): adjusts the number of pods in a deployment or replica set.

Vertical Pod Autoscaler (VPA): adjusts the CPU and memory reservations for pods. https://github.com/kubernetes/autoscaler/tree/master/vertical-pod-autoscaler
Cluster Autoscaler (CA): adjusts the number of nodes in a cluster.

Cluster Autoscaler: The Kubernetes Classic

Cluster Autoscaler (CA) is like your reliable old hatchback. It's been around since the early days of Kubernetes, ensuring that clusters scale out when needed and scale in when the demand wanes.

History of Cluster Autoscaler

Developed as part of the Kubernetes project, CA has been the go-to for Kubernetes users needing to implement node-level autoscaling. It monitors for pods that couldn't be scheduled due to resource constraints and adjusts the cluster size accordingly.

How Cluster Autoscaler Works

Cluster Autoscaler dynamically optimizes the number of nodes in your EKS cluster. It scales up when it detects unscheduled pods due to resource constraints and scales down when nodes have been underutilized for an extended period.

Scaling Up with Cluster Autoscaler

Cluster Autoscaler checks for unscheduled pods approximately every 10 seconds. If it finds any, it triggers the provisioning of new EC2 instances with specifications mirroring existing nodes, ensuring pods are scheduled promptly.

Scaling Down

It also identifies nodes with low resource utilization and marks them as unnecessary. If these nodes remain unneeded for about 10 minutes, Cluster Autoscaler will terminate them, optimizing resource usage and cost.

Issues Faced with Cluster Autoscaler

Despite its functionality, we encountered challenges with Cluster Autoscaler, especially with volume node affinity conflicts. Stateful pods requiring EBS Volume Storage faced scheduling issues due to node termination and subsequent provisioning in different AWS zones.

Karpenter: The New Kid on the Block

Karpenter is AWS's response to next-generation cluster autoscaling, focusing on optimizing resource utilization and reducing costs through intelligent provisioning.

History of Karpenter

Introduced in late 2021, Karpenter was created to improve the efficiency of resource allocation within Kubernetes. Karpenter emerged as a solution to the limitations of Cluster Autoscaler, offering intelligent node provisioning based on actual workload requirements, significantly optimizing resource utilization and cost.

How Karpenter Works

Karpenter takes a proactive approach to autoscaling, using custom Kubernetes resources called “provisioners” to define and manage resource provisioning based on application requirements. This method ensures optimal resource utilization, as Karpenter can tailor provision nodes to the workload's specific needs.

Key Features of Karpenter

Elimination of Multiple Autoscaling Groups: Karpenter is zone-aware, capable of provisioning nodes in required zones without managing numerous autoscaling groups.
Intelligent Node Allocation: It provisions nodes based on precise workload requirements, optimizing costs and system availability.
Versatile Node Configuration: Supports diverse configurations, allowing for an optimal match between instance types and pod requirements.
Enhanced Scaling and Scheduling: Offers fast, efficient scaling based on workload and resource utilization, maintaining optimal performance.

Comparing Karpenter to Cluster Autoscaler

User Friendliness: Karpenter simplifies setup and integration with Kubernetes, contrasting with the Cluster Autoscaler's potentially complex configuration.

Optimal Resource Utilization: Karpenter's ability to provision nodes based on application requirements leads to more efficient use of resources.

Customization: Karpenter allows for detailed scaling configurations, such as scaling based on custom metrics.

Optimized Compute Costs: Unlike Cluster Autoscaler, Karpenter's efficient bin packing and workload consolidation can reduce computation costs.

Volume Node Affinity Conflict: Karpenter's intelligent scheduling resolves the challenge of stateful pods by ensuring node and volume affinity, mitigating unscheduled pod issues.

Conclusion

While the Cluster Autoscaler provides a reliable, time-tested solution for autoscaling, Karpenter introduces a more dynamic and cost-effective approach when paired with AWS EKS. By understanding the unique features and capabilities of each, you can choose the right tool for your Kubernetes environment, ensuring efficient resource management and cost savings.

Unraveling Kubernetes: From EC2 to EKS

Oluwafemi Lawal — Fri, 09 Feb 2024 14:37:03 +0000

What is Kubernetes?

The concept of isolating application code is something that has been around since the 1970s. By the year 2012, containers running on Linux platforms were transformed into an operating system-level virtualization technology, the need to manage multiple interconnected containers at scale became increasingly evident, and multiple technologies were created to solve just that.

Kubernetes was released in September of 2014 (almost two most after Terraform 👀). At its core, it is an open-source platform designed to automate the deployment, scaling, and operation of application containers across clusters of hosts. It provides the framework to run distributed systems resiliently, taking care of scaling and failover for your application, providing deployment patterns, and more. Essentially, Kubernetes is about abstracting away the complexity of managing a fleet of containers, which are lightweight, portable, self-sufficient packages that can run cloud-native applications.

The Benefits of Kubernetes

Kubernetes offers a plethora of benefits that have made it the de facto standard for container orchestration:

Scalability: Effortlessly scale your applications up or down based on demand, with Kubernetes handling the complexity of managing container distribution across the cluster.
High Availability: Kubernetes ensures your application is always available, regardless of individual server failures, by distributing containers across multiple nodes.
Portability: Run your applications on any public cloud, private cloud, or on-premises server.
Self-healing: Automatically restarts containers that fail, replaces and reschedules containers when nodes die, kills containers that don't respond to user-defined health checks, and doesn't advertise them to clients until they are ready to serve.

Historical Need to Run Kubernetes on EC2 VMs

Before the advent of Kubernetes as a managed service in AWS, deploying it on virtual machines, such as AWS EC2, was the norm. This setup involved manually installing Kubernetes on each VM, configuring the network, setting up storage, and ensuring security and isolation between the containers. The process was intricate, involving:

Setting up an EC2 instance for the Kubernetes master.
Configuring additional EC2 instances to serve as worker nodes.
Installing Kubernetes on all instances and configuring them to communicate with each other.
Setting up networking to allow containers to communicate within the cluster and with the outside world.

This manual setup was time-consuming and required a deep understanding of Kubernetes internals.

What is EKS?

Amazon Elastic Kubernetes Service (EKS) is AWS's managed service, making it easy to run Kubernetes on AWS and on-premises with Amazon EKS Anywhere. EKS abstracts away the complexity of installing and managing Kubernetes, providing a highly available and secure Kubernetes control plane. With EKS, AWS manages the Kubernetes master nodes, relieving developers and sysadmins from the operational overhead of managing the Kubernetes infrastructure.

Differences between EKS vs Running your own Kubernetes Cluster on EC2 VMs

The choice between EKS and self-managed Kubernetes on EC2 boils down to the trade-off between control and convenience:

Managed Service vs Manual Configuration: EKS provides a managed Kubernetes service where the control plane is managed by AWS, reducing the operational complexity. In contrast, running Kubernetes on EC2 requires manual setup and management.
Scalability and Availability: EKS automatically scales the Kubernetes control plane, ensuring high availability without manual intervention. Running Kubernetes on EC2 requires manual scaling and failover management.
Security and Compliance: EKS is integrated with AWS security services, such as IAM, for authentication, simplifying compliance and security management. A self-managed Kubernetes setup requires manual configuration of security policies and networking rules.
Cost: While EKS simplifies Kubernetes management, it comes with additional costs for the managed control plane. Running Kubernetes on EC2 gives you more control over costs, especially if you can optimize your cluster's utilization and manage its complexity.The EKS control plane requires nodes in multiple availability zones to ensure high availability, this translates to a minimal cost of $75 a month for each Amazon EKS cluster with a standard 3-node configuration.

Conclusion

Kubernetes has significantly simplified containerised applications' deployment, scaling, and management. While running Kubernetes on EC2 VMs offers full control over the Kubernetes cluster, it comes with the added complexity of manual management. EKS, on the other hand, provides a managed Kubernetes service that abstracts much of this complexity, allowing developers to focus on building their applications rather than managing the underlying infrastructure. Ultimately, choosing between EKS and a self-managed Kubernetes cluster on EC2 depends on your specific needs, skills, and budget constraints. Regardless of the path chosen, Kubernetes remains a powerful tool in the cloud-native ecosystem, enabling businesses to deploy and manage applications at scale efficiently.

How to Deploy to AWS from Fargate backed Gitlab Runners

Oluwafemi Lawal — Sat, 18 Feb 2023 14:27:02 +0000

Introduction

I love AWS CodePipeline. It integrates with GitHub and Bitbucket, you can add as many stages for your pipeline as you want, be it for Approval, Build, Invoking, Testing, and efficiently deploying to multiple AWS Services. One thing it does not to integrate with, unfortunately (at the time of my writing this article), is Gitlab.

This article will cover the steps required to deploy to AWS from a GitLab runner backed by the Amazon Fargate service. I will not talk about how to set everything up, Gitlab covers it in detail on their documentation https://docs.gitlab.com/runner/configuration/runner_autoscale_aws_fargate/

Prerequisites

Before we get started, there are a few things you need to have set up:

A GitLab account with a repository that will run CI jobs with Fargate.
An AWS account.
GitLab runner set up and running in your AWS account.
Fargate task definition that Gitlab runner will use to run your CI jobs.

A modified version of the example debian image in the instructions from Gitlab will be used for the Fargate task Gitlab runner invokes, the only addition is the AWS CLI



FROM debian:buster

# ---------------------------------------------------------------------
# Install https://github.com/krallin/tini - a very small 'init' process
# that helps processing signalls sent to the container properly.
# ---------------------------------------------------------------------
ARG TINI_VERSION=v0.19.0

RUN apt-get update && \
    apt-get install -y curl && \
    curl -Lo /usr/local/bin/tini https://github.com/krallin/tini/releases/download/${TINI_VERSION}/tini-amd64 && \
    chmod +x /usr/local/bin/tini

# --------------------------------------------------------------------------
# Install and configure sshd.
# https://docs.docker.com/engine/examples/running_ssh_service for reference.
# --------------------------------------------------------------------------
RUN apt-get install -y openssh-server && \
    # Creating /run/sshd instead of /var/run/sshd, because in the Debian
    # image /var/run is a symlink to /run. Creating /var/run/sshd directory
    # as proposed in the Docker documentation linked above just doesn't
    # work.
    mkdir -p /run/sshd

EXPOSE 22

# ----------------------------------------
# Install GitLab CI required dependencies.
# ----------------------------------------
ARG GITLAB_RUNNER_VERSION=v12.9.0

RUN curl -Lo /usr/local/bin/gitlab-runner https://gitlab-runner-downloads.s3.amazonaws.com/${GITLAB_RUNNER_VERSION}/binaries/gitlab-runner-linux-amd64 && \
    chmod +x /usr/local/bin/gitlab-runner && \
    # Test if the downloaded file was indeed a binary and not, for example,
    # an HTML page representing S3's internal server error message or something
    # like that.
    gitlab-runner --version

RUN apt-get install -y bash ca-certificates git git-lfs && \
    git lfs install --skip-repo

# ----------------------------------------
# Install AWS CLI.
# ----------------------------------------
RUN apt-get install -y awscli

# -------------------------------------------------------------------------------------
# Execute a startup script.
# https://success.docker.com/article/use-a-script-to-initialize-stateful-container-data
# for reference.
# -------------------------------------------------------------------------------------
COPY docker-entrypoint.sh /usr/local/bin/docker-entrypoint.sh

RUN chmod +x /usr/local/bin/docker-entrypoint.sh

ENTRYPOINT ["tini", "--", "/usr/local/bin/docker-entrypoint.sh"]

You will have to create an ECR repository for the image, build your image and push it ECR, then create the task definition that uses the image.

Setting Up The Deployment

The scenario is you have a Gitlab repository that holds a CloudFormation template for creating ECR Repositories, merges to the main branch should update the template in your AWS account. Our pipeline requires two key components

Creating a ecr.yaml CloudFormation template.



AWSTemplateFormatVersion: "2010-09-09"

Description: >
  This template creates ECR resources

Parameters:
  IAMUserName:
    Type: String
    Description: IAM User Name
    Default: "YOUR_USER_NAME"
    AllowedPattern: "[a-zA-Z0-9-_]+"
    ConstraintDescription: must be a valid IAM user name  

Resources:
  ECRRepository:
    Type: AWS::ECR::Repository
    Properties:
      RepositoryName: !Ref AWS::StackName
      RepositoryPolicyText:
        Version: "2012-10-17"
        Statement:
          - Sid: "AllowPushPull"
            Effect: Allow
            Principal:
              AWS:
                !Join [
                  "",
                  [
                    "arn:aws:iam::",
                    !Ref AWS::AccountId,
                    ":user/",
                    !Ref IAMUserName
                  ],
                ]
            Action:
              - "ecr:GetDownloadUrlForLayer"
              - "ecr:BatchGetImage"
              - "ecr:BatchCheckLayerAvailability"
              - "ecr:PutImage"
              - "ecr:InitiateLayerUpload"
              - "ecr:UploadLayerPart"
              - "ecr:CompleteLayerUpload"

Outputs:
  ECRRepository:
    Description: ECR repository
    Value: !Ref ECRRepository

Modifying the .gitlab-ci.yml file to include a deployment stage that will deploy your changes to AWS. ```yaml

variables:
STACK_NAME: ecr-stack
TEMPLATE_PATH: /templates/ecr.yaml

stages:

deploy

cloudformation_deploy:
stage: deploy
script:
- aws cloudformation deploy --template-file $TEMPLATE_PATH --stack-name $STACK_NAME --capabilities CAPABILITY_NAMED_IAM


## The Problem
The pipeline is not actually going to work, the AWS CLI will be unable to locate the credentials and will ask you to run `aws configure`. You might be thinking the simple solution is to just create credentials for Gitlab then set the AWS_SECRET_ACCESS_KEY and AWS_ACCESS_KEY_ID environment variables, but that's completely unnecessary (and not very secure). 

The pipeline is running in a Fargate container within your AWS account and has proper permissions, so why doesn't it work?


![Illustration of the Fargate task](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/berld3eid8iejxipt8w0.png)





Ordinarily, the container makes a request to the task metadata endpoint to get temporary AWS credentials, but unfortunately, the environment variable it uses for that is only available to the init container process (PID 1). So to fix this issue, we have to set that environment variable.

How do we get an environment variable from an entirely different process? PID 1 environment variables are stored in the environ file of the process, so we can retrieve it from there by running:
```bash


export $(strings /proc/1/environ | grep AWS_CONTAINER_CREDENTIALS_RELATIVE_URI)

This will make the variable avaialable to the CI process and enable to container to retrieve temporary AWS credentials, the final .gitlab.ci.yaml file will look like this:



variables:

  STACK_NAME: ecr-stack

  TEMPLATE_PATH: /templates/ecr.yaml

stages:

  - deploy

deploy:

  stage: deploy

  script:

    - export $(strings /proc/1/environ | grep AWS_CONTAINER_CREDENTIALS_RELATIVE_URI)

    - aws cloudformation deploy --template-file $TEMPLATE_PATH --stack-name $STACK_NAME --capabilities CAPABILITY_NAMED_IAM

Conclusion

In this article, we covered the steps required to use the AWS CLI for deployments from a GitLab runner backed by the Amazon Fargate service. With these steps in place, you can automate the deployment of your resources to AWS.

How To SSH Into An ECS Fargate Container

Oluwafemi Lawal — Thu, 07 Jul 2022 08:03:21 +0000

The Problem

Best practices dictate that you should not SSH into containers; instead, you should implement proper observability mechanisms for monitoring, debugging, and log analysis. So it was impossible with ECS, at least not until the 16th of March 2021. After that, it was a highly requested feature on the AWS Container roadmap. AWS eventually gave in because they understood that engineers must be able to debug and test quickly in the early stages of development. It can also be helpful to debug high severity issues in production.

The Solution

Before getting into your ECS container, you would have to open the SSH ports, get keys/passwords to the host instance, SSH into that instance, and then docker exec into the container. If you were running the serverless Fargate model, you couldn't even do this because you do not have access to the host machine, so you'd have to redeploy it to an EC2 instance.

Then ECS exec came to the rescue, and it allows you to docker exec right into the container you want without the overhead of worrying about security issues with distributing SSH keys or passwords

Implementation

I will use the Fargate container from this post and make modifications to specific parts of the CloudFormation template.

IAM Policy for the Task Role
KMS key to encrypt the ECS Exec data channel
Enable Execute command
Place cluster in public subnets The template already has the cluster in public subnets, so first, we'll add the IAM Policy. ``` yaml TaskRole: Type: AWS::IAM::Role Properties: RoleName: !Join ["-", [!Ref ServiceName, TaskRole]] AssumeRolePolicyDocument: Statement: - Effect: Allow Principal: Service: ecs-tasks.amazonaws.com Action: "sts:AssumeRole" Policies: - PolicyName: "systems-manager" PolicyDocument: Version: "2012-10-17" Statement: - Effect: "Allow" Action: - "ssmmessages:CreateControlChannel" - "ssmmessages:CreateDataChannel" - "ssmmessages:OpenControlChannel" - "ssmmessages:OpenDataChannel" Resource: "*" - Effect: "Allow" Action: - "kms:Decrypt" Resource: !Ref KMSKey

Then the KMS key:
```yaml


  ############################################################
  #              KMS KEY
  ############################################################
  KMSKey:
    Type: AWS::KMS::Key
    Properties:
      Description: Key needed to encrypt docker exec data channel
      KeyPolicy:
        Version: "2012-10-17"
        Statement:
          - Effect: "Allow"
            Principal:
              AWS: !Sub "arn:aws:iam::${AWS::AccountId}:root"
            Action:
              - "kms:*"
            Resource: "*"

Now we enable the exec command on the service, that can be done by adding EnableExecuteCommand: true



  ############################################################
  #              ECS SERVICE
  ############################################################
  Service:
    Type: AWS::ECS::Service
    # This dependency is needed so that the load balancer is setup correctly in time
    DependsOn:
      - ApplicationLoadBalancer
      - ALBHTTPListener
      - ListenerRule
      - TargetGroup
    Properties:
      ServiceName: !Ref ServiceName
      Cluster: !Ref Cluster
      TaskDefinition: !Ref TaskDefinition
      EnableExecuteCommand: true        
      DeploymentConfiguration:
        MinimumHealthyPercent: 100
        MaximumPercent: 200
      DesiredCount: !Ref MinContainers
      HealthCheckGracePeriodSeconds: 300
      LaunchType: FARGATE
      NetworkConfiguration:
        AwsvpcConfiguration:
          AssignPublicIp: ENABLED
          Subnets:
            - !Ref PublicSubnetOne
            - !Ref PublicSubnetTwo
          SecurityGroups:
            - !Ref ContainerSecurityGroup
      LoadBalancers:
        - ContainerName: !Ref ServiceName
          ContainerPort: !Ref ContainerPort
          TargetGroupArn: !Ref TargetGroup

You can deploy the template:



aws cloudformation deploy --template-file ecs.yml --stack-name ecs-infrastructure --capabilities CAPABILITY_NAMED_IAM --profile default

Then you can get the Task Id by running:



aws ecs list-tasks --cluster rest-api-cluster

The task id is the part highlighted in red below:

This is an alpine linux docker image, so instead of "/bin/bash", to docker exec into it, "/bin/sh" will have to be used instead.



aws ecs execute-command     --region us-east-1    --cluster rest-api-cluster     --task INSERT_YOUR_TASK_ID_HERE     --container rest-api     --command "/bin/sh"     --interactive

Sample output:

You can use this tool to troubleshoot issues.

Cleanup

Always remember to delete resources you do not need, run:



aws cloudformation delete-stack --stack-name ecs-infrastructure

You can check the status of the stack to confirm if it was deleted successfully:



aws cloudformation describe-stacks --stack-name ecs-infrastructure

Confirm the stack was deleted successfully; remember to clean up the ECR repository if you implemented that.

Creating An ECS cluster with CloudFormation

Oluwafemi Lawal — Wed, 06 Jul 2022 22:36:09 +0000

What is ECS?

Amazon Elastic Container Service (Amazon ECS) is the AWS container orchestration service that runs and manages Docker containers. With ECS, you can run clusters of virtual machines with either the EC2-backed option or the serverless option with Fargate.
You can read more about ECS in the AWS documentation

Infrastructure

Disclaimer: This is not created for a production use case. It is just for illustration purposes.
Things that will be created included in this post:

VPC
Internet Gateway
Public Subnets
Route tables
ECS cluster
ECS Service
Task definition
IAM roles
Application Load balancer
Target group
Security groups
CloudWatch Log groups

That's a long list, but we will accomplish this with CloudFormation. To use HTTPS with the load balancer, you would need to create an SSL certificate in AWS Certificate Manager, but this post will make a load balancer with HTTP, and the cluster will be in public subnets instead of private.
The application used is a simple rest api.
You can create an ECR repository by following this post and push the docker image to the repo following this post.

I could divide the CloudFormation template into multiple templates, then export and import values when needed, and give detailed explanations for what each template does, but that would make this post very long...

I'm going to post everything in a single template, you can consult the CloudFormation documentation for explanations



AWSTemplateFormatVersion: "2010-09-09"

Description: >
  This template creates ECS Fargate Services.

############################################################
#              PARAMETERS RECORDS BLOCK
############################################################

Parameters:
  ImageTag:
    Type: String
    Default: latest
  ServiceName:
    Type: String
    Default: rest-api
  ContainerPort:
    Type: Number
    Default: 80
  ContainerCpu:
    Type: String
    Default: 256
  ContainerMemory:
    Type: String
    Default: 1GB
  HealthCheckPath:
    Type: String
    Default: /
  MinContainers:
    Type: Number
    Default: 1
  VpcCIDR:
    Description: "IP range for this VPC"
    Type: "String"
    Default: "10.0.0.0/16"
  PublicSubnetOneCIDR:
    Description: "IP range for the public subnet in the first Availability Zone"
    Type: "String"
    Default: "10.0.1.0/24"
  PublicSubnetTwoCIDR:
    Description: "IP range for the public subnet in the second Availability Zone"
    Type: "String"
    Default: "10.0.2.0/24"

Resources:

  ############################################################
  #            VPC & PUBLIC SUBNETS
  ############################################################

  RestVPC:
    Type: "AWS::EC2::VPC"
    Properties:
      CidrBlock: !Ref "VpcCIDR"
      Tags:
        - Key: "Name"
          Value: "Rest-API-VPC"
  ####### Create Public Subnet #######
  PublicSubnetOne:
    Type: "AWS::EC2::Subnet"
    Properties:
      VpcId: !Ref RestVPC
      CidrBlock: !Ref "PublicSubnetOneCIDR"
      AvailabilityZone: !Select ["0", !GetAZs ]
      MapPublicIpOnLaunch: "True"
      Tags:
        - Key: "Name"
          Value: !Sub "${PublicSubnetOneCIDR}-PublicSubnetOne"

  PublicSubnetTwo:
    Type: "AWS::EC2::Subnet"
    Properties:
      VpcId: !Ref RestVPC
      CidrBlock: !Ref "PublicSubnetTwoCIDR"
      AvailabilityZone: !Select ["1", !GetAZs ]
      MapPublicIpOnLaunch: "True"
      Tags:
        - Key: "Name"
          Value: !Sub "${PublicSubnetTwoCIDR}-PublicSubnetTwo"

  ######## Create Public Route Table #######
  PublicRouteTable1:
    Type: "AWS::EC2::RouteTable"
    Properties:
      VpcId: !Ref RestVPC
      Tags:
        - Key: "Name"
          Value: "PublicRoute1"

  PublicRouteTable2:
    Type: "AWS::EC2::RouteTable"
    Properties:
      VpcId: !Ref RestVPC
      Tags:
        - Key: "Name"
          Value: "PublicRoute2"

  ######## Create Internet Gateway #######
  InternetGateway:
    Type: "AWS::EC2::InternetGateway"
    Properties:
      Tags:
        - Key: "Name"
          Value: "InternetGateway"

  ######## Attach Internet Gateway to VPC #######
  GatewayToInternet:
    Type: "AWS::EC2::VPCGatewayAttachment"
    Properties:
      VpcId: !Ref RestVPC
      InternetGatewayId: !Ref "InternetGateway"

  ######## Route-out Public Route Table to Internet Gateway (Internet connection)  #######
  PublicRouteIGW1:
    Type: "AWS::EC2::Route"
    DependsOn: "GatewayToInternet"
    Properties:
      RouteTableId: !Ref "PublicRouteTable1"
      DestinationCidrBlock: "0.0.0.0/0"
      GatewayId: !Ref "InternetGateway"

  PublicRouteIGW2:
    Type: "AWS::EC2::Route"
    DependsOn: "GatewayToInternet"
    Properties:
      RouteTableId: !Ref "PublicRouteTable2"
      DestinationCidrBlock: "0.0.0.0/0"
      GatewayId: !Ref "InternetGateway"

  ######## Associate Public Route Table with Public Subnet1 & Subnet2 #######
  PublicSubnetOneRouteTableAssociation:
    Type: "AWS::EC2::SubnetRouteTableAssociation"
    Properties:
      SubnetId: !Ref "PublicSubnetOne"
      RouteTableId: !Ref "PublicRouteTable1"

  PublicSubnetTwoRouteTableAssociation:
    Type: "AWS::EC2::SubnetRouteTableAssociation"
    Properties:
      SubnetId: !Ref "PublicSubnetTwo"
      RouteTableId: !Ref "PublicRouteTable2"

  ######## Create Custom Network ACL #######
  PublicNetworkACL:
    Type: "AWS::EC2::NetworkAcl"
    Properties:
      VpcId: !Ref RestVPC
      Tags:
        - Key: "Name"
          Value: "PublicNetworkACL"

  PublicInboundPublicACL:
    Type: "AWS::EC2::NetworkAclEntry"
    Properties:
      NetworkAclId: !Ref PublicNetworkACL
      RuleNumber: "100"
      Protocol: "-1"
      RuleAction: "allow"
      Egress: "false"
      CidrBlock: "0.0.0.0/0"

  PublicOutboundPublicACL:
    Type: "AWS::EC2::NetworkAclEntry"
    Properties:
      NetworkAclId: !Ref PublicNetworkACL
      RuleNumber: "100"
      Protocol: "-1"
      RuleAction: "allow"
      Egress: "true"
      CidrBlock: "0.0.0.0/0"

  ######## Associate Public Subnet to Network ACL #######
  PublicSubnetOneNetworkAclAssociation:
    Type: "AWS::EC2::SubnetNetworkAclAssociation"
    Properties:
      SubnetId: !Ref "PublicSubnetOne"
      NetworkAclId: !Ref "PublicNetworkACL"

  PublicSubnetTwoNetworkAclAssociation:
    Type: "AWS::EC2::SubnetNetworkAclAssociation"
    Properties:
      SubnetId: !Ref "PublicSubnetTwo"
      NetworkAclId: !Ref "PublicNetworkACL"

  ############################################################
  #              ECS CLUSTER BLOCK
  ############################################################
  Cluster:
    Type: AWS::ECS::Cluster
    Properties:
      ClusterName: !Join ["-", [!Ref ServiceName, cluster]]

  ############################################################
  #              ECS TASK DEFINITION
  ############################################################
  TaskDefinition:
    Type: AWS::ECS::TaskDefinition
    DependsOn: LogGroup
    Properties:
      # Name of the task definition. Subsequent versions of the task definition are grouped together under this name.
      Family: !Join ["-", [!Ref ServiceName, TaskDefinition]]
      # awsvpc is required for Fargate
      NetworkMode: awsvpc
      RequiresCompatibilities:
        - FARGATE
      Cpu: !Ref ContainerCpu
      Memory: !Ref ContainerMemory
      ExecutionRoleArn: !Ref ExecutionRole
      TaskRoleArn: !Ref TaskRole
      ContainerDefinitions:
        - Name: !Ref ServiceName
          Image:
            !Join [
              ".",
              [
                !Ref AWS::AccountId,
                "dkr.ecr",
                !Ref AWS::Region,
                !Sub "amazonaws.com/ecr-repository:${ImageTag}",
              ],
            ]
          PortMappings:
            - ContainerPort: !Ref ContainerPort
          # Send logs to CloudWatch Logs
          LogConfiguration:
            LogDriver: awslogs
            Options:
              awslogs-region: !Ref AWS::Region
              awslogs-group: !Ref LogGroup
              awslogs-stream-prefix: ecs

  ############################################################
  #              IAM ROLE
  ############################################################

  ExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: !Join ["-", [!Ref ServiceName, ExecutionRole]]
      AssumeRolePolicyDocument:
        Statement:
          - Effect: Allow
            Principal:
              Service: ecs-tasks.amazonaws.com
            Action: "sts:AssumeRole"
      ManagedPolicyArns:
        - "arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy"

  TaskRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: !Join ["-", [!Ref ServiceName, TaskRole]]
      AssumeRolePolicyDocument:
        Statement:
          - Effect: Allow
            Principal:
              Service: ecs-tasks.amazonaws.com
            Action: "sts:AssumeRole"

  ############################################################
  #              SECURITY GROUP
  ############################################################
  ContainerSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: !Join ["-", [!Ref ServiceName, ContainerSecurityGroup]]
      VpcId: !Ref RestVPC
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: !Ref ContainerPort
          ToPort: !Ref ContainerPort
          SourceSecurityGroupId: !Ref LoadBalancerSecurityGroup
  LoadBalancerSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription:
        !Join ["-", [!Ref ServiceName, LoadBalancerSecurityGroup]]
      VpcId: !Ref RestVPC
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 80
          ToPort: 80
          CidrIp: 0.0.0.0/0

  ############################################################
  #              ECS SERVICE
  ############################################################
  Service:
    Type: AWS::ECS::Service
    # This dependency is needed so that the load balancer is setup correctly in time
    DependsOn:
      - ApplicationLoadBalancer
      - ALBHTTPListener
      - ListenerRule
      - TargetGroup
    Properties:
      ServiceName: !Ref ServiceName
      Cluster: !Ref Cluster
      TaskDefinition: !Ref TaskDefinition
      DeploymentConfiguration:
        MinimumHealthyPercent: 100
        MaximumPercent: 200
      DesiredCount: !Ref MinContainers
      HealthCheckGracePeriodSeconds: 300
      LaunchType: FARGATE
      NetworkConfiguration:
        AwsvpcConfiguration:
          AssignPublicIp: ENABLED
          Subnets:
            - !Ref PublicSubnetOne
            - !Ref PublicSubnetTwo
          SecurityGroups:
            - !Ref ContainerSecurityGroup
      LoadBalancers:
        - ContainerName: !Ref ServiceName
          ContainerPort: !Ref ContainerPort
          TargetGroupArn: !Ref TargetGroup

  ############################################################
  #              ECS TARGET GROUP
  ############################################################
  TargetGroup:
    Type: AWS::ElasticLoadBalancingV2::TargetGroup
    Properties:
      HealthCheckIntervalSeconds: 10
      HealthCheckPath: !Ref HealthCheckPath
      HealthCheckTimeoutSeconds: 5
      UnhealthyThresholdCount: 2
      HealthyThresholdCount: 2
      Name: !Join ["-", [!Ref ServiceName, TargetGroup]]
      Port: !Ref ContainerPort
      Protocol: HTTP
      TargetGroupAttributes:
        - Key: deregistration_delay.timeout_seconds
          Value: 60
      TargetType: ip
      VpcId: !Ref RestVPC
      Matcher:
        HttpCode: 200

  ############################################################
  #              APPLICATION LOAD BALANCER
  ############################################################
  ApplicationLoadBalancer:
    Type: AWS::ElasticLoadBalancingV2::LoadBalancer
    Properties:
      LoadBalancerAttributes:
        - Key: idle_timeout.timeout_seconds
          Value: 60
      Name: !Join ["-", [!Ref ServiceName, ApplicationLoadBalancer]]
      Scheme: internet-facing
      SecurityGroups:
        - !Ref LoadBalancerSecurityGroup
      Subnets:
        - !Ref PublicSubnetOne
        - !Ref PublicSubnetTwo

  ALBHTTPListener:
    Type: "AWS::ElasticLoadBalancingV2::Listener"
    Properties:
      LoadBalancerArn: !Ref ApplicationLoadBalancer
      DefaultActions:
        - FixedResponseConfig:
            StatusCode: 200
          Type: fixed-response
      Port: "80"
      Protocol: HTTP

  # ---- Applcation Load Balancer Listener Rule ---- #
  ListenerRule:
    Type: "AWS::ElasticLoadBalancingV2::ListenerRule"
    Properties:
      Actions:
        - Type: forward
          TargetGroupArn: !Ref TargetGroup
      Conditions:
        - Field: path-pattern
          Values:
            - "*"
      ListenerArn: !Ref ALBHTTPListener
      Priority: 1

  ############################################################
  #              CLOUDWATCH LOG GROUPS
  ############################################################
  LogGroup:
    Type: AWS::Logs::LogGroup
    Properties:
      LogGroupName: !Join ["", [/ecs/, !Ref ServiceName, TaskDefinition]]

I named the template ecs.yaml. You can deploy it by running:



aws cloudformation deploy --template-file ecs.yml --stack-name ecs-infrastructure --capabilities CAPABILITY_NAMED_IAM --profile default

You can navigate to the ECS console and confirm your cluster was created successfully.

You can also open the load balancer address to see if the container works as expected.

Cleanup

Always remember to delete resources you do not need, run:



aws cloudformation delete-stack --stack-name ecs-infrastructure

You can check the status of the stack to confirm if it was deleted successfully:



aws cloudformation delete-stack --stack-name ecs-infrastructure

Confirm the stack was deleted successfully; remember to clean up the ECR repository if you implemented that.

Creating Your Private Versions of Public Docker Images in ECR

Oluwafemi Lawal — Wed, 06 Jul 2022 11:20:18 +0000

Why do this?

One reason could be avoiding Docker Hub's free tier rate limits. If your team uses free Docker Hub, you might run into rate limits while pulling images into something like AWS CodeBuild. The rate limit is set to 100 pulls per 6 hours per IP address if you are anonymous and 200 pulls per 6 hours for authenticated users with a Docker ID.

Creation process

You can create an ECR Repository following this guide.
We will save an image of ubuntu

First, pull the docker image:

docker pull ubuntu:20.04

You should get an output similar to this:

Then create a target image tag that refers to the source image:

docker tag ubuntu:20.04 YOUR_AWS_ACCOUNT_ID.dkr.ecr.us-east-1.amazonaws.com/ecr-repository

Before you can push to ECR, you have to authenticate:

aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin YOUR_AWS_ACCOUNT_ID.dkr.ecr.us-east-1.amazonaws.com

The output should be:

Then push the image:

docker push YOUR_AWS_ACCOUNT_ID.dkr.ecr.us-east-1.amazonaws.com/ecr-repository

The output should be:

You can confirm it's creation in the AWS console:

Cleanup

Always remember to clean up resources you do not need.
First delete the image:

aws ecr batch-delete-image --repository-name ecr-repository --image-ids imageTag=latest

Assuming you used the CloudFormation template from this lesson, delete the repository by running:

aws cloudformation delete-stack --stack-name ecr-repository

Creating An Elastic Container Registry Repo With CloudFormation

Oluwafemi Lawal — Tue, 05 Jul 2022 15:54:16 +0000

What is Amazon ECR?

A Container Registry is a centralized way to distribute and manage container images. Amazon ECR is simply a container registry that AWS provides, and you can create public and private images, just like on Docker Hub.

Why Amazon ECR?

Ecosystem

Some teams prefer to keep all their services in the same ecosystem. In this case, AWS can make things much easier to manage and control user access. For example, you can use ECR instead of Docker Hub or Amazon Elastic Container Service (a fully managed container orchestration service) instead of Kubernetes (the de-facto standard for container orchestration). Instead, you can use Kubernetes on AWS with Amazon Elastic Kubernetes Service.

Rate limits

There is at least one other reason to use ECR, if your team uses free Docker Hub, you might run into rate limits while pulling images into something like AWS CodeBuild. The rate limit is set to 100 pulls per 6 hours per IP address if you are anonymous and 200 pulls per 6 hours for authenticated users with a Docker ID.

The cheapest way to overcome this might be to add the image to your AWS accounts ECR and use that instead.

Creating the ECR Repository

You can create an ECR Repository manually in the ECR console, but we will focus on the CloudFormation method.
You can create a file called ecr.yml and paste the template below inside it:

AWSTemplateFormatVersion: "2010-09-09"

Description: >
  This template creates ECR resources

Parameters:
  IAMUserName:
    Type: String
    Description: IAM User Name
    Default: "YOUR_USER_NAME"
    AllowedPattern: "[a-zA-Z0-9-_]+"
    ConstraintDescription: must be a valid IAM user name  

Resources:
  ECRRepository:
    Type: AWS::ECR::Repository
    Properties:
      RepositoryName: !Ref AWS::StackName
      RepositoryPolicyText:
        Version: "2012-10-17"
        Statement:
          - Sid: "AllowPushPull"
            Effect: Allow
            Principal:
              AWS:
                !Join [
                  "",
                  [
                    "arn:aws:iam::",
                    !Ref AWS::AccountId,
                    ":user/",
                    !Ref IAMUserName
                  ],
                ]
            Action:
              - "ecr:GetDownloadUrlForLayer"
              - "ecr:BatchGetImage"
              - "ecr:BatchCheckLayerAvailability"
              - "ecr:PutImage"
              - "ecr:InitiateLayerUpload"
              - "ecr:UploadLayerPart"
              - "ecr:CompleteLayerUpload"

Outputs:
  ECRRepository:
    Description: ECR repository
    Value: !Ref ECRRepository

The above template is responsible for creating the repo and proving an IAM user, specified in the YOUR_USER_NAME parameter, the necessary permissions to use the repository. Please replace YOUR_USER_NAME with your actual IAM username, or the creation will fail.
To deploy the template, run:

aws cloudformation deploy --template-file ecr.yml --stack-name ecr-repository --capabilities CAPABILITY_NAMED_IAM --profile default --parameter-overrides IAMUserName=YOUR_USER_NAME

and you will get an output similar to this:

You can check the status of the stack creation by running

AWS cloudformation describe-stacks --stack-name ecr-repository

If it created successfully, you should get an output similar to this with a StackStatus of CREATE_COMPLETE:

You can also check on the CloudFormation console in the region your default profile is set to.

Then in the ECR console, your repository should be there.

Cleanup

Always clean up resources you don't need to avoid unexpected bills.
To delete to the repository, run:

aws cloudformation delete-stack --stack-name ecr-repository

Confirm the delete was successful by checking the status of the stack,

AWS cloudformation describe-stacks --stack-name ecr-repository

and you should get a response similar to this

You can double-check in the AWS console to be sure.

DEV Community: Oluwafemi Lawal

Go vs Python on AWS Lambda: a practical Terraform shootout (with Function URLs)

Heads-up: Go's managed runtime is deprecated

What you'll build

Why Go outshines Python in this specific setup

Go advantages (for this use case)

Python advantages (in general)

The trade-off

Project layout

The Lambda code (same logic, two languages)

1) Python: lambda-py/app.py

2) Go: lambda-go/main.go

Terraform: deploy both functions the same way

infra/versions.tf

infra/main.tf

infra/outputs.tf

Build & deploy

1) Build the Go bootstrap for arm64

2) Terraform apply

Generate a payload and hit both endpoints

scripts/gen_payload.py

Quick benchmark (side-by-side)

Option A: hey

Option B: a tiny bench.sh

What to look for in the results

Actual Benchmark Results

Latency Summary (all times in milliseconds)

Go vs Python Speedup Factor

Key Observations

Cost Implications

When I'd choose Go vs Python for Lambda

I reach for Go when:

I reach for Python when:

Final notes and improvements

Additional Benchmark Scenarios to Try

1. Memory Configuration Comparison

2. Concurrent Request Simulation

3. I/O-Bound Workload Comparison

4. Package Size Impact

5. Provisioned Concurrency Test

Real-World Use Cases Where These Results Matter

✅ Good fit for Go Lambda:

✅ Good fit for Python Lambda:

Cleanup

Reproducibility

Summary

Navigating AWS EKS with Terraform: Configuring Karpenter for Just-in-Time Node Provisioning

Prerequisites

Setup Overview

Key Architectural Difference: Self-Managed Nodes

Prepare Your EKS Cluster

Modifications

Update EKS Module to Support New Parameters

Update VPC and EKS Configuration

Create IAM Role for Karpenter

Install Karpenter using Helm

Deploy Applications To Trigger Scale Up Operation

Scale Down Operation

Understanding Karpenter's Advantages

Faster Provisioning

No Pre-defined Node Groups

Better Bin Packing

Faster Scale Down

Consolidation

Spot Instance Support

Comparison: Cluster Autoscaler vs Karpenter

Troubleshooting Common Issues

Issue: Nodes Stuck in NotReady State

Issue: Capacity-Block Warnings in Logs

Issue: Why Are My Nodes "Self-Managed" in AWS Console?

Conclusion

Navigating AWS EKS with Terraform: Implementing Cluster Auoscaler in your EKS Cluster

Prerequisites

Setup Overview

Prepare Your EKS Cluster

Modifications

Create IAM Role for Cluster Autoscaler

Install Cluster Autoscaler using Helm

Deploy Applications To Trigger Scale Up Operation

Scale Down Operation

1) Python: `lambda-py/app.py`

2) Go: `lambda-go/main.go`

`infra/versions.tf`

`infra/main.tf`

`infra/outputs.tf`

1) Build the Go `bootstrap` for arm64

`scripts/gen_payload.py`

Option A: `hey`

Option B: a tiny `bench.sh`