This article was originally published at sivaro.in

How to Build Cost Efficient Kubernetes Cluster

And why your $47K/month EKS bill is mostly your own fault.

Last March, a Series B fintech I advise came to me with a problem. Their Kubernetes footprint on AWS had grown to $47,000 a month. Twenty-two nodes. Maybe 30% average utilization across the fleet. Their head of platform asked me if they should switch to GKE.

I told them no. Switching clouds wasn't their problem. Their problem was that they were paying for idle compute with premium networking attached, and they'd never once looked at their request/limit ratios.

Ninety days later: same workloads, same cluster topology patterns, $19,400/month. That's the thing about how to build cost efficient kubernetes cluster architectures — the savings rarely come from the shiny vendor swap. They come from a dozen unglamorous decisions made in the right order.

This guide is the comparison table I wish someone had handed me in 2019, before I spent a year learning it the expensive way. We'll compare managed vs. self-hosted, spot vs. on-demand vs. reserved, autoscaling strategies, bin-packing engines, and the observability stack that tells you which of those actually moved your bill.

Grab coffee. This is long.

The Three Cost Layers You're Actually Paying For

Most teams think of Kubernetes cost as "node hours." That's about 60% of the truth.

You're paying for:

Compute layer. Node hours, spot interruptions, capacity reservations. This is the visible bill.

Control plane + network layer. EKS charges $0.10/hour per cluster as of their current pricing, GKE has a free tier waiver, AKS standard tier is $0.10/hour. Then NAT gateway charges — and this is where teams get ambushed. A single NAT gateway in us-east-1 runs roughly $32/month base plus $0.045/GB processed. I've seen clusters with a $900/month NAT bill and a $4,000 compute bill.

The hidden layer. Idle namespaces, zombie PersistentVolumes, orphaned load balancers, and containers that request 2 CPU but use 40m. This layer doesn't show up on any single invoice. It shows up as "why is our bill 3x our Grafana utilization graph."

You can't build a cost efficient cluster until you can attribute spend to all three. Start with Kubecost or OpenCost (the CNCF sandbox project). Free, self-hosted, and it labels every pod with a dollar figure. Ran it on a 60-node cluster in 2024 — took 20 minutes to deploy, surfaced $6,200/month of pure waste in the first week.

Managed Kubernetes vs. Self-Hosted: The Real Math

The dominant argument is "managed is worth the control plane fee." Sometimes true. Often false.

Here's the comparison, and I'm going to be blunt about where each option wins.

Factor	EKS / GKE / AKS	Self-hosted (kubeadm, k3s, Talos)
Control plane cost	$70–$150/mo per cluster	$0 (you run it on worker nodes)
Upgrade burden	Vendor-driven, still your testing	Fully yours
Networking (CNI)	Vendor default, sometimes slow	Pick Cilium, save 10–15% CPU
Node lifecycle	Managed node groups, still your config	You own everything
Multi-cloud portability	Locked to vendor APIs	Portable
Saves money when…	Small teams, <10 clusters	You have platform engineers

I ran a 40-node self-hosted k3s cluster on Hetzner in 2025 that did the job of a $9K/month EKS setup for €2,400/month. But it required a full-time platform engineer to keep it healthy. That's $12K/month of salary. The "cheaper" option wasn't cheaper.

My rule: if you don't have a dedicated platform engineer, go managed. If you do, self-host only the workloads that justify the operational overhead.

Instance Selection: Where 70% of Your Savings Hide

Most people pick instance families because the console defaults to them. Don't do that.

Spot, On-Demand, and Reserved: A Portfolio Approach

Last I checked, spot instances on AWS typically price 60–90% below on-demand. GCP's spot VMs land in a similar range. That's not a rounding error — that's the whole game for stateless workloads.

The mistake I see constantly: teams run everything on on-demand because "spot is risky." It's only risky if you don't design for it.

# spot-node-pool.yaml
apiVersion: v1
kind: Node
metadata:
  labels:
    node.kubernetes.io/lifecycle: spot
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: batch-processor
spec:
  replicas: 20
  template:
    spec:
      nodeSelector:
        node.kubernetes.io/lifecycle: spot
      tolerations:
      - key: "sku"
        operator: "Equal"
        value: "spot"
        effect: "NoSchedule"
      terminationGracePeriodSeconds: 30
      containers:
      - name: worker
        image: myapp/worker:1.19.2
        resources:
          requests:
            cpu: 500m
            memory: 512Mi

Then run on-demand for the control-plane-adjacent stuff, and reserved instances for the baseline you know you'll need every hour of every day.

My typical portfolio split:

• Spot: 55–65% of compute
• Reserved (1-year, no upfront): 25–35% for baseline
• On-demand: remaining 10% for burst and critical

That split cuts the compute bill by roughly 45–55% versus all on-demand. I've verified this across three client clusters in 2025 and 2026.

Instance Family Doesn't Matter as Much as You Think

The ARM vs. x86 debate is real but overblown. Graviton (AWS) and Tau T2A/Axion (GCP) deliver 15–40% better price-performance on average for compatible workloads. But if your team isn't building multi-arch images, the migration cost eats the savings.

What actually matters: avoid the t (burstable) families for production Kubernetes. They throttle under sustained load, which means your autoscaler reacts to throttling instead of real traffic, and you end up with 40% more nodes than you need. I've seen this exact mistake cost teams five figures monthly.

Autoscaling: The Layer Most Teams Get Half Right

There are three autoscalers you likely need, and they aren't interchangeable.

Cluster Autoscaler vs. Karpenter vs. KEDA

Cluster Autoscaler — mature, works, slow (1–3 minutes to scale up). Fine for predictable workloads.

Karpenter — AWS-native, provisioner-based, scales in seconds, picks optimal instance types automatically. If you're on EKS in 2026 and not using Karpenter, you're paying a tax. Consolidated nodes alone typically shave 20–30% off compute.

A Karpenter provisioner that lets the scheduler pick from many instance types:

apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: general
spec:
  template:
    spec:
      requirements:
      - key: karpenter.sh/capacity-type
        operator: In
        values: ["spot", "on-demand"]
      - key: kubernetes.io/arch
        operator: In
        values: ["amd64", "arm64"]
      - key: karpenter.k8s.aws/instance-category
        operator: In
        values: ["c", "m", "r"]
      - key: karpenter.k8s.aws/instance-generation
        operator: Gt
        values: ["5"]
      disruption:
        consolidationPolicy: WhenEmptyOrUnderutilized
        consolidateAfter: 30s
  limits:
    cpu: 1000
    memory: 2000Gi

KEDA — event-driven pod autoscaling. Scale to zero on idle queues. If you're running workers that poll SQS/Kafka/RabbitMQ and you aren't using KEDA, you're paying for pods that do nothing 80% of the time.

The combination matters. Karpenter handles nodes. HPA/KEDA handles pods. If you only tune one, you leave 30%+ on the table.

The Requests/Limits Conversation

This is where I get preachy. Most teams set CPU requests to what they think the pod needs, then never revisit it.

Real example from a client in January: their API service requested 800m CPU. Actual p95 usage was 110m. Across 40 pods, they were reserving enough CPU to run 20 additional nodes' worth of workload that didn't exist.

# Use this to find your worst offenders
kubectl top pods -A --sort-by=cpu | head -30

Then compare against requests:

kubectl get pods -A -o json | jq -r '
  .items[] | 
  "\(.metadata.namespace)/\(.metadata.name) " +
  "req=\(.spec.containers[0].resources.requests.cpu // "none")"
'

The VPA (Vertical Pod Autoscaler) in recommendation-only mode does this for you. Deploy it, let it observe for a week, then apply the recommendations carefully — they tend to be aggressive.

Bin-Packing: The Underused Lever

Default Kubernetes scheduler is a first-fit algorithm. It doesn't pack tightly. That's fine — it's optimized for availability, not cost.

But you can shift the balance. Taints, affinity rules, and topology spread constraints all tell the scheduler where to be efficient. And projects like Trimaran provide load-aware scheduling that factors actual utilization into placement.

For most teams, this is a 5–15% win, not a 50% win. Worth doing after you've done everything else.

Storage and Networking: The Silent Budget Killers

I mentioned NAT earlier. Here's the deeper issue: Kubernetes defaults to over-networked.

Load balancers. Every Service of type LoadBalancer creates a cloud LB. At $16–$22/month each, a cluster with 30 of them is spending $600/month on LBs alone. Use an Ingress controller (Traefik, NGINX, or Gateway API implementations) and a single ALB. $20/month instead of $600.

Persistent volumes. GP3 (AWS) is cheaper and faster than GP2 for most workloads. But orphaned PVs are the real problem — a PVC that outlives its pod keeps billing forever. Write a CronJob to flag unattached volumes older than 30 days.

apiVersion: batch/v1
kind: CronJob
metadata:
  name: orphaned-pv-reporter
spec:
  schedule: "0 9 * * 1"
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: reporter
            image: bitnami/kubectl:1.31
            command:
            - /bin/sh
            - -c
            - |
              kubectl get pv -o json | jq -r '
                .items[] | select(.status.phase=="Available") |
                "\(.metadata.name) \(.spec.capacity.storage)"
              '
          restartPolicy: OnFailure

Cross-AZ traffic. If your nodes are spread across three AZs and pods talk between them, you pay $0.01/GB in each direction. This adds up faster than you'd guess in high-throughput clusters. Zonal affinity for chummy services — Cilium's topology-aware routing — can cut this substantially.

A Tiered Decision Framework

Here's how I'd think about it if I were starting from zero today.

If you're under 20 nodes:
Go managed (EKS/GKE/AKS). Enable Karpenter or GKE Autopilot. Deploy OpenCost. Tune requests with VPA recommendations. You're done. This gets you to roughly 55–65% of your "naive" bill.

If you're 20–100 nodes:
Managed plus spot portfolio plus KEDA for event-driven workloads. Trimaran for scheduler efficiency. Ingress consolidation. Multi-arch images to unlock Graviton/Tau. You should get to 40–50% of naive.

If you're 100+ nodes:
Consider self-hosting the control plane if you have the team. Investigate dedicated tenancy or committed use discounts at a scale that actually moves. Cilium with kube-proxy replacement for CPU savings. Real capacity planning. This can land at 30–40% of naive spend, but the operational cost is real.

Nobody should do all three at once. I've watched teams try. They break production and go back to on-demand everything.

FAQ

How much can I realistically save on my Kubernetes bill?

40–65% is the common range, done over 6–9 months. Anything promising 80%+ is either lying or describing a workload you don't run.

Is spot safe for production?

Safe for stateless, retry-tolerant workloads. Not safe for stateful databases without operator-level failover, and definitely not safe for anything with a single replica. Design for graceful shutdown (30–60 second termination grace periods), spread replicas across capacity types, and use PodDisruptionBudgets.

Do I need Karpenter if I'm on GKE or AKS?

No — GKE's cluster autoscaler and AKS's node autoprovisioning (NAP) play similar roles. Karpenter's biggest wins are on EKS, though it now supports Azure in preview.

What's the single biggest mistake teams make?

Setting CPU requests to peak usage instead of p95. It cascades into oversized nodes, oversized autoscaling, and idle capacity reserves. Fix this before anything else.

How do I convince my CFO to fund the migration effort?

Bring a Kubecost report to the meeting. Showing them $22K/month of identified waste with a 90-day remediation plan is more persuasive than any architecture diagram.

Is ARM (Graviton) worth the migration?

Yes, if your images are already multi-arch or you can flip them cheaply. If you're maintaining 40 hand-crafted Dockerfiles, the migration cost may exceed the savings. Measure both.

What about serverless Kubernetes (Fargate, GKE Autopilot, ACI)?

Great for spiky, unpredictable workloads. Terrible for steady-state compute — the per-vCPU-second pricing is often 2–3x the equivalent node cost. Use it for burst, not baseline.

Should I use a cost-optimization SaaS product?

Only if you've already deployed OpenCost and hit its ceiling. Most teams haven't. The SaaS layer adds context (allocation by team, chargeback reports) — it doesn't find waste you couldn't find yourself.

The Order of Operations That Actually Works

If you take one thing from this, take the sequence.

First, observe. Deploy OpenCost. Get labels on everything. Understand where every dollar goes, including the hidden ones.

Second, tune requests and limits. This is free money and it's the highest-leverage change.

Third, fix the floor. Convert baseline workload to reserved instances, burst to spot, and eliminate the on-demand-by-default.

Fourth, add Karpenter (or equivalent) for consolidation. This is where the node sprawl dies.

Fifth, attack networking and storage. Ingress consolidation, PV cleanup, topology-aware routing.

Sixth, only then consider architecture changes — ARM migration, self-hosting, serverless for burst.

The teams that do this in order cut 50%+ from their bill in a quarter. The teams that do it out of order break production, blame Kubernetes, and pay more next quarter.

How to build cost efficient kubernetes cluster isn't a secret knowledge problem. It's a discipline problem. The tools have existed since roughly 2023. Almost nobody strings them together in the right sequence. That's your edge.

---

Nishaant Dixit — Founder of SIVARO. Building data infrastructure and production AI systems since 2018. Built systems processing 200K events/sec.