DEV Community

Cover image for The AWS Charge Silently Eating Your K8s Budget: NAT Gateway Explained
KloudAudit
KloudAudit

Posted on

The AWS Charge Silently Eating Your K8s Budget: NAT Gateway Explained

#kubernetes #finops #aws #devops

You get your AWS bill. EC2 looks reasonable. RDS looks fine. Then there's a line item called "NAT Gateway" sitting at $800 and you have no idea why.

This is the charge that catches almost every team running Kubernetes on AWS. Here is exactly what it is, why it grows silently, and how to fix it in 20 minutes.

What NAT Gateway Actually Does

A NAT Gateway lets resources in your private subnets reach the internet without exposing them directly. Every byte of traffic that flows through it costs $0.045 per GB — on top of the hourly charge of $0.045 per hour ($32/month just for existing).

For small workloads that's negligible. For a K8s cluster with 20+ pods constantly pulling images, sending logs, and calling external APIs, it compounds fast.

Why K8s Makes It Worse

Three patterns specific to Kubernetes that silently inflate NAT Gateway costs:

1. ECR image pulls routing through NAT

Every time a node pulls a container image from ECR, that traffic goes through NAT Gateway by default. A cluster that scales frequently — pulling images on new nodes — can generate hundreds of GB of NAT traffic per month just from image pulls.

Fix: Create a VPC endpoint for ECR. Traffic stays inside AWS and costs nothing.

aws ec2 create-vpc-endpoint \
  --vpc-id vpc-xxxxxxxx \
  --service-name com.amazonaws.eu-west-1.ecr.dkr \
  --vpc-endpoint-type Interface \
  --subnet-ids subnet-xxxxxxxx \
  --security-group-ids sg-xxxxxxxx
Enter fullscreen mode Exit fullscreen mode

2. Cross-AZ pod traffic

When a pod in eu-west-1a calls a service whose pod is scheduled in eu-west-1b, that traffic crosses availability zones. Each GB costs $0.01 in data transfer. At scale this adds up fast.

Fix: Use topology-aware routing to prefer same-AZ endpoints:

apiVersion: v1
kind: Service
metadata:
  name: your-service
  annotations:
    service.kubernetes.io/topology-mode: Auto
spec:
  ...
Enter fullscreen mode Exit fullscreen mode

3. S3 traffic routing through NAT

If your pods are reading or writing to S3 without a VPC endpoint, every byte goes through NAT Gateway. At $0.045/GB this destroys any S3 cost savings from storage tiering.

Fix: Create a VPC Gateway endpoint for S3 — it is free:

aws ec2 create-vpc-endpoint \
  --vpc-id vpc-xxxxxxxx \
  --service-name com.amazonaws.eu-west-1.s3 \
  --vpc-endpoint-type Gateway \
  --route-table-ids rtb-xxxxxxxx
Enter fullscreen mode Exit fullscreen mode

How Much Are You Actually Paying

Check your current NAT Gateway spend:

aws ce get-cost-and-usage \
  --time-period Start=2026-05-01,End=2026-05-31 \
  --granularity MONTHLY \
  --metrics BlendedCost \
  --filter '{"Dimensions":{"Key":"SERVICE","Values":["Amazon Virtual Private Cloud"]}}' \
  --group-by Type=DIMENSION,Key=USAGE_TYPE \
  --output table
Enter fullscreen mode Exit fullscreen mode

Look for lines containing NatGateway-Bytes and NatGateway-Hours. That is your total NAT cost split by data processed and hourly charges.

The Fix Priority Order

Fix Effort Typical saving
S3 VPC Gateway endpoint 5 minutes 20-40% of NAT cost
ECR VPC Interface endpoint 15 minutes 30-50% of NAT cost
Topology-aware routing 30 minutes 10-20% of NAT cost
Review remaining traffic Ongoing Varies

Start with S3 — it is free, takes 5 minutes, and has zero risk. Most teams see immediate impact on their next bill.

What a Real Fix Looks Like

A team running 15 microservices on EKS, spending $1,200/month on NAT Gateway:

  • Added S3 VPC Gateway endpoint: saved $380/month
  • Added ECR VPC Interface endpoint: saved $290/month
  • Enabled topology-aware routing: saved $140/month

Total: $810/month recovered. 50 minutes of work.

The ECR endpoint costs $7.30/month for the interface endpoint itself. The net saving was still $283/month after that cost.

Check All 18 Cost Patterns at Once

NAT Gateway is one of 18 checks that commonly hide recoverable AWS spend. If you want a systematic view of everything — EBS volumes, RDS scheduling, Reserved Instances, security misconfigurations — run the free audit at kloudaudit.eu

No AWS credentials. No signup. 15 minutes.

Samuel Ayodele Adomeh is a Senior DevOps Engineer and Azure Solutions Architect based in Wrocław, Poland. He built KloudAudit after seven years of reviewing cloud bills and seeing the same waste patterns on every infrastructure he worked with.

Top comments (0)