DEV Community

Sanket Patharkar
Sanket Patharkar

Posted on

Real-World AWS Cost Optimization Strategies — A Technical Deep Dive

In a cloud-first world, keeping costs under control without losing performance, scalability or reliability matters for any business that wants to last. AWS gives you a lot of tools for measuring, managing and optimizing spend. The hard part is wiring them into an engineering system instead of doing a clean-up once a quarter.

I'm a Senior Cloud Operations Engineer with 6+ years across AWS, DevOps, CI/CD and enterprise cloud architecture, much of it in the automotive systems domain. The approaches below mix automation, governance and real-time monitoring, and each comes with the CLI commands, JSON/YAML configs and gotchas you need to use it in production.

Who this is for: Cloud Architects, DevOps/SRE engineers and FinOps practitioners who want measurable, automated, production-ready cost control, not just a checklist.

About the prices: Dollar amounts below are illustrative us-east-1 list prices at the time of writing. They vary by Region and change over time. Discount figures such as “up to 72%” or “up to 90%” are what AWS advertises as a maximum, not a saving you should expect on every workload. Check the current AWS pricing pages before you build a business case on any of them.


TL;DR — The Eight Levers

# Strategy Key AWS services Where the savings come from
1 Measure before you optimize Data Exports (CUR 2.0), Athena, Cost Explorer, Budgets, Cost Anomaly Detection Knowing who spends what, and finding spikes within hours instead of at month-end
2 Right-size compute Compute Optimizer, CloudWatch Agent, Auto Scaling, Graviton Removing idle headroom; ~20% lower price per instance on Graviton
3 Commitments + Spot Savings Plans, Reserved Instances, EC2 Fleet, Karpenter AWS advertises up to 72% (Savings Plans) and up to 90% (Spot) off On-Demand
4 Storage tiering S3 Lifecycle, Intelligent-Tiering, Storage Lens, EBS gp3, EFS IA About 20% on gp2→gp3 in many Regions; AWS advertises up to ~95% for cold archive tiers
5 Network cost engineering VPC Gateway/Interface Endpoints, CloudFront, AZ-aware routing Avoiding NAT processing and cross-AZ charges
6 Non-prod shutdown EventBridge Scheduler, Lambda, Instance Scheduler on AWS 60–70% of non-prod compute hours
7 Database optimization Aurora Serverless v2, Aurora I/O-Optimized, DynamoDB modes, RDS on Graviton Paying for actual load instead of peak provisioning
8 Continuous governance Organizations SCPs, Tag Policies, AWS Config, Budgets Actions Stopping expensive mistakes before they reach the bill

The Big Picture — A Closed FinOps Loop

Cost optimization isn't a project. It's a control loop: measure, detect, act, govern, then measure again. Every strategy in this article plugs into one of those four stages.

Figure: AWS cost optimization reference architecture — Measure, Detect & Alert, Act/Automate, and Govern.

  • Measure: Billing and usage data (CUR 2.0) lands in S3, is queried with Athena and visualized in QuickSight/Cost Explorer. Tags turn line items into team-level accountability.
  • Detect & Alert: Anomaly Detection, Budgets, Compute Optimizer and Cost Optimization Hub produce signals. SNS sends them to Slack/Teams/email.
  • Act / Automate: Schedulers, Lambda remediations, SSM runbooks and Karpenter make the change, ideally without a human opening the console.
  • Govern: SCPs, Tag Policies and Config rules stop the waste from coming back.

Production note: The examples below are reference implementations. Validate pricing, service availability, quotas, IAM permissions, and regional behavior against your AWS environment before applying them to production.

1. Measure Before You Optimize

You can't optimize what you can't see. Treat cost like any other telemetry signal: granular, tagged and close to real time.

1.1 Set up CUR 2.0 (Data Exports) + Athena

Cost Explorer is great for exploring, but a Cost and Usage Report (CUR 2.0) export, delivered through Billing and Cost Management → Data Exports, is the practical dataset for line-item analysis. What it contains depends on the export you configure. Hourly granularity and resource IDs are settings you turn on. They are not a promise that every charge has a resource identifier: some fees, credits, and tax lines do not. Treat the export as the working record for the charges it is configured to include, and confirm the columns before you build chargeback on it.

Recommended export settings:

  • Export type: Standard data export → CUR 2.0
  • Include resource IDs: ✅ (you can't chase waste without them)
  • Time granularity: Hourly
  • Format: Parquet, overwrite, delivered to a dedicated S3 bucket
  • Use the Athena/Glue integration (or a Glue crawler) to create the table

Query 1: top spend by service and environment

-- Partition/column names depend on how your Glue table was created
SELECT
  line_item_product_code                    AS service,
  resource_tags['user_environment']         AS environment,
  ROUND(SUM(line_item_unblended_cost), 2)   AS cost_usd
FROM cur2
WHERE billing_period = '2026-08'
GROUP BY 1, 2
ORDER BY cost_usd DESC
LIMIT 20;
Enter fullscreen mode Exit fullscreen mode

Query 2: untagged spend (your accountability gap)

SELECT
  line_item_product_code AS service,
  ROUND(SUM(line_item_unblended_cost), 2) AS untagged_cost
FROM cur2
WHERE billing_period = '2026-08'
  AND line_item_line_item_type = 'Usage'
  AND (resource_tags['user_owner'] IS NULL OR resource_tags['user_owner'] = '')
GROUP BY 1
ORDER BY 2 DESC;
Enter fullscreen mode Exit fullscreen mode

Query 3: the silent killers, NAT processing and cross-AZ transfer

SELECT
  line_item_usage_type,
  line_item_resource_id,
  ROUND(SUM(line_item_usage_amount), 0)  AS gb,
  ROUND(SUM(line_item_unblended_cost), 2) AS cost_usd
FROM cur2
WHERE billing_period = '2026-08'
  AND (line_item_usage_type LIKE '%NatGateway-Bytes%'
       OR line_item_usage_type LIKE '%DataTransfer-Regional-Bytes%')
GROUP BY 1, 2
ORDER BY cost_usd DESC
LIMIT 25;
Enter fullscreen mode Exit fullscreen mode

💡 Insight: unblended vs amortized. If you've bought Savings Plans or RIs, unblended cost puts all the upfront or recurring fees on one line and makes teams look "free". For chargeback, use amortized cost (savings_plan_savings_plan_effective_cost and reservation_effective_cost in CUR), so every team carries its fair share of the commitment.

1.2 Activate cost allocation tags

Tags only show up in billing data after you activate them in the payer account:

aws ce update-cost-allocation-tags-status \
  --cost-allocation-tags-status TagKey=Environment,Status=Active TagKey=Project,Status=Active TagKey=Owner,Status=Active
Enter fullscreen mode Exit fullscreen mode

Required baseline: Environment, Project, Owner (plus CostCenter if finance charges back). Section 8 covers how to enforce them.

1.3 Cost Explorer API: query programmatically, but know it costs money

aws ce get-cost-and-usage \
  --time-period Start=2026-08-01,End=2026-09-01 \
  --granularity DAILY \
  --metrics UnblendedCost \
  --group-by Type=TAG,Key=Environment
Enter fullscreen mode Exit fullscreen mode

⚠️ Gotcha: AWS currently charges $0.01 per paginated Cost Explorer API request against the primary billing view. A custom billing view is charged at $0.01 per source in that view. The Cost Explorer console is free. A dashboard polling the API every minute runs up a real bill. Hourly granularity is a separate opt-in charge and only covers the last 14 days. Confirm the current rate on the Cost Explorer pricing page before you automate it. For heavy analysis, query CUR in Athena instead.

1.4 Budgets with forecast alerts

Alert on forecasted spend, not just actual. By the time actual spend crosses 100%, the money is already gone.

budget.json

{
  "BudgetName": "platform-monthly",
  "BudgetLimit": { "Amount": "10000", "Unit": "USD" },
  "TimeUnit": "MONTHLY",
  "BudgetType": "COST",
  "CostFilters": { "TagKeyValue": ["user:Project$platform"] }
}
Enter fullscreen mode Exit fullscreen mode

notifications.json

[
  {
    "Notification": { "NotificationType": "FORECASTED", "ComparisonOperator": "GREATER_THAN", "Threshold": 100, "ThresholdType": "PERCENTAGE" },
    "Subscribers": [{ "SubscriptionType": "SNS", "Address": "arn:aws:sns:us-east-1:111122223333:finops-alerts" }]
  },
  {
    "Notification": { "NotificationType": "ACTUAL", "ComparisonOperator": "GREATER_THAN", "Threshold": 80, "ThresholdType": "PERCENTAGE" },
    "Subscribers": [{ "SubscriptionType": "SNS", "Address": "arn:aws:sns:us-east-1:111122223333:finops-alerts" }]
  }
]
Enter fullscreen mode Exit fullscreen mode
aws budgets create-budget \
  --account-id 111122223333 \
  --budget file://budget.json \
  --notifications-with-subscribers file://notifications.json
Enter fullscreen mode Exit fullscreen mode

Send the SNS topic to Slack or Teams through Amazon Q Developer in chat applications (formerly AWS Chatbot).

1.5 Cost Anomaly Detection

A budget catches how much. Anomaly Detection catches something changed, like a runaway Lambda loop or a debug log level left on in production.

aws ce create-anomaly-monitor --anomaly-monitor '{
  "MonitorName": "per-service",
  "MonitorType": "DIMENSIONAL",
  "MonitorDimension": "SERVICE"
}'

aws ce create-anomaly-subscription --anomaly-subscription '{
  "SubscriptionName": "anomalies-over-100-usd",
  "MonitorArnList": ["arn:aws:ce::111122223333:anomalymonitor/abcd-1234"],
  "Subscribers": [{ "Type": "SNS", "Address": "arn:aws:sns:us-east-1:111122223333:finops-alerts" }],
  "Frequency": "IMMEDIATE",
  "ThresholdExpression": {
    "Dimensions": { "Key": "ANOMALY_TOTAL_IMPACT_ABSOLUTE", "Values": ["100"], "MatchOptions": ["GREATER_THAN_OR_EQUAL"] }
  }
}'
Enter fullscreen mode Exit fullscreen mode

💡 Insight: track unit cost, not just total cost. A rising bill can be healthy growth. Divide spend by a business metric (cost per 1,000 API requests, per tenant, per CI build) and put it on the same dashboard as your latency SLOs. When unit cost goes up, that's a regression.


2. Right-Sizing Compute Resources

Right-sizing is more than CPU. You also have to look at memory, network bandwidth, EBS throughput and CPU credit behaviour against real performance baselines.

2.1 Feed Compute Optimizer the metric it can't see: memory

By default, EC2 doesn't publish memory metrics. Without them, Compute Optimizer can't safely recommend moving to a smaller or memory-optimized family. Install the CloudWatch Agent:

{
  "metrics": {
    "append_dimensions": { "InstanceId": "${aws:InstanceId}" },
    "metrics_collected": {
      "cpu":  { "measurement": ["cpu_usage_active"], "totalcpu": true, "resources": ["*"] },
      "mem":  { "measurement": ["mem_used_percent"] },
      "disk": { "measurement": ["used_percent"], "resources": ["/"] }
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

Compute Optimizer picks up mem_used_percent from the CWAgent namespace automatically. Then pull the over-provisioned list:

aws compute-optimizer get-ec2-instance-recommendations \
  --filters name=Finding,values=Overprovisioned \
  --query 'instanceRecommendations[].{id:instanceArn,current:currentInstanceType,best:recommendationOptions[0].instanceType,risk:recommendationOptions[0].performanceRisk}' \
  --output table
Enter fullscreen mode Exit fullscreen mode

💡 Insight: size on p95/p99, not averages. A 12% average CPU can hide a 90% p99 spike during batch windows. Per-core metrics (resources: ["*"]) also show single-threaded apps pinning one core while the others sit idle. Those need a faster core, not more cores. Turn on enhanced infrastructure metrics in Compute Optimizer for a 3-month lookback on seasonal workloads.

⚠️ Gotcha: burstable (T-family) instances. t3/t4g default to unlimited mode. A T instance that burns through its CPU credits all day gets billed for surplus credits and can cost more than an m-family instance. Watch CPUSurplusCreditsCharged.

2.2 Graviton migration checklist

Graviton (arm64) instances usually cost ~20% less than comparable x86 instances, and AWS quotes up to 40% better price-performance.

  1. Check compatibility: Interpreted and JVM languages (Java, Python, Node.js, Go, .NET 6+) usually just work. Check native dependencies and agents.
  2. Build multi-arch images:
   docker buildx build --platform linux/amd64,linux/arm64 \
     -t 111122223333.dkr.ecr.us-east-1.amazonaws.com/api:1.4.0 --push .
Enter fullscreen mode Exit fullscreen mode
  1. Canary: Put a Graviton ASG or node pool behind the same ALB target group and shift traffic gradually while watching latency and error rates.
  2. Go beyond EC2: Lambda on arm64 is ~20% cheaper per GB-second. RDS/Aurora (db.r7g, db.m7g), ElastiCache and OpenSearch all offer Graviton classes. These are often the easiest wins because there's no code to change.

2.3 Scale on the right signal

aws autoscaling put-scaling-policy \
  --auto-scaling-group-name web-asg \
  --policy-name cpu-target-50 \
  --policy-type TargetTrackingScaling \
  --target-tracking-configuration '{
    "PredefinedMetricSpecification": { "PredefinedMetricType": "ASGAverageCPUUtilization" },
    "TargetValue": 50.0
  }'
Enter fullscreen mode Exit fullscreen mode
  • For web tiers, ALBRequestCountPerTarget tracks real demand more closely than CPU.
  • Add predictive scaling for workloads with a daily pattern so capacity arrives before the morning ramp instead of sitting there all night.
  • Use attribute-based instance selection (InstanceRequirements) so the ASG isn't tied to one instance type. That matters again in section 3.

3. Savings Plans & Spot Fleet Engineering

The biggest single lever on compute is how you buy it. The rule is simple: layer your capacity.

Figure: Compute purchasing strategy — Savings Plans for the baseline, Spot for variable demand, On-Demand for short peaks.

3.1 Savings Plans: commit to the floor, not the average

Plan Max discount Flexibility Best for
Compute Savings Plans up to 66% Any family, size, Region, OS; also covers Fargate & Lambda Most organizations; containers and serverless
EC2 Instance Savings Plans up to 72% Locked to a family in a Region (size/OS flexible) Very stable, known fleets
Reserved Instances (RDS, ElastiCache, OpenSearch, Redshift) varies Per service Databases and data stores

How to size the commitment:

  1. Pull the recommendation from AWS as a starting point:
   aws ce get-savings-plans-purchase-recommendation \
     --savings-plans-type COMPUTE_SP \
     --term-in-years ONE_YEAR \
     --payment-option NO_UPFRONT \
     --lookback-period-in-days THIRTY_DAYS
Enter fullscreen mode Exit fullscreen mode
  1. Commit to roughly 90–95% of your lowest sustained hourly usage, after right-sizing. Committing first and right-sizing afterwards leaves you paying for a commitment you no longer use.
  2. Ladder your purchases, for example a small tranche every quarter. Commitments then expire on a rolling basis, and you're never locked into one big bet.
  3. Track utilization (are you using what you bought?) and coverage (how much of your eligible spend is covered?). These are Cost Explorer APIs, ce:GetSavingsPlansUtilization and ce:GetSavingsPlansCoverage:
   aws ce get-savings-plans-utilization --time-period Start=2026-08-01,End=2026-09-01
   aws ce get-savings-plans-coverage   --time-period Start=2026-08-01,End=2026-09-01 --granularity MONTHLY
Enter fullscreen mode Exit fullscreen mode

🎯 Targets I use: utilization > 95%, coverage of eligible compute 70–85%. Coverage near 100% usually means you over-committed and have no room left for Spot.

3.2 Spot: engineering for interruption

AWS advertises Spot discounts of up to 90% versus On-Demand. In exchange, the instance can be reclaimed with a 2-minute notice. Three things make it production-safe.

a) Diversify and use price-capacity-optimized. It's the recommended allocation strategy. It picks the deepest Spot pools and weighs price, so you get fewer interruptions than with lowest-price and better prices than with pure capacity-optimized.

mixed-instances.json (ASG):

{
  "LaunchTemplate": {
    "LaunchTemplateSpecification": { "LaunchTemplateName": "web-lt", "Version": "$Latest" },
    "Overrides": [
      {
        "InstanceRequirements": {
          "VCpuCount": { "Min": 2, "Max": 8 },
          "MemoryMiB": { "Min": 4096 },
          "CpuManufacturers": ["intel", "amd"],
          "InstanceGenerations": ["current"]
        }
      }
    ]
  },
  "InstancesDistribution": {
    "OnDemandBaseCapacity": 2,
    "OnDemandPercentageAboveBaseCapacity": 20,
    "SpotAllocationStrategy": "price-capacity-optimized"
  }
}
Enter fullscreen mode Exit fullscreen mode
aws autoscaling create-auto-scaling-group \
  --auto-scaling-group-name web-asg \
  --mixed-instances-policy file://mixed-instances.json \
  --min-size 2 --max-size 40 \
  --vpc-zone-identifier "subnet-aaa,subnet-bbb,subnet-ccc" \
  --capacity-rebalance
Enter fullscreen mode Exit fullscreen mode

--capacity-rebalance launches a replacement as soon as AWS sends a rebalance recommendation, which often comes before the 2-minute notice.

b) Karpenter for EKS. Karpenter provisions nodes directly from pod requirements, prefers Spot when you allow it, and consolidates under-utilized nodes all the time.

apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: general
spec:
  template:
    spec:
      requirements:
        - key: karpenter.sh/capacity-type
          operator: In
          values: ["spot", "on-demand"]
        - key: kubernetes.io/arch
          operator: In
          values: ["arm64", "amd64"]       # multi-arch images required
        - key: karpenter.k8s.aws/instance-category
          operator: In
          values: ["c", "m", "r"]
        - key: karpenter.k8s.aws/instance-generation
          operator: Gt
          values: ["5"]
      nodeClassRef:
        group: karpenter.k8s.aws
        kind: EC2NodeClass
        name: default
      expireAfter: 720h
  disruption:
    consolidationPolicy: WhenEmptyOrUnderutilized
    consolidateAfter: 1m
  limits:
    cpu: "1000"
Enter fullscreen mode Exit fullscreen mode

Point Karpenter at an SQS interruption queue (the settings.interruptionQueue Helm value) so it drains nodes gracefully on Spot interruption and rebalance events. If you run self-managed node groups without Karpenter, use the AWS Node Termination Handler.

c) Graceful shutdown on plain EC2. Poll instance metadata (IMDSv2) and drain in time:

#!/bin/bash
# /usr/local/bin/spot-watch.sh — run as a systemd service
while true; do
  TOKEN=$(curl -s -X PUT "http://169.254.169.254/latest/api/token" \
    -H "X-aws-ec2-metadata-token-ttl-seconds: 300")
  CODE=$(curl -s -o /dev/null -w "%{http_code}" \
    -H "X-aws-ec2-metadata-token: $TOKEN" \
    http://169.254.169.254/latest/meta-data/spot/instance-action)
  if [ "$CODE" -eq 200 ]; then
    logger "Spot interruption notice received - draining"
    systemctl stop myapp        # finish in-flight work, deregister, flush buffers
    break
  fi
  sleep 5
done
Enter fullscreen mode Exit fullscreen mode

💡 Insight: what should not run on Spot. Single-instance stateful services, long jobs without checkpoints, and anything with a license tied to a host. Everything else, including CI runners, batch/ETL, stateless APIs behind a load balancer and EKS workers with PodDisruptionBudgets, is a candidate.


4. Storage Cost Optimization

Storage costs grow quietly because nothing ever gets deleted.

4.1 S3: pick the class by access pattern

Storage class ~$/GB-month Min. duration Retrieval Use for
S3 Standard 0.023 — instant Hot data
S3 Intelligent-Tiering 0.023 → auto-tiered — instant (frequent/IA/archive-instant tiers) Unknown or changing access patterns
S3 Standard-IA 0.0125 30 days instant, per-GB fee Monthly-access data
S3 Glacier Instant Retrieval 0.004 90 days milliseconds Quarterly-access data, backups you might restore
S3 Glacier Flexible Retrieval 0.0036 90 days minutes–hours Archives
S3 Glacier Deep Archive 0.00099 180 days ~12–48 hours Compliance retention

A production-grade lifecycle policy. Note that Filter replaces the legacy top-level Prefix, and there's a separate "hygiene" rule:

{
  "Rules": [
    {
      "ID": "logs-tiering",
      "Status": "Enabled",
      "Filter": { "Prefix": "logs/" },
      "Transitions": [
        { "Days": 30,  "StorageClass": "STANDARD_IA" },
        { "Days": 90,  "StorageClass": "GLACIER_IR" },
        { "Days": 180, "StorageClass": "DEEP_ARCHIVE" }
      ],
      "Expiration": { "Days": 730 }
    },
    {
      "ID": "bucket-hygiene",
      "Status": "Enabled",
      "Filter": {},
      "AbortIncompleteMultipartUpload": { "DaysAfterInitiation": 7 },
      "NoncurrentVersionExpiration": { "NoncurrentDays": 30, "NewerNoncurrentVersions": 3 }
    }
  ]
}
Enter fullscreen mode Exit fullscreen mode
aws s3api put-bucket-lifecycle-configuration \
  --bucket my-app-logs \
  --lifecycle-configuration file://lifecycle.json
Enter fullscreen mode Exit fullscreen mode

⚠️ Gotchas that cost real money:

  • Incomplete multipart uploads are invisible in the console but billed. The AbortIncompleteMultipartUpload rule fixes that.
  • Versioned buckets keep every overwrite forever unless you expire noncurrent versions.
  • Small objects: by default, lifecycle doesn't transition objects under 128 KB. IA classes bill a 128 KB minimum, and Intelligent-Tiering doesn't auto-tier objects under 128 KB. Millions of tiny files belong in aggregated/compacted formats instead.
  • Transition requests cost money (per 1,000 objects). Moving 100M tiny objects to Glacier can cost more than it saves.

Use S3 Storage Lens (the free metrics are enough to start) to find buckets with no recent activity, large noncurrent-version bytes or incomplete multipart uploads. Use S3 Inventory for object-level audits.

4.2 EBS: gp2 → gp3 is often about a 20% storage saving

In many Regions, gp3 is about 20% cheaper per GB than gp2 and includes a 3,000 IOPS / 125 MiB/s baseline, independent of size. Confirm the current per-GB price before you migrate a large fleet. One catch: gp2 volumes above 1 TiB get more than 3,000 baseline IOPS (3 IOPS/GiB), so set IOPS explicitly or large volumes will lose performance:

#!/bin/bash
# Migrate all gp2 volumes to gp3, preserving the gp2 baseline IOPS (3 IOPS/GiB, capped at 16,000)
aws ec2 describe-volumes --filters Name=volume-type,Values=gp2 \
  --query 'Volumes[].[VolumeId,Size]' --output text |
while read -r VOL SIZE; do
  IOPS=$(( SIZE * 3 )); (( IOPS < 3000 )) && IOPS=3000; (( IOPS > 16000 )) && IOPS=16000
  echo "Migrating $VOL (${SIZE}GiB) -> gp3 @ ${IOPS} IOPS"
  aws ec2 modify-volume --volume-id "$VOL" --volume-type gp3 --iops "$IOPS" --throughput 250
done
Enter fullscreen mode Exit fullscreen mode

The migration is online, with no detach and no downtime. The same volume can't be modified again for 6 hours.

Hunt for orphans:

# Unattached EBS volumes
aws ec2 describe-volumes --filters Name=status,Values=available \
  --query 'Volumes[].{ID:VolumeId,GB:Size,Type:VolumeType,Created:CreateTime}' --output table

# Unassociated Elastic IPs (each public IPv4 address is billed hourly)
aws ec2 describe-addresses --query 'Addresses[?AssociationId==null].[PublicIp,AllocationId]' --output table
Enter fullscreen mode Exit fullscreen mode
  • Move old snapshots you must keep to the EBS Snapshots Archive tier (about 75% cheaper, 90-day minimum, 24–72h restore):
  aws ec2 modify-snapshot-tier --snapshot-id snap-0abc123 --storage-tier archive
Enter fullscreen mode Exit fullscreen mode
  • Automate snapshot retention with Amazon Data Lifecycle Manager or AWS Backup instead of hand-written cron jobs.

4.3 EFS and CloudWatch Logs, the ones people forget

  • EFS: Turn on lifecycle management to move cold files to Infrequent Access and Archive, with transition back to Standard on access for files that heat up again.
  • CloudWatch Logs: New log groups keep data forever by default, and ingestion (~$0.50/GB) is often a top-10 line item. Set retention on everything:
  for LG in $(aws logs describe-log-groups --query 'logGroups[?!retentionInDays].logGroupName' --output text); do
    aws logs put-retention-policy --log-group-name "$LG" --retention-in-days 30
  done
Enter fullscreen mode Exit fullscreen mode

For high-volume, rarely queried logs, use the Infrequent Access log class (about half the ingestion price). Also drop DEBUG logging in production.


5. Network Traffic Cost Engineering

Data transfer is the silent killer of AWS budgets. It's spread across dozens of usage types, so it rarely looks like one big problem.

Figure: Network cost engineering — before-and-after paths for S3/DynamoDB, AWS APIs, service-to-service traffic, and content delivery.

5.1 Gateway VPC Endpoints: free, and frequently missing

Traffic from private subnets to S3 or DynamoDB through a NAT Gateway pays NAT data-processing charges. AWS has listed this at about $0.045/GB in us-east-1. Confirm the rate for your Region. A Gateway Endpoint costs nothing and takes one command:

aws ec2 create-vpc-endpoint \
  --vpc-id vpc-0abc123 \
  --vpc-endpoint-type Gateway \
  --service-name com.amazonaws.us-east-1.s3 \
  --route-table-ids rtb-0priv1a rtb-0priv1b rtb-0priv1c
Enter fullscreen mode Exit fullscreen mode

Worked example (illustrative, using $0.045/GB): A data pipeline moves 20 TB/month from EC2 to S3 through NAT.
20,480 GB × $0.045 = $921.60/month. With a Gateway Endpoint, the NAT processing charge on that path becomes $0. The dollar result changes if your Region’s rate is different.

5.2 Interface Endpoints (PrivateLink): do the break-even math

For ECR, STS, CloudWatch Logs, SQS and similar services, an Interface Endpoint has often been listed at about $0.01/hr per AZ plus about $0.01/GB. Compared with NAT data processing at about $0.045/GB, the per-GB difference is about $0.035. Recalculate with your Region’s prices.

  • 3 AZs × $0.01 × 730 h ≈ $21.90/month fixed
  • Break-even ≈ 21.90 / 0.035 ≈ ~625 GB/month through NAT to that service

EKS clusters pulling large container images from ECR through NAT usually pass that break-even quickly. Put ECR endpoints (ecr.api, ecr.dkr) in place, plus the S3 gateway endpoint, because ECR image layers are stored in S3.

5.3 Cross-AZ traffic: stay multi-AZ, route locally

Cross-AZ transfer has commonly been listed at about $0.01/GB in each direction. Chatty microservices, Kafka replication and read replicas in another AZ add up fast. The fix is not to give up multi-AZ. Keep the redundancy and keep the traffic local:

  • Kubernetes: topology-aware routing (service.kubernetes.io/topology-mode: Auto) or trafficDistribution: PreferClose on Services.
  • Kafka/MSK: configure client.rack so consumers fetch from the closest replica.
  • Databases: send read traffic to a replica in the same AZ as the caller.
  • ALB: check whether cross-zone load balancing is actually needed for each target group.

5.4 CloudFront in front of everything public

  • Data transfer from AWS origins to CloudFront is free, and CloudFront's egress rates are generally lower than EC2 internet egress at volume.
  • Every cache hit is a request your origin fleet doesn't serve, so the ALB/EC2 tier can be smaller.
  • Origin Shield adds a regional caching layer that collapses duplicate origin fetches from many edge locations.
  • Track cache hit ratio as a cost KPI. Tune Cache-Control headers and cache keys (drop unnecessary query strings and cookies).

5.5 Public IPv4 addresses are no longer free

AWS currently bills in-use and idle public IPv4 addresses. The published rate has been about $0.005 per hour (about $3.65 per month). Confirm it before you forecast a large cleanup. Hundreds of instances with public IPs "just in case" add up. Use private subnets and load balancers, and look at IPv6 where you can. VPC IP Address Manager (IPAM) shows public IP usage across accounts.


6. Automating Non-Prod Shutdown

A dev environment that runs only during office hours (12h × 5 days = 60 of 168 weekly hours) cuts that compute bill by ~64%. It's the highest-ROI automation in this article.

6.1 No-code option: EventBridge Scheduler universal targets

EventBridge Scheduler can call almost any AWS API directly, so you don't need a Lambda:

aws scheduler create-schedule \
  --name stop-dev-ec2-evening \
  --schedule-expression "cron(0 20 ? * MON-FRI *)" \
  --schedule-expression-timezone "Asia/Kolkata" \
  --flexible-time-window Mode=OFF \
  --target '{
    "Arn": "arn:aws:scheduler:::aws-sdk:ec2:stopInstances",
    "RoleArn": "arn:aws:iam::111122223333:role/scheduler-ec2-startstop",
    "Input": "{\"InstanceIds\":[\"i-0abc123\",\"i-0def456\"]}"
  }'
Enter fullscreen mode Exit fullscreen mode

Create a matching startInstances schedule for the morning.

6.2 Tag-driven option: one Lambda for EC2 + RDS

Hard-coded instance IDs don't scale. Drive the schedule from tags (Schedule=OfficeHours) instead:

import boto3

ec2 = boto3.client("ec2")
rds = boto3.client("rds")

def lambda_handler(event, context):
    action = event.get("action", "stop")          # EventBridge passes {"action": "stop"|"start"}
    current_state = "running" if action == "stop" else "stopped"

    # EC2: find tagged instances in the right state
    paginator = ec2.get_paginator("describe_instances")
    ids = [
        i["InstanceId"]
        for page in paginator.paginate(Filters=[
            {"Name": "tag:Schedule", "Values": ["OfficeHours"]},
            {"Name": "instance-state-name", "Values": [current_state]},
        ])
        for r in page["Reservations"] for i in r["Instances"]
    ]
    if ids:
        (ec2.stop_instances if action == "stop" else ec2.start_instances)(InstanceIds=ids)

    # RDS: stop/start tagged DB instances (Aurora clusters need stop_db_cluster/start_db_cluster)
    for db in rds.describe_db_instances()["DBInstances"]:
        tags = {t["Key"]: t["Value"] for t in db.get("TagList", [])}
        if tags.get("Schedule") != "OfficeHours":
            continue
        if action == "stop" and db["DBInstanceStatus"] == "available":
            rds.stop_db_instance(DBInstanceIdentifier=db["DBInstanceIdentifier"])
        elif action == "start" and db["DBInstanceStatus"] == "stopped":
            rds.start_db_instance(DBInstanceIdentifier=db["DBInstanceIdentifier"])

    return {"action": action, "ec2": ids}
Enter fullscreen mode Exit fullscreen mode

For many accounts with complex calendars, deploy the Instance Scheduler on AWS solution. It works off the Schedule tag and keeps schedule definitions in DynamoDB.

⚠️ Gotchas:

  • Auto Scaling groups replace stopped instances. For ASGs, scale to zero instead: aws autoscaling update-auto-scaling-group --auto-scaling-group-name dev-asg --min-size 0 --desired-capacity 0.
  • RDS restarts a stopped instance after 7 days. Your schedule has to stop it again. It's not a set-and-forget "off" switch.
  • EKS: scale node groups or Karpenter limits down, or scale deployments to zero after hours.
  • A stopped EC2 instance still pays for its EBS volumes and Elastic IPs.

7. Database Optimization

Databases are usually the second-largest line item, and the hardest to change later. Pick the right model early.

7.1 Aurora

  • Aurora Serverless v2 scales capacity (ACUs) in fine-grained steps, within seconds. It fits spiky or unpredictable workloads and dev/test databases, and it can now scale down to 0 ACUs with auto-pause, so idle non-prod databases cost almost nothing for compute.
  • Aurora I/O-Optimized: if I/O charges are more than ~25% of your Aurora bill, switching the cluster storage configuration removes per-I/O charges in exchange for higher instance and storage prices. You can switch back once every 30 days.
  • Use Graviton classes (db.r7g, db.r8g where available) for provisioned instances.

7.2 RDS

  • Right-size with Performance Insights / CloudWatch Database Insights (load by wait event, top SQL) instead of CPU alone. A slow query is cheaper to fix than a bigger instance.
  • Move RDS storage to gp3 and set IOPS independently of size.
  • Tune max_connections and use RDS Proxy when many Lambda functions or short-lived connections would otherwise force a larger instance.
  • ⚠️ Extended Support surcharges: engine versions past end of standard support (e.g., older MySQL/PostgreSQL majors) are automatically enrolled in RDS Extended Support, billed per vCPU-hour. The rate goes up in later years. Upgrading is a cost optimization.
  • Buy Reserved Instances for steady production databases once they're right-sized.

7.3 DynamoDB

  • On-Demand vs Provisioned: On-Demand fits unpredictable or spiky traffic and got much cheaper after AWS's late-2024 price cut. Provisioned capacity with auto scaling (plus reserved capacity) is still cheaper for steady, high-throughput tables.
  • Standard-IA table class: about 60% cheaper storage and more expensive reads/writes. Use it for tables where storage dominates cost (history, audit, old orders).
  • TTL deletes expired items at no cost, which beats a scan-and-delete job.
  • DAX reduces read cost and latency for read-heavy hot keys, but it's a cluster you pay for by the hour. Measure before you add it.

7.4 Caching

ElastiCache for Valkey is priced lower than the Redis OSS engine (AWS quotes ~20% lower for nodes and ~33% lower for Serverless), and it's a drop-in replacement for most Redis workloads.


8. Continuous Cost Governance

Optimization without governance decays. Six months later, the waste is back. Make cost a DevOps responsibility backed by guardrails.

8.1 Service Control Policies: block expensive mistakes

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "DenyExpensiveInstanceTypes",
      "Effect": "Deny",
      "Action": "ec2:RunInstances",
      "Resource": "arn:aws:ec2:*:*:instance/*",
      "Condition": {
        "StringLike": {
          "ec2:InstanceType": ["p4d.*", "p5.*", "p5e.*", "u-*", "x2*", "*.metal*", "*.24xlarge", "*.32xlarge", "*.48xlarge"]
        },
        "ArnNotLike": { "aws:PrincipalArn": "arn:aws:iam::*:role/PlatformAdmin" }
      }
    },
    {
      "Sid": "DenyLaunchWithoutCostCenter",
      "Effect": "Deny",
      "Action": "ec2:RunInstances",
      "Resource": "arn:aws:ec2:*:*:instance/*",
      "Condition": { "Null": { "aws:RequestTag/CostCenter": "true" } }
    }
  ]
}
Enter fullscreen mode Exit fullscreen mode

Attach it to non-prod and sandbox OUs first. The PlatformAdmin exception gives a justified GPU or large-instance request a controlled path.

8.2 Tag Policies + AWS Config

  • Tag Policies (AWS Organizations) standardize tag keys and allowed values (Environment ∈ prod|staging|dev), so env, Env and ENV don't fragment your reports.
  • AWS Config required-tags flags non-compliant resources:
aws configservice put-config-rule --config-rule '{
  "ConfigRuleName": "required-tags",
  "Source": { "Owner": "AWS", "SourceIdentifier": "REQUIRED_TAGS" },
  "InputParameters": "{\"tag1Key\":\"Environment\",\"tag2Key\":\"Project\",\"tag3Key\":\"Owner\"}",
  "Scope": { "ComplianceResourceTypes": ["AWS::EC2::Instance","AWS::EC2::Volume","AWS::RDS::DBInstance","AWS::S3::Bucket"] }
}'
Enter fullscreen mode Exit fullscreen mode

💡 Insight: Config has its own cost. Recording every configuration change of high-churn resources can be expensive. Use periodic recording for noisy resource types and record only what you actually govern.

8.3 Budgets Actions: guardrails that act

AWS Budgets can apply an IAM policy or SCP, or stop specific EC2/RDS instances, when a threshold is crossed, either automatically or after approval. A sandbox account that hits 120% of its budget can be locked down without anyone being paged at 2 a.m.

8.4 The monthly FinOps review

Run it from CUR + Athena (or QuickSight/CUDOS dashboards) and review these KPIs every month:

KPI Target (starting point)
% of spend with valid owner tags > 95%
Savings Plans / RI utilization > 95%
Commitment coverage of eligible compute 70–85%
Spot share of stateless compute growing quarter over quarter
Idle / orphaned resource spend < 2%
Unit cost (e.g., $ per 1k requests) flat or decreasing

Every item gets an owner, a date and a ticket. A review without owners is just a meeting.


Bonus: Hidden Cost Killers Checklist

These show up in almost every account I review:

  • [ ] NAT Gateway processing for S3/DynamoDB/ECR traffic (fix: endpoints)
  • [ ] CloudWatch Logs with no retention, and DEBUG logs in production
  • [ ] Idle load balancers and NAT Gateways in unused AZs
  • [ ] Unattached EBS volumes, old snapshots and unused AMIs
  • [ ] Public IPv4 addresses on instances that don't need them
  • [ ] gp2 volumes that were never migrated to gp3
  • [ ] EKS clusters on Kubernetes versions in extended support (AWS has published about $0.60/hr instead of about $0.10/hr per cluster, roughly $438 vs $73/month; confirm the current extended-support rate)
  • [ ] RDS engines in Extended Support
  • [ ] T-family instances in unlimited mode charging surplus credits
  • [ ] S3 incomplete multipart uploads and endless noncurrent versions
  • [ ] Cross-Region replication or backups nobody asked for
  • [ ] Dev/test environments running 24×7

AWS Services Reference: What Each One Does for Your Bill

Service What it is How it helps cost
Data Exports (CUR 2.0) Line-item billing data delivered to S3 Working dataset for chargeback, anomaly hunting and unit cost, for the columns the export includes
Amazon Athena Serverless SQL over S3 Query CUR and VPC Flow Logs without running a database
Amazon QuickSight (CUDOS) BI dashboards Self-service cost visibility per team
AWS Cost Explorer Interactive cost analysis Trends, forecasts, SP/RI coverage and utilization
Cost Categories Rules that group costs Map accounts and tags to business units
AWS Budgets / Budgets Actions Thresholds + automated responses Early warning; automatic lockdown of runaway accounts
Cost Anomaly Detection ML-based spend monitoring Catches spikes within hours, not at month-end
Cost Optimization Hub Consolidated recommendations One deduplicated, prioritized list of savings
AWS Compute Optimizer ML right-sizing EC2, ASG, EBS, Lambda, ECS on Fargate, RDS recommendations
AWS Trusted Advisor Best-practice checks Idle resources, low-utilization instances, unassociated EIPs
Savings Plans / Reserved Instances Commitment discounts AWS advertises up to 72% off steady compute; RIs for databases
EC2 Spot / EC2 Fleet / ASG mixed instances Spare capacity at a discount AWS advertises up to 90% off fault-tolerant compute
Karpenter Kubernetes node autoscaler Spot-first provisioning, bin-packing, consolidation
AWS Graviton ARM-based instances Lower price and better price-performance across EC2, Lambda, RDS, ElastiCache
EC2 Auto Scaling (target tracking, predictive) Elastic capacity Pay for demand instead of peak
S3 Lifecycle / Intelligent-Tiering / Storage Lens Tiering and visibility Move cold data to cheaper classes automatically
Amazon EBS gp3 / Snapshots Archive / Data Lifecycle Manager Block storage controls Often about 20% cheaper volumes, cheaper snapshot retention, automated cleanup
Amazon EFS lifecycle (IA/Archive) File storage tiering Cold files at a fraction of Standard price
VPC Gateway Endpoints Private route to S3/DynamoDB Removes NAT processing charges at no cost
AWS PrivateLink (Interface Endpoints) Private access to AWS services Cheaper than NAT above the break-even volume
Amazon CloudFront + Origin Shield CDN Lower egress, fewer origin requests, smaller origin fleet
Amazon EventBridge Scheduler Serverless scheduler Start/stop non-prod directly through API calls
AWS Lambda / SSM Automation Serverless compute / runbooks Automated remediation and right-sizing
Instance Scheduler on AWS Tag-based scheduling solution Org-wide office-hours schedules
Aurora Serverless v2 / I/O-Optimized Elastic / I/O-flat database pricing Pay for real load; predictable cost for I/O-heavy workloads
DynamoDB On-Demand / Standard-IA / TTL Capacity modes and table classes Match pricing to traffic and storage profile
ElastiCache for Valkey Managed in-memory cache Lower price than the Redis OSS engine
AWS Organizations (SCPs, Tag Policies) Multi-account guardrails Prevent costly launches; consistent tagging
AWS Config Resource compliance Enforce tags and flag non-compliant resources
Amazon Q Developer in chat apps ChatOps (formerly AWS Chatbot) Cost alerts where engineers already work

A 30-60-90 Day Rollout Plan

Days 0–30: visibility + quick wins

  • Enable CUR 2.0 → Athena, activate cost allocation tags, create Budgets and Anomaly Detection
  • Gateway Endpoints for S3/DynamoDB in every VPC
  • gp2 → gp3, delete orphaned volumes and EIPs, set CloudWatch Logs retention
  • S3 lifecycle "hygiene" rule on every bucket

Days 31–60: structural savings

  • Right-size from Compute Optimizer (with memory metrics)
  • Non-prod schedules through EventBridge Scheduler / Instance Scheduler
  • Spot for CI and batch; Karpenter with Spot on EKS
  • First Savings Plans tranche, sized to the post-right-sizing baseline

Days 61–90: make it stick

  • SCPs, Tag Policies, Config required-tags
  • Graviton canaries for stateless services and managed databases
  • Unit-cost dashboards and the monthly FinOps review, with owners

Conclusion

Real-world AWS cost optimization isn't one trick. It's an engineering discipline:

  1. Measure with granular, tagged, near-real-time data.
  2. Right-size on real percentiles, then commit to the floor and put the rest on Spot.
  3. Tier storage and engineer network paths deliberately.
  4. Automate the boring parts, like schedules, clean-ups and remediation.
  5. Govern with guardrails so the waste doesn't come back.

Combine measurement, automation and governance, and you reduce waste, maximize savings and enforce accountability without giving up performance or reliability.

If this helped, drop a comment with the biggest hidden cost you've found in your AWS account. I'd love to compare notes. 👇


🧑‍💼 About the Author

Sanket Satish Patharkar is a Senior Cloud Operations Engineer with 6+ years of experience in AWS, DevOps, CI/CD and enterprise-grade cloud architecture, particularly in the automotive systems domain.

He specializes in building highly available, cost-efficient and automated infrastructure. He has designed cloud architectures that meet high availability (HA) and scalability requirements and keep applications running during component failures or traffic spikes. Sanket combines practical engineering, automation and governance to help organizations cut costs while keeping their cloud environments resilient and efficient.

For any issue or support: LinkedIn.

Top comments (0)