Cloud bills creep up silently. Last quarter, our AWS bill hit $12,000/month. I spent one weekend optimizing it and cut it by 40%. Here is exactly what I did, with the scripts I used.
Step 1: Find the Waste
Before optimizing, you need to find where the money is going.
import boto3
from collections import defaultdict
def analyze_costs(days=30):
ce = boto3.client('ce')
start = (datetime.now() - timedelta(days=days)).strftime('%Y-%m-%d')
end = datetime.now().strftime('%Y-%m-%d')
response = ce.get_cost_and_usage(
TimePeriod={'Start': start, 'End': end},
Granularity='DAILY',
Metrics=['UnblendedCost'],
GroupBy=[{'Type': 'DIMENSION', 'Key': 'SERVICE'}]
)
costs = defaultdict(float)
for day in response['ResultsByTime']:
for group in day['Groups']:
service = group['Keys'][0]
cost = float(group['Metrics']['UnblendedCost']['Amount'])
costs[service] += cost
return sorted(costs.items(), key=lambda x: x[1], reverse=True)
The output told me: EC2 was 45% of the bill, RDS was 25%, S3 was 15%, and the rest was scattered.
Step 2: Kill Idle Resources
The easiest savings. Resources that nobody uses but nobody deleted.
import boto3
def find_idle_ec2():
ec2 = boto3.client('ec2')
instances = ec2.describe_instances()
idle = []
for res in instances['Reservations']:
for inst in res['Instances']:
if inst['State']['Name'] == 'stopped':
idle.append({
'id': inst['InstanceId'],
'type': inst['InstanceType'],
'name': next((t['Value'] for t in inst.get('Tags', []) if t['Key'] == 'Name'), 'unnamed'),
'state': 'stopped'
})
return idle
def find_unattached_volumes():
ec2 = boto3.client('ec2')
volumes = ec2.describe_volumes()
unattached = []
for vol in volumes['Volumes']:
if vol['State'] == 'available':
unattached.append({
'id': vol['VolumeId'],
'size': vol['Size'],
'type': vol['VolumeType']
})
return unattached
Found 3 stopped EC2 instances and 7 unattached EBS volumes. Savings: $340/month.
Step 3: Right-Size Over-Provisioned Instances
Most instances are sized for peak load that never comes.
def get_cpu_utilization(instance_id, days=14):
cloudwatch = boto3.client('cloudwatch')
response = cloudwatch.get_metric_statistics(
Namespace='AWS/EC2',
MetricName='CPUUtilization',
Dimensions=[{'Name': 'InstanceId', 'Value': instance_id}],
StartTime=datetime.now() - timedelta(days=days),
EndTime=datetime.now(),
Period=3600,
Statistics=['Average', 'Maximum']
)
if not response['Datapoints']:
return None
avg_cpu = sum(d['Average'] for d in response['Datapoints']) / len(response['Datapoints'])
max_cpu = max(d['Maximum'] for d in response['Datapoints'])
return {'avg': avg_cpu, 'max': max_cpu}
Rule: if average CPU is under 20% and max CPU is under 50% for 14 days, downsize by one step.
Found 5 instances that could be downsized. Savings: $890/month.
Step 4: Reserved Instances and Savings Plans
The biggest single saving. If you run instances 24/7, you should have reserved instances.
def check_reserved_coverage():
ec2 = boto3.client('ec2')
# Get running instances
running = ec2.describe_instances(Filters=[{'Name': 'instance-state-name', 'Values': ['running']}])
on_demand = defaultdict(int)
for res in running['Reservations']:
for inst in res['Instances']:
on_demand[inst['InstanceType']] += 1
# Get reserved instances
reserved = ec2.describe_reserved_instances(Filters=[{'Name': 'state', 'Values': ['active']}])
reserved_count = defaultdict(int)
for ri in reserved['ReservedInstances']:
reserved_count[ri['InstanceType']] += ri['InstanceCount']
# Calculate coverage
coverage = {}
for itype, count in on_demand.items():
covered = reserved_count.get(itype, 0)
coverage[itype] = {
'running': count,
'reserved': covered,
'uncovered': max(0, count - covered)
}
return coverage
We had 0 reserved instances. Bought 1-year reserved instances for our 4 most-used instance types. Savings: $2,100/month.
Step 5: S3 Lifecycle Policies
Old data does not need to live on S3 Standard.
def setup_s3_lifecycle(bucket_name):
s3 = boto3.client('s3')
s3.put_bucket_lifecycle_configuration(
Bucket=bucket_name,
LifecycleConfiguration={
'Rules': [{
'Status': 'Enabled',
'Filter': {'Prefix': ''},
'Transitions': [
{'Days': 30, 'StorageClass': 'STANDARD_IA'},
{'Days': 90, 'StorageClass': 'GLACIER'},
{'Days': 365, 'StorageClass': 'DEEP_ARCHIVE'}
],
'Expiration': {'Days': 730}
}]
}
)
Applied to all 12 buckets. Savings: $420/month.
Step 6: Snapshot Cleanup
Old backups that nobody will ever restore from.
def clean_old_snapshots(days=30):
ec2 = boto3.client('ec2')
snapshots = ec2.describe_snapshots(OwnerIds=['self'])
cutoff = datetime.now() - timedelta(days=days)
old = []
for snap in snapshots['Snapshots']:
if snap['StartTime'].replace(tzinfo=None) < cutoff:
old.append(snap['SnapshotId'])
return old
Found 89 snapshots older than 30 days. Deleted them. Savings: $180/month.
Step 7: NAT Gateway Audit
NAT Gateways cost $32/month base plus $0.045/GB processed. They are often left running in dev environments.
def audit_nat_gateways():
ec2 = boto3.client('ec2')
nats = ec2.describe_nat_gateways()
for nat in nats['NatGateways']:
vpc = nat['VpcId']
subnet = nat['SubnetId']
state = nat['State']
print(f"NAT Gateway: {nat['NatGatewayId']} | VPC: {vpc} | State: {state}")
Found 2 NAT gateways in dev VPCs that should have been deleted. Savings: $64/month.
The Results
| Optimization | Monthly Savings |
|---|---|
| Kill idle resources | $340 |
| Right-size instances | $890 |
| Reserved instances | $2,100 |
| S3 lifecycle | $420 |
| Snapshot cleanup | $180 |
| NAT gateway audit | $64 |
| Total | $3,994 |
From $12,000 to $8,006. A 33% reduction in one weekend.
What I Learned
- Billing alerts first. Set up AWS Budget alerts before optimizing. You need to know your baseline.
- Idle resources are free money. Always check for stopped instances and unattached volumes first.
- Reserved instances are the biggest win. If you run 24/7, you are burning money without them.
- Automate the audit. Run the cost analysis script weekly. Costs creep back up.
For complete cloud cost optimization scripts and templates, check out the Ops Starter Kit.
Top comments (0)