DEV Community

Geminate Solutions
Geminate Solutions

Posted on

Find underused EC2 instances and unattached EBS volumes with Node.js

You can find underused EC2 instances and unattached EBS volumes with one Node.js script. It pulls 14 days of CloudWatch CPU and network metrics for every running instance, flags the ones that barely move, and lists every volume whose status is available. That covers most of the quick wins in cloud optimization, and it only needs read-only permissions.

This is for you if your AWS bill keeps growing while traffic stays flat and nobody can say for sure which servers still do real work. The script writes two CSV files that your team can go through in one meeting.

Why do idle instances and orphaned volumes pile up?

Creating resources is easy, and nobody is in charge of deleting them. Someone spins up a staging box for a demo. A load test leaves a fleet at its peak size. An Auto Scaling group gets replaced and its old instances stay behind.

Volumes are worse. When you terminate an instance, only volumes with DeleteOnTermination set to true are deleted with it. Data volumes attached later default to false, so they stay behind, detached and still billed for every provisioned GiB.

Oversizing has a different cause. Teams pick a size at launch with room for a spike that never comes. Once the app works, nobody looks at the size again.

What does it cost you to leave this alone?

The direct cost is paying every hour, with no end date, for capacity nobody uses. Four things drive it: instance size, instance count, hours running and provisioned GiB on every volume.

The indirect cost is harder to see. An account full of unnamed resources with no owner slows every security review and hides real capacity problems. The next person who tries to clean up is afraid to touch anything, and with good reason.

Delete the wrong volume and you can lose data you will never get back.

Who does not need this script?

If you run a handful of instances in one region, use AWS Compute Optimizer and the rightsizing view in Cost Explorer. Both look at a similar 14 day window and need no code. Compute Optimizer only needs you to opt in.

The script is worth it when you have several accounts or regions, want to set your own thresholds, or need a report you can run in CI every week and compare over time.

How does the script decide what is idle or oversized?

It pulls hourly data points for 14 days and applies a few simple rules. Fourteen days covers two full weeks, so a batch job that only runs on Sundays still shows up.

Verdict Rule Suggested action
IDLE p95 of hourly max CPU under 5% and under 50 MB of network a day Find the owner, then stop it
OVERSIZED Average CPU under 10% and p95 peak under 40% Test one size smaller
NO_DATA No CloudWatch data points Probably launched recently, check again later
OK Anything else Leave it

The script uses the 95th percentile of hourly peaks instead of the single highest value. That way one spike during a deploy does not hide a server that is otherwise idle.

One limit you should know about: EC2 does not send memory metrics to CloudWatch by default. A database or cache with low CPU can still be using most of its RAM. If you run the CloudWatch agent, add mem_used_percent from the CWAgent namespace before you trust an OVERSIZED verdict on those machines.

The script

Install the two SDK clients:

npm i @aws-sdk/client-ec2 @aws-sdk/client-cloudwatch
Enter fullscreen mode Exit fullscreen mode

The role running it needs only ec2:DescribeInstances, ec2:DescribeVolumes and cloudwatch:GetMetricData. GetMetricData is billed per metric requested, so the cost of each run grows with your instance count (four metrics per instance). Run it weekly, not every minute.

// rightsize.mjs
import { EC2Client, paginateDescribeInstances, paginateDescribeVolumes } from '@aws-sdk/client-ec2'
import { CloudWatchClient, GetMetricDataCommand } from '@aws-sdk/client-cloudwatch'
import { writeFileSync } from 'node:fs'

const region = process.env.AWS_REGION || 'us-east-1'
const DAYS = 14
const ec2 = new EC2Client({ region })
const cw = new CloudWatchClient({ region })
const end = new Date()
const start = new Date(end.getTime() - DAYS * 86400000)

async function listRunningInstances() {
  const out = []
  const pages = paginateDescribeInstances({ client: ec2 }, {
    Filters: [{ Name: 'instance-state-name', Values: ['running'] }]
  })
  for await (const page of pages) {
    for (const r of page.Reservations ?? []) {
      for (const i of r.Instances ?? []) {
        const name = i.Tags?.find(t => t.Key === 'Name')?.Value ?? ''
        out.push({ id: i.InstanceId, type: i.InstanceType, name: name.replace(/,/g, ' ') })
      }
    }
  }
  return out
}

function queriesFor(inst, n) {
  const dims = [{ Name: 'InstanceId', Value: inst.id }]
  const q = (id, metric, stat) => ({
    Id: id + '_' + n,
    MetricStat: {
      Metric: { Namespace: 'AWS/EC2', MetricName: metric, Dimensions: dims },
      Period: 3600,
      Stat: stat
    }
  })
  return [
    q('cpuavg', 'CPUUtilization', 'Average'),
    q('cpumax', 'CPUUtilization', 'Maximum'),
    q('netin', 'NetworkIn', 'Sum'),
    q('netout', 'NetworkOut', 'Sum')
  ]
}

async function fetchMetrics(instances) {
  const values = {}
  for (let b = 0; b < instances.length; b += 100) {
    const queries = instances.slice(b, b + 100).flatMap((inst, k) => queriesFor(inst, b + k))
    let NextToken
    do {
      const res = await cw.send(new GetMetricDataCommand({
        StartTime: start, EndTime: end, MetricDataQueries: queries, NextToken
      }))
      for (const r of res.MetricDataResults ?? []) {
        values[r.Id] = (values[r.Id] ?? []).concat(r.Values ?? [])
      }
      NextToken = res.NextToken
    } while (NextToken)
  }
  return values
}

async function listUnattachedVolumes() {
  const out = []
  const pages = paginateDescribeVolumes({ client: ec2 }, {
    Filters: [{ Name: 'status', Values: ['available'] }]
  })
  for await (const page of pages) {
    for (const v of page.Volumes ?? []) {
      out.push({
        id: v.VolumeId,
        type: v.VolumeType,
        sizeGiB: v.Size,
        ageDays: Math.round((end - v.CreateTime) / 86400000),
        snapshot: v.SnapshotId || ''
      })
    }
  }
  return out
}

const sum = a => a.reduce((s, v) => s + v, 0)
const avg = a => (a.length ? sum(a) / a.length : null)
function p95(a) {
  if (!a.length) return null
  const sorted = [...a].sort((x, y) => x - y)
  return sorted[Math.floor(sorted.length * 0.95)]
}

function classify(cpuAvg, cpuPeak, netMBPerDay) {
  if (cpuAvg === null) return 'NO_DATA'
  if (cpuPeak < 5 && netMBPerDay < 50) return 'IDLE'
  if (cpuAvg < 10 && cpuPeak < 40) return 'OVERSIZED'
  return 'OK'
}

const instances = await listRunningInstances()
const metrics = await fetchMetrics(instances)

const rows = instances.map((inst, n) => {
  const cpuAvg = avg(metrics['cpuavg_' + n] ?? [])
  const cpuPeak = p95(metrics['cpumax_' + n] ?? [])
  const netBytes = sum(metrics['netin_' + n] ?? []) + sum(metrics['netout_' + n] ?? [])
  const netMBPerDay = netBytes / DAYS / 1e6
  return { ...inst, cpuAvg, cpuPeak, netMBPerDay, verdict: classify(cpuAvg, cpuPeak, netMBPerDay) }
})
const volumes = await listUnattachedVolumes()

const fmt = v => (typeof v === 'number' ? v.toFixed(1) : (v ?? ''))
const toCsv = list => [Object.keys(list[0]).join(',')]
  .concat(list.map(r => Object.values(r).map(fmt).join(',')))
  .join('\n')

if (rows.length) writeFileSync('instances.csv', toCsv(rows))
if (volumes.length) writeFileSync('volumes.csv', toCsv(volumes))
console.table(rows.filter(r => r.verdict !== 'OK'))
console.table(volumes)
Enter fullscreen mode Exit fullscreen mode

GetMetricData accepts up to 500 queries per call. The script sends 100 instances (400 queries) at a time and follows NextToken until it has every data point. Run it with AWS_REGION=eu-west-1 node rightsize.mjs, then repeat for each region you use.

How do you act on the report without breaking production?

Go from the cheapest and safest change to the riskiest.

  1. Unattached volumes older than 30 days. Take a snapshot, tag it with the volume ID and the date, then delete the volume. A snapshot is billed on the data actually stored, not the provisioned size.
  2. IDLE instances. If the instance has an owner tag, message the owner. If nobody claims it within a week, stop it but do not terminate it. A stopped instance keeps its volumes, so you can start it again if someone turns out to need it.
  3. OVERSIZED instances. Move down one size in the same family, for example m5.2xlarge to m5.xlarge, during a quiet window. Watch latency and error rates for a few days before you go smaller again.

These rules are worth writing down for the team:

  • If it is a burstable T family instance, check CPUCreditBalance before you downsize. A smaller T instance earns credits more slowly, and an app that runs out of credits gets throttled.
  • If it belongs to an Auto Scaling group, change the launch template, not the instance. If you change only the instance, the group replaces it at the old size.
  • If CPU is low but the app is slow, the problem is somewhere else: disk, database or memory. Downsizing will make it worse.

What should you check before deleting a volume?

Check three things. First, ageDays counts from when the volume was created, not when it was detached. Look up the DetachVolume event in CloudTrail to see how long it has really been unused. If CloudTrail shows nothing in its 90 day event history, the volume has been detached for longer than that.

Second, read its tags for a project or owner. Third, if the snapshot column is empty, nothing backs this volume up yet, so take a snapshot first.

The cleanup is the easy part. Keeping the account clean takes tagging rules, named owners and sensible default sizes in your infrastructure code. If you want a second opinion on your AWS setup, Geminate Solutions offers a free review through its cloud and DevOps service and replies within 24 hours.

Top comments (0)