DEV Community: Muskan Bandta

AWS Trusted Advisor vs Compute Optimizer vs Cost Explorer: What Each One Actually Finds

Muskan Bandta — Tue, 01 Sep 2026 10:49:53 +0000

AWS gives you three native tools that all claim to help with cost, and the "my provider says I'm already optimized" confidence usually comes from having glanced at one of them. The problem is that Trusted Advisor, Compute Optimizer, and Cost Explorer do genuinely different jobs, and no single one gives you the full picture. Most comparisons just line up their feature checkboxes. Here is what each actually finds, where each is blind, and how they fit together.

Cost Explorer: what you spent and where

Cost Explorer is the visualization and reporting layer. It answers "where is the money going": spend by service, by tag, by account, over time, with forecasting.

Finds: trends, spikes, which service or team is growing, and it surfaces Reserved Instance and Savings Plans recommendations based on your usage history.
Blind to: whether individual resources are right-sized or idle. Cost Explorer tells you EC2 is your biggest line; it does not tell you that a specific instance is at 4% CPU. It is the map, not the inspection.

Use it for the big-picture question and for commitment purchase recommendations. Do not expect it to point at individual waste.

Compute Optimizer: is each resource the right size

Compute Optimizer is the machine-learning rightsizing engine. It analyzes utilization metrics and recommends a better configuration per resource.

Finds: over- and under-provisioned EC2 instances, Auto Scaling groups, EBS volumes, and Lambda functions, with a concrete recommended target and the projected savings. It is the tool that says "this m5.xlarge should be an m5.large."
Blind to: things that are not a sizing question. It will not tell you a resource is entirely unused (that is idle/orphan territory), and its scope is limited to the resource types it supports. It also needs enough utilization history to be confident.

Use it for the rightsizing question specifically. Turn it on (it is free), because it produces the most actionable per-resource dollar figures of the three.

Trusted Advisor: broad checks across pillars

Trusted Advisor runs a catalogue of best-practice checks across cost, security, performance, fault tolerance, and service limits. On cost, it flags things like idle load balancers, underutilized EBS volumes, and idle RDS instances.

Finds: a broad sweep of best-practice violations, including some idle-resource cost checks the other two do not cover.
Blind to / limited by: the free tier only exposes a subset of checks; the full cost checks require a Business or Enterprise Support plan. And its checks are threshold-based best practices, not deep per-resource rightsizing analysis.

Use it as the broad hygiene sweep, but know that on the free tier you are seeing a fraction of what it can flag.

The honest gap: none of them covers everything

Here is what the "we're already optimized" crowd misses. Each tool has a scope, and the union of all three still leaves gaps:

Cost Explorer: spend visibility + commitment recommendations.
Compute Optimizer: rightsizing for a specific set of resource types.
Trusted Advisor: broad best-practice checks (subset free).

None of them, on its own, gives you a complete "here is every dollar of recoverable waste across every resource type and every cloud" answer. They do not cover orphaned resources thoroughly, they are AWS-only, and the coverage is split across three consoles with different scopes and access tiers. That is exactly the gap third-party FinOps tools exist to fill: one continuous view across resource types and clouds with the dollar figure attached (which is the category ZopNight sits in). The native tools are a real and free starting point; they are not the whole map.

How to actually use the three together

Cost Explorer to see the big rocks and grab RI/SP recommendations.
Compute Optimizer (turn it on, it is free) for per-resource rightsizing with concrete targets.
Trusted Advisor for the broad hygiene sweep, understanding the free tier is partial.
Then look for what all three miss: orphaned and idle resources, cross-account and cross-cloud waste, and anything outside Compute Optimizer's supported types.

The take

"My provider says I'm optimized" usually means "I looked at one of three tools that each see a different slice." Cost Explorer shows spend, Compute Optimizer rightsizes specific resources, Trusted Advisor runs broad checks (mostly behind a support plan). Use all three, then be honest that even together they leave gaps. Optimized is not a checkbox one console gives you; it is the union of these plus the waste none of them catches.

Which of the three do you actually check, and has it ever given you a false sense of "we're fine"? For me it was Cost Explorer, a clean-looking trend line hid a pile of idle resources that only Compute Optimizer and a manual sweep surfaced.

What an Idle NAT Gateway, Idle Load Balancer and Sub-5% EC2 Instance Actually Cost You Per Month

Muskan Bandta — Tue, 01 Sep 2026 10:49:43 +0000

"Idle" sounds free. It is not. Some of the most reliable waste in an AWS account comes from resources that are running, doing almost nothing, and billing you the full rate anyway. The frustrating part is that none of them is expensive enough on its own to trigger a second look, so they survive for months. Put a real monthly number on each and they stop looking harmless.

Here are the three most common idle-but-billing resources, roughly what each costs, and how to find yours.

Idle NAT gateway: roughly $32+ per month, before traffic

A NAT gateway bills two ways: an hourly charge just for existing, plus a per-GB data processing charge for traffic through it. The hourly charge alone is about $0.045 per hour in most regions, which is roughly $32 a month per gateway whether or not a single byte flows through it.

The waste pattern: a NAT gateway left behind in a region or VPC you migrated out of, or one provisioned per-AZ "for high availability" in an environment that no longer needs all of them. Each idle one is $32+ a month for nothing, and teams often have several.

Find NAT gateways with near-zero traffic by checking their BytesOutToDestination CloudWatch metric over the last couple of weeks. Flatlined at zero means it is billing you for existing and nothing else.

Idle load balancer: roughly $16 to $22+ per month with no healthy targets

An Application or Network Load Balancer has an hourly charge (around $0.0225/hour for an ALB, so ~$16+ a month) plus capacity-unit charges. A load balancer with zero healthy targets is the giveaway: the thing it was routing to got deprovisioned, but the load balancer stayed, still billing.

Find them by checking HealthyHostCount per target group. Zero healthy hosts over a sustained period means the LB is routing to nothing and can almost certainly go.

Sub-5% CPU EC2 instance: the full instance price for near-idle

An EC2 instance at 3% average CPU costs exactly the same as one at 80%. A box that has averaged under 5% CPU with negligible network for two weeks is either wildly over-provisioned or genuinely unused, and either way you are paying full rate. Depending on size that is anywhere from a few dollars to hundreds a month per instance.

The detection rule most teams use: average CPU under 5% and low network IO for 14 days. That combination separates "idle" from "low but real" (a box can be low-CPU but doing important memory or IO work, so check network and memory before acting, not just CPU).

Why these survive for months

Two reasons, and they are worth naming because they tell you how to fix the process, not just the resources:

Each is individually small. $32 here, $16 there, one small instance. None trips a budget alert, so none gets investigated. The waste is real only in aggregate.
Nobody owns "idle." These resources have no obvious owner to notice them. They are the residue of changes, migrations, and experiments that moved on.

How to find and fix yours

NAT gateways: list them all, check BytesOutToDestination. Delete the ones flatlined at zero (confirm no private subnet still needs egress through them first).
Load balancers: check HealthyHostCount per target group. Zero healthy hosts sustained means delete.
EC2: pull 14-day average CPU and network. Under-5% CPU plus low network means rightsize or stop, after checking it is not memory or IO bound.

Then make it recurring. Idle resources regenerate constantly, so a one-time cleanup is a one-time win. A monthly detection pass (a script, or a tool that continuously flags idle resources with the dollar figure attached, which is part of what ZopNight does) is what actually keeps the number down.

The take

Idle does not mean free. An idle NAT gateway is ~$32 a month, an idle load balancer is ~$16 or more, and a near-idle EC2 instance is the full instance price for almost no work. Individually forgettable, collectively a real line on the bill, and among the safest cuts you can make because nobody is using them. Put the dollar figure on each and they stop being invisible.

Which idle resource has cost you the most before you caught it? For me it was a pair of per-AZ NAT gateways in a region we had mostly left, quietly billing ~$64 a month between them for traffic that had dropped to nearly zero.

The Cloud Zombie Index: Unattached EBS Volumes, Orphaned Snapshots, Unused Elastic IPs and Incomplete Multipart Uploads

Muskan Bandta — Tue, 01 Sep 2026 10:49:31 +0000

Every cloud account is haunted. Not by anything dramatic, just by resources that outlived whatever created them and now bill you every hour for nothing. I call them cloud zombies: things nobody owns, nobody uses, and nobody remembers, quietly draining the budget. They are the cheapest waste to kill because deleting them breaks nothing, and the easiest to ignore because none of them is big on its own.

Here is a field guide to the most common zombies, how to find each, and how to kill them safely.

1. Unattached EBS volumes

When you terminate an EC2 instance, its data volumes do not always go with it. Volumes with "delete on termination" off survive, detached, billing per GB forever. A 500 GB gp3 volume nobody has touched in a year is pure waste.

Find them:

aws ec2 describe-volumes --filters Name=status,Values=available \
  --query "Volumes[].{ID:VolumeId,GB:Size,Created:CreateTime}" --output table

status=available means "not attached to anything." Snapshot first if you are unsure, then delete.

2. Orphaned EBS snapshots

Snapshots pile up from old backup scripts and long-gone volumes. The dangerous ones to delete are snapshots referenced by an AMI, so check before you purge. Everything older than your retention policy that is not backing an AMI is a candidate. This is usually the single largest zombie category by count.

3. Unused Elastic IPs

This one actively bills you because it is idle. AWS charges for an Elastic IP that is allocated but not associated with a running instance. An idle EIP is a rare case of paying specifically for not using something.

aws ec2 describe-addresses \
  --query "Addresses[?AssociationId==null].{IP:PublicIp,Alloc:AllocationId}" --output table

Release any that come back, after confirming nothing plans to use them.

4. Incomplete multipart uploads

The sneakiest zombie. When a large S3 upload fails partway, the uploaded parts are not automatically cleaned up. They sit in the bucket, invisible in the normal object listing, billing you for storage indefinitely. Entire teams have gigabytes of these they have never seen.

The fix is a lifecycle rule that aborts incomplete multipart uploads after a few days:

{
  "Rules": [{
    "ID": "abort-incomplete-mpu",
    "Status": "Enabled",
    "Filter": {},
    "AbortIncompleteMultipartUpload": { "DaysAfterInitiation": 7 }
  }]
}

Set this on every bucket. It is a one-time fix that stops a category of invisible waste forever.

5. The cross-cloud cousins

The same pattern repeats everywhere:

Azure: unattached managed disks (they bill exactly like unattached EBS volumes), orphaned snapshots, unassociated public IPs.
GCP: orphaned persistent disks, unused static external IPs, old machine images.
Everywhere: load balancers with no healthy targets, abandoned node groups, idle dashboards and alarms.

How to run a sweep without breaking anything

The reason people avoid this cleanup is fear of deleting something that matters. Two rules make it safe:

Snapshot before delete, by default. For anything with data (volumes, disks), take a snapshot first. Storage for a snapshot is cheap; restoring from one is easy. This turns "delete" from irreversible into reversible.
Age plus unattached, not just unattached. Only target resources that are both detached/unassociated and older than some threshold (14 to 30 days). This avoids catching something created an hour ago mid-setup.

Run it as a monthly habit, not a one-time heroic cleanup. Zombies regenerate: every terminated instance, failed upload, and decommissioned service can leave one behind. A recurring sweep (a scheduled Lambda, or a tool that does discovery and flags orphans automatically, which is part of what ZopNight handles) keeps the account clean instead of letting it silt up again.

The take

None of these zombies is a big number by itself. Collectively, on a neglected account, they are routinely thousands of dollars a month, and they are the safest savings you will ever book because deleting them breaks nothing. Build the "cloud zombie index" for your account this week: unattached volumes, orphaned snapshots, idle EIPs, incomplete multipart uploads, and their cross-cloud cousins. Snapshot, age-filter, delete, repeat monthly.

What is the biggest zombie you have found, and how long had it been billing before anyone noticed? Mine was a set of orphaned snapshots from a backup script that had been "temporary" for three years.

Azure VM Stopped vs Deallocated: Why You're Still Being Charged (and the Disks Nobody Mentions)

Muskan Bandta — Fri, 28 Aug 2026 06:39:02 +0000

You shut the VM down to save money, and next month it is still on the bill. This is one of the most common Azure billing surprises, and it comes down to a distinction Azure does not make obvious: there is a difference between a VM that is Stopped and one that is Stopped (deallocated), and only one of them stops the compute charges.

Here is exactly what is happening, and the cost that survives even when you do it right.

Stopped vs Stopped (deallocated)

Azure has two "off" states, and they bill completely differently.

Stopped (from inside the OS). If you run shutdown inside the guest OS, the VM powers off but Azure keeps the compute resources allocated to it. The status shows Stopped. You are still paying full compute price for a VM doing nothing. This is the trap.
Stopped (deallocated). If you stop the VM from the Azure Portal, CLI, or PowerShell, Azure deallocates it, releasing the underlying compute. The status shows Stopped (deallocated), and compute billing stops.

So the rule: shutting down from inside the guest does not save you money. You must deallocate, and deallocation only happens when you stop it through Azure, not through the OS.

# This deallocates and stops compute billing:
az vm deallocate --resource-group my-rg --name my-vm

# Inside-the-OS "shutdown" does NOT deallocate. Status stays "Stopped", billing continues.

Check which state you are actually in:

az vm get-instance-view --resource-group my-rg --name my-vm \
  --query "instanceView.statuses[?starts_with(code, 'PowerState')].displayStatus" -o tsv

If that returns VM stopped you are still paying. If it returns VM deallocated you are not paying for compute.

The disks nobody mentions

Here is the part that catches people even after they deallocate correctly: deallocation stops compute billing, not storage billing. The managed disks attached to the VM (the OS disk and any data disks) keep costing money whether the VM is running, stopped, or deallocated. A deallocated VM with a 512 GB Premium SSD is still billing you for that 512 GB every hour.

Other charges that also survive deallocation:

Static public IP addresses. A reserved (static) public IP keeps billing when the VM is off. Dynamic IPs are released on deallocation.
Reserved capacity or licenses tied to the VM, depending on how they were purchased.

So "I turned it off and I am still charged" almost always means one of two things: you stopped from inside the OS (compute still billing), or you deallocated correctly but are seeing the disk and IP charges that never stop.

What to actually do

To pause a VM you will use again soon: deallocate it (through Azure, not the OS). You stop compute, keep the disks, and it starts back up with its data intact. This is the right move for dev and test boxes overnight.
To stop paying entirely for something you are done with: deallocating is not enough. You must delete the VM and its disks (and release the static IP). A deleted VM whose disks you forgot to remove is a classic zombie cost.
For non-production VMs you deallocate nightly: automate it on a schedule rather than trusting people to remember. A start/stop schedule on dev and test VMs is one of the highest-return Azure cost moves there is, and it only works if it is automatic. (We schedule non-prod VMs the same way across clouds, that scheduling is part of what ZopNight does, but an Automation runbook or a Logic App on a timer does the crude version.)

The one-line summary

Stopping a VM from inside Windows or Linux saves you nothing. Deallocating it through Azure stops the compute charge but keeps billing you for the disks and any static IP. To fully stop paying, delete the VM and its disks. Most "why am I still being charged" cases are one of those three facts.

Have you been bitten by the inside-the-OS shutdown, or by the disks that keep billing after deallocation? For me it was the disks, I deallocated a fleet of test VMs, felt clever, and still saw a chunk of the bill I could not explain until I looked at the storage line.

Scheduling EC2 and RDS Start/Stop at Scale: Why Your Shutdown Script Breaks at 300 Instances

Muskan Bandta — Fri, 28 Aug 2026 06:38:48 +0000

Everybody's cloud cost journey has the same first chapter: someone writes a Lambda that stops the dev instances at night and starts them in the morning. It works. It saves real money. And then the environment grows, and one morning the script that ran fine for a year quietly causes an outage. The shutdown script that works on one instance breaks at three hundred, and it breaks in four specific ways. Here is each one, because knowing them is the difference between saving money and writing a postmortem.

The script that works on one instance

# stop_dev.py, EventBridge at 20:00
import boto3
ec2 = boto3.client("ec2")
ids = [i["InstanceId"] for r in ec2.describe_instances(
        Filters=[{"Name":"tag:env","Values":["dev"]}])["Reservations"]
        for i in r["Instances"]]
ec2.stop_instances(InstanceIds=ids)

At small scale this is fine. At scale, here is what goes wrong.

Break 1: dependency order

Your app instance depends on a database. Stop them in a random order and starting back up, the app comes alive before the database is ready and lands in a crash loop. On one box you get away with it. Across an environment with app tiers, databases, and caches, ordering is not optional: databases up before apps, apps up before the things that call them. A flat list of instance IDs has no concept of "start this after that." Real scheduling needs dependency-aware sequencing (storage, then compute, then application), with delays between tiers.

Break 2: timezones

The script fires at 20:00. Whose 20:00? As you add teams in different regions, a single UTC cron either shuts down someone's environment in the middle of their afternoon or leaves it running all night. At scale, schedules have to be timezone-aware per environment or per team, not one global time that is wrong for most of the world.

Break 3: no overrides, so people disable it

The night QA needs staging up late for a release, the script kills it at 20:00 anyway. This happens twice, and then someone disables the schedule "just for now." It stays off for a year, and all the savings evaporate. At scale you need a first-class override: a "keep this on until midnight" or "hold this environment" button that anyone can use and that expires on its own. Without a safe override, people protect themselves by turning the whole thing off, and an override that requires editing a cron expression is not a real override.

Related trap: an override with no expiry. Someone forces an environment on during an incident and forgets to remove the hold. The savings quietly disappear because half the fleet is now permanently exempt. Overrides must be time-bounded.

Break 4: no proof it actually fired

The script has no memory. Did last night's shutdown actually run? Did every instance stop, or did three fail because of a state transition and keep billing? On one instance you would notice. On three hundred, a 5% silent failure rate means fifteen instances running 24/7 that you think are scheduled, and nothing tells you. At scale you need reconciliation: compare what was scheduled to stop against what actually stopped, and alert on the gap.

What "scheduling at scale" actually requires

Put those four together and the requirements are clear. Real scheduling is:

Tag-driven discovery, so new resources inherit the schedule automatically instead of being hand-added to a list that rots.
Dependency-aware sequencing across resource types, with delays.
Timezone-aware schedules per environment.
First-class, time-bounded overrides anyone can use.
Reconciliation and alerting so you know it actually fired.
Coverage beyond EC2: RDS (which you stop differently, and which auto-restarts after 7 days), Auto Scaling groups, and more.

AWS Instance Scheduler exists and covers some of this, but it is a solution you deploy and operate. This is also exactly the problem ZopNight was built to handle across resource types and clouds, precisely because the DIY script era taught everyone these four failure modes the hard way. Whatever you use, the point is that "stop the dev instances at night" is a harder problem than the first Lambda makes it look.

The take

The one-instance shutdown script is a great way to start and a terrible way to scale. It breaks on dependency order, timezones, overrides, and silent failures, and each break is invisible until it causes an incident or quietly stops saving money. If your environment has outgrown the single Lambda, you do not need a bigger Lambda, you need scheduling that understands dependencies, timezones, overrides, and proof-of-firing.

Which of the four broke first for you? For most people it is the override problem, one badly-timed shutdown during a release, and the whole schedule gets switched off in self-defense.

GPU Rightsizing Without Breaking Production: G5, G6, P4, P5 and the CUDA Check Nobody Mentions

Muskan Bandta — Fri, 28 Aug 2026 06:38:27 +0000

CPU rightsizing is a solved, well-documented practice. GPU rightsizing is where the real money is now, and almost nobody writes about it, because GPU instances are expensive enough that people are scared to touch them and unsure how. Given how much a GPU box costs per hour, an over-provisioned one is the single most expensive rightsizing mistake in your account. Here is how to rightsize AWS GPU instances without breaking the workload, including the compatibility check that quietly bites people.

Know what each GPU family is for

Rightsizing starts with using the right family, not just the right size. On AWS:

G5 / G6 (NVIDIA A10G / L4): inference, graphics, smaller training. The workhorses for serving models and lighter ML. Cheaper per hour.
P4 / P5 (A100 / H100): large-scale training and heavy inference. The expensive tier, built for jobs that genuinely need the horsepower and interconnect.

The most common GPU waste is running a training-class P-family instance for an inference workload that a G-family instance would serve fine at a fraction of the cost. Wrong family is a bigger error than wrong size.

Rightsize on the binding resource, and it is usually not CPU

GPU workloads have several resources that can be the bottleneck, and CPU utilization, the thing you would check for a normal instance, is often the least relevant:

GPU utilization: is the GPU actually busy, or idle between requests? (CloudWatch does not report this by default; you need the CloudWatch agent with GPU metrics or nvidia-smi telemetry.)
GPU memory: many inference workloads are GPU-memory-bound, not compute-bound. A model that fits in less VRAM can move to a smaller GPU.
Host CPU and RAM: sometimes the GPU is fine but the instance is over-sized on host resources.

The rightsizing signal is a GPU sitting at low utilization or using a fraction of its VRAM over a sustained window (a 90-day-style baseline, same idea as CPU rightsizing). That is your candidate to move down a size or across to a cheaper family.

The CUDA check nobody mentions

Here is the gotcha that turns a clean rightsizing into an outage. Different GPU families use different NVIDIA architectures, which means different CUDA driver and library requirements. Move a workload from, say, an A100-based P4 to an L4-based G6, and the driver version, CUDA toolkit, and framework build that worked on one may not match the other.

If you rightsize the instance but do not verify CUDA compatibility, the box comes up and the workload fails to initialize the GPU, or silently falls back to CPU and runs at a crawl. Before any GPU family change:

Confirm the target family's supported NVIDIA driver and CUDA version.
Confirm your framework build (the CUDA-compiled wheels for your ML library) supports that target.
Test on one instance before you roll the fleet.

This step is the difference between "rightsized and saving money" and "rightsized and paged at 2am because inference is down." It is also why teams avoid GPU rightsizing entirely, they got burned once and never went back. Do the compatibility check and it is safe.

The other GPU cost lever: do not run them idle

Rightsizing the instance matters, but the bigger GPU waste is often time, not size. GPU utilization on "always-on" fleets is routinely far below what people assume, and eval, dev, and training-experiment pools have no reason to run overnight or on weekends.

Schedule non-production GPU pools to scale down off-hours. A 50%-cheaper right-sized GPU still burns money at 3am doing nothing. (We schedule GPU node groups the same way as any other non-prod resource, which is part of what ZopNight does, but a scheduled scale-down gets you the crude version.)
Right-size and schedule together. They compound: the correct family and size, running only when needed, is dramatically cheaper than an oversized one running 24/7.

The take

GPU rightsizing is the highest-value, least-covered cost work in most AI-touched accounts, because the per-hour price makes every mistake expensive. Use the right family (G5/G6 for inference, P4/P5 for heavy training), rightsize on GPU utilization and VRAM rather than CPU, and always run the CUDA compatibility check before a family change so you do not trade savings for an outage. Then make sure the thing is not idling overnight, which is often the bigger win.

Have you rightsized GPU instances, and did the driver/CUDA compatibility catch you the way it caught a lot of us? That check is the step every GPU-cost article skips and every GPU incident includes.

Stop Comparing GPU Clouds Only by $/Hour

Muskan Bandta — Mon, 24 Aug 2026 05:53:30 +0000

Every GPU cloud comparison I see is a table of dollars per hour. Provider A is $2.10 an hour for an H100, provider B is $1.89, the newcomer is $1.40, therefore the newcomer wins. I understand the appeal, one number, easy to sort. It is also how teams end up paying more for GPUs while believing they got a deal, because $/hour is the least important number in GPU economics once you are running real workloads.

Here is what the hourly rate hides, and what I actually compare instead.

The number that matters is $/unit-of-work, not $/hour

You do not buy GPU time to have GPU time. You buy it to train a model, serve inference, run a batch. The honest metric is cost per unit of work done, cost per token served, per training run, per job. A GPU that is 20% cheaper per hour but delivers 40% less throughput on your actual workload is more expensive, full stop.

Throughput per dollar depends on things the hourly rate says nothing about: the exact GPU generation, memory bandwidth, interconnect, and how well your framework is tuned for that hardware. Two "H100" listings can perform differently depending on networking and host. Benchmark your workload, not the spec sheet.

The four costs the $/hour hides

1. Utilization. This is the big one. A GPU billed by the hour costs you the full hour whether it runs at 90% or 9%. If your workload is bursty and the GPU sits idle between bursts, your effective cost per unit of work is a multiple of the sticker rate. A slightly pricier provider with better autoscaling or faster provisioning can beat a cheap one you leave idle. Utilization, not rate, is where most GPU money leaks.

2. Data transfer and storage. GPUs eat data. If your training data or model weights live in one place and your GPUs in another, egress and cross-region transfer can quietly rival the compute bill. The cheap-GPU provider that charges premium egress, or forces you to move terabytes in and out, is not cheap. Price the whole data path, not the accelerator.

3. Provisioning time and cold starts. If it takes minutes to get a GPU node up and pull a large model into memory, and you churn nodes often, you pay that startup repeatedly, in both wall-clock and instance-hours. A provider with instant capacity can deliver more useful work per dollar than a cheaper one you spend half your time waiting on.

4. Availability and eviction. The cheapest GPU capacity is often the least available, spot or preemptible pools that vanish when demand spikes, which right now is often. If evictions kill your jobs and you restart from scratch, the cheap rate bought you expensive re-work. Reliability is a cost input, not a footnote.

The comparison I actually run

Instead of sorting a table by hourly rate, I answer these:

What is the cost per unit of my actual work (token, epoch, job) on this provider, from a real benchmark, not the spec sheet?
What utilization can I realistically hit here, given provisioning speed and autoscaling? A cheap rate at 30% utilization loses to a fair rate at 80%.
What is the fully-loaded path cost, compute plus storage plus egress plus the ops time to run it?
How available is the capacity at the tier I need, and what does an eviction cost me in re-work?

Only after those does the hourly number mean anything, and by then it is usually not the deciding factor.

The FinOps habit underneath this

The reason $/hour is seductive is the same reason idle instances persist: the sticker price is visible and the waste is not. GPU cost discipline is the same discipline as the rest of FinOps, pointed at the most expensive resource you rent. Measure utilization. Kill idle capacity. Schedule non-production GPU pools so they are not burning the sticker rate at 3am doing nothing (we schedule GPU node groups the same way we schedule any other non-prod resource, that is part of what ZopNight handles, but a scheduled scale-down does the crude version). The provider you pick matters less than whether you actually use what you rent.

The take

Comparing GPU clouds by $/hour is like comparing cars by fuel-tank size. It is a real number that tells you almost nothing about what the thing costs to run. Benchmark your workload, price the whole data path, factor utilization and availability, and compute cost per unit of work. Do that and the "expensive" provider often turns out cheaper, and the cheap one turns out to be the one quietly draining the budget.

How do you compare GPU providers, do you benchmark your own workload or trust the spec sheet? I trusted spec sheets exactly once, and the throughput gap on my real job was not close to what the numbers promised.

FinOps Can't Stop at the Cloud Bill Anymore: Tracking AI Token Spend

Muskan Bandta — Mon, 24 Aug 2026 05:53:09 +0000

FinOps grew up managing one kind of cost: cloud infrastructure. Instances, storage, data transfer, the stuff on your AWS or GCP or Azure bill. That scope is now too narrow, because a new operational cost has shown up that behaves differently from everything FinOps was built for, and most teams have no idea how big it is: AI token spend.

The industry conversation this year has been blunt about it. Token usage has become a major operational expense, and organizations still lack a good way to connect that spend to developer productivity or business outcomes. That gap is the whole problem. Let me lay out why AI spend breaks the old FinOps model and what tracking it actually looks like.

Why token spend is not just another line item

You might think, it is on the cloud bill (Bedrock, Vertex) or an API invoice (OpenAI, Anthropic), so treat it like any other cost. It is not that simple, for three reasons:

1. It is usage-metered at a granularity nothing else is. An EC2 instance costs the same whether it does important work or spins idle. A token costs the same either way too, but the volume is driven by application behavior in a way instance-hours are not. One inefficient prompt, one chatty agent, one retry loop, and volume explodes. The cost is coupled to code behavior, not provisioned capacity.

2. It hides across invoices. Some AI spend is on the cloud bill, some is on a separate SaaS-style API invoice, some is buried in a vendor's platform fee. Unlike cloud cost, which at least lives in one console, AI spend is scattered, so nobody sees the total. The first task is often just adding it all up.

3. It resists attribution. Ask "which team's EC2 is this" and tags get you an answer. Ask "which feature generated this token spend" and most teams have nothing. The requests go through a shared API key, and the key does not know which product feature, which customer, or which team is behind each call.

What tracking AI spend actually requires

Bringing AI cost under FinOps is not exotic. It is the same muscles, retrained on a new resource:

Aggregate it first. Pull every source of AI spend into one view: cloud-billed model usage, direct API invoices, platform fees. You cannot manage a number you have not assembled.
Attribute it. Route requests so you can tag them by feature, team, or customer. In practice this means per-team or per-feature API keys, or a gateway that stamps metadata on each call. Without this, every optimization is guesswork.
Right-size the model, not just the instance. The new rightsizing question is "which model does this request actually need." Most requests are over-modeled the way most instances are over-provisioned. Routing easy requests to a cheap tier and reserving the frontier model for hard ones is often the single biggest saving.
Catch runaway usage. A retry loop, a caching bug, an agent stuck in a plan-execute cycle. These are the new "forgot to turn off the GPU box," and they show up as a token-spend anomaly before anyone files a ticket. Anomaly detection on token volume is now as necessary as anomaly detection on cloud spend.

The part the industry is stuck on: connecting spend to value

Here is where AI FinOps is genuinely harder than cloud FinOps, and where nobody has fully cracked it. With infrastructure, "is this worth it" is often obvious, the server runs the product. With AI, the question is sharper: is this feature's token spend producing proportional value? A summarization feature costing more in tokens than it saves users in time is a bad trade, but you only see that if you track spend and outcome together.

The teams getting ahead are the ones instrumenting both sides: cost per feature on one axis, usage or productivity or revenue impact on the other. That is unit economics applied to AI, and it is the direction FinOps has to go. The bill alone cannot tell you whether the spend is smart.

The practical starting point

You do not need a platform to begin. This week:

Add up every source of AI spend into one number. Most people are surprised by the total.
Put the top two or three AI features behind separate keys or a tagging gateway so you can attribute the biggest chunks.
Set one anomaly alert on total token volume so a runaway loop does not become a month-end surprise.
Pick your highest-volume request path and test whether a cheaper model handles it acceptably.

That is a week of work and it moves AI spend from "invisible and growing" to "measured and managed," which is the whole game.

The take

FinOps that stops at the cloud bill is now measuring the wrong boundary. AI token spend is an operational cost with its own behavior, scattered across invoices, coupled to code, and stubbornly hard to attribute, and it is growing fast. The same discipline applies, aggregate, attribute, right-size, alert, but you have to actually point it at tokens, and then take the harder step of tying that spend to value.

Are you tracking AI token spend as a first-class cost yet, with attribution, or is it still landing on invoices nobody reconciles? And has anyone here actually connected a feature's token cost to its value? That last part is where I keep seeing teams stall.

Kubernetes at the Edge Hit a Wall. Fleet Management Is the Way Through.

Muskan Bandta — Mon, 24 Aug 2026 05:52:53 +0000

Kubernetes at the edge has been "the next big thing" for a few years now, and this month the honest assessment showed up in the reporting: edge Kubernetes has hit a wall, and fleet management is emerging as the way through. That matches what I have seen. The moment you go from one cluster in a datacenter to a hundred small clusters in stores, factories, or cell sites, the thing that breaks is not Kubernetes itself. It is the assumption that you manage clusters one at a time.

Here is why the edge breaks the normal model, and what "fleet management" actually means as a fix.

Why one-cluster habits fall apart at the edge

Everything you do comfortably with a single cluster assumes properties the edge does not have:

You assume the cluster is always reachable. At the edge it is not. A retail location loses its uplink, a factory network is flaky, a remote site is offline for hours. Your control plane cannot assume it can reach every cluster on demand.
You assume a handful of clusters. At the edge you have tens or hundreds, sometimes thousands. Anything that requires a human to kubectl into a cluster does not survive contact with 300 of them.
You assume roughly uniform, capable hardware. Edge nodes are often small, heterogeneous, and resource-constrained. A control-plane-heavy distro that is fine in a datacenter is too heavy for a box in a closet.
You assume a physical ops team nearby. There is nobody at the edge site to reboot a node. Recovery has to be automatic or remote.

Individually these are annoyances. Together they mean "manage each cluster" simply does not scale, and that is the wall.

What fleet management actually means

Fleet management flips the unit of operation from the cluster to the fleet. You stop operating clusters and start operating a population of them declaratively. The core ideas:

Declarative, pull-based config. Instead of pushing changes to each cluster, every cluster pulls its desired state from a central source (GitOps at fleet scale). A cluster that was offline reconciles itself when it reconnects. This is the single most important shift, because it removes the "must be reachable right now" assumption.
Group by policy, not by name. You act on labels, "all stores in this region," "all clusters on this app version," not on individual cluster names. Rollouts, config, and policy target groups.
Staged rollouts across the fleet. You never ship to all 300 at once. Canary to a few clusters, watch, then ring out. A bad change that would brick the fleet gets caught at 5 clusters, not 300. This is the same blast-radius thinking that matters anywhere, just applied to a population.
Fleet-wide observability and drift detection. One place to see which clusters are healthy, which are behind on version, which have drifted from desired state. At fleet scale, "which clusters are wrong" is the question you live in.

The tools converging on this

You do not have to build it from scratch. The ecosystem has been moving here: lightweight distros for constrained nodes (k3s and friends), fleet controllers and GitOps at scale (Fleet, Argo CD ApplicationSets, Flux), and managed fleet offerings from the big clouds. The pattern across all of them is the same: declarative desired state, pull-based reconciliation, label-based grouping, staged rollout.

The cost angle nobody mentions

Since I spend my time on cloud cost, here is the part that gets ignored in edge conversations: a fleet is a cost-visibility nightmare by default. A hundred clusters is a hundred places for over-provisioning to hide, and the usual "look at the cluster" cost tooling does not scale to a fleet any better than manual ops does. Cost attribution and rightsizing have to become fleet-level too, or you end up with the same idle-resource waste multiplied by your cluster count. Whatever fleet management you adopt, make sure "what is this fleet costing and where is it wasted" is a question it can answer, not just "is it healthy."

The take

Edge Kubernetes did not fail. The one-cluster-at-a-time way of operating it failed, and that was always going to happen once the cluster count went from one to hundreds. Fleet management, declarative pull-based config, label-based grouping, staged rollouts, fleet-wide visibility, is the model that actually scales, and it is worth adopting the mindset even if your fleet is small today, because the habits that work at 3 clusters are the ones that survive at 300.

If you run Kubernetes across many sites, what broke first for you, the reachability assumption or the cost visibility? For me it was reachability, I built everything push-based and then watched it fall apart the first time a site went dark.

OpenAI Just Paused Its Own Model for Being Too Good at Finding Zero-Days. Read That Again.

Muskan Bandta — Mon, 24 Aug 2026 05:20:43 +0000

The AI story this week that should stop you mid-scroll is not another benchmark or price cut. It is that OpenAI reportedly paused internal development of its Astra model after evaluations found it might be capable of developing zero-day exploits on its own, the first model to trip the "Critical" cybersecurity threshold under their Preparedness Framework. They moved it to isolated testing with government and safety review before anything ships.

Sit with that. A frontier lab looked at its own model, saw it could autonomously find and weaponize unknown software vulnerabilities, and hit the brakes. I am not writing this to fearmonger. I am writing it because I run cloud infrastructure, and this news changes the threat model I have to plan for whether or not Astra ever ships.

Why this is different from "AI writes malware"

We have had "AI can help write malicious code" headlines for two years. This is not that. Writing malware from a known technique is a productivity boost for an attacker. Autonomously discovering a zero-day, a vulnerability nobody knows exists yet, and building a working exploit for it, is a different capability class entirely. It compresses the most expensive, most skilled step in offensive security into compute.

The economics of attacking infrastructure have always rested on scarcity: finding novel vulnerabilities takes rare expertise and a lot of time. If that step becomes something you rent by the hour, the scarcity that quietly protected most of us evaporates. That is the part worth taking seriously.

What it means if you run cloud infrastructure

You are not going to get access to Astra to attack yourself with it, and that is not the point. The point is that capability, once demonstrated, diffuses. Assume that within some number of quarters, attackers have something in this class. What does that change for the boring day job of keeping a cloud account safe?

Honestly, less than you would fear, and it mostly rewards fundamentals you already know you are behind on:

Patch latency becomes the whole game. If exploit development speeds up, your window between a vulnerability going public and it being weaponized shrinks toward zero. The teams that get hurt are the ones with month-long patch cycles. This is not a new lecture, it is the old lecture with the timer sped up.
Attack surface you forgot about is where it lands. Autonomous vulnerability discovery does not care what is on your roadmap. It probes what is exposed. Every forgotten public S3 bucket, every dev environment with a public IP, every abandoned load balancer still routing to something is now a candidate. The orphaned and idle resources I usually talk about as a cost problem are also your unmonitored attack surface.
Detection matters more than prevention. If novel exploits become cheaper to produce, you cannot assume you will prevent every one. What you can do is notice fast. Anomaly detection on API velocity, on unexpected state changes, on spend spikes, is the layer that catches "something is wrong" even when you did not predict the specific something.

The uncomfortable overlap with the cost work

Here is the connection I keep coming back to, and it is genuinely useful, not a stretch. The hygiene that controls cloud cost and the hygiene that controls this kind of risk are the same hygiene. Killing orphaned resources shrinks the bill and the attack surface. Knowing what you actually have running (discovery) is the prerequisite for both a clean bill and a defensible perimeter. Independent monitoring of resource state catches both a runaway cost anomaly and an intrusion that changed something it should not have. I did not expect "do your FinOps" to be a security recommendation, but a well-managed, fully-inventoried, low-waste account is a smaller, more observable target. It is the same discipline pointed at a different bill.

The part I respect

Pausing a model that just cleared a capability bar the whole industry is racing toward is not a small decision, competitively or financially. Whatever you think of the labs, hitting the brakes because your own eval said "this is too dangerous to keep building right now" is the behavior you would want. It is also a signal: the capability is real enough that the people closest to it flinched.

What I am actually doing about it

Nothing dramatic, which is the honest answer. I am tightening patch latency where I can, running a fresh sweep of what we expose publicly (the forgotten stuff, not the documented stuff), and making sure the independent state monitoring covers the accounts I care about. None of it is new advice. The news just moved all of it up my priority list, because the assumption underneath my threat model, that finding novel exploits is expensive and slow, is the assumption that is expiring.

If a model that can autonomously find zero-days becomes real and available, what breaks first in your environment? For me it is patch latency on a couple of long-lived boxes I have been meaning to deal with. This week I will actually deal with them.

ChatGPT and Gemini Both Crossed a Billion Users. The Infrastructure Story Is the One Nobody's Telling

Muskan Bandta — Mon, 24 Aug 2026 05:20:34 +0000

The headline this week is the round number: ChatGPT reportedly crossed a billion weekly users, Gemini is in the same territory within weeks of it, and ChatGPT got there faster than any consumer product in history, under four years for something that took Facebook eight. Everyone is writing about the adoption. Almost nobody is writing about the part that actually keeps me up as an infrastructure person: what does it take to serve a billion people asking an LLM questions all day, and who is paying for it.

Because a billion users is not a marketing stat. It is an infrastructure event, and the economics under it are stranger than the adoption curve makes them look.

A billion users of an LLM is not like a billion users of anything before it

Here is the thing that makes this different from previous billion-user milestones. A billion people loading a web page is a solved problem: caching, CDNs, cheap static serving. The marginal cost of the billionth pageview rounds to zero.

A billion people running LLM inference is the opposite. Every single query is compute. There is no cache for "what is a novel question," and a huge share of these queries are novel. The marginal cost of the billionth inference does not round to zero, it is a real GPU-second on a real, expensive, supply-constrained accelerator. Facebook at a billion users got cheaper per user over time as infrastructure amortized. LLM serving at a billion users has a floor under it made of silicon that is currently the scarcest thing in tech.

That is why this milestone and the aggressive price cuts happening at the same time (GPT-5.6's cheapest tier reportedly dropped to around $0.20 per million input tokens) are in genuine tension. Prices are falling and the absolute compute demand is exploding. Something has to give, and mostly what gives is that the providers eat margin now to own the users, betting the unit economics improve later.

What this looks like from inside a normal company

I do not run ChatGPT. But almost every company I know is now wiring these APIs into their own products, and the billion-user backdrop directly shapes the bill we all get:

You are renting time on a constrained resource. When the whole world is pulling on the same GPU supply, capacity and price are not stable inputs you can plan around for a year. They move. Building a product on inference means building on a cost base that is more volatile than the cloud compute we are used to.
Your "small" AI feature rides the same curve. The chatbot you added to your app is inference too. It does not get the cache discount either. Teams routinely underestimate this because they mentally file it next to "an API call" (cheap) instead of "a compute job" (not cheap, and priced per token).
The cheap-model headline is a trap if you do not route. Prices dropping to $0.20 per million tokens is real, but only for the cheapest tier. Send every request to the frontier model out of habit and you pay the frontier price a billion times over at your own small scale. Model routing (easy queries to the cheap tier, hard ones to the expensive tier) is the single biggest lever most teams have not pulled.

The FinOps discipline this demands

A billion-user backdrop makes AI spend behave like a new kind of cloud cost, and the same FinOps muscles apply, just pointed at tokens and GPU-hours instead of instances:

Attribute it. Which feature, which team, which customer is generating the inference spend? If you cannot answer that, you cannot manage it, same as any cloud cost.
Right-size the model, not just the instance. "Which model does this request actually need" is the new "which instance size does this workload actually need." Most requests are over-modeled the way most instances are over-provisioned.
Watch for runaway loops. An agent stuck retrying, a batch job hammering the API, a caching bug replaying requests. At a billion-user scale of pricing pressure, a runaway inference loop is the new "forgot to turn off the GPU box," and it shows up as an anomaly on the bill before anyone files a ticket.
Schedule and cache what you can. Not every inference is novel. Cache the repeatable ones, batch the deferrable ones, and do not run eval or dev inference pipelines around the clock. The same scheduling logic that sleeps a non-prod database applies to a non-prod inference pipeline.

The take

The billion-user milestone is being reported as an adoption story, and it is one. But under it is an infrastructure story that matters more to anyone building on this: the world just wired itself to a compute-metered service whose marginal cost does not fall to zero, running on the most supply-constrained hardware in the industry, while prices are being cut for strategic reasons that may not last. That is not a stable foundation, it is a moving one, and the teams that treat their AI spend with real FinOps discipline instead of "it's just an API call" are the ones who will not get a nasty surprise when the strategic pricing normalizes.

Are you tracking your inference spend as a real cloud cost line yet, with attribution and model routing, or is it still filed under "API calls we'll look at later"? I suspect most teams are in the second camp, and the billion-user economics are going to make that expensive.

Railway vs Render vs Fly.io vs Just Using AWS: The Real Monthly Cost Once You Outgrow the Free Tier

Muskan Bandta — Thu, 20 Aug 2026 05:33:46 +0000

Every few months a wave of "we left Railway" posts shows up, and every one of them is really about the same moment: the app outgrew the hobby tier and the bill stopped looking cute. So instead of another migration story, here is the thing I actually wanted when I was making this call: an honest look at what Railway, Render, Fly.io, and plain AWS cost once you are past the free tier and running something real.

I am a cloud associate, so my bias is toward understanding the bill rather than avoiding it. Here is how these actually compare.

The tradeoff in one sentence

PaaS (Railway, Render, Fly.io) sells you time: you pay a premium per unit of compute in exchange for not running infrastructure. Raw cloud (AWS) sells you leverage: it is cheaper per unit and infinitely more flexible, in exchange for you doing the ops. The entire decision is where your app sits on the "is my time worth more than the markup" curve, and that point moves as you grow.

Where each one fits

Railway. Fantastic developer experience, deploy-from-git simplicity, great for a small team shipping fast. The catch is the one everyone writes about: usage-based pricing that is gentle at hobby scale and climbs quickly as your always-on services and databases grow. You are paying for the smooth experience, and at low scale that is a great trade.

Render. Similar PaaS comfort with more predictable service-based pricing. Managed databases, background workers, cron, all first-class. Tends to stay reasonable a bit further up the curve than pure usage-based billing, but you are still paying a managed premium and you hit ceilings on control.

Fly.io. The most infrastructure-flavored of the three. Runs your containers close to users, gives you more knobs, and is often cheaper for compute-heavy or globally distributed apps. In exchange you are closer to the metal and doing more yourself, which is either a feature or a cost depending on your team.

Plain AWS. The cheapest per unit and the most flexible, full stop. It is also the most work: you own networking, security, scaling, and the bill's complexity. The per-unit savings are real and large at scale, but they are not free, you pay in engineering time and the risk of misconfiguration.

The honest cost crossover

Here is the pattern I have seen, stated plainly rather than as a fake pricing table (your real numbers depend entirely on your workload):

Hobby / early stage: PaaS wins easily. Your time is the scarce resource, the bills are small, and running your own infra to save a few dollars is a bad trade. Use Railway or Render and move on.
Growing / steady traffic: the crossover zone. This is where the "we left Railway" posts come from. Always-on services and databases on usage-based PaaS pricing start to cost multiples of the equivalent raw compute. If your infrastructure is stable and predictable, the PaaS premium is now buying you less.
Scale / cost-sensitive: AWS (or another raw cloud) usually wins on cost, if and only if you have someone to run it. At this size the per-unit savings dwarf a fractional engineer's time, and you need the flexibility anyway.

The mistake is treating this as a permanent religious choice. It is a stage-dependent decision. The right answer genuinely changes as you grow, and switching when the crossover arrives is not a failure, it is just noticing the trade flipped.

Before you migrate to "save money," read this

Most teams jumping from PaaS to AWS for cost reasons underestimate two things:

The managed services you were getting for free. Railway/Render gave you a database, TLS, deploys, logs, and networking as part of the price. On AWS every one of those is a service you now provision, secure, and pay for separately. The raw compute is cheaper; the fully-loaded stack is closer than the sticker suggests.
The ops time is a real cost. "We save $400/month on AWS" is not a win if it costs you a week of engineering a month to run it. Put a dollar figure on the time before you call it savings.

And if you do move to AWS, the cost only stays lower if you actually do the FinOps: rightsize, schedule non-prod, watch data transfer, and clean up idle resources. AWS being cheaper per unit does not make it cheaper by default, it makes it cheaper if you manage it. A neglected AWS account can easily cost more than the Railway bill you fled.

My take

If you are small, use a PaaS and stop thinking about it. When your bill crosses into "a chunk of an engineer's salary" territory and your infra is stable, price out the AWS version honestly, including the managed services and the ops time, not just the compute. Migrate when the math flips, not because a blog post told you PaaS is a trap. It is not a trap, it is a stage.

Where did the crossover land for you, and did the AWS bill actually come in lower after you added back all the managed pieces? That last part is where a lot of migrations quietly disappoint.