This article was originally published at sivaro.in

How to Reduce Cloud Costs Without Sacrificing Performance

Two years ago, a fintech client showed me their AWS bill. $340,000 a month. They'd just signed a three-year commitment for more reserved instances because their CFO wanted "predictability." I asked one question: what percentage of your compute runs below 40% CPU utilization? Nobody knew. We instrumented it. The answer was 71%. They were paying premium prices to run idle silicon at scale.

That's how to reduce cloud costs without sacrificing performance: you don't. Not the way most people think. You stop buying the wrong thing. Most cost-cutting advice treats cloud bills like a coupon-clipping exercise. Wrong framing. Cloud cost is an architecture problem wearing a finance disguise. Fix the architecture, the bill drops. Leave the architecture alone, and you're just negotiating how fast you bleed.

This guide compares the real options — ARM vs x86 migrations, spot and reserved purchasing, GPU scheduling, data egress paths, autoscaling strategies — and tells you which ones actually work. I'll give you numbers from systems I've built at SIVARO, honest trade-offs, and a decision framework you can apply this quarter. Not abstract principles. Actual purchasing decisions.

---

Cloud Cost Optimization Architecture Beats Purchasing Tricks Every Time

Here's the contrarian take. Most teams optimize in the wrong order.

They start with reserved instance commitments, savings plans, and vendor negotiations. That's the last 15% of the answer. The first 60% comes from architecture choices that make the workload fundamentally cheaper to run — before any discount applies.

Three architecture decisions drive most of the savings:

Right-sizing the compute shape. Not "smaller instances." The wrong shape. I've seen n8i.4xlarge instances running Python APIs that never touched a GPU. That's $2,400/month for nothing.

Data locality. Moving 40 TB across regions monthly to feed a model that could run in-region costs roughly $800/month in egress alone on AWS. Same workload, co-located: $0.

Elasticity matching. A service with flat 24/7 traffic and a service with 10x daytime spikes need different purchasing models. Treating them the same wastes 30-40%.

At first I thought cloud cost was mostly a tagging and visibility problem. Then I watched a team with perfect FinOps dashboards still waste $90K/month. They knew where the money went. They just hadn't fixed the architecture generating the spend.

FinOps visibility tells you where. Architecture tells you why. You need both, in that order.

# Quick utilization audit before you do anything else
import boto3
from datetime import datetime, timedelta

def find_idle_compute(region="us-east-1", threshold=40):
    cw = boto3.client("cloudwatch", region_name=region)
    ec2 = boto3.client("ec2", region_name=region)
    idle = []
    for r in ec2.describe_instances()["Reservations"]:
        for i in r["Instances"]:
            if i["State"]["Name"] != "running":
                continue
            metrics = cw.get_metric_statistics(
                Namespace="AWS/EC2",
                MetricName="CPUUtilization",
                Dimensions=[{"Name": "InstanceId", "Value": i["InstanceId"]}],
                StartTime=datetime.utcnow() - timedelta(days=14),
                EndTime=datetime.utcnow(),
                Period=3600,
                Statistics=["Average"],
            )
            points = [p["Average"] for p in metrics["Datapoints"]]
            if points and (sum(points) / len(points)) < threshold:
                idle.append((i["InstanceId"], round(sum(points)/len(points), 1)))
    return idle

print(find_idle_compute())

Run that. If more than 30% of your fleet shows sub-40% average CPU, you have an architecture problem, not a purchasing problem.

---

ARM vs x86 Cloud Cost Efficiency: What the Benchmarks Actually Show

This is the single highest-leverage move most teams haven't finished. Graviton4 and AmpereOne closed the gap. In many workloads, ARM isn't just cheaper — it's faster.

Graviton3 instances deliver up to 40% better price-performance than comparable x86 instances for many workloads, per AWS's own published benchmarks. That's not marketing. I've measured it.

Numbers from our own migrations:

Workload	x86 instance	ARM instance	Cost delta	Latency delta
Go API server	c6i.2xlarge	c7g.2xlarge	-18%	-9%
Postgres read replica	m6i.xlarge	m7g.xlarge	-21%	-3%
Rust event processor	c6i.4xlarge	c8g.4xlarge	-26%	-14%
Python ML inference	m6i.xlarge	m7g.xlarge	-12%	+22%

That last row matters. Python + ARM was a wash on older interpreters. On Python 3.11+ with properly built wheels, it flipped positive. If you're on 3.9 or worse, fix that first.

The rule I use: anything compiled, ARM wins. Anything interpreted with C extensions, test before you commit.

gRPC and protobuf workloads migrate with zero code change. Anything with a vendored binary — some Postgres extensions, a few older ML libraries — needs a rebuild.

# Multi-arch build for a Go service, single command
docker buildx build \
  --platform linux/amd64,linux/arm64 \
  -t registry.example.com/myapi:latest \
  --push .

Now the honest trade-off. ARM savings evaporate if your CI doesn't build multi-arch and you're hand-patching production. I've seen a team "migrate to ARM" and end up with two divergent pipelines and a 6-week maintenance debt. Do the build system first. Then migrate.

For a lot of teams I've worked with, ARM migration alone cuts compute bills 20-25% with no measurable latency regression. That's the cheapest 20% you'll ever find.

---

Spot, Reserved, and Savings Plans: Getting the Mix Right

Most teams buy too much reserved. Here's why: reserved instances bake in a bet on constant demand. Modern services don't have constant demand. Demand is spiky, and every reserved hour you don't use is money you can't recover.

The mix that has worked across the workloads I've run:

• Spot: 40-60% of stateless compute. Non-negotiables are that the workload must be interruption-tolerant and have a graceful shutdown under 2 minutes.
• Savings Plans or reserved: only the minimum baseline you've observed for 6+ months. Not the average. The floor.
• On-demand: everything else. Yes, pay more per hour. You pay less overall.

I built a queue-based worker system in 2024 that ran 100% spot. Interruptions 4-6 times per day. Zero user-visible impact because every job drained cleanly via the SQS visibility timeout. Cost went from $18K/month to $4,200/month for the same throughput.

The trap: teams buy 3-year reserved because it's the cheapest per hour, then their architecture changes, their instance family gets discontinued, and they're stuck.

# Kubernetes spot node pool with automatic fallback
apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: spot-workers
spec:
  template:
    spec:
      requirements:
        - key: karpenter.sh/capacity-type
          operator: In
          values: ["spot", "on-demand"]
        - key: kubernetes.io/arch
          operator: In
          values: ["arm64"]
      disruption:
        consolidationPolicy: WhenEmptyOrUnderutilized
        consolidateAfter: 30s
  limits:
    cpu: "400"
  disruption:
    budgets:
      - nodes: "10%"

Karpenter choosing spot-first with on-demand fallback gives you the price without the 3 AM pager. Do this before you sign anything with a salesperson.

---

Where the Money Actually Leaks: Data Transfer and Storage

Nobody wants to hear this, but your egress bill is probably your worst-kept secret.

AWS charges $0.09/GB for internet egress in most regions. Inter-AZ transfer is $0.01/GB each direction. Get your replicas in the wrong AZ and a chatty service will quietly burn $5-10K/month.

Three fixes that pay for themselves fast:

Co-locate chatty services. Same AZ. Same node pool if you can. Network traffic between pods on the same node is free.

Compress everything cross-AZ. gRPC with gzip for inter-service calls. We saw a 60% reduction in cross-AZ bytes on a service mesh after enabling compression.

Object storage lifecycle rules. This is the boring one that everyone skips. Data older than 30 days that nobody reads belongs in S3 Infrequent Access or Glacier Instant Retrieval. We found 2.3 PB of "hot" storage in one client environment that hadn't been read in 90+ days. Cost to move to Glacier IR: about $3K one-time in API calls. Monthly savings: $28K.

# Terraform: S3 lifecycle that banks real savings
resource "aws_s3_bucket_lifecycle_configuration" "data" {
  bucket = aws_s3_bucket.data.id
  rule {
    id     = "tier-down"
    status = "Enabled"
    filter { prefix = "logs/" }
    transition {
      days          = 30
      storage_class = "STANDARD_IA"
    }
    transition {
      days          = 90
      storage_class = "GLACIER_IR"
    }
    expiration {
      days = 365
    }
  }
}

Storage costs are the least glamorous lever and often the biggest single line-item win. Do it before you touch purchasing.

---

GPU Cost Optimization: The 2026 Problem

Training and serving models is where cloud bills go to die. And in 2026, the situation is weirder than it was two years ago.

H100 and H200 capacity is still constrained in some regions. B200 availability is improving. AMD MI300X is finally competitive for inference at scale. Google TPU v5e is genuinely good for certain workloads.

Here's what I've learned the hard way:

Training and inference want different hardware. Training wants max memory bandwidth and interconnect (NVLink, InfiniBand). Inference wants memory capacity per dollar and good batching throughput. Don't buy the same shape for both.

Reserved GPU is a trap for most teams. Unless you're running the same workload 24/7 for 12+ months, don't commit. GPU generations turn over in 18 months. That A100 you reserved is now worth half.

Inference routing saves 30-50%. Route easy queries to a small model, hard queries to a bigger one. I wrote about this extensively — a two-tier router on a customer support workload cut inference cost from $47K/month to $19K/month with no quality regression on the eval set.

For serving, prioritize:

1. vLLM or TensorRT-LLM over naive transformers inference. 3-8x throughput gain.
2. Continuous batching enabled. Non-negotiable.
3. KV cache offloading to CPU memory for long-context workloads. Free throughput.
4. Spot GPU for batch jobs only. Don't serve user traffic on spot GPUs unless you have graceful failover across regions.

In 2025, we ran a batch embedding job on spot H100s at 70% off on-demand pricing, with checkpointing every 100 batches. Zero restarts caused data loss across 3 weeks of operation.

---

The Build-vs-Buy Call Nobody Wants to Make

Should you build your own cost tooling or buy it?

I'll be direct: buy the visibility, build the enforcement.

Datadog, CloudZero, Vantage, AWS Cost Explorer — pick one and buy it. The cost of building your own tag-compliance and cost-attribution system is higher than the license fee. I've watched three teams try and abandon it.

But build the guardrails in your infrastructure-as-code. Terraform policies, Kubernetes admission controllers, OPA rules that block deploys that violate cost constraints. That's where the money actually stays saved.

# OPA policy: block pods requesting more than 4 CPU without a cost tag
package kubernetes.admission

deny[msg] {
  input.request.kind.kind == "Pod"
  some container in input.request.object.spec.containers
  cpu_limit := container.resources.limits.cpu
  to_number(replace(cpu_limit, "m", "")) > 4000
  not input.request.object.metadata.labels["cost-center"]
  msg := sprintf("Pod %v requests >4 CPU without cost-center label", [input.request.object.metadata.name])
}

That policy alone stopped about $12K/month of accidental overspend at one client. Nobody was malicious. They just didn't have a forcing function.

---

Frequently Asked Questions

Does moving to ARM really help without performance loss?

For compiled languages and modern runtimes, yes — often ARM is faster. Graviton3 delivers up to 40% better price-performance than comparable x86 for many workloads per AWS's published data. For Python with heavy C extensions, test first. The savings are typically 15-25% on compute, which compounds across your whole fleet.

How much can I save with spot instances without breaking things?

40-70% off on-demand for interruption-tolerant workloads. The trick is graceful shutdown under 2 minutes and a durable queue. I've run production systems on 100% spot with zero user-visible incidents. Don't run stateful databases on spot unless they support fast failover.

Is reserved capacity ever the right call?

Yes — for the floor of your steady-state demand, not the average. If you can prove a workload has run at 40+ vCPU constantly for 6+ months, reserve 40. Not 60. The overage is on-demand. This usually lands 20-30% cheaper than all-on-demand with far less risk than 100% reserved.

What's the biggest lie about cloud cost optimization?

That it's a finance problem. It isn't. It's an engineering problem. The best FinOps dashboard in the world can't fix a service that does 40 queries per request when 4 would do. Fix the code first.

How do I reduce cloud costs without sacrificing performance on AI workloads?

Route inference: small model for easy queries, large model for hard ones. Use continuous batching. Enable KV cache offloading. For training, spot with checkpointing. For serving, reserved only if your traffic is genuinely flat — which it usually isn't.

Are Kubernetes autoscalers worth the complexity?

Yes, but use Karpenter or the cluster-autoscaler carefully. The complexity cost is real. If you're running fewer than 20 nodes, plain ASG scaling is simpler and just as good. Above that, Karpenter's consolidation logic routinely saves 20-35% by packing workloads tighter.

Does multi-cloud actually save money?

Rarely. It saves negotiating leverage, sometimes. But you pay for it in operational complexity, duplicated tooling, and team cognitive load. I've seen roughly equal numbers where the savings were offset by the cost of two platforms. Go multi-cloud for resilience or regulation, not for the bill.

How often should I revisit my cloud cost architecture?

Every quarter for the mix (your traffic patterns shift). Every 18 months for the hardware (instance families turn over). Every week for tagging drift — human habits degrade. Set a recurring calendar block. Cost hygiene is a habit, not a project.

---

Putting It Together: How to Reduce Cloud Costs Without Sacrificing Performance

Here's my recommendation stack, in order of impact:

1. Instrument first. Run the utilization audit. Find the 30% of your fleet running idle.
2. Migrate compiled workloads to ARM. 15-25% off compute, minimal effort.
3. Fix data movement. Co-locate chatty services, compress cross-AZ, lifecycle your object storage.
4. Right-size the purchasing mix. Spot for elastic, reserved for only the floor, on-demand for the rest.
5. Route AI inference. Two-tier, continuous batching, KV offload.
6. Add guardrails in IaC. Not dashboards — admission policies.
7. Reassess quarterly. Every architecture drifts.

None of this is sexy. And none of it is "just sign this savings plan and we'll fix your bill." I've watched teams try the purchasing-first approach for years and land at 8-12% savings with a 3-year lock-in hangover. Teams that fix the architecture first routinely hit 35-50% reductions without changing a single user-facing metric.

If you're serious about how to reduce cloud costs without sacrificing performance, treat it like an engineering roadmap, not a sales negotiation. The bill is a symptom. The architecture is the disease.

Audit. Migrate. Compress. Purchase. Enforce. Repeat.

That's the whole game. There's no secret lever. There's just doing the boring, careful, high-leverage work — and not signing a three-year commitment before you've done it.

---

Nishaant Dixit — Founder of SIVARO. Building data infrastructure and production AI systems since 2018. Built systems processing 200K events/sec.