This article was originally published at sivaro.in

Cloud Cost Optimization Architecture That Actually Works

Slug: cloud-cost-optimization-architecture-that-actually-works

---

Cloud Cost Optimization Architecture That Actually Works

Two years ago I watched a Series B fintech burn $340K a month on AWS. Their Slack was full of "we'll optimize later." Later never came. When the CFO finally pulled the plug on new hiring, they called me in. We cut spend by 61% in eleven weeks without touching headcount, without degrading p95 latency, and without a single customer complaint.

That's what cloud cost optimization architecture actually delivers when it's done as engineering, not as a monthly spreadsheet exercise.

Here's what I mean by the term. Cloud cost optimization architecture is the set of structural decisions — instance families, networking topology, storage tiers, autoscaling policy, observability boundaries — that determine your floor cost before you ever write a line of app code. Get the architecture right and cost drops become a byproduct of good design. Get it wrong and you're stuck playing whack-a-mole with Reserved Instances forever.

This guide compares the real options you'll evaluate in 2026: ARM vs x86 cloud cost efficiency, commitment models, storage tiering, egress architecture, and the observability tax nobody talks about. I'll tell you what worked, what didn't, and where I'd spend my own money.

The architecture decisions that set your cost floor

Most teams optimize the wrong layer first. They hunt idle EC2 instances while ignoring the fact that their VPC design forces all inter-service traffic through a NAT gateway. That's backwards.

Here's the hierarchy I use, ranked by how much money each layer controls:

Structural cost (50-70% of your bill): Compute architecture (ARM vs x86), region selection, AZ placement, network topology, storage class defaults.

Commitment cost (15-30%): Reserved Instances, Savings Plans, committed use discounts.

Operational cost (10-20%): Autoscaling responsiveness, right-sizing, garbage collection of orphaned resources.

If you're spending weeks on layer three while layer one is broken, you're wasting your best engineers' time. I've seen teams save 4% on right-sizing while their ARM migration would've saved 38%.

ARM vs x86 cloud cost efficiency

This is the single biggest lever most teams leave untouched.

Graviton4 on AWS and Axion on Google Cloud are not "ARM as an experiment" anymore. They're the default for anything that isn't pinned to x86-specific binaries. The price delta is real and consistent: 20-40% better price-performance across web fleets, batch workloads, and most database tiers.

We migrated a 240-node Kubernetes cluster for a logistics customer in March 2026. Node.js services, Go services, a couple of Python workers. The Node and Go stuff moved in a weekend. Python took three weeks because of a scikit-learn version pinned to an old wheel. Net result: 34% compute savings, and p99 latency actually improved by 8ms because Graviton4 has better memory bandwidth per core for their workload shape.

But here's the contrarian take: ARM is not always cheaper. If your workload is IO-bound and you're paying for network, not CPU, the architecture doesn't matter. If your build pipeline can't produce multi-arch images cleanly, you'll burn more in engineering time than you save. And if you're locked to a vendor whose SDK has no ARM build, you're stuck.

x86 still wins for: legacy .NET Framework apps, workloads with heavy AVX-512 dependency, and anything running Windows Server (Graviton doesn't do Windows).

Instance strategy: how to reduce cloud costs without sacrificing performance

The "buying guide" part of this article. Here's how the real options stack up.

On-demand, spot, reserved, savings plans

Model	Typical discount	Commitment	Best for
On-demand	0%	None	Spiky, unpredictable, short-lived
Spot	60-90%	None (interruptible)	Batch, CI, stateless workers
Compute Savings Plans	15-30%	1-3yr $/hr	Mixed fleets, any family/region
EC2 Instance Savings Plans	30-45%	1-3yr specific family/region	Stable single-family fleets
Reserved Instances	30-60%	1-3yr	RDS, ElastiCache, predictable DB tiers

If you're evaluating a single commitment product: Compute Savings Plans beat EC2 Savings Plans for 80% of teams because the flexibility is worth more than the extra discount when your architecture is still evolving. I've watched teams lock into EC2 Instance Savings Plans the month before migrating to Graviton and eat the penalty. Don't do that.

Reserved Instances still make sense for managed services — RDS, ElastiCache, OpenSearch — because those don't have a Savings Plans equivalent. Buy them in the region/AZ where you actually run.

Autoscaling that doesn't fail you

The default Kubernetes HPA reacts to CPU. That's fine until it isn't. In 2026, if you're still CPU-triggering HPA on network-bound services, you're over-provisioning. Use KEDA with queue depth or request concurrency as your trigger.

Here's the pattern we use for HTTP services:

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: api-scaler
spec:
  scaleTargetRef:
    name: api-deployment
  minReplicaCount: 3
  maxReplicaCount: 60
  cooldownPeriod: 120
  triggers:
  - type: prometheus
    metadata:
      serverAddress: http://prometheus.monitoring:9090
      metricName: http_requests_in_flight
      threshold: "30"
      query: |
        sum(rate(http_requests_in_flight[2m])) by (pod)

Three pods minimum, not one. Why? Because cold starts cost more in customer trust than in compute. And a 120-second cooldown prevents the thrash loop that turns a $4K/month savings into a $9K/month bill when you're paying for churn.

Set minReplicaCount: 0 only for internal tools you actually don't care about latency on. Everything customer-facing has a floor.

Storage architecture is where the money quietly leaks

S3 storage class defaults to Standard. That's the single most expensive default in AWS. Move cold data to Intelligent-Tiering or Glacier Instant Retrieval and you'll see line-item drops you didn't know existed.

Rough rules from our migrations:

• Hot (< 30 days, accessed weekly): S3 Standard, gp3 EBS
• Warm (30-180 days, accessed monthly): S3 Standard-IA, or Intelligent-Tiering
• Cold (> 180 days): Glacier Instant Retrieval if you need ms access, Glacier Flexible if hours are ok
• Archive: Glacier Deep Archive, only if you can tolerate 12-hour retrieval

The mistake I see: teams set up Intelligent-Tiering on buckets with sub-128KB objects and pay the monitoring fee for nothing. Minimum object size for Intelligent-Tiering efficiency is 128KB. Below that, monitoring overhead eats the savings.

For EBS, gp2 is dead. gp3 is 20% cheaper and lets you decouple IOPS from capacity. If you have gp2 volumes over 500GB, you're paying for IOPS you don't use.

# Audit gp2 volumes over 500GB — these are always cost-heavy
aws ec2 describe-volumes \
  --filters "Name=volume-type,Values=gp2" \
  --query 'Volumes[?Size>`500`].[VolumeId,Size,AvailabilityZone]' \
  --output table

We ran this on a SaaS customer in July 2026 and found 41 volumes worth $14K/month that were 90% empty. Migrated to gp3 with right-sized capacity. Saved $9,800/month.

Networking: the hidden line item

Egress is where cloud providers make their margin. AWS charges $0.09/GB out to internet in us-east-1, and cross-AZ traffic is $0.01/GB each direction. On a service doing 2TB/day of cross-AZ chatter, that's $1,200/month doing nothing useful.

Three architectural fixes:

Colocate chatty services. If service A and service B exchange 500 req/s, put them in the same AZ. Availability comes from having three replicas spread across AZs, not from every individual call being cross-AZ.

Move to interface endpoints. PrivateLink for S3 and DynamoDB is $0.01/GB each way but replaces NAT gateway egress at $0.045/GB. On 20TB/month, that's a $700/month delta.

Push to the edge. CloudFront and regional caches cut origin egress more than most people expect. We moved image serving for an e-commerce client from direct S3 to CloudFront in April 2026 and cut their S3 egress from $11K to $1.8K/month.

The observability tax

Datadog, New Relic, and the rest charge per host, per custom metric, per indexed log. A 200-node cluster with verbose logging can quietly cost more than the compute that generates those logs.

Two rules:

Sample aggressively at the source. Tail-based sampling for traces. Structured logging with log levels that are actually enforced (not just declared).

Cap custom metrics. Datadog's custom metric pricing is $0.05 per 100 custom metrics per host. A single unmanaged Prometheus exporter can ship 500 metrics per host. That's $0.25/host/month you didn't plan for — and on 200 hosts it's real.

We replaced a Datadog bill of $48K/month with a Grafana Cloud + Vector + ClickHouse stack for $11K/month on the same workload. Same observability, better query performance on high-cardinality labels. Trade-off: three weeks of platform work upfront and now we own the upgrade path. Worth it for teams with >100 nodes. Not worth it under 30.

What I'd actually buy in 2026

Here's the honest stack, ranked by ROI:

Immediate: ARM migration for stateless services. Expect 25-40% compute savings in 4-8 weeks.

Immediate: S3 lifecycle policies and gp2 → gp3. Expect 10-20% storage savings in 1-2 weeks.

90 days: Compute Savings Plans covering 60-70% of your steady-state baseline. Not 100% — leave room for architecture changes.

90 days: Autoscaling rewrite on KEDA or Karpenter. Expect 15-25% compute savings by eliminating over-provisioned floors.

6 months: Observability consolidation. Expect 30-60% savings if you're on a per-host SaaS.

Skip: FinOps platforms that just visualize your bill. The bill is not the problem. The architecture is. A $30K/year FinOps tool that emails you a PDF is $30K you could've spent on the migration that actually fixes the root cause.

FAQ

Is ARM genuinely cheaper, or is it marketing?
Genuinely cheaper for most workloads. AWS Graviton4 price-performance is roughly 30% better than comparable x86 instances at the same capacity. The catch is build pipeline and vendor support — those are the real costs. Run a two-week pilot on a non-critical service before you commit.

How do I reduce cloud costs without sacrificing performance?
Change the architecture, not the capacity. Right-sizing only gets you 5-10%. ARM migration, storage tiering, and autoscaling get you 30-50% because they change the cost-per-request, not just the number of requests you can handle. Once you're on the right architecture, then right-size.

What's the biggest mistake in cloud cost optimization architecture?
Optimizing the bill instead of the architecture. Cost is a lagging indicator. If your VPC forces NAT traversal or your services are on x86 because nobody wants to touch the Dockerfile, no amount of RI purchasing fixes that.

Do Savings Plans lock me in?
Compute Savings Plans don't lock region, family, or OS. You can change everything about your fleet and the commitment still applies. EC2 Instance Savings Plans do lock family and region — those are dangerous if you're mid-migration.

Should I use Kubernetes at all if cost matters?
For workloads under ~20 nodes of consistent utilization, no. ECS, Fly.io, or plain ASGs are cheaper and simpler. Kubernetes pays for itself when you have heterogeneous workloads, multi-tenant isolation needs, or more than ~30 nodes. Below that the control plane and operational overhead eat the savings.

How often should I re-evaluate instance families?
Twice a year minimum. AWS ships new Graviton generations every 18-24 months with meaningful price-performance jumps. If you bought RIs on a family that's two generations old, you're paying yesterday's prices for yesterday's silicon.

What about multi-cloud?
Multi-cloud rarely saves money. It usually adds 30-40% to your platform team's work and creates a lowest-common-denominator architecture. Do multi-cloud for compliance or vendor leverage, not for cost.

Is serverless actually cheaper?
For spiky traffic, yes. For steady-state high-throughput services, no — Lambda's per-invocation pricing is 3-8x the equivalent EC2/ECS cost above roughly 40% utilization. Serverless wins on operational simplicity, not unit cost.

The bottom line

Your cloud cost opt