DEV Community: Rajesh Medampudi

Why I Rewrote Four Services in Go

Rajesh Medampudi — Sun, 06 Sep 2026 15:51:52 +0000

I had four small services. Each one was a Model Context Protocol adapter — a thin wrapper that lets an AI agent call out to some external thing. One talked to Replicate for image generation. One talked to a Nostr-friendly social poster. One was a Git-aware research helper. One was a Tavily-powered web search.

They were all written in Python. They all ran on Knative on a small Kubernetes cluster. They all worked. And they were all just slightly too slow to use.

A six-second cold start is fine for nothing. It is the precisely wrong amount of time — slow enough to be noticed, fast enough to feel almost loaded. An AI agent waiting six seconds for a single tool call does not know it is waiting for a cold start; it just knows the tool is sluggish. The user does not know either. The user just thinks the agent is broken.

And six seconds was a good day. Some of the services took longer.

So I rewrote them in Go. This is what that cost me, and what the measurements actually were before and after.

The actual problem

Cold starts on serverless platforms are an old problem with a well-known shape. The platform spins your container up only when traffic arrives, so the first request after an idle period pays the full startup tax — image pull (or warm cache hit), container start, language runtime initialisation, application bootstrap.

For Python, application bootstrap is where the bill arrives. The interpreter has to start. import statements run. The dependency tree gets walked. If you have ever wondered why a hello world Flask app feels so much heavier than a hello world Go binary, this is why. Python is doing real work before your code runs. Go has already started.

On a small Kubernetes cluster — small as in I am paying for it personally — you do not keep a fleet of warm replicas around. You scale-to-zero. You scale-to-zero because that is the entire point of using serverless on small infrastructure. The trade-off is that every idle service eats a cold start the next time it is invoked.

For my four MCPs, next time it is invoked was approximately every time an agent decided to use them. Which was constantly. Which meant cold starts were not the rare edge case. They were the common case.

The measured cold-start latency on each Python MCP, taken from production:

postiz-mcp — 5.9 seconds
git-mcp — 4.9 seconds
research-mcp — 5.4 seconds
image-mcp — 7.2 seconds

For comparison, the one MCP I had already written in Go (clickup-mcp) was cold-starting in 200 to 400 milliseconds. An order of magnitude faster, on the same cluster, in the same Knative configuration.

But the headline number was the workflow chain. A real agent run — drafting a blog post — hits four MCPs sequentially in its draft phase. On Python, that meant 20 to 25 seconds of cumulative cold-start latency every time the chain ran cold. On Go, the same chain came down to roughly 2 seconds.

Twenty seconds of staring at a loading spinner, every time, in steady state. That was the ceiling I was bumping into.

The rewrite

I would like to tell you the rewrite was elegant. It was not. It was four small slogs, in a row, on weekends, with the same set of decisions made four times in a row because I was too lazy to extract the boilerplate properly until service three.

The actual work, per service, looked like this.

Setting up the project. Pick a Go module name, decide on a directory structure, pick an HTTP framework or just use the standard library, pick a logger. Repeat per service. (I eventually settled on net/http plus slog. The number of choices Go gives you for stdlib HTTP is zero, and that is why it is faster than the equivalent in any other ecosystem.)

Translating the SDK calls. The Python services used native Python SDKs for Replicate, the Nostr toolkit, and so on. Go either had a vendor SDK, a community one, or — in two cases — nothing usable. In those two cases I just hand-rolled the HTTP requests because the APIs were small enough. This took less time than I expected. Most modern APIs are thin wrappers over REST and JSON, which is exactly what net/http and encoding/json are built for.

Translating the state. Two of the services held a small amount of in-memory state — caches, mostly. Python made this trivial; Go made me think about it. In both cases the right answer was use sync.Map and stop overengineering.

Containerising. This is where the win started becoming visible. A multi-stage Go Dockerfile (golang:1.25-alpine builder going to a FROM scratch final) produces a static binary in an image of about 20 megabytes. The Python images had been about 300 megabytes with all their dependencies. Pull times went from "perceptible" to "negligible." This is a real cold-start contribution that has nothing to do with language runtime — it is image-pull latency, which is dominated by image size.

Wiring observability. The Python services had structured logging via Loguru. The Go services got slog with the same JSON output. Same dashboards still worked. Same alerts still worked. Nobody in the observability stack knew or cared that the language had changed, which is a very nice property for a rewrite.

The total time, across four services and a lot of starting-and-stopping, was something like three weekends. None of them were full weekends. None of them were free of context-switching to other work. So call it twenty hours, generously.

The unexpected wins

A few things landed in my lap that I had not planned for.

No more Python image dance. Building Python container images involves a lot of dancing — the right base image, the right Python version, the right system packages, the right pip cache strategy, the right --no-deps if you have already got a vendored wheel directory. Half the Dockerfiles in my old repo were Python wheel-and-glue. The Go equivalents are six lines and they all look the same.

Single binary deploys. When the only artefact your service produces is a compiled binary, deployment becomes a COPY instruction in a FROM scratch image. There is no virtualenv. There is no missing system library. There is no "works on my laptop, fails on the cluster" because you have shipped exactly the same bytes.

Easier to read in production. Go's lack of magic — its terrible-by-design lack of expressiveness — turned out to be a virtue when I was on-call at two in the morning trying to figure out why an MCP was throwing 500s. There is one way to do error handling. There is one way to do concurrency. There is one way to read a JSON body. The code is boring, and the boring code was easier to debug.

Smaller blast radius for dependency updates. Go modules with go.sum produce reproducible builds. A go mod tidy followed by a build verifies the entire dependency graph. When you update a Python package, you find out at runtime whether it was a breaking change. When you update a Go package, you find out at compile time. This is not a small difference at two in the morning.

The unexpected pains

Three things hurt that I had underestimated.

Boilerplate. Go has a verbose-by-design philosophy. Where Python lets me read a JSON request body in two lines, Go takes six. Where Python has list comprehensions, Go has for loops. Where Python has decorators, Go has wrapper functions. Multiply this by the surface area of four services and you write a lot of code that, in Python, was implicit. I learned to like this. But I had to learn.

Error handling discipline. if err != nil is the most-mocked line in Go and it deserves the mockery — until you spend an afternoon debugging a Python service that swallowed an exception three layers deep. Go's every error is in your face model is exhausting to write and reassuring to read. I now know exactly what every one of my services does when replicate.Run() returns an error. I did not know that for the Python equivalents.

The library gap on niche packages. For Replicate and Tavily, the Go SDKs were either community-maintained or non-existent, and I rolled my own HTTP. That is fine if the API is small. It is painful if the API is large or undocumented. One of my services lost a weekend to what does this API actually do when you pass it an empty string for a required field research that I would not have had to do if a vendor-maintained SDK existed.

The numbers

These are the actual measurements from before and after — taken on the same cluster, the same Knative configuration, the same workload.

Cold-start latency — Python services were measuring 4.9 to 7.2 seconds (about 6 seconds average across the four). The Go rewrites came in at 200 to 400 milliseconds. Roughly 20× faster at the per-call level.
Memory at idle — Python services were sitting at about 80 MiB each. Go binaries sit at about 15 MiB. Across five services, that is the difference between 400 MiB always reserved and 75 MiB always reserved — and on a small cluster with limited CPU per VM, this is the kind of footprint that matters.
Image sizes — about 300 MB Python images became about 20 MB Go images. 15× smaller, with proportional improvements to image-pull latency and registry storage churn.
The headline number — workflow chain latency. A real agent run that chains four MCPs sequentially in its draft phase used to eat 20–25 seconds of cumulative cold-start tax every time the chain ran cold. After the rewrite, the same chain takes about 2 seconds end-to-end. That is the user-facing win, and it is the one I actually care about. The rest is plumbing.

The agents, which were the actual customers, started working better. Tool-call chains that had felt sluggish now feel responsive. That is the only metric I deeply care about. Everything else is vanity.

When you should not do this

You should not rewrite a Python service in Go if:

The service is fast enough. If your cold start is already under a second and you are not running on a small cluster, this is not your problem. Do not make it your problem.
You are doing data science. Python's library ecosystem for ML, statistics, image processing, anything-with-numpy is incomparable. Go is not where your model-serving service should live.
You are using a heavy framework that Go does not have. If your Python service is built on Django or some specific async framework with deep ecosystem dependencies, the Go port is not a port — it is a rewrite. Different math.
You are a Python team. If you do not already have Go in production, the operational cost of introducing a new language for one service is not worth the latency win on that service. Pick your battles.

I had four small adapters with simple request-response shapes, no heavy library deps, and a latency floor that mattered. That is the exact shape Go was good for.

What I would do differently

If I had to do this over, I would extract the boilerplate before the second service, not the third. Two-thirds of each MCP is the same — HTTP setup, JSON request-and-response handling, MCP protocol scaffolding, error envelopes, structured logging. By service three I had a small internal library that did all of that. By service four it was a copy-paste. Services one and two had a lot of code I would have happily deleted in retrospect.

I would also keep at least one Python service around as a comparison baseline, so when someone asks "could you not have just optimised the Python," I have a same-shape service to point to. (I can confidently tell you the answer is no, not at the cold-start level, but I cannot show you, which is annoying.)

The bigger meta-lesson — small services on a small cluster are a different optimisation problem than big services on big infrastructure. On EKS with a fleet of warm replicas, none of this matters; keep your Python. On a six-node cluster scaling-to-zero between requests, every second of cold start is a second the user is staring at a loading spinner. Pick the language that respects the loading spinner.

I respect the loading spinner now. The agents do too. The four services do not time out any more.

That is the whole win, in one sentence.

Cloud Cost Management: Your Bill Is a Product Metric

Rajesh Medampudi — Sun, 06 Sep 2026 15:51:49 +0000

The cloud bill is the only number in most companies that nobody on the team owns until it's already a problem.

Engineering owns latency. It owns error rates, p99, uptime, the whole observability wall. Finance owns the invoice. And between those two ownerships there's a gap wide enough to drive a fifth to a third of your cloud spend straight into a wall — which, across the industry, is roughly what happens. The fix isn't a smarter spreadsheet at month-end. Real cloud cost management isn't an accounting function at all — the fix is to stop treating the bill as accounting and start treating it as a product metric: cost per request, cost per tenant, cost per feature, sitting on the same dashboard as latency and error rate, owned by the same people who own those numbers.

That's the whole argument. The rest of this post is why it's true and how it's done.

The bill is a lagging accounting artifact, and that's the bug

Here's how cloud cost is treated almost everywhere. A bill arrives. Someone in finance reconciles it against a budget. If it's higher than expected, a thread gets opened, an engineer gets pulled in, and everyone spends a week spelunking through Cost Explorer trying to reconstruct why a number that's already been spent is what it is. Then it happens again next month.

Every part of that loop is broken. The signal arrives weeks after the decision that caused it. The person reading the signal can't act on it. The person who can act on it never sees it. And the unit of measurement — total dollars — tells you nothing about whether the spend was good. A bill that doubled because you doubled revenue is a triumph. A bill that doubled because someone left a debug log streaming to an expensive tier is a fire. Total dollars can't tell those two apart. They look identical on the invoice.

This is the same mistake we'd never make with any other production signal. Nobody reviews latency once a month from a PDF. Nobody waits for finance to tell engineering that p99 regressed. We put it on a graph, we attach it to the deploy that moved it, we alert when it crosses a line. Cost is the one production signal we still run like a 1990s expense report. The waste isn't an accident — it's the structural consequence of measuring the wrong thing, late, in front of the wrong people.

What changes when cost becomes a unit metric

The unlock is dividing. Instead of asking "what did we spend," you ask "what did we spend per unit of the thing the business actually sells." Total infra cost over the number of requests gives you cost per request. Over active tenants, cost per tenant. Over inferences, if you're running models, cost per inference. The FinOps Foundation — the industry body that codified this practice — calls this capability Unit Economics, and defines it plainly: it "brings together what an organization spends on technology and the value that technology spending creates" (FinOps Foundation, Unit Economics capability).

The arithmetic is trivial. The shift it forces is not.

Once cost is per unit, a rising bill stops being alarming by default. If cost per request is flat and the bill is up, you grew — celebrate. If the bill is flat but cost per request is climbing, you have a real problem hiding behind a calm-looking invoice, and you found it before finance did. The unit metric separates the two failure modes the raw total fused together. That separation is the entire point.

The Foundation splits these into two useful buckets. Resource-efficiency metrics — cost per GB stored, cost per vCPU, cost per GB transferred, cost per token — tell engineers whether the machinery is tight. Business metrics — cost per tenant, cost per transaction, cost to serve, cost per case resolved — tell the business whether the product makes money at the unit level (FinOps Foundation, Unit Economics capability). You want both. The first tells you how you're wasting; the second tells you whether it matters.

I'll be honest about the limit here: getting a clean cost-per-tenant number in a genuinely multi-tenant system, where tenants share clusters, share databases, share a NAT gateway, is hard. Allocation is the unglamorous, real work of this whole discipline — tagging, cost-allocation keys, splitting shared infrastructure on a defensible ratio. Anyone who tells you cost-per-tenant falls out of the bill for free hasn't built it. But "hard to attribute perfectly" is not "not worth approximating." An 80%-right cost-per-tenant on a dashboard beats a 100%-right total invoice nobody reads. (The hands-on version of finding where the money actually goes is what I'd audit first on a $50K AWS bill.)

Put it on the dashboard, or it isn't real

A metric that lives in a monthly finance review is not a metric the people who move it ever see. This is the part most "we do FinOps" claims quietly skip.

Cost per request belongs on the same Grafana board as latency and error rate — same screen, same refresh, looked at by the same on-call engineer at the same moment. Not because engineers should obsess over money, but because the cost of a code path is a property of that code path, exactly like its latency. An engineer who can see that a new endpoint costs 4x per call what the old one did will fix it in the pull request, while the context is hot, for the price of a code review. The same regression caught six weeks later in a finance reconciliation costs a forensic investigation, a context-switch back into code nobody remembers, and a meeting. Same bug. Two orders of magnitude difference in what it costs to fix, decided entirely by when and where the number was visible.

The FinOps framework names this the Inform phase — make cost, usage, and efficiency data visible and timely before you try to optimise anything (FinOps Foundation, FinOps Phases). Visibility first. You cannot optimise a number nobody can see, and you cannot create ownership of a number that only appears in someone else's department's PDF.

Showback before chargeback

Once cost is per-unit and visible, the next question is who carries it. Two models, and the order you adopt them matters.

Showback shows each team what its slice of the bill is — without billing them for it. Chargeback actually moves the cost onto the team's own budget. Most engineering orgs that reach for chargeback first end up in a turf war: teams dispute the allocation, argue the shared-infra split is unfair, and the energy that should go into reducing cost goes into contesting it instead.

Showback first. Let teams see their number for a quarter or two with no money attached. Visibility alone moves behaviour, because most over-spend isn't malice — it's invisibility. The team running the over-provisioned cluster usually doesn't know it's over-provisioned; they've just never seen the number isolated to them. Show it, and a meaningful fraction self-corrects before anyone has to enforce anything. Chargeback is the tool you reach for after showback has done the easy 60%, when you need accountability with teeth on the stubborn remainder. Lead with the budget transfer and you'll spend your political capital on the dispute instead of the fix.

This is a culture metric, not a tooling metric

You can buy every cost tool on the market and still waste a quarter of your spend, because the tools surface the number and the culture decides whether anyone acts on it. FinOps, in the Foundation's own framing, is "a cultural practice" — collaboration between engineering, finance, and product, with everyone taking ownership of their own technology usage (FinOps Foundation, Framework Overview). The tool is the easy part. The hard part is making cost a thing engineers are proud to have tight, the way they're proud of a clean p99.

And the stakes scale with the bill. Globally, Flexera's 2025 State of the Cloud report found 84% of organisations name managing cloud spend as their top cloud challenge (vendor: Flexera, 2025 State of the Cloud press release), and the long-running industry estimate of wasted spend sits at roughly a fifth to a third — a range that's barely moved in years. That's not a rounding error. On a serious cloud bill, a quarter of it evaporating is the difference between a profitable product and one that's quietly subsidising its own infrastructure. The companies that fix it aren't the ones with the fanciest dashboards. They're the ones where the engineer who wrote the expensive query saw the number, owned it, and shipped the fix in the same afternoon — because the cost was sitting right there next to the latency, where it always should have been.

Treat the bill as accounting and you'll reconcile it forever. Treat it as a product metric and you'll engineer it — which is the only thing that's ever actually moved the number.

Sources

FinOps Foundation — Unit Economics capability (definition; resource-efficiency vs business unit metrics; cost-per-X list). https://www.finops.org/framework/capabilities/unit-economics/
FinOps Foundation — Framework Overview (FinOps as a cultural practice; collaboration between engineering, finance, business; ownership principle). https://www.finops.org/framework/
FinOps Foundation — FinOps Phases (Inform / Optimize / Operate; Inform = make cost/usage/efficiency data visible and timely). https://www.finops.org/framework/phases/
FinOps Foundation — Introduction to Cloud Unit Economics working group (unit cost = total infra cost ÷ units produced; cost per request, per tenant, per inference). https://www.finops.org/wg/introduction-cloud-unit-economics/
Flexera — 2025 State of the Cloud, press release (84% name managing cloud spend as top challenge). https://www.flexera.com/about-us/press-center/new-flexera-report-finds-84-percent-of-organizations-struggle-to-manage-cloud-spend

The S3 Cost Optimization Playbook

Rajesh Medampudi — Sun, 06 Sep 2026 15:51:46 +0000

Most S3 bills are wrong, and the fix takes an afternoon. The data sits in the most expensive class AWS offers (S3 Standard, $0.023/GB-month), nobody set a lifecycle policy, incomplete multipart uploads are silently billing for storage you can't even see in the console, and every byte your EC2 fleet pulls from S3 is routed out through a NAT Gateway when a free VPC Gateway Endpoint would do the same job for $0. None of this needs an architecture rewrite. It needs a checklist run in the right order.

Here is the order. The savings depend entirely on your access pattern — I will not promise you a number I can't see — but the mistakes below are so common that the question is usually how much, not whether. One number is just arithmetic: cold data that moves from S3 Standard ($0.023/GB-month) to Glacier Deep Archive ($0.00099/GB-month) drops about 96% on the storage line for those bytes, and on an observability platform I ran — logs aged past 90 days into Deep Archive — that is exactly the lever that did the work. S3 cost optimization is the same boring discipline as the rest of the bill: see it, then decide what each byte should actually cost.

First, see the bill before you touch it

You cannot optimize what you cannot measure, and S3's default billing view tells you nearly nothing useful. Turn on S3 Storage Lens before anything else. The free tier gives you 62 metrics at the bucket level with 14 days of history, and crucially it includes cost-optimization metrics out of the box — including "Incomplete multipart upload bytes greater than 7 days old," which is the single most common source of money disappearing into storage nobody knows exists (AWS S3 Storage Lens docs, accessed 2026-06-18).

Storage Lens free metrics answer the three questions that decide everything that follows:

Which buckets hold the most bytes?
What storage class is that data sitting in right now?
Where are the incomplete multipart uploads?

For deeper per-prefix analysis or a longer history, Advanced metrics (15 months of data, recommendations) costs extra — worth it for a large estate, overkill for a single small account. Start free. Pay for advanced only once the free tier has told you the estate is big enough to justify it.

Pair this with S3 Storage Class Analysis on your busiest buckets. It watches access patterns and tells you which objects are candidates to move to Infrequent Access — so you set lifecycle thresholds from data, not from a guess.

Second, fix the silent leak: incomplete multipart uploads

This is the one nobody finds on their own, so it goes near the top.

When you upload a large object in parts and the upload fails partway — a dropped connection, a crashed job, an SDK that didn't clean up — the parts that did land stay in the bucket. You are billed for that storage. They do not appear in the normal object listing. They accumulate for years, and on a long-lived account the orphaned parts can add up to a meaningful fraction of the bill before anyone notices.

The fix is one lifecycle rule, applied to every bucket:

{
  "Rules": [
    {
      "ID": "abort-incomplete-mpu",
      "Status": "Enabled",
      "Filter": { "Prefix": "" },
      "AbortIncompleteMultipartUpload": { "DaysAfterInitiation": 7 }
    }
  ]
}

AWS supports a lifecycle rule that stops multipart uploads not completed within a set number of days and deletes the orphaned parts — and it applies to both existing and future uploads (AWS lifecycle config for incomplete MPU, accessed 2026-06-18). Seven days is the standard value; it is long enough that no legitimate in-flight upload gets killed, short enough that garbage doesn't pile up. And per AWS, removing incomplete multipart parts via lifecycle does not trigger early-delete charges — so there is no downside.

Set this rule on every bucket you own, today, before you do anything else. It is the cheapest win in the whole playbook.

Third, get the data into the right storage class

S3 storage classes span a roughly 23x price range, and the only thing separating them is access pattern. Here is the ladder, US East (N. Virginia), per GB-month:

Class	$/GB-month	Min duration	Min object size	Use it for
S3 Standard	$0.023	none	none	Active, frequently read data
Standard-IA	$0.0125	30 days	128 KB	Read a few times a month
One Zone-IA	~$0.01	30 days	128 KB	Reproducible IA data, single-AZ OK
Glacier Instant Retrieval	$0.004	90 days	128 KB	Archive needing instant access
Glacier Flexible Retrieval	$0.0036	90 days	—	Archive, minutes-to-hours retrieval
Glacier Deep Archive	~$0.00099	180 days	—	Compliance, rarely-ever read

Storage prices and the minimum-duration / minimum-size rules are from AWS primary docs (storage classes, Glacier classes, accessed 2026-06-18). Two rules carry most of the risk:

Minimum storage duration. Delete a Standard-IA object before 30 days and you still pay the full 30. Glacier Flexible and Instant bill a 90-day minimum; Deep Archive bills 180. Move data down the ladder only when it will actually sit there. Putting short-lived data in Glacier is a way to pay more, not less.
Minimum billable object size. Standard-IA, One Zone-IA, and Glacier Instant bill every object as if it were at least 128 KB. A bucket of 10 KB thumbnails moved to Standard-IA gets billed at 128 KB each — you pay for 12x the bytes you store. Small objects stay in Standard. This is the trap that quietly reverses the savings.

The decision rule is simple. Frequently read → Standard. Read occasionally, objects over 128 KB → Standard-IA. Rarely read but must be instant → Glacier Instant Retrieval. Archive you can wait minutes-to-hours for → Glacier Flexible. Compliance data you'll likely never read → Deep Archive.

Fourth, automate the transitions with lifecycle policies

You do not move data by hand. You write a lifecycle policy and S3 does it on a schedule. A typical policy for log or backup data:

{
  "Rules": [
    {
      "ID": "tier-down-logs",
      "Status": "Enabled",
      "Filter": { "Prefix": "logs/" },
      "Transitions": [
        { "Days": 30,  "StorageClass": "STANDARD_IA" },
        { "Days": 90,  "StorageClass": "GLACIER_IR" },
        { "Days": 180, "StorageClass": "DEEP_ARCHIVE" }
      ],
      "Expiration": { "Days": 2555 }
    }
  ]
}

Two things to know before you ship a lifecycle policy:

Transitions cost money per request. Each lifecycle transition is a billed request, and the per-1,000 transition cost is higher for the colder classes (vendor: AWS S3 pricing — confirm the exact us-east-1 cents on the live page, the table is JS-rendered and these move). For a bucket of millions of tiny objects, the transition requests can cost more than the storage you save. This is the second reason small objects don't belong in IA — the move itself isn't worth it.
Expiration is the most underused line. If the data has a legal or practical end-of-life, set Expiration. Storage you delete is storage you stop paying for forever. Most teams tier data down and then keep it for eternity because nobody wrote the expiry rule.

Fifth, when access is unpredictable, use Intelligent-Tiering

Lifecycle policies assume you know the access pattern. When you don't — user uploads, a data lake, anything where some objects go cold and others stay hot unpredictably — S3 Intelligent-Tiering is the right default. It monitors each object and moves it between tiers automatically: after 30 consecutive days with no access it drops to Infrequent Access (about 40% cheaper), and after 90 days to Archive Instant Access (about 68% cheaper), with no retrieval fee when an object gets hot again and is read (AWS Intelligent-Tiering, accessed 2026-06-18).

The cost is a monitoring-and-automation charge of $0.0025 per 1,000 objects per month (vendor: AWS, us-east-1). That math has one sharp edge:

Intelligent-Tiering is wrong for billions of tiny objects. The monitoring fee is per object, not per GB. A bucket of a billion small objects pays a monitoring charge that can dwarf any tiering savings. Per AWS, objects smaller than 128 KB are never auto-tiered and are always billed at the Frequent Access rate — but they can still rack up monitoring charges. For huge counts of small objects, a plain lifecycle policy (or just Standard) beats Intelligent-Tiering.

Rule of thumb: unknown access pattern + reasonably sized objects → Intelligent-Tiering and forget it. Known pattern, or billions of tiny objects → explicit lifecycle policy.

Sixth, stop paying for retrieval and requests you didn't budget for

Storage is the line everyone watches. Requests and retrievals are the lines that ambush you.

Retrieval fees scale with how cold the class is. Standard-IA and Glacier Instant charge a per-GB read fee; Glacier Flexible and Deep Archive charge a per-GB retrieval fee plus a per-request fee, and Deep Archive's slowest retrieval tier is measured in hours, not seconds (vendor: AWS — confirm the exact per-GB cents and the retrieval-time SLA on the live S3 pricing and retrieval-options pages). The lesson: a class is only cheap if you read it as rarely as its design assumes. Putting frequently-read data in Glacier to save on storage and then paying retrieval on every read is the most expensive mistake in this whole document — the retrieval bill can exceed what Standard would have cost outright.
Request pricing punishes chatty workloads. S3 Standard GETs are cheap individually (~$0.0004 per 1,000) but a service doing millions of tiny GETs per minute turns "cheap" into a real line item. Batch, cache, and use CloudFront in front of read-heavy buckets so the requests never hit S3.

Before you tier anything down, ask the one question that governs the whole ladder: how often is this actually read? If you don't know, that's what Storage Class Analysis and Intelligent-Tiering are for. If you guess wrong toward "cold," the retrieval fees make you pay for the guess.

Seventh, the free win everyone leaves on the table: VPC Gateway Endpoints

If your EC2, ECS, or Lambda workloads in a VPC talk to S3, check how that traffic leaves the VPC. By default, instances in a private subnet reach S3 through a NAT Gateway — and NAT Gateway bills both an hourly charge and a per-GB data-processing charge on every byte. For a workload pulling terabytes from S3, that is a tax you are paying for nothing. (The full version of that problem is its own post: the NAT Gateway hidden tax.)

An S3 Gateway VPC Endpoint routes that traffic privately, and AWS charges nothing for it — no hourly fee, no per-GB fee, and traffic to S3 in the same Region incurs no data transfer charge (AWS Gateway endpoints for S3, accessed 2026-06-18). You add a route, and the same S3 traffic that was flowing through a metered NAT Gateway now flows free.

One caveat worth stating honestly: Gateway endpoints work for traffic originating inside the VPC in the same Region. They do not serve on-premises networks, peered VPCs in other Regions, or transit-gateway paths — those need an Interface endpoint, which does cost money ($0.01/AZ/hour plus $0.01/GB). For the common case — instances in a VPC reading from S3 in the same Region — the Gateway endpoint is free and you should have created it on day one.

The order matters

Run it top to bottom: turn on Storage Lens, kill incomplete multipart uploads, right-size storage classes, automate with lifecycle (or Intelligent-Tiering when the pattern is unknown), respect the retrieval and request fees, and add the free VPC Gateway Endpoint. Every step is reversible, none of it touches your application code, and the whole thing is an afternoon's work for savings that compound every month the data sits there.

The reason this works is not cleverness. It is that S3's defaults are tuned for the most expensive, most available configuration, and almost nobody changes them. The money is sitting in plain sight. You just have to run the checklist.

Sources

AWS — Object Storage Classes (storage prices, min sizes): https://aws.amazon.com/s3/storage-classes/ — accessed 2026-06-18
AWS — Understanding S3 Glacier storage classes (min durations 90/90/180): https://docs.aws.amazon.com/AmazonS3/latest/userguide/glacier-storage-classes.html — accessed 2026-06-18
AWS — Understanding and managing S3 storage classes: https://docs.aws.amazon.com/AmazonS3/latest/userguide/storage-class-intro.html — accessed 2026-06-18
AWS — S3 Intelligent-Tiering (30d→IA 40%, 90d→Archive Instant 68%, no retrieval fee, 128 KB rule): https://aws.amazon.com/s3/storage-classes/intelligent-tiering/ — accessed 2026-06-18
AWS — How S3 Intelligent-Tiering works: https://docs.aws.amazon.com/AmazonS3/latest/userguide/intelligent-tiering-overview.html — accessed 2026-06-18
AWS — Configuring lifecycle to delete incomplete multipart uploads (AbortIncompleteMultipartUpload, 7 days, no early-delete charge): https://docs.aws.amazon.com/AmazonS3/latest/userguide/mpu-abort-incomplete-mpu-lifecycle-config.html — accessed 2026-06-18
AWS — S3 Storage Lens metrics & recommendations (62 free metrics, 14 days, incomplete MPU cost metric): https://docs.aws.amazon.com/AmazonS3/latest/userguide/storage_lens_basics_metrics_recommendations.html — accessed 2026-06-18
AWS — Gateway endpoints for Amazon S3 (no charge, same-Region no data-transfer charge, caveats): https://docs.aws.amazon.com/vpc/latest/privatelink/vpc-endpoints-s3.html — accessed 2026-06-18
AWS — S3 pricing page (storage / request / retrieval tables; live source of truth): https://aws.amazon.com/s3/pricing/ — accessed 2026-06-18
AWS — VPC pricing (Interface endpoint $0.01/AZ/hr + $0.01/GB): https://aws.amazon.com/vpc/pricing/ — accessed 2026-06-18

AWS Savings Plans vs Reserved Instances: Which to Buy

Rajesh Medampudi — Sun, 06 Sep 2026 15:51:44 +0000

For AWS Savings Plans vs Reserved Instances, the default answer is: buy a Savings Plan, not a Reserved Instance. The exception is OpenSearch, Redshift, and (until December 2025) databases, which still need the older Reserved model. That is the whole decision. For most teams a Compute Savings Plan is the right default: same discount as a Convertible Reserved Instance, far less to manage, and it follows your workload across instance families, regions, Fargate, and Lambda. The cases where a Reserved Instance still wins are narrow and specific, and the December 2025 launch of Database Savings Plans shrank them further.

This post is the decision — RI vs Savings Plan, when each wins. It is not a deep-dive on how Savings Plans work under the hood; I cover the mechanics — how the $/hour commitment gets applied, the billing-hour math, the queue order against On-Demand — in a separate post. Here I only want to answer the question you actually have when the Cost Explorer recommendation pops up: which one do I buy? (Commitments are step three of a full bill audit — where they sit in the order is in what I'd audit first on a $50K AWS bill.)

The short version

Both Reserved Instances and Savings Plans are the same trade: you promise AWS a one- or three-year commitment, AWS gives you a discount over On-Demand. The difference is what you commit to.

A Reserved Instance commits you to a specific instance configuration — family, and depending on type, size, region, OS, tenancy.
A Savings Plan commits you to a dollar amount of usage per hour (e.g. "$10/hour of compute"), and AWS applies that discount to whatever matching usage you actually run.

The Savings Plan is the more flexible instrument at the same discount level, which is why AWS itself now recommends Savings Plans over Reserved Instances for compute. (AWS, Compute Savings Plans and Reserved Instances, accessed June 2026.) The Reserved Instance has not gone away — but for EC2 compute, it is mostly the legacy choice now.

The discount numbers, side by side

The headline rates line up almost exactly. From AWS's own comparison:

Instrument	Max discount vs On-Demand	Flexibility
Compute Savings Plan	up to 66%	EC2 + Fargate + Lambda, any family, any region
EC2 Instance Savings Plan	up to 72%	one instance family in one region, any size/OS/tenancy
Convertible Reserved Instance	up to 66%	exchangeable, but manual
Standard Reserved Instance	up to 72%	locked to configuration, best rate

(AWS, Compute Savings Plans and Reserved Instances, accessed June 2026.)

That table is the entire argument. The Compute Savings Plan matches the Convertible RI's discount (up to 66%) but applies across instance families, regions, and serverless compute, automatically, with no exchanges. The EC2 Instance Savings Plan matches the Standard RI's discount (up to 72%) while still giving you size, OS, and tenancy flexibility within the family you committed to.

So at every discount tier the Savings Plan gives you more flexibility for the same rate. That is why the default flipped.

Standard vs Convertible RI — the old trade-off, briefly

If you are still considering Reserved Instances, the choice between the two types is the same shape: more discount for less flexibility.

Standard RIs give the deepest discount (up to 72% off On-Demand) but you cannot change what you reserved — you can only modify size within a family (for Regional RIs) and sell unused ones on the Reserved Instance Marketplace. (AWS, Standard vs. Convertible offering classes, accessed June 2026.)
Convertible RIs give a smaller discount (up to 66%) but you can exchange them for RIs with different attributes — different family, OS, tenancy — by performing a manual exchange. (AWS, Standard vs. Convertible offering classes, accessed June 2026.)

Here is the catch that makes Savings Plans the better tool for most teams. The Convertible RI's whole reason to exist is flexibility, and the Compute Savings Plan does that better with no manual exchanges. The Standard RI's reason to exist is the deepest rate, and the EC2 Instance Savings Plan matches it while staying more flexible. So for EC2, the RI is rarely the right answer in 2026.

When a Reserved Instance still wins

The Savings Plan does not cover everything. These are the real cases where you still reach for an RI or a Reserved Node:

1. Databases, before December 2025. Until very recently, RDS, Aurora, ElastiCache, Redshift, and OpenSearch had no Savings Plan at all — Reserved Instances (or Reserved Nodes) were the only way to get a committed-use discount on them. This is the single biggest reason teams still had large RI portfolios. The December 2025 launch changed most of this — see the next section.

2. Redshift and OpenSearch — still Reserved-only. Even after December 2025, Amazon Redshift uses Reserved Nodes and Amazon OpenSearch Service uses Reserved Instances as their committed-discount model. Neither is covered by any Savings Plan today. (AWS, Amazon Redshift reserved nodes; AWS, Reserved Instances in Amazon OpenSearch Service, accessed June 2026.) If your bill is heavy on either, the Reserved model is not legacy — it is the only lever you have.

3. Capacity guarantees. A zonal Reserved Instance reserves capacity in a specific Availability Zone. A Savings Plan does not reserve capacity at all — AWS is explicit that "Savings Plans doesn't provide capacity reservations." (AWS, Compute Savings Plans and Reserved Instances, accessed June 2026.) If you need a guarantee that the instance will be available in a constrained AZ — not just discounted — you want a zonal RI or an On-Demand Capacity Reservation, not a Savings Plan.

4. The Reserved Instance Marketplace. Standard RIs can be sold to other AWS customers if your needs change. Savings Plans cannot be cancelled or sold — "[they] can't be cancelled during the term." (AWS, Compute Savings Plans and Reserved Instances, accessed June 2026.) If you genuinely expect to need an exit and are willing to take a haircut, a Standard RI has a resale path a Savings Plan does not.

Outside of these, the Savings Plan is the better instrument.

What the December 2025 Database Savings Plans changed

On 2 December 2025, AWS launched Database Savings Plans, finally bringing the Savings Plan model to managed databases — the gap that kept most database spend stuck on Reserved Instances. (AWS, Introducing Database Savings Plans for AWS Databases, accessed June 2026.)

What it covers. The supported services are: Amazon Aurora, Amazon RDS, Amazon DynamoDB, Amazon ElastiCache, Amazon DocumentDB, Amazon Neptune, Amazon Keyspaces, Amazon Timestream, and AWS Database Migration Service (DMS). (AWS, Introducing Database Savings Plans, accessed June 2026.)

What it does NOT cover. Note two absences that matter: Amazon Redshift and Amazon OpenSearch Service are not on the list. They remain Reserved-Node / Reserved-Instance only. So if you read a summary that says Database Savings Plans cover OpenSearch, it is wrong — check the official service list. Redshift and OpenSearch are exactly the two database-family services where the RI/Reserved-Node model is still the only committed-discount option.

The discount. Database Savings Plans save up to 35% on serverless deployments and up to 20% on provisioned instances, with smaller tiers for DynamoDB and Keyspaces throughput. (AWS, Introducing Database Savings Plans, accessed June 2026.) These are lower than EC2's headline numbers — databases were always a shallower discount — but the flexibility is the same draw: the plan applies regardless of engine, instance family, size, deployment option, or region, so you can move an Aurora workload from db.r7g to db.r8g, shift regions, or modernise from RDS for Oracle to Aurora PostgreSQL and keep the discount. (AWS, Introducing Database Savings Plans, accessed June 2026.)

The terms — read this before you commit. As launched, Database Savings Plans are one-year term, No Upfront payment only. (AWS, Announcing Database Savings Plans, accessed June 2026.) There is no three-year option and no upfront-payment option at launch — which is a meaningfully different shape from EC2 Savings Plans and from database Reserved Instances, both of which offer 1- or 3-year terms and All/Partial/No Upfront. If you have been buying 3-year database RIs for the deepest rate, the Savings Plan does not yet replace that specific play. Available in all AWS Regions except China. (AWS, Announcing Database Savings Plans, accessed June 2026.)

The practical effect: for RDS, Aurora, ElastiCache, DocumentDB, Neptune, DynamoDB, Keyspaces, Timestream, and DMS, you now have a flexible Savings Plan option that did not exist before December 2025, and for steady-state database spend on a 1-year horizon it is usually the easier instrument to manage than database RIs. For Redshift and OpenSearch, nothing changed — keep buying Reserved Nodes / Reserved Instances.

The decision, step by step

Here is how I would route the decision today:

Is it Redshift or OpenSearch? → Reserved Node (Redshift) or Reserved Instance (OpenSearch). No Savings Plan exists. Done.
Is it another managed database (RDS, Aurora, ElastiCache, DynamoDB, DocumentDB, Neptune, Keyspaces, Timestream, DMS)? → Database Savings Plan (1-year, No Upfront) for steady-state spend. Database RIs only if you specifically need a 3-year term or upfront payment for a deeper rate.
Is it EC2 / Fargate / Lambda compute?
- Do you need flexibility across instance families and regions, or you run Fargate/Lambda? → Compute Savings Plan (up to 66%).
- Is your usage concentrated and stable in one instance family in one region, and you want the deepest rate? → EC2 Instance Savings Plan (up to 72%).
- Do you need a capacity guarantee in a specific AZ, or a resale exit? → Reserved Instance (zonal for capacity; Standard for the Marketplace).
Everything else (steady compute, no special constraint)? → Compute Savings Plan. It is the safe default.

The honest summary: Savings Plans are the default for almost everything now, Database Savings Plans closed the biggest remaining gap in December 2025, and Reserved Instances survive in three specific corners — Redshift/OpenSearch, capacity guarantees, and the resale exit.

One caveat I will state plainly: I have not personally run a Database Savings Plan commitment for a full year yet — it launched in December 2025 and I am writing this in 2026. The EC2 Savings Plan guidance is from production use; the database guidance is from the AWS documentation and the launch terms, which I have verified and cited above. Treat the database section as "what the docs say and what I would do," not "what I have run for twelve months."

Sources

AWS — Compute Savings Plans and Reserved Instances (comparison table, discounts, flexibility, "no capacity reservations", "can't be cancelled"): https://docs.aws.amazon.com/savingsplans/latest/userguide/sp-ris.html — accessed June 2026.
AWS — Standard vs. Convertible offering classes (Standard up to 72% / Convertible up to 66%, exchange vs no-exchange): https://docs.aws.amazon.com/whitepapers/latest/cost-optimization-reservation-models/standard-vs.-convertible-offering-classes.html — accessed June 2026.
AWS — Introducing Database Savings Plans for AWS Databases (launch 2 Dec 2025, supported services list, up to 35% serverless / 20% provisioned, engine/family/region flexibility): https://aws.amazon.com/blogs/aws/introducing-database-savings-plans-for-aws-databases/ — accessed June 2026.
AWS — Announcing Database Savings Plans with up to 35% savings (1-year term, No Upfront only, all Regions except China): https://aws.amazon.com/about-aws/whats-new/2025/12/database-savings-plans-savings/ — accessed June 2026.
AWS — Amazon Redshift reserved nodes (Redshift remains Reserved-Node only): https://docs.aws.amazon.com/redshift/latest/mgmt/purchase-reserved-node-instance.html — accessed June 2026.
AWS — Reserved Instances in Amazon OpenSearch Service (OpenSearch remains Reserved-Instance only; 1/3-year, No/Partial/All Upfront): https://docs.aws.amazon.com/opensearch-service/latest/developerguide/ri.html — accessed June 2026.
AWS — Compute and EC2 Instance Savings Plans pricing (1 or 3 year terms, up to 66% / 72%): https://aws.amazon.com/savingsplans/compute-pricing/ — accessed June 2026.

AWS NAT Gateway Pricing: The Hidden Tax, and How to Kill It

Rajesh Medampudi — Sun, 06 Sep 2026 15:51:41 +0000

If your AWS bill has a NAT Gateway line, you are paying twice for the same packet: once for the gateway to merely exist, and again for every gigabyte it carries. The fix for most teams is dull and free. Add an S3 and a DynamoDB gateway endpoint, route the heavy traffic away from NAT, and only then argue about anything fancier. That single change is free to turn on, takes minutes, and stops the most expensive traffic from ever touching the meter.

This is a playbook, not a lecture. The trick with NAT Gateway pricing is that the two charges hide in different places on the bill, so most teams only ever see half of it. Numbers first, then the fixes, in the order I would actually do them.

What you are actually being charged for

NAT Gateway has two charges, and people forget the second one until they read the bill closely.

Hourly charge — you pay for every hour the gateway is provisioned and available, whether or not a single byte moves through it. In us-east-1 (N. Virginia) and us-east-2 (Ohio) this is $0.045 per NAT Gateway-hour. That is roughly $32.85 a month per gateway just to keep the lights on. Partial hours bill as full hours.
Data processing charge — you pay $0.045 per GB processed through the gateway, in the same region, on top of the hourly charge. This applies to every gigabyte, inbound or outbound, regardless of source or destination.

And then there is the part the pricing page mentions almost in passing: standard AWS data transfer charges still apply on top. NAT processing is an extra meter on traffic you were already paying to move.

The hourly charge is fixed and visible. The per-GB charge is the one that catches teams out, because it scales with traffic you mostly cannot see: package installs, container image pulls, S3 reads from private subnets, telemetry shipped out, cross-region calls. The rate varies by region (it runs higher in places like São Paulo, where both the hourly and per-GB rates sit around $0.093), so check your own region rather than trusting the Ohio number. (AWS VPC Pricing; NAT Gateway pricing docs, accessed June 2026.)

Why it hides

Three reasons the NAT line stays invisible until it is large.

It is bundled with traffic you think is "free." A private subnet pulling a Docker image from a public registry, or an app reading from S3 over the public S3 endpoint, looks like ordinary egress. Every one of those gigabytes is also a NAT data-processing gigabyte at $0.045.

It scales with success, not with headcount. You provision one NAT Gateway on day one and forget it. Traffic grows with users and deploys; the per-GB charge grows with it, silently, while the line item stays named the same boring thing.

Cross-AZ traffic gets routed through it the expensive way. If your workloads sit in one Availability Zone and your NAT Gateway sits in another, that traffic crosses an AZ boundary to reach NAT and crosses back — racking up inter-AZ data transfer charges in addition to NAT processing. AWS's own guidance is explicit: keep resources in the same AZ as the NAT Gateway, or run a NAT Gateway in each AZ that has resources. (NAT Gateway pricing docs, accessed June 2026.)

How to spot it on your bill

Before you fix anything, measure. Fifteen minutes in the console tells you whether this is a $30 problem or a $3,000 one.

Cost Explorer — group by Usage Type and filter for NatGateway-Hours and NatGateway-Bytes. The first is your fixed cost (number of gateways × hours). The second is the one worth chasing.
VPC Flow Logs — turn them on for the NAT'd subnets and look at where the bytes go. If a large share of destinations are S3 or DynamoDB IP ranges, that is your easiest win: those bytes should never touch NAT.
The smell test — a single NAT Gateway carrying multiple TB a month, in an account doing heavy S3 or container work, almost always means you are paying NAT to do a job a free endpoint should be doing.

The fixes, in the order I would do them

1. Add S3 and DynamoDB gateway endpoints — free, do this today

This is the first move and it is not close. Gateway VPC endpoints for Amazon S3 and DynamoDB carry no hourly charge and no data-processing charge. AWS states it plainly: there are no data-processing or hourly charges for using Gateway-type VPC endpoints. (AWS VPC Pricing; Gateway endpoints docs, accessed June 2026.)

A gateway endpoint adds a route to your subnet's route table so traffic to S3 or DynamoDB goes over a private AWS path instead of out through NAT. Every gigabyte you move to that path is a gigabyte that stops costing you $0.045 of NAT processing. If you run anything S3-heavy from private subnets — backups, data pipelines, log shipping, static asset reads — this one change can take a meaningful bite out of the bill for the price of a route-table edit.

There is no real downside. Create one in any VPC that talks to S3 or DynamoDB. The only catch worth knowing: gateway endpoints work from inside the VPC only — they do not serve on-premises networks, peered VPCs in other regions, or transit gateways. For those, you need the interface type below.

2. Add interface (PrivateLink) endpoints for the next-heaviest services

Gateway endpoints exist for exactly two services. Everything else — ECR (container image pulls), CloudWatch Logs, SSM, Secrets Manager, STS, Kinesis, and so on — uses interface endpoints, which run on AWS PrivateLink and are not free.

Interface endpoint pricing: $0.01 per hour per endpoint, per Availability Zone, plus $0.01 per GB processed (tiered down for very large volumes — $0.006/GB past 1 PB, $0.004/GB past 5 PB). (AWS PrivateLink pricing, accessed June 2026.)

So an interface endpoint is roughly $7.30 a month per AZ in hourly cost, plus a per-GB rate that is less than a quarter of NAT's $0.045/GB. The decision is arithmetic:

High-volume service traffic (think ECR image pulls on every deploy, or chatty CloudWatch Logs) — the $0.01/GB endpoint beats the $0.045/GB NAT path easily, and you stop paying twice.
Low-volume, occasional traffic — the per-AZ hourly fee can cost more than just letting it ride over NAT. Do the math per service. A handful of MB a month does not justify $7.30 × number-of-AZs in standing charges.

The classic win here is ECR. Container-heavy accounts pull large images through NAT on every deploy and autoscale event; moving ECR (plus its S3 backing layer via the gateway endpoint, and CloudWatch Logs) onto endpoints is often where the NAT bill collapses.

3. Fix the cross-AZ routing

If your NAT Gateway lives in one AZ and your workloads in another, you are paying inter-AZ transfer on top of NAT processing. Either co-locate the workloads with the gateway, or run one NAT Gateway per AZ that has resources so traffic never crosses a boundary to reach it. Per-AZ NAT Gateways cost more in fixed hourly fees but can be cheaper overall once cross-AZ transfer is large — again, measure before deciding. (NAT Gateway pricing docs, accessed June 2026.)

4. Question whether you need NAT at all

A surprising number of private subnets only ever talk to AWS services and a short list of known external hosts. Once S3, DynamoDB, ECR, and your logging/secrets services are on endpoints, what is actually left going to the public internet? Sometimes the honest answer is "almost nothing," and the NAT Gateway is $32.85/month of standing charge guarding a trickle. If a workload needs no inbound and only talks to AWS, it may not need NAT or even a public path at all.

5. Replace NAT with a NAT instance (fck-nat) — only with eyes open

For the cost-sensitive, the open-source pattern is fck-nat: a self-managed NAT instance on a small ARM box instead of the managed gateway. On a t4g.nano it runs at roughly $3–4 a month, supports up to ~5 Gbps of burst traffic (the EC2 egress cap), and — the real saving — charges no per-GB processing fee, because it is just an EC2 instance forwarding packets. Reported savings run up to ~90%+ versus managed NAT. (fck-nat project, accessed June 2026.)

But be honest about what you are buying. The managed NAT Gateway is fully managed and highly available within its AZ; fck-nat is an instance you own. By default it has no automatic failover — if the instance dies, that subnet's egress dies until it recovers. You handle patching, monitoring, and scaling. The project's own authors recommend the managed gateway for workloads that need five-nines uptime. (fck-nat project, accessed June 2026.)

So the rule I would apply: fck-nat for dev, staging, and internal/batch workloads where a few minutes of egress downtime is survivable. Managed NAT Gateway for production paths where it is not. This is the one fix on the list with a real operational cost; do not reach for it before you have done the free endpoints, which carry no trade-off at all.

The order is the point

The reason to do these in sequence is that the cheap, no-downside fixes capture most of the savings. Gateway endpoints are free and risk-free; do them first. Interface endpoints are cheap and a simple cost calculation; do them next, per service. Cross-AZ and architecture cleanups are free but need a little thought. fck-nat is the only one that trades money for operational responsibility, so it goes last and only where the trade is acceptable.

Most teams never need to get past step two. The S3 and DynamoDB endpoints alone are free, take minutes, and remove the single most common reason the NAT line creeps up. If you read only one sentence of this and act on it, make it that one.

NAT is usually the first place I look on a mid-size bill — the broader order I work an account in is what I'd audit first on a $50K AWS bill, and the S3 side of the same traffic problem is the S3 cost optimization playbook.

Sources

Amazon VPC Pricing — NAT Gateway $0.045/hr + $0.045/GB (us-east, Ohio); gateway endpoints for S3/DynamoDB have no hourly or data-processing charges. https://aws.amazon.com/vpc/pricing/
Pricing for NAT gateways (VPC User Guide) — two-charge model; same-AZ recommendation to avoid cross-AZ transfer; guidance to use interface/gateway endpoints for AWS-service traffic. https://docs.aws.amazon.com/vpc/latest/userguide/nat-gateway-pricing.html
AWS PrivateLink Pricing — interface endpoint $0.01/hr per AZ + $0.01/GB (tiered to $0.006/GB past 1 PB, $0.004/GB past 5 PB). https://aws.amazon.com/privatelink/pricing/
Gateway endpoints (PrivateLink docs) — S3/DynamoDB gateway endpoints, no PrivateLink, no additional charge; VPC-internal only (no on-prem / cross-region peering / transit gateway). https://docs.aws.amazon.com/vpc/latest/privatelink/gateway-endpoints.html
fck-nat (open-source NAT instance AMI) — t4g.nano ~$3-4/mo, ~5 Gbps burst, no per-GB processing fee, no default auto-failover; authors recommend managed NAT for five-nines. https://github.com/AndrewGuenther/fck-nat and https://fck-nat.dev

AWS Cost Optimization: What I'd Audit First on a $50K Bill

Rajesh Medampudi — Sun, 06 Sep 2026 15:51:38 +0000

Give me read access to a $50,000/month AWS account and I will tell you within a day where the first 20-30% is hiding, because on a mid-size bill it is almost always hiding in the same four places, in the same order: data transfer you can't see in the console, instances sized for a load test that ran two years ago, on-demand pricing on a baseline that never moves, and storage rotting in the most expensive class AWS sells. None of this needs an architecture rewrite. AWS cost optimization, at least the first and biggest pass of it, is just the bill read in the right order by someone who knows where AWS buries the meter.

This is the order I work. It is the same audit I run on every account I'm handed, and it is the offer — if you want me to run it on yours, the post ends with how. But you can run most of it yourself today, and you should, because nobody is going to care about your bill as much as you do.

A note before the recipe: I deal in ranges, not promises. The exact saving on your account depends on what you've built. What I can promise is that the mistakes below are common enough that the question is usually how much, not whether.

Hour zero: get the real bill, not the dashboard

Before touching a single resource, I want the granular data. The AWS console's cost dashboard rounds, groups, and hides the things that matter. Two tools give you the truth.

Cost Explorer, with rightsizing recommendations turned on, is the fast view — group by service, then by usage type, and the bill stops being one big number and starts being a list of decisions. Resource-level and hourly granularity costs extra ($0.01 per 1,000 usage records per month), but for one audit pass it's worth pennies.

The Cost and Usage Report (CUR) is the ground truth — line-item, hourly, every charge AWS makes, delivered to your own S3 bucket. Generating it is free; you pay only the few cents of S3 storage. If you're going to do this seriously, set up CUR (now delivered via AWS Data Exports) on day one. Everything below is a query against it.

The first thing I look at isn't a resource. It's the shape of the bill: what fraction is compute, what fraction is storage, what fraction is the line most people never read — data transfer.

First place I look: data transfer (the invisible 15%)

This is first because it's the one nobody instruments, and on a networked workload it's frequently the single most wasteful line. AWS charges for moving bytes, and the meter runs in places the console never surfaces.

NAT Gateway. This is the one I find money in most often. A NAT Gateway costs $0.045 per hour just to exist, plus $0.045 for every GB it processes (us-east-1). The hourly charge is trivial; the per-GB charge is where it hurts. If your private-subnet instances pull container images, packages, or — the classic — objects from S3 through the NAT Gateway, you are paying 4.5 cents a GB to route traffic that should be free. A VPC Gateway Endpoint for S3 and DynamoDB costs nothing and takes that traffic off the NAT path entirely. I check this on every account, and on data-heavy ones it's often the biggest single line-item fix. (The full NAT teardown — both charges, every fix — is its own post: the NAT Gateway hidden tax.)

Cross-AZ traffic. AWS charges $0.01/GB in each direction for data crossing Availability Zones in the same region — $0.02 round trip. Spread a chatty app and its database across three AZs for "high availability" and you can pay a real tax on every query. Sometimes the HA is worth it. Often the chattiness is an accident of where things got scheduled, and pinning the hot path to one AZ (while keeping failover) cuts the line without cutting resilience.

Data transfer out to the internet. First 100 GB/month is free, aggregated across all services and regions; after that it's $0.09/GB up to 10 TB. If you're serving meaningful traffic straight off EC2 or an ALB, CloudFront in front often costs less per GB and offloads the origin — the egress math frequently pays for the CDN by itself.

The reason this is step one: data transfer is the only major cost that doesn't show up as a resource you can point at. You have to read it out of the bill on purpose.

Second: rightsizing — the load test that never ended

Now the obvious one, done properly. Most over-provisioning isn't malice; it's an m5.2xlarge somebody picked for a launch-day spike that never came back, running at 8% CPU ever since.

AWS Compute Optimizer is free and does the heavy lifting. It analyses CloudWatch metrics and gives rightsizing recommendations for EC2, Auto Scaling groups, EBS volumes, Lambda, and ECS-on-Fargate (and, as of 2026, RDS and idle-resource recommendations for things like NAT Gateways too). Default lookback is 14 days; pay a small per-resource fee for Enhanced Infrastructure Metrics and it'll look back ~93 days, which I'd do before resizing anything seasonal — you don't want to shrink a box right before its busy month.

My discipline here: I trust the direction of the recommendation, not the exact target. Compute Optimizer is right that the box is too big; whether you drop one size or two depends on headroom you understand and it doesn't. Rightsizing is also the one step you can get wrong — under-provision a latency-sensitive service and you've traded a cost problem for an outage. Move one size at a time, watch the metrics, repeat.

Two rightsizing moves that aren't just "smaller":

Move to gp3 EBS. gp2 is $0.10/GB-month; gp3 is $0.08/GB-month — 20% cheaper, by AWS's own number — and gp3 includes 3,000 IOPS and 125 MB/s baseline free. For the overwhelming majority of volumes, gp3 is the correct default and gp2 is just an older, costlier setting nobody changed. io2 is for genuine high-IOPS workloads only.
Move to Graviton. AWS's ARM64 chips run at materially lower cost than comparable x86 — the current Graviton page cites up to 20% less cost (the older "up to 40% better price-performance" figure was the Graviton2-generation claim). If your stack is interpreted or already cross-compiles cleanly (most Go, Java, Python, Node services do), the migration is often a base-image change and a redeploy. On the observability platform I ran on EKS — Grafana LGTM, 6 TB/day for 15 departments and 200+ developers — moving the fleet to ARM Graviton2 under Karpenter took 40% off the compute line, with no performance loss. Java compiled cleanly to ARM64; the work was node-pool config and a redeploy, not a rewrite.

Third: commitments — stop paying on-demand for a baseline that never moves

Rightsize first, commit second — never the other way round, or you'll buy a commitment for capacity you're about to delete.

Once the fleet is the right size, look at the baseline that runs 24/7. Paying on-demand for steady-state compute is leaving the largest predictable discount on the table.

Savings Plans are the current answer for compute. A Compute Savings Plan gives up to 66% off and — this is the point — applies across EC2, Fargate, and Lambda, across regions, instance families, and operating systems. You commit to a dollar-per-hour spend, not a specific instance, so it keeps applying as your fleet changes. An EC2 Instance Savings Plan goes deeper, up to 72%, in exchange for locking to an instance family in a region. Standard Reserved Instances also reach up to 72% but are rigid; for EC2, Savings Plans have largely superseded them.

The trap I see: people assume Savings Plans cover everything. Compute Savings Plans do not cover databases. For that, AWS launched a separate Database Savings Plans product (December 2025) covering RDS, Aurora, ElastiCache (Valkey), OpenSearch, DynamoDB and more — up to ~35%. Redshift remains Reserved-Node-only. So on a real bill you may need two or three commitment instruments, not one. Map them to the steady-state portion of each — not the peak, or you'll over-commit and pay for unused commitment.

Rule of thumb I hold to: commit to the floor, pay on-demand (or Spot) for the spikes. Start with a 1-year, no-upfront Compute Savings Plan sized to your reliable baseline. You can always commit deeper once you trust the number. (Which instrument to buy — and why Savings Plans beat Reserved Instances for most teams — is the Reserved Instances vs Savings Plans decision guide.)

Fourth: storage and the things nobody deleted

Storage is rarely the biggest line, but it's the easiest free money, because most of it is waste nobody is defending.

S3 in the wrong class. S3 Standard is $0.023/GB-month — the most expensive tier — and most data sitting in it hasn't been read in months. For unpredictable access, S3 Intelligent-Tiering moves objects between tiers automatically with no retrieval fees and no operational overhead (you pay $0.0025 per 1,000 objects/month for monitoring). For known-cold data, a lifecycle policy to Standard-IA ($0.0125), Glacier Instant Retrieval ($0.004), or Glacier Deep Archive (~$0.001) cuts the line by 5-20x. Standard at $0.023 vs Deep Archive at ~$0.001 is not a rounding difference; it's a 20x difference on the same bytes. (The whole S3 lever — classes, lifecycle, multipart cleanup, retrieval fees — is the S3 cost optimization playbook.)

The graveyard. Three things bill silently:

Unattached EBS volumes keep billing per GB-month after the instance they served is long gone.
Old EBS snapshots — $0.05/GB-month each — accumulate forever unless something deletes them.
Public IPv4 addresses. Since 1 February 2024, AWS charges $0.005/hour for every public IPv4 address, attached or not — about $3.60/month each. On an account with a sprawl of idle Elastic IPs and load balancers, this adds up to a line worth reading.

AWS Trusted Advisor flags most of this — idle load balancers, underutilised EBS volumes, unassociated Elastic IPs — but the full cost-optimization check set requires a paid Support plan (Business Support and above). If you're on Basic, Compute Optimizer plus a CUR query gets you most of the same answers for free.

The order is the recipe

The sequence matters more than any single fix:

Get the real bill — CUR + Cost Explorer, read the shape before touching anything.
Data transfer first — it's invisible and often the biggest single waste (NAT Gateway, cross-AZ, egress).
Rightsize — Compute Optimizer for direction, gp3 and Graviton for structural wins, one size at a time.
Commit — Savings Plans on the rightsized baseline, never before.
Storage cleanup — S3 classes, then delete the graveyard.

Run it top to bottom and the early steps make the later ones cheaper — you don't want to buy a 3-year commitment on an instance you're about to delete, or rightsize a fleet before you've stopped it routing free traffic through a paid NAT Gateway.

I've run this enough times that the pattern is boringly consistent. On the cost work I take on, the range I quote is 20-40% with no performance loss — and on a mid-size bill the first pass usually lands in that band without touching the architecture. The exact number is yours to discover; the order is the same on every account.

If you'd rather not run it yourself: this audit is my AWS cost-optimization offer. I read your bill, run this sequence against your CUR, and hand you a prioritised list — biggest, safest wins first, with the numbers attached. No architecture rewrite, no lock-in, no commitment to me beyond the audit. Reach me through rajesh.medampudi.com/work-with-me — a short email about what you're dealing with is the best starting point.

Sources

(all checked 2026-06-18, us-east-1 unless noted)

https://aws.amazon.com/vpc/pricing/ — NAT Gateway $0.045/hr + $0.045/GB; VPC Gateway Endpoint for S3/DynamoDB is free
https://docs.aws.amazon.com/vpc/latest/userguide/nat-gateway-pricing.html — NAT Gateway pricing detail
https://aws.amazon.com/ec2/pricing/on-demand/ — DTO first 100GB/mo free then $0.09/GB to 10TB; cross-AZ $0.01/GB each direction
https://aws.amazon.com/ebs/pricing/ — gp2 $0.10, gp3 $0.08 (GB-mo); snapshots $0.05/GB-mo; unattached volumes still bill
https://aws.amazon.com/ebs/general-purpose/ — gp3 "20% less expensive than gp2", 3000 IOPS + 125 MB/s baseline included
https://docs.aws.amazon.com/ebs/latest/userguide/ebs-volume-types.html — gp3 as default, io2 for high-IOPS
https://aws.amazon.com/s3/pricing/ — S3 Standard $0.023, Standard-IA $0.0125, Glacier Instant $0.004, Flexible $0.0036, Deep Archive ~$0.00099 (GB-mo); Intelligent-Tiering monitoring $0.0025/1000 objects
https://aws.amazon.com/s3/storage-classes/intelligent-tiering/ — Intelligent-Tiering auto-tiers, no retrieval fees; lifecycle transition request costs
https://aws.amazon.com/savingsplans/compute-pricing/ — Compute SP up to 66%, EC2 Instance SP up to 72%; covers EC2/Fargate/Lambda across regions/families
https://aws.amazon.com/savingsplans/database-pricing/ — Database Savings Plans (Dec 2025) cover RDS/Aurora/ElastiCache-Valkey/OpenSearch/DynamoDB up to ~35%
https://aws.amazon.com/blogs/aws/introducing-database-savings-plans-for-aws-databases/ — Database Savings Plans launch (2025-12-02); Redshift remains Reserved-Node-only
https://aws.amazon.com/ec2/graviton/ — current page: "up to 20% less cost" vs comparable x86; ARM64/Neoverse (40% figure was Graviton2-gen)
https://aws.amazon.com/compute-optimizer/pricing/ — free; covers EC2/ASG/EBS/Lambda/ECS-Fargate (+RDS, idle-resource 2026); 14-day default, ~93-day with paid Enhanced Infrastructure Metrics
https://aws.amazon.com/aws-cost-management/aws-cost-explorer/pricing/ — rightsizing recs; hourly/resource granularity $0.01 per 1,000 usage records/mo
https://docs.aws.amazon.com/cur/latest/userguide/what-is-cur.html — CUR is most granular billing data, free to generate (pay only S3 storage), delivered via Data Exports
https://aws.amazon.com/blogs/aws/new-aws-public-ipv4-address-charge-public-ip-insights/ — $0.005/hr per public IPv4 since 2024-02-01, attached or not
https://docs.aws.amazon.com/awssupport/latest/user/cost-optimization-checks.html — Trusted Advisor cost checks (idle LBs, underutilised EBS, unassociated EIPs)
https://aws.amazon.com/premiumsupport/technology/trusted-advisor/ — full cost-optimization checks require paid Support plan (Business+)

Building My Own Cloud

Rajesh Medampudi — Sun, 06 Sep 2026 15:51:35 +0000

I rent six dedicated servers from a company in Germany. Together they have more cores, more memory, and more SSD than most production clusters I worked on a decade ago.

I run my own Kubernetes on them. Not managed. Not EKS. Not GKE. The whole stack, from the immutable OS up to the workloads.

People who hear this ask me why, and the question usually arrives in one of two tones. The dangerous tone is "that's amazing, how do I do it". The responsible tone is "why on earth would you do that to yourself".

This post is for the second group.

Why the cloud is the right answer for almost everyone

Let me get this out of the way honestly: by every conventional metric, I should be using the cloud.

Managed Kubernetes has become genuinely good. EKS has dramatically improved over the last three years. GKE has always been better than people gave it credit for. The serverless options are mature. The serverless databases are mature. The observability is mature. The bill is predictable in the way a Tuesday is predictable.

Self-hosting violates almost every assumption that makes a startup productive. Time is the most expensive resource you have. The cloud sells you abstractions that turn that time into product. Running your own substrate means the time goes into the substrate.

If you are trying to ship a product to customers — go use the cloud. Stop reading this post. It will only confuse you.

What the cloud does not sell you

Here is what the cloud will not sell you, even if you are willing to pay extra: control over your own roadmap.

The cloud's roadmap is the cloud's. They decide which APIs deprecate. They decide which regions get the new feature. They decide what your egress bill looks like. They decide whether your monitoring vendor — sitting on top of their infrastructure — is allowed to charge you eight times what it would cost you to host the same software yourself. They decide whether the small ML company hosting your fine-tuned model gets acquired by someone with very different priorities than the founders had.

Most of the time, none of this matters. The roadmap is fine. The bill is fine. The egress is fine. You do not think about it.

Then one day a regulation, an acquisition, or a pricing change makes you think about it, and you realise that the abstraction you bought was not an abstraction. It was a contract. With one customer.

I bought my own substrate for the same reason I write on my own blog instead of posting on a single platform. Not because the platform is bad — but because the platform is not mine.

What I actually run

The shape of the thing, layer by layer, told plainly.

The hardware. Six dedicated servers at a Hetzner data centre. Bare metal — not VMs, not "instances." Each node has multiple cores, real NVMe, real network bandwidth. The bill is a fraction of what equivalent compute on AWS would cost, and most months I am using a fraction of that.

The OS. Talos Linux on every node. Immutable, API-driven, no SSH. You do not log into Talos boxes; you reconcile them. After running mutable Linux for fifteen years, the experience of "I cannot break this server even if I try" is psychologically corrective. I wish I had switched sooner.

The Kubernetes substrate. A multi-tenant Kubernetes distribution that gives me per-tenant control planes, per-tenant ingress, built-in storage, built-in observability, and a real package model. I treat tenants as isolation boundaries — one for each environment or product surface I want to keep separate. This is the layer that took me longest to understand and pays the most consistently.

Serverless. Knative on top of the substrate, with resource patches tuned aggressively for the cluster size. Cold starts matter on a small cluster, so I rewrote four Python services in Go specifically because the cold starts mattered for my AI workloads. There is a separate post about that rewrite.

Durable workflows. Temporal, exposed inside one tenant and bridged into another via a selector-based service so the UI works through the cluster's ingress. Replaced what would otherwise have been a sprawl of SQS-and-Lambda glue with a single workflow engine that survives restarts and treats retries as a first-class concept.

Secrets. OpenBAO — a Vault drop-in — with AppRole auth scoped per service. Every service that needs a credential gets one through OpenBAO. Nothing lives in environment variables in a Helm values file. Rotating a credential is a single operation, not an archaeology dig.

LLM gateway. LiteLLM in front of a small fleet of model providers, with per-token pricing tiers configured explicitly so my AI agents have a budget. When an agent goes off the rails — and they do — the budget is the seatbelt.

Backups. Velero, snapshotting the platform on a schedule. I restored from one of these snapshots in anger exactly once, which was enough to convert me from "backups are a chore" to "backups are oxygen."

Edge. A Nostr relay running on Cloudflare Workers, handling globally distributed traffic for a fraction of what a VPS would cost. The edge does what the edge is good at; the metal does what the metal is good at. The discipline is figuring out which is which.

That is the substrate. Everything else I build — the personal blog, the family Bitcoin wallet, the consulting tools, the AI agents — runs on this stack.

The mistakes

Three I will admit to in public.

The first one was treating Kubernetes like AWS. I tried to recreate cloud primitives one for one. NAT Gateways. Per-tenant load balancers. Internet egress proxies. Each of these has a Kubernetes-native equivalent that is better than the cloud version, and I spent two months reinventing things that were already in the platform's package catalogue. If the platform ships an opinion, take the opinion before you write a flag to override it.

The second was undersizing the control plane. When you run nested control planes — child clusters whose API servers are pods on the parent — you can absolutely starve them by being stingy with CPU. A child cluster whose API server is being CPU-throttled looks indistinguishable from a child cluster that is just slow. I lost a weekend tracing a problem that turned out to be a requests.cpu: 100m set by a copy-paste two months earlier.

The third was ignoring the runbook. I wrote a Day-2 operations runbook early. Excellent. Then I did not update it for six months. Excellent operational hygiene right up until the morning I had to follow it, and discovered the version I was following described a cluster topology I had since changed. The runbook is not a write-once artefact. It is a contract you renew every time the cluster moves.

What it actually costs

The bill, all in: a couple hundred dollars a month for the bare metal. A small fraction of that for the edge bits and DNS. The Cloudflare side is free at my volume.

The same workload on AWS — managed control plane, NAT Gateways, egress, observability, secrets — would cost me five to ten times more. I have done the math. I keep doing the math. The math keeps coming out the same.

But the real cost is not the bill. The real cost is the time I spend keeping it running. Some weeks that is an hour. Some weeks it is a weekend. Some weeks it is the entire weekend, and I think dark thoughts about EKS at two in the morning.

Average it out and the time cost is significant — easily worth more than the bill differential at any honest hourly rate. So if you only care about cash plus time, this is a bad trade.

When you absolutely should not do this

You should not do this if:

You are building a product and your customers are not yet sure they want it. Self-hosting will eat the runway you need to find product-market fit.
You do not enjoy infrastructure. This is a hobby that pays in skill, not in money. If you do not like the hobby, you will resent every minute of it.
You do not have a fallback. The cluster will go down. You need to be okay with that, or you need someone else who is.
You think it will be cheaper without budgeting for time. It is not cheaper if you bill yourself honestly.

In other words: do this if and only if the learning is worth the cost on its own, separate from the workload it hosts.

Why I keep running it anyway

A platform you control is a platform that will host whatever you decide to build next.

I have a Bitcoin wallet I am building for my family. I have a content engine that publishes to a personal blog. I have a Frappe ERP I deploy for consulting clients. I have AI agents that need to run cheaply and reliably. I have a CLI that searches forty-seven thousand vectors of my own writing in under a second.

None of these existed when I built the cluster. The cluster was built so that they could exist when I needed them — on terms I controlled, with no third party between me and the workload.

That is the deepest difference between renting your substrate and building one. The rented substrate is a contract. The substrate you built is an option — a low-cost option, with a long expiry, on whatever you decide to ship next.

I will keep paying my Hetzner bill and my Talos curiosity tax for as long as I am building things. The day I stop is the day I migrate to managed Kubernetes and admit it.

That day is not today.