If you need GPUs for rendering or for training, and you are paying on-demand prices for them, you are probably paying about five times what you have to.
That is not a figure from a vendor page. It is what I pay. A single-card instance in my family costs $3.0421 an hour on demand, and the same machine on interruptible capacity has been costing me between 55 and 70 cents.
There is a catch, and it is a real one, and most of what follows is about the catch. But the headline deserves to be stated plainly first, because a lot of teams never look at interruptible capacity at all, having been told it is for batch jobs and brave people.
The discount on GPUs is better than the guides suggest
AWS advertises "up to 90% off On-Demand". Everybody quotes that line and everybody then explains that it is marketing.
They are mostly right. Most writing about interruptible instances is about CPU fleets, where the workhorse families sit somewhere around 55 to 75 percent off, and the 90 percent number belongs to some unloved instance type in a region nobody uses.
Graphics cards are the exception, and I have not seen anyone say so. My own survey, taken across three regions on one afternoon:
| instance | on demand | interruptible | share of on demand |
|---|---|---|---|
| single card, best zone | 3.0421 | 0.5503 | 18% |
| single card, my zone | 3.0421 | 0.5996 | 20% |
| two cards, best zone | 7.1283 | 1.2721 | 18% |
| single card, a third region | 3.0421 | 1.4175 | 47% |
Eighty percent off, sitting near the top of the advertised range rather than in the middle of it. On a machine that costs three dollars an hour, that is the difference between running an experiment and not running it.
One number in that table matters more than the discount, though. On-demand pricing is identical in every region. Interruptible pricing is not: in one region the spread between availability zones was 27 percent, in another 1.2 percent. So the question is never "is this region cheap". It is "which zone, this week".
What you actually give up
Three things, in ascending order of how much they hurt.
The machine can be taken back. You get a two-minute notice, and then it is gone. This is the constraint everyone knows about, and it is the least interesting of the three, because it is the one you can engineer around.
The cheapest zone is cheap for a reason. In the table above, the best price was in a zone that AWS itself rated lowest for capacity. I had already learned that the hard way: that same zone dropped me, and the savings of four cents an hour bought me an outage. The price you do not pay is not free. It is quoted somewhere else, in interruptions.
Capacity is not a dial you can turn. This is the one that actually constrains me, and almost nobody mentions it. Your account has a quota, in vCPUs, for interruptible GPU instances. Mine is 64, and it is 64 in every region I checked, so relocating does not help.
Raising it is not a form you fill in. I asked for 128 and got an answer the same day: partially approved, 64. When I asked again, I learned that increases beyond that threshold have been removed from self-service entirely and now require a review process you can only reach through an account team.
Live with that number long enough and it reshapes your architecture. My two-card instance is 48 vCPUs. Two of them would be 96. The ladder has no rung in between, so "add a bit more capacity" is not an available move. I do not scale; I choose a shape and fit inside it.
And one asymmetry worth knowing before you design your fallback: the quota for regular on-demand GPUs is set separately and differs wildly by region. In mine it was 64 in one, 16 in another, 8 in a third. Your emergency exit may be much narrower than your front door, and in a different building.
Who this is for
If your work is batch shaped, meaning it can stop and resume and nobody is waiting, take the eighty percent and do not think too hard. Training runs with checkpoints, offline rendering, evaluation sweeps, data preparation. The machinery you need is a retry.
If somebody is waiting on the other end of a live session, it is still worth it, but the discount now has to fund the engineering that hides the interruptions. That is my case, and the rest of this is what that engineering turned out to be.
The machinery, in plain words
Four pieces, none of them clever, all of them necessary.
A controller that owns a target, not a machine. It knows how many healthy slots the service needs, and it holds that number. Instances are cattle in the usual sense, but the important part is that no human and no config file names a specific machine or a specific zone. Which zone I am in is an outcome, not a setting. It changes with every replacement, and any belief of the form "I live in zone A" will eventually be wrong and will take something down with it.
A ladder, tried in order. Two cards on one machine; if that shape is unavailable, single-card machines; if interruptible capacity is unavailable entirely, an on-demand bridge to keep serving while I wait. The bridge is expensive on purpose and temporary on purpose.
A detail that took me a while to see: because on-demand prices are identical everywhere, the bridge should go wherever you have on-demand quota, not wherever your primary region is. There is no price penalty for leaving home, only a quota one.
A price watcher. Zone prices drift. Mine checks what I am paying against what the alternatives cost, and says so when the gap grows.
A backoff with a floor that must not be tuned. When capacity is short, the controller retries, and the interval grows: thirty minutes, then sixty, then one hundred and twenty. Three consecutive failures are normal weather, not an incident, and the most tempting change in the whole system is to make the retries more eager. That converts a short shortage into a long one, because a failed request burns the interval it was granted.
I got to watch all of this work without touching it. A real interruption came through in the middle of the day, and the system replaced the machine, brought the slots back and reattached the address in about half an hour, while nobody was looking.
Three times my own guards lied to me
The system works. That is not the interesting part. The interesting part is that every one of these failures reported success while it was happening.
The price watcher was blind inside its own region. It compared my region against the others, faithfully, for months. But zones within a single region do not cost the same, and the comparison could not express that, so a zone that was running about $91 a month more expensive than its neighbour sat there unmentioned. The alert was green the entire time, because the thing it was built to notice was below the grain of what it measured.
The alert topic had no subscribers. The self-healing system published its events dutifully to a channel that nobody was listening to. Every message was delivered successfully. To nobody. There is no error state for that; a publisher with no subscribers looks exactly like a publisher whose subscribers are content.
The reaper does not reap after a planned replacement. When I deliberately roll a machine, the old one is supposed to be shut down. It is not, because that job only runs against the primary region. I never noticed in production, and the reason I never noticed is the best part: during a real interruption, AWS shuts the old machine down for me. The bug only appears when things go well.
That trio is the actual lesson of running on interruptible capacity, and it is not about interruptions at all. Once the machinery is between you and the hardware, your view of the hardware is only as good as the machinery's self-reporting, and self-reporting fails quietly by construction.
If you are starting on Monday
Take the discount, it is real and it is larger on GPUs than the guides imply. Pick your zone by capacity rating first and price second, because the cheapest zone will eventually charge you in outages. Check your quota before you design anything, and check the quota for your fallback separately. Write the retry backoff before you write the happy path.
And for every guard you build, arrange for it to fail loudly at least once, on purpose, before you start trusting it. Mine all worked. Three of them were worthless anyway, and I found that out by accident.
What does your infrastructure report when the thing it is watching is not there at all?
Top comments (0)