I got a task at my internship that sounded simple. Move one of our servers to a new AWS region, keep it the same as before, and spend as little money as possible.
That last part stuck with me. If you've spent any time reading about saving money on AWS, you've seen Spot instances everywhere. AWS says they can be up to 90% cheaper than normal on-demand instances. Ninety percent! I thought I'd found the easy win.
Then at standup, a teammate asked one question that killed the idea in about ten seconds:
"Where does this server keep its data?"
Here's what I learned, and the question I now ask before picking Spot for anything.
1. What a Spot Instance Actually Is
AWS has huge data centres full of servers, and not all of them are being used at any moment. Spot instances let you rent that spare capacity for a lot less money.
The catch is in the deal. AWS can take it back whenever it needs the capacity. You get a two minute warning, and then your instance is gone.
By default, an interrupted Spot instance is terminated, and unless you've set it up differently, the disk that came with it goes too.
Two minutes is enough to save a bit of work. It's not enough to save a server's whole memory.
2. Our Server Couldn't Be Thrown Away
This is where the standup question came in.
The server I was moving keeps its context on its own disk. Everything it knows lives right there on that one machine. There's no separate database or storage holding a copy.
So if AWS took it back, we wouldn't just lose a running server. We'd lose everything it had built up and have to start from zero.
The cheap option could have ended up being the most expensive one.
3. Pets vs Cattle
There's an old saying in DevOps that finally clicked for me after this.
Cattle are servers you don't get attached to. They're all the same, and if one dies, you replace it and nobody notices. Think of a web server behind a load balancer, where the real data lives in a database somewhere else.
Pets are servers you care about. They have their own name, their own data, and if one dies, you're having a bad day.
Spot instances are great for cattle. They're risky for pets.
Our server was very much a pet
4. The Question I Ask Now
Before choosing Spot for anything, I ask myself:
"If this server disappeared right now, what would we lose?"
If the answer is "nothing, we'd just start a new one", Spot is probably a great fit. Things like:
- Batch jobs that can be restarted
- CI runners
- Test environments
- Servers in an auto scaling group that keep their data somewhere else
If the answer is "a lot", stick with on-demand, at least until the data lives somewhere safer.
5. We Still Saved Money, Just Differently
The goal was still to keep costs down, so cheap wasn't off the table. It just had to be the right kind of cheap.
The plan is to run the server on an on/off schedule, so it only runs when it's needed. With on-demand, you don't pay for the compute while the instance is stopped, only a small amount for the disk. And because stopping an instance keeps its disk, nothing gets lost overnight.
Spot isn't ruled out forever either. Once the migration is done, we'll look at it again. If the data moves off the server, it stops being a pet, and Spot becomes an option.
Final Thoughts
I went in thinking the cheapest option was the smartest one. One question at standup showed me that cheap isn't cheap if it can lose your data.
Honestly, some of the best things I've learned in this internship didn't come from docs. They came from teammates asking simple questions I didn't think to ask. This is why I love working with people, you learn something new every day.
So I'm curious: what's a "cheap" decision that ended up costing you more?
Top comments (0)