I had a hobby app running on one small VPS for about three years. It served maybe forty people, most of them me. Last month the host had a bad night and I spent a weekend rebuilding it from a backup that turned out to be eleven days old. So I finally did the thing I had been putting off.
The first thing I got wrong was treating this as an infrastructure problem. It was a backup problem. Moving to fancier hosting would not have saved me a single hour that weekend, because the data I lost was gone before the move was ever on the table. Restore drills are boring and they are the only part of this that actually mattered.
The second thing: I tried to containerise everything in one pass. Two days in I had a docker-compose file I did not understand and an app that worked slightly worse than before. I threw it away and moved one piece at a time, starting with the database, and that went fine.
If you are doing the same thing, the part worth reading properly is the transcoding and file handling, because that is where the surprises live.
Three weekends total. The app is not meaningfully faster. But I can lose the whole machine now and be back in twenty minutes, which is the only number I actually cared about.
I leaned on the official docs for this.
Top comments (1)
"It was a backup problem, not an infrastructure problem" is the sentence most of us have to learn by paying for it. Eleven days of data is a brutal way to find out that a backup you never restored is just a file that makes you feel safe. The restore drill being the only part that mattered is exactly right, and it is the part everyone skips because nothing about it looks urgent.
The one-pass containerise is also familiar: two days in you own a compose file you can't explain, which is the same failure as an agent rewriting the same file twelve times. One piece at a time, database first, is the boring path that actually finishes. Twenty minutes to rebuild the whole machine from scratch is a better number than any latency graph. How do you make sure the drill happens again in three months, once the muscle memory of that weekend has faded?