I lost an evening last March because I assumed my host's backups covered me. They didn't. The nightly snapshot on my 1 vCPU / 2 GB RAM box only kept seven days, and I needed a CSV from eleven days back. That was the Adobe 5-column upload file for a batch of 40 assets I had already submitted. Rebuilding it meant re-running make_metadata.py against a folder of files I had since renamed. I got it back, but only because I had one sloppy habit: a _system folder I never cleaned. That moment is why backup is now a designed part of my pipeline instead of an afterthought.
Here is what actually works when your whole business lives on a tiny server.
Know what you are actually backing up
My catalogue is 39 finished products, 856 MB of packed output. That number matters because it decides the strategy. 856 MB is small enough to version or sync, but the raw material behind it is not. The parent project holds roughly 19,000 lines of pipeline code and over 300 pages of research notes. If you only back up the zips, you lose the ability to regenerate or repackage anything.
So I split the catalogue into three tiers:
- Irreplaceable and small: metadata CSVs, the upload tracker ledger, titles and keywords JSON, research notes. Kilobytes. These go to version control and a remote sync, every day.
- Regenerable but expensive: the 856 MB of product zips, covers, and thumbnails. Synced weekly off-server.
- Raw assets: the largest folder, and the one I regenerate rather than fully duplicate if I have to.
The mistake I made before was treating all of it the same and then backing none of it properly.
The CSVs are the real crown jewels
People back up their art and forget the metadata. That is backwards. The art exists as files on disk. The metadata encodes decisions: which title, which keywords, which order. All three channels reject a wrong CSV header instantly. Adobe wants a 5-column header, Vecteezy wants 4 columns, Dreamstime wants 15. make_metadata.py writes the exact header each platform demands and nothing else, but the output is still a decision record. If the CSV is gone and the source filenames have changed, you are re-doing editorial work, not just re-running a script.
Same for upload_tracker.py. That ledger is what makes a re-run safe. The fastest way to get a stock account flagged is double-posting a batch, so the tracker remembers every asset marked done. Lose the tracker file and you cannot tell what you already submitted. A file that small should be committed after every single batch run. Mine is, now.
Backups on a 2 GB box need their own rules
The same constraints that shape rendering shape backup. One vCPU means one process at a time, including archive jobs. Running a tar or rsync while a render is in flight is how you get an OOM kill and a corrupted archive.
My habits, in order of how much pain each one saved me:
-
Measure peak RSS before scheduling anything.
resource.getrusage(RUSAGE_SELF).ru_maxrssgives the real ceiling. My archive jobs peak well under the limit, but I checked before trusting them in a cron slot. -
Free memory between items.
gc.collect()after each asset, same as rendering. A long backup loop over thousands of small files leaks without it. - One file per subprocess for heavy steps. Memory returns to the OS. I never batch-compress inside a single long-lived Python process.
- Render at the resolution you need. This is a backup rule too. Inspecting a preview at 60 dpi instead of 300 means the verification pass over my archive does not blow the RAM budget.
- Verify restores, not backups. Once a month I restore one product zip and one CSV to a scratch folder and open them. An untested backup is a hope, not a backup.
The folder layout is the backup plan
The structure this kit assumes is also what makes tiered backup possible:
project/
assets/ raw output (one file per asset)
output/ ready-to-submit files + metadata.csv
packs/ finished product zips + covers
research/ your notes and sources
scripts/ these tools
Each tier maps to a folder, so each folder gets its own schedule and its own retention. research/ gets the strongest protection. I once deleted research notes to "clean up" and lost raw material for future products. Never again. Your scrape logs and notes are the input to your next product. Deleting them to save disk on a 2 GB box is a false economy when the whole research folder is smaller than a single product zip.
Cron, retention, and the honest limits
I run three jobs: a daily commit of trackers, CSVs, and notes; a weekly sync of packs/ and output/; a monthly verified restore. Retention on the server itself is short because disk is thin, but the off-server copies are the real backup. The server copy only exists to make restores fast.
What I do not have: a DR site that can run the whole pipeline. If the box dies, I can restore every product and every CSV and keep selling, but rebuilding the 1 vCPU runtime takes an afternoon of setup. For a solo operation at 39 products, that trade is honest and acceptable. Pretending otherwise would mean paying for infrastructure I do not need yet.
Where this comes from
None of this is theory. The four scripts in my kit, make_metadata.py, upload_tracker.py, md2pdf.py, and pack_product.py, packaged all 39 products and ran on exactly this hardware: 1 vCPU, 2 GB RAM, no GPU. The metadata rules they enforce came out of 300+ pages of research into what each platform actually rejects. The backup habits above are the operational layer around those scripts, written down because I lost that evening in March to not having them.
I packaged the whole thing as the compliance metadata kit: the four standalone Python scripts, the folder layout, and the notes on running it all on a tiny server. Two third-party packages total, Pillow and reportlab. The two metadata scripts need nothing but the standard library. If you are building the same kind of pipeline, it is the tool I built from this exact work.
Building digital products? Free guides & kits for digital product sellers.
Top comments (0)