I lost an entire Sunday afternoon to a rejection email that said, in full, "Invalid CSV header." I had 40 Dreamstime uploads queued from a batch run, and the spreadsheet I'd built by hand had the columns in the wrong order. Dreamstime's importer doesn't guess. It reads the header row, checks it against its expected 15-column layout, and fails the whole file if one column is missing or renamed. That was the moment I stopped hand-editing CSVs and wrote make_metadata.py.
If you're pushing stock art or templates to multiple platforms from one pipeline, the metadata and the re-run problem will bite you before anything else does. Here's what I learned fixing it.
Every platform has a fixed column count, and they're all different
The three platforms I submit to have three different schemas, and none of them tolerate extras:
- Adobe Stock expects a 5-column CSV. Title, keywords, category, releases, and the filename. Get the header text slightly wrong and the whole upload errors out.
- Vecteezy wants 4 columns. Fewer fields, same strictness about the header.
- Dreamstime is the 15-column one that cost me the Sunday. Fifteen columns, exact names, exact order.
My first script tried to be clever and write one "universal" CSV with superset columns. That failed everywhere. What actually works is boring: the script knows the exact header each platform demands and writes that header and nothing else. make_metadata.py ./my-assets --platform dreamstime --out dreamstime.csv --ext .jpg produces a file whose first row matches what the importer checks for, byte for byte. No extra columns, no reordered fields, no "helpful" additions.
One honest note: without a titles file, the script title-cases the filename and uses that as the title. That's fine for a first pass, but you should replace it with real titles and keywords before you submit. Generic filenames as titles is a fast track to soft rejections on quality grounds, which are harder to diagnose than a hard header failure.
The double-post problem is worse than the rejection problem
A wrong CSV gets rejected, you fix it, you move on. A double-posted batch is different. Running your upload job twice, because you re-ran a script, or a cron job fired while you were testing, means the same asset goes up twice. On stock platforms that's the kind of behavior that gets accounts flagged, and a flagged account is a much bigger loss than one rejected file.
My pipeline runs batches, and batches get re-run. That's just true. So I wrote a second script, upload_tracker.py, that keeps a simple ledger of what has already been posted. The workflow is three commands:
python upload_tracker.py pending items.txt # what is NOT done yet
python upload_tracker.py done "asset-001.jpg"
python upload_tracker.py status
The key habit is marking done immediately after each successful upload, before the next one starts, not at the end of the batch. If the run dies at item 23, the ledger knows about items 1 through 22, and the re-run starts at 23. It's a plain text ledger, standard library only, no database to corrupt or migrate. That simplicity is a feature. The ledger has to be more reliable than the thing it's tracking.
Both scripts run on nothing but Python
This matters more than it sounds. The whole pipeline ran on a 1 vCPU, 2 GB RAM server with no GPU. Every dependency you add is memory you don't have. make_metadata.py and upload_tracker.py use the standard library alone, so there's nothing to install and nothing sitting in RAM between runs. The only two third-party packages in the entire kit are Pillow for the cover renders and reportlab for PDF output, and those live in separate scripts you only run when packaging.
The same constraint shaped the rest of the pipeline habits: one process at a time on a single core, gc.collect() after each asset, measure peak RSS with resource.getrusage(RUSAGE_SELF).ru_maxrss instead of average usage, and render previews at 60 dpi when you're only inspecting, not delivering.
The folder structure is part of the fix
Neither script works well against a junk drawer. The layout the pipeline assumes is fixed: assets/ for raw output, output/ for ready-to-submit files plus the CSVs, packs/ for finished zips, research/ for notes and logs, scripts/ for the tools. Point make_metadata.py at output/ and it picks up exactly the files that are actually ready.
One mistake I made once and won't repeat: I deleted research/ during a cleanup. Those scrape logs and notes were the raw material for three later products. Keep it forever. Disk is cheaper than redoing research.
What I'd tell past me
The failure wasn't carelessness. Hand-maintaining a 15-column CSV across 40 assets is a process that cannot survive contact with a real batch. The fix was two small scripts that enforce the platform rules mechanically instead of me remembering them. Metadata headers come from the script, never from memory. Upload state comes from the ledger, never from "I think I already did that one."
Those two scripts, plus md2pdf.py for turning Markdown docs into styled PDFs and pack_product.py for bundling zips with a 1280×720 cover and 600×600 thumbnail, are what packaged the 39 products this project shipped, 856 MB of output, on that same tiny server. I've bundled them as the Pipeline Starter Kit, with the install notes and the folder layout included. If any part of this sounds like your week, it's the tool built from exactly this work.
Building digital products? Free guides & kits for digital product sellers.
Top comments (0)