I ran my batch script twice on a Tuesday night. Not on purpose. The first run finished at 11:40 PM, I tweaked one title, and ran it again at 11:55 PM to "pick up the change". My tracker at the time was a text file I sometimes updated. Sometimes is the problem word there. By morning I had 84 duplicate submissions sitting in two contributor portals, and duplicates are one of the fastest ways to get a review queue backed up or an account flagged on stock platforms. Adobe's contributor review checks look for repeated and near-identical submissions, and Vecteezy reviewers see the same file twice in the queue just as plainly. I spent the next day withdrawing files by hand. That night I wrote upload_tracker.py, and it has run before every batch since.
The CSV header is a hard gate, so stop guessing it
Before the tracker mattered, the metadata had to be right. Each platform demands a specific column count and order, and the upload form rejects a wrong header before your files even get reviewed.
- Adobe Stock wants a 5-column CSV
- Vecteezy wants a 4-column CSV
- Dreamstime wants a 15-column CSV
make_metadata.py writes exactly those headers, nothing extra, from a folder of files. It uses only the Python standard library, so there is nothing to install and nothing to break. The point for this post is simpler: once your CSVs are generated from one folder, your batch script is the single source of truth for what got submitted. That is exactly the thing a double-run corrupts.
A ledger beats memory, and it beats a text file
My old text file failed for an obvious reason: updating it was a manual step, and manual steps get skipped at midnight. The fix is to make recording automatic and checking mandatory.
The tracker is a plain ledger with three commands:
python upload_tracker.py pending items.txt # what is NOT done yet
python upload_tracker.py done "asset-001.jpg"
python upload_tracker.py status
The habit that stopped my duplicates: the batch script never reads the raw item list. It reads pending output. If a re-run happens, the already-posted filenames are simply not in the pending list, so nothing gets submitted twice. Marking done is one command after each successful upload, and status before every batch takes two seconds. The whole thing is standard library only, no dependencies, so it runs on any box I SSH into.
Why platforms care more than you think
It is tempting to think a duplicate is harmless, one extra file among thousands. From the reviewer's side it looks different. Stock platforms pay per download and reviewers check submissions against what already exists. Repeated near-identical uploads read as spam or as keyword gaming, and both put your whole account under scrutiny, not just the duplicated files. Withdrawing by hand also costs real time. My Tuesday cleanup took about four hours for 84 files. The tracker took one evening to write and zero minutes since.
The rest of the pipeline still has to fit a tiny server
The tracker is cheap, but the surrounding pipeline is not, and I run everything on a 1 vCPU, 2 GB RAM server with no GPU. Three habits keep it alive:
- One process at a time. Fanning out image jobs in parallel on a single core just thrashes.
-
gc.collect()after each asset, because Python holds memory longer than you expect. - Measure peak RSS with
resource.getrusage(RUSAGE_SELF).ru_maxrss, not average usage. Peak is what gets you OOM-killed.
For heavy render steps I run one file per subprocess so memory returns to the OS, and I inspect previews at 60 dpi instead of rendering at 300. The tracker fits this budget trivially since it is a text ledger, which is the point: the boring safety tool should be the cheapest thing in the stack.
The folder layout makes the tracker possible
A tracker only works if filenames are stable. That means a fixed structure, not a junk drawer:
project/
assets/ raw output
output/ ready-to-submit files + metadata CSVs
packs/ finished product zips + covers
research/ notes and sources
scripts/ the tools
One file per asset in assets/, one name carried through to output/. If you rename files between steps, your ledger goes stale, so pick names once and keep them. I also keep research/ forever now. Deleting notes to clean up cost me raw material for later products once, and I will not repeat it.
Where this ended up
All of this lives in the compliance metadata kit: the same four scripts I use, including make_metadata.py with the exact Adobe 5-column, Vecteezy 4-column and Dreamstime 15-column headers, and upload_tracker.py as the ledger described above. It is the tool built from this work, not a course about it. It also includes md2pdf.py for turning Markdown into a styled PDF and pack_product.py for zipping products with a 1280×720 cover and 600×600 thumbnail, since packaging is the other half of selling. Two dependencies total, Pillow and reportlab, and the metadata and tracker scripts need neither.
If you run batches at all, add a ledger this week. The duplicate cleanup I did by hand was the most boring four hours of this project, and it was entirely preventable.
Building digital products? Free guides & kits for digital product sellers.
Top comments (0)