DEV Community

Cover image for The Second Quality Layer That Catches What Automation Misses

The Second Quality Layer That Catches What Automation Misses

Last March I had a batch of 40 vectors ready for Adobe Stock. The metadata CSV was generated by my pipeline, the uploads went out at 2 AM while I slept, and I woke up feeling like a genius. By noon, 11 of the 40 were rejected. Not for the CSV header, the header was right. For things the pipeline cannot see: a title that was technically accurate but read like a filename, keywords that were technically valid but described nothing in the image, and one file where the export had flattened a layer group in a way that left a visible seam. My automation had done its job perfectly. The job was just too small.

That rejection email was the moment I understood the pipeline has two quality layers, and I had only built one.

Layer one is rules. Layer two is judgment.

The first layer is what scripts are good at. My make_metadata.py writes the exact column count each platform demands: Adobe wants 5 columns, Vecteezy wants 4, Dreamstime wants 15. Get the header wrong and the upload fails instantly, before any human looks at it. That is a rules problem and a script solves it completely. Same with double-posting: upload_tracker.py keeps a ledger of what is already submitted, because running a batch twice is one of the fastest ways to get an account flagged. Rules, enforced by code, no exceptions.

But the 11 rejections were not rules violations. Adobe's reviewer did not say "wrong column count". They said the content did not match the metadata, or the asset had a quality issue. No CSV format check catches that. This is the second layer: does the asset actually look right, and does the description actually describe it.

Why the second layer cannot be fully automated

I tried. My parent pipeline is about 19,000 lines of Python, and a meaningful chunk of early code attempted to auto-judge assets. The honest result: an LLM can flag obvious failures (blank output, truncated render, duplicated keyword lists), but it cannot reliably tell you whether a sticker design reads well at sticker size, or whether a template's spacing feels off. Those checks cost real money at scale, too. Every AI-assisted review pass on an image costs API credits, and on a pipeline producing hundreds of assets, "just have the model look at everything" adds up fast.

What works instead is a hybrid, and the split is roughly this:

  • Automated pre-checks catch the mechanical failures. Render each asset at a low preview resolution, around 60 dpi, instead of 300, and check for blank regions, zero-byte files, and wrong dimensions. Inspecting at 60 dpi instead of 300 is roughly 25x fewer pixels to process, which matters when your box has 1 vCPU and 2 GB RAM.
  • A human pass catches everything else, in batches, with a checklist. Ten minutes per batch of 40. Not per asset. Per batch.

The checklist is short and specific: does the title describe the subject in plain words, are the keywords things actually visible in the image, is there any visible artifact at 100% zoom, does the file open cleanly. That is it. Four questions, and they would have caught all 11 rejections.

The batching habit matters more than the tool

The reason the human pass works is that it happens before upload, on the whole batch, not after rejection, one file at a time. Rejection cycles cost days. Adobe review turnaround is not instant, and every rejected asset is a slot in your batch that produced zero revenue for that cycle. Catching 11 bad assets in a 10-minute review is worth more than any amount of upload automation, because the upload was never the bottleneck. Judgment was.

Practically, my folder layout enforces this. Raw output goes to assets/, reviewed and corrected files go to output/, and the metadata CSV is built from output/, never from assets/. The pipeline physically cannot package an asset that has not passed through the review step, because the review step is where files move folders. It is a gate made of directory structure instead of code, and it has held up better than any validation script I wrote.

Where automation genuinely does the rest

Once the second layer has signed off, automation takes over completely and does it well. Packaging a reviewed product is a solved problem: zip the files, render a 1280x720 cover and a 600x600 thumbnail with Pillow, write the styled PDF with reportlab. My pack_product.py does that in one command, and md2pdf.py turns the documentation into a sellable PDF. All of this ran on that same 1 vCPU, 2 GB RAM server, one process at a time, with gc.collect() between items and peak RSS measured with ru_maxrss instead of guessing. The full run produced 39 finished products, about 856 MB of output, on hardware that costs a few dollars a month.

The lesson from the 11 rejections is not "automate less". It is "automate the right layer". Rules and packaging: full automation, zero regret. Visual and semantic judgment: a cheap, batched, human checklist gated by folder structure. Trying to push judgment into the code was where I lost a week and 11 assets, and the fix cost 10 minutes per batch.

The tool I built from this

Everything above is baked into the Pipeline Starter Kit: the four scripts (make_metadata.py with the exact Adobe 5-column, Vecteezy 4-column and Dreamstime 15-column headers, upload_tracker.py, md2pdf.py, pack_product.py), the folder layout that acts as the review gate, and the notes on running it all on a 1 vCPU / 2 GB box. Two dependencies total, and the metadata and tracking scripts need nothing but Python. It is the packaging layer behind those 39 products, with the second quality layer left where it belongs: in your hands, ten minutes per batch.

See the kit


Building digital products? Free guides & kits for digital product sellers.

Top comments (0)