Short answer: validate the upload synchronously, quarantine the original, and publish smart-cropped derivatives only after an asynchronous worker records a complete validation result. That boundary keeps a bad file out of listing pages without forcing every upload request to wait for image work.
This is the pattern I use for a fintech marketplace where sellers attach classified-ad photos and the UI needs several aspect ratios. The upload endpoint owns identity, bytes, and policy. A worker owns decoding and cropping. Neither path gets to quietly promote an unverified object.
What should a classified-photo upload prove before publication?
An extension is not evidence. A request called image/jpeg can contain something else, and a correctly named JPEG can still be far too large for a thumbnail worker. Read the first bytes, decode with a library that enforces pixel and memory limits, and compare the decoded format with the allow-list. MDN's media guide is a useful reminder that containers, codecs, and browser support are separate concerns.
I keep three states in Postgres: received, validated, and published, with rejected and expired as terminal states. The object key includes an immutable upload id, never a user-supplied filename. The database row is the source of truth; object storage is payload, not workflow.
The request path does only bounded work. It authenticates the seller, caps the request body, computes a digest while streaming, and stores the original in a private quarantine prefix. A successful response means “received,” not “safe to display.”
That distinction matters during retries. A client can resend the same upload after a timeout, so the digest and seller id form an idempotency check. A duplicate should return the existing upload id rather than create a second crop job.
Ship the state.
How do upload-time checks and on-demand crops share one safe boundary?
Upload-time processing is predictable for a listing feed: every required derivative is ready before moderation or publication. It also makes the seller wait for CPU and queue capacity. On-demand processing gives a fast upload and avoids generating unused ratios, but the first viewer pays the latency and a hot listing can stampede the worker.
For classified photos I choose a hybrid. Validate and normalize once at upload, then generate the small, known set of crops in a job. Keep an on-demand path only for a ratio the product adds later. Both paths consume the same validated original and write to versioned derivative keys, so a crop cannot accidentally read a still-quarantined object.
Here is the boundary in TypeScript. The storage and decoder interfaces are deliberately boring: they can wrap an S3-compatible service, a local disk, or a test double without changing the state transitions.
type UploadState = "received" | "validated" | "published" | "rejected" | "expired";
type Ratio = "1:1" | "4:3" | "16:9";
interface UploadRow {
id: string;
sellerId: string;
digest: string;
state: UploadState;
sourceKey: string;
derivatives: Partial<Record<Ratio, string>>;
}
interface BlobStore {
put(key: string, body: AsyncIterable<Uint8Array>, contentType: string): Promise<void>;
get(key: string): Promise<AsyncIterable<Uint8Array>>;
copy(source: string, target: string, contentType: string): Promise<void>;
remove(key: string): Promise<void>;
}
interface ImageDecoder {
inspect(input: AsyncIterable<Uint8Array>): Promise<{
format: "jpeg" | "png" | "webp";
width: number;
height: number;
}>;
crop(input: AsyncIterable<Uint8Array>, ratio: Ratio): Promise<Uint8Array>;
}
const allowed = new Set(["jpeg", "png", "webp"]);
const maxPixels = 40_000_000;
async function validateAndQueue(row: UploadRow, store: BlobStore, decoder: ImageDecoder) {
if (row.state !== "received") return row;
const meta = await decoder.inspect(await store.get(row.sourceKey));
if (!allowed.has(meta.format) || meta.width * meta.height > maxPixels) {
row.state = "rejected";
await store.remove(row.sourceKey);
return row;
}
row.state = "validated";
return row;
}
async function publishDerivatives(row: UploadRow, store: BlobStore, decoder: ImageDecoder) {
if (row.state !== "validated") return row;
for (const ratio of ["1:1", "4:3", "16:9"] as Ratio[]) {
const key = `listing/${row.id}/${ratio.replace(":", "-")}.webp`;
const bytes = await decoder.crop(await store.get(row.sourceKey), ratio);
await store.put(key, (async function* () { yield bytes; })(), "image/webp");
row.derivatives[ratio] = key;
}
row.state = "published";
return row;
}
The important detail is the ordering: write the derivative, persist its key, and transition to published only when every required ratio exists. If the worker stops after the second crop, a retry sees validated; it can overwrite the same versioned keys and finish. No cleanup script has to guess whether a listing is safe.
Where do validation systems usually fail?
The first failure is trusting metadata supplied by the browser. Treat it as a hint for logging only. The second is checking dimensions after allocating a huge bitmap; a compressed image can expand dramatically. Set decoder limits before decode, and enforce a request byte limit at the HTTP layer as well.
The third is publishing through a CDN URL that points at the quarantine key. Keep quarantine private and issue URLs only for derivative keys after the state transition. A delete or expiry job should remove both the source and derivatives for rows that never reach publication. One failure mode is easy to miss: a worker can successfully write a square crop, crash while writing the landscape crop, and leave a cacheable URL behind. If the listing row is marked published too early, clients will observe a plausible image set that is actually incomplete. On restart, the worker must inspect the row, treat missing ratios as pending, and write deterministic keys again; the final state transition belongs after that check, in the same database transaction that records the derivative keys. This is slower than flipping a boolean after the first file, but it gives support staff one answer when they ask whether an upload is ready.
I also make errors explicit. 415 Unsupported Media Type means the declared or detected format is outside policy; 422 Unprocessable Content means the bytes are an accepted container but fail dimensions or pixel limits. Those responses are safe to retry only after the client changes the file. A network timeout is different: the client should query the upload id and read the current state.
Moderation adds another boundary. A technically valid image can still violate marketplace rules, so published should mean “derivatives are available,” while a separate listing policy decides whether they are visible. Keeping those decisions separate prevents a crop worker from becoming an accidental trust service.
What does the operational checklist look like in production?
Measure time from received to validated, queue age, decode rejection rate, derivative completion rate, and orphaned objects. Put the upload id and digest in structured logs; do not log the image bytes or a signed URL. Alert on a growing queue and on rows stuck in received, but make the alert threshold a deployment setting because traffic patterns differ.
Test the state machine with fixtures for truncated files, valid images with misleading extensions, oversized dimensions, duplicate digests, and worker restarts between derivative writes. Run a browser compatibility pass against the formats your product actually serves; the MDN format tables are a starting point, not a policy decision.
The catch is storage and queue complexity. This hybrid is not suitable when a listing must be visible immediately and you have no asynchronous worker budget; in that case, generate one conservative thumbnail in the request and accept the latency. Stick with pure on-demand derivatives when most ratios are rarely viewed and your cache can absorb the first-request spike. Your mileage may vary, and I'm not sure a single threshold fits every seller mix, so record real queue and abandonment data before changing the boundary.
Keep the contract boring.
Before shipping, verify that every public URL maps to a published row, that retries are idempotent, and that expiry removes private originals. Keep the policy in configuration, pin decoder versions, and review the limits when camera resolutions change. Small rules, enforced at one boundary, beat a clever upload controller spread across five services.
Top comments (0)