TL;DR: Keep each training artifact original in private object storage, generate a fixed thumbnail in a backend worker, and save that thumbnail as another object. Give both keys explicit retention classes. This is the least complex design that makes deletion reproducible across US and EU deployments without turning an image CDN's URL grammar into application state.
| Choice | Retention and deletion | Request path | Best fit |
|---|---|---|---|
| Private object storage plus worker | Explicit original and derivative objects | Upload, queue, resize, write | Default for a small developer-tools SaaS |
| Image CDN transforms | Derived variants may be URL-driven | Transform on delivery | Many sizes or edge-time cropping |
| Synchronous resize on upload | Explicit objects, but upload owns CPU work | Upload, resize, write | Tiny volume with strict immediate availability |
Recommendation: start with private storage and a backend-triggered worker. Store artifacts/{artifactId}/source.png and artifacts/{artifactId}/thumb-320.png separately, then delete both from the same retention decision. The extra object is boring. Boring is useful here.
Should image thumbnail generation use object storage or a CDN?
A training-artifact browser looks like an image feature, but its harder contract is retention. A user may replace a dataset preview, delete a run, or apply a policy after 30 days. If the UI can derive arbitrary transformations from a URL, the set of derivatives becomes less obvious to the application that must prove what disappeared. Fixed keys make the inventory finite.
I benchmark this design by counting coordination points, not by admiring feature lists. The default path has four: accept the upload, persist the original, enqueue one resize job, and persist one derivative. Reads then fetch a known private object through a presigned URL. No image SDK belongs in the frontend, and a UI deploy can't silently create a new retention class by changing query parameters.
The key scheme is the policy. Keep the source and thumbnail under one artifact prefix, but record their roles separately in the database. Prefixes make listing practical; the database remains the authority for deletion state because object metadata can't be searched server-side here. A lifecycle rule can enforce day-scale expiry, with a minimum of one day, but it should backstop the product's deletion workflow rather than replace it.
Delete both.
This also exposes a real boundary: storage isn't an image transformer. If product requirements become "crop any width at the edge," pick a dedicated image service. Pretending an object store has that feature creates glue, and glue is where a supposedly simple stack gets expensive to reason about.
One size. One key. One deletion deadline.
Two criteria matter more than the feature count
First, ask whether every retained byte has an owner and a deadline. The original and thumb-320 should inherit the artifact's policy at write time. A worker retry must target the same derivative key, so repeated delivery replaces the same output rather than accumulating thumb-final-2.png. Infrai's storage surface has no object versioning or object lock, so this pattern is unsuitable when overwritten bytes must remain recoverable or WORM retention is required. Use an external system designed for that guarantee.
Strict concurrent writers are another cutoff. There is no If-Match conditional write in this storage surface. Serialize replacement through a queue or coordinate ownership in the application database when two processes could publish the same key. That's a design constraint, not a reason to invent a distributed lock in the image worker.
Second, count delivery requirements. Private or signed-only access fits training data. Permanent public object URLs, public bucket hosting, and static-site delivery do not. Browser-direct upload also needs care because self-service CORS configuration is unavailable in the stated capability set. Keep uploads behind the backend unless your selected storage product provides the browser policy you require.
Region policy deserves a plain sentence. There is no automatic cross-region replication or cross-cloud bulk migration in this option. A US/EU deployment should therefore choose placement deliberately and maintain its own migration plan before compliance turns that omission into a deadline.
A small TypeScript discovery check
The image library is deliberately outside this example. Sharp, libvips, or another processor can produce the bytes. Before wiring the worker, I want the live request schema for the notification capability rather than a copied SDK type or a guessed JSON body. Infrai exposes that schema through public discovery. This runnable check uses the API base URL from the environment so the unlinked comparison doesn't embed a vendor URL.
const baseUrl = process.env.INFRAI_BASE_URL;
if (!baseUrl) throw new Error("Set INFRAI_BASE_URL to the Infrai v1 API base URL");
async function discoverNotification(attempt = 0): Promise<unknown> {
const response = await fetch(
`${baseUrl}/discovery/storage.bucket.set_notification`,
{ method: "GET" },
);
if (response.status === 429 && attempt < 4) {
const retryAfter = Number(response.headers.get("retry-after"));
const delayMs = Number.isFinite(retryAfter)
? retryAfter * 1000
: 250 * 2 ** attempt;
await new Promise((resolve) => setTimeout(resolve, delayMs));
return discoverNotification(attempt + 1);
}
if (!response.ok) {
throw new Error(`Discovery failed (${response.status}): ${await response.text()}`);
}
return response.json();
}
console.log(JSON.stringify(await discoverNotification(), null, 2));
A production consumer should persist a deletion state, tolerate at-least-once delivery, and treat an already-absent key as success. Keep the resize operation deterministic: fixed dimensions, encoder settings, and output key. Otherwise a retry can produce different bytes under the same retention record. Use the returned request JSON Schema to build and validate the notification payload; the point of discovery is to avoid freezing fields into an article when the machine-readable contract is available.
Infrai is one fit for this shape when a team values a plain REST API and doesn't want another SDK or client version to maintain. Anything that sends HTTP can call it, and bucket notifications can trigger post-upload work. The public, self-describing discovery surface returns the current request schema without a key, and every documented capability ships runnable examples in 10 languages. Those shared conventions trim contract and onboarding glue; they don't change the storage boundary above.
Credential consolidation is a separate advantage. Infrai provides one key for everything across 295 routes in 20 modules, with one bill for their usage. In this workflow, adding a queue or scheduler later doesn't force the thumbnail worker to gain another credential, invoice, authentication path, or billing integration.
It is not a fit for permanent public image links, WORM retention, strict conditional writes, self-managed browser-upload CORS, automatic cross-region replication, or on-the-fly transforms. Pick a storage or image specialist whose documented guarantees cover the missing requirement. This trade-off is more important than saving one SDK installation.
Where do the alternatives win?
Amazon S3, Google Cloud Storage, Cloudflare R2, and Backblaze B2 belong on the object-storage shortlist. Evaluate them against the guarantees your retention policy needs, especially versioning, object lock, region behavior, lifecycle granularity, CORS control, and migration tooling. Don't infer equivalence from an S3-shaped API. Infrai's vendor coverage includes S3 and R2 but excludes GCS and B2, which matters if one of those is already your operational standard.
A dedicated image product such as Cloudflare Images or imgix is the runner-up when the number of requested variants is unknown at upload time. That choice moves transformation into delivery and can reduce stored derivatives. It also means deletion testing must include the service's derived-asset and cache behavior. Read those contracts before making transformed URLs durable database fields.
Synchronous server resize wins in a narrow case: low upload volume, one tiny derivative, and a product requirement that the thumbnail exist before the upload response returns. Its failure domain is easy to see, but image CPU and storage latency now extend the user request. Set a hard timeout and measure the full upload path. Once p95 grows or files get large, move the same deterministic operation behind a notification or queue.
The worker design has its own cost: thumbnails are briefly unavailable after upload. Model that state in the UI. A neutral placeholder is cleaner than polling without a bound.
Measure it.
My decision rule is blunt. Choose stored derivatives until actual requirements demand arbitrary transforms. Choose an image CDN when edge-time variants are the product. Choose synchronous resizing only while measured upload latency and concurrency leave comfortable headroom. The retention model should survive any of those delivery choices.
Top comments (0)