Short answer: let the browser upload the original image directly to object storage with a narrowly scoped, short-lived signed request, then have the backend verify the stored object and create thumbnails asynchronously under separate immutable keys. Proxy the bytes through the application when validation must happen before storage, the client cannot reach the storage endpoint, or the signing mechanism cannot enforce the constraints your threat model requires.
Treat the original and every derivative as different lifecycle states, not as one file that gets overwritten. A database record should own those transitions: requested, uploaded, verified, processing, and ready. The public application must not claim an image is ready merely because the browser says its upload completed.
This is an SLO problem disguised as a form submission. Direct upload removes image bytes from the Next.js request path, but it also splits one apparent user action across a browser, an object store, a metadata service, and a worker. Capacity planning gets easier only after those boundaries are explicit.
How should a browser upload an image before a Node.js backend creates thumbnails?
The safe flow begins with metadata, not bytes. The browser asks the application for permission to upload one object and supplies the proposed media type, size, and a client-generated idempotency key. The backend authenticates the user, chooses an opaque object key, records an upload intent, and returns a signed upload request that permits exactly the intended operation. The browser sends the original directly to storage and then tells the application that the transfer finished.
That notification is a hint.
The backend verifies the object from storage before it queues resizing work.
The distinction matters.
A client can disappear after receiving permission, retry after a network interruption, or report completion for an object that does not match its declared metadata. The application therefore owns the state machine, while storage owns the bytes. Keep the original private. Give the worker read access to the original prefix and write access to a derivative prefix; give the browser neither permission.
Use opaque keys built from an internal image identifier and a generation number. Don't use the original filename as identity. A name supplied by a browser is useful display metadata, but it is a poor authorization boundary and an unreliable cache key. Derivatives should include the transformation recipe or version in their keys so a new crop policy can coexist with the old output during a rollout.
The capacity-planning reflex is simple: budget each accepted original for one storage write, verification work, a worker read, and one write per derivative, then separately model retries. I'm not sure what retry multiplier fits your traffic; request logs, object-store metrics, and a representative load test should settle it. Without that accounting, moving bytes out of Node.js can make the web tier look healthy while the resize queue quietly misses its SLO.
Make authorization smaller than the object
A signed request is delegated authority. It should expire quickly enough to limit replay, allow only one key and one method, and bind enforceable properties such as content length and media type where the storage signing mechanism supports them. The backend should generate the key. CORS is a browser permission check, not authentication, so configure only the application origins and methods that are actually required, but never treat an allowed origin as proof of who uploaded an object.
There is a sharp edge: declared media type is not verified content. After upload, the worker or verification step must inspect the actual bytes with a maintained image decoder, reject malformed or unsupported input, enforce pixel and file-size ceilings, and strip metadata if the product's privacy policy calls for it. Perform decoding in an isolated process or constrained worker because image parsers operate on untrusted data.
Retries need names. Reusing the same upload intent should return the same object key until that intent expires; creating a new intent should create a new generation. Worker jobs should be idempotent, which in practice means deterministic output keys and an atomic database transition that prevents two successful workers from publishing competing manifests. Duplicate delivery is normal.
Design for it.
A 403 response is diagnostic rather than mysterious: check expiry, clock skew, method, signed headers, key, and CORS separately. Do not respond by broadening the storage credential or allowing every origin. That may hide the immediate symptom while turning a scoped upload path into a general write capability.
Keep thumbnail work off the request path
Thumbnail generation consumes CPU and memory in bursts that correlate with user uploads, and decoded pixel buffers can be much larger than compressed originals. An asynchronous queue gives the web tier a bounded job: authorize, verify, enqueue, and report status. Workers can then have their own concurrency limit, memory limit, retry budget, and dead-letter policy.
The Go example below shows the ownership boundaries rather than a vendor SDK. The storage adapter implements signed upload and object inspection against whichever service you choose. The queue provides at-least-once delivery, so confirmation is deliberately idempotent.
package images
import (
"context"
"errors"
"fmt"
"time"
)
type Store interface {
SignPut(ctx context.Context, key, mediaType string, maxBytes int64, expires time.Time) (string, error)
Stat(ctx context.Context, key string) (ObjectInfo, error)
}
type Queue interface {
Enqueue(ctx context.Context, job ResizeJob) error
}
type ObjectInfo struct {
Size int64
MediaType string
}
type ResizeJob struct {
ImageID string
Generation int
SourceKey string
}
type Service struct {
store Store
queue Queue
repo Repository
now func() time.Time
}
func (s *Service) Authorize(ctx context.Context, ownerID, mediaType string, maxBytes int64) (string, error) {
image, err := s.repo.CreateUploadIntent(ctx, ownerID, mediaType, maxBytes)
if err != nil {
return "", err
}
expires := s.now().Add(image.UploadTTL)
return s.store.SignPut(ctx, image.SourceKey, mediaType, maxBytes, expires)
}
func (s *Service) Confirm(ctx context.Context, imageID string) error {
image, err := s.repo.GetForUpdate(ctx, imageID)
if err != nil {
return err
}
if image.State == Ready || image.State == Processing {
return nil
}
if image.State != AwaitingUpload {
return errors.New("image is not awaiting upload")
}
info, err := s.store.Stat(ctx, image.SourceKey)
if err != nil {
return fmt.Errorf("verify source object: %w", err)
}
if info.Size <= 0 || info.Size > image.MaxBytes || info.MediaType != image.MediaType {
return errors.New("uploaded object does not match the upload intent")
}
if err := s.repo.MarkProcessing(ctx, image.ID, image.Generation); err != nil {
return err
}
return s.queue.Enqueue(ctx, ResizeJob{
ImageID: image.ID, Generation: image.Generation, SourceKey: image.SourceKey,
})
}
Metadata checks are necessary but insufficient; byte-level validation belongs in the worker before decoding. The publication step should write all required derivatives first, verify their metadata, and then commit a manifest or database state that makes the set visible. Readers either see the previous complete generation or the new complete generation. They should never see half of a thumbnail set.
Cache policy follows identity. Versioned derivative keys are immutable, so they can use a long freshness lifetime; mutable aliases need revalidation or short freshness. Set Cache-Control on the object response deliberately, and don't mark private originals as publicly cacheable.
Choose the boundary with a buy-versus-build table
No architecture wins every workload. The useful comparison is the operational boundary your team is willing to own, including upload bandwidth, signing, data retention, decoder patching, queue delay, egress, and deletion. Price is one input, but the on-call surface and exit plan often dominate after launch.
| Boundary | Good fit | The catch | Capacity and SLO question |
|---|---|---|---|
| Browser to object storage, managed queue and workers | Large or bursty uploads where the web tier should stay out of the byte path | More distributed states, CORS configuration, and provider-specific signing behavior | Can queue age stay inside the image-ready SLO at peak arrival rate? |
| Backend proxy to object storage | Small uploads, strict inspection before persistence, or clients that cannot reach storage | Application bandwidth, request duration, and memory pressure grow with every upload | How many simultaneous bodies can each instance safely stream without harming API latency? |
| Self-hosted object storage and workers | Data-local deployments or teams with existing storage operations | The team owns durability, upgrades, replication, recovery tests, and capacity headroom | Is there staffed on-call ownership for the data plane and tested recovery objectives? |
| Synchronous resizing in the backend | Tiny, tightly bounded images where immediate output is mandatory | Decode latency and memory compete directly with ordinary web requests | Can worst-case decoding fit the request deadline without lowering the API SLO? |
Direct-to-storage upload is not suitable when bytes must be scanned or transformed before they may be persisted, when client networks cannot reach the object endpoint, or when signing cannot constrain the properties required by policy. Stick with a streaming backend proxy in those cases. Synchronous resizing is a reasonable exception only after input dimensions and concurrency are tightly bounded; otherwise, one hostile or merely enormous image can consume the same resources needed for authentication and page data.
Lock-in appears in more than an SDK. Signed-request formats, event payloads, lifecycle rules, consistency expectations, and image transformation semantics all become dependencies. Keep a small storage interface in application code, but don't pretend that an interface erases operational differences. An exit test should copy a sample generation, preserve metadata and cache behavior, rerun derivatives, and compare the published manifest before a migration becomes urgent.
Verify the pipeline and rehearse rollback
A runbook needs observable gates. Track authorization attempts, completed uploads, verification rejects, queue age, worker attempts, decode failures, derivative publication latency, and abandoned intents. Alert on user impact: the share of accepted images that become ready inside the promised window, plus the age of the oldest eligible job. CPU utilization alone cannot tell you that users are waiting.
Before rollout, test an expired signature, a wrong method, a mismatched media type, an oversized object, a truncated image, duplicate confirmation, duplicate queue delivery, worker termination between derivative writes, and deletion during processing. Confirm that each case ends in a known state and retries cannot publish partial output. Load testing should use the distribution of compressed sizes and decoded dimensions expected in production, because average file size hides memory risk.
Rollback should change routing, not rewrite history. Keep the previous derivative generation readable, stop publishing new manifests, drain or pause workers, and direct new upload intents to the last known-good pipeline. Once the fault domain is understood, replay jobs from original private objects into a fresh generation. Never overwrite a known-good derivative during recovery.
Short rollback paths cost storage. That's acceptable when the alternative is coupling recovery to a mass re-upload from users. Retention still needs a limit: expire abandoned upload intents, garbage-collect unpublished generations after a safety window, and delete originals and derivatives together according to the product's retention policy. Test that deletion path just as seriously as creation.
The final review question is blunt: can an operator determine, from one image identifier, who authorized the upload, which immutable source was verified, which recipe produced each derivative, what state is currently public, and how to replay or delete it? If the answer is no, the system isn't ready to carry user images, regardless of how elegant the upload form looks.
Top comments (0)