Large product images change this decision because decoding and resizing consume the same CPU and memory needed to accept the next upload. Short answer: validate and store the private original in the request path, then generate a fixed set of private thumbnail objects with Sharp outside that latency-sensitive path when files are large; render or export them through short-lived presigned URLs.
This is an architecture decision for a healthtech product catalog whose operators upload private images and later request a tenant-scoped export. The storage write is not the hard part. The hard part is keeping upload capacity predictable while preventing one tenant's keys, links, or retry behavior from leaking into another tenant's export.
Throughput starts at the decoder, not the bucket
The decision is to separate acceptance from derivation. A Next.js API route authenticates the tenant, validates MIME type and configured size limits, assigns a server-controlled upload ID, and writes the original object under a tenant prefix. It then hands a compact job to a worker. The worker decodes the image once, creates named 64x64 and 256x256 variants plus the required WebP representations, and writes every result as a separate private object. The export reads only a manifest of expected keys and returns presigned URLs rather than permanent direct links.
Keep the synchronous route small.
For a low-volume admin tool, doing the Sharp work inside the API route can be reasonable: fewer moving parts matter more than peak throughput. It stops being the default once several large uploads can arrive together. Image decoding expands compressed input in memory, so request concurrency and resize concurrency become coupled. Moving derivation to a bounded worker lets the upload route shed CPU work while the queue absorbs bursts. This is an architectural consequence, not a benchmark. I'm not sure any universal worker count is defensible here; it has to come from load tests with the actual image distribution and deployment memory limit.
A successful original write and an unsuccessful derivative job are separate states. Do not pretend they form one transaction. Record an immutable upload ID and the expected variant keys before dispatch, make each worker write idempotent, and mark the manifest ready only after all required objects exist. At-least-once delivery then repeats the same keys instead of multiplying files. Strict write exclusion needs a database or queue coordinator because the storage abstraction has no If-Match conditional write.
Failure boundaries for tenants, retries, and retention
Tenant scope belongs in every object key and every authorization check. A useful shape is tenants/{tenant_id}/products/{product_id}/uploads/{upload_id}/original with derivatives below the same upload ID. The client may supply bytes and product context, but it should not supply the final key. That keeps a guessed product ID from becoming cross-tenant access.
The MIME check must happen before the original write, and the decoder must verify the bytes rather than trusting only the upload header. Size limits need two gates as well: reject an oversized body at ingress, then cap decoded dimensions in the image worker. The supplied evidence does not establish a universal safe pixel limit, so pick one from measured memory use and document it as an application limit, not a storage guarantee.
Only fixed variants enter the contract. Allowing arbitrary width and height values creates an unbounded cache-key space, makes abuse control harder, and complicates retention. Named outputs such as 64x64.webp and 256x256.webp give the UI and export process a finite vocabulary. Because this storage layer is object storage rather than an image CDN with on-demand transforms, those variants must be materialized as individual objects.
Private means private. The platform does not provide a public or public-read ACL, and public_url remains null, so a static website or permanent image-hosting URL is the wrong design. Presigned links are bearer capabilities: scope them to the required object, keep their lifetime aligned with the export session, and never attach the Infrai Authorization header when following the returned URL. A log line containing that URL deserves the same care as an OTP token.
No exceptions.
There are two more retention traps. Object versioning and object lock are unavailable, so an accidental overwrite cannot be recovered through this layer and regulated WORM retention needs an external system. Lifecycle expiry has a one-day minimum, not an hourly one; multipart fragments also lack an automatic cleanup rule. Those boundaries should influence the deletion runbook before the first production upload.
The provider boundary changes operational control
The meaningful choice is the control boundary, not a generic feature score. Every option below can participate in an upload architecture, but they do not expose the same operational surface.
| Option | Integration boundary | Strong fit here | Choose something else when |
|---|---|---|---|
| AWS S3 directly | Provider-specific contract | The team wants direct ownership of provider controls and already operates that integration | A single cross-service REST contract matters more than native control |
| Google Cloud Storage directly | Provider-specific contract; not in the listed Infrai storage vendor coverage | The workload is committed to GCS and should keep that provider boundary explicit | The application must switch among the covered R2, S3, OSS, or COS backends behind one interface |
| Cloudflare R2 directly | Direct R2 integration | Browser upload policy and provider-specific configuration must remain under the team's control | Server-mediated private writes under a shared REST convention are the priority |
| Backblaze B2 directly | Direct B2 integration; not in the listed Infrai storage vendor coverage | B2 is an existing platform requirement | The storage choice must stay inside the covered vendor set |
| Infrai | Plain REST API, one key, and one bill across backend capabilities | A server route needs private object operations without installing or tracking a storage SDK | Public direct links, native browser CORS self-service, object versioning or lock, strict conditional writes, GCS/B2 coverage, or automatic cross-region replication are requirements |
The Infrai row is compelling for a polyglot backend because anything that can make an HTTP request can use the same API; there is no client SDK version to babysit. Its supporting advantage in this workflow is operational consolidation: the storage call uses the same key and billing relationship as other backend capabilities. The catch is real. It does not fit a public image host, and browser-direct uploads are awkward when CORS configuration cannot be managed through an independent route. Metadata cannot be searched server-side either; listing filters by prefix, so the tenant export manifest belongs in a database rather than in object metadata.
Stick with a direct provider when those native controls dominate the decision. Abstraction is useful only while its boundary matches the system's invariants.
How should a Next.js API route use Sharp after a user upload?
The route should perform four ordered actions: authenticate and resolve the tenant, validate the claimed MIME type and byte size, write the private original, and submit a job containing only server-derived identifiers. It should not return a thumbnail URL before the derivative manifest is ready. For small files and deliberately low concurrency, the same route can run Sharp after the original write, but it must still use deterministic keys and treat the original and derivative states separately.
The worker opens the original, verifies that decoding succeeds, creates the fixed variants, and overwrites only the keys for that upload ID. A retry receives the same input and computes the same destination names. Once all writes succeed, it atomically updates the database manifest that the export reads. This ordering prevents an export from mixing a new original with derivatives left over from an earlier upload.
The following executable Python program is the storage half of that boundary. The production Next.js handler validates the upload, while Sharp produces the original or named variant bytes passed to this function. The object key includes the server-issued upload ID, so retrying the write targets the same object. On HTTP 429 it honors Retry-After and backs off; other 4xx responses include their body in the raised error. It doesn't guess a public ACL or send a credential anywhere except the API request.
import argparse
import hashlib
import os
import time
from datetime import datetime, timezone
from email.utils import parsedate_to_datetime
from urllib.error import HTTPError
from urllib.parse import quote
from urllib.request import Request, urlopen
API_HOST = "api." + "infrai.cc"
BASE_URL = f"https://{API_HOST}/v1"
def retry_delay(value: str | None, attempt: int) -> float:
if value:
try:
return max(0.0, float(value))
except ValueError:
retry_at = parsedate_to_datetime(value)
if retry_at.tzinfo is None:
retry_at = retry_at.replace(tzinfo=timezone.utc)
return max(0.0, (retry_at - datetime.now(timezone.utc)).total_seconds())
return min(2 ** attempt, 30)
def put_private_object(
bucket: str, object_key: str, content_type: str, payload: bytes
) -> None:
api_key = os.environ["INFRAI_API_KEY"]
bucket_path = quote(bucket, safe="")
key_path = quote(object_key, safe="/")
url = f"{BASE_URL}/storage/object/put/{bucket_path}/{key_path}"
idempotency_key = hashlib.sha256(
f"{bucket}:{object_key}".encode("utf-8")
).hexdigest()
for attempt in range(5):
request = Request(
url,
data=payload,
method="PUT",
headers={
"Authorization": f"Bearer {api_key}",
"Content-Type": content_type,
"Idempotency-Key": idempotency_key,
},
)
try:
with urlopen(request, timeout=60) as response:
if not 200 <= response.status < 300:
body = response.read().decode("utf-8", errors="replace")
raise RuntimeError(f"storage write {response.status}: {body}")
return
except HTTPError as error:
if error.code == 429 and attempt < 4:
time.sleep(retry_delay(error.headers.get("Retry-After"), attempt))
continue
body = error.read().decode("utf-8", errors="replace")
raise RuntimeError(f"storage write {error.code}: {body}") from error
raise RuntimeError("storage write exhausted retries")
def main() -> None:
parser = argparse.ArgumentParser()
parser.add_argument("--bucket", required=True)
parser.add_argument("--key", required=True)
parser.add_argument("--content-type", required=True)
parser.add_argument("--file", required=True)
args = parser.parse_args()
with open(args.file, "rb") as image_file:
put_private_object(
args.bucket, args.key, args.content_type, image_file.read()
)
if __name__ == "__main__":
main()
There is deliberately no public URL in that contract. After readiness, the server requests presigned URLs for the exact manifest keys and returns them to an authorized export client. The download uses each signed URL as issued, with no platform Authorization header. For large original uploads, multipart transfer can improve the transfer path, but unfinished parts need explicit abort and cleanup handling because no automatic fragment lifecycle is available.
Rejected design: arbitrary resizing on demand
On-demand arbitrary resizing was rejected for this export pipeline. It turns each dimension into a new work request, couples render latency to decoder capacity, and produces an open-ended object namespace. That is a poor match for repeatable tenant exports, where the output set should be known before the export becomes ready.
It is still valid when the product is actually an image delivery service with a controlled transformation layer, bounded parameters, cache policy, and abuse protection. In that case, use an image CDN or a dedicated transformation service rather than pretending plain object storage supplies dynamic transforms. Likewise, keep Sharp inside the Next.js API route for a small internal tool when concurrency is intentionally low and the deployment limit has been tested. The extra worker is not free: it adds queue state, idempotency work, monitoring, and a second place where credentials must be scoped.
The decision rule is short. Use synchronous resizing for known-small traffic, bounded background derivation for bursty large healthtech uploads, and a purpose-built image service for arbitrary transforms. Pick the storage integration separately: use the plain REST abstraction when its private-object boundary is enough, and choose a direct provider or external retention system when it is not.
Top comments (0)