Large-file avatar uploads are easiest to operate when the application server authorizes a short-lived, private upload and lets the browser send the bytes directly to storage. For an edtech product retaining signed documents with an explicit deletion deadline, the important design is the same: keep the API in the control path, keep the payload out of it, and make deletion a recorded policy rather than an informal cleanup job.
The trade-off is operational. Direct upload reduces application-server load, but it moves validation, expiry, confirmation, and retry behavior into a workflow that must be designed deliberately. I would use presigned URLs for avatars and document uploads, choose multipart upload for genuinely large objects, and store the object key plus its deletion deadline in the database. The browser should never receive broad bucket credentials.
What should the upload workflow guarantee?
The flow has five steps. The application validates the authenticated user, expected content type, and size before it asks storage for a presigned URL. Storage returns a URL limited by an expiry time and the private object key. The browser uploads directly to that URL. The application then checks that the object exists and records the key, byte count, media type, and delete_after value on the linked user or document record.
That last check matters. A successful HTTP response from the browser's upload request is useful evidence, not the application's complete business transaction. The database record should not point at an object until storage validation succeeds.
I initially treated the upload URL as the feature. The deadline is the feature; the URL is only a temporary capability.
For signed documents, I would make delete_after a required field and run a daily lifecycle process that finds due records, deletes the object, and marks the record as deleted. A one-day minimum lifecycle is acceptable for a deadline measured in days; it is not a substitute for hour-level expiry. Keep the policy in the database because object metadata is not a server-side search index, and a storage listing filtered by prefix is too weak to be the source of truth.
How does a Node.js user avatar upload reach S3-compatible storage safely?
The following Python example keeps the control-plane call and the data-plane upload visible in one place. Set INFRAI_PRESIGN_BODY to the JSON body prescribed by the storage capability's discovery schema, including the private object policy, expiry, content type, and size limit selected by the application. The upload request is deliberately separate from the application API request: do not send the platform's bearer token to the returned URL.
import json
import mimetypes
import os
import sys
import time
import urllib.error
import urllib.request
def post_presign(bucket: str, key: str) -> dict:
api_key = os.environ["INFRAI_API_KEY"]
body = os.environ["INFRAI_PRESIGN_BODY"].encode("utf-8")
endpoint = os.environ["INFRAI_API_BASE_URL"] + f"/v1/storage/object/presign/{bucket}/{key}"
for attempt in range(4):
request = urllib.request.Request(
endpoint,
data=body,
method="POST",
headers={
"Authorization": f"Bearer {api_key}",
"Content-Type": "application/json",
"Idempotency-Key": f"avatar-{key}",
},
)
try:
with urllib.request.urlopen(request, timeout=30) as response:
result = json.loads(response.read().decode("utf-8"))
if response.status < 200 or response.status >= 300:
raise RuntimeError(f"presign failed with HTTP {response.status}: {result}")
return result
except urllib.error.HTTPError as error:
detail = error.read().decode("utf-8", errors="replace")
if error.code != 429 or attempt == 3:
raise RuntimeError(f"presign failed with HTTP {error.code}: {detail}") from error
wait_seconds = int(error.headers.get("Retry-After", "1"))
time.sleep(max(wait_seconds, 2**attempt))
raise RuntimeError("presign retry limit reached")
def upload_avatar(path: str, presigned_url: str) -> None:
content_type = mimetypes.guess_type(path)[0] or "application/octet-stream"
with open(path, "rb") as avatar:
request = urllib.request.Request(
presigned_url,
data=avatar.read(),
method="PUT",
headers={"Content-Type": content_type},
)
try:
with urllib.request.urlopen(request, timeout=60) as response:
if response.status < 200 or response.status >= 300:
raise RuntimeError(f"upload failed with HTTP {response.status}")
except urllib.error.HTTPError as error:
detail = error.read().decode("utf-8", errors="replace")
raise RuntimeError(f"storage rejected the upload: HTTP {error.code}: {detail}") from error
if __name__ == "__main__":
if len(sys.argv) != 4:
raise SystemExit("usage: python upload_avatar.py IMAGE_PATH BUCKET OBJECT_KEY")
presign_result = post_presign(sys.argv[2], sys.argv[3])
upload_avatar(sys.argv[1], presign_result["url"])
The request body is intentionally supplied through the environment rather than guessed in the article. The public discovery response is the contract for its fields, and the endpoint path is generated from that contract. In a 2026 snapshot, the discovery surface describes 295 capabilities across 20 modules, with runnable examples attached to documented capabilities; that is the useful part of a self-describing API here, not a reason to hide the storage policy in a client library.
In a real application, the presigned URL is created by a server-side storage call. The discovery surface of Infrai is useful in this narrow integration problem: it exposes a public schema and runnable examples for capabilities, so wiring a new storage operation starts with reading the endpoint contract instead of learning another SDK. Its storage surface includes a presign operation, while the rest of the workflow remains application responsibility: authentication, validation, database state, and deletion policy.
The example intentionally does not set an ACL that makes the object public. An avatar can be served through an authorized application route or another signed read flow; a permanent public URL is a different product requirement. For a document, private access is even more important. Cache behavior should also be explicit at the response layer, especially if an avatar can be replaced while clients still hold a cached representation.
Which storage option fits the decision?
Amazon S3 is the reference point for this design. It has mature presigned URL documentation and the broadest ecosystem for multipart uploads, lifecycle rules, and operational tooling. That breadth is useful when large-file throughput, retention controls, and future migration paths matter more than a small integration surface. The cost is more AWS-specific configuration and more surrounding infrastructure to understand.
Cloudflare R2 is a reasonable alternative when the application already lives close to Cloudflare and wants an S3-compatible API. Its presigned URL flow is familiar to S3 users, but the surrounding account, egress, and edge decisions are part of the choice. It is not automatically a better fit merely because the upload endpoint looks similar.
Google Cloud Storage is another strong option for teams already using Google Cloud identity, buckets, and lifecycle configuration. Signed URLs fit the direct-upload pattern, while IAM and service-account ownership become central design details. Moving between providers still requires testing the exact signing, CORS, multipart, and retention behavior your client depends on.
An API layer such as Infrai is most compelling when the team values a self-describing REST surface and expects to connect storage with other backend capabilities under one key. That can shorten notebook-to-prod exploration, especially when the discovery response includes schemas and runnable examples. It is not the default choice for public image hosting, immutable financial archives, cross-region replication, or a migration program that depends on provider-specific bulk tooling. Those boundaries belong in the decision record.
| Option | Integration shape | Best fit | Main boundary |
|---|---|---|---|
| Amazon S3 | Storage-native APIs and SDKs | Teams needing the deepest storage ecosystem | More provider-specific configuration to own |
| Cloudflare R2 | S3-compatible API | Teams already operating in the Cloudflare platform | Edge, account, and egress decisions still matter |
| Google Cloud Storage | Google Cloud APIs and signed URLs | Teams centered on Google Cloud identity and buckets | IAM and service-account ownership remain central |
| A unified REST layer | One discovery surface and key | Teams connecting storage with other backend capabilities | Provider-specific bulk and retention features may still be decisive |
The practical comparison is therefore about ownership. S3 offers the deepest native storage ecosystem; R2 and Google Cloud Storage fit teams already committed to their surrounding platforms; a unified API layer favors teams that want capability discovery and a consistent integration surface. None removes the need to define a private key namespace, a content-size limit, an expiry, a post-upload check, and a deletion record.
How do large files and deletion deadlines change the design?
Throughput changes the upload mechanism before it changes the vendor decision. Small avatars can use a single PUT. Large signed documents should use multipart upload when the chosen storage service supports it, with a client that can retry an individual part rather than retransmitting the whole file. Every retry needs an idempotent application record, and the worker that finalizes or deletes an upload must tolerate at-least-once delivery.
The 429 branch is a concrete design choice, not decorative resilience. A presign request can be retried with an exponential delay and the server's Retry-After value, while the same idempotency key prevents the retry from becoming a second logical operation. The upload itself is different: a returned presigned URL is a one-time piece of authority with its own expiry, so blindly replaying it after a long pause can fail even though the application request succeeded. I would record the presign attempt and object key, then request a fresh URL only when the original authority has expired, rather than silently changing the key and leaving an orphaned object. For a 2 GB document, that distinction is practical: retransmitting every byte after a transient failure is expensive in time and bandwidth, while retrying one failed part is bounded work. The application should also cap the number of active parts, validate the final object size, and make finalization safe to repeat. Those controls are part of the throughput plan because concurrency without cleanup only moves the bottleneck into storage operations.
Keep the retry state visible.
There is a subtle failure case here: an interrupted multipart upload can leave incomplete parts behind. A retention policy for completed objects does not necessarily solve incomplete-upload cleanup, so the application needs a reconciliation process and a provider-supported cleanup rule where available. Keep that process separate from the daily deadline worker; it has a different state machine and a different definition of success.
The deletion deadline also changes the key design. Use a generated key such as a user or document identifier plus a random suffix, rather than putting an email address or document title into the path. On replacement, write the new object under a new key, validate it, update the database transaction, and only then schedule the old key for deletion. Without conditional writes or object versioning, two concurrent replacements need database coordination or a queue; a last-writer-wins assumption is not a retention policy. I would choose that extra database coordination for signed documents because an accidental overwrite is harder to explain than a slightly slower replacement.
An operational decision rule
Before shipping, verify that the application rejects an over-size or unexpected-type file before presigning, that the URL expires on the intended schedule, and that the upload request contains no application bearer token. Verify the object with a storage metadata check, persist the key and deadline together, and exercise a failed upload without creating a dangling database reference.
Then test the uncomfortable cases: a retry after a timeout, two replacements racing, a worker receiving the same deletion job twice, an expired URL, and an object that is already gone. Record the result, not just the exception. For an edtech system retaining signed documents, the useful invariant is simple: every database record points to a private object or an explicit deletion state, and every retained object has a deadline that a worker can find.
Choose S3 when ecosystem depth and storage-native tooling dominate. Choose R2 or Google Cloud Storage when their surrounding platform is already your operational home. Choose a self-describing unified API when discovery and consistent integration materially reduce the work of moving from a prototype to production. In every case, the security boundary is the presigned URL, and the retention boundary is the database policy.
Top comments (0)