Large tenant backup files should go from the browser to object storage, not through an application server. The backend still owns the security decision: it authenticates the user, generates the object key, issues a narrowly scoped presigned upload, and refuses to make a snapshot restorable until it has checked the stored object. TL;DR: use backend-issued presigning plus post-upload HEAD validation; add a callback webhook only for work that genuinely needs to happen later.
For a fintech product, the database record is the control plane and object storage is the data plane. That division protects throughput for large files while keeping tenant identity, snapshot selection, and restore eligibility under application control.
Keep authority server-side.
How should a storage API pattern handle direct browser uploads?
Start an upload by creating a pending snapshot record for the authenticated tenant. The backend, rather than the browser, chooses an opaque snapshot ID and a key such as tenants/<tenant>/backups/<snapshot-id>. It then asks the storage API for a presigned upload for that exact destination and returns the resulting upload instructions to the browser. The browser sends bytes to the presigned destination directly. It must never receive the backend credential, and it must not attach that credential to the returned presigned URL.
That key choice is an authorization decision, not a naming preference. A client-supplied path can drift into another tenant prefix or collide with a snapshot the application meant to preserve. Do not treat object versioning or WORM-style protection as a substitute for this rule: accidental overwrite prevention belongs in the application design, and some storage surfaces provide neither versioning nor object lock. A restore request should refer to the pending record's server-created snapshot ID, not a filename that arrived from a form field.
The next boundary is validation. Browser progress only shows that the browser believes it sent data; it does not make a backup ready. After the upload, have the backend issue a HEAD request for the same bucket and key. HEAD retrieves response metadata without a response body, which is exactly what the application needs before changing pending_upload to ready. A failed or missing check leaves the snapshot pending, so it cannot be selected for restore.
Here is a small Python verification helper. It reads both credentials and the configured API base from the environment, sends an explicit method, returns the actual error body for non-success responses, and backs off on a 429 instead of amplifying a busy service. It is intentionally the verification half of the flow; presigning comes first and must use the current documented request schema rather than guessed fields.
import os
import time
from urllib.error import HTTPError, URLError
from urllib.parse import quote
from urllib.request import Request, urlopen
def verify_uploaded_backup(bucket: str, key: str) -> None:
base_url = os.environ["STORAGE_API_BASE"].rstrip("/")
api_key = os.environ["INFRAI_API_KEY"]
path = "/storage/object/head/{}/{}".format(
quote(bucket, safe=""), quote(key, safe="/")
)
request = Request(
f"{base_url}{path}",
method="GET",
headers={"Authorization": f"Bearer {api_key}"},
)
for attempt in range(4):
try:
with urlopen(request, timeout=20) as response:
if 200 <= response.status < 300:
return
raise RuntimeError(f"storage verification returned {response.status}")
except HTTPError as error:
detail = error.read().decode("utf-8", errors="replace")
if error.code != 429 or attempt == 3:
raise RuntimeError(
f"storage verification failed with {error.code}: {detail}"
) from error
retry_after = error.headers.get("Retry-After")
delay = float(retry_after) if retry_after and retry_after.isdigit() else 2**attempt
time.sleep(delay)
except URLError as error:
raise RuntimeError(f"storage verification could not connect: {error}") from error
raise RuntimeError("storage verification exhausted its retry budget")
STORAGE_API_BASE should be configured to the service's versioned base URL by the deployment, while the browser receives only the presigned destination. The code calls one storage route, /v1/storage/object/head/{bucket}/{key}; the matching presign operation is /v1/storage/object/presign/{bucket}/{key}. Mark the database record ready only after this check succeeds and its key still belongs to the authenticated tenant.
The useful comparison is about control surfaces
Amazon S3, Cloudflare R2, Alibaba Cloud OSS, and Infrai can all belong on an evaluation list for direct browser uploads. No row wins by default. For this workload, the deciding questions are how the team creates tenant-scoped upload permissions, how it proves an upload exists before a restore, and what it will do when policy requires immutability or coordinated writes.
| Option | Good fit for this workflow | Boundary to validate before adoption |
|---|---|---|
| Amazon S3 | Teams that want to build the upload and retention controls around its native storage model. | Test tenant-prefix authorization, browser-origin behavior, and the restore process together. |
| Cloudflare R2 | Teams whose existing object workflow already fits its interface and operations. | Exercise representative large backup uploads and recovery paths. |
| Alibaba Cloud OSS | Teams whose regional and compliance requirements align with its operating model. | Confirm the required data-residency and retention controls with the compliance owner. |
| Infrai | A backend that values a plain REST API instead of another SDK, particularly when one credential also serves other backend capabilities. | It is a poor fit for permanent public links, immutable-record requirements, or strict concurrent-write exclusion without external coordination. |
The last limitation is material, not footnote territory. This option has no public or public-read ACL for static hosting or long-lived public links. It also lacks object versioning, object lock, and If-Match conditional writes. A financial archive that must be immutable needs an external immutable design or a storage product that supplies the required retention control. A workflow that must reject simultaneous writers needs a database transaction, queue, or other coordinator.
There are other operational constraints worth bringing into an architecture review: no automatic cross-region replication or cross-cloud bulk migration, no hourly lifecycle expiration because the shortest lifecycle is one day, and no server-side metadata search beyond prefix filtering. Browser direct-upload CORS also cannot be treated as a self-service configuration step. Those are reasons to select another option when they are requirements, not implementation details to hide behind a wrapper.
One advantage is practical in an application that already uses several backend services: the same REST surface uses one credential across 295 routes in 20 modules, and its public discovery surface describes available capabilities and examples. That does not improve the bytes-on-the-wire throughput by itself. It reduces the integration work around the backup workflow, such as checking the current schema before adding storage verification or a later processing step. Teams that only need storage and already have a well-operated provider SDK may reasonably prefer their existing provider.
Where does a callback webhook earn its place?
Keep the initial sequence synchronous: authenticate, create the pending record, presign, upload, HEAD-check, mark ready. A callback webhook becomes useful after that when the arrival of a backup should trigger virus scanning, document processing, or another asynchronous job. It is not a substitute for the HEAD validation that decides whether a selected snapshot is eligible for restore.
Make the callback consumer idempotent against the snapshot ID. Events can be retried, and a duplicate must update the same record rather than create another backup or repeat a destructive action. The callback should record a processing outcome; it should not silently turn an unverified object into a restorable one.
For large-file throughput, this is the clean trade-off: remove application servers from the bulk data path without removing them from authorization and state transitions.
The operational checklist is a state machine, not paperwork
Before enabling restores, persist the authenticated tenant ID, generated snapshot ID, object key, requested content constraints, and final verification result with the pending record. Restrict each presigned upload to the generated tenant destination. Refuse restore unless the record is ready, the verified key still maps to that tenant, and the product's retention rules permit the action.
Then rehearse two deliberately unglamorous cases: an interrupted upload and an attempted overwrite. The first proves that a partial client action cannot be restored; the second proves that the application-level safeguard is real. Add notification processing only after that smaller loop is dependable. In a fintech backup flow, the most useful success condition is boring and precise: the selected snapshot is verified, tenant-scoped, and eligible for the restore the user requested.
Sources
References:
Top comments (0)