DEV Community

XenonCross2718
XenonCross2718

Posted on

Support Object Storage: Browser Direct Upload for Generated Image Throughput

A customer-support product that accepts large generated images has one constraint that dominates the architecture: image bytes should not occupy an application worker twice, once on arrival and again on the way to storage. Short answer: use a browser-to-storage upload only when you can control and test the bucket's CORS policy; otherwise, upload from the server and accept the extra byte hop as the reliable boundary.

That is not the answer a throughput chart wants. It is the answer the browser enforces.

A valid presigned URL does not grant a web page permission to send a request across origins. The browser may first issue an OPTIONS preflight, and storage can reject that preflight before the signed PUT ever runs. For customer-support media, this distinction matters because the failure may look like a bad upload credential while the object service never evaluates the signed write at all.

How should a US/EU SaaS troubleshoot browser direct upload CORS errors?

Start by separating browser policy from storage authorization. Send the same image bytes from a backend process to the intended write path. If the backend write succeeds while the browser attempt stops at preflight, investigate CORS. If the backend write also fails, inspect the signature, object key, method, content type, and expiration rather than changing browser headers at random.

Keep the browser's network trace. An OPTIONS failure and a rejected PUT are different incidents, even if the UI reduces both to “upload failed.” Record the request origin, requested method, requested headers, final status, and whether the actual PUT appeared. I have learned from OTP delivery gaps and rate limits that collapsing several transport stages into one application error destroys the evidence needed for a useful retry policy. Storage uploads deserve the same discipline.

Do not retry a failed preflight in a tight loop. It will not repair a missing policy, and it turns one support-ticket attachment into noise. A 429 from an API is different: honor Retry-After, use exponential backoff, and keep the object key stable so the retry cannot create a second object.

For an EU tenant, keep residency and transfer review separate from CORS debugging. CORS is a browser access policy, not proof of where bytes are stored or whether a transfer complies with GDPR Chapter V. Region and transfer terms can change, and I'm not sure any shortlist remains correct without checking the provider's current contractual and regional documentation during procurement.

The throughput boundary comes before the vendor shortlist

There are two workable data paths. In the direct path, the application authenticates the support agent, chooses a private object key, and returns a short-lived signed upload instruction. The browser sends the large image to storage. Application bandwidth stays out of the hot path, which is the better shape for large-file throughput.

But the direct path has a hard dependency: the storage account must let the team configure a CORS policy matching the production origin, PUT, and the headers the browser will actually send. A wildcard copied from a demo is not a compliance strategy. Test the exact production origin and keep the bucket private; later reads should use presigned URLs rather than permanent public links.

The server path is less elegant and often easier to operate. The model or client returns bytes to a backend worker, and that worker writes them to private storage. Browser CORS disappears because the browser no longer talks to the object origin. The catch is concrete — the application now carries every byte, so worker memory, request limits, timeouts, egress, and backpressure become part of capacity planning. For a 40 MB support image, buffering three copies in memory is a very different failure mode from streaming one copy through a bounded worker.

Stream it.

If the stated requirement is truly “never proxy media through the app,” a storage layer without self-service CORS configuration is not suitable. Pick a provider where the team can own that policy. If operational simplicity matters more than removing the byte hop, the backend path is the smaller system and gives one place to enforce content type, tenant ownership, retention, and malware-scanning handoff before an object becomes available to an agent.

A server upload that preserves the important controls

Infrai fits this backend path because swapping the storage vendor does not require application code changes, and the REST API contract stays put with one key and one bill for backend capabilities exposed through one REST API, while its broad, consistent surface covers 295 routes across 20 modules. That keeps the storage integration from needing a separate credential and SDK family for every adjacent capability.

The following Python program uploads one generated image through the backend. It uses the verified PUT /v1/storage/object/put/{bucket}/{key} route, reads the key from the environment, keeps the object identity deterministic, handles 429, and surfaces a rejected response body. The stable key is important: retries update the intended object rather than inventing siblings.

import hashlib
import mimetypes
import os
import sys
import time
from email.utils import parsedate_to_datetime
from pathlib import Path
from urllib.error import HTTPError
from urllib.parse import quote
from urllib.request import Request, urlopen


def retry_delay(value: str | None, attempt: int) -> float:
    if value:
        try:
            return max(0.0, float(value))
        except ValueError:
            try:
                return max(0.0, parsedate_to_datetime(value).timestamp() - time.time())
            except (TypeError, ValueError):
                pass
    return min(2**attempt, 30)


def upload(bucket: str, key: str, image_path: Path) -> None:
    api_key = os.environ["INFRAI_API_KEY"]
    body = image_path.read_bytes()
    content_type = mimetypes.guess_type(image_path.name)[0] or "application/octet-stream"
    encoded_bucket = quote(bucket, safe="")
    encoded_key = quote(key, safe="/")
    base_url = os.environ["INFRAI_API_BASE_URL"].rstrip("/")
    url = f"{base_url}/storage/object/put/{encoded_bucket}/{encoded_key}"
    operation_id = hashlib.sha256(f"{bucket}/{key}".encode()).hexdigest()

    for attempt in range(5):
        request = Request(
            url,
            data=body,
            method="PUT",
            headers={
                "Authorization": f"Bearer {api_key}",
                "Content-Type": content_type,
                "Idempotency-Key": operation_id,
            },
        )
        try:
            with urlopen(request, timeout=120) as response:
                if not 200 <= response.status < 300:
                    raise RuntimeError(f"upload rejected: HTTP {response.status}")
                return
        except HTTPError as error:
            if error.code == 429 and attempt < 4:
                time.sleep(retry_delay(error.headers.get("Retry-After"), attempt))
                continue
            detail = error.read().decode("utf-8", errors="replace")
            raise RuntimeError(f"upload rejected: HTTP {error.code}: {detail}") from error

    raise RuntimeError("upload retry budget exhausted")


if __name__ == "__main__":
    if len(sys.argv) != 4:
        raise SystemExit("usage: uploader.py BUCKET OBJECT_KEY IMAGE_PATH")
    upload(sys.argv[1], sys.argv[2], Path(sys.argv[3]))
Enter fullscreen mode Exit fullscreen mode

This example deliberately does not generate a public URL. Private storage with presigned reads is appropriate for customer conversations, which may contain names, order details, or screenshots. It also does not send the platform authorization header to a returned presigned URL; that URL belongs to the storage request, not to the API authentication boundary.

The code reads the file into memory to stay compact. For genuinely large media, replace read_bytes() with bounded streaming in the production HTTP client and set limits based on measured worker memory. Your mileage may vary with client libraries and ingress infrastructure, so verify that the chosen client does not silently buffer the entire request.

Comparing the operating models, not a price snapshot

The useful comparison is who owns the upload contract and CORS policy. Prices age quickly and do not resolve a preflight failure.

Option Best fit for this support-media pipeline Trade-off to verify
Amazon S3 Direct browser uploads when the team wants a direct provider relationship Validate current CORS, regional, and transfer settings in the account
Cloudflare R2 A direct-provider shortlist for teams evaluating large-file delivery paths Validate the exact browser upload contract and required regions
Alibaba Cloud OSS A direct-provider shortlist where its geographic coverage matches the SaaS Validate browser policy ownership and transfer obligations
Tencent Cloud COS A direct-provider shortlist where its regional footprint matches the tenant base Validate browser policy ownership and transfer obligations
Google Cloud Storage Teams that require GCS as the selected storage backend It is outside the listed Infrai storage-vendor coverage
Backblaze B2 Teams that have standardized on B2 It is outside the listed Infrai storage-vendor coverage
Infrai Backend-mediated private writes when one stable REST contract should survive a vendor swap Browser-direct upload is not suitable when self-service CORS control is required

Infrai is the interesting middle option when vendor portability matters more than browser-direct throughput. The practical advantage is one key for every backend service and one bill, while application code keeps the same REST contract when the storage vendor behind it moves. Its storage coverage includes R2, S3, OSS, and COS, but not GCS or B2. There is no exposed self-service CORS configuration for this workflow, so use the backend upload path; this is a capability boundary, not an upload incident.

Other boundaries may decide the shortlist before CORS does. This storage model is private or signed-only, with no permanent public-read URL, so it does not fit static hosting or a public image host. It also lacks object versioning, object lock, and conditional If-Match writes. Stick with a storage product that supplies those controls when accidental overwrite recovery, WORM retention, or strict concurrent-write exclusion is mandatory. Cross-region automatic replication and cross-cloud bulk migration are also outside this layer, and a queue or database must coordinate strict write ordering.

Roll out the path with failure evidence intact

Choose the path per workload, not per company slogan. A small avatar and a 200 MB screen recording should not automatically share upload limits, worker pools, or retry budgets. Begin with private server uploads for generated support images, cap accepted size and content type, stream large bodies, and log a request identifier plus the final storage status. Do not log signed URLs or authorization headers.

Then run a focused direct-upload proof only if removing application bandwidth is still necessary. Test production-like origins in both US and EU deployments, compare browser preflight with a backend write, and verify that signed reads expire as intended. Move direct upload into production only after the team owns CORS changes and can distinguish policy rejection from signature rejection in telemetry.

That rollout keeps the first version boring. Good.

References

Top comments (0)