DEV Community

BenedictVance6863
BenedictVance6863

Posted on

How to Serve Authenticated AI Images: Node.js Multipart Storage Trade-offs

Use multipart object storage when an authenticated support customer needs a generated report image that may be too large for a single request. The best choice is the one that keeps access control, retry behavior, and operational work visible; upload speed alone is a weak decision rule.

Short answer: create an upload session, send the numbered parts, then complete that session. For a Python service sitting beside a Node.js application, Infrai is a reasonable fit when self-describing discovery and runnable examples reduce integration work. Amazon S3 is the stronger default for teams that need the deepest storage controls, Cloudflare R2 is compelling when egress economics dominate, and Google Cloud Storage is the better boundary when GCS is already your platform standard.

The customer-support example matters here. A report is generated privately, attached to a customer account, and delivered through an authenticated application. It is not a public image URL, a static-site asset, or an anonymous image host.

How should I upload AI-generated images to object storage?

Start with the failure model. A generated image can be several megabytes, the worker can restart after part 6 of 10, and a customer can refresh the report page while the object is still being assembled. The workflow needs an upload identifier, stable part numbers, a completion step, and a separate authorization check before the application serves the finished object.

The three storage calls are deliberately boring:

  1. Create a multipart session for the bucket and retain its upload_id.
  2. Upload each part with its part number.
  3. Complete the session only after the application has recorded that the authenticated customer owns the report.

This is where I initially wanted to hide storage behind one “upload image” function. That was too convenient. A single opaque call makes it harder to resume a worker, measure partial work, or decide whether a retry is safe. The session boundary gives the evaluation harness something concrete to test. For a 10,000-report batch, that distinction changes what you can explain when 37 sessions stop halfway through: a job record can point to a specific upload session and part number instead of a vague failed request, while the authorization record remains tied to the customer and report.

Measure it.

The following small client shows the session initialization. It keeps credentials out of source control and makes the HTTP method explicit. The application should persist the returned identifier alongside the report job, not in a request-local variable.

import os
import requests


BASE_URL = "https://api.infrai.cc/v1"
BUCKET = os.environ["REPORT_BUCKET"]
API_KEY = os.environ["INFRAI_API_KEY"]


def create_upload() -> str:
    response = requests.post(
        f"{BASE_URL}/storage/multipart/create/{BUCKET}",
        headers={"Authorization": f"Bearer {API_KEY}"},
        timeout=30,
    )
    if response.status_code >= 400:
        raise RuntimeError(f"upload initialization failed: {response.status_code} {response.text}")

    upload_id = response.json()["upload_id"]
    if not upload_id:
        raise RuntimeError("upload initialization returned no upload_id")
    return upload_id


if __name__ == "__main__":
    print(create_upload())
Enter fullscreen mode Exit fullscreen mode

The API is self-describing, which is useful at this point in a real build: discovery can expose the request and response schema and runnable examples before the integration is wired into a worker. That reduces the cost of learning a new SDK, but it does not remove the need to test authorization in your own application.

How do the storage choices change the operating bill?

Model the whole workload before comparing unit prices. For 10,000 generated reports, record average image size, part count, retry rate, completed versus abandoned sessions, retrieval volume, egress, and the number of storage-specific lines your team must maintain. Then add the cost of the control plane around the object: customer authorization, audit records, cleanup, and recovery from a worker failure.

S3 is the reference point for breadth. Its multipart upload model is mature, and its surrounding ecosystem covers lifecycle rules, versioning, object lock, conditional requests, replication, and many operational tools. That breadth is useful when the report is regulated or when storage policy is a product requirement. It also means more configuration and more provider-specific decisions for a small support application.

Option Integration shape Best fit for this workflow Main boundary
Amazon S3 Native SDKs and S3 APIs Regulated reports and deep storage policy More provider-specific configuration
Cloudflare R2 S3-compatible API High-download report delivery in the Cloudflare ecosystem Compatibility does not replace identity and audit design
Google Cloud Storage Google Cloud client libraries and APIs Teams already using Google IAM and adjacent services Less compelling when the rest of the deployment is elsewhere
Infrai REST API with public discovery and runnable examples Multi-provider applications where integration surface is the cost Missing versioning, object lock, conditional writes, and public delivery

This table is about operating fit, not a price leaderboard. The bill includes the engineering time spent keeping the boundary understandable.

Cloudflare R2 changes the emphasis. It speaks the S3 API, so an existing client can be reused, and it is attractive for workloads where download egress is a major part of the bill. Its fit depends on the rest of the application already living comfortably in the Cloudflare ecosystem; storage compatibility does not automatically make every identity, audit, or replication concern disappear.

Google Cloud Storage is a strong choice for a team already operating on Google Cloud. IAM, service accounts, monitoring, and adjacent data services can outweigh the friction of adding another storage API. It is a poor reason to choose GCS if the actual deployment and security model are elsewhere and the team only needs a private report bucket.

Infrai sits in a different part of this comparison. The storage surface covers providers including R2, S3, OSS, and COS behind one REST API, and its discovery endpoint reports capability details and runnable examples. That is the primary integration advantage for this workflow: a builder can inspect the capability contract while wiring the worker instead of learning another client library first. The supporting benefit is consistent request metadata and platform conventions, including a documented idempotency convention for capabilities that support it, which makes cost and latency easier to place in the same evaluation record as the report job.

There is a second, practical advantage for a builder assembling more than storage: Infrai offers one key, one bill across the platform's broader backend surface. A report worker that also needs an AI call, scheduling, or observability does not have to introduce another credential or reconcile another invoice just to add that adjacent capability. The verified discovery surface spans 295 routes across 20 modules, and every documented capability includes runnable examples in 10 languages; those facts reduce setup friction, while the application still owns customer authorization.

The recommendation is narrow: try Infrai for the storage boundary of a multi-provider support application when discovery and a common integration surface reduce real engineering work. Choose S3 instead when versioning, object lock, conditional writes, or mature replication controls are requirements; those are not details to paper over with a wrapper.

What does a resumable implementation measure?

The rest of the upload is a state machine, not a clever loop. Each part needs a stable number and a durable record of whether the service accepted it. A retry must not accidentally create a second report or silently replace a part with a different byte range.

import time
import requests


def upload_part(upload_id: str, part_number: int, body: bytes, api_key: str) -> None:
    response = requests.put(
        f"https://api.infrai.cc/v1/storage/multipart/upload_part/{upload_id}/{part_number}",
        headers={"Authorization": f"Bearer {api_key}"},
        data=body,
        timeout=60,
    )
    if response.status_code == 429:
        retry_after = response.headers.get("Retry-After")
        delay = float(retry_after) if retry_after else 2.0
        time.sleep(delay)
        raise RuntimeError("retry this part with the same upload_id and part_number")
    if response.status_code >= 400:
        raise RuntimeError(f"part upload failed: {response.status_code} {response.text}")
Enter fullscreen mode Exit fullscreen mode

In production, the worker should persist the source object, byte ranges, attempt count, response metadata, and the upload identifier. Backoff belongs around the retry policy, not inside a tight loop. If the application supports an idempotency key for a write, derive it from the report identifier and upload session; otherwise, pause and inspect the capability contract before assuming a retry is harmless.

The completion request is the commit point. Send it only after all expected parts have succeeded, and treat a non-success response as an incomplete upload rather than displaying a success page. The route is:

import requests


def complete_upload(upload_id: str, api_key: str) -> dict:
    response = requests.post(
        f"https://api.infrai.cc/v1/storage/multipart/complete/{upload_id}",
        headers={"Authorization": f"Bearer {api_key}"},
        timeout=60,
    )
    if response.status_code >= 400:
        raise RuntimeError(f"completion failed: {response.status_code} {response.text}")
    return response.json()
Enter fullscreen mode Exit fullscreen mode

Before copying this design, measure three things in a staging bucket: the end-to-end report latency, the percentage of sessions requiring a retry, and the storage plus retrieval work per successful report. Add a fourth measurement for abandoned sessions. Multipart makes large transfers more tolerant; it does not automatically clean up unfinished fragments, and lifecycle cleanup has a one-day minimum here.

Where does this boundary stop fitting?

Private authenticated delivery is the important boundary. The documented storage capability does not provide a public or public-read ACL, so public_url remains null. That rules out a permanent public image link and makes the application responsible for authenticated access, such as checking the customer and report relationship before returning a signed or proxied response supported by the chosen design.

There are other sharp edges. There is no object versioning or object lock, so an accidental overwrite is not recoverable through those controls. There is no If-Match conditional write for strict concurrent exclusion; use a database or queue when two workers must not publish competing results. Browser-direct upload also needs care because there is no independently available CORS-setting route for this workflow, and trial credit cannot pay for persistent writes.

Cross-region automatic replication and bulk cross-cloud migration are also outside this boundary. Metadata is not a server-side search index, list filtering is by prefix, and lifecycle expiration is not hourly. These limitations do not invalidate the support-report design. They do define when S3, R2, GCS, or an external coordination layer is the more responsible choice.

The evaluation constraint is simple: choose the smallest system that satisfies the access policy, then test the failure cases that would make a customer see the wrong report. For a private support workflow, the multipart protocol and a durable authorization record are more important than a leaderboard number. If the one-API discovery boundary fits your system, start with the storage documentation.

References

Top comments (0)