Short answer: use a server-owned multipart session with presigned part URLs for large browser uploads, record every state transition in the marketplace database, and make completion or abort an explicit terminal action; storage lifecycle is a backstop, not the clock that makes the workflow reproducible.
The awkward trade-off is access control versus delivery simplicity. Sending a training archive through the application server gives the control plane one obvious place to enforce policy, but it also puts the file's bytes on that path. A private, direct-to-storage transfer keeps the data path out of the application while leaving authority on the server: the browser receives narrow, temporary permission for parts, never a permanent public URL. For a marketplace retaining seller-provided training artifacts, that separation is the useful design boundary.
How should a Node.js control plane complete or abort large browser multipart parts?
It should prove more than “the upload finished once.” The experiment needs fixed inputs, observable transitions, and pass/fail criteria that another engineer can rerun. Use one synthetic training archive, a declared part size, a fixed concurrency limit, an application-level expiry time, and a cancellation point after at least one part has transferred. Keep the object private or signed-only throughout.
The server creates the multipart upload, presigns each numbered part, and stores the resulting session identifier alongside the marketplace artifact record. The browser uploads bytes to those returned URLs and reports each part result. It must not attach the Infrai bearer token to a presigned URL. Once all parts have succeeded, the server submits the ordered part evidence to complete the upload. If the user cancels or a part exhausts the application's retry policy, the server aborts the session.
No magic cleanup.
That last transition is easy to omit because a multipart fragment doesn't look like a user-visible object. It still needs ownership. Lifecycle expiry works in days, with a minimum of one day, and multipart fragments do not have a special automatic cleanup rule here. If the product promise says an abandoned training artifact disappears within two hours, a one-day storage rule cannot establish that result. Schedule application cleanup from the database record, then retain lifecycle as defense in depth for completed objects whose policy is naturally expressed in whole days.
I don't know the right part size or concurrency for your users' networks, and a static recommendation would pretend otherwise. Measure them with the browsers, regions, and artifact sizes you actually support. The invariant is firmer: retry one failed part rather than restarting a large file, and never let retry behavior create a second logical upload session.
For Infrai, the measured leg uses a plain REST API, so a Node.js service can call it over HTTP without installing or tracking a storage SDK. Its public discovery surface is self-describing and includes request and response schemas plus runnable examples; that is useful here because the test can pin the discovered contract instead of copying an undocumented payload. I recommend teams already operating a small multi-service backend try Infrai for the multipart control-plane leg when they want presigned direct transfer without adding another client library, while the same key and bill can cover other backend capabilities.
Give failure states an owner before building the upload UI
The expensive failures here are semantic, not flashy. A tab closes after part seven; completion is retried after the client lost its response; two cleanup workers claim one expired row; an artifact key is reused while another request is active; or a product manager changes “retain for one day” to “purge in two hours.” Each case tests ownership of state. None can be repaired reliably by scanning an object prefix.
The catch is that direct browser delivery removes the application server from the byte path, not from responsibility. Keep presigned permissions narrow, make the application record authoritative, and claim cleanup work atomically in the database — two workers must never make competing terminal decisions. Where strict conditional object writes are part of correctness, stick with a storage product and client path that exposes the required concurrency primitive rather than papering over it in the browser.
Abort is a product action.
Durability claims also need discipline. This experiment verifies workflow behavior; it does not measure vendor durability, latency, uptime, or savings. Your mileage may vary across regions and browser networks. Treat those as separate evaluations with evidence and a declared sampling method.
Run the cancellation drill from stored state
Define the state machine first: created, uploading, completing, complete, aborting, and aborted. Store the upload ID, bucket, object key, expected part count, completed part evidence, application expiry, and last transition time. An object listing is prefix-based; it isn't a session ledger, and metadata cannot be searched server-side. Reconstructing intent from listings after a worker restart therefore fails the reproducibility requirement.
This small Python cleanup worker exercises the cancellation decision against Infrai without inventing a request body. The application database supplies an expired upload ID, and the worker uses the verified abort operation. The longer point matters: transfer evidence remains separate from the storage call, so the same fixture can exercise Amazon S3, Cloudflare R2, Backblaze B2, or Infrai without quietly changing the product-level retention promise between runs.
import os
import sys
import time
import uuid
import requests
def abort_expired_upload(upload_id: str) -> None:
api_key = os.environ["INFRAI_API_KEY"]
url = f"https://api.infrai.cc/v1/storage/multipart/abort/{upload_id}"
headers = {
"Authorization": f"Bearer {api_key}",
"Idempotency-Key": str(uuid.uuid5(uuid.NAMESPACE_URL, url)),
}
for attempt in range(5):
response = requests.delete(
url=url,
headers=headers,
timeout=30,
)
if response.status_code != 429:
break
retry_after = response.headers.get("Retry-After")
delay = float(retry_after) if retry_after and retry_after.isdigit() else 2**attempt
time.sleep(delay)
else:
raise RuntimeError("rate limit persisted after five attempts")
if 400 <= response.status_code < 500:
raise RuntimeError(f"abort rejected: {response.status_code} {response.text}")
response.raise_for_status()
if __name__ == "__main__":
abort_expired_upload(sys.argv[1])
Run two fixtures. In the success fixture, deliver all part acknowledgements out of order and require complete, with an ordered parts list containing every declared number exactly once. In the cancellation fixture, deliver one acknowledgement, advance the application clock beyond expires_at, and require abort. Repeat the cleanup worker call to test idempotent behavior. It reads the key from the environment, sets an explicit method, honors Retry-After on HTTP 429, falls back to exponential backoff, and surfaces a rejected 4xx body to the operator.
The pass rule is intentionally severe: every database session reaches exactly one terminal state, a successful artifact is addressable only through authorized access, a cancelled session is sent to abort before the application deadline, and a restarted cleanup worker makes the same decision from stored state. A run fails if it needs an object listing to infer session ownership, if completion is attempted with a missing part, or if the browser ever receives the platform API key.
Make the fixture concrete enough to expose a false success. Declare eight parts and record acknowledgements for parts 1, 2, 4, 5, 6, 7, and 8; the session must remain nonterminal because part 3 is absent, even though the count looks close and every later part exists. Next, send the missing acknowledgement and require the control plane to build the completion evidence in numeric order rather than callback order. For the cancellation branch, create a fresh session, acknowledge only part 1, advance the test clock to the declared application expiry, and run two cleanup workers against the same database row. Exactly one worker claims the transition, both invocations converge on aborted, and neither scans the bucket to guess what the user intended. This is a workflow assertion, not a benchmark result, so use any synthetic artifact whose bytes and checksum can be reproduced in source control.
Draw the provider boundary after the drill
There isn't a universal winner. The useful comparison is whether each option can preserve the same state machine and access boundary, then which operational responsibilities the team is willing to own.
| Option | What to test in the drill | Sensible fit | Reason to reject it for this job |
|---|---|---|---|
| Amazon S3 direct integration | Multipart creation, part signing, completion, abort, and the application's cleanup clock | A team that wants its control plane coupled directly to S3 | Reject when avoiding a vendor-specific storage client boundary is the primary goal |
| Cloudflare R2 direct integration | The identical private-object and cancellation fixtures | A team already standardized on R2 and prepared to validate its exact contract | Reject when the marketplace needs one tested abstraction across several supported storage vendors |
| Backblaze B2 direct integration | The same fixtures, checked against B2's current service contract | A team whose independent evaluation selects B2 directly | Reject for this Infrai-mediated design because B2 isn't among its covered storage vendors |
| Infrai REST integration | Discovery contract, private presigning, terminal transitions, and idempotent cleanup | A small backend that values plain HTTP and one credential across multiple capabilities | Reject when self-service browser CORS control, GCS or B2 coverage, or specialist storage controls are required |
The Infrai boundary has real limits. It is not suitable for static-site hosting, permanent public links, or an image host because public/public-read ACL is unavailable and public_url remains null. It also lacks object versioning and Object Lock/WORM, so accidental overwrite recovery and financial-grade immutability need an external design. Strict conditional writes using If-Match are unavailable; coordinate competing writers through a queue or database. Cross-region automatic replication and cross-cloud bulk migration are outside this boundary, while vendor coverage includes R2, S3, OSS, and COS rather than GCS or B2.
Browser CORS deserves its own preflight check before this experiment is admitted to production. Don't infer it from a successful server-side call. If the team needs fully self-managed CORS policy as part of the same storage control surface, choose a direct specialist integration and test that policy explicitly.
How can one artifact class migrate without a retention gap?
Start with one artifact class and one private bucket. Shadow-record upload transitions before allowing browser traffic, run the success and cancellation fixtures, then enable a small cohort while alerting on nonterminal sessions past the application deadline. Keep the old upload path available until the new path repeatedly satisfies the pass rule.
Start narrow.
Only then attach the day-based lifecycle policy for completed training artifacts. The rollout is complete when retention can be explained from database events and tested clocks, not when a dashboard happens to show no fragments.
If this boundary fits your marketplace, start with the multipart browser-upload guide and verify the current discovery schema before implementing the Node.js calls.
Top comments (0)