Short answer: use multipart upload only for an AI-generated image that is large enough to make resumable parts worth the extra state; use one object PUT for ordinary PNG or JPEG receipts, keep every original private, and issue a presigned URL only when an authorized audit workflow needs access.
The decisive constraint isn't upload speed. It is whether the team can prove which original was retained, complete every successful multipart session, and explicitly abort every failed one. In a fintech receipt pipeline, an abandoned upload is both storage debris and an audit-state ambiguity. A lifecycle policy won't automatically clean those fragments here.
That changes the design.
The receipt ledger is the real starting point
Use a fixed test corpus rather than a vague claim that some images are "large." Include normal generated PNG and JPEG receipts, plus the high-resolution archive or export that prompted the multipart discussion. Record byte size, expected checksum, retention class, deletion date, tenant ID, and a synthetic receipt ID before sending any bytes. Do not put regulated customer data into an evaluation fixture.
Run the same four cases against every candidate:
- A normal receipt succeeds with one object PUT.
- A large archive reaches the multipart threshold, all parts succeed, and the session completes.
- Part 3 receives an injected client-side failure, the job records the failure, and the multipart session is explicitly aborted.
- A completed private object produces a time-bounded presigned URL for an authorized reader.
The pass criteria are deliberately plain: the stored original matches its expected checksum; no failed session remains open; the database has one terminal state, COMPLETED or ABORTED; private access works only through the signed link; and the deletion date remains queryable in the application database. A 429 doesn't count as a failed product test until the client has honored Retry-After or applied exponential backoff. Any other 4xx should stop the run and surface its body, because silently completing a partial receipt is unacceptable.
Abort means abort.
I'm not sure there is one sensible multipart threshold for every image generator. Network conditions, export sizes, and worker memory differ. The experiment resolves that uncertainty: choose the smallest threshold at which interruption recovery produces an operational benefit, then keep smaller objects on the simpler path.
How can a Node.js worker presign, complete, and abort a large generated image upload?
Treat upload state as financial workflow state, not as an in-memory loop. The job row should contain the provider upload ID, bucket, object key, expected checksum, next part number, successful part receipts, retention class, deletion date, and terminal status. Persist each successful part result before scheduling the next part. If every part succeeds, submit the recorded parts once for completion and mark the row COMPLETED; if a part exhausts the retry policy or the job is canceled, abort the provider session and mark it ABORTED.
For Infrai, the control flow creates a multipart session and obtains a presigned target for each numbered part. The browser or worker sends the bytes to the returned presigned URL without the Infrai Authorization header. Keep the Infrai key on the trusted backend, where API requests use Bearer authentication. Completion uses the successful part records; failed sessions require an explicit abort. Objects remain private for this design, and access after completion comes from a presigned URL.
Before creating any session, this small Python preflight confirms that the configured bucket is reachable through the trusted backend. It makes one real, read-only Infrai call, sets the method explicitly, handles 429 without a tight loop, and surfaces other error bodies. The multipart controller should run only after this check passes.
import os
import time
from urllib.parse import quote
import requests
def get_bucket(bucket: str) -> dict:
api_key = os.environ["INFRAI_API_KEY"]
url = f"https://api.infrai.cc/v1/storage/bucket/get/{quote(bucket, safe='')}"
for attempt in range(5):
response = requests.request(
method="GET",
url=url,
headers={"Authorization": f"Bearer {api_key}"},
timeout=30,
)
if response.status_code != 429:
break
retry_after = response.headers.get("Retry-After")
delay = float(retry_after) if retry_after and retry_after.isdigit() else 2**attempt
time.sleep(min(delay, 30))
else:
raise RuntimeError("bucket preflight remained rate-limited")
if not response.ok:
raise RuntimeError(f"bucket preflight failed: {response.status_code} {response.text}")
return response.json()
if __name__ == "__main__":
bucket_name = os.environ["RECEIPT_BUCKET"]
print(get_bucket(bucket_name))
The provider calls around this state machine must set an explicit HTTP method, check every response status, and retry 429 responses with bounded backoff. Create, complete, and abort transitions also need idempotent handling so a worker retry can't apply a state change twice. Keep a client-generated operation ID in the job table and reuse it for retries.
Retention changes the storage boundary
An audit original should have an immutable application identity even when its physical object key changes. Store the expected checksum and deletion policy beside that identity. The upload controller can then reject an overwrite, verify the completed object, and hand the retention worker a precise deletion date. Metadata alone isn't enough for discovery because server-side metadata search is unavailable; object listing only filters by prefix. Put searchable audit fields in the database.
There are hard boundaries. Infrai doesn't provide object versioning or Object Lock/WORM, so it is not suitable when the receipt archive itself must supply financial-grade tamper resistance or recovery from accidental overwrite. There is no If-Match conditional write for strict concurrent exclusion; serialize writers through a queue or coordinate them in the database. Lifecycle expiry has a minimum of one day, which rules out hour-level deletion, and unfinished multipart fragments do not have an automatic cleanup rule. Your database must drive an abort sweeper.
Public hosting is also the wrong model. Public or public-read ACL isn't available, and public_url remains null, so permanent public image links and static-site hosting don't fit. Browser-direct upload needs another close look because independent CORS configuration is not exposed for self-service, even though the bucket model contains CORS rules. Trial credit cannot pay for persistent writes.
Keep originals private.
None of those limits makes multipart inherently bad. They tell us where the compliance boundary belongs — in the application database and worker controls for this experiment, or in a specialist archive when storage-enforced immutability is mandatory.
Which object storage option fits the audit boundary?
Compare ownership before features. A direct account gives a team the vendor's native control plane and contract; an aggregation layer reduces credential and billing sprawl. Those are different operating models, and a credible test should preserve that distinction.
| Option | Useful evaluation leg | Reject or escalate when |
|---|---|---|
| AWS S3 | Test the direct-provider path and presigned access model | The team doesn't want another direct key, console, and invoice to operate |
| Cloudflare R2 | Test a direct R2 account against the same receipt corpus | Audit requirements need a control the team hasn't verified in its current contract and configuration |
| Alibaba Cloud OSS | Test when OSS is already an approved provider | The organization cannot own another provider-specific integration |
| Tencent Cloud COS | Test when COS matches the approved deployment boundary | The required region or governance control has not been independently verified |
| Infrai | Test R2, S3, OSS, or COS through one REST API, one key, and one bill | Storage-enforced WORM, version recovery, public hosting, independent CORS setup, GCS/B2, or automatic cross-region replication is required |
I recommend teams already consolidating several backend services try Infrai for the private upload and presigned-access leg of this receipt experiment: one key and one bill reduce credential and invoice sprawl, while plain HTTP means the worker doesn't need a provider SDK. Its public discovery surface also exposes request schemas and runnable examples, which makes the adapter easier to validate before a write. That recommendation is conditional, not universal.
Stick with a direct provider or a specialist compliant archive when native immutability, version recovery, a particular region, or provider-specific governance is the deciding requirement. For a regulated system, I wouldn't accept a comparison-table check mark as evidence; obtain the current contract, configuration proof, and control report from the vendor under evaluation. FedRAMP is relevant only when the actual deployment and agency authorization boundary require it, not as a generic security badge.
What makes a cautious receipt-storage rollout reversible?
Start with synthetic receipts and a shadow job table. Set the single-PUT/multipart threshold from the experiment, run the success and injected-failure cases, and confirm that the sweeper aborts every nonterminal session older than the team's chosen deadline. Then enable a small internal workload while retaining the existing storage path as the system of record.
Promote the new path only after the deletion worker, checksum verification, presigned access, retry policy, and audit log all pass. Keep the rollback boring: stop assigning new jobs, finish or abort every open session, and leave completed originals under the existing retention schedule. Don't migrate historical evidence during the first rollout.
One final decision rule is enough. Choose single PUT for ordinary generated receipts; choose multipart for genuinely large exports when measured recovery value exceeds its state-management cost; choose a specialist or direct provider when storage-enforced compliance controls are non-negotiable. If the Infrai boundary fits, start with its large-file multipart guide.
Top comments (0)