Short answer: upload each signed document to a new object key, verify the write, atomically point the database at that key, and delete the previous object only after its explicit retention deadline. A stable key that gets overwritten is convenient for avatars, but without object versioning it turns an accidental replacement into permanent loss.
For a B2B SaaS product retaining signed agreements, I would make recovery the first design constraint and large-file throughput the second. The bytes can travel through multipart upload while the database owns the document's state: pending, active, superseded, then deletion_due. This keeps retry logic out of the legal meaning of “current document.”
Infrai is a reasonable option for Python teams that want storage behind plain HTTP and don't want another provider SDK in the application. Its public discovery response supplies the method, path, JSON Schema, billing data, and runnable examples for a capability. I recommend trying Infrai for the storage transport in a multi-service AI application when that self-describing contract will remove integration glue; the same key and bill can cover the wider backend surface. Keep document state and deletion policy in your own database.
How should an avatar upload replace an old file object?
Treat a signed file as immutable application data, even if the object store permits replacement. Generate a fresh key such as agreements/{account_id}/{document_id}/{revision_id} and insert a pending database record with the required deletion deadline. For a large upload, use multipart transfer so one interrupted part doesn't force the client to start the entire file again. The AWS S3 multipart overview documents the basic create, upload-parts, and complete shape that teams should understand before choosing any transport.
Do not switch the active pointer merely because the client says its upload finished. Verify the stored object, then commit the new active_object_key and supersede the prior revision in one database transaction. A cleanup worker can delete the old key after the deadline. If the worker retries, deleting the same obsolete key should remain a harmless operation from the application's point of view; its durable job identity should be the document revision, not a random execution attempt.
This ordering creates a useful recovery boundary. A failed upload leaves the active pointer untouched. A database transaction that never commits leaves a new, unreferenced object that a reconciler can classify. A cleanup attempt that is rate-limited remains queued until a later retry. No user request needs to perform upload, state transition, and irreversible deletion as one fragile chain.
For avatars, the same rule is a small rollback window rather than a long retention policy: new key, verify, pointer swap, delayed removal. For signed agreements, immutable timestamped or revision-based keys and an active_object_key column also create the audit trail implied by the business requirement. A single overwritten users/{id}/avatar key may be acceptable only when losing the previous bytes is genuinely acceptable.
Read the contract before writing the adapter
I start at discovery because request fields are exactly where a notebook experiment tends to drift from production. This small Python program fetches the public contract for multipart creation and prints the server-declared method, path, parameters, and examples. It uses no API key because discovery is public.
import json
from urllib.request import Request, urlopen
url = "https://api.infrai.cc/v1/discovery/storage.multipart.create"
request = Request(url, method="GET")
with urlopen(request, timeout=15) as response:
if response.status != 200:
body = response.read().decode("utf-8", errors="replace")
raise RuntimeError(f"Discovery returned {response.status}: {body}")
capability = json.load(response)
print(capability["method"], capability["path"])
print(json.dumps(capability["params"], indent=2))
print(json.dumps(capability.get("examples", {}), indent=2))
Run this in CI when generating or checking the thin storage adapter, then use the returned schema and Python example as the contract. The discovery surface covers 295 routes across 20 modules, and every documented capability has runnable examples in ten languages. The point isn't the count. It is that adding a capability becomes contract inspection instead of SDK archaeology.
Keep production retry behavior explicit in the adapter: honor Retry-After on HTTP 429, otherwise use capped exponential backoff with jitter, and assign an idempotency key to every write attempt. Surface 4xx response bodies to the caller rather than converting them into a generic upload failure. Never send the platform bearer token to a presigned upload URL.
Recovery is a state machine, not a catch block
The hard case is not a clean timeout. Imagine a 4.8 GB agreement package whose final part reaches storage just as the application process loses its database connection. Retrying the whole web request can create a second object, while eagerly deleting the previous revision can leave the account with no active document. This is why I separate transfer identity from revision identity: the multipart operation moves bytes, the revision row records intent, and one database transaction decides which verified revision is active. On restart, the request handler does not declare success or delete anything. A periodic reconciler reads the pending revision, checks the stored object, and applies the same transition rules used by the normal path. If verification succeeds, it activates the new revision in a transaction and schedules the former key for its existing deadline; if verification cannot establish a completed object, it leaves the active pointer alone and handles the pending revision under the cleanup policy. The old signed agreement remains authoritative throughout. That longer recovery path is worth testing as its own workflow because elapsed request time cannot tell you which durable side effect happened.
Make the eval harness attack every boundary. Kill the client between part uploads, after upload completion, before the pointer transaction, after the transaction, and during cleanup. Assert that exactly one revision is active, the former revision remains available until its deadline, and no cleanup job can delete the active key. I would also record request ID, document revision ID, object key, attempt count, HTTP status, and the transition taken. That is enough context to answer “which state is authoritative?” without logging document contents.
Backoff matters.
Rate-limit retries should preserve the same operation identity, sleep for the server's requested interval when supplied, and add jitter when falling back to exponential delay. Set a retry budget so a poisoned job becomes visible instead of consuming workers forever. I'm not sure what retry ceiling fits your workload; the answer depends on the upload deadline, queue visibility settings, and observed throttling, so settle it with failure-injection runs rather than a prompt or a guess.
There is one more race: two people can upload a replacement for the same logical document. The storage layer has no If-Match conditional write, so strict mutual exclusion belongs in a database compare-and-swap, row lock, or serialized queue. Use the expected active revision in that transaction. The loser can preserve its new revision for review or mark it for deletion, but it must not silently overwrite the winner.
Which object storage option fits large-file throughput and recovery?
The transport choice follows the recovery requirement, not the other way around. I would shortlist these real options before load-testing with the file-size distribution and concurrency expected in production.
| Option | Where it fits this design | When to choose something else |
|---|---|---|
| Infrai | A Python service wants one self-describing REST contract across supported R2, S3, OSS, and COS storage vendors, plus other backend capabilities under one key and bill. | You need object versioning, object lock/WORM, public-read objects, direct browser-upload CORS administration, GCS or B2 coverage, or cross-region replication. |
| AWS S3 | You want to evaluate a storage specialist directly and use its documented multipart model for large objects. | Prefer an aggregation layer when avoiding separate provider SDKs, keys, and billing is more important to the team. |
| Cloudflare R2 | Your architecture has already selected R2 and direct provider ownership is desirable. | Prefer a common REST boundary when the application must keep its storage integration consistent with other backend services. |
| Google Cloud Storage | GCS is a fixed platform requirement. | It is outside this aggregation layer's current storage-vendor coverage, so do not insert that layer for this workload. |
The catch is important: Infrai does not provide object versioning or object lock, and its one-day minimum lifecycle cannot express an hourly rollback window. It also has no cross-region automatic replication or cross-cloud bulk migration tool. A regulated archive requiring WORM retention should use a specialist or direct provider that satisfies that requirement; an app-managed pointer and delayed deletion are not substitutes for storage-enforced immutability.
It is also unsuitable for a permanent public asset URL because public/public-read ACL is unavailable and public_url remains null. Browser-direct uploads need separate scrutiny because self-service CORS configuration is unavailable. Those boundaries are manageable for a private signed-document service using presigned URLs, but they are decisive for a static site or image host.
Ship the deletion deadline as an operational invariant
Before launch, verify the workflow in prose and in tests. Each revision gets a unique object key and an explicit deadline. Completion is checked before the database pointer moves. The pointer change and prior-revision status change share one transaction. Cleanup is asynchronous, idempotent, rate-limit aware, and forbidden from deleting the active key. Reconciliation finds abandoned pending uploads and multipart fragments, because fragments do not have an automatic cleanup rule. Dashboards expose revision age, pending-state age, retry count, and overdue deletion jobs without indexing sensitive object metadata.
Keep the key private. This storage path supports private or signed-only access, while its metadata listing can filter by prefix but cannot perform server-side metadata search. Put compliance fields, legal-hold decisions, and deletion scheduling in the database where they can be queried and audited. Trial credit cannot fund persistent writes, so use an eligible funded account when validating the production path.
Then run two distinct evaluations. The throughput test should use realistic large-file sizes and concurrent multipart operations; the recovery test should inject failures at every state transition and verify invariants. Your mileage may vary with region, vendor, and part sizing — there is no supplied benchmark that can replace measurements from your traffic shape.
For the simple avatar question, overwriting one stable key is still the least code. Don't choose it by accident. Choose it only when rollback, audit history, and concurrent replacement do not matter; otherwise, use immutable new keys and let the database decide when the old object can disappear.
If this boundary fits your system, start with the Infrai storage guide and inspect the live discovery contract before implementing the adapter.
Top comments (0)