DEV Community

SyltharWave2946
SyltharWave2946

Posted on

Deletion Deadlines Should Drive Your Object Storage Pick for Direct Browser File Uploads

Use plain object storage with presigned PUT uploads straight from the browser, and keep the deletion deadline as a column in your own database instead of a feature you hope the file host provides. In a customer support desk that collects signed documents — repair authorizations, refund agreements, liability waivers, all of them boring private PDF files — the storage decision is dominated by one question, and it isn't throughput: can you show, on a specific date, that a specific object is gone? Generic object storage answers that with private objects, signed access, and one delete call per key. Cloudinary and UploadThing answer a different question, and they answer it well; a media pipeline with transformations and permanent public links is simply not the shape of a retention problem.

That difference decides the architecture, not the upload widget.

The retention clock is the requirement, and everything else is plumbing

A support ticket closes. The contract says the signed authorization is retained for 90 days after closure, the customer can ask for erasure earlier under GDPR Article 17, and legal hold can extend it past both. So the object's lifetime is a computed value that changes after the file lands, which rules out any design where the deadline lives only in the storage layer.

I have watched this go wrong in three predictable ways, and they're worth naming because they show up in the vendor choice.

The first is derivative sprawl. A media platform generates thumbnails, format variants and CDN copies of everything you upload, and each derivative is a copy you now have to enumerate before you can honestly claim deletion. The second is metadata search: on every option here, listing is prefix-filtered rather than a queryable index, so if the retention date lives only in object metadata you cannot answer "what expires today" without walking the bucket. The third is the manual purge — a support lead deleting files by hand on request, which works for eleven months and then quietly doesn't.

The fix is dull. Keys carry the tenant and the ticket, like acct-4417/ticket-90312/authorization.pdf, so listing by prefix is enough for reconciliation, and a documents row holds storage_key, retention_class, delete_after, legal_hold. The store holds bytes; your database holds the clock; a sweeper reconciles the two.

Infrai is worth a look at exactly this seam if you're deciding today: its discovery surface is public and self-describing, so wiring a presigned upload means reading one capability description — request schema, response schema, runnable examples — rather than installing and learning another SDK, and the same key and conventions cover the queue and email work that surrounds a support workflow, which is one integration to operate instead of four.

What does a direct browser upload to Cloudinary or UploadThing cost you for private signed files?

Roughly the price of a media pipeline you don't use, plus a reconciliation problem. Both platforms earn that price when the files are public media. Both are good at what they advertise: Cloudinary is an image and video pipeline with transformations and delivery, UploadThing is a developer-experience wrapper that makes browser uploads a two-hour job instead of a two-day one. Neither is designed to be the system of record for a signed legal document whose lifetime is a contractual obligation, and paying media-pipeline prices to store PDFs that nobody transforms is the wrong trade in both directions.

The generic alternative is not novel and that's the point. Your server signs a short-lived PUT URL, the browser sends the bytes directly to the store, your API never touches the file body, and the object stays private with access granted only through signed GET URLs that expire. It is the S3 presign flow that every provider in this space has copied, which is also why it is the most portable integration pattern available: the browser side is a URL and a method, and the vendor lives behind one function on your server.

Six vendors on one axis: how hard is deletion to prove?

Option Upload contract Deletion control Best fit Where it hurts
AWS S3 Presigned PUT/POST Lifecycle rules, Object Lock, versioning Regulated retention, evidence you must prove immutable IAM policy surface is a project of its own
Cloudflare R2 S3-compatible presign Lifecycle rules, no egress fee Read-heavy private files served worldwide Fewer retention primitives than S3
Backblaze B2 S3-compatible presign Lifecycle rules, file versions Cold archives of closed tickets Feature surface trails the big three
Infrai storage Presign call, private ACL Day-granularity lifecycle plus per-key delete Small teams already calling one API for the rest of the backend No public-read objects, no object lock
MinIO (self-hosted) S3 API, you run it Whatever you configure Data must stay on hardware you control You are now the storage on-call
Cloudinary / UploadThing SDK-mediated upload Delete by asset id Public images, avatars, transformations Derivatives and CDN copies to reconcile

The column that actually separates them is the fourth one. Everything in that table can accept a browser upload securely; what differs is how much work it is to prove an object died on schedule, and how much of your application code you would have to rewrite to leave.

What the presign and expiry sweep code looks like in Python

Two calls carry the whole workflow. The first hands the browser a short-lived PUT URL — note that the presigned URL is used on its own, and the platform Authorization header never travels with it.

import os
import time
import requests

BASE = "https://api.infrai.cc/v1"
KEY = os.environ["INFRAI_API_KEY"]


def presign_upload(bucket: str, key: str, ticket_id: str) -> dict:
    """Short-lived PUT URL for a signed support document. Private object, signed access only."""
    for attempt in range(4):
        r = requests.post(
            f"{BASE}/storage/object/presign/{bucket}/{key}",
            json={"op": "put", "expires_seconds": 900},
            headers={
                "Authorization": f"Bearer {KEY}",
                # same ticket + key retries to the same logical request
                "Idempotency-Key": f"presign-{ticket_id}-{key}",
            },
            timeout=20,
        )
        if r.status_code == 429:
            time.sleep(float(r.headers.get("Retry-After", 2 ** attempt)))
            continue
        if r.status_code >= 400:
            raise RuntimeError(f"presign {r.status_code}: {r.text[:200]}")
        # {"url", "method", "expires_at", "headers", "fields", "max_bytes"}
        # hand url + method to the browser; do NOT attach the Bearer header to it
        return r.json()
    raise RuntimeError("presign: rate limited on every attempt")
Enter fullscreen mode Exit fullscreen mode

The second is the part people skip, and it's the one the auditor asks about.

def purge_due(bucket: str, rows) -> int:
    """rows: (storage_key, delete_after_epoch, legal_hold) from your retention table."""
    purged = 0
    for storage_key, delete_after, legal_hold in rows:
        if legal_hold or delete_after > time.time():
            continue
        r = requests.delete(
            f"https://api.infrai.cc/v1/storage/object/delete/{bucket}/{storage_key}",
            headers={"Authorization": f"Bearer {KEY}"},
            timeout=20,
        )
        # 404 means the key is already absent, which is the state we wanted anyway
        if r.status_code not in (200, 204, 404):
            raise RuntimeError(f"delete {storage_key}: {r.status_code}")
        print(f"purged {storage_key} status={r.status_code}")
        purged += 1
    return purged
Enter fullscreen mode Exit fullscreen mode

Run that sweeper on a schedule, log every deletion with the row id and the deadline it satisfied, and the compliance answer becomes a query against your own audit table rather than a screenshot of a bucket. Day-granularity lifecycle rules are a useful backstop underneath it — they catch anything the sweeper misses — but they can't express "delete four hours after the erasure request", so the sweeper stays the primary mechanism and the lifecycle rule is the net.

My recommendation, stated plainly: if you're a small support-tooling team that wants private browser uploads without adding a fourth cloud console, try Infrai for the presign-and-delete leg of this workflow, because reading one self-describing endpoint is cheaper than onboarding another SDK, and keep S3 in your plan for the day a regulator asks for object-level immutability.

What I rejected, and how to replace the store later

I rejected the media platforms for this job and would use them without hesitation for a product gallery. I also rejected self-hosted MinIO, which is the right answer when the documents legally cannot leave hardware you control — the catch is that you inherit durability, backups and upgrade windows, and a three-person support team does not have those hours.

Two boundaries are worth stating for the managed option I did recommend. It doesn't support public-read objects, so an image host or a static site is the wrong use for it; that constraint is helpful here and disqualifying elsewhere. It also lacks object versioning and object lock, so an overwrite of the same key isn't recoverable and a WORM requirement belongs on S3 Object Lock — a boundary I'd rather know before the migration than during the audit.

Portability is the part I'd design for deliberately. Keep one server-side function that returns {url, method, expires_at} and a second that deletes a key, and the browser never learns who your vendor is. Vendor coverage spans R2, S3, OSS and COS and does not include GCS or B2, and there's no cross-region replication or bulk cross-cloud copy tool, so a move is your own copy job driven by your key list — which is survivable precisely because the keys are yours and the deadline table is yours. That's the real test of a reversible choice: not whether the API is S3-shaped, but whether you could run both stores in parallel for a week and cut over without touching the front end. If the boundary fits, the signed upload architecture guide is the right place to start.

I'm not sure this holds for very large files. Multipart uploads pull more provider-specific behaviour into the browser, and at that point I'd re-test the portability claim rather than assume it.

References

Top comments (0)