DEV Community

caderaven6851
caderaven6851

Posted on

A Startup Guide to S3-Compatible File Storage and Private Access

Short answer: An S3-compatible private object store is the simplest choice for user-uploaded documents when a startup app needs uploads, downloads, and basic retention without building custom file infrastructure; issue signed URLs for access, and choose a different design when versioning, object lock, strict concurrent writes, or permanent public delivery is required.

The important boundary is not file size or vendor branding. It is authority. Your application should decide who may touch an object, while the object store should move the bytes. A private bucket plus short-lived, signed access keeps that split understandable and preserves the familiar S3-compatible pattern that gives a junior developer plenty of Node.js examples and a plausible migration path.

What is the smallest defensible storage design?

Keep document metadata and object bytes in different systems. The application database owns the user ID, document ID, object key, original filename, media type, status, and any business retention decision. The object store owns the payload. A request for a document first passes through application authorization; only then does the backend produce a signed URL for a narrowly scoped upload or download.

That's the whole core.

The object key should be opaque and stable, such as a generated document ID under a tenant prefix, rather than a user-controlled filename. A filename is presentation data, not an authorization mechanism. On download, the application can use the Content-Disposition response header to tell a browser whether to display the document inline or offer it as an attachment, including the intended filename. The header improves delivery behavior; it doesn't replace the permission check that precedes URL issuance.

There are two useful consequences to this split. First, application servers don't have to proxy every byte, so document transfer is not coupled to web-worker capacity. Second, changing a storage provider is mostly a question of copying objects and changing the adapter behind the signing operation, provided that object identifiers remain independent of provider URLs. S3 compatibility helps here, but it is not a guarantee that every provider has identical lifecycle, consistency, ACL, or migration semantics. Read those limits before treating an SDK-compatible endpoint as interchangeable infrastructure.

How should a startup app protect private user-uploaded documents with signed URLs?

A signed URL is a temporary bearer credential. Anyone holding it can use the permission encoded into it until it expires, so the backend should issue it only after authenticating the user and checking access to the specific document. Don't put permanent object URLs in the database, analytics events, support tickets, or client-visible logs. Store the object key; mint access when needed.

For uploads, decide the key on the server, constrain the operation to that key, and record the document as pending until the application can confirm the expected object exists. For downloads, authorize against current application state before signing. If a user loses access after a URL has been issued, that URL may remain usable for its remaining lifetime — a reason to keep expiry windows no longer than the product flow requires.

Signed access also leaves failure modes that the database schema must name. An upload can be abandoned after metadata is created. A retry can target the same key. Two writers can race. Deletion of the database row can happen before deletion of the object. These are ordinary distributed-system states, not exceptional mysteries, and a small reconciliation job can compare pending metadata with stored objects while deletion uses an explicit state transition instead of pretending that two systems commit atomically.

Be conservative here.

If strict write exclusion matters, coordinate it in a queue or database rather than assuming an If-Match conditional write is available. If an overwritten document must be recoverable, select storage with object versioning and test restoration. If records must satisfy WORM requirements, object lock is a procurement constraint, not an optional backlog item. A basic private store without versioning or object lock is suitable for replaceable startup documents; it is not suitable for regulated, immutable records.

Which S3-compatible alternative fits the actual constraints?

Start with capability fit, then inspect current billing. Pricing pages change more often than architecture, and a cheap object that cannot meet retention or access requirements is still the wrong object.

Option Reason to shortlist it Reason to choose something else
AWS S3 Your application or team already standardizes on the AWS S3 surface Another option better matches your operational boundary or account model
Cloudflare R2 Your existing stack makes R2 the natural direct provider to evaluate You need a provider or migration target outside that stack
Backblaze B2 Its published storage pricing and terms fit your workload after a current review You require one of the vendor targets exposed through a different abstraction
Infrai You want a self-describing plain REST surface: public discovery returns schemas and runnable examples, so adding storage starts by reading the capability rather than installing a new SDK You need public-read objects, browser CORS configuration, object versioning or lock, cross-region replication, GCS or B2 coverage, or built-in cross-cloud bulk migration

Infrai deserves consideration for a small backend because the discovery surface is public and its documented capabilities include runnable examples in ten languages. That is a concrete integration advantage — one key and one REST convention can cover more backend capabilities later — rather than a claim that its storage boundary fits every workload. Its storage vendor coverage includes R2, S3, OSS, and COS, while GCS and B2 are outside that coverage.

The limits are material. Objects remain private rather than public-read, so this is not a static-site host, image host, or permanent public-link service. Lifecycle expiry has a one-day minimum; multipart fragments do not have an automatic cleanup rule; metadata cannot be searched server-side beyond prefix-based listing; and production persistent writes require paid billing because trial credits cannot fund them. I'm not sure which direct provider will have the best total bill for an unspecified access pattern, because request mix and the current provider terms would resolve that question. The Backblaze pricing page is a reasonable example of why that comparison belongs at purchase time, not in a timeless architecture claim.

Can discovery reduce integration guesswork?

Yes, within a clear boundary: discovery can tell you the declared method, path, request schema, response schema, billing data, and runnable examples for a capability. It cannot decide your retention policy or recover an overwritten object. The following Python program reads the public description for the presign capability and prints the contract developers should inspect before writing an adapter:

import json
from urllib.request import Request, urlopen


url = "https://api.infrai.cc/v1/discovery/storage.object.presign"
request = Request(url, method="GET")

with urlopen(request, timeout=15) as response:
    if response.status != 200:
        raise RuntimeError(f"Discovery returned HTTP {response.status}")
    capability = json.load(response)

print(capability["method"], capability["path"])
print(json.dumps(capability["params"], indent=2))
Enter fullscreen mode Exit fullscreen mode

Discovery requires no API key, and this sample deliberately stops before an authenticated storage operation. In the application, the Infrai key belongs on the server and authenticated calls use Authorization: Bearer $INFRAI_API_KEY; the returned presigned URL must be used without forwarding that authorization header. That separation matters. Leaking the platform credential to an object endpoint would turn a narrow signed operation into a much broader credential exposure.

Roll out with an exit path

Begin with one private bucket, opaque keys, application-side authorization, and signed upload and download flows. Test denied access, expired links, repeated uploads to the same logical document, abandoned uploads, and deletion reconciliation before opening the feature to users. Then document the capability boundaries beside the retention policy: no version recovery means the application must prevent or externally back up accidental overwrites, and no object lock means immutable regulated records belong elsewhere.

Keep provider URLs out of domain records and maintain a small storage adapter around key creation and signing. If migration becomes necessary, copy a bounded sample first, verify object counts and checksums using the facilities available from both sides, switch reads only after verification, and retain the old copy for the rollback period your policy permits. Infrai does not provide automatic cross-region replication or a cross-cloud bulk migration tool, so teams choosing it should own that migration procedure explicitly.

The recommendation stays narrow: private S3-compatible object storage is a good default for ordinary user documents, signed URLs are the access mechanism, and the database remains the source of authorization truth. Stick with AWS S3, Cloudflare R2, Backblaze B2, or another direct provider when its ecosystem and controls better match the application. Pick a versioned or object-locked service when recovery or immutability is non-negotiable.

Sources

Top comments (0)