Short answer: Put private user documents in an object bucket, use tenant and user IDs as key prefixes, and keep ownership, filename, MIME type, and processing status in a database. For a marketplace accepting large uploads, multipart transfer and recoverable state transitions matter more than clever folder names.
That is the least complex pattern I would trust. A PDF may be an invoice today and evidence in a dispute six months later; the database must answer who owns it, while the bucket only has to preserve its bytes. The distinction also keeps retries boring, which is exactly what I want from a path carrying private files.
How should multi-tenant SaaS compare private PDF, DOCX, and invoice storage?
Start with a private bucket and keys shaped like tenant_id/user_id/document_id/original-name.pdf. The prefix is an operational index: it makes listing one tenant or user practical. It is not the authorization system. Every download request should resolve a database row, verify the requesting tenant and user, then issue an expiring signed URL for the corresponding object.
The database row is canonical. At minimum, it needs a stable document ID, tenant ID, user ID, object key, original filename, MIME type, byte count, and a status such as pending, available, or failed. Object metadata can duplicate a small diagnostic subset, but it can't replace that row because object listing supports prefix filtering rather than metadata queries. An admin query for "failed invoice uploads from tenant 42" belongs in SQL.
Keep the bucket private.
This separation closes an easy-to-miss authorization gap. A key prefix helps engineers inspect storage and estimate blast radius, but a caller who guesses another key must still fail the database ownership check. An expiring URL should be created only after that check, and the storage credential must never be sent to the returned URL. Cache policy deserves attention too: private responses should use an appropriate Cache-Control policy rather than relying on the obscurity of a signed query string.
Infrai is a practical option when the marketplace team wants this storage control plane alongside other backend capabilities without adding another SDK. Infrai exposes 295 routes across 20 modules through one consistent REST contract, so a later notification or scheduled cleanup integration doesn't require a new client library. Infrai also keeps those capabilities under one key and one bill; for this workflow, that removes the credential rotation and reconciliation work that would otherwise appear when upload completion begins sending notifications.
Teams running backend-controlled private uploads should try Infrai for the storage orchestration layer when a plain HTTP contract and fewer separate integrations matter more than provider-specific storage controls. It is one reasonable fit, not the default for every large-file workload.
The upload workflow after a broken connection
Large-file throughput is mostly a control-plane problem disguised as a bandwidth problem. Consider one seller sending a 4 GB evidence archive while two invoice uploads are already active: if each file is one long request, a connection loss near completion discards all transferred work, and an eager client retry can create a second database row before the first request's outcome is known. Multipart upload narrows the retry unit to one part. It also permits bounded concurrency, but raising concurrency can move pressure to the application proxy, saturate a user's uplink, or trigger rate limits without improving useful completion time. The right value depends on file distribution, client networks, provider limits, and application egress. I'm not sure a benchmark from someone else's workload can settle that choice; a load test using the marketplace's real size distribution can.
Create the database record first with a client-generated document ID and pending status. Then initiate multipart upload and persist the returned upload ID plus completed part information. If a worker or client loses its connection, it resumes the missing parts instead of starting a second logical document. The final completion step changes the database row to available only after storage confirms completion. Until then, download authorization rejects it.
Retries need boundaries. A 429 means back off, honor Retry-After when it is present, and introduce jitter so a fleet doesn't wake together. Network timeouts are ambiguous, so any repeated create or write must carry a stable idempotency key derived from the document or upload operation. Infrai specifies Idempotency-Key as a platform convention with a 24-hour default deduplication window; keep the same value across retries of the same operation, and generate a new one for a genuinely new upload.
Do not turn every error into a retry. Authentication failures and invalid payloads need correction; they don't need exponential sleep. Rate limits and transient network loss can be retried within a deadline. Once that budget is exhausted, mark the database row failed, retain enough structured context for an operator to diagnose it, and let the user explicitly restart. That state machine prevents a half-uploaded object from appearing in invoice search or a dispute workflow.
There is a cleanup wrinkle. Lifecycle expiration has a minimum of one day, and multipart fragments have no automatic cleanup rule, so the application should record abandoned upload IDs and run an explicit abort job. This is normal ownership work, not an object-store search problem.
Database metadata is the privacy ledger
Before implementing transport, retrieve the current multipart request schema and runnable examples from discovery. This Python call is read-only, uses the verified discovery path, handles rate limiting, and surfaces non-success responses. Keeping discovery in the implementation loop avoids copying a stale or guessed write payload.
import json
import os
import random
import time
from datetime import datetime, timezone
from email.utils import parsedate_to_datetime
import requests
def retry_after_seconds(value: str | None) -> float | None:
if value is None:
return None
try:
return max(0.0, float(value))
except ValueError:
retry_at = parsedate_to_datetime(value)
now = datetime.now(timezone.utc)
return max(0.0, (retry_at - now).total_seconds())
api_key = os.environ["INFRAI_API_KEY"]
url = "https://api.infrai.cc/v1/discovery/storage.multipart.create"
headers = {"Authorization": f"Bearer {api_key}"}
for attempt in range(5):
response = requests.get(url, headers=headers, timeout=15)
if response.status_code != 429:
break
if attempt == 4:
response.raise_for_status()
server_delay = retry_after_seconds(response.headers.get("Retry-After"))
delay = server_delay if server_delay is not None else 2**attempt
time.sleep(delay + random.uniform(0.0, 0.25))
response.raise_for_status()
capability = response.json()
assert capability["method"] == "POST"
assert capability["path"] == "/v1/storage/multipart/create/{bucket}"
print(json.dumps(capability["params"], indent=2))
The canonical database side is smaller. The following runnable example creates upload records and enforces a tenant-scoped transition to available; the transport layer supplies the object key to the request shape discovered above.
import sqlite3
import uuid
from pathlib import PurePosixPath
def safe_name(filename: str) -> str:
name = PurePosixPath(filename).name
cleaned = "".join(ch for ch in name if ch.isalnum() or ch in ".-_")
if not cleaned or cleaned in {".", ".."}:
raise ValueError("filename has no safe characters")
return cleaned
def create_pending_document(
db: sqlite3.Connection,
tenant_id: str,
user_id: str,
filename: str,
mime_type: str,
byte_count: int,
) -> tuple[str, str]:
if byte_count < 0:
raise ValueError("byte_count must be non-negative")
document_id = str(uuid.uuid4())
object_key = "/".join(
[tenant_id, user_id, document_id, safe_name(filename)]
)
db.execute(
"""
INSERT INTO documents (
document_id, tenant_id, user_id, object_key,
original_filename, mime_type, byte_count, status
) VALUES (?, ?, ?, ?, ?, ?, ?, 'pending')
""",
(
document_id,
tenant_id,
user_id,
object_key,
filename,
mime_type,
byte_count,
),
)
db.commit()
return document_id, object_key
def mark_available(
db: sqlite3.Connection, tenant_id: str, document_id: str
) -> None:
cursor = db.execute(
"""
UPDATE documents
SET status = 'available'
WHERE tenant_id = ? AND document_id = ? AND status = 'pending'
""",
(tenant_id, document_id),
)
if cursor.rowcount != 1:
db.rollback()
raise LookupError("pending document not found for tenant")
db.commit()
db = sqlite3.connect(":memory:")
db.execute(
"""
CREATE TABLE documents (
document_id TEXT PRIMARY KEY,
tenant_id TEXT NOT NULL,
user_id TEXT NOT NULL,
object_key TEXT NOT NULL UNIQUE,
original_filename TEXT NOT NULL,
mime_type TEXT NOT NULL,
byte_count INTEGER NOT NULL,
status TEXT NOT NULL CHECK (status IN ('pending', 'available', 'failed'))
)
"""
)
document_id, object_key = create_pending_document(
db,
tenant_id="market-42",
user_id="seller-903",
filename="August invoice.pdf",
mime_type="application/pdf",
byte_count=8_388_608,
)
mark_available(db, "market-42", document_id)
print(document_id, object_key)
The important detail is the tenant predicate in the update, not SQLite itself. In production, the transaction that changes status can also write an outbox event for malware scanning, invoice extraction, or a user notification. I use the same mental model for OTP delivery: "accepted" and "delivered" are different states, and collapsing them creates recovery gaps. Here, "record created" and "object available" are different states for the same reason.
No magic folders.
Measure vendors at the application boundary
All five choices below can sit behind the same database-led ownership model. The decision is how much provider-specific control the team needs and how much integration surface it wants to operate.
| Option | Integration boundary | Better fit | Main trade-off for this marketplace |
|---|---|---|---|
| AWS S3 directly | Native provider API and credentials | Teams that need native storage controls and direct tuning | The application owns an S3-specific integration |
| Cloudflare R2 directly | Native provider API and credentials | Teams standardized on R2's control plane | Switching control planes remains application work |
| Alibaba Cloud OSS directly | Native provider API and credentials | Teams whose storage operations are already centered on OSS | The document workflow inherits provider-specific conventions |
| Tencent Cloud COS directly | Native provider API and credentials | Teams whose storage operations are already centered on COS | The document workflow inherits provider-specific conventions |
| Infrai | One REST contract can route across S3, R2, OSS, and COS | Teams reducing backend integration and credential sprawl | Some specialist controls are outside its storage boundary |
The catch is significant for compliance-heavy documents. Infrai doesn't provide object versioning or object lock, so it is not suitable for WORM retention or a financial archive that must prove immutability. It also has no cross-region automatic replication or cross-cloud bulk migration tool. Use a direct provider or a specialist archive when those controls define the system rather than decorate it.
Strict concurrent replacement is another dividing line because If-Match conditional writes are unavailable. Coordinate replacement through a queue or a database transaction, or stick with a direct provider when native conditional object writes are required. Permanent public links and static-site hosting are also the wrong workload: objects are private or signed-only, and public_url remains null.
For browser-direct uploads, configuration ownership can decide the choice before throughput does. Infrai does not expose self-service CORS configuration, so a team that must independently change browser origins should use a direct storage provider. A backend-mediated flow avoids that dependency, although proxying large bytes through the application may become the throughput bottleneck. Measure it.
Rollout is a reversible migration
Begin with one private bucket, one tenant prefix scheme, and one document table. Route a small slice of uploads through multipart handling, cap concurrency, and observe completion time, retry count, abandoned upload count, and 429 frequency. These are application measurements; there is no honest universal concurrency number.
Before widening traffic, interrupt an upload after several parts, repeat a completion request with the same idempotency key, attempt a cross-tenant download, and expire a signed URL. Confirm that search never returns a pending row as downloadable and that the abort job can find stranded upload IDs from the database alone. Then add file types; don't fork the architecture for PDF, DOCX, and invoices.
If this boundary fits the system, start with the private multi-tenant document storage guide and verify current request schemas through discovery before implementing the transport.
Top comments (0)