Short answer: keep signed user documents in private object storage, but make the database the owner of retention intent and tenant isolation; use lifecycle deletion only as a coarse backstop, and keep backups and exports outside the user-deletion promise until their separate retention rules are explicit.
That answer is deliberately less tidy than “set a one-day TTL.” A deletion deadline is a product guarantee. Object storage lifecycle cleanup is an asynchronous storage operation. They can cooperate, but they are not the same clock.
The decision record: protect the tenant boundary first
For a gaming service storing signed tournament waivers, parental-consent forms, or account documents, the first invariant is that a request for tenant A can never select tenant B's key. Every object record should therefore carry a tenant identifier, an opaque object key, its retention class, its creation time, and a deletion state. The bucket is private; authorization is checked against the database row before a read, export, or delete is scheduled.
Use a key that is easy to scope, such as tenants/t_42/documents/d_981/signature.pdf. The prefix helps operators inspect a bounded area, but it is not an authorization system and it is not a query engine. A database query should decide which keys belong to the requesting tenant.
The object key is not the tenant boundary.
The failure boundary matters. If the database says deleted when a queue message was merely accepted, an audit report can claim erasure that never happened. Record requested, in_progress, and confirmed states instead, and make the worker idempotent. A repeated delete request should converge on the same state without touching another tenant's object.
How should document retention storage handle lifecycle delete, backups, and exports?
Separate four clocks: the document's business retention deadline, the object lifecycle rule, the backup retention period, and the generated-export expiry. A signed document may need durable retention while a download export can be disposable. A backup may be kept for disaster recovery even after an application row is erased, subject to the legal and privacy policy that governs that backup. Do not imply that deleting the live object instantly removes every copy.
| Layer | Owns | Useful guarantee | Boundary to test |
|---|---|---|---|
| Database | Tenant, policy, state, and audit intent | Queryable ownership and a deletion ledger | It does not prove the bytes are gone |
| Private object storage | Document and export bytes | Access-controlled durable storage | Lifecycle timing is coarse and asynchronous |
| Lifecycle policy | Routine stale-object cleanup | A storage-side backstop for day-scale expiry | It is unsuitable for an hourly deadline |
| Backups | Recovery copies | Restoration after an operational mistake | Erasure and legal hold need a separate policy |
| Export worker | Temporary user-facing packages | Explicit expiry and repeatable generation | A failed job must not widen tenant access |
The practical rule is simple: use a one-day minimum only when “no later than the next day-scale sweep” is acceptable for that class of data. If the requirement is “delete within 60 minutes,” the application must schedule and verify deletion itself. If the requirement is “never delete before a legal hold is released,” lifecycle policy needs an authoritative hold check or a different retention system.
There is a second trap in exports. An export often combines rows from several services and is then copied to a temporary key. Its retention metadata must be derived from the export job, not inherited blindly from the oldest source document. The job record should name the requesting tenant, the exact object key, the expiry decision, and the delivery status. That makes a support investigation possible without scanning object metadata, and it creates a useful reconciliation point when a player downloads a package, loses the connection halfway through, and asks for the link again: the service can check the tenant and job state before issuing a new short-lived access grant instead of treating a filename as proof of ownership. The export worker should also distinguish a source document that is still retained from a package that is already expired, because those are separate policy decisions with separate audit consequences.
Failure modes that the storage diagram hides
Tenant isolation fails in ordinary-looking places. A listing endpoint may accept a prefix supplied by the client; a retry may reuse a signed URL after its intended window; a bulk deletion query may omit the tenant predicate; or an export filename may be treated as a globally unique identifier when it is only unique inside one account. These are application failures around storage, not properties that a bucket label fixes.
Retention has its own sharp edges. A lifecycle sweep can be late relative to an hourly promise. An overwrite can destroy the prior value when version history is absent. A multipart upload can leave incomplete parts if cleanup is not configured and monitored. A backup can preserve a copy after the live object is gone. None of those cases should be hidden behind a green “delete requested” metric.
I would instrument the boundary, not just the API call: objects requested for deletion, objects confirmed absent, age of the oldest pending deletion, export expiry lag, and backup-policy exceptions. Alert on a growing confirmation gap. A queue that is accepting work is not evidence that the work is complete.
Keep it boring.
I'm not sure a product summary can settle durability, consistency, or legal-retention questions for a regulated workload. The service contract, current documentation, restore tests, and counsel's interpretation have to resolve those questions. Your mileage may vary by jurisdiction, especially when signed documents contain minors' data or cross-border transfers are involved.
Option trade-offs and the rejected shortcut
The choice is not “which bucket has the nicest feature list?” It is which layer can make each promise observable and enforceable.
| Option | Fits when | Trade-off |
|---|---|---|
| Private object storage plus database ledger | Documents are file-shaped and access is tenant-scoped | Requires application coordination for deletion, audit, and exports |
| Database binary storage | Payloads are small and must commit with relational state | Large documents, backups, and export packages increase database pressure |
| Managed archive or compliance store | Immutability, legal hold, or formal retention evidence is mandatory | More operational and policy complexity; ordinary temporary exports may not belong there |
| Self-hosted object storage | Regional control and operational ownership justify the work | The team owns upgrades, durability design, recovery tests, and isolation controls |
The rejected shortcut is lifecycle-only retention. It is valid for regenerable exports that contain no legal record, tolerate a full day of retention, and have no promise of immediate erasure. It is a poor sole system for signed documents because it cannot, by itself, explain ownership, prove a user-scoped deletion, or coordinate a legal hold with the database.
The catch is that private object storage is not suitable when tenant isolation depends on provider-side metadata queries, when deletion must be measured in minutes, or when immutable retention is non-negotiable. Choose a storage or compliance service that explicitly supplies those controls, and keep the same database-ledger pattern around it.
The critical path in Python
The code below shows the provider-neutral shape of an erasure worker. The storage adapter is intentionally abstract: its contract must return success only after the object is gone or already absent, while the database owns tenant verification and state transitions.
from dataclasses import dataclass
from typing import Protocol
class PrivateStore(Protocol):
def delete_if_owned(self, tenant_id: str, object_key: str) -> bool:
"""Return True when the key is absent after this call."""
...
@dataclass(frozen=True)
class DocumentDeletion:
tenant_id: str
object_key: str
def erase_document(store: PrivateStore, deletion: DocumentDeletion) -> str:
if not deletion.tenant_id or not deletion.object_key:
raise ValueError("tenant and object key are required")
# The adapter must verify ownership before issuing the storage delete.
confirmed = store.delete_if_owned(deletion.tenant_id, deletion.object_key)
return "confirmed" if confirmed else "pending"
In production, the database transaction that loads the row and marks it in_progress belongs around this call, with a retryable worker and a reconciliation job for pending. Do not put a bearer token in a browser, and do not turn a storage error into a successful deletion event. The exact request method and endpoint are provider-specific; inventing a route here would make a neutral example look more portable than it is.
Top comments (0)