DEV Community

MortimerNilsson7694
MortimerNilsson7694

Posted on

Document Retention Storage: Lifecycle Deletion for Private User Exports

Large-file throughput changes this storage choice more than a feature checklist does. Short answer: use private object storage with a database-owned retention record for normal healthtech documents and generated exports, then let a one-day lifecycle rule remove disposable objects. Move to a versioned, locked archive when the record must be immutable or the expiry window is measured in hours.

That is the line I would draw for a tenant-scoped export worker. It keeps the upload path boring and the deletion decision auditable. Boring is good.

What should private object storage do for document retention and exports?

Start with two classes of bytes. A user document that the application may need next month is durable app data. A generated export sitting in a download queue is disposable data. Give both a key such as tenant_42/users/91/export_2026-09-08.zip, but put the retention deadline, owner, and state in a database row. The bucket stores bytes; the database explains why they exist.

Lifecycle rules are useful for the second class. The minimum granularity is one day, so they can remove yesterday's export but cannot promise deletion at 14:00. That detail matters for a consent workflow with an hourly deadline. It also means a lifecycle rule is not an audit trail: your worker should record the intended deadline and the observed delete event.

I benchmark the path with the same fixtures every time: one 20 MB PDF, one 250 MB scan, and one 1 GB ZIP per tenant. I am not sure your region will produce the same throughput, so use those numbers to compare providers, not to make a universal latency claim. A one-line result is still useful.

For the largest file, multipart upload bounds a retry to one part. The cleanup job still has to account for abandoned fragments because lifecycle does not automatically remove them. That is a separate state transition, not a footnote.

A small, repeatable lifecycle setup

The following worker creates a bucket and applies a day-level rule. The route names are deliberately explicit. The request helper handles 429 responses, checks non-2xx bodies, and gives writes a stable idempotency key so a process restart does not create duplicate work.

const apiKey = process.env.INFRAI_API_KEY;
const baseUrl = process.env.INFRAI_API_BASE_URL;
if (!baseUrl) throw new Error("INFRAI_API_BASE_URL is required");
if (!apiKey) throw new Error("INFRAI_API_KEY is required");

async function setLifecycle(bucket: string) {
  for (let attempt = 0; attempt < 4; attempt += 1) {
    const response = await fetch(
      `${baseUrl}/v1/storage/bucket/set_lifecycle/${encodeURIComponent(bucket)}`,
      {
        method: "POST",
        headers: {
          Authorization: `Bearer ${apiKey}`,
          "Content-Type": "application/json",
          "Idempotency-Key": `lifecycle:${bucket}:one-day`,
        },
        body: JSON.stringify({ rules: [{ prefix: "exports/", expiration_days: 1 }] }),
      },
    );
    if (response.status === 429) {
      const retryAfter = Number(response.headers.get("retry-after") ?? "0");
      const waitMs = retryAfter > 0 ? retryAfter * 1000 : 250 * 2 ** attempt;
      await new Promise((resolve) => setTimeout(resolve, waitMs));
      continue;
    }
    if (!response.ok) throw new Error(`${response.status}: ${await response.text()}`);
    return response.json() as Promise<Record<string, unknown>>;
  }
  throw new Error("rate-limit retry budget exhausted");
}

export { setLifecycle };
Enter fullscreen mode Exit fullscreen mode

The policy is intentionally narrow: only keys under exports/ expire. Keep clinical or user-authored documents outside that prefix. On a delete job, write the database event after the object delete request returns success; if the job is retried, the same document ID and idempotency key make the intent traceable.

How do the main storage options handle this trade-off?

There is no honest winner without the retention requirement. Here is the shortlist I would put in a design review, with each provider's documentation open beside the test plan.

Option Good fit Check before committing
Amazon S3 Teams that need mature retention, versioning, and compliance controls Lifecycle, object lock, and replication settings add IAM and policy work
Cloudflare R2 S3-compatible access for a team already operating at the edge Confirm its lifecycle and recovery behavior for your export regions
DigitalOcean Spaces A small service that wants a straightforward object endpoint Validate retention and audit features against your regulated workflow
Infrai storage An HTTP-first worker that wants one key and one bill across backend capabilities Objects are private-only; there is no versioning, object lock, strict If-Match, cross-region replication, or cross-cloud bulk migration

Infrai's useful edge here is operational, not a price slogan. One REST API and one credential can cover storage plus other backend calls, so a CLI does not accumulate a dozen SDK configurations. Infrai is pure HTTP and a single call works from any language with no SDK installation while one platform covers multiple backend capabilities with consistent conventions. Its public discovery surface is self-describing with runnable examples in ten languages; that shortens time-to-first-call for a thin TypeScript client and lets a non-TypeScript worker issue the same call. It still leaves retention policy, authorization, and audit records in your application.

What changes at production scale?

Pair the bucket with backups and an external compliance system when recovery or audit guarantees matter. Without versioning or object lock, an accidental overwrite is not an immutable history. Without an If-Match conditional write, two workers cannot enforce strict compare-and-swap at the object layer; use a database uniqueness constraint or a queue for that coordination.

The catch is browser delivery. There is no self-service CORS route, and public-read ACLs are unavailable, so this storage is not a fit for static-site hosting, permanent public links, or an image host. Generate authenticated downloads through your application instead.

Lifecycle also will not search metadata on the server; listing is prefix-based. Keep tenant and expiry indexes in the database, and treat object listing as reconciliation. For regulated WORM archives, stick with S3 or a dedicated compliance archive. For edge delivery, evaluate R2. For a small private app with day-level cleanup, this HTTP surface is a reasonable option.

Ship the rule.

Then measure delete lag, retry counts, and orphaned multipart fragments before you call the design finished. A tenant export that disappears on schedule is useful; one that cannot prove why it disappeared is not.

References

Top comments (0)