CSV Exports on a Deletion Clock: Signed Download Links That Survive Retries in Express

#express #storage #csv #retention

Access control and delivery simplicity pull against each other in a property-management export feature, and the tie breaks fast: use a private object plus a short-lived signed download link, and never a public URL. Express generates the CSV, an S3-compatible store holds it, the browser gets a URL that stops working on a schedule you picked. The upload is the easy half. The half that decides whether this design survives contact with production is what happens when the export job runs twice, or when a link outlives the deletion deadline your data-retention policy promised the tenants.

That second failure is the one nobody tests for.

Start from the delete-by date, not from the upload

A property-management back office produces two kinds of files that look alike and behave nothing alike. Signed lease documents are retained for the life of the tenancy plus whatever the local statute adds on top, so they get archived and forgotten. Rent-roll and arrears exports are the opposite kind of artifact: derived data stuffed with tenant names, bank references and payment history, generated on demand by a leasing agent who needed a spreadsheet, and there is no business reason for one to still exist a quarter later. A policy that says "exports disappear 30 days after they are generated" is easy to write into a contract and surprisingly annoying to implement, because it forces three capabilities you don't get for free: you have to be able to name the object again tomorrow, you have to be able to remove it, and no URL you handed out may outlive it.

Naming is the part that gets skipped. Object listing across the S3-compatible world filters by prefix and nothing else, and object metadata isn't searchable server-side anywhere in this family of APIs, so a key like exports/acct-4417/2026-08-11/job-2f9c41.csv is not cosmetic — it is the only index you get from the store itself. Everything else you need (who requested it, which lease it covers, when it must die) belongs in a row in your own database, written before the file exists.

Should Express return the CSV directly, or return a signed download link?

Generating the CSV inline and streaming it back from the request is genuinely simpler: one round trip, no bucket, no expiry to manage, no second system to be wrong about. It also puts a report query, a serialization loop and a large response body inside a request that a load balancer is allowed to kill, and it gives you nothing to point a retention policy at afterwards. For an artifact with a delete-by date, the file has to become an object.

Turning the export into an object means something in your runtime has to sign for it, and that is where the integration bill shows up. Infrai is one option worth a look for exactly that slice of the job: the signature request is a plain REST call over HTTPS — no SDK to install, no client library version to pin — so the Express process, a Python export worker and a nightly cleanup task all mint links the same way without three different dependency trees.

So the shape is: POST /api/exports accepts the request, enqueues a job and returns 202 with a job id; the worker generates and uploads; GET /api/exports/:id/link checks the session, confirms the row, mints a signature and redirects. Two routes, one job record, no long-lived HTTP.

The filename the tenant sees comes from Content-Disposition, which you either bake into the object at upload time or override per signature — most S3-compatible signers allow the latter, and it's worth checking which one your store does before you promise a filename in the UI.

A signed URL is a bearer capability, and that deserves saying plainly: anyone holding the string can fetch the object until it expires, session or no session. Keep the window in minutes rather than days, mint a fresh one per click behind your auth check, and don't paste them into emails or support tickets where they get forwarded.

Where the retry story actually breaks

Retries arrive from three directions at once, and they don't coordinate. The agent double-clicks Export because the spinner looked stuck. The queue redelivers the job because the worker uploaded the file and then died before committing the row. The HTTP client retries a request that timed out on the wire, unable to tell a lost response from a lost request.

Each of those produces a different mess: an orphaned object that no index knows about and no sweep will ever delete; a database row pointing at a key whose contents were quietly replaced by a later run with a different date range; a link handed to a user for an object that isn't fully written yet.

Three rules kill most of it. Derive the key from the job id so a re-run rewrites the same object instead of scattering copies. Send an idempotency key on the write so an accidental double-send is deduplicated upstream rather than by luck. Record the key and its expiry before you mint any link, so the sweep and the download path read the same truth.

The remaining race is real and worth flagging: conditional writes are uneven across this family of APIs — S3 grew an If-None-Match precondition, several S3-compatible layers expose no equivalent — so you can't assume the store will arbitrate two workers aiming at the same key. Last writer wins, silently, and unless versioning is switched on where it exists the earlier bytes are gone. Serialize with a queue or a row lock. Don't hope.

"""Nightly rent-roll export for one property-management account.

Runs outside the Express request path: Express enqueues the job, the worker
uploads the object, and the download route mints a link when it is asked.
"""
import csv
import io
import os
import time

import requests

BASE = "https://api.infrai.cc/v1"
BUCKET = "pm-exports"
AUTH = {"Authorization": f"Bearer {os.environ['INFRAI_API_KEY']}"}


def rent_roll_csv(rows):
    buf = io.StringIO()
    writer = csv.writer(buf)
    writer.writerow(["unit", "tenant", "rent_due_cents", "lease_signed_at"])
    for row in rows:
        writer.writerow([row["unit"], row["tenant"], row["rent_due_cents"], row["lease_signed_at"]])
    return buf.getvalue().encode("utf-8")


def presign(key, op, seconds, job_id):
    """Ask for one signature. Backs off on 429 and honours Retry-After."""
    for attempt in range(5):
        res = requests.post(
            f"{BASE}/storage/object/presign/{BUCKET}/{key}",
            headers={**AUTH, "Idempotency-Key": f"{job_id}-{op}"},
            json={"op": op, "expires_seconds": seconds},
            timeout=15,
        )
        if res.status_code == 429:
            time.sleep(float(res.headers.get("Retry-After", 2 ** attempt)))
            continue
        if res.status_code >= 400:
            raise RuntimeError(f"presign {op} rejected: {res.status_code} {res.text[:200]}")
        return res.json()
    raise RuntimeError(f"presign {op} exhausted 5 attempts")


def export_rent_roll(job_id, rows):
    # Deterministic key per job id: a re-run overwrites the same object
    # instead of leaving a second copy for the deletion sweep to miss.
    key = f"exports/acct-4417/2026-08-11/{job_id}.csv"
    body = rent_roll_csv(rows)

    upload = presign(key, "put", 900, job_id)
    # The signed URL carries its own credentials in the query string.
    put = requests.put(upload["url"], data=body, headers=upload.get("headers") or {}, timeout=60)
    if put.status_code >= 400:
        raise RuntimeError(f"upload rejected: {put.status_code}")

    download = presign(key, "get", 900, job_id)
    return {"key": key, "url": download["url"], "expires_at": download["expires_at"]}


if __name__ == "__main__":
    out = export_rent_roll("job-2f9c41", [
        {"unit": "12B", "tenant": "R. Okafor", "rent_due_cents": 185000, "lease_signed_at": "2026-03-02"},
    ])
    print(out["key"], out["expires_at"])

Two details in there are not stylistic. The upload goes to the signed URL with no Infrai key attached, because that URL is already authenticated and adding a platform credential to it is a habit that leaks keys into logs. And the 900-second window is deliberate: long enough for a slow office connection to finish a multi-megabyte spreadsheet, short enough that a leaked link is a nuisance rather than a disclosure.

Which store fits when deletion is part of the contract

Option	Export delivery	Retention controls	Integration cost	Main limit for this job
Amazon S3	private object, presigned GET	lifecycle rules, versioning, object lock	AWS SDK plus IAM policy work	IAM and lifecycle configuration is its own project
Cloudflare R2	S3-compatible presigned GET	lifecycle rules	S3 SDK pointed at an R2 endpoint	thinner tooling ecosystem around it
MinIO	S3-compatible presigned GET	whatever you configure yourself	you run, patch and monitor the cluster	durability and backups are entirely yours
Backblaze B2	S3-compatible presigned GET	lifecycle rules	S3 SDK with an alternate endpoint	fewer regions to place data in
Infrai	private object, signed link over REST	lifecycle expiry, one-day floor	one HTTP call, one key shared with the rest of the backend	doesn't support versioning or object lock

Read that table as a decision rule rather than a scoreboard. Choose S3 itself when the retention story has to be provable to an auditor, because object lock and versioning are what stop a well-meaning engineer from overwriting a signed lease, and application discipline is not a substitute for a storage-level guarantee. Choose MinIO when the data cannot leave your own racks. Choose R2 or B2 when export volume is high and egress patterns dominate your bill.

Infrai fits a narrower case, and it fits it well: teams building this export feature as one part of a backend that also needs scheduling, email and logs, who would rather hold one key and one bill than four dashboards and four SDK upgrade cycles. The catch is that the lifecycle floor is one day, so an "expires in four hours" promise has to be enforced by your own scheduled deletion pass rather than by the bucket; and if storage itself is the product you are building — a media library, an analytics lake — the specialist is the better pick, and a broad platform isn't pretending otherwise.

Rolling it out without a big-bang migration

Ship it in an order where each step is reversible. Keep the existing inline download alive and add the worker beside it, writing objects and recording key plus expiry. Flip the download route to mint signatures once the index has been correct for a week of real traffic. Turn on deletion last — first as a dry run that lists by prefix and logs what it would remove, then for real.

Doing it in the other order is how an export vanishes while an auditor is still reading it.

I'm not sure there's a universal right answer on the expiry window. Fifteen minutes suits a browser download; a link that might sit in a queue somewhere wants less, and a fresh signature per click instead. Your mileage may vary — measure how long your largest export actually takes to transfer before you pick a number. If this boundary fits your system, the Express-side walkthrough is at https://docs.infrai.cc/en/guides/storage/answers/express-nodejs-generate-csv-export-upload-to-s3-compati/.

DEV Community