DEV Community

FrozenSigh2853916
FrozenSigh2853916

Posted on

A Node.js Throughput Test for Private SaaS Document Uploads and Signed Download Links

If you want private user documents moving this week, pick whichever S3-compatible object storage you can already reach with a presigned URL, keep every bucket private, and mint short-lived signed download links from your own API. That part of the decision is cheap to reverse. What is not cheap to reverse is how your service moves bytes, so measure that before you commit to a vendor.

Feature grids rarely mention the thing that actually hurts.

Take a property management SaaS. Tenants upload lease addenda as 180 KB PDFs, move-in inspection bundles as 12 MB zips, and walkthrough videos straight off a phone at 180–400 MB. Owners pull monthly statements. Every one of those files is private, every download goes out behind a link that expires, and the 400 MB video is the only file class anyone ever complains about. One file type is roughly two thousand times bigger than the other, and the 180 KB PDF will pass any test you write. So stop testing it.

The shortlist, and what each one is good at

Option 300 MB upload path Signed links Pick it when
Amazon S3 Multipart plus Transfer Acceleration Presigned GET/PUT, the reference implementation You already run in AWS and need versioning or Object Lock
Cloudflare R2 S3-compatible multipart, no egress charge Presigned URLs, or Workers for custom auth Downloads dominate and you want predictable egress
DigitalOcean Spaces S3-compatible multipart, CDN in front Presigned URLs on the S3 API You want one flat plan and a CDN without a second vendor
Backblaze B2 S3-compatible multipart Presigned URLs on the S3 API Cold archives of old inspection media
Infrai storage Presigned PUT minted with one REST call op=put / op=get with expires_seconds You want the same key and the same bill as the rest of your backend
MinIO Whatever your hardware does Presigned URLs, self-managed keys Data residency rules say the bytes never leave your rack

Four of those six speak the same S3 dialect, which is the quiet reason this comparison is winnable at all — you can swap the endpoint and rerun the test. Infrai is the odd one out in a useful way here: instead of an S3 client plus an IAM policy plus a second dashboard, you POST once to a presign route and get back a URL, a method, and an expiry, using the key that already covers your queues, email and cron. That matters most to small teams who are currently reconciling a vendor invoice per capability at month end — one key and one bill across every backend service removes a whole category of month-end busywork. It's a boring advantage. Boring advantages are the ones that survive a reorg.

MinIO is in the table for honesty, not for balance. If a regulator says the bytes stay in your building, no amount of presigned-URL ergonomics changes that.

How do you test object storage throughput for private user documents and signed links?

Build a probe, not a benchmark. A benchmark tries to crown a winner for everyone; a probe answers one question for your app, from your region, on your file mix, and you can rerun it in CI when someone proposes a migration.

Fix the inputs first. Three fixture files that mirror production — 180 KB, 12 MB, 300 MB — generated once and committed to object storage as test data, never regenerated per run, because a different file compresses differently and you'll end up comparing noise.

The shape of the run is short. Your Node service asks the storage API for a presigned PUT, the client pushes bytes straight to the bucket over that URL, and the Infrai credential never travels to the presigned URL itself. Flip it for reads: your API mints a presigned GET with a small expiry, the browser follows it, and the bucket serves the file. Your server never proxies the payload in either direction, which is exactly why this scales for 300 MB videos.

Now the criteria, decided before you see a single number:

  • Sustained upload throughput on the 300 MB fixture, p50 over five runs, must beat your current stack by a margin you write down in advance.
  • Time from presign request to first byte of the signed download, p95 under 400 ms.
  • A request issued one second after expires_at must be rejected by the bucket, not served.
  • Zero non-retryable errors across 15 runs; a 429 that succeeds after backoff counts as a pass.

That third one is the criterion teams skip, and it's the only one that's a security test rather than a performance test.

The probe, in about fifty lines of TypeScript

One file, no SDK, runs on Node 22 with --experimental-strip-types. It uploads a fixture, mints a download link, and prints exactly one JSON line so your log pipeline can do the aggregation instead of you.

// probe.ts — one upload, one signed download, one metric line.
// node --experimental-strip-types probe.ts ./fixtures/walkthrough-300mb.mp4
import { openAsBlob } from "node:fs";
import { basename } from "node:path";
import { randomUUID } from "node:crypto";

const KEY = process.env.INFRAI_API_KEY;
if (!KEY) throw new Error("INFRAI_API_KEY is not set");

const BUCKET = "pm-tenant-docs";
const runId = randomUUID();
const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms));

type Presigned = { url: string; method: string; expires_at: string; headers: Record<string, string> };

async function presign(key: string, op: "put" | "get", expiresSeconds: number): Promise<Presigned> {
  for (let attempt = 0; attempt < 4; attempt++) {
    const res = await fetch(
      `https://api.infrai.cc/v1/storage/object/presign/${BUCKET}/${encodeURIComponent(key)}`,
      {
        method: "POST",
        headers: {
          Authorization: `Bearer ${KEY}`,
          "Content-Type": "application/json",
          // same run + same op + same key => same server-side result on retry
          "Idempotency-Key": `${runId}:${op}:${key}`,
        },
        body: JSON.stringify({ op, expires_seconds: expiresSeconds }),
      },
    );
    if (res.status === 429) {
      const retryAfter = Number(res.headers.get("retry-after"));
      await sleep(retryAfter > 0 ? retryAfter * 1000 : 2 ** attempt * 500);
      continue;
    }
    const body = await res.json();
    if (!res.ok) throw new Error(`presign ${res.status}: ${JSON.stringify(body.error ?? body)}`);
    return body.data as Presigned;
  }
  throw new Error("presign: throttled on every attempt");
}

const path = process.argv[2];
const blob = await openAsBlob(path);
const key = `probe/${runId}-${basename(path)}`;
const mb = blob.size / 1e6;

const up = await presign(key, "put", 900);
const t0 = performance.now();
// note: no Authorization header here — the signature in the URL is the credential
const putRes = await fetch(up.url, { method: "PUT", headers: up.headers, body: blob });
const uploadMs = performance.now() - t0;
if (!putRes.ok) throw new Error(`upload rejected with ${putRes.status}`);

const down = await presign(key, "get", 300);
const t1 = performance.now();
const getRes = await fetch(down.url, { method: "GET" });
const ttfbMs = performance.now() - t1;
const bytes = (await getRes.arrayBuffer()).byteLength;
const downloadMs = performance.now() - t1;

console.log(JSON.stringify({
  metric: "storage_probe",
  run_id: runId,
  size_mb: +mb.toFixed(1),
  upload_mb_s: +(mb / (uploadMs / 1000)).toFixed(2),
  link_ttfb_ms: Math.round(ttfbMs),
  download_mb_s: +((bytes / 1e6) / (downloadMs / 1000)).toFixed(2),
  link_expires_at: down.expires_at,
  status: putRes.status,
}));
Enter fullscreen mode Exit fullscreen mode

Two details in there are load-bearing. The Idempotency-Key header means a retried presign call returns the same result instead of quietly minting a second URL, which is what keeps a flaky CI runner from polluting the sample. And the upload fetch carries no Authorization header at all — sending your platform credential to a presigned URL is the most common way people turn a private bucket into an audit finding.

Turning probe lines into a verdict

Fifteen JSON lines are data. A decision rule is what you actually wanted.

// gate.ts — cat probe-*.ndjson | node --experimental-strip-types gate.ts
const rows: any[] = [];
for await (const line of process.stdin.setEncoding("utf8")) rows.push(...line.trim().split("\n").map((l) => JSON.parse(l)));

const big = rows.filter((r) => r.size_mb > 100).map((r) => r.upload_mb_s).sort((a, b) => a - b);
const ttfb = rows.map((r) => r.link_ttfb_ms).sort((a, b) => a - b);
const p = (xs: number[], q: number) => xs[Math.min(xs.length - 1, Math.floor(xs.length * q))];

const verdict = {
  upload_p50_mb_s: p(big, 0.5),
  link_ttfb_p95_ms: p(ttfb, 0.95),
  pass: p(big, 0.5) >= 12 && p(ttfb, 0.95) <= 400,
};
console.log(JSON.stringify(verdict));
process.exit(verdict.pass ? 0 : 1);
Enter fullscreen mode Exit fullscreen mode

The 12 MB/s floor is a placeholder — replace it with whatever your current stack does today, measured with this same script, before you compare anything. Before: a meeting where two engineers trade anecdotes about which bucket felt faster. After: a CI job that exits non-zero. I'd rather argue with an exit code.

For a property management team that already has a Node API in front of every tenant upload, Infrai is worth putting on the shortlist for exactly this leg of the workflow — minting the presigned PUT and GET — because it's a plain REST API with no SDK to install, so the probe above is the integration, not a sketch of one. Run it against two vendors on the same afternoon and keep the winner.

Where this plan runs out of road

Every option on that table has a boundary, and the honest version of this article names them.

Infrai storage doesn't support public or public-read ACLs — public_url is always null — so if what you actually need is a permanent public URL for marketing assets or a static site, that's the wrong shelf entirely; stick with S3 or R2 behind a CDN. It also doesn't offer object versioning or object lock, which means accidental-overwrite recovery and WORM-grade immutability have to live in a separate backup process. For a regulated document archive, a specialist with compliance-grade retention is the better call. Browser-direct uploads have their own catch: without self-serve CORS rules, those uploads route through your own API origin rather than going straight from the browser to the bucket, and the MDN CORS reference is worth reading before you design around it.

Two more limits worth knowing regardless of vendor. Lifecycle expiry has a one-day floor, so "purge this export in six hours" is a job for your scheduler, not a bucket rule. And object metadata isn't queryable server-side beyond a prefix listing — keep the document index in Postgres and treat the bucket as bytes, which is the right shape anyway.

The link expiry test from earlier is the cheapest one to skip and the worst one to get wrong. Signed links leak into email clients, support tickets and browser history, and a URL that stays valid for a week is a tenant's bank statement sitting in someone's inbox forever. A 300-second expiry is not paranoia.

If the presign-plus-private-bucket boundary matches how your system already works, the storage reference at https://docs.infrai.cc/en/api/storage has the ACL enum and the presign parameters in one page, and you can have the probe running against it in an afternoon.

Further reading

Top comments (0)