Short answer: implement medical referral intake in a Node.js service with a durable asynchronous job, validation before persistence, bounded retries, and secure temporary files. The audit record should outlive the file, while the file itself should disappear on a policy-driven schedule.
That rule keeps a one-person SaaS moving weekly. I can outsource undifferentiated PDF rendering, but I cannot outsource the decision about who may retrieve a referral or why it was retained.
The decision note
| Choice | Good fit | Trade-off |
|---|---|---|
| Synchronous render | A tiny, non-sensitive preview | Request timeout becomes a data and UX problem |
| Durable asynchronous job | Medical referral intake and monthly marketplace reports | Requires idempotency, status tracking, and retry policy |
| Local temporary file | A single trusted worker with strict permissions | Cleanup and disk exhaustion become your responsibility |
| Encrypted object storage | Multiple workers or delayed archive retrieval | Key management and deletion verification add work |
For referral intake, I choose the durable job plus encrypted storage path. The API accepts metadata and a bounded upload, records an idempotency key, and returns a job identifier. A worker validates, renders, archives, and emits an audit event. The HTTP request never waits for a PDF renderer.
The catch is operational overhead. If you only need an ephemeral preview and can tolerate a failed request, synchronous work is simpler. Stick with it when no protected health information crosses the boundary and the render fits inside your request timeout.
Keep the boundary boring.
What should a Node.js intake pipeline validate before it queues work?
Validation has two layers. The edge checks the envelope: authenticated caller, content length, allowed media type, and a stable referral identifier. The worker checks the meaning: required patient and referring-clinician fields, date formats, permitted status values, and a schema version. Rejecting malformed input before a retry queue saves both compute and confusion.
Never trust a filename or a client-supplied MIME type. Read bytes, use a parser appropriate to the declared format, and cap decompression and page counts. Store a digest of accepted bytes so a retry can prove it processed the same input. I once chased a duplicate report whose names differed only by case; the idempotency key, not the filename, should have been the identity.
A Blob is a useful boundary for byte-oriented handling: its size, type, and arrayBuffer() methods let code inspect payloads without pretending they are trusted paths. The browser API does not make data private by itself, so keep authorization and retention decisions in the service.
type ReferralJob = {
jobId: string;
idempotencyKey: string;
sha256: string;
status: 'queued' | 'running' | 'archived' | 'failed';
expiresAt: string;
};
function validateEnvelope(input: {
bytes: Uint8Array;
mediaType: string;
referralId: string;
idempotencyKey: string;
}): void {
if (input.bytes.byteLength > 10 * 1024 * 1024) throw new Error('payload_too_large');
if (input.mediaType !== 'application/pdf') throw new Error('media_type_not_allowed');
if (!/^[A-Za-z0-9_-]{8,80}$/.test(input.referralId)) throw new Error('referral_id_invalid');
if (input.idempotencyKey.length < 16) throw new Error('idempotency_key_required');
}
The 10 MiB ceiling is an example policy, not a medical standard. Pick a limit from your real referral forms, then test it at the edge and in the worker.
How do asynchronous jobs, retries, validation, and secure temporary files work together?
Make the state machine explicit: queued -> running -> archived, with failed as a terminal state after bounded attempts. Claim a job with a lease. On timeout, let the lease expire so another worker can continue. Every transition records actor, timestamp, job version, and reason; the audit trail is append-only.
Retries are for transient dependencies, not bad referrals. Use exponential backoff with jitter, a maximum attempt count, and a dead-letter review path. A renderer that says “invalid page” should not be retried five times. A temporary network timeout may be retried. Keep that distinction in a typed error classification, and make the archive operation idempotent so a worker crash after upload does not create two records.
For files, create a random directory with mode 0700, write with exclusive creation, and never concatenate user input into a path. Close file handles before deletion. A cleanup process should delete by expiresAt, not by “old-looking” names, and should emit a deletion audit event without logging the document contents. Memory is safer for small payloads; disk is safer than an unbounded buffer for larger ones, provided the disk is encrypted and monitored.
Privacy and retention are product behavior
Privacy is a data-flow property. Separate referral metadata from the document bytes, minimize what enters logs, redact identifiers in metrics, and make download authorization check the requesting principal on every read. Encryption in transit and at rest is table stakes; key rotation and access review are the work that keeps the table standing.
Retention needs a written schedule per record class: intake payload, rendered PDF, audit event, and failure metadata may have different lifetimes. The schedule should name the legal or contractual owner, the deletion trigger, and the evidence that deletion ran. Do not promise a universal number of days. I am not sure which rule applies to your jurisdiction; your compliance owner and counsel should resolve that before launch.
Run deletion tests against backups and replicas, too. A “delete” button that removes one database row while an export bucket keeps the bytes is not deletion. Your monitoring should alert on expired files, retry storms, lease starvation, and audit writes that lag behind job completion.
For a low-volume, non-sensitive preview tool, a queue is the wrong center of gravity. A direct stream to a renderer can reduce moving parts and latency. It becomes the wrong choice as soon as a caller needs a durable status, a signed audit trail, or a worker that can recover after a process restart.
For the medical referral case, keep the durable job boundary. For the marketplace monthly report, the same boundary lets a PDF be archived while the audit record captures the report period, source snapshot, renderer version, and retrieval decision. That is the useful abstraction: a small, inspectable state machine that makes privacy and failure handling visible.
Ship the boring path first.
Top comments (0)