Short answer: for a property-management system that stores AI-generated receipt images, use multipart object storage with a short-lived presign, an explicit completion record, and a retention job that deletes only after the audit window closes. The upload is a workflow, not a single HTTP request.
At 3 a.m., the useful question is not "did the dashboard turn green?" It is "which page fired, and can we still prove which original file was attached to this expense?" A one-request upload can leave a proxy buffering a 1.2 GB image, a process restart halfway through, and no durable evidence that the source bytes survived. Keep the original immutable object, write its hash and receipt ID to a database, and make every state transition observable.
What should Node.js teams retain before completing a multipart object upload?
Start with a receipt row containing pending, the object key, tenant, uploader, expected byte count, and an expiry timestamp. Generate an upload ID through the S3-compatible service, then presign individual part uploads. The browser or worker sends parts directly; your Node.js API only coordinates. A part is addressed by number, and its returned checksum or ETag is recorded beside that number. Never infer completion from the client saying "100%."
The retention decision belongs outside the upload session. For example, keep original/tenant-42/receipt-9812.png for the lease and tax-audit period, while a derivative thumbnail may have a shorter lifecycle. A deletion worker must check the receipt state, legal hold, and retention timestamp in one transaction before issuing a delete. If a hold appears, deletion is skipped and the reason is logged. That is boring by design.
| State | Object action | Audit record |
|---|---|---|
pending |
Presign parts; allow retry | Upload ID, expiry, expected bytes |
complete |
Keep original under policy | Hash, receipt ID, completion time |
aborted |
Remove unfinished parts | Actor, reason, timestamp |
Keep it boring.
The following Go sketch shows the control-flow boundary. The storage adapter can target any S3-compatible implementation; the important part is that completion and abort are authenticated server actions, while part bytes bypass the application process.
type Part struct {
Number int
ETag string
}
type Upload struct {
ID string
ObjectKey string
Parts []Part
}
func complete(w http.ResponseWriter, r *http.Request, u Upload) {
if !authorized(r, u.ObjectKey) {
http.Error(w, "forbidden", http.StatusForbidden)
return
}
if !contiguous(u.Parts) {
http.Error(w, "missing part", http.StatusConflict)
return
}
// Commit storage first, then mark the receipt complete in one idempotent command.
if err := storageComplete(u.ID, u.Parts); err != nil {
http.Error(w, "completion failed", http.StatusBadGateway)
return
}
markReceiptComplete(u.ObjectKey)
w.WriteHeader(http.StatusNoContent)
}
func abort(w http.ResponseWriter, r *http.Request, u Upload) {
if !authorized(r, u.ObjectKey) {
http.Error(w, "forbidden", http.StatusForbidden)
return
}
storageAbort(u.ID)
markReceiptAborted(u.ObjectKey)
w.WriteHeader(http.StatusNoContent)
}
In production, make both commands idempotent. A retry after a network timeout must not create a second receipt or turn an already-complete upload back into pending. Store the part list server-side, cap part size and count, and reject keys containing tenant data supplied by the client. The object key should be derived from the authenticated tenant and receipt ID. When a leasing clerk uploads a large scan from a weak connection, the worker may resume three hours later, after credentials have rotated and a legal hold has been added; the reconciler must therefore reload authorization and retention state rather than trusting the original session metadata, compare every recorded checksum, and leave a precise event for the auditor if completion is no longer allowed.
How do presign, complete, abort, and retention behave after a failure?
Treat each failure as a state transition with a recovery owner. An expired presign is safe to regenerate while the session is pending. A missing part causes completion to remain pending, so a reconciler can request that part again. A worker crash after storage completion but before the database update is repaired by a periodic head-and-reconcile pass keyed by upload ID. An abandoned session is aborted after its deadline, and its receipt row records who or what performed the cleanup.
Test the ugly paths: kill the coordinator between calls, replay completion twice, upload parts out of order, send a wrong checksum, and run deletion while a legal hold is being added. Your alerts should include tenant, receipt ID, upload ID, age, and state. Page on an old pending session or an audit object missing its metadata; do not page on every transient client retry. I have learned to distrust "successful uploads" that have no corresponding audit row.
The catch is that this design is not suitable when users need a single atomic transaction spanning object storage and several unrelated databases; use an orchestrated saga or a storage system with that transaction model. Stick with a simple single-put upload for small, bounded files where restart cost is negligible. Multipart adds state, reconciliation, and cleanup work, and that operational burden is the price of resumability.
Verification and rollback checklist
Before rollout, verify that a completed receipt can be fetched by an auditor without application credentials, that the stored hash matches the source bytes, and that a retention simulation never deletes a held object. Measure pending-session age and orphaned-part bytes, then exercise the abort path in staging.
Rollback is a configuration change: stop issuing new sessions, let active sessions expire or abort, and leave completed objects readable. Do not bulk-delete the prefix during rollback. Preserve the receipt rows so the audit trail explains what happened, even when a generated image is later removed under policy.
Top comments (0)