Short answer: keep the original image in object storage under an application-owned, user-scoped key, and treat the database as the authority for retention and deletion. For a media service processing receipts, large-file throughput should decide the upload path first; lifecycle cleanup and user-requested deletion are separate controls with separate SLOs.
The useful design is boring: one immutable object key per source file, a small record describing that object, bounded workers, and an audit trail for every deletion decision. A folder is only a prefix in a flat object namespace. It is not an access-control boundary, a transaction, or a substitute for metadata.
How should I store generated images per user folder?
Use a key that is difficult to confuse with another receipt, for example users/{userID}/receipts/{receiptID}/original.png. Generate receiptID server-side, store the complete key in the database, and keep the original separate from derived previews. A per-user prefix makes an operational listing possible, but deletion should use the recorded key rather than infer intent from a filename or prompt. I don't regard the prefix as authorization; the API still needs an authorization check against the user ID in the record.
The record should include the user identifier, object key, byte size, content type, upload state, creation time, retention deadline, and a deletion-request timestamp. Keep generation and receipt-processing state there too. Object storage is good at durable bytes; it is a poor place to discover why a file exists.
Use a direct, explicit delete for a user request or a legal-retention transition. Use a lifecycle rule for material that is allowed to expire by age, such as failed intermediates or temporary derivatives. Do not promise an hour-level deletion SLO from a day-granularity lifecycle policy. If the policy requires removal sooner, the application worker owns that deadline and must report it.
There is a sharp boundary here. Lifecycle is housekeeping. Deletion intent is a business event.
That distinction is the first review question.
What does large-file throughput require from the upload path?
For receipt images, the original file is the audit artifact, so an upload that silently becomes a thumbnail is a data-loss event even if the UI looks healthy. Measure the whole path: bytes accepted, upload duration, completed objects, incomplete multipart uploads, processing latency, and the age of the oldest pending receipt. The SLO should describe the user-visible obligation, such as the fraction of accepted receipts whose original is durable before processing begins, rather than only the HTTP response time. A 429 from a storage API is a capacity signal, not a reason to drop the receipt.
Capacity planning starts with bytes, not request count. If the service accepts R receipts per minute and the p95 source size is B bytes, the ingress estimate is R * B; add headroom for retries, reprocessing, and a burst window. A worker pool sized for ten small images can be badly undersized for two large originals. Record size distributions, not just averages, because a few large files dominate bandwidth and temporary disk.
Multipart upload is the right shape when a large object must be transferred in parts and retried without restarting the entire file. The upload identifier belongs in durable job state. On success, mark the object complete and persist its key; on an abandoned attempt, abort the multipart upload and track that cleanup separately. A lifecycle rule for completed objects does not, by itself, make an incomplete upload part of the completed-object retention story. The runbook has to cover the half-finished state because retries, worker restarts, and deployments all create it; a dashboard that counts only completed objects will call this path healthy while storage continues to accumulate unfinished work.
The application should not hold a large file in Node memory while it waits for the storage request. Stream from the accepted input to the storage client or issue a bounded presigned upload, then verify the completed object before enqueueing receipt processing. The exact stream implementation depends on the storage client, so I would test it under the p99 file size and a deliberately interrupted connection before choosing worker limits.
How can a deletion worker preserve an audit trail?
The worker below uses a generic HTTP contract rather than a vendor SDK. The important behavior is independent of the endpoint: select a bounded set of records, save the selection, delete only their recorded keys, retry transient throttling, and make the result observable. Secrets should come from a managed secret mechanism or an injected environment value; they should not be committed or printed in logs.
package main
import (
"context"
"fmt"
"io"
"net/http"
"os"
"time"
)
type Deleter struct {
BaseURL string
Token string
Client *http.Client
}
func (d Deleter) Delete(ctx context.Context, bucket, key string) error {
endpoint := d.BaseURL + "/objects/" + bucket + "/" + key
for attempt := 0; attempt < 4; attempt++ {
req, err := http.NewRequestWithContext(ctx, http.MethodDelete, endpoint, nil)
if err != nil {
return err
}
req.Header.Set("Authorization", "Bearer "+d.Token)
res, err := d.Client.Do(req)
if err != nil {
return err
}
_, readErr := io.Copy(io.Discard, res.Body)
res.Body.Close()
if readErr != nil {
return readErr
}
if res.StatusCode == http.StatusTooManyRequests {
timer := time.NewTimer(time.Duration(1<<attempt) * time.Second)
select {
case <-ctx.Done():
timer.Stop()
return ctx.Err()
case <-timer.C:
}
continue
}
if res.StatusCode < http.StatusOK || res.StatusCode >= http.StatusMultipleChoices {
return fmt.Errorf("delete %q: status %d", key, res.StatusCode)
}
return nil
}
return fmt.Errorf("delete %q: retry budget exhausted", key)
}
func main() {
deleter := Deleter{
BaseURL: os.Getenv("OBJECT_STORE_URL"),
Token: os.Getenv("OBJECT_STORE_TOKEN"),
Client: &http.Client{Timeout: 30 * time.Second},
}
if err := deleter.Delete(context.Background(), os.Getenv("OBJECT_BUCKET"), os.Getenv("OBJECT_KEY")); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
}
The sample intentionally does not list a prefix. A cleanup job that lists users/alice/ and deletes whatever looks old can remove a newly reprocessed receipt, miss a key moved by a migration, or cross an authorization boundary if the prefix is assembled incorrectly. Store the selection set before execution, include an idempotency key for the run, and make a retry safe to repeat. Depending on the storage contract, a successful delete may be idempotent; verify that behavior in the target service rather than assuming it. If the first attempt gets a 429, the job should retain its selection and retry that same decision; it should not run a new broad query and hope the result still means the same thing.
Keep the token out of request URLs and logs. The OWASP guidance on secrets management is a useful baseline for storage, rotation, access control, and exposure review. A deletion log needs the record ID, object key, actor or job ID, decision, attempt count, and outcome, but it does not need the secret or the receipt contents.
Verify bytes first, then enable retention
Start with a fixture set containing a small file, a p95-sized file, an interrupted multipart upload, an already-expired derivative, and two users whose identifiers share a prefix. Run the worker in dry-run mode, inspect the persisted selection, then execute against only that fixture set. Verify the original bytes and content type before processing, verify that the intended keys are absent after deletion, and check that unrelated keys remain. This is where a receipt archive earns its SLO: the test must prove that the original survives a worker restart, a retry, and a delayed lifecycle sweep, not merely that one happy-path request returns 2xx.
Alert on age, not just volume: oldest pending deletion, selected-versus-successful count, retry count, incomplete-upload count, and the percentage of accepted receipts without a durable original are more informative than one cleanup_success counter. A zero-delete run may be correct for an empty queue. A growing oldest age is not.
Rollback is a policy decision because deleting the only copy cannot be undone by the worker. Before physical deletion, a product may hide the image and hold the record during a grace period. That is unsuitable when a legal deletion deadline requires immediate removal. In that case, require a reviewed selection query, a small initial batch, and an independent audit record before widening the job.
The trade-offs that matter in production
| Choice | Strength | Cost or boundary |
|---|---|---|
| Application-owned keys and metadata | Clear ownership, retention intent, and auditability | Requires a durable database record and migration discipline |
| Explicit deletion worker | Can be measured against a user-facing SLO | Adds queueing, retries, authorization checks, and on-call signals |
| Age-based lifecycle policy | Simple housekeeping for temporary data | Cannot express every business rule or necessarily meet a short deletion deadline |
| Multipart upload | Better retry behavior for large originals | Requires upload-state tracking and abandoned-upload cleanup |
| Self-hosted object storage | Greater control over placement and operations | Your team owns disks, upgrades, capacity, replication, and incidents |
| Managed object storage | Less storage infrastructure to operate | Provider-specific policy surfaces and lock-in need review |
The recommendation is therefore conditional: use user-scoped immutable keys, explicit metadata, multipart transfer for genuinely large originals, and lifecycle rules only for data whose age is the policy. It is not suitable when the service needs transactional rename semantics, an immediate globally verified deletion guarantee, or a replication and retention control that the selected storage system does not provide; choose a storage design that exposes those guarantees and test them at the expected file-size distribution.
Your mileage may vary. I am not sure a single throughput target can be stated without the source-size distribution, network shape, and processing SLO, so those measurements should be collected before capacity limits are fixed.
Top comments (0)