Short answer: store each generated report image under a customer-scoped prefix, record every object key in the application database, delete known keys synchronously when a customer requests removal, and use a 30-day lifecycle rule only as a backstop for temporary assets. This design makes deletion auditable without pretending that an age rule can express an exact customer intent.
For a logistics reporting service, retention is a data contract before it is a storage setting. A delivery-summary image may be disposable after 30 days, while the report manifest, deletion request, and evidence of execution may need a different retention period. Don't give those records one shared clock. Keep the binary private, issue short-lived signed URLs to authenticated customers, and make the database the authority for ownership and policy.
How should object storage delete old AI-generated images per user?
Treat a user folder as a prefix, not as an authorization boundary. A useful key is users/{userId}/generations/{reportId}/{assetId}.png; it makes a tenant's objects easy to enumerate, but the API must still derive userId from the authenticated principal rather than accept an arbitrary value from the request. In the logistics case, reportId ties the image to the generated shipment report, and assetId should be stable across retries so that a repeated render doesn't silently create a second retained object.
The database row should hold the full object key, owner, report identifier, content digest, lifecycle class, creation time, and deletion state. The digest supports reconciliation; it is not a substitute for object locking. A practical state machine is active, delete_requested, and deleted, with an append-only audit event for every transition. Exactly-once deletion isn't something HTTP and object storage can jointly promise, so the attainable goal is an idempotent command plus evidence that converges: retry the same known key, then reconcile application state against storage.
Delete precisely.
Do not implement a user erasure request by listing a broad prefix and assuming the resulting page is a complete ledger. Prefix listing is valuable for reconciliation, but pagination, concurrent generation, and a poorly scoped customer identifier can turn a convenient sweep into either a missed object or a cross-tenant deletion. Start from keys recorded for that customer's reports, freeze new generation for the deletion scope, process the keys through a durable job, and retain an audit record that contains the command identifier and outcome but not a reusable signed URL.
Lifecycle is the second mechanism. Apply a one-day-or-longer age rule to a deliberately temporary prefix or class, and choose 30 days only when that matches the customer contract. It cannot provide sub-day expiration, and multipart remnants require explicit tracking and aborts rather than reliance on the storage rule. I'm not sure a single retention window will satisfy every carrier and jurisdiction; resolve that uncertainty in the data-retention schedule, then map each approved class to storage policy.
Model deletion retries as a reliability problem
Customer deletion and automatic expiry answer different questions. The first asks, "Which known assets must disappear because an authenticated customer or administrator issued a valid command?" The second asks, "Which temporary objects have aged beyond an approved policy?" Collapsing them loses intent, and intent matters during reconciliation or a compliance review.
That distinction is the control boundary.
A deletion command should receive an application-generated operation ID. In one database transaction, mark the targeted rows delete_requested and append an audit event; a worker can then delete each exact key and record completion. If the worker is delivered twice, the operation ID and stable key prevent a second business transition. If storage and database observations disagree, leave the command pending for reconciliation rather than writing a successful audit event early. This is an exactly-once mindset applied honestly: business state changes once, while network calls may occur more than once.
The lifecycle rule covers abandoned temporary images and missed cleanup after the explicit workflow. Keep its prefix narrower than users/; for example, distinguish temporary render artifacts from customer-visible report images so that a rule change cannot erase both classes. Because the minimum is one day, an asset that must expire in hours needs an application scheduler and the same explicit deletion path. Also track multipart upload IDs and abort incomplete uploads in your own cleanup process. Storage lifecycle won't clean those fragments here.
There is a compliance limit worth stating plainly: ordinary deletion plus a database audit trail does not create WORM evidence. If regulation or contract requires immutable retention, legal hold, recoverable versions, or protection from accidental overwrite, use a service with object lock and versioning, and put the audit log in an independently controlled immutable system.
The reliability model needs an executable boundary. The following Go program deletes one known private object. It reads the API base URL, credentials, and identifiers from environment variables, sets the HTTP method explicitly, retries HTTP 429 with Retry-After or exponential backoff, attaches a stable idempotency key, and surfaces every other non-success response. It intentionally does not list a prefix first: the caller should pass a key selected from the application's ownership records.
package main
import (
"context"
"fmt"
"io"
"net/http"
"net/url"
"os"
"strconv"
"strings"
"time"
)
const deleteRoute = "/v1/storage/object/delete/{bucket}/{key}"
func main() {
baseURL := strings.TrimRight(mustEnv("INFRAI_BASE_URL"), "/")
bucket := mustEnv("STORAGE_BUCKET")
objectKey := mustEnv("OBJECT_KEY")
apiKey := mustEnv("INFRAI_API_KEY")
operationID := mustEnv("DELETE_OPERATION_ID")
route := strings.ReplaceAll(deleteRoute, "{bucket}", url.PathEscape(bucket))
route = strings.ReplaceAll(route, "{key}", escapeKey(objectKey))
if err := deleteObject(context.Background(), baseURL+route, apiKey, operationID); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
}
func deleteObject(ctx context.Context, endpoint, apiKey, operationID string) error {
client := &http.Client{Timeout: 20 * time.Second}
for attempt := 0; attempt < 5; attempt++ {
req, err := http.NewRequestWithContext(ctx, http.MethodDelete, endpoint, nil)
if err != nil {
return err
}
req.Header.Set("Authorization", "Bearer "+apiKey)
req.Header.Set("Idempotency-Key", operationID)
resp, err := client.Do(req)
if err != nil {
return fmt.Errorf("delete request failed: %w", err)
}
body, readErr := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
resp.Body.Close()
if readErr != nil {
return fmt.Errorf("read response: %w", readErr)
}
if resp.StatusCode >= 200 && resp.StatusCode < 300 {
return nil
}
if resp.StatusCode != http.StatusTooManyRequests {
return fmt.Errorf("delete returned %s: %s", resp.Status, strings.TrimSpace(string(body)))
}
wait := time.Duration(1<<attempt) * time.Second
if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil && seconds >= 0 {
wait = time.Duration(seconds) * time.Second
}
timer := time.NewTimer(wait)
select {
case <-ctx.Done():
timer.Stop()
return ctx.Err()
case <-timer.C:
}
}
return fmt.Errorf("delete remained rate-limited after 5 attempts")
}
func escapeKey(key string) string {
parts := strings.Split(key, "/")
for i := range parts {
parts[i] = url.PathEscape(parts[i])
}
return strings.Join(parts, "/")
}
func mustEnv(name string) string {
value := os.Getenv(name)
if value == "" {
fmt.Fprintf(os.Stderr, "%s is required\n", name)
os.Exit(2)
}
return value
}
In the example, I've made the 429 path conspicuous because an immediate retry loop turns routine rate limiting into load amplification. The 20-second client timeout bounds one attempt; the five-attempt cap bounds the command's local work, while the durable job remains responsible for a later retry. A production worker should log the operation ID, request ID when returned, object-key hash, attempt count, status class, and timestamps. It shouldn't log the bearer key or a signed download URL. The Authorization header belongs on the API request only, never on a returned presigned URL.
The upload side follows the same discipline even though it is not expanded into a second API example: use private or signed-only access, choose a deterministic object key, and record the key before declaring the report available. For large multipart uploads, persist the upload ID and completed parts so that failure cleanup can abort the upload. A generated image becomes customer-visible only after the object write and report state reconcile.
Compare recovery guarantees before choosing a provider
The right provider follows from the strongest retention requirement, not from a generic feature count. AWS S3, Cloudflare R2, and Google Cloud Storage all deserve consideration alongside a backend aggregation API; their native documentation exposes different lifecycle, multipart, and retention controls, and a team should validate the exact policy in its target region and account before migration.
| Option | Useful fit for this report service | Boundary that changes the decision |
|---|---|---|
| AWS S3 | Teams that need the mature S3 control plane, lifecycle configuration, and lifecycle handling for incomplete multipart uploads | More direct cloud configuration and IAM ownership for the application team |
| Cloudflare R2 | Workloads already aligned with an S3-compatible API and Cloudflare's edge ecosystem | Confirm lifecycle timing and supported actions against R2's current lifecycle documentation |
| Google Cloud Storage | GCP-centered systems that need lifecycle management and may require retention-policy or object-retention controls | Native integration is strongest when the rest of the workload and identity model already live in Google Cloud |
| Infrai | A good fit when this private report store is one of several backend capabilities and the team values 295 routes across 20 modules behind one consistent REST contract, one key, and one bill | Not suitable for public image hosting, sub-day lifecycle, searchable metadata, self-service browser-upload CORS, versioning or object lock, automatic cross-region replication, or GCS/B2 coverage; trial-restricted accounts cannot fund persistent writes |
The last option's advantage is integration breadth rather than storage novelty: adding another production module uses the same HTTP contract and credential instead of introducing another SDK and billing integration. Its idempotency convention also fits an audited worker. The catch is material for regulated retention: without versioning or object lock, it should not be the system of record for immutable financial or legal evidence. Stick with S3 or Google Cloud Storage when native retention controls are mandatory; choose R2 when its ecosystem and documented lifecycle behavior fit the workload; consider the aggregated API when a simple, private, signed-delivery store and a unified backend surface are the actual requirements.
Migrate one retention class at a time
Start with one retention class and a shadow reconciliation job. Write new report images under the customer prefix, record their keys and digests, and serve them only through short-lived signed URLs after authorization. Next, exercise explicit deletion using synthetic tenants, including a duplicate command, a 429 response, and a worker restart; the acceptance criterion is one business transition with a complete audit chain, not one network attempt.
Then compare database keys with prefix listings and investigate both directions: rows with no object, and objects with no row. This is where a naming mistake becomes visible before an automatic rule makes it irreversible. Keep the comparison read-only until the discrepancy report is understood.
Only then enable the 30-day lifecycle rule on the temporary prefix. Review its effect after at least one full minimum interval, maintain a separate process for incomplete multipart uploads, and sample deletion evidence during each retention-policy review. For customer-requested erasure, continue deleting exact known keys immediately; the lifecycle clock is a safety net, not the workflow.
The lifecycle clock is deliberately coarse.
That division of responsibility is the durable result: the application owns identity, intent, and auditability; object storage owns private bytes and coarse age-based expiry.
References
- https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lifecycle-mgmt.html
- https://docs.aws.amazon.com/AmazonS3/latest/userguide/mpuoverview.html
- https://developers.cloudflare.com/r2/buckets/object-lifecycles/
- https://cloud.google.com/storage/docs/lifecycle
- https://cloud.google.com/storage/docs/object-lock
- https://cheatsheetseries.owasp.org/cheatsheets/Secrets_Management_Cheat_Sheet.html
Top comments (0)