Short answer: use a short-lived staging bucket, a database retention record, and an explicit deletion worker that proves every original image and generated video reached a terminal state. Lifecycle rules are a backstop, not the policy.
The bill starts with bytes, but the expensive part is usually the behavior around those bytes. A campaign upload can create an original image, several resized thumbnails, a moderation derivative, and a generated video preview. If the campaign is rejected, teams often delete the original and quietly retain the derivatives, logs, and multipart fragments. Storage grows while nobody can answer which copy is still allowed to exist.
For a fintech workflow, moderation coverage is the decision axis. A thumbnail that escaped moderation is a customer-facing risk; a deleted evidence image can be a compliance problem. I treat retention as a state machine with an owner, a deadline, and an audit event. That is less clever than hoping a provider's lifecycle sweep runs at the right time.
What should a retention record prove before deleting campaign images and generated videos?
The record needs immutable identifiers, not just a path. Store the campaign ID, object version or generation, media kind, moderation decision, legal hold flag, creation time, expiry time, and deletion attempt history. A worker may then make a narrow decision: delete only the exact version that the policy selected, and only after moderation has reached a terminal result.
Here is a small policy evaluator. It does not call a storage vendor; the adapter behind delete_object can speak S3-compatible HTTP, a filesystem gateway, or an internal service.
from dataclasses import dataclass
from datetime import datetime, timezone
@dataclass(frozen=True)
class MediaRecord:
object_key: str
version: str
kind: str
moderation: str
expires_at: datetime
legal_hold: bool
def deletion_decision(record: MediaRecord, now: datetime) -> str:
if record.legal_hold:
return "hold"
if record.moderation not in {"approved", "rejected"}:
return "wait_for_moderation"
if now < record.expires_at:
return "retain"
return "delete_exact_version"
now = datetime.now(timezone.utc)
The important detail is delete_exact_version. A retry must be idempotent and must not remove a newly uploaded object that reused the same human-readable filename. Keep a tombstone for the version, then verify that a read-after-delete check (or the storage system's documented consistency contract) agrees with the audit event.
How do cost and retention choices change the media pipeline?
Count each byte class separately: originals, thumbnails, moderation artifacts, generated video frames, and failed multipart uploads. Processing and egress can dominate a small storage bill when a moderation service repeatedly downloads a large source. A policy that keeps every derivative for 30 days may look tidy while multiplying both transfer and scan work.
I start with a retention budget per campaign, then ask which artifact is actually needed after publication. In one design review, the proposed rule retained four thumbnail sizes plus a full-resolution image for every rejected asset. The safer compromise was to keep a cryptographic digest, moderation decision, and a low-resolution review copy for seven days, while deleting the source and generated video at the campaign deadline. Your mileage may vary: regulated investigations may require a legal hold that overrides this budget.
| Artifact | Typical purpose | Deletion trigger | What is lost |
|---|---|---|---|
| Original upload | Reprocessing and evidence | Moderation terminal + expiry | Ability to regenerate every derivative |
| Thumbnail | User interface | Campaign end + short grace period | Fast replay of the campaign view |
| Generated video | Preview or export | Campaign end unless published | Original render for later edits |
| Moderation metadata | Decision audit | Policy-defined retention | Easy reconstruction of the decision |
The catch is operational: deleting aggressively reduces exposure and storage, but it also removes the easiest path to investigate a disputed moderation result. Keep metadata that explains the decision even when the pixels are gone, and make that split visible to reviewers.
Delete twice.
That sounds absurd until the race is written down. At 09:00, the upload service records campaign-418/original.jpg as version a7 and enqueues a seven-day expiry. At 09:01, a retrying thumbnailer writes a fresh version under the same key because the first response was lost. At 09:02, the expiry worker lists by key, sees the new version first, and removes it even though its retention clock has not started. The fix is not a longer sleep: the queue payload must carry the immutable version, and the delete request must include that version. The worker then records a successful deletion, or an already-absent result, against that exact identifier. A later upload can use the same key without inheriting an old campaign's deadline. This is why I prefer an auditable state transition over a nightly "delete everything older than N days" query, even when the latter is easier to explain on a diagram.
Which failure modes make explicit deletion unreliable?
The common failures are mundane. A queue message is delivered twice. A worker crashes after deleting the source but before recording the event. A clock is wrong by six minutes. A legal hold arrives between selection and deletion. Multipart uploads remain incomplete because the cleanup job only lists completed objects.
Use an outbox transaction when changing the retention record: commit the new state and deletion command together, then let workers retry with exponential backoff. Emit metrics for selected, deleted, already_absent, held, and failed, with campaign and media-kind dimensions. Alert on age of the oldest eligible object, not just on worker process health.
Tests should include a version collision, duplicate delivery, a hold added during a retry, and a storage timeout after the server accepted the delete. Property-based tests are useful here because the invariant is simple: no object under legal hold is deleted, and no expired eligible version remains indefinitely without an alert.
When is this retention design not suitable?
It is not suitable when the product promises user-initiated restoration, frame-accurate video editing after campaign close, or forensic access to the original pixels for years. In those cases, retain a governed archive and separate it from the hot staging bucket; do not stretch a temporary-media policy until it becomes an accidental archive.
Standards still matter. Object naming, media sniffing, and thumbnail generation should follow the format guidance in MDN, while retention semantics belong in your policy and audit model. A lifecycle rule can enforce a maximum age, but it cannot know whether moderation finished, a hold was placed, or a human appealed a decision. Treat it as a final guardrail.
References
- https://developer.mozilla.org/en-US/docs/Web/Media/Guides/Formats
- https://datatracker.ietf.org/doc/html/rfc9110
- https://csrc.nist.gov/publications/detail/sp/800-57-part-1/rev-5/final
Further reading
The same sources provide the format, HTTP semantics, and key-management context behind this design.
Top comments (0)