Tenant-Isolated Object Storage: Delete Old Thumbnails After Image Replace

Tenant isolation changes the cleanup decision. For a customer-support service accepting private uploads, store each thumbnail generation under a tenant-scoped, image-scoped prefix, publish the complete replacement in the database, and only then enqueue deletion of the old generation. A one-day lifecycle rule can be a backstop, but it is not the replacement transaction.

That order is the answer. The storage system owns bytes; the database owns which bytes a support agent is allowed to request. Mixing those responsibilities creates a particularly unpleasant failure: a cleanup worker for tenant A can remove objects that a later retry, or even another tenant, still considers live.

How should you delete old thumbnails after an image replacement in object storage?

Use an immutable generation rather than overwriting keys such as thumbs/160.webp. A useful key shape is:

tenants/{tenant_id}/images/{image_id}/generations/{generation_id}/{variant}.webp

For tenant acme, image img_42, and generation g_18, the 160-pixel variant becomes tenants/acme/images/img_42/generations/g_18/160.webp. The application record stores the active generation or its complete prefix. It should never discover the active generation by scanning storage after the fact.

The write path has three durable boundaries:

1. Validate that the caller belongs to the tenant and that the upload is private.
2. Write every expected thumbnail under a new generation prefix.
3. In a database transaction, publish that generation and record the old prefix as a cleanup job.
4. Let a worker list the recorded old prefix and batch-delete only those returned keys.

The database commit is the visibility boundary. Before it, readers use the old set. After it, readers use the new set, even if the cleanup worker has not run yet. That temporary duplicate storage is an acceptable failure boundary; serving a thumbnail that has already been deleted is not.

The cleanup target must be captured at publication time. Imagine replacement A publishes g_18, then replacement B publishes g_19 before A's worker gets scheduled. A must delete the prefix belonging to g_17; it must not reread the row and calculate “everything except g_19.” That calculation can accidentally include the generation B had just published or a generation needed by a delayed reader. A recorded target makes retries narrow and auditable.

This is also where a Node.js implementation needs discipline. The client library, promise chain, or queue changes the syntax, not the state machine: prepare objects, publish the active pointer, enqueue the old prefix, list it, then batch-delete the listed keys. Keep the storage interface small enough that the application can test this order without a real bucket.

The architecture decision record

The invariants are more important than the particular object-storage API.

Decision	Invariant	Failure boundary
Prefix includes tenant, image, and generation	A cleanup job has one explicit ownership scope	A malformed or missing scope is rejected before deletion
New variants are written before publication	The active database pointer never names a partial set	An incomplete new generation is abandoned and later reaped
Publication and cleanup are separate	A worker cannot make the live record disappear	The old set may remain temporarily
Cleanup stores its target prefix	A retry repeats the same deletion set	A newer replacement does not change an older job's target
Batch deletion is idempotent	Replaying a completed job is harmless	An already-absent key is treated as success by the adapter

The tenant prefix is a security boundary in the naming scheme, not an authorization system. Every read and write still needs an authorization check against the tenant in the request and the tenant stored with the image. A prefix filter reduces the blast radius of a mistake; it does not prove that the caller is entitled to the prefix.

Object versioning is a separate recovery control. It can preserve prior object versions after an overwrite, but it does not tell the application which thumbnail generation is active, and it does not replace tenant checks. If rollback requirements depend on retaining old object versions, enable and test that feature deliberately; generation-scoped keys remain useful because they make publication and cleanup sets explicit. The general storage documentation for object versioning describes this distinction between a current version and retained prior versions.

There is a concurrency question here that storage alone cannot settle. Two image replacements need a database transaction, row lock, compare-and-swap field, or serialized queue, depending on the service's write rate and database. I'm not sure which coordination primitive fits without those details. The non-negotiable part is that a job records the version it replaced and never performs a broad “delete all non-current objects” scan.

A small, testable cleanup path

The following standard-library example models the critical path. Its adapter exposes prefix listing and batch deletion, while the database keeps the active generation and cleanup target. A production Node.js service can implement the same contract with its chosen object-storage client.

from __future__ import annotations

import sqlite3
from dataclasses import dataclass, field
from typing import Iterable, Protocol


class ThumbnailStore(Protocol):
    def put(self, key: str, content: bytes) -> None: ...
    def list_prefix(self, prefix: str) -> list[str]: ...
    def delete_batch(self, keys: Iterable[str]) -> None: ...


@dataclass
class MemoryStore:
    objects: dict[str, bytes] = field(default_factory=dict)

    def put(self, key: str, content: bytes) -> None:
        self.objects[key] = content

    def list_prefix(self, prefix: str) -> list[str]:
        return sorted(key for key in self.objects if key.startswith(prefix))

    def delete_batch(self, keys: Iterable[str]) -> None:
        for key in keys:
            self.objects.pop(key, None)


def prefix(tenant_id: str, image_id: str, generation_id: str) -> str:
    return (
        f"tenants/{tenant_id}/images/{image_id}/"
        f"generations/{generation_id}/"
    )


def replace_image(
    db: sqlite3.Connection,
    store: ThumbnailStore,
    tenant_id: str,
    image_id: str,
    generation_id: str,
    variants: dict[str, bytes],
) -> None:
    row = db.execute(
        "SELECT active_prefix FROM images WHERE tenant_id = ? AND image_id = ?",
        (tenant_id, image_id),
    ).fetchone()
    old_prefix = row[0] if row else None
    new_prefix = prefix(tenant_id, image_id, generation_id)

    for variant, content in variants.items():
        store.put(f"{new_prefix}{variant}", content)

    with db:
        db.execute(
            """
            INSERT INTO images (tenant_id, image_id, active_prefix)
            VALUES (?, ?, ?)
            ON CONFLICT(tenant_id, image_id) DO UPDATE SET
                active_prefix = excluded.active_prefix
            """,
            (tenant_id, image_id, new_prefix),
        )
        if old_prefix and old_prefix != new_prefix:
            db.execute(
                "INSERT INTO cleanup_jobs (tenant_id, prefix, state) VALUES (?, ?, ?)",
                (tenant_id, old_prefix, "pending"),
            )


def run_cleanup_job(store: ThumbnailStore, prefix_to_remove: str) -> None:
    keys = store.list_prefix(prefix_to_remove)
    if keys:
        store.delete_batch(keys)

The example deliberately does not delete inside replace_image. A request can return after publication while a worker continues safely, and a worker can retry after a process restart. In production, the cleanup job should also carry an image identifier and generation identifier, record attempts, and use bounded batches so one unusually large upload does not monopolize the worker.

The database transaction shown here assumes the row and job tables have the matching uniqueness constraints. It also leaves a subtle production concern visible: writing objects and committing a database transaction are different systems. If the process dies after the object writes but before publication, the new prefix is orphaned. A periodic reaper may remove generations that are neither active nor referenced by a pending job, but that reaper must use the same tenant-scoped naming and authorization rules as the normal worker.

What can go wrong during listing and batch deletion?

Listing is a snapshot for a cleanup attempt, not a permanent inventory. New objects should not be written into an old generation prefix, and the worker should treat a repeated listing as normal after a retry. If the storage API paginates results, the adapter must consume every page before declaring the prefix empty. If batch deletion has a request-size limit, split the listed keys into bounded batches and persist progress or make replay safe.

The error policy should distinguish an already-absent stale key from an authorization failure, malformed request, or rate limit. A missing object can be success for an idempotent cleanup job. A permission failure must remain visible and must not trigger a retry loop that silently expands its target. A throttled request should back off according to the service response when possible. Never send the service's authorization header to a presigned download URL.

Observability should answer four questions for every job: which tenant owns it, which generation was published, which prefix is being removed, and how many keys remain after the attempt. Log identifiers and counts rather than private image contents. Alert on cleanup age and repeated authorization failures; a growing queue is a storage hygiene problem, while a database pointer to missing active objects is an availability and integrity problem.

The private-upload requirement also affects the download path. Return short-lived signed links only after checking tenant ownership, and do not treat an unguessable object key as permission. This design is not suitable for public static assets, permanent cache URLs, or a workflow that requires users to share immutable public links. For those cases, use a separate public-asset policy and keep it outside the private support-upload namespace.

Why a one-day lifecycle rule is only a backstop

Lifecycle expiration has a minimum age of one day in this decision. That is useful for abandoned generations, crash leftovers, and retention hygiene when delayed removal is acceptable. It is the wrong mechanism for an image replacement that should stop exposing the old generation as soon as the database publishes the new one.

The rejected lifecycle-only design has a clean boundary: it knows object age, not the database event that made a generation obsolete. Keep explicit prefix listing and batch deletion on the replacement path, then use lifecycle as a coarse safety net. The trade-off is extra worker code and more operational signals in exchange for a small, explicit deletion set and a clear tenant boundary.

Before shipping, test concurrent replacements for one image and for two tenants with similar image IDs. Test a crash after the last thumbnail write, a crash after publication, an empty prefix, a paginated listing, a repeated batch delete, and a rejected tenant authorization. Test the retention rule with a clock you control. Storage cleanup is uninteresting when it works; the test suite is what makes the destructive path trustworthy.

References

• https://csrc.nist.gov/pubs/sp/800/66/r2/final
• https://docs.aws.amazon.com/AmazonS3/latest/userguide/Versioning.html