DEV Community

JasperFlint6947
JasperFlint6947

Posted on

Private Image Intake with Tenant Isolation: Multipart Uploads and Python Thumbnail Jobs

Short answer: for a large image upload, make tenant identity part of every object key and job record, use multipart upload before resizing, and issue short-lived download links only after an authorization check in the Node.js or Python service. The thumbnail is a derivative, not a new permission boundary.

That rule matters more than the choice between a Node.js and Python service, or between Sharp and another image library. A media application usually has three different moments: a browser sends bytes, a worker decodes an image, and a user receives a private result. Tenant isolation has to survive all three.

I build this sort of flow with an eval harness nearby. A passing upload test is not enough if tenant A can fetch tenant B's thumbnail, or if a retry makes two jobs write the same derivative. The first experiment should therefore measure authorization outcomes and object ownership, not just resize latency.

How should a large image upload become a private thumbnail without breaking tenant isolation?

Start with an opaque upload record owned by the authenticated tenant. Give it a server-generated identifier, an expected media type, and a state such as pending, complete, or rejected. The client may upload parts, but it must never choose the final storage prefix. A safe shape is tenant/{tenant_id}/original/{asset_id}; the derivative can be tenant/{tenant_id}/thumbnail/{asset_id}/320.jpg.

The prefix is useful, but it is not authorization. Every read still needs a database lookup that confirms the caller's tenant owns the asset and that the requested derivative belongs to that asset. This distinction catches a common bug: filtering the upload list by tenant while leaving the download endpoint addressable by an unscoped object key.

The resize worker consumes a completed upload event. It loads the original through a storage adapter, checks the recorded asset state, creates a thumbnail, and writes only to the derivative key. It does not accept a raw destination key from an untrusted request. The job should carry tenant_id, asset_id, source_key, derivative_key, and a transformation version so retries are explicit and inspectable.

Three checks are worth making executable:

  • A pending asset cannot create a thumbnail job.
  • A job for tenant A cannot read a source key owned by tenant B.
  • A download link is minted only after the ownership check, and it expires.

Small state machines beat clever callbacks here.

A Python experiment for keys, jobs, and expiring access

The following example is deliberately storage-neutral. It models the contract around an S3-compatible private bucket without pretending that a local dictionary is the bucket itself. The image bytes are represented by a placeholder payload; production code should decode and resize them with the image library already standardized by the team, such as Sharp in a Node.js worker or Pillow in Python.

from __future__ import annotations

from dataclasses import dataclass
from datetime import datetime, timedelta, timezone


@dataclass(frozen=True)
class Asset:
    tenant_id: str
    asset_id: str
    source_key: str
    state: str = "pending"


class IsolationError(PermissionError):
    pass


def source_key(tenant_id: str, asset_id: str) -> str:
    return f"tenant/{tenant_id}/original/{asset_id}"


def thumbnail_key(tenant_id: str, asset_id: str, width: int) -> str:
    return f"tenant/{tenant_id}/thumbnail/{asset_id}/{width}.jpg"


def enqueue_thumbnail(asset: Asset, requested_tenant: str, width: int) -> dict[str, str]:
    if asset.tenant_id != requested_tenant:
        raise IsolationError("asset belongs to another tenant")
    if asset.state != "complete":
        raise ValueError("thumbnail requires a completed upload")
    return {
        "tenant_id": asset.tenant_id,
        "asset_id": asset.asset_id,
        "source_key": asset.source_key,
        "derivative_key": thumbnail_key(asset.tenant_id, asset.asset_id, width),
        "transform_version": "thumb-v1",
    }


def make_download_link(asset: Asset, requested_tenant: str, now: datetime) -> str:
    if asset.tenant_id != requested_tenant:
        raise IsolationError("asset belongs to another tenant")
    if asset.state != "complete":
        raise ValueError("asset is not available")
    expires_at = now + timedelta(minutes=10)
    # The adapter signs this private key with the storage service.
    return f"private-link:{asset.source_key}?expires={expires_at.isoformat()}"


asset = Asset("tenant-a", "asset-42", source_key("tenant-a", "asset-42"), "complete")
job = enqueue_thumbnail(asset, "tenant-a", 320)
assert job["derivative_key"] == "tenant/tenant-a/thumbnail/asset-42/320.jpg"

try:
    enqueue_thumbnail(asset, "tenant-b", 320)
except IsolationError:
    pass

link = make_download_link(asset, "tenant-a", datetime.now(timezone.utc))
assert "expires=" in link
Enter fullscreen mode Exit fullscreen mode

The important part is the data carried between boundaries. In a real worker, the multipart completion handler writes the object metadata and changes the asset state in one application-level transition before publishing the job. A retry can then use the same asset_id and transformation version. The original remains immutable, while a new version writes a new derivative key.

Do not trust the filename or the browser's Content-Type as the final decision. OWASP recommends allowlisting extensions, validating the declared type against the file signature, limiting size, generating storage names, and keeping uploaded files outside the webroot or behind an application-controlled download path. Those checks belong before the worker spends CPU decoding pixels.

Sharp can be a good fit for a Node.js thumbnail worker, but it does not solve tenant authorization. The storage adapter, queue consumer, and download handler still need the same ownership contract. My Python test above is useful precisely because it can run before the production image library is selected.

What fails first in a multipart thumbnail pipeline?

The most dangerous failures are quiet ones.

An upload can be marked complete while the application record still points at a client-supplied key. A queue retry can overwrite a thumbnail from a different transformation. A cleanup task can delete a source while a delayed derivative job is reading it. A download endpoint can validate asset_id but forget to validate tenant_id. None of these failures needs an exotic storage feature; they come from splitting ownership across too many uncoordinated records.

I test the boundaries with a small matrix: wrong tenant, pending state, missing derivative, repeated job, expired link, malformed image, oversized image, and an abandoned multipart session. Each case should have a named outcome. For example, a wrong-tenant request should produce the same externally visible response as a missing asset if the application does not want to reveal that the asset exists.

Observability needs the same identifiers as the job payload. Log tenant ID, asset ID, transform version, upload state, and request correlation ID. Avoid logging private URLs or image bytes. Track rejected uploads, completed uploads, derivative failures, link issuance, and authorization denials separately; one combined “upload success” counter hides too much.

For an AI feature, add downstream checks. Store the thumbnail dimensions and byte size, then compare retrieval accuracy, OCR fidelity, or caption agreement by transformation version. Token cost is part of the experiment: sending a needlessly large derivative into a vision model can change both latency and spend, while an over-aggressive resize can erase the text the model needs. I don't know which metric will win for every media corpus; your mileage may vary, so use representative tenant data and a fixed evaluation set.

When is this private storage pattern the wrong choice?

The pattern is not suitable when the product needs permanent public image URLs, anonymous hotlinking, or a public CDN origin. Use a delivery design intended for public content then. It is also a poor fit when the storage layer must provide object lock, versioned recovery, or strict cross-region replication as a hard requirement; those controls need to be verified in the selected service and may change the architecture.

Stick with a direct provider integration when its native identity controls, lifecycle tooling, and operational support are central to the team. A generic S3-compatible interface is not a substitute for checking multipart semantics, private-link behavior, retention controls, and audit requirements. Conversely, an adapter is valuable when the application needs a single HTTP contract across storage backends and wants the rest of the Python or Node.js code to remain unaware of provider-specific request details.

That is the trade-off I would put in the design review. Portability reduces coupling; it can also hide the controls that make isolation provable. Test the actual boundary you care about.

What should the eval harness prove before production?

Use two tenants, two originals, and two derivative sizes. Complete one multipart upload, enqueue one job, and assert that the source and derivative keys contain the same tenant prefix. Then replay the job, request both links with the wrong tenant, expire one link, and interrupt one upload before completion. The expected result is deterministic: no cross-tenant read, no derivative from a pending asset, no long-lived credential in a download URL, and no mutation of the original.

Measure completion-to-derivative latency, rejected upload rate, abandoned-session cleanup, authorization-denial rate, derivative byte size, and the quality metric used by the AI feature. Those measurements tell you whether the pipeline is merely moving bytes or producing safe, useful inputs.

The practical decision is simple: select the storage implementation last. First make tenant ownership, state transitions, private delivery, and transformation versions observable. Then test the implementation against that contract.

References

Top comments (0)