DEV Community

SullivanReed1247
SullivanReed1247

Posted on

FastAPI Receipt Architecture: Private File Upload with Signed Object Downloads

Short answer: keep receipt originals in private object storage, let FastAPI issue narrow signed upload and download grants, and move bytes directly between the browser and the correct US or EU storage region.

The deciding constraint is large-file throughput. FastAPI should authorize a transfer, record its state, and verify its completion; it shouldn't relay a 600 MB scan through an application worker. The file is evidence, so an HTTP success from storage is only one event in a longer chain: upload, verification, malware screening, receipt processing, retention, and deletion all need distinct states.

This is an architecture decision record for a media application that processes receipts while retaining each original for audit. It is deliberately narrower than a generic object-storage roundup. The hard part isn't generating a URL. It's preserving authorization, regional placement, and audit meaning while the data plane bypasses the app.

How should a beginner design private signed browser uploads for US and EU users?

Split the system into a control plane and a data plane. The browser asks the FastAPI control plane for permission to upload one intended receipt. After authenticating the app user, the control plane chooses the storage region from trusted tenant policy, generates an opaque object key, creates a pending database row, and returns a short-lived signed grant. The browser then sends the bytes directly to private object storage.

Don't let the browser choose us or eu in a request field. Region is an authorization decision, not presentation data. A user's current IP address is a poor substitute for the tenant's documented data-handling policy, and moving an account later is a migration rather than a different upload parameter. GDPR obligations also aren't reduced to a region toggle; retention, deletion, access control, and evidence of those actions remain application responsibilities.

After transfer, the browser asks the control plane to finalize the pending record. The service checks the stored object's exact key, size, expected media type, and supplied integrity value before queueing any parser. Upload metadata is evidence, not proof: OWASP recommends validating extension, type, signature, filename, and size as defense in depth, with files stored outside the webroot and scanned when appropriate. A receipt stays unavailable for download until the checks and malware-screening stage have passed.

That last transition matters.

I've learned from OTP delivery gaps that “accepted” and “completed” are different states. File ingestion deserves the same suspicion — a grant can be issued without a transfer, a transfer can finish without finalization, and a clean original can exist while receipt extraction is still pending. One uploaded=true flag erases those distinctions just when support or compliance needs them.

Invariants and failure boundaries

The first invariant is that the bucket or container remains private. A signed URL is a temporary bearer credential for one operation and one generated key; it is not a reason to relax the underlying storage policy. Keep signing credentials only in the backend secret store, rotate them under an explicit procedure, and never send a storage credential to browser code. The OWASP secrets guidance is useful here because it treats access, rotation, revocation, and auditing as lifecycle concerns rather than a .env formatting exercise.

No public fallback.

The second invariant is ownership binding. Every receipt row binds an opaque object key to a tenant, an app user, a region, a declared size, a lifecycle state, and retention metadata. Download authorization queries that row first. It does not accept an arbitrary object key from the caller and sign it. This prevents a valid user from changing receipt_id into a storage-key oracle.

Use explicit states such as pending_upload, verifying, screening, ready, rejected, and deleted. State transitions should be idempotent. If two finalize requests race, one transition wins and the other reads the resulting state; neither schedules two processors. A 409 Conflict is reasonable when the supplied receipt identifier belongs to a different state transition, while malformed declarations can fail with 400 before any grant exists.

Failure boundaries should also be boring and visible:

  • An expired upload grant creates no authority to mint another one automatically. The authenticated client asks again, and policy is reevaluated.
  • An abandoned pending_upload row is expired by a scheduled reconciliation job; an unreferenced object is investigated and removed under the same retention policy.
  • A size or content mismatch moves the record to rejected. The object never becomes downloadable to the end user.
  • A processor retries from a durable receipt identifier, not from a signed download URL that may expire while waiting in a queue.
  • Deletion records the policy decision and outcome without logging a reusable signed URL or sensitive object metadata.

Logs need identifiers, state changes, actor IDs, region, and policy decisions. They don't need bearer grants. This is the same deliverability instinct I apply to message flows: enough telemetry to explain the gap, never the secret that can reproduce the action.

I'm not sure a universal signed-URL lifetime exists, because the correct value depends on measured file sizes, client uplink distributions, and multipart behavior. Resolve that uncertainty with transfer telemetry by size band, then choose the shortest lifetime that covers a defensible percentile plus retry overhead. Guessing five minutes because it looks secure can punish mobile users; choosing a day because it avoids support tickets expands the credential's exposure window.

Compare the transfer paths before choosing one

There are three credible shapes. The table is a decision aid, not a vendor ranking.

Transfer path Application bandwidth Large-file behavior Security and operational cost Best fit
App-proxied upload Bytes cross the app tier Worker, proxy, and request limits become part of the transfer Central inspection is straightforward, but scaling and timeout ownership grow Small, tightly bounded files or networks that cannot reach storage
Direct single-request upload Bytes bypass the app tier Simple, but a failed long transfer may restart from zero Narrow signing and post-upload verification are mandatory Moderate files on reliable connections
Direct multipart upload Bytes bypass the app tier Parts can be retried independently and uploaded with controlled concurrency More client state, abort cleanup, part limits, and completion verification Large receipts or variable uplinks where throughput is the primary axis

For this media workload, direct multipart upload is the recorded decision once real file sizes justify its complexity. Cap concurrency rather than opening every part at once, because browser memory, local networks, and storage throttling are finite. Persist the multipart session against the pending receipt, authorize every part against the same generated key, and abort abandoned sessions through a reconciliation process. Start with direct single-request upload if measurements show that files are modest and retries are rare. That's not an architectural failure; it is a smaller implementation with the same control-plane boundary. Preserve the receipt state machine and opaque-key scheme so multipart can replace the transfer mechanism without rewriting authorization. The important choice is the bypassed data plane, not multipart for its own sake: a team should be able to show that part-level retries or parallel transfer solve an observed problem before accepting another client state machine, another cleanup path, and another set of completion tests.

There is a compliance edge case hiding in performance tests. Synthetic load must exercise both regional policies, but test fixtures should not be copied production receipts. Measure grant latency, time to first byte, total transfer time by size band, part retry count, finalize lag, screening backlog, and deletion lag. Alert on stuck states rather than only request error rates. A quiet API can coexist with thousands of never-finalized uploads.

The critical FastAPI control path

The following Python sketch keeps storage details behind a generic adapter. It omits login implementation and database plumbing, but it shows the boundary that matters: trusted policy selects the region; the server creates the key; and downloads are signed only for a ready receipt owned by the current tenant. The adapter's head result must come from storage, not from browser claims.

from dataclasses import dataclass
from datetime import datetime, timedelta, timezone
from enum import Enum
from uuid import UUID, uuid4

from fastapi import Depends, FastAPI, HTTPException
from pydantic import BaseModel, Field

app = FastAPI()


class ReceiptState(str, Enum):
    PENDING_UPLOAD = "pending_upload"
    VERIFYING = "verifying"
    READY = "ready"
    REJECTED = "rejected"


class UploadRequest(BaseModel):
    filename: str
    content_type: str
    size_bytes: int = Field(gt=0)
    checksum: str


class FinalizeRequest(BaseModel):
    checksum: str


@dataclass(frozen=True)
class Actor:
    user_id: UUID
    tenant_id: UUID


def current_actor() -> Actor:
    # Replace with the application's authenticated-session dependency.
    raise NotImplementedError


@app.post("/receipts/upload-grants")
def create_upload_grant(
    body: UploadRequest,
    actor: Actor = Depends(current_actor),
) -> dict:
    policy = tenant_policies.get(actor.tenant_id)
    validate_declaration(body.filename, body.content_type, body.size_bytes)

    receipt_id = uuid4()
    object_key = f"receipts/{actor.tenant_id}/{receipt_id}/original"
    receipt = receipts.insert_pending(
        receipt_id=receipt_id,
        tenant_id=actor.tenant_id,
        user_id=actor.user_id,
        region=policy.storage_region,
        object_key=object_key,
        declared_size=body.size_bytes,
        declared_checksum=body.checksum,
    )
    grant = storage.for_region(receipt.region).sign_multipart_upload(
        key=receipt.object_key,
        content_type=body.content_type,
        max_bytes=body.size_bytes,
        expires_at=datetime.now(timezone.utc) + timedelta(minutes=15),
    )
    return {"receipt_id": str(receipt.id), "upload": grant}


@app.post("/receipts/{receipt_id}/finalize")
def finalize_upload(
    receipt_id: UUID,
    body: FinalizeRequest,
    actor: Actor = Depends(current_actor),
) -> dict:
    receipt = receipts.get_for_update(receipt_id, actor.tenant_id)
    require_state(receipt, ReceiptState.PENDING_UPLOAD)

    stored = storage.for_region(receipt.region).head(receipt.object_key)
    if stored.size != receipt.declared_size or stored.checksum != body.checksum:
        receipts.transition(receipt.id, ReceiptState.REJECTED)
        raise HTTPException(status_code=400, detail="Stored object failed verification")

    receipts.transition(receipt.id, ReceiptState.VERIFYING)
    screening_queue.enqueue(receipt.id)
    return {"receipt_id": str(receipt.id), "state": "verifying"}


@app.post("/receipts/{receipt_id}/download-grants")
def create_download_grant(
    receipt_id: UUID,
    actor: Actor = Depends(current_actor),
) -> dict:
    receipt = receipts.get(receipt_id, actor.tenant_id)
    require_state(receipt, ReceiptState.READY)

    grant = storage.for_region(receipt.region).sign_download(
        key=receipt.object_key,
        expires_at=datetime.now(timezone.utc) + timedelta(minutes=5),
        download_name="receipt-original",
    )
    audit.record("receipt.download_granted", actor.user_id, receipt.id)
    return {"download": grant}
Enter fullscreen mode Exit fullscreen mode

Those two expiry values are examples to replace with measured policy, not universal recommendations. Likewise, checksum semantics vary by storage implementation and multipart construction; define the adapter contract precisely and test it against files with repeated bytes, zero-byte rejection, interrupted parts, swapped checksums, and filenames containing multiple extensions. Content type supplied by the browser is only a declaration. The screening worker must identify the actual file type and apply the allowlist before transitioning to ready.

Deployment needs a second, less glamorous path: reconciliation. Run it with bounded batches per region. It should find expired pending rows, abandoned multipart sessions, objects without database ownership, and records stuck beyond each stage's service objective. Treat reconciliation actions as audited state transitions, and make deletion retries idempotent.

Rejected option and the case where it wins

The rejected option is routing every original through FastAPI. It is not suitable for this workload when large-file throughput dominates, because each byte consumes application ingress and egress capacity while proxy limits, worker lifetimes, and deploy drains join the transfer's failure surface. A browser receiving 413 Request Entity Too Large from the app proxy has learned nothing about whether storage itself could accept the object.

Still, app-proxied upload has a valid use case. Stick with it when files are small and strictly capped, clients cannot reach the storage endpoint, or policy requires inline inspection before any byte enters the object store. The catch is that the app tier must then own backpressure, streaming rather than buffering, request limits, timeout alignment, temporary-file handling, and capacity tests. That can be the right trade. It just conflicts with the primary axis here.

Direct multipart transfer has its own limitations. It adds client logic, incomplete-upload cleanup, and a more involved integrity contract. Teams without evidence of large or failure-prone transfers should resist paying that complexity tax early. Begin with the control-plane/data-plane split, instrument actual transfers, and upgrade the data path when the numbers justify it.

The final decision rule is plain: keep authorization and lifecycle state in FastAPI, keep receipt bytes in private regional storage, and choose single-request or multipart direct upload from observed size and retry distributions. Auditability comes from explicit ownership and state transitions, not from forcing the application server to touch every byte.

References

Top comments (0)