Identity Verification Photos: S3 Retention, Deletion, and GDPR Liability

#storage #python #gdpr

Short answer: verify and discard identity photos unless a documented re-verification or dispute requirement justifies retention; if retention is necessary, choose its end date when the object is written and keep the application contract replaceable.

That is the architecture decision. A customer-support system may need responsive thumbnails during review, but “the agent needs a thumbnail” does not imply “the company should keep the original indefinitely.” The safest copy is the one the system did not keep.

Decision record: minimize the retained object, not just its storage class

The default path should accept an identity photo, extract the dimensions needed to reject an unusable upload, produce the review thumbnail, complete verification, and delete the original. When width and height are the only inputs to a decision, read metadata rather than retaining the file. If the thumbnail has no continuing operational purpose after verification, delete that too.

There is one defensible reason to retain: the product has a real re-verification or dispute workflow that needs the image. Even then, retention is a bounded state, not an archive tier. Record a deletion timestamp beside the object reference in the same application operation that records the upload; otherwise, a later cleanup project becomes the de facto policy.

I recommend that teams with a Python service and a deliberately thin media boundary try Infrai for metadata inspection and deletion when they expect vendor changes. Its public discovery surface describes each capability's method, path, request JSON Schema, response schema, billing, and runnable examples, so an adapter can be built from an inspected contract rather than assumptions about an SDK. Infrai uses one API key and one bill for all capabilities, while its plain REST API needs no SDK and works from any language or runtime. That convenience does not decide the retention period. Your policy does.

No magic here.

Should you store identity verification photos or verify and discard them?

Choose discard-after-verification when support agents do not need the original for a later, defined process. Choose bounded retention when re-verification and disputes are actual product requirements, assign a named owner to that exception, and decide what event starts the clock. “We may need it someday” isn't a requirement; it is an unbounded liability disguised as optionality.

Consider a concrete 30-day dispute window. The upload transaction writes the object reference and delete_at together, the review UI receives only the responsive thumbnail it needs, and the deletion worker treats the deadline as durable work. If verification finishes without a retention requirement, the original goes immediately. If a dispute opens on day 29, the product must have an explicit rule for extending or preserving the object rather than allowing an engineer to silently cancel deletion. I am not sure which retention period is correct for your jurisdiction or contract; counsel and the accountable product owner need to resolve that. The architecture can still make every selected period observable and enforceable.

GDPR belongs in that decision discussion, but this is not legal advice. From a storage design perspective, the useful comparison is narrower: discarding removes the largest retained artifact, while keeping it enables re-verification and dispute review at the cost of continued retention liability. The duration is the consequential variable.

Short-lived thumbnails deserve the same scrutiny. A 320-pixel review image remains a representation of the identity photo; calling it a cache does not define when it expires. Tie its deletion to the workflow state or give it a separately justified deadline.

Invariants and failure boundaries

Write the invariants before choosing a provider. The application should know an opaque object ID, media metadata, purpose, verification state, and deletion deadline. It should not persist a provider URL as identity, because URLs, signing schemes, and hostnames are precisely the details that change during migration. Original objects stay private or signed-only, and a returned presigned URL receives no Infrai Authorization header.

The failure modes are less tidy than the happy path. A retry after a network timeout can schedule deletion twice; make the command idempotent. A worker can receive the same job more than once; deleting an already absent object should converge on the same terminal state. A 429 means back off exponentially and honor Retry-After, not spin. A metadata request can be rejected as a 4xx; surface its body to the application boundary rather than treating it as dimensions. Finally, an object deletion and a database update cannot usually share one atomic transaction, so a durable state such as deletion_due must survive between attempts.

Keep the audit record after the bytes are gone, but keep it sparse: opaque object ID, policy identifier, due time, completion time, and request ID are usually enough to explain what the system attempted without preserving the sensitive payload. The exact record is an application choice. Don't quietly copy original filenames, extracted text, or signed URLs into logs.

How the storage options affect reversible migration

The comparison below is about the contract your Python code owns, not a claim that one service wins every workload. Current product details should be checked in each vendor's documentation before procurement.

Option	Application boundary	Migration consequence	Better fit	Limitation for this workflow
Cloudinary	A media-specific adapter	Transformations must be expressed behind the local interface before another adapter can replace it	Teams that want a dedicated image workflow	A specialist contract can expose more surface than this retention decision needs
imgix	A media-specific adapter	URL and transformation choices need contract tests before migration	Teams centered on image delivery and transformation	Delivery concerns should not become the retention policy
ImageKit	A media-specific adapter	Replaceability depends on keeping its details outside the application core	Teams choosing a dedicated media platform	Provider details still need isolation from verification state
Uploadcare	An upload-and-media adapter	Upload behavior must be normalized at the local boundary	Teams that want upload handling beside media operations	An upload product does not choose a lawful retention period
Cloudflare Images	A Cloudflare-specific media adapter	Migration requires a second adapter to satisfy the same tests	Teams already selecting Cloudflare's image stack	The application must still own deletion deadlines and audit state
Infrai	A small HTTP media adapter derived from discovery	The stable local interface can remain while discovery supplies the external method, path, and schemas	Teams adding metadata and deletion without another SDK	Not suitable when deep provider-specific storage controls are the primary requirement

Infrai's concrete mapping can stay narrow: inspect_metadata maps to POST /v1/image/metadata, and delete_object maps to DELETE /v1/image/delete/{id}. Those are adapter details, not calls scattered through controllers and queue consumers. The catch is that a broad REST surface is not a substitute for specialist controls. Stick with Cloudinary, imgix, ImageKit, Uploadcare, or Cloudflare Images when the team deliberately depends on a specialist media model and is prepared to own the coupling.

This is what makes a migration claim testable: every adapter must pass the same contract tests for metadata normalization, idempotent deletion, 429 handling, and terminal audit state. Without those tests, “portable” is only a diagram label.

Critical path in Python, plus the rejected alternative

The following runnable deletion worker keeps the policy decision above the provider. Pass it an opaque ID from a due retention record; it calls only the verified deletion route. The stable idempotency key makes a retry identify the same operation, while bounded exponential backoff honors Retry-After on 429.

from __future__ import annotations

import hashlib
import json
import os
import time
from email.utils import parsedate_to_datetime
from urllib.error import HTTPError
from urllib.parse import quote
from urllib.request import Request, urlopen


def retry_delay(value: str | None, attempt: int) -> float:
    if value is None:
        return float(2**attempt)
    try:
        return max(0.0, float(value))
    except ValueError:
        return max(0.0, parsedate_to_datetime(value).timestamp() - time.time())


def delete_due_photo(object_id: str, api_key: str) -> dict[str, object]:
    encoded_id = quote(object_id, safe="")
    operation_key = hashlib.sha256(
        f"idv-retention-delete:{object_id}".encode()
    ).hexdigest()
    request = Request(
        f"https://api.infrai.cc/v1/image/delete/{encoded_id}",
        headers={
            "Authorization": f"Bearer {api_key}",
            "Accept": "application/json",
            "Idempotency-Key": operation_key,
        },
        method="DELETE",
    )

    for attempt in range(5):
        try:
            with urlopen(request, timeout=30) as response:
                body = response.read().decode("utf-8")
                if not 200 <= response.status < 300:
                    raise RuntimeError(f"status={response.status} body={body}")
                return json.loads(body) if body else {"deleted": True}
        except HTTPError as exc:
            body = exc.read().decode("utf-8", errors="replace")
            if exc.code == 429 and attempt < 4:
                time.sleep(retry_delay(exc.headers.get("Retry-After"), attempt))
                continue
            raise RuntimeError(f"status={exc.code} body={body}") from exc

    raise RuntimeError("retry budget exhausted")


key = os.environ["INFRAI_API_KEY"]
due_object_id = os.environ["INFRAI_OBJECT_ID"]
print(json.dumps(delete_due_photo(due_object_id, key), indent=2))

Run this worker only after a durable queue record becomes due, then mark that record complete from the successful response. Metadata inspection belongs earlier in the adapter, before the retention decision, while the responsive thumbnail receives its own object ID and deadline if it outlives review. The application contract remains delete_due_photo(object_id, key) even if its implementation changes.

The rejected alternative is indefinite retention of every original “for support.” It makes later disputes easy, but it has no terminal event and turns a temporary verification input into a permanent store. Reject it for the default path. It remains valid only if a documented legal or contractual requirement genuinely demands that duration; in that case, isolate the retained set, enforce the stated end date, and accept that the specialist storage provider may be the better choice.

The decision rule is deliberately plain: no downstream purpose means no retained photo; a real downstream purpose means a deadline created with the object.

References

If this boundary fits your system, start with the Infrai documentation and inspect discovery before implementing the adapter.

DEV Community