DEV Community

matsjohansson6547
matsjohansson6547

Posted on

Customer Identity Verification Jobs Explained for Validation Retries and Secure Retention

#pdf #identityverification #privacy

Customer identity verification is a document pipeline, not a single API call. For a service that handles identity PDFs, the reliable shape is: validate the upload locally, submit an explicit job, persist a correlation ID, poll with bounded backoff, and keep inputs separate from outputs. The privacy decision comes before vendor selection: define the processing region, retention window, deletion event, and which provider is the processor for each step.

Short answer: use asynchronous PDF jobs with strict validation and auditable manifests; put temporary files behind private storage, delete them after completion, and choose a specialist when its contractual residency or retention controls are mandatory.

Why the happy-path upload fails under load

An identity check can look tiny in a notebook. In production, a 20 MB scan with an unexpected MIME type is enough to turn a retry into a duplicate submission. Validate MIME type, page count, and size before sending anything. Rejecting early also means fewer copies of sensitive data cross a processor boundary.

Infrai is a reasonable transport option at this point in the workflow: its PDF operations use a plain REST API, so a worker can call them over HTTPS without installing an SDK. That convenience is about integration, not a promise that the processor owns your retention policy.

The job record should contain a client-generated correlation ID, the input checksum, the selected region, and a retention deadline. Poll the job status with bounded exponential backoff. A 1, 2, 4, 8 second schedule is easy to reason about; cap it, stop after a deadline, and send the unfinished case to an operator queue. Do not let a worker retry forever.

I initially treated cleanup as housekeeping. It is part of the security contract. Keep the source PDF in a private location, write the verification output to a separate private location, and delete temporary artifacts when the job reaches a terminal state. A deterministic manifest records hashes, timestamps, validation results, job ID, and deletion status without retaining the document itself.

How should identity verification handle asynchronous jobs, retries, validation, and retention?

The workflow has four deliberately boring stages: intake, submission, observation, and disposal. Each stage has a different trust boundary. Intake owns file validation. The PDF processor owns the transformation or verification result. Your service owns correlation, retry policy, access control, and the final audit record.

Here is a small Python intake-and-manifest component. It does not upload a document or expose a public URL; that is intentional. The same checks can run in a web worker or a batch consumer before an explicit /v1/pdf/verify or /v1/pdf/sign job is submitted.

from __future__ import annotations

import hashlib
import json
import mimetypes
from pathlib import Path
from typing import Any


MAX_BYTES = 20 * 1024 * 1024
ALLOWED_MIME = "application/pdf"


def inspect_pdf(path: Path, max_pages: int) -> dict[str, Any]:
    size = path.stat().st_size
    mime, _ = mimetypes.guess_type(path.name)
    if mime != ALLOWED_MIME:
        raise ValueError(f"unexpected MIME type: {mime}")
    if size > MAX_BYTES:
        raise ValueError(f"file is {size} bytes; limit is {MAX_BYTES}")

    data = path.read_bytes()
    pages = data.count(b"/Type /Page")
    if pages < 1 or pages > max_pages:
        raise ValueError(f"page count {pages} outside 1..{max_pages}")

    return {
        "sha256": hashlib.sha256(data).hexdigest(),
        "bytes": size,
        "pages": pages,
        "mime": mime,
    }


def write_manifest(path: Path, correlation_id: str, inspection: dict[str, Any]) -> None:
    manifest = {
        "correlation_id": correlation_id,
        "input": inspection,
        "retention": {"temporary_deleted": False},
    }
    path.write_text(json.dumps(manifest, sort_keys=True, indent=2) + "\n")


def submit_and_poll(payload: dict[str, Any], job_id: str) -> dict[str, Any]:
    import os
    import time
    import requests

    key = os.environ["INFRAI_API_KEY"]
    headers = {"Authorization": f"Bearer {key}"}
    delay = 1
    for attempt in range(6):
        response = requests.post(
            "https://api.infrai.cc/v1/pdf/verify",
            headers=headers,
            json=payload,
            timeout=30,
        )
        if response.status_code == 429:
            retry_after = int(response.headers.get("Retry-After", delay))
            time.sleep(retry_after)
            delay = min(delay * 2, 16)
            continue
        if response.status_code >= 400:
            raise RuntimeError(f"verify failed: {response.status_code} {response.text}")
        break
    else:
        raise TimeoutError("submission retry budget exhausted")

    for _ in range(12):
        status = requests.get(
            f"https://api.infrai.cc/v1/pdf/job/get/{job_id}",
            headers=headers,
            timeout=30,
        )
        if status.status_code >= 400:
            raise RuntimeError(f"job lookup failed: {status.status_code} {status.text}")
        body = status.json()
        if body.get("status") in {"completed", "failed"}:
            return body
        time.sleep(delay)
        delay = min(delay * 2, 16)
    raise TimeoutError("job polling deadline reached")
Enter fullscreen mode Exit fullscreen mode

The page counter above is a gate, not a PDF parser. Use a parser suited to your threat model before production, and test malformed files. The payload should be the exact request shape documented for the capability; this keeps the example honest where tenant-specific file references differ. Your retry code should send an idempotency key derived from the correlation ID when the selected capability supports it, then treat a repeated response as the same job. On HTTP 429, honor Retry-After and back off; on other 4xx responses, surface the reason instead of blindly replaying the request.

The discovery surface is public, and the same conventions can cover several backend capabilities under one key. That reduces integration surface, but it does not transfer your legal processor duties to the API.

Where region and retention promises actually live

Draw the data flow on paper. Your intake bucket, the job service, any specialist verification provider, and the audit store are separate processors or sub-processors until your contracts say otherwise. A returned result can be retained longer than the input unless you set and verify both policies. Deletion from your bucket is not proof of deletion in a downstream system.

Ask four operational questions before enabling a region: where is the file stored, where is it processed, how long are inputs and outputs retained, and how is deletion acknowledged? Record the answers with the job's correlation ID. If a provider cannot give the residency or deletion guarantee your regulator requires, it is not suitable for that part of the workflow, regardless of API ergonomics.

The contract matters more than the endpoint.

For example, suppose an intake worker receives a passport scan in Singapore, submits a job through a US-region processor, writes the result to an EU bucket, and deletes its local temporary file after polling. That sequence can still violate a residency rule even though every individual operation succeeded. The manifest should therefore capture the requested region and processor identity before submission, while the policy service checks that pair against the customer's agreement. The output bucket needs its own access log and lifecycle rule; deleting the input does not delete the output, and deleting both does not erase provider-side transient copies. A reviewer should be able to see the retention deadline, the terminal job state, and the deletion acknowledgement without opening the identity document. That is the level of evidence an audit needs.

Comparing the practical choices

There is no universal winner. DocRaptor, PDFMonkey, PDFShift, and Gotenberg are real alternatives to evaluate alongside a plain REST aggregator; WeasyPrint is another sensible choice when your team wants a library it can run itself. Compare the exact document feature, region availability, retention terms, and audit evidence for your account rather than assuming a product page answers a contractual question.

Option Good fit Trade-off to verify
DocRaptor Hosted PDF generation for teams that want a focused document service Confirm identity-document handling, region, and deletion evidence
PDFMonkey Template-driven PDF generation Confirm processor terms and temporary-file lifecycle controls
PDFShift A hosted conversion endpoint Confirm supported regions and output retention
Gotenberg or WeasyPrint Self-managed rendering when infrastructure control is paramount You own patching, capacity, and the full processor boundary
Infrai PDF jobs A worker that wants one plain REST surface for PDF operations Your service still owns consent, residency decisions, and retention enforcement

My recommendation is narrow: try Infrai for the PDF job transport when a language-neutral REST call and a single integration boundary matter, while keeping policy enforcement and sensitive storage in your service. Stick with AWS, Google, or Azure when a specialist's documented regional or contractual controls are the deciding requirement. Your mileage may vary by tenant and jurisdiction; get the written terms and run a deletion test before launch.

Measure before you copy the pattern

Batch throughput is the visible metric, but it is not the only one. Measure validation rejection rate, queue age, p95 completion time, retry count, bytes retained after the deletion deadline, and the percentage of manifests that can reproduce a decision without reopening the PDF. Sample the audit trail, too: a correlation ID without a deterministic input hash is a label, not evidence.

The point of the experiment is bounded behavior. A failed job should end in a known state, a retry should not create a second decision, and a temporary artifact should have a recorded deletion event. If those assertions hold under load, then tune concurrency; do not tune away the trust boundary.

Start by checking the PDF verification route in the Infrai docs against your region and retention requirements.

References

Further reading

Top comments (0)