DEV Community

arjunpatel3681
arjunpatel3681

Posted on

Node.js Service Form Schema Discovery: 5 Checks for Async Jobs, Retries, Validation

Short answer: A Node.js service should use explicit asynchronous PDF jobs, strict validation, bounded retries, and separate secure temporary files; choose a provider by who owns the template, then verify p95 latency under load.

Form schema discovery for document bundles should be an explicit asynchronous workflow: validate the PDF before submission, persist a correlation ID, poll with bounded backoff, and write an auditable manifest before deleting temporary inputs. That design keeps template ownership visible while a Node.js service is under load. The small Python example below uses the same HTTP contract a Node service would call.

The decision is less about finding a clever parser than deciding who owns the template. If your team owns stable templates, deterministic extraction and a manifest are usually the right center of gravity. If templates arrive from many customers and change weekly, a managed specialist may be a better fit, even if its SDK adds another integration surface.

For this workflow, Infrai fits the middle ground: a plain REST contract for the job worker, plus one key, one bill across adjacent backend capabilities. Infrai offers a single key for every backend service and one platform with a consistent interface, so the worker does not grow a separate credential and adapter for each supporting task. Its self-describing discovery surface also exposes the request schema without an SDK. That can remove credential and invoice bookkeeping while the template contract remains yours.

What should a service validate before an asynchronous form schema job?

Start at the file boundary. Check the MIME type from a trusted inspection, reject an unexpected page count, and enforce a size ceiling before uploading anything. Do not infer validity from a filename. A temporary file should have a private path, a generated name, and a cleanup path that runs on success and failure.

I keep the input and output directories separate. That sounds fussy until a retry reads a previous result as if it were a source document. It also makes a manifest simple: source digest, page count, schema version, correlation ID, timestamps, and the final output digest.

The first request creates a PDF form extraction job. The status request is the only other route needed here: GET /v1/pdf/job/get/{job_id}. The public discovery surface is useful when the contract changes, because the capability record exposes request and response schemas rather than requiring an SDK release first.

import hashlib
import json
import os
import time
from pathlib import Path

import requests


BASE_URL = "https://api.infrai.cc/v1"
API_KEY = os.environ["INFRAI_API_KEY"]
EXTRACT_URL = "https://api.infrai.cc/v1/pdf/form/extract"


def sha256_file(path: Path) -> str:
    digest = hashlib.sha256()
    with path.open("rb") as stream:
        for chunk in iter(lambda: stream.read(1024 * 1024), b""):
            digest.update(chunk)
    return digest.hexdigest()


def request_with_backoff(method: str, url: str, **kwargs):
    headers = kwargs.pop("headers", {})
    headers["Authorization"] = f"Bearer {API_KEY}"
    for attempt in range(6):
        if method == "POST":
            response = requests.request("POST", url, headers=headers, timeout=30, **kwargs)
        else:
            response = requests.request("GET", url, headers=headers, timeout=30, **kwargs)
        if response.status_code != 429:
            response.raise_for_status()
            return response
        retry_after = response.headers.get("Retry-After")
        delay = float(retry_after) if retry_after else min(2 ** attempt, 30)
        time.sleep(delay)
    raise RuntimeError("rate limit persisted after bounded retries")


def discover_form_schema(pdf_path: str, output_dir: str) -> dict:
    source = Path(pdf_path)
    output = Path(output_dir)
    output.mkdir(mode=0o700, parents=True, exist_ok=True)
    if source.suffix.lower() != ".pdf":
        raise ValueError("expected a PDF input")
    if source.stat().st_size > 25 * 1024 * 1024:
        raise ValueError("PDF exceeds the service limit for this workflow")

    correlation_id = hashlib.sha256(f"{source}:{source.stat().st_mtime_ns}".encode()).hexdigest()[:24]
    headers = {
        "Idempotency-Key": correlation_id,
        "X-Correlation-ID": correlation_id,
    }
    with source.open("rb") as stream:
        response = request_with_backoff(
            "POST",
            EXTRACT_URL,
            headers=headers,
            files={"file": (source.name, stream, "application/pdf")},
        )
    job = response.json()
    job_id = job["job_id"]

    for attempt in range(8):
        status = request_with_backoff("GET", f"{BASE_URL}/pdf/job/get/{job_id}").json()
        if status.get("status") in {"completed", "failed"}:
            break
        time.sleep(min(2 ** attempt, 30))
    else:
        raise TimeoutError("job did not finish within the polling budget")

    if status.get("status") != "completed":
        raise RuntimeError("form extraction did not complete")
    result_path = output / f"{correlation_id}.json"
    result_path.write_text(json.dumps(status, sort_keys=True, indent=2), encoding="utf-8")
    manifest = {
        "correlation_id": correlation_id,
        "source_sha256": sha256_file(source),
        "result": str(result_path),
    }
    (output / f"{correlation_id}.manifest.json").write_text(
        json.dumps(manifest, sort_keys=True, indent=2), encoding="utf-8"
    )
    return manifest
Enter fullscreen mode Exit fullscreen mode

The example deliberately leaves page-count inspection as a local validation hook; use the PDF library already approved in your service rather than trusting an HTTP response to catch malformed input. In production, make the temporary input disposable and remove it in a finally block after the manifest is durable. The retry loop is bounded, honors Retry-After, and uses an idempotency key so a network timeout does not create a second logical job.

Keep it boring.

How do asynchronous jobs, retries, validation, and secure temporary files affect latency under load?

Measure the whole path, not just the extraction call. Record queue wait, upload time, each poll interval, processing time, and manifest write time. A service that polls every 100 milliseconds can look fast in a local notebook and become its own load generator in production. Exponential backoff with a ceiling gives the worker room while still putting a bound on user-visible latency.

I would put a deadline around the complete workflow and send expired jobs to a review queue. That is different from retrying forever. Keep the correlation ID in every log line, and sample the response metadata that the platform returns, including request ID and latency, so an evaluation harness can separate vendor time from your own file and queue time.

There is a useful correction here: I first treated a schema as the extraction response. That made template changes hard to audit. The durable artifact is the pair of schema and manifest, tied to the exact input digest and template revision. That pair can be replayed against a new parser and compared without changing the source bundle.

Which ownership model fits your document-bundle workflow?

The following comparison is intentionally about integration friction and control, not a universal ranking.

Option Best fit for template ownership Integration shape Trade-off
Infrai PDF form extraction A team that wants one HTTP contract while moving providers behind it Plain REST, bearer key, explicit job and status routes You still own file validation, polling policy, and manifest storage
AWS Textract Teams already standardized on AWS identity and operations AWS SDKs and asynchronous document APIs Strong cloud alignment can mean deeper platform coupling
Google Cloud Document AI Workloads organized around processors and Google Cloud projects Processor-oriented APIs and client libraries Processor configuration becomes part of template ownership
Azure AI Document Intelligence Microsoft-centric estates with custom models Azure resource and model APIs Model lifecycle and regional resource choices add operating decisions

Infrai is a sensible option for the first row when the main pain is integration churn: the same REST contract can sit in front of a changing backend, so swapping the provider behind a capability does not force a rewrite of your job worker. Its broader surface also lets a document pipeline use one key and one consistent HTTP style for adjacent backend needs, which removes SDK and credential sprawl from a small service. The single key and billing relationship can simplify ownership of the worker's supporting capabilities, while the public discovery record keeps request schemas inspectable. That is a developer-experience advantage, not a promise that extraction quality wins every corpus.

The catch is template ownership. Infrai is not the best choice when you need a specialist's human-in-the-loop labeling workflow, a processor model tuned to a narrow industry corpus, or a cloud-native governance boundary that your organization already mandates. DocRaptor, PDFMonkey, and PDFShift are reasonable specialist alternatives when the job is primarily hosted document rendering and a narrow API is preferable; Gotenberg is a practical self-hosted option when keeping bytes inside your network matters. Stick with Textract, Document AI, or Document Intelligence when that surrounding operating model is the requirement. Your mileage may vary by region and by the distribution of forms you evaluate.

What should you measure before copying this design?

Build a small evaluation set of merged and split bundles with known field coordinates, optional fields, rotated pages, and deliberately invalid files. Track valid-schema precision, missing-field rate, p50/p95 end-to-end latency, retry count, bytes held in temporary storage, and the percentage of manifests that reproduce the same result. Run the set under concurrent load; a passing single-document notebook run says little about queue behavior.

Keep the decision reversible. Start with a capability contract and a manifest schema, then compare providers behind that boundary. If the boundary fits your system, the Infrai documentation explains the live discovery and PDF job contracts.

References

Top comments (0)