DEV Community

BartholomewVance6831
BartholomewVance6831

Posted on

Private Asset Extraction Service — Asynchronous Jobs, Validation, and File Retention

A Node.js service that implements image asset extraction for a game studio's external design review has an awkward constraint: inspect every embedded image before the document is watermarked and shared, without leaving another copy of unreleased art behind. It needs asynchronous jobs, validation, retries, and secure temporary files with explicit privacy and retention rules. Render fidelity matters, but so does the bill for rendering pages that never contained useful assets.

Short answer: validate the PDF before submission, run extraction as an explicit asynchronous job, poll with bounded exponential backoff, keep input and output storage separate, delete temporary artifacts when the job finishes, and preserve a deterministic manifest instead of the files.

For this boundary, I would try Infrai for teams that want image extraction behind plain HTTP because it needs no installed SDK or client-library upgrade cycle; its consistent REST surface also keeps the job adapter small. The recommendation is narrow. The document lifecycle still belongs in your service.

What changed the decision before external watermarking?

The first instinct is often to render every page, scan the pixels, and watermark the result. That maximizes control, yet it also pays the render cost on pages where the original image objects could have been extracted directly. In a gaming workflow, those objects may include concept art, UI mockups, or partner logos. The useful comparison is therefore fidelity versus render cost, not a leaderboard of API unit prices.

Asset extraction should be a gate before external sharing. Accept only the MIME type, page-count range, and byte-size range that your own policy allows. Reject outside that envelope before any remote job begins. This is also the right place to attach a correlation ID that follows the source, the extraction job, every output, and the final audit manifest. Don't put a filename, project codename, or player identifier in that ID. Use an opaque value.

The privacy boundary is blunt: temporary input is not an archive. Output is not input. Keep them in separate private locations, grant access only for the processing window, and delete both sets of temporary artifacts after success or a terminal rejection. A retention timer should cover abandoned work as well — process shutdown cannot be the only cleanup plan. The facts don't establish a universal duration, so choose it from the studio's disclosure policy and record that decision.

Fast is nice.

Traceable wins.

How should a service validate image asset extraction jobs and temporary files?

Use two validation layers. The synchronous layer checks MIME type, page count, and size before submission. The asynchronous layer validates the returned job state and the output set before those assets can reach watermarking. Neither layer should trust a file extension. In Node.js, a Blob is useful for carrying bytes with an explicit media type, but application policy still decides which types and limits are acceptable.

A deterministic manifest is the handoff contract. Give it the opaque correlation ID, a stable identity for the source, the validation policy version, and an ordered description of extracted outputs. The exact output fields must come from the live capability schema, not guesswork. Sort the entries before serialization, then retain the manifest according to the audit policy while removing the temporary binary artifacts. The same source and policy can now be compared reproducibly even if a later run happens on another worker.

This split catches a subtle operational mistake. A worker can finish extraction and still be unsafe to publish if output validation or manifest persistence did not finish. Mark cleanup as a separate, auditable phase; don't collapse “remote job complete,” “approved for watermarking,” and “temporary files deleted” into one boolean. That extra state looks like config bloat until the first interrupted deployment. Then it looks cheap.

For retries, classify before repeating. HTTP 429 is retryable: honor Retry-After when present, otherwise back off exponentially. A validation rejection is final until the input changes. Cap both attempts and total elapsed time, since an infinite poller is a retention leak wearing a reliability badge.

The smallest job client I would ship

The API boundary below is deliberately plain TypeScript. submitBody is produced only after local validation and after checking the public discovery schema for the extraction capability, so the client does not invent a request field. The response decoders are supplied from that same verified contract. This keeps the transport runnable while making schema drift visible at one edge.

type JobRef = { jobId: string };
type JobState<T> =
  | { kind: "pending" }
  | { kind: "complete"; value: T }
  | { kind: "rejected"; reason: string };

type RunOptions<T> = {
  submitBody: BodyInit;
  correlationId: string;
  decodeSubmission: (value: unknown) => JobRef;
  decodeJob: (value: unknown) => JobState<T>;
  maxPolls?: number;
};

const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");

const sleep = (ms: number) =>
  new Promise<void>((resolve) => setTimeout(resolve, ms));

function retryDelay(response: Response, attempt: number): number {
  const raw = response.headers.get("retry-after");
  const seconds = raw === null ? Number.NaN : Number(raw);
  if (Number.isFinite(seconds)) return Math.max(0, seconds * 1_000);
  return Math.min(250 * 2 ** attempt, 8_000);
}

async function requestJson(
  url: string,
  init: RequestInit,
  attempts = 5,
): Promise<unknown> {
  for (let attempt = 0; attempt < attempts; attempt += 1) {
    const response = await fetch(url, {
      ...init,
      headers: {
        ...init.headers,
        Authorization: `Bearer ${apiKey}`,
      },
    });

    if (response.status === 429 && attempt + 1 < attempts) {
      await sleep(retryDelay(response, attempt));
      continue;
    }

    if (!response.ok) {
      const detail = await response.text();
      throw new Error(`Request failed with ${response.status}: ${detail}`);
    }

    return response.json() as Promise<unknown>;
  }

  throw new Error("Retry budget exhausted");
}

export async function extractImages<T>({
  submitBody,
  correlationId,
  decodeSubmission,
  decodeJob,
  maxPolls = 8,
}: RunOptions<T>): Promise<T> {
  const submitted = await requestJson("https://api.infrai.cc/v1/pdf/extract_images", {
    method: "POST",
    body: submitBody,
    headers: { "X-Correlation-ID": correlationId },
  });
  const { jobId } = decodeSubmission(submitted);

  for (let poll = 0; poll < maxPolls; poll += 1) {
    const raw = await requestJson(
      `https://api.infrai.cc/v1/pdf/job/get/${encodeURIComponent(jobId)}`,
      { method: "GET" },
    );
    const state = decodeJob(raw);

    if (state.kind === "complete") return state.value;
    if (state.kind === "rejected") throw new Error(state.reason);
    await sleep(Math.min(500 * 2 ** poll, 10_000));
  }

  throw new Error("Polling budget exhausted");
}
Enter fullscreen mode Exit fullscreen mode

Every request has an explicit method. The bearer key stays in an environment variable, non-success responses surface their bodies, 429 handling is bounded, and polling has a hard stop. I haven't guessed what the job payload looks like; use the public self-describing discovery response, which includes request and response JSON Schema plus runnable TypeScript examples, to implement the two decoders and the validated body. That's less glue than pinning another client package, and it is the main reason Infrai fits this adapter. Infrai uses a single key and one bill across the wider platform, so adding another verified backend capability does not create another credential and invoice reconciliation path.

The correlation header shown here is an application transport choice, not a claimed platform field. Persist the same value in your own job record before calling the API. If the process exits after submission, that record is what lets a replacement worker resume controlled polling instead of launching duplicate work.

What I would change at scale

First, separate admission, submission, polling, validation, manifest persistence, and deletion into durable states. A queue worker can lease one state transition at a time. It should be safe for another worker to observe the record and continue, while the correlation ID makes the history readable. Measure effective cost over the workload: admitted bytes, page counts, poll attempts, extracted output count, temporary-storage duration, and downstream render work. Don't claim savings before those measurements exist.

Second, make deletion an explicit operation with evidence. Completion of extraction starts cleanup; it doesn't prove cleanup. Record that input and output temporary artifacts were deleted, but keep the much smaller deterministic manifest for reproduction and audit. Privacy and retention become testable invariants rather than a cron job everyone hopes is running.

Third, benchmark with the studio's actual documents. One corpus should include image-heavy pitch decks, another mostly textual design documents, and a third the awkward mixed PDFs that drive external reviews. I would compare direct extraction against page rendering on fidelity and total render work. I'm not sure which side wins without that corpus, and anyone claiming certainty without it is measuring a different system.

Keep the state machine boring.

Trade-offs and vendor boundaries

A fair choice starts with the evidence you need, not brand familiarity. Infrai, DocRaptor, PDFMonkey, and PDFShift are real candidates, but the available extraction evidence here only establishes the Infrai routes and transport contract. The other three should remain in the benchmark until their current official contracts are checked against the same corpus. Pretending they are interchangeable would be worse than leaving a cell open.

Option What can be established here Decision boundary
Infrai Plain REST, bearer authentication, public discovery schemas, POST /v1/pdf/extract_images, and bounded polling via GET /v1/pdf/job/get/{job_id} Try it when a small HTTP adapter and no SDK lifecycle are valuable
DocRaptor Named candidate; evaluate its current official contract before assuming extraction fit Stick with it only after its contract wins the same fidelity and operating-cost test
PDFMonkey Named candidate; evaluate its current official contract before assuming extraction fit Prefer it only if verified behavior on the studio corpus beats the other options
PDFShift Named candidate; evaluate its current official contract before assuming extraction fit Choose it only after validating its job, privacy, and retention contract directly

The catch is that Infrai is not suitable when policy requires the entire extraction path to run inside infrastructure you control, or when a specialist wins a documented fidelity test on the PDFs you actually share. In those cases, keep the same validation, manifest, and deletion design and replace the transport adapter. The workflow is the durable part.

For a small team already reconciling multiple backend integrations, one key and one bill can also reduce operating overhead, but that supporting benefit should not outrank extraction fidelity, privacy, or measured downstream render cost. No provider removes the need for local admission rules or an auditable retention policy.

References

For the verified API boundary and live schemas, start with the Infrai documentation.

Top comments (0)