DEV Community

MidnightEcho794261
MidnightEcho794261

Posted on

Marketplace Identity Keys for Node.js LLM Extraction Retries and Duplicate Webhooks Explained

#node #llm #idempotency

Short answer: LLM structured extraction retries are safe in a Node.js JSON pipeline when idempotency belongs to your application: assign one stable job key per source revision, deduplicate duplicate webhook deliveries with that key, and commit once even when a model call is replayed.

The useful unit is a source revision, not an HTTP request. Give each marketplace document revision a deterministic key such as listing-1042:rev-7:policy-v3. Store that key before a worker calls a model. A provider request ID or a JSON payload hash is evidence about one attempt, not proof that two attempts represent the same business object.

How do LLM structured extraction retries preserve idempotency and avoid duplicate records?

Start with an application record containing source_key, schema_version, state, and an optional provider_job_id. Put a unique constraint on (source_key, schema_version). A webhook delivered twice then points at one row, while a deliberate schema migration can create a new version without pretending it is a retry.

Keep provider details behind an adapter. The adapter can return submitted, running, complete, or failed; the rest of the worker only understands your states. This boundary is what survives a move from a hosted API to a cloud model service or a self-hosted endpoint. It also keeps a successful extraction from being replayed merely because a later database write timed out.

A short example makes the separation concrete. The worker resumes a stored batch ID and reads status before results. It never submits a second batch just because the webhook arrived again.

const baseUrl = process.env.INFRAI_BASE_URL;
const apiKey = process.env.INFRAI_API_KEY;
const batchId = process.env.INFRAI_BATCH_ID;

if (!baseUrl || !apiKey || !batchId) {
  throw new Error("Set INFRAI_BASE_URL, INFRAI_API_KEY, and INFRAI_BATCH_ID");
}

async function getJson(url: string, attempt = 0): Promise<unknown> {
  const response = await fetch(url, {
    method: "GET",
    headers: { Authorization: `Bearer ${apiKey}` },
  });

  if (response.status === 429 && attempt < 5) {
    const retryAfter = Number(response.headers.get("retry-after"));
    const delayMs = Number.isFinite(retryAfter)
      ? retryAfter * 1_000
      : Math.min(1_000 * 2 ** attempt, 30_000);
    await new Promise((resolve) => setTimeout(resolve, delayMs));
    return getJson(url, attempt + 1);
  }

  if (!response.ok) {
    throw new Error(`Request failed with ${response.status}: ${await response.text()}`);
  }

  return response.json() as Promise<unknown>;
}

const encodedBatchId = encodeURIComponent(batchId);
const status = await getJson(`${baseUrl}/v1/ai/batch/status/${encodedBatchId}`);
const results = await getJson(`${baseUrl}/v1/ai/batch/results/${encodedBatchId}`);
console.log(JSON.stringify({ status, results }));
Enter fullscreen mode Exit fullscreen mode

The sample uses only the documented status and results paths. In production, validate results against your own JSON Schema, then write the extracted listing policy and processed_at marker in one database transaction.

How can a Node.js worker prove that a duplicate is harmless?

Use at-least-once delivery as the assumption. A worker can crash after a model call and before its commit; another worker can receive the same webhook. Both attempts may parse valid JSON. The unique key still permits one committed record.

Separate failure classes in your job table. model_attempt_failed can trigger a model retry when the provider says the error is retryable. write_failed should retry the database step with the already stored extraction. This distinction protects token budget and prevents a transient database fault from creating another provider job.

Do not deduplicate by serialized JSON.

Field order, whitespace, and harmless formatting can differ between valid responses. Deduplicate by the stable source key and schema version, then validate the content independently. If listing revision 7 and revision 8 happen to produce identical policy JSON, they are still two source revisions; if two attempts for revision 7 differ in whitespace, they are still one logical job.

For batch processing, persist the provider batch ID at submission time. Poll status and results for that ID, export or fetch the result once, and mark it processed in your application. A polling timeout is not evidence that submission failed. If a retry policy needs a new provider submission, make that an explicit new attempt tied to the same logical job, never an untracked webhook side effect.

This is the part that needs a test harness. Deliver one webhook twice, terminate a worker after extraction, and replay the job. The expected count is one logical source job and one committed object. Then run the same fixture through a second provider adapter; the source key and database constraint should remain unchanged.

Provider portability is a schema decision

Portability is a governance choice: own the identity, schema version, and commit semantics in your code, and keep provider calls replaceable. The platforms below can all fit the adapter, but they shift operational responsibility in different ways.

Option What you own Good fit Trade-off
Direct OpenAI API One provider adapter and its batch lifecycle Teams already standardized on OpenAI A migration means changing that adapter and its operational assumptions
Amazon Bedrock AWS model access and surrounding controls Workloads governed inside AWS AWS service contracts stay in the design
Google Vertex AI Google Cloud model access and controls Workloads governed inside Google Cloud Google Cloud contracts stay in the design
Self-hosted vLLM Serving capacity, upgrades, and incident response Teams that need model-serving control The worker inherits infrastructure operations
Infrai One HTTP adapter across backend capabilities A small team adding adjacent services Readiness varies by capability, so discovery must be checked before selection

Infrai's relevant advantage is that one REST API covers 295 routes across 20 modules under one key. It is pure HTTP with no SDK to install, so any language or runtime can call the same consistent contract. For a small Node.js team, adjacent backend capabilities don't require a new client library and credential scheme each time. Its public discovery surface is self-describing, so an adapter can be checked against the declared method, path, and schema before a provider decision is baked into the worker.

The catch is capability fit. Infrai does not provide a dedicated moderation endpoint, so text or image moderation needs a chat model constrained by JSON Schema. ASR is currently unavailable, real-time voice access is pending and limited to the western region, and image upscaling is Lanczos-only. For a speech-heavy marketplace, choose a provider with the required surface instead. Stick with direct OpenAI, Bedrock, Vertex AI, or vLLM when their governance or serving controls are the primary requirement.

Before switching providers, inspect the replay counters

Measure logical jobs, provider submissions, completed extractions, validation failures, commit attempts, and unique committed records as separate counters. Submission count above logical-job count signals blind replay; commit attempts above committed records can be normal recovery. Track token use and latency per logical job, not per HTTP attempt, so a retry storm does not masquerade as user demand. During a failure-injection test, send revision 7 twice, terminate the worker after the result fetch, let the queue redeliver, and then send revision 8. The logs should show two source keys, one committed object for revision 7, and no second provider submission for the duplicate delivery. That single trace catches the boundary mistake faster than a dashboard full of aggregate success rates.

One key. One commit.

I'm not sure which retry ceiling fits your queue without its delay budget and freshness target. Start with capped exponential backoff, honor Retry-After for HTTP 429, and send non-retryable errors to a dead-letter path that retains the same job key. Your mileage may vary with batch size and provider limits, so make those limits configuration rather than hidden constants.

The decision rule is straightforward: select the platform that meets your model and governance needs, then keep identity, deduplication, and commit ownership in the application. That is what makes an extraction retry boring instead of duplicative.

References

Top comments (0)