Short answer: make LLM structured extraction retries idempotent by giving each moderation report one stable identity, claiming it before inference, and guarding the final database upsert with that same key. A retry may repeat computation; it must not create a second report.
The deciding constraint isn't model quality. It's structured output correctness across crash boundaries. A customer-support webhook can be delivered twice, a worker can lose its lease after the model responds, and malformed JSON can trigger another attempt. If those events share no durable identity, a perfectly valid classifier can still produce duplicate records for one human reviewer.
That distinction matters. Attempts are disposable; report identity is permanent.
What should a Node.js webhook worker do when LLM JSON extraction retries create duplicate records?
Treat the pipeline as three separately retryable steps: accept, classify, and commit. The ingress handler authenticates and stores the raw report under an idempotency key. A worker claims that stored input and asks for a schema-shaped classification. The commit step validates the value and writes it under the original key. Each boundary gets a uniqueness constraint; none relies on an in-memory flag.
For a moderation report, a useful business identity is usually an immutable event ID issued by the upstream system. If no such ID exists, derive one from stable fields such as tenant, source report ID, and payload version. Don't hash the whole request blindly — timestamps, signature headers, or field ordering can change while the underlying report remains the same. Also don't use the queue message ID as business identity. Redelivery systems are allowed to wrap the same business event in a new transport message.
The three fences have different jobs:
| Fence | Durable key | Duplicate behavior |
|---|---|---|
| Ingress | tenant + source report ID + payload version | Return the already accepted job |
| Worker claim | report key + classifier version | Reuse or resume the same classification job |
| Result commit | report key + schema version | Update one row, never append another |
The classifier version belongs in the claim because changing a prompt or model contract may justify recomputation. The result key has a schema version for the same reason. Keep those versions explicit. Config hidden in a dozen environment variables makes incident review miserable.
One wrinkle: a report can legitimately be re-opened after an agent adds evidence. That is a new input version, not a retry. If the upstream event doesn't distinguish those cases, the pipeline cannot infer intent with certainty. I'm not sure any generic deduplication rule can fix an ambiguous source contract; the clean resolution is an immutable event ID plus a monotonic revision supplied at ingress.
The smallest working implementation
Start with one function that computes a stable key and one repository operation that claims it atomically. The repository is deliberately generic here; its claim method must map to a database insert protected by a unique constraint, not a read followed by a write.
import { createHash } from "node:crypto";
type ModerationReport = {
tenantId: string;
sourceReportId: string;
revision: number;
text: string;
};
type Classification = {
category: "abuse" | "spam" | "self_harm" | "other";
confidence: number;
needsHumanReview: boolean;
};
type Claim = {
reportKey: string;
status: "new" | "running" | "complete";
result?: Classification;
};
interface ReportStore {
claim(reportKey: string, input: ModerationReport): Promise<Claim>;
commit(reportKey: string, value: Classification): Promise<void>;
reject(reportKey: string, reason: string): Promise<void>;
}
function reportKey(report: ModerationReport): string {
const identity = [
report.tenantId,
report.sourceReportId,
String(report.revision),
"moderation-schema-v3",
].join(":");
return createHash("sha256").update(identity).digest("hex");
}
Then keep the model boundary narrow. The parser checks the exact keys, allowed category values, Boolean type, and confidence range. A JSON parse success is not a schema success.
function parseClassification(raw: string): Classification {
const value: unknown = JSON.parse(raw);
if (typeof value !== "object" || value === null) {
throw new Error("classification must be an object");
}
const row = value as Record<string, unknown>;
const categories = new Set(["abuse", "spam", "self_harm", "other"]);
const exactKeys = ["category", "confidence", "needsHumanReview"];
if (
Object.keys(row).sort().join(",") !== exactKeys.sort().join(",") ||
typeof row.category !== "string" ||
!categories.has(row.category) ||
typeof row.confidence !== "number" ||
row.confidence < 0 ||
row.confidence > 1 ||
typeof row.needsHumanReview !== "boolean"
) {
throw new Error("classification failed schema validation");
}
return row as Classification;
}
type ExtractJson = (text: string) => Promise<string>;
async function classifyOnce(
report: ModerationReport,
store: ReportStore,
extractJson: ExtractJson,
): Promise<Classification | undefined> {
const key = reportKey(report);
const claim = await store.claim(key, report);
if (claim.status === "complete") return claim.result;
if (claim.status === "running") return undefined;
try {
const raw = await extractJson(report.text);
const result = parseClassification(raw);
await store.commit(key, result);
return result;
} catch (error) {
const reason = error instanceof Error ? error.message : "unknown failure";
await store.reject(key, reason);
throw error;
}
}
There is a deliberate gap between extractJson and any vendor SDK. Good. The rest of the service shouldn't care which model produced the candidate JSON, and swapping a client library shouldn't rewrite deduplication logic. Keep the adapter thin enough to test with a function.
The nasty crash is after the model returns but before commit finishes. The job will run again, so the model call may repeat. Yet the unique report key still prevents two committed moderation rows. If avoiding repeated inference is required, persist the raw model response before validation under the same key; that adds storage and lifecycle rules, so it isn't automatically the better default.
Retry policy is part of the data model
Retries need a taxonomy. JSON syntax failure and schema rejection are extraction failures. A closed database connection is a commit failure. Duplicate delivery isn't a failure at all; it is a lookup of existing work. Collapsing all four into throw plus a queue retry counter hides the state that operators need.
Use bounded attempts for classification, record every attempt against the stable report key, and send exhausted work to human review. Never generate a fresh report key on retry. Preserve the original text, classifier version, schema version, validation reason, and attempt number so an engineer can explain why a report reached the manual queue without storing a second canonical classification.
Prompt injection is relevant in this customer-support setting because report text is untrusted input. It can contain instructions that try to alter the task or output shape. Separate instructions from report data, constrain output to the moderation schema, validate after generation, and treat the result as untrusted until it passes policy checks. OWASP's LLM application guidance is useful here, while prompt examples alone are not a security boundary.
Fast retries can make things worse. If twenty workers receive the same webhook and all perform the LLM call before claiming the key, the database eventually deduplicates the rows but compute is still multiplied. Claim first. Then measure claim conflicts, validation failures, attempt counts, time from ingress to human review, and the age of the oldest unfinished report. I benchmark those boundaries because a low average latency can hide a stuck tail.
What I would change at scale
At higher volume, move the claim into the same transactional database as the canonical result and publish queue work through a transactional outbox. That closes the gap where an ingress record commits but process termination happens before queue publication. Consumers still remain idempotent because outbox delivery can repeat.
Partitioning should follow tenant and report identity, not random attempt IDs. This keeps competing deliveries for one report near the same ownership boundary, although the unique constraint remains the authority. Add lease expiry for abandoned running claims, but make lease takeover conditional on the stored lease token; an old worker must not commit after a new worker owns the claim.
Keep the dashboard boring: accepted, running, validation-rejected, ready for review, and terminal. Five states beat fifteen half-documented flags.
Trade-offs and the decision rule
This pattern favors correctness and auditability over the fewest database writes. The catch is that it needs durable coordination before inference, explicit versioning, and cleanup for stored attempts. It is not suitable when every input is intentionally independent, duplicate outputs are harmless, and the extra claim round trip costs more than recomputation. In that narrow case, a stateless worker may be enough.
Stick with a database-backed unique key when duplicate records would show the same moderation report twice to a reviewer, trigger two downstream actions, or corrupt audit counts. Use a queue's deduplication feature as an optimization, not the only fence, because transport identity and business identity solve different problems. For teams already running a relational database, a unique constraint plus an upsert is usually the smallest coordination surface. A dedicated workflow engine can make sense when classifications span long timers, approvals, and compensating actions, but its operational model is more machinery than a short extraction job needs.
The final test is blunt: replay the identical webhook concurrently, terminate a worker after extraction, redeliver the queue item, and submit a schema-invalid response. The system passes only if one canonical report exists, every attempt remains inspectable, and a human sees at most one review item. Don't accept a green happy-path test as evidence of idempotency.
Top comments (0)