Short answer: treat historical post and comment moderation as a resumable data pipeline, not a giant API request; snapshot each revision, classify bounded batches, commit idempotently, and export only after the counts reconcile.
I build CLIs and SDKs for other developers, so my first benchmark is time-to-first-verified-row. The model call is rarely the hard part. The hard part is proving what happened after a worker was killed, a response was malformed, or a reviewer opened an export while it was still being written.
Keep the enforcement decision separate from the backfill. A historical run should produce evidence first. A person or a later policy can decide what to do with that evidence.
What should a Node.js bulk job do before it classifies posts and comments?
Freeze the input boundary. Record a job ID, policy version, classifier version, and a stable ordering key. For each source item, store its content ID, revision, and body (or a pointer to a permitted immutable snapshot). A cursor over stable IDs is safer than an offset because new rows can otherwise shift the meaning of “page 12.”
The result identity needs the revision, not only the content ID. A comment edited during a run is a different classification target. A unique constraint such as (job_id, content_id, content_revision) makes a replay converge on one logical result. The cursor update and result insert belong in one transaction, so a crash cannot advance progress without durable evidence.
I learned to check the database, not the progress counter. In one test run, 37,200 records produced successful HTTP responses, but a transaction helper returned before its commit branch. The dashboard looked green; the result table was empty. The fix was small: count committed rows and read one row back after every batch. That check catches an entire class of “the API worked” illusions.
The longer version of that failure is worth spelling out. A worker fetched page A, sent its payload, received a valid-looking response, and incremented an in-memory counter. A process restart then fetched page A again because the cursor had never committed. A later patch changed the query filter, so an offset-based retry skipped several edited comments while duplicating others. Nothing in the HTTP access log could reveal the mistake: every request had a 2xx status, and every JSON body parsed. Only a comparison between the immutable input snapshot, the unique result key, and the exported line count exposed the gap. That is why the job record, cursor, result insert, and post-commit probe are one design problem, not four optional observability features.
Three words: snapshot, classify, reconcile.
A small TypeScript runner that can be restarted safely
Start with a narrow interface. The classifier can later point at an HTTP service, a local process, or a test double without making storage depend on a provider response shape.
type SourceItem = {
id: string;
revision: string;
body: string;
};
type Verdict = {
label: "allow" | "review" | "remove";
reasons: string[];
};
type StoredResult = SourceItem & Verdict & {
jobId: string;
policyVersion: string;
classifierVersion: string;
};
type Classifier = (items: readonly SourceItem[]) => Promise<readonly Verdict[]>;
interface Store {
readPage(afterId: string | null, limit: number): Promise<readonly SourceItem[]>;
commitBatch(results: readonly StoredResult[], nextCursor: string): Promise<void>;
findResult(jobId: string, itemId: string, revision: string): Promise<StoredResult | null>;
}
export async function runModerationJob(
store: Store,
classify: Classifier,
jobId: string,
policyVersion: string,
classifierVersion: string,
): Promise<number> {
let cursor: string | null = null;
let committed = 0;
for (;;) {
const items = await store.readPage(cursor, 100);
if (items.length === 0) return committed;
const verdicts = await classify(items);
if (verdicts.length !== items.length) {
throw new Error("Classifier returned the wrong result count");
}
const results = items.map((item, index): StoredResult => ({
...item,
...verdicts[index],
jobId,
policyVersion,
classifierVersion,
}));
const nextCursor = items[items.length - 1].id;
await store.commitBatch(results, nextCursor);
const probe = results[Math.floor(results.length / 2)];
const saved = await store.findResult(jobId, probe.id, probe.revision);
if (!saved || saved.label !== probe.label) {
throw new Error(`Commit verification failed for ${probe.id}`);
}
cursor = nextCursor;
committed += results.length;
}
}
The production adapter should match by echoed item ID, not trust array position. It should reject unknown labels, missing reasons, duplicate IDs, and responses that contain an item from another revision before storage begins. I am not sure every external classifier preserves ordering under every batching mode; an explicit identity check removes that uncertainty.
Retry semantics need the same discipline. RFC 9110 describes which HTTP methods are idempotent, but an idempotent HTTP request does not automatically make your database side effect idempotent. Use an attempt key, bounded retries, and exponential backoff for transient transport failures. Leave the cursor unchanged when a batch is rejected.
Test the policy and the failure modes before the full archive
Prompt text is executable policy. Version it beside the allowed labels and reason codes, then evaluate it on a fixed sample before spending the full run. Include obvious allows, obvious removals, ambiguous review cases, quoted abuse, sarcasm, code snippets, empty text, and truncated text. A polished prompt is not evidence that your community's language is covered.
Measure errors by label and reason. False removals deserve a different response from missed spam. Keep a review outcome so uncertainty reaches a person instead of being hidden inside a forced binary answer. Free-form explanations can be retained for context, but controlled reason codes are what make a report filterable.
| Failure mode | Detection | Recovery |
|---|---|---|
| Missing or extra verdict | Compare input IDs and revisions | Reject the batch; leave the cursor unchanged |
| Unknown label or reason | Validate the policy contract | Quarantine the response for inspection |
| Transport timeout | Record an attempt and retry with a cap | Replay the same idempotency key |
| Edited source content | Compare the stored revision with the current revision | Queue the new revision as separate work |
| Export interrupted | Reconcile rows, lines, and label totals | Delete the temporary artifact and regenerate |
Inject failures around every boundary: before classification, after the response, during commit, and immediately after commit. Restart the process. The invariant is simple: every intended content revision has zero or one accepted result, never two, and no cursor skips an input. Fast calls are nice. Predictable replays are better.
The Prompt Engineering Guide is useful for thinking about instruction structure, but acceptance criteria must come from a labeled sample and a review process, not from prompt style alone.
Export results as a reviewable audit artifact
Generate JSONL for programs and CSV for reviewers from the same stored rows. Include content ID, revision, label, controlled reasons, job ID, policy version, classifier version, and completion time. Copy the original text only when the review workflow permits it; moderation data can be sensitive, and every extra copy expands access risk.
type ExportRow = {
contentId: string;
contentRevision: string;
label: "allow" | "review" | "remove";
reasons: readonly string[];
jobId: string;
policyVersion: string;
classifierVersion: string;
completedAt: string;
};
export async function writeJsonLines(
rows: AsyncIterable<ExportRow>,
output: NodeJS.WritableStream,
): Promise<void> {
for await (const row of rows) {
const line = JSON.stringify(row) + "\n";
if (!output.write(line)) {
await new Promise<void>((resolve) => output.once("drain", resolve));
}
}
}
Reconcile before publishing: stored completed-row count, exported line count, and label totals must agree. Write to a temporary filename, close the stream, verify the manifest, then rename it to the public export. A hash identifies exactly which artifact a reviewer received. Raw provider payloads do not belong in the main CSV; they make the file harder to inspect and couple downstream tools to an unstable response format.
Do not present an uncalibrated confidence score as probability. If you retain a score, document its meaning and use it to order human review. Your mileage may vary because content mix changes the error distribution.
The trade-off when volume grows
One process is enough when a measured run meets the deadline. At larger volume, partition by stable ID ranges and put bounded work units on a queue with leases. Workers may duplicate delivery after a lease expires, which is why the result key, transaction, and export contract must stay unchanged. Add a global concurrency limit and observe latency, retry rate, invalid-result rate, and reviewer disagreement.
The catch is operational weight. A queue brings lease expiry, shutdown behavior, dashboards, and more credentials. It is not suitable for instant enforcement, legal adjudication, or automatic deletion without an appeals path. Use a synchronous path for new content that needs an immediate answer, deterministic rules for exact matches, and trained reviewers for high-impact ambiguity.
Choose the larger architecture only when the benchmark shows a real throughput gap or one machine cannot hold the required network concurrency reliably. Cost belongs in the report, but it should not choose an architecture whose results cannot be reconstructed.
The job is done when a reviewer can trace every exported verdict to an immutable revision, a policy version, and a classifier version, and when restarting the worker produces the same durable set.
References
- RFC 9110: HTTP Semantics — https://www.rfc-editor.org/rfc/rfc9110
- Prompt Engineering Guide — https://www.promptingguide.ai
Top comments (0)