Existing Posts and Comments: Node.js Batch Moderation with Auditable LLM Results

#contentmoderation #llmclassification #auditability

Short answer: snapshot each post or comment revision, classify it under a versioned policy with bounded concurrency, commit one durable result for that input, and export only after input, decision, and exception counts reconcile.

A bulk moderation run is an accounting problem with an uncertain function in the middle. The classifier may return probabilistic labels, but the surrounding Node.js system still has to establish which bytes were evaluated, which policy interpreted the response, and why a later retry did not create a second enforcement action. A large Promise.all answers none of those questions.

The least complex implementation that preserves those properties is a manifest plus a resumable worker. The Node.js application owns a stable extract of existing content; the worker owns request pacing and durable attempts; a separate policy step turns classification output into allow, review, or restrict; and the exporter reads committed decisions rather than live application rows. This separation is deliberate. It permits a policy threshold to change without quietly changing the historical model response.

What should a Node.js bulk job export after LLM classification?

Export an audit record, not just a label. Each row should identify the content object and immutable revision, the input hash, policy version, prompt-template hash, model identifier returned by the API, normalized classification output, derived enforcement decision, attempt lineage, and completion time. A decision ID derived from (content_id, revision, policy_version) gives downstream consumers a stable deduplication key.

The text hash matters because an existing comment can be edited while the job is running. Without a snapshot or immutable revision, comment_id=42 names an object but does not prove what the classifier saw. The export should normally omit raw text unless a defined review or audit purpose requires it; hashes, identifiers, and decision evidence reduce exposure, though they do not replace a retention policy. Applicable privacy, employment, consumer-protection, and sector rules vary by jurisdiction, so counsel and the data owner must set access and deletion periods. The architecture can preserve evidence. It can't decide the legal basis for keeping it.

A useful row has three conceptual layers:

Layer	Representative fields	Reason
Input identity	content ID, revision, text SHA-256	Proves which immutable unit was evaluated
Classification evidence	model ID, policy and prompt versions, labels	Preserves the basis for later evaluation
Operational record	decision ID, attempt, timestamps, export manifest ID	Supports retry analysis and downstream deduplication

Keep the raw classifier response separately when its schema contains evidence needed for review, but validate and normalize it before enforcement. A provider changing field order must not alter a policy decision, and an unexpected label must go to review rather than fall through to allow. It is tempting to combine inference and enforcement in one handler because the first prototype is shorter. Don't. A reversible classification record and an account-affecting action have different audit and authorization requirements.

Treat delivery as at-least-once and commitment as unique

HTTP does not make a classification request exactly once. RFC 9110 defines safe and idempotent method semantics, while a typical inference call uses POST, whose retry safety cannot be assumed. A client can lose the connection after the remote service accepted a request but before the response arrived; a retry may therefore repeat computation. If the API explicitly supports an idempotency key, send the stable decision ID. Independently, enforce uniqueness in local storage, because remote request deduplication and local decision commitment solve different problems.

The state machine can remain small: pending -> leased -> classified -> exported, with failed_review as a terminal exception that requires an operator. A lease needs an owner and an expiry. The commit of classified needs a unique constraint on content ID, revision, and policy version. Only after that transaction succeeds should the queue item be acknowledged. Two workers may perform the same call after a lease race, but they must not create two authoritative rows or two moderation actions.

Exactly once is the accounting invariant, not the transport claim.

Retry policy must be derived from the API contract. A timeout or connection reset is ambiguous; 429 may be retryable when the service specifies pacing information; a malformed request is not repaired by exponential backoff. Use capped backoff with jitter, a per-attempt deadline, and a maximum attempt count, then preserve the final exception with its decision ID. I'm not sure a universal retry count exists: payload size, service quotas, latency objectives, and operator coverage determine it. A canary run resolves that uncertainty better than folklore.

Reconciliation closes the loop. For a manifest containing N immutable revisions, require N = committed + terminal_exception + pending, and require pending to reach zero before declaring completion. Also reconcile by content type and source partition, because a correct grand total can hide a missing comments partition offset by duplicated posts. This is the same discipline used around a ledger migration: totals are necessary, but control totals expose where the drift occurred.

Implement a bounded, auditable classifier client

The following Go client is intentionally only the request boundary. The existing Node.js service can write an NDJSON manifest or enqueue stable work IDs; language choice does not change the contract. The classifier URL is injected configuration, avoiding an invented vendor route, and the code records hashes and identifiers needed by the repository's conditional insert.

package moderation

import (
    "bytes"
    "context"
    "crypto/sha256"
    "encoding/hex"
    "encoding/json"
    "fmt"
    "io"
    "net/http"
    "time"
)

type Work struct {
    ContentID string `json:"content_id"`
    Revision  int64  `json:"revision"`
    Policy    string `json:"policy_version"`
    Prompt    string `json:"prompt_version"`
    Text      string `json:"text"`
}

type Result struct {
    DecisionID string             `json:"decision_id"`
    ContentID  string             `json:"content_id"`
    Revision   int64              `json:"revision"`
    TextSHA256 string             `json:"text_sha256"`
    Model      string             `json:"model"`
    Labels     map[string]float64 `json:"labels"`
    FinishedAt time.Time          `json:"finished_at"`
}

type Client struct {
    HTTP *http.Client
    URL  string
}

func stableID(w Work) string {
    sum := sha256.Sum256([]byte(fmt.Sprintf("%s\x00%d\x00%s", w.ContentID, w.Revision, w.Policy)))
    return hex.EncodeToString(sum[:])
}

func (c Client) Classify(ctx context.Context, w Work) (Result, error) {
    payload := struct {
        Policy string `json:"policy_version"`
        Prompt string `json:"prompt_version"`
        Text   string `json:"text"`
    }{w.Policy, w.Prompt, w.Text}

    body, err := json.Marshal(payload)
    if err != nil {
        return Result{}, fmt.Errorf("encode request: %w", err)
    }
    req, err := http.NewRequestWithContext(ctx, http.MethodPost, c.URL, bytes.NewReader(body))
    if err != nil {
        return Result{}, fmt.Errorf("build request: %w", err)
    }
    req.Header.Set("Content-Type", "application/json")
    req.Header.Set("Idempotency-Key", stableID(w))

    resp, err := c.HTTP.Do(req)
    if err != nil {
        return Result{}, fmt.Errorf("classification request: %w", err)
    }
    defer resp.Body.Close()
    if resp.StatusCode < 200 || resp.StatusCode >= 300 {
        return Result{}, fmt.Errorf("classification status %d", resp.StatusCode)
    }

    var decoded struct {
        Model  string             `json:"model"`
        Labels map[string]float64 `json:"labels"`
    }
    if err := json.NewDecoder(io.LimitReader(resp.Body, 1<<20)).Decode(&decoded); err != nil {
        return Result{}, fmt.Errorf("decode response: %w", err)
    }
    textHash := sha256.Sum256([]byte(w.Text))
    return Result{
        DecisionID: stableID(w), ContentID: w.ContentID, Revision: w.Revision,
        TextSHA256: hex.EncodeToString(textHash[:]), Model: decoded.Model,
        Labels: decoded.Labels, FinishedAt: time.Now().UTC(),
    }, nil
}

Place a semaphore outside Classify, set an explicit timeout on the HTTP client and each context, and checkpoint after each page whose results are durably committed. The repository operation should be an insert guarded by the natural-key uniqueness constraint. If the row already exists, compare its immutable hashes and versions; equality means the repeat can be recorded as deduplicated, while disagreement belongs in review. Silent overwrite destroys the very evidence the bulk job was meant to produce.

Before production volume, test cancellation during request execution, process termination between response receipt and database commit, expired leases, duplicate queue delivery, truncated response bodies, and export interruption. Inject these failures. For example, terminate a worker after it has received a successful response but before its conditional insert, then restart it with the same lease record: the second attempt may make another remote call, yet the repository must still contain one immutable decision and one downstream action. Next, deliver the same queue item to two workers, corrupt an export partway through publication, and advance the source cursor while a partition is paused. The expected evidence is concrete: one natural-key row, one decision ID, a named terminal exception when a retry budget is exhausted, and a manifest whose checksum is either absent or valid. The happy path proves syntax; interruption tests prove resumability.

Validate policy quality, observability, and operating limits

Start with a stratified, human-reviewed evaluation set covering short comments, long posts, quotations, code, supported languages, and policy boundaries. Prompt examples are executable policy inputs — version and hash them — while precision and recall should be reported per label and content segment rather than collapsed into one accuracy number. A small harmful-content class can disappear inside an attractive aggregate. Threshold changes require a new policy version and a replay against stored classification evidence, assuming that evidence is sufficient for the revised rule.

Observe request latency, payload bytes, attempts, classifier units when the API reports them, label distribution, review rate, deduplication count, lease expiry, and age of the oldest pending item. Never place user text or stable user identifiers in metric labels. Logs should carry a decision ID and attempt ID; access-controlled audit storage can resolve those identifiers to details when an investigation is authorized.

Cost control belongs beside correctness, although price is not the architectural criterion. Sample a canary from the real length distribution, measure actual input expansion caused by policy instructions and quoted context, project a range, and stop when an approved usage ceiling is crossed. Row count alone is weak because one long discussion tree may consume more model input than hundreds of short comments. Your mileage may vary — especially across languages and markup-heavy posts — so keep the estimate tied to observed payload units rather than a guessed average.

This design is not suitable for every moderation path. Historical backfills can tolerate asynchronous completion; content that must be blocked before publication needs synchronous or near-synchronous controls on the write path. Fully automatic enforcement is also inappropriate where a decision could trigger an irreversible deletion, account suspension, statutory report, or other high-impact consequence without the review required by policy or law. Route uncertain and high-impact outcomes to trained reviewers, preserve chain of custody, and make downstream actions idempotent under the same decision ID.

Operationally, choose the smallest execution shape that meets recovery requirements. A supervised command can handle a modest, bounded corpus; leased workers suit partitions that must resume over a longer period; a database claim loop can keep ownership close to transactional data but adds polling and lease contention. There is no universal winner. The catch is that every step up in distribution adds coordination state that must itself be reconciled.

Roll out by proving the counts

Begin with one read-only snapshot partition and a policy that records decisions without applying them. Compare its results with human review, inspect malformed and ambiguous cases, verify that duplicate delivery produces one committed decision, and reconstruct a sample solely from the export and audit store. Then enable enforcement for reversible actions on a narrow partition, with an explicit rollback record rather than an ad hoc script.

Increase concurrency only after latency, retry, review, and cost signals remain within approved limits. At each stage, publish an export to a temporary object, verify its sorted row count and checksum, and atomically promote it with a manifest containing schema version, source snapshot ID, policy version, creation time, and control totals. Importers must accept the same file twice without applying the same restriction twice.

Finish when the arithmetic closes and an operator can explain every exception.