Short answer: treat each supplier invoice summary as a leased, idempotent job, validate its JSON before committing it, and keep the model provider behind a narrow adapter. The Node.js API may accept the request, but the durable job record owns completion; a client connection doesn't.
That rule matters more than prompt polish. I've been paged for missed cron jobs and duplicate queue deliveries, and the recurring lesson is blunt: delivery is not completion. In an edtech accounts-payable flow, a duplicate "contact supplier" action can waste time, while a missing "verify purchase order" action can hold up course materials. The invariant is one committed result for one source document and one schema version, even when a worker restarts or a request is retried.
Retries are normal.
The model's task here is bounded. It turns previously extracted invoice text into a title, concise bullets, normalized invoice fields, and action items. Optical or speech extraction is a separate stage. Keeping those stages apart makes provenance clearer and lets the summarization contract survive a change in either component.
How should a Node.js API schedule LLM summary JSON action items?
Return an accepted job identifier after the request has been durably recorded. Then let workers lease jobs, call the configured model adapter, validate the candidate, and atomically commit the result. A read endpoint can expose queued, running, succeeded, or failed without tying work to an open HTTP connection.
This is an asynchronous API even if a first prototype completed inside one Node.js request handler. The important boundary is the persisted job, not the language used by the edge service. A deterministic job key can be derived from the tenant, source-document digest, transformation name, and schema version. Repeating the same logical request then points to the same job. Don't include an attempt number in that key; attempts belong in execution metadata, or every retry becomes new work. HTTP semantics help, but they don't create application-level deduplication. RFC 9110 defines idempotent methods and explains why clients may automatically retry an idempotent request after a communication failure. A summary-creation operation will often be a POST, so the service must define its own idempotency contract and store the result associated with the caller's key. The client should send the same key after a timeout, and the server should return the existing job rather than enqueue another one. A lease then needs an expiry and an owner token. Only that token may extend the lease or commit the result. If a worker disappears, another can take the job after expiry; if the old worker wakes later, its stale commit is rejected. This is the fence that keeps at-least-once delivery from becoming duplicate side effects.
Fencing decides ownership.
Keep side effects after validation and commit. Sending an email, creating a payable task, or updating a ledger directly from raw model output makes replay dangerous. Publish an outbox event from the same transaction that commits the summary, and give each downstream action its own stable key such as job_id + action_item_id. That is the part I put in the runbook in capital letters: a successful model call is not a committed job.
Make the schema an operational boundary
The JSON contract should represent what downstream code can safely consume, not everything the model might say. Use a version field, tight cardinality, explicit nullability, enumerated action kinds, and evidence copied from the source text. Money should have an ISO currency code and an integer minor-unit amount rather than a floating-point number. Dates need one declared representation. If the source omits a value, preserve that absence; don't invite the model to fill it from habit.
For this invoice workflow, a practical response has four layers:
| Layer | Required content | Failure policy |
|---|---|---|
| Identity | schema version, job ID, source digest | Reject on mismatch |
| Display | title and one to five bullets | Reject empty or oversized values |
| Invoice facts | supplier, invoice number, currency, total, due date | Allow declared missing fields |
| Actions | stable ID, allowed kind, source evidence | Reject unknown kinds or absent evidence |
The title is for a review queue, not a substitute for the invoice. Bullets should summarize facts that appear in the extracted text. Action items are proposed work, so their kinds must map to known handlers such as verify_purchase_order, review_tax, or contact_supplier. Free-form instructions can be displayed to a person, but they should never select executable code.
Here is the provider-facing boundary I would put behind the queue worker. It uses Go because the worker implementation is incidental; a Node.js edge API can produce the same job envelope. The adapter receives plain text and returns bytes, while validation and state transitions remain owned by the application.
package summary
import (
"context"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"strings"
)
const schemaVersion = "invoice-summary-v1"
type ActionItem struct {
ID string `json:"id"`
Kind string `json:"kind"`
Evidence string `json:"evidence"`
}
type InvoiceFacts struct {
SupplierName string `json:"supplier_name"`
InvoiceNo string `json:"invoice_number"`
Currency *string `json:"currency"`
TotalMinor *int64 `json:"total_minor"`
DueDate *string `json:"due_date"`
}
type Summary struct {
SchemaVersion string `json:"schema_version"`
Title string `json:"title"`
Bullets []string `json:"bullets"`
Invoice InvoiceFacts `json:"invoice"`
ActionItems []ActionItem `json:"action_items"`
}
type Model interface {
GenerateJSON(ctx context.Context, sourceText []byte) ([]byte, error)
}
func JobKey(tenant string, sourceText []byte) string {
digest := sha256.Sum256(sourceText)
key := tenant + "\x00invoice-summary\x00" + schemaVersion + "\x00" + hex.EncodeToString(digest[:])
sum := sha256.Sum256([]byte(key))
return hex.EncodeToString(sum[:])
}
func GenerateAndValidate(ctx context.Context, model Model, sourceText []byte) (Summary, error) {
raw, err := model.GenerateJSON(ctx, sourceText)
if err != nil {
return Summary{}, fmt.Errorf("generate candidate: %w", err)
}
var candidate Summary
decoder := json.NewDecoder(strings.NewReader(string(raw)))
decoder.DisallowUnknownFields()
if err := decoder.Decode(&candidate); err != nil {
return Summary{}, fmt.Errorf("decode candidate: %w", err)
}
if err := validate(candidate, string(sourceText)); err != nil {
return Summary{}, err
}
return candidate, nil
}
func validate(s Summary, source string) error {
if s.SchemaVersion != schemaVersion {
return errors.New("unexpected schema version")
}
if strings.TrimSpace(s.Title) == "" || len(s.Bullets) < 1 || len(s.Bullets) > 5 {
return errors.New("invalid display summary")
}
allowed := map[string]bool{
"verify_purchase_order": true,
"review_tax": true,
"contact_supplier": true,
}
seen := make(map[string]bool)
for _, action := range s.ActionItems {
if action.ID == "" || seen[action.ID] || !allowed[action.Kind] {
return errors.New("invalid action identity or kind")
}
if action.Evidence == "" || !strings.Contains(source, action.Evidence) {
return errors.New("action evidence is absent from source")
}
seen[action.ID] = true
}
return nil
}
This validator is deliberately incomplete. Production code should also cap byte lengths, verify date and currency syntax, reject a second JSON value after the first, and define Unicode normalization before hashing. I'm not sure which invoice fields your finance team treats as authoritative; only that team's acceptance policy can resolve that. Put those decisions in deterministic validation, where they can be reviewed and tested, rather than in prompt prose.
Portability lives above the model adapter
Provider portability means the stored job and published result do not expose a provider's response envelope, finish reason, tool-call representation, or model-specific identifier as business data. The adapter translates the application's contract into a provider request and translates the response back into candidate JSON. Provider metadata can still live in restricted execution logs for diagnosis, but downstream invoice consumers should depend on invoice-summary-v1.
Compare three architectural options. A provider-native schema can reduce adapter work, but it couples validation keywords and response parsing to that API. Prompt-only JSON is easy to start with, but malformed or extra fields become routine control-flow cases. Application-owned validation adds code and may reject output a human could understand, yet it creates one admission rule across providers. For a queue that can replay work, I favor the third option because the commit decision stays deterministic.
The catch is real. This pattern is not suitable when a person is interactively exploring one document and can correct output before anything is stored; a synchronous request with a review screen may be simpler. Stick with a provider-native contract when a single-provider deployment is an explicit constraint and its advanced structured-output features are central to the product. Also avoid automatic action generation when source evidence cannot be retained under your privacy policy. Portability has a maintenance cost: every adapter needs conformance tests, and the common contract cannot expose every provider-specific feature.
Changing providers should be a deployment decision backed by a replayable evaluation set, not a type migration. Save consented, redacted invoice fixtures with expected invariants: required title, bounded bullets, exact evidence spans, correct minor-unit totals, and stable action kinds. Run the same fixtures through each adapter, then compare admission rates and semantic review results. Your mileage may vary on model behavior, so promote an adapter only after it passes the workload you actually own.
Test the ugly execution paths
Happy-path schema tests are necessary and small. The useful tests force the worker through the places where ownership changes: the HTTP response disappears after the job is stored, a lease expires during generation, two workers receive the same delivery, a stale worker tries to commit, validation rejects a candidate, and an outbox publisher retries. Assert the durable state and number of downstream effects after each sequence.
I once assumed queue acknowledgment was the finish line; being paged by both missed jobs and duplicate deliveries corrected that mental model. Acknowledgment only describes the broker interaction. The database transition guarded by the lease token decides which result won, and the outbox key decides whether the downstream action happens once.
Make retry policy classification explicit. Connection loss before a response is ambiguous, so reuse the job key. Validation rejection is not fixed by an immediate identical retry; record the rejection category and route it to controlled regeneration or review. Rate limiting can be retried with bounded backoff when the provider's contract permits it. Exhausted attempts should land in a visible terminal state with the source job intact, not disappear from the queue.
Watch four signals: oldest queued-job age, lease recovery count, validation rejection rate by schema version and adapter, and time from accepted to committed. Add duplicate-suppression counts and outbox lag. Don't put invoice text, supplier bank details, or complete model payloads in metrics labels or routine logs. Trace with job IDs and digests, then gate access to any retained source content.
One more trap: a scheduler can be healthy while a tenant is starved. Slice queue age and throughput by a bounded tenant class or priority lane, enforce concurrency limits, and test that a large school district's batch cannot block every smaller account. Fairness is part of reliability here.
Release and rollback as contract changes
Schema changes need the same discipline as database changes. Add invoice-summary-v2 beside v1, make consumers declare what they accept, and keep the job key versioned. A retry of a v1 request must not silently become v2. During rollout, shadow a bounded sample through the new validator without publishing its actions, compare results, and then shift traffic gradually.
Rollback the adapter or prompt independently from the public result schema. If a new prompt increases validation rejection, workers can return to the prior prompt while queued jobs retain the same identity. If the contract itself must roll back, stop admitting the new version first and drain or explicitly migrate its jobs. Never reinterpret stored JSON under a different schema version.
The final decision rule is operational: choose an application-owned JSON contract when summaries feed durable workflows, use leases and fenced commits when work may be delivered more than once, and isolate provider details when replacement is a real requirement. For disposable, human-reviewed summaries, keep it smaller. Either way, no action item should escape until its evidence and shape have passed code you control.
References
- RFC 9110, HTTP Semantics: https://www.rfc-editor.org/rfc/rfc9110
- openai/whisper, open-source speech recognition: https://github.com/openai/whisper
Top comments (0)