Node.js Chatbot API Compare: Long Context Good Quality via Schema Drift Controls

TL;DR: For a chatbot API with long context, good quality means a contract pipeline, not a prompt call. Validate the model's JSON at the boundary, measure field-level rejection and repair rates, and retain a small, redacted sample of failures. The decision rule is straightforward: choose the runtime path that preserves required attributes and auditability under malformed input, even when its first-pass output rate is lower.

This article records that decision for a marketplace that turns supplier descriptions into searchable product attributes. The input is inconsistent prose: units mixed with marketing language, missing dimensions, variant details embedded in paragraphs, and occasionally contradictory claims. The output is a versioned object that downstream search and merchandising can trust.

What must remain true when the description is messy?

Three invariants define the boundary. Every accepted object must validate against the active schema. Every uncertain attribute must retain an explicit state rather than becoming a plausible value. Every rejected object must be replayable without storing more customer or supplier text than the investigation requires.

The failure boundary belongs between generation and persistence. A response that looks like JSON but contains a string where an array is required is not an almost-successful catalog record. It is an unaccepted result. That distinction prevents a partial write from becoming a silent search defect. JSON itself does not define an application schema; RFC 8259 defines the data interchange format, while a schema system supplies application-level constraints.

I initially treated long descriptions as a context-window problem. The more expensive discovery was that context length did not make a missing unit less ambiguous. A 1,200-word description can still fail a required net_weight_g field. Quality is therefore measured by valid, useful fields, not by the amount of text sent.

The resulting record keeps provenance beside the value:

{
  "schema_version": "catalog.v3",
  "title": "Insulated bottle",
  "attributes": {
    "capacity_ml": {"value": 750, "state": "observed", "source_span": "750 ml"},
    "material": {"value": null, "state": "unknown", "source_span": null}
  }
}

unknown is a product decision, not an error message. Search can exclude it; a reviewer can resolve it; an automated repair step must not invent it.

How can a chatbot API keep long context quality good?

The architecture decision compares three paths.

Path	What it guarantees	Failure it exposes	Appropriate use
Parse and validate once	A clear acceptance boundary	Missing fields, wrong types, extra keys	Default path for every item
Validate, then repair with the original evidence	A second chance under the same contract	Persistent ambiguity and repeated invalid fields	Bounded retry for recoverable shape errors
Send to a review queue	No fabricated attributes	Human throughput and queue age	High-value or safety-sensitive items

The rejected option is “accept valid-looking JSON and normalize later.” It has a convenient happy path, but normalization after persistence loses the distinction between an absent attribute and one the generator guessed. That option is valid for exploratory analytics where records are disposable and no downstream action depends on the value. It is unsuitable for a catalog index.

The critical path is deliberately boring. The caller asks for an object, parses it, validates it, records counters, and persists only an accepted result. A generic endpoint keeps the mechanism portable.

The familiar comparison set—GPT-4.1 mini, Claude 3.5 Haiku, and Gemini 1.5 Flash—can be used as labels in a fixed evaluation, but the names do not decide acceptance. A good chatbot API is the one that preserves the same schema and evidence rules across those runs. A smaller context budget may be a valid choice for short descriptions; long context is useful only when the extra material changes a field decision.

This boundary has a real trade-off: strict validation creates rejects and review work, while permissive parsing creates faster writes with weaker evidence. For a marketplace index, I accept the queue and its latency because an unverified attribute can affect search and merchandising for much longer than a review wait.

curl -sS https://api.example.invalid/v1/ai/batch/submit \
  -H 'content-type: application/json' \
  -H 'idempotency-key: item-8472-attempt-1' \
  --data '{
    "schema": "catalog.v3",
    "description": "Double-wall steel bottle, 750 ml, keeps drinks cold.",
    "required_fields": ["capacity_ml", "material"]
  }'

In production, the service validates the returned object locally against the pinned schema as well. The endpoint may enforce a contract, but the persistence service owns the final write decision. Two boundaries make failures visible instead of distributing trust across an undocumented assumption.

How should telemetry count a bad result?

Observability is part of the data contract because each label can multiply storage and query work. I keep low-cardinality counters such as schema_version, failure_class, attempt, and pipeline_stage. I do not put the item identifier, raw description, or full validation message into a metric label. Those belong in a redacted event stream with controlled retention, if they are needed at all.

A useful dashboard can answer five questions: what fraction passed first validation, which field failed most often, how many repairs succeeded, how old the review queue is, and how many records were dropped before persistence. The first question alone is weak. A 98% pass rate can conceal a 40% failure rate for a mandatory attribute if the denominator is total objects rather than field checks.

For a rough storage estimate,

retained_bytes = events_per_day × sampled_fraction × average_event_bytes × retention_days.

That is not a billing claim; it is an engineering control. Sampling 1% of successful events while retaining every schema failure gives a more useful signal than retaining every successful payload. The sampled event should contain schema version, field names, error class, latency bucket, and a short hash for correlation. Keep source text out unless an access-controlled replay actually needs it.

OpenTelemetry defines a common model for traces, metrics, and logs, but it does not decide which business fields are safe to retain. That policy remains ours. The same distinction applies to token measurement: a tokenizer library can estimate input size for a particular encoding, while retention and sampling determine how much operational evidence we keep.

What does the test corpus need to prove?

A golden set should be stratified by failure mode, not assembled from only clean descriptions. I would include at least: missing units, multiple units, variant-specific dimensions, negation, conflicting measurements, non-Latin text, long boilerplate, and descriptions that contain valid JSON as part of the prose. Each fixture has an expected state for every required attribute: observed, unknown, or review.

The useful test metric is field-level correctness under the acceptance contract. Track exact type validity separately from semantic review, because a number with the wrong unit can pass JSON Schema while still being wrong for merchandising. Add a replay test for every production failure class. The test should assert that a retry cannot change an explicitly unknown value into an observed value without evidence in the input.

A compact request for an evaluation run can carry only fixture identifiers and the schema revision; the descriptions stay in the controlled test store.

curl -sS https://api.example.invalid/v1/ai/batch/submit \
  -H 'content-type: application/json' \
  --data '{
    "dataset": "catalog-fixtures-2026-10",
    "schema": "catalog.v3",
    "cases": ["missing-unit-014", "variant-conflict-022"]
  }'

Latency and token counts still matter, but they are secondary signals here. A faster response that increases review volume transfers work to people and delays searchable inventory. The operational comparison should use a fixed corpus, the same schema, the same retry budget, and the same acceptance policy.

Decision

Adopt parse-and-validate as the mandatory first boundary, allow one evidence-preserving repair attempt for shape errors, and route unresolved or high-impact attributes to review. Version the schema with the record, keep telemetry labels bounded, and sample successful payload evidence while retaining failure metadata.

This is a deliberately conservative runtime. It accepts less, explains more, and gives the catalog team a stable place to improve prompts, fixtures, and policies. The architecture can later replace the generation component without changing the acceptance contract or the measurement vocabulary.

References

• RFC 8259, The JavaScript Object Notation (JSON) Data Interchange Format: https://www.rfc-editor.org/rfc/rfc8259
• JSON Schema specification: https://json-schema.org/specification
• OpenTelemetry documentation: https://opentelemetry.io/docs/
• OpenAI tiktoken repository: https://github.com/openai/tiktoken
• OpenAI Whisper repository: https://github.com/openai/whisper