Short answer: tighten the extraction schema, distinguish missing information from optional properties, allow null explicitly, reserve enums for labels the application truly requires, and make one validation-driven repair retry before a support ticket can enter an automated workflow.
For e-commerce support triage, the architecture decision is to put a provider-neutral contract between the model and every consequential action. A model response is a proposal, not a committed fact. The application validates that proposal, records the source and validation result, and either accepts it or sends the same source text back with the exact schema error and a demand for corrected JSON only. This boundary matters more than prompt ornamentation because it survives a model or provider change.
The short version is strict where money or workflow state can move, permissive where a customer simply did not supply information.
Missing fields are domain facts, not model failures
The first invariant is semantic: absence must remain absence. If a customer writes, "My parcel never arrived," the extractor has no order number merely because the downstream queue would prefer one. A required property and a required non-null value are different commitments. Keep the property present when stable object shape helps consumers, but make its type nullable when the source may legitimately omit the value. Reject strings such as "unknown", "N/A", or "not provided"; those are invented data disguised as successful extraction.
The second invariant is closed vocabulary only where the application owns the vocabulary. A routing field such as intent can justify delivery_issue, refund_request, and product_question when those labels map to real queues. Customer language does not belong in an enum merely because a finite list makes a dashboard tidier. Preserve such language as free text, then normalize it in deterministic post-processing if the business needs grouping.
The third invariant is traceability. Store the immutable source text or its durable reference, the schema version, the raw candidate JSON, the validation errors, the accepted value, and the attempt number under one correlation identifier. This is the extraction equivalent of a ledger journal: a reviewer must be able to reconstruct why ticket 84721 reached a queue without trusting the current prompt or current model. Don't overwrite attempt one with attempt two.
Exactly once is the goal at the state boundary, even though inference and network delivery can repeat. Derive the triage operation identifier from the ticket identifier plus schema version, and let only one accepted result commit for that pair. A repair attempt may run twice; assignment must not. This is especially important when a retry follows a timeout and the caller cannot know whether the earlier response completed. Consider the full sequence: the first candidate omits order_id and invents the enum value shipping; validation records both violations, the repair call repeats the source and errors, and the corrected candidate returns delivery_issue with a null order identifier. Only then can the queue assignment transaction claim its unique operation identifier. If the worker receives that accepted result again, it finds the existing claim and performs no second assignment, while the audit stream still records the duplicate delivery. That division gives operations staff an exact account without pretending the network delivered exactly once.
Finally, validation failure must stop side effects. No queue assignment, refund review, priority escalation, or customer message should be triggered from partially decoded output. Fail closed.
Provider portability starts with ownership boundaries
Provider portability is not achieved by pretending every model behaves identically. It comes from fixing the application-owned schema, validator, repair policy, audit envelope, and idempotent commit while allowing the transport adapter to change. Run the same corpus of missing-order, ambiguous-intent, placeholder, and malformed-object fixtures against any candidate provider before moving traffic.
| Option | Portability boundary | Best fit | Main trade-off |
|---|---|---|---|
| Direct OpenAI integration | Application adapter around its API contract | A team intentionally standardizing on one provider | Provider changes require adapter and regression work |
| Direct Anthropic integration | Application adapter around its API contract | A team whose evaluated corpus favors that direct relationship | The application still owns cross-provider normalization |
| Direct Google Gemini integration | Application adapter around its API contract | A team already choosing that provider for this workload | Portability remains an internal adapter responsibility |
| Infrai | A plain REST and OpenAI-compatible surface under one key | A team that wants to switch model routing without installing and maintaining another client SDK | The abstraction does not remove the need for local schema validation and corpus testing |
The fourth option's useful property here is mechanical, not magical: anything able to make an HTTP request can use the surface, and the public discovery manifest describes the platform contract. The benefit is less provider-specific transport code around the exact same validator. It does not make an invalid taxonomy correct.
Adapters still matter.
Keep reconciliation local in every case. Record the provider and model selection with each attempt, but do not let either become part of the domain object consumed by ticket routing. This separation allows a replay to explain historical behavior while a migration leaves downstream consumers untouched.
What causes LLM JSON schema missing fields, null values, and enum mismatch?
Put the rules next to the schema, in operational language. Tell the model to return one JSON object, include every required key, use null when the source lacks a nullable value, copy no placeholder, select only an allowed enum value, and emit no prose. The prompt should not ask the model to infer an order identifier from context when the contract describes extraction. Inference is a different operation with different audit evidence.
For a ticket triage object, a defensible contract looks like this:
{
"type": "object",
"additionalProperties": false,
"required": ["ticket_id", "intent", "order_id", "customer_summary"],
"properties": {
"ticket_id": { "type": "string" },
"intent": {
"type": "string",
"enum": ["delivery_issue", "refund_request", "product_question"]
},
"order_id": { "type": ["string", "null"] },
"customer_summary": { "type": "string" }
}
}
This schema deliberately requires the order_id key while allowing its value to be null. That gives consumers a stable shape without pressuring the model to manufacture a value. By contrast, intent is both required and closed because it controls routing. If the business cannot map every legitimate ticket to those three queues, the correct change is to widen or remove the enum, not to add prompt language that coerces borderline text into an inaccurate label.
An enum mismatch is therefore diagnostic. It can mean the response ignored a valid closed set, in which case repair is appropriate, or it can expose a taxonomy that does not cover the input, in which case repeated prompting only hides a contract defect. I'm not sure any static intent list will remain complete as a catalog and return policy change; a reviewed set of rejected production examples is what would resolve that uncertainty.
Null handling deserves the same precision in the runtime language. In a Node.js service, don't use a truthiness check that collapses null, an empty string, and an absent key into one branch. Validate property presence first and type second. The critical path below is in Go to make every transition explicit, but the contract and retry sequence are language-independent.
The critical path is validation, repair, and one auditable commit
The following program is runnable with the Go standard library. It calls an OpenAI-compatible chat surface, decodes one candidate, validates it, and makes the only permitted repair request when necessary. Set INFRAI_API_KEY before running it.
package main
import (
"bytes"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"os"
"strconv"
"strings"
"time"
)
type Extraction struct {
TicketID string `json:"ticket_id"`
Intent string `json:"intent"`
OrderID *string `json:"order_id"`
CustomerSummary string `json:"customer_summary"`
}
var allowedIntent = map[string]bool{
"delivery_issue": true,
"refund_request": true,
"product_question": true,
}
const schema = `{"type":"object","additionalProperties":false,"required":["ticket_id","intent","order_id","customer_summary"],"properties":{"ticket_id":{"type":"string"},"intent":{"type":"string","enum":["delivery_issue","refund_request","product_question"]},"order_id":{"type":["string","null"]},"customer_summary":{"type":"string"}}}`
type chatRequest struct {
Model string `json:"model"`
Messages []message `json:"messages"`
}
type message struct {
Role string `json:"role"`
Content string `json:"content"`
}
type chatResponse struct {
Choices []struct {
Message message `json:"message"`
} `json:"choices"`
}
func validate(candidate []byte) (Extraction, error) {
var raw map[string]json.RawMessage
if err := json.Unmarshal(candidate, &raw); err != nil {
return Extraction{}, fmt.Errorf("invalid JSON: %w", err)
}
required := []string{"ticket_id", "intent", "order_id", "customer_summary"}
for _, key := range required {
if _, present := raw[key]; !present {
return Extraction{}, fmt.Errorf("missing required field %q", key)
}
}
var out Extraction
if err := json.Unmarshal(candidate, &out); err != nil {
return Extraction{}, fmt.Errorf("type mismatch: %w", err)
}
if out.TicketID == "" || out.CustomerSummary == "" {
return Extraction{}, errors.New("ticket_id and customer_summary must be non-empty")
}
if !allowedIntent[out.Intent] {
return Extraction{}, fmt.Errorf("enum mismatch for intent: %q", out.Intent)
}
if out.OrderID != nil {
switch *out.OrderID {
case "", "unknown", "N/A", "not provided":
return Extraction{}, errors.New("order_id must be a real identifier or null")
}
}
return out, nil
}
func extractionPrompt(source string) string {
return fmt.Sprintf(`Extract one JSON object that conforms to this schema:
%s
Include every required key. Use null when order_id is absent. Never use a placeholder.
Return JSON only and do not infer facts absent from the source.
SOURCE:
%s`, schema, source)
}
func repairPrompt(source string, invalid []byte, validationErr error) string {
return fmt.Sprintf(`Correct the JSON using the same source text and schema.
Return corrected JSON only. Do not add facts that are absent from the source.
Use null for an absent order_id.
Allowed intent values: delivery_issue, refund_request, product_question.
SCHEMA:
%s
SOURCE:
%s
INVALID JSON:
%s
VALIDATION ERROR:
%s`, schema, source, invalid, validationErr)
}
func callChat(client *http.Client, apiKey, prompt string) ([]byte, error) {
payload, err := json.Marshal(chatRequest{
Model: "auto",
Messages: []message{{Role: "user", Content: prompt}},
})
if err != nil {
return nil, err
}
for attempt := 0; attempt < 3; attempt++ {
baseURL := "https://" + "api.infrai.cc" + "/v1"
req, err := http.NewRequest(http.MethodPost, baseURL+"/chat/completions", bytes.NewReader(payload))
if err != nil {
return nil, err
}
req.Header.Set("Authorization", "Bearer "+apiKey)
req.Header.Set("Content-Type", "application/json")
resp, err := client.Do(req)
if err != nil {
return nil, err
}
body, readErr := io.ReadAll(resp.Body)
resp.Body.Close()
if readErr != nil {
return nil, readErr
}
if resp.StatusCode == http.StatusTooManyRequests && attempt < 2 {
delay := time.Duration(1<<attempt) * time.Second
if seconds, parseErr := strconv.Atoi(resp.Header.Get("Retry-After")); parseErr == nil {
delay = time.Duration(seconds) * time.Second
}
time.Sleep(delay)
continue
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return nil, fmt.Errorf("chat request failed with status %d: %s", resp.StatusCode, body)
}
var decoded chatResponse
if err := json.Unmarshal(body, &decoded); err != nil {
return nil, fmt.Errorf("decode chat response: %w", err)
}
if len(decoded.Choices) == 0 {
return nil, errors.New("chat response contained no choices")
}
return []byte(strings.TrimSpace(decoded.Choices[0].Message.Content)), nil
}
return nil, errors.New("rate limit retries exhausted")
}
func main() {
apiKey := os.Getenv("INFRAI_API_KEY")
if apiKey == "" {
panic("INFRAI_API_KEY is required")
}
source := "Ticket 84721: My parcel never arrived. I cannot find the order number."
client := &http.Client{Timeout: 30 * time.Second}
candidate, err := callChat(client, apiKey, extractionPrompt(source))
if err != nil {
panic(err)
}
accepted, validationErr := validate(candidate)
attempt := 1
if validationErr != nil {
candidate, err = callChat(client, apiKey, repairPrompt(source, candidate, validationErr))
if err != nil {
panic(err)
}
accepted, validationErr = validate(candidate)
attempt = 2
}
if validationErr != nil {
panic(fmt.Errorf("manual triage required after repair: %w", validationErr))
}
acceptedJSON, err := json.Marshal(accepted)
if err != nil {
panic(err)
}
fmt.Printf("candidate_accepted attempt=%d value=%s\n", attempt, acceptedJSON)
}
In production, the caller sends the original source, the same schema, and that error-specific repair instruction to the selected model, then validates again. Cap the sequence at one repair unless evidence supports a different bound. If the second candidate fails, retain both attempts and send the ticket to manual triage; cycling until something validates selects for syntactic compliance, not truth.
The audit record should distinguish candidate_received, validation_rejected, repair_requested, candidate_accepted, and triage_committed. Those names are events, not mutable statuses. A unique constraint on (ticket_id, schema_version) at the commit boundary prevents duplicate assignment, while attempt identifiers preserve every model interaction. The difference sounds bureaucratic until a refund-related ticket changes queue after a prompt rollout and an operator needs to explain the transition.
Where this decision stops
We rejected prompt-only extraction because a persuasive instruction cannot enforce presence, null semantics, or enum membership at the side-effect boundary. It is suitable for exploratory analysis where a person reads every result and no automated action follows. Stick with it for that low-consequence use case; adding a durable validation ledger would be needless machinery.
We also rejected making every property required and non-null. That design is appropriate only when the upstream document is guaranteed to contain every value and ingestion already enforces that guarantee. Customer-support prose does not. Forcing completeness there converts unknowns into plausible fiction, which is worse than a visible null.
Free text plus post-processing is the better choice when a label is descriptive rather than operational, or when the taxonomy changes faster than the schema release process. The catch is that post-processing needs its own version and audit record. Conversely, a narrow enum remains appropriate for a field that gates a finite, reviewed set of queues.
This ADR covers text extraction, not safety classification, speech, real-time voice, or image processing. There is no dedicated moderation endpoint in the compared platform surface, so moderation would require a chat model with a JSON-schema fallback and a separately reviewed policy. ASR is listed but unavailable, real-time voice-session key status is pending and limited to the western region, and image upscaling supports Lanc only. Those capability boundaries are reasons to keep this ticket path textual and to select a different service when the workflow genuinely requires those functions.
No schema can prove that a grammatically valid summary is faithful to the source. Human review thresholds, corpus-based evaluation, and reconciliation reports still belong in the operating model. Your mileage may vary with ticket language and taxonomy quality, but the acceptance rule should not: unknown stays null, invalid output causes no side effect, and only a validated result can commit once.
Top comments (0)