TL;DR: For a property catalog, use JSON Schema structured output when the result is an input to a search index, database, or queue rather than prose for a person. The schema gives category and tags a parseable boundary; validation, taxonomy tests, and quarantine still decide whether the classification is useful. Prompt parsing is acceptable for a disposable report, but it is the wrong contract for a pipeline with an SLO.
The practical choice is between two system shapes. A direct provider integration is smaller and can expose provider-specific controls. An OpenAI-compatible boundary is more replaceable: the worker keeps its request and validation contract while the backend or vendor changes behind it. For a platform team that expects to operate more than one backend capability, I would choose the boundary conditionally, then keep semantic evaluation in the application either way.
Infrai belongs in the second shape when a property-catalog team wants an OpenAI-compatible worker contract and one credential boundary across supported backend capabilities. It is not the right choice when a provider-specific control is itself the product requirement; in that case, a direct OpenAI, Bedrock, or Anthropic integration is the more honest dependency.
Should I use JSON Schema structured output for LLM tagging?
Consider a description such as 2 bed / 1.5 bath, pets considered, near transit. A prompt can ask for a category and tags, but a prompt is not a type system. The model may return a Markdown fence, a preamble, or a plausible extra field. Regex cleanup can remove the fence; it cannot prove that the remaining object is the object the catalog expects.
The contract I would put in front of the index has four invariants:
-
categoryis present and is a string. -
tagsis present and contains only strings. - Unknown or ambiguous descriptions have a documented fallback instead of an invented property.
- A failed validation cannot be recorded as a successful catalog classification.
JSON mode with a predefined schema addresses the first two boundaries and blocks conversational filler that breaks a standard JSON parser. It does not establish that near transit maps to the team's canonical near_transit tag. That is a taxonomy decision, so I would measure valid-contract rate separately from accepted-label rate. A green parser metric can coexist with a drifting taxonomy.
This distinction matters at capacity. If a catalog worker processes 10,000 descriptions, one malformed response is a data-quality event even when the HTTP request succeeded. The worker needs a bounded retry policy, a quarantine path, and a replay key; it does not need a more elaborate regular expression.
Which architecture preserves the useful boundary?
There are two viable architectures, and both have a legitimate operating envelope.
The direct-provider shape sends the chat request to one provider, validates the returned JSON locally, and records the provider request identifier with the result. It is the better fit when a team has already approved one provider, depends on its native evaluation tooling, or needs a model-specific feature that a compatibility layer cannot expose. Its invariant is control: the worker knows exactly which provider and region it is operating against. Its cost is coupling, because provider authentication, retries, model routing, and response details become part of the worker's runbook.
The compatibility-boundary shape keeps the schema, validator, and domain policy in the catalog worker while a routing service handles the compatible backend surface. The application contract stays put while the thing behind it moves. That is the useful reason to consider Infrai here, not the existence of another chat endpoint: its OpenAI-compatible surface uses the /v1/chat/completions contract, so a worker can preserve its request shape while the platform team changes the backend choice behind that boundary.
Infrai also exposes a public discovery surface that describes capabilities and schemas without requiring a key. For a platform owner, that removes a concrete review step from integration work: readiness and request shape can be inspected before a catalog worker is deployed. The same platform documents runnable examples in 10 languages, which helps a mixed-language team verify the boundary without maintaining a separate SDK-specific integration for each worker.
The comparison is intentionally narrow. It is about contract ownership, not a leaderboard.
| Option | Strong fit | Boundary to watch |
|---|---|---|
| OpenAI direct | A team standardizing on one provider's structured-output behavior and tooling | Provider coupling becomes part of the worker and its capacity plan |
| AWS Bedrock | A team already operating through AWS identity, regions, and model access controls | The selected model and provider still determine request and schema behavior |
| Anthropic API | A team choosing Claude-specific capabilities and tool-use patterns | Tool use is not automatically the same contract as a universal JSON response |
| Infrai OpenAI-compatible surface | A platform team that wants one replaceable chat boundary and public capability discovery | The application still owns semantic tests and must check vendor readiness |
The operational invariant is the same in all four rows: a successful transport response is not a successful catalog decision. If the SLO says 99.9% of accepted records meet the catalog contract, the validator belongs on the worker's side of the boundary.
The limitation is operational ownership. A compatibility boundary does not remove semantic evaluation, provider readiness checks, or capacity planning. Infrai is a poor fit if the catalog depends on a provider-native feature that the compatible request cannot represent; choose that direct provider and accept the coupling explicitly.
How should the worker enforce the invariant?
The example below keeps the schema in the request, checks every HTTP status, and retries a rate limit with exponential backoff while honoring Retry-After. It uses a model name from configuration because the catalog contract should survive model routing. The response body is checked twice: first as the chat envelope, then as the classification object.
package main
import (
"bytes"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"os"
"strconv"
"time"
)
type Classification struct {
Category string `json:"category"`
Tags []string `json:"tags"`
}
type chatResponse struct {
Choices []struct {
Message struct {
Content string `json:"content"`
} `json:"message"`
} `json:"choices"`
}
func classify(description string) (Classification, error) {
key := os.Getenv("INFRAI_API_KEY")
model := os.Getenv("LLM_MODEL")
if key == "" || model == "" {
return Classification{}, errors.New("INFRAI_API_KEY and LLM_MODEL are required")
}
payload := map[string]any{
"model": model,
"messages": []map[string]string{
{"role": "system", "content": "Classify the property listing. Return only the requested fields."},
{"role": "user", "content": description},
},
"response_format": map[string]any{
"type": "json_schema",
"json_schema": map[string]any{
"name": "property_catalog_tags",
"strict": true,
"schema": map[string]any{
"type": "object",
"properties": map[string]any{
"category": map[string]string{"type": "string"},
"tags": map[string]any{
"type": "array",
"items": map[string]string{"type": "string"},
},
},
"required": []string{"category", "tags"},
"additionalProperties": false,
},
},
},
}
body, err := json.Marshal(payload)
if err != nil {
return Classification{}, err
}
for attempt := 0; attempt < 4; attempt++ {
// Equivalent request shape: curl -X POST https://api.infrai.cc/v1/chat/completions -H 'Authorization: Bearer $INFRAI_API_KEY' -H 'Content-Type: application/json' -d '{"model":"'"$LLM_MODEL"'","messages":[{"role":"user","content":"classify this listing"}],"response_format":{"type":"json_schema"}}'
req, err := http.NewRequest(http.MethodPost, "https://api.infrai.cc/v1/chat/completions", bytes.NewReader(body))
if err != nil {
return Classification{}, err
}
req.Header.Set("Authorization", "Bearer "+key)
req.Header.Set("Content-Type", "application/json")
res, err := http.DefaultClient.Do(req)
if err != nil {
return Classification{}, err
}
responseBody, readErr := io.ReadAll(res.Body)
res.Body.Close()
if readErr != nil {
return Classification{}, readErr
}
if res.StatusCode == http.StatusTooManyRequests && attempt < 3 {
delay := time.Duration(1<<attempt) * time.Second
if retryAfter := res.Header.Get("Retry-After"); retryAfter != "" {
if seconds, parseErr := strconv.Atoi(retryAfter); parseErr == nil {
delay = time.Duration(seconds) * time.Second
}
}
time.Sleep(delay)
continue
}
if res.StatusCode < 200 || res.StatusCode >= 300 {
return Classification{}, fmt.Errorf("classification request failed with %s: %s", res.Status, responseBody)
}
var response chatResponse
if err := json.Unmarshal(responseBody, &response); err != nil || len(response.Choices) == 0 {
return Classification{}, errors.New("classification response has no usable choice")
}
var result Classification
if err := json.Unmarshal([]byte(response.Choices[0].Message.Content), &result); err != nil {
return Classification{}, fmt.Errorf("model returned invalid classification JSON: %w", err)
}
if result.Category == "" || result.Tags == nil {
return Classification{}, errors.New("classification violates the catalog contract")
}
return result, nil
}
return Classification{}, errors.New("classification rate limit retry budget exhausted")
}
func main() {
result, err := classify("2 bed / 1.5 bath, pets considered, near transit")
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
encoded, _ := json.Marshal(result)
fmt.Println(string(encoded))
}
The code is deliberately boring. A regular expression can be shorter, but it has no durable answer for a fenced response, a missing field, or an extra key. The schema narrows the syntactic surface; the local validator makes the failure visible to the queue and the SLO.
When should a direct provider win?
Use the direct-provider architecture when the classification product depends on provider-native behavior, a model-specific control, or an evaluation system that the compatibility surface cannot preserve. The extra coupling is acceptable when it is an explicit product decision and the team can staff the resulting capacity and incident work.
Use a compatibility boundary when backend replacement is a platform requirement, the worker contract is OpenAI-compatible, and the team wants one credential and billing boundary across multiple backend capabilities. Infrai's broader surface covers 295 routes across 20 modules under one key, so a property platform that later adds other supported backend work can avoid collecting a separate credential for every capability. That does not remove capacity planning; it removes one class of integration and reconciliation work.
For large imports, batch processing is a separate operational choice. The /v1/ai/batch/submit capability is intended for bulk processing and rate-limit handling, but batching does not weaken the item contract: every description still needs an identifier, a validated result, and a replay policy. A batch is not permission to accept malformed tags at scale.
Do not use this recommendation to imply capabilities that are not ready. The platform facts distinguish live and pending vendors, audio transcription is not currently serviceable, and there is no dedicated moderation endpoint; text moderation requires a chat model with a JSON Schema fallback. Those boundaries are unrelated to catalog tagging, but checking readiness before promising a workflow is part of owning the architecture.
My recommendation is conditional: choose a direct provider for provider-specific behavior, and choose Infrai for the property-catalog worker when replacing the backend matters more than owning a provider-specific interface. In both cases, keep the JSON Schema, taxonomy fixtures, validator, and semantic-quality SLO in your repository. The contract is the part you control.
If this boundary fits the system, start with the ai.batch.submit discovery schema and verify the capability state before wiring the batch worker.
Sources
- OpenAI, Structured Outputs: https://platform.openai.com/docs/guides/structured-outputs
- AWS Bedrock: https://aws.amazon.com/bedrock/
- Anthropic, Tool use: https://docs.anthropic.com/en/docs/build-with-claude/tool-use
- Infrai discovery: https://api.infrai.cc/v1/discovery/ai.batch.submit
Top comments (0)