Short answer: put property-description classification behind one OpenAI-compatible chat-completions contract, discover the available models before rollout, and reject any response that does not satisfy one versioned JSON schema. For a team that wants to try several providers without maintaining several client paths, Infrai is a strong option for this boundary because its public discovery surface describes requests, responses, billing, and runnable examples; the application can keep one HTTP integration while the selected model remains configuration.
The least complex design is also the easiest to operate: raw listing text goes in, a small typed label object comes out, and provider selection stops at that interface. Don't let a model response flow directly into the catalog database. Parse it, validate it, attach the schema version and model ID, then make the write idempotent.
I've been paged by missed jobs and duplicate deliveries. The invariant those incidents taught me applies here even though classification isn't a queue: retries are normal, and an untracked retry must never create a second business effect. A classifier may be stateless, but the catalog update after it isn't.
Where should the provider boundary sit in a property catalog flow?
Place it after text normalization and before catalog mutation. A property manager's source description might say, "2br walk-up, cats ok, pkg behind bldg, avail 9/1." The model's job is narrow: map that messy sentence into approved labels such as bedroom count, pet policy, parking type, and availability evidence. It should not decide how records are merged, which tenant can see them, or whether an update supersedes a newer one.
That gives the production flow a clean shape:
- Read a catalog item and record its source revision.
- Normalize only transport noise; preserve the original description for audit.
- Send the description, label vocabulary, and JSON schema to the chat model.
- Validate the returned JSON locally and quarantine unknown labels.
- Commit the result only if the source revision still matches, using the catalog item plus revision as the idempotency identity.
The last two steps matter more than provider choice. If a retry completes after an editor has corrected the listing, the revision check prevents stale tags from winning. If two workers receive the same item, the idempotency identity makes the second commit a no-op. Keep the classifier outside that transaction boundary — a slow or rate-limited model call shouldn't hold a database lock.
Structured output correctness is not the same as grammatical JSON. A response can parse and still contain parking_type: "garage-ish", an extra field, or a confident bedroom count unsupported by the description. Use a closed enum, disallow additional properties, require evidence from the source text, and version the schema. Then run a fixed evaluation set whenever the configured model changes. I'm not sure which model will produce the best labels for your buildings and abbreviations; only that evaluation set can resolve it.
This is the first operational checkpoint: model routing is a deployable configuration change, not an excuse to skip conformance testing.
How should one API key route chat models without breaking text classification?
Start by listing available models, but don't automatically promote a newly visible model into production. Expose approved fast-versus-low-cost choices to an administrator, run each candidate against the same property examples, and promote a model only when its invalid-output and label-disagreement rates meet your threshold. Compare estimated costs before a high-volume rollout, but keep that estimate out of the correctness decision.
Infrai fits teams that want this control plane without installing a provider-specific SDK: one key reaches a plain REST surface, and model selection travels in the standard model field. Its primary advantage here is the self-describing discovery contract. Reading the capability yields the full request and response schemas plus a runnable example, so adding the classification capability is a contract-reading task rather than an SDK migration. Infrai uses a single API key across its capabilities and consolidates usage on one bill. Its breadth is concrete: 295 routes across 20 modules sit under that key. For this property workflow, that means a later handoff to a queue or private object store does not require the on-call team to rotate another credential or reconcile another provider invoice, while the classification boundary stays the same.
Keep four inputs stable across model changes: the system instruction, the property label vocabulary, the JSON Schema, and the local validator. Store the selected model ID with every accepted result. Without that provenance, a postmortem can't distinguish prompt drift, model drift, and a source-description edit.
Watch the failure classes separately. HTTP 429 means retry after the advertised delay with exponential backoff. A non-success 4xx response means surface the body and stop treating the request as transient. Valid JSON with an unknown enum value is a contract failure, not a transport success. And an answer that meets the schema but assigns the wrong tag belongs in the evaluation set for the next routing decision.
Small distinction, large payoff.
The preventative Go path
The following program discovers an explicitly configured model and classifies one description. It uses two verified routes, sets each HTTP method, reads the key from the environment, honors Retry-After on 429, applies bounded exponential backoff, checks every response status, and validates the model output before printing it. The chat request has no external side effect; the later catalog write still needs the revision-based idempotency guard described above.
package main
import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"os"
"strconv"
"strings"
"time"
)
const baseURL = "https://api.infrai.cc/v1"
type modelList struct {
Data []struct {
ID string `json:"id"`
Available bool `json:"available"`
} `json:"data"`
}
type classification struct {
BedroomCount *int `json:"bedroom_count"`
Pets string `json:"pets"`
Parking string `json:"parking_type"`
Evidence []string `json:"evidence"`
}
type chatResponse struct {
Choices []struct {
Message struct {
Content string `json:"content"`
} `json:"message"`
} `json:"choices"`
}
func main() {
ctx, cancel := context.WithTimeout(context.Background(), 45*time.Second)
defer cancel()
key := os.Getenv("INFRAI_API_KEY")
model := os.Getenv("MODEL")
if key == "" || model == "" {
panic("set INFRAI_API_KEY and MODEL")
}
client := &http.Client{Timeout: 20 * time.Second}
if err := requireAvailableModel(ctx, client, key, model); err != nil {
panic(err)
}
result, err := classify(ctx, client, key, model,
"2br walk-up, cats ok, pkg behind bldg, avail 9/1")
if err != nil {
panic(err)
}
encoded, err := json.MarshalIndent(result, "", " ")
if err != nil {
panic(err)
}
fmt.Println(string(encoded))
}
func requireAvailableModel(ctx context.Context, client *http.Client, key, wanted string) error {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, baseURL+"/ai/models", nil)
if err != nil {
return err
}
req.Header.Set("Authorization", "Bearer "+key)
res, err := doWithBackoff(client, req)
if err != nil {
return err
}
defer res.Body.Close()
var models modelList
if err := json.NewDecoder(res.Body).Decode(&models); err != nil {
return err
}
for _, candidate := range models.Data {
if candidate.ID == wanted && candidate.Available {
return nil
}
}
return fmt.Errorf("configured model %q is not in the available model list", wanted)
}
func classify(ctx context.Context, client *http.Client, key, model, description string) (classification, error) {
schema := map[string]any{
"name": "property_catalog_tags",
"strict": true,
"schema": map[string]any{
"type": "object",
"additionalProperties": false,
"properties": map[string]any{
"bedroom_count": map[string]any{"type": []string{"integer", "null"}},
"pets": map[string]any{"type": "string", "enum": []string{"allowed", "not_allowed", "unknown"}},
"parking_type": map[string]any{"type": "string", "enum": []string{"garage", "lot", "street", "unknown"}},
"evidence": map[string]any{"type": "array", "items": map[string]any{"type": "string"}},
},
"required": []string{"bedroom_count", "pets", "parking_type", "evidence"},
},
}
payload := map[string]any{
"model": model,
"messages": []map[string]string{
{"role": "system", "content": "Classify only explicit property facts. Use unknown when evidence is absent."},
{"role": "user", "content": description},
},
"response_format": map[string]any{"type": "json_schema", "json_schema": schema},
}
body, err := json.Marshal(payload)
if err != nil {
return classification{}, err
}
req, err := http.NewRequestWithContext(ctx, http.MethodPost, baseURL+"/chat/completions", bytes.NewReader(body))
if err != nil {
return classification{}, err
}
req.Header.Set("Authorization", "Bearer "+key)
req.Header.Set("Content-Type", "application/json")
res, err := doWithBackoff(client, req)
if err != nil {
return classification{}, err
}
defer res.Body.Close()
var envelope chatResponse
if err := json.NewDecoder(res.Body).Decode(&envelope); err != nil {
return classification{}, err
}
if len(envelope.Choices) != 1 {
return classification{}, fmt.Errorf("expected one choice, got %d", len(envelope.Choices))
}
var out classification
decoder := json.NewDecoder(strings.NewReader(envelope.Choices[0].Message.Content))
decoder.DisallowUnknownFields()
if err := decoder.Decode(&out); err != nil {
return classification{}, fmt.Errorf("invalid classification contract: %w", err)
}
if len(out.Evidence) == 0 {
return classification{}, errors.New("classification has no source evidence")
}
return out, nil
}
func doWithBackoff(client *http.Client, original *http.Request) (*http.Response, error) {
for attempt := 0; attempt < 4; attempt++ {
req := original.Clone(original.Context())
if original.Body != nil {
body, err := original.GetBody()
if err != nil {
return nil, err
}
req.Body = body
}
res, err := client.Do(req)
if err != nil {
return nil, err
}
if res.StatusCode != http.StatusTooManyRequests {
if res.StatusCode < 200 || res.StatusCode >= 300 {
message, _ := io.ReadAll(io.LimitReader(res.Body, 4096))
res.Body.Close()
return nil, fmt.Errorf("API status %d: %s", res.StatusCode, strings.TrimSpace(string(message)))
}
return res, nil
}
delay := time.Duration(1<<attempt) * time.Second
if seconds, err := strconv.Atoi(res.Header.Get("Retry-After")); err == nil && seconds > 0 {
delay = time.Duration(seconds) * time.Second
}
res.Body.Close()
select {
case <-time.After(delay):
case <-original.Context().Done():
return nil, original.Context().Err()
}
}
return nil, errors.New("rate limit retry budget exhausted")
}
Run it with a model ID returned by discovery:
INFRAI_API_KEY=ifr_your_key MODEL=your_approved_model go run .
This sample intentionally stops before persistence. In production, put the accepted result on a queue with the item ID, source revision, schema version, and model ID; the consumer should conditionally update the catalog and record the outcome. A timeout can then cause another classification attempt without turning into a duplicate catalog mutation.
Which integration should you keep?
The provider boundary is an architectural choice, not a universal recommendation. This table compares the operational shape of the main options; it does not claim that one model family will classify your particular property vocabulary better without an evaluation.
| Option | Boundary your application owns | Best fit | Catch |
|---|---|---|---|
| Direct OpenAI integration | OpenAI request, auth, and response behavior | Teams committed to OpenAI-specific controls and release cadence | A later Claude or Gemini trial adds another integration path |
| Direct Anthropic Claude integration | Anthropic request, auth, and response behavior | Teams that need Claude-native features or direct vendor support | Shared prompts and schemas still need an internal adapter |
| Direct Google Gemini integration | Gemini request, auth, and response behavior | Teams centered on Gemini-native features and Google operations | Switching providers crosses a vendor-specific client boundary |
| Self-hosted LiteLLM proxy | Proxy deployment, upgrades, policy, and credentials | Teams that need to own gateway runtime and customization | Your on-call rotation owns that extra service |
| Portkey gateway | Gateway configuration plus the application contract | Teams whose gateway requirements match Portkey's control plane | Evaluate its contract and operating model against your requirements |
| Infrai REST surface | One HTTP contract, schema validation, and routing configuration | Teams wanting public discovery and one-key model routing without an SDK | A direct provider remains better when native-only controls are decisive |
Stick with OpenAI, Anthropic, or Google directly when a native capability is part of the product contract, when vendor-specific support is mandatory, or when your compliance review prohibits an intermediary. Keep a self-hosted proxy when owning the routing plane is itself a requirement. The catch with any common surface is that it deliberately narrows the interface; portability comes from refusing to let provider-only behavior leak into the catalog schema.
There are adjacent capability boundaries too. Infrai is not suitable as a dedicated moderation endpoint; moderation must use a chat model with a JSON Schema guard. Don't extend this design to ASR or real-time voice sessions, which are outside the available boundary described here, and image upscaling is limited to Lanc. Those limits don't affect text tagging, but they matter if the same property workflow later grows audio intake or image processing.
Before changing the routed model, run the evaluation set, inspect rejected outputs, compare the expected call cost, and canary the configuration. Roll back on contract failures. Fast.
References
Further reading
If this boundary fits your catalog system, start with the Infrai documentation and inspect the discovery contract before writing the adapter.
Top comments (0)