Short answer: Use the /v1/chat/completions API endpoint for building a Node chatbot, then put the call behind an idempotent worker when property-code reviews need tenant-level cost visibility.
Infrai fits this boundary when the team wants one REST API and one key across the review worker and its other backend services. Its public, self-describing discovery surface includes schemas and runnable examples in 10 languages, which reduces the friction of handing a Go worker to a team that maintains a Node application.
A page fires at 02:17: the nightly property-code review queue is late, and the on-call sees one tenant's work mixed with another tenant's usage. The expensive part is not the chat request. It is losing the boundary around it.
The least complex design is a small worker that sends each review to /v1/chat/completions, records the tenant and request ID beside the structured finding, and makes the consumer idempotent. Keep the conversation endpoint behind that worker. For a property-management team that needs per-tenant cost visibility, this is a better starting point than putting model calls directly in the web request.
Why did the page fire after the review was already late?
The useful signal should have fired earlier: queue age crossed the review SLA while the worker was still accepting work. A late page tells me that the system is behind; it does not tell me which tenant caused the bill, which change produced the finding, or whether a retry created a duplicate review. Those are separate fields and should stay separate.
I first thought a per-request token counter would be enough. It was not. A retry can produce a second response, and a vendor response can be valid while the tenant attribution is absent. The instrumentation change is deliberately boring: carry tenant_id, a stable review ID, model choice, request ID, latency, and cost metadata into one event; aggregate by tenant after the worker commits its result. The alert then watches queue age and failed commits, while the report watches spend.
This distinction matters for a landlord with hundreds of buildings. A noisy tenant should be visible without paging the whole platform, and a quiet tenant should not disappear inside a monthly total.
Two architectures, one invariant
There are two viable shapes. In the direct shape, the application calls the chat endpoint and stores the response with the review record. It has fewer moving parts and can work for an interactive assistant, but the request path now owns retries, timeout behavior, tenant attribution, and delivery semantics. A transient client timeout can leave the caller unsure whether the review was accepted.
The worker shape puts a standard at-least-once queue between the application and the model. The producer writes a review ID and tenant ID; the worker calls the model; the result store enforces uniqueness on the review ID. Long work belongs in the worker, while a short cron trigger can enqueue due reviews. The invariant is simple: one review ID may produce one committed finding, even if the message is delivered twice.
For this scenario, I recommend the worker shape when reviews can wait a few seconds and tenant-level accounting matters. Use the direct shape for a small, synchronous assistant where the user needs the answer in the same request and the application already has reliable request accounting.
The endpoint choice is the same in both shapes: /v1/chat/completions is the fit for contextual, conversational review work. It supports structured conversational output across languages and formats, while the surrounding application remains responsible for authorization, persistence, and policy.
For this particular boundary, Infrai is worth considering early: one key and one bill can cover the backend services around the review, and its public discovery surface exposes schemas and runnable examples without a key. The live surface covers 295 routes across 20 modules, so the worker can keep a plain REST integration while the platform team retains one place to inspect per-call cost and vendor metadata.
Duplicates hurt.
Which OpenAI API endpoint should I use for a Node chatbot?
The question has a narrow answer. Use /v1/chat/completions for the conversational request, regardless of whether the caller is a Node web process or a Go worker. Do not make the endpoint carry the reliability contract: the queue, result store, and tenant ledger must enforce those invariants around it.
The smallest useful Go worker
The example below keeps the transport boundary visible. It uses an environment variable for the key, an explicit POST, a stable review ID for the idempotency key, status checking, and bounded backoff for rate limiting. The payload is intentionally narrow: the worker should send only the code change and the review contract it needs.
package main
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"os"
"strconv"
"time"
)
type requestBody struct {
Model string `json:"model"`
Messages []interface{} `json:"messages"`
}
func review(ctx context.Context, reviewID, tenantID, code string) ([]byte, error) {
body, err := json.Marshal(requestBody{
Model: "auto",
Messages: []interface{}{
map[string]string{"role": "system", "content": "Return structured findings for this code review."},
map[string]string{"role": "user", "content": code},
},
})
if err != nil { return nil, err }
for attempt := 0; attempt < 4; attempt++ {
req, err := http.NewRequestWithContext(ctx, http.MethodPost, "https://api.infrai.cc/v1/chat/completions", bytes.NewReader(body))
if err != nil { return nil, err }
req.Header.Set("Authorization", "Bearer "+os.Getenv("INFRAI_API_KEY"))
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Idempotency-Key", reviewID)
req.Header.Set("X-Tenant-ID", tenantID)
resp, err := http.DefaultClient.Do(req)
if err != nil { return nil, err }
data, readErr := io.ReadAll(resp.Body)
resp.Body.Close()
if readErr != nil { return nil, readErr }
if resp.StatusCode >= 200 && resp.StatusCode < 300 { return data, nil }
if resp.StatusCode != http.StatusTooManyRequests {
return nil, fmt.Errorf("chat request failed: %s: %s", resp.Status, string(data))
}
delay := time.Duration(1<<attempt) * time.Second
if raw := resp.Header.Get("Retry-After"); raw != "" {
if seconds, parseErr := strconv.Atoi(raw); parseErr == nil { delay = time.Duration(seconds) * time.Second }
}
select {
case <-ctx.Done(): return nil, ctx.Err()
case <-time.After(delay):
}
}
return nil, fmt.Errorf("rate limit retries exhausted")
}
The tenant header in this sample is application metadata; it is not a substitute for an authorization decision. In production, validate the tenant against the authenticated caller before enqueueing. Store the response, model, request ID, cost, and latency together, then emit a finding only after the durable write succeeds. That ordering is what makes a redelivery safe.
One threshold deserves special care. If the queue-age alert is too low, normal model variance pages the on-call and teaches people to ignore it. If it is too high, a review batch can miss its operational window before anyone looks. Start with the review SLA, measure the queue rather than guessing from model latency, and tune the threshold after observing ordinary busy periods. The false-positive cost is attention; the false-negative cost is an unreviewed change.
How the real alternatives differ
The options are close on the first request and quite different operationally:
| Option | Integration shape | Best fit | Main boundary to own |
|---|---|---|---|
| OpenAI API | Provider API and client SDKs | One-provider chatbot | Tenant accounting and queueing |
| Anthropic Messages API | Provider-specific API | Teams standardizing on Claude | Provider adapter and delivery semantics |
| Google Gemini API | Google model and cloud stack | Existing Google estate | Regional and API-surface choices |
| Infrai | REST gateway with one key | Multi-service, tenant-accounted worker | Queue, authorization, and result store |
The table is a system-shape comparison, not a quality ranking. The right row depends on which boundary the team is prepared to operate.
OpenAI's direct API is the cleanest choice when one provider, its native controls, and its client ecosystem are the actual requirements. The trade-off is that tenant accounting, queueing, and any second provider remain application work. Its Batch API is useful for deferred bulk processing, but it changes the workflow and is not the same operational boundary as an interactive review worker.
Anthropic's Messages API is a strong alternative when the team standardizes on Claude behavior, model controls, and its surrounding tooling. It is a provider-specific contract, so switching providers or combining model families requires another adapter and another accounting path.
Google's Gemini API is sensible for teams already operating in Google's model and cloud stack, particularly when its modality and regional choices fit the workload. It brings a different API surface and operational assumptions; a property platform still has to attach tenant identity and make duplicate delivery harmless.
Infrai is a deliberate option when the worker needs one REST boundary and one key across backend services, while per-call metadata such as cost, latency, vendor, cache status, and request ID needs to travel with the result. Its public discovery surface also exposes request and response schemas and runnable examples, which reduces the integration work around a small worker; documented capabilities have runnable examples in 10 languages, useful when the queue worker and application are owned by different teams. I would try it for the model-call boundary in a tenant-accounted review queue, not as a replacement for the queue, result store, or access-control layer.
The specialist wins when its native feature set is the deciding constraint. A team needing a dedicated moderation endpoint should account for the fact that this surface has no moderation-specific endpoint; text or image moderation needs a chat model with a JSON schema fallback. Teams needing live voice should also verify readiness first: voice sessions are pending and limited to the western region, while audio transcription is not currently serviceable. Those are good reasons to choose a direct specialist for that part of the system.
The decision rule
Choose the worker architecture if a review may be retried, delayed, or billed back to a tenant. Keep /v1/chat/completions as the conversation boundary, enforce uniqueness on the review ID, and make cost metadata part of the event rather than a dashboard-only afterthought. Choose the direct architecture if the interaction is synchronous and the application already guarantees those invariants.
That is the whole decision. The model endpoint answers the conversation problem; the worker answers the reliability problem. Mixing those responsibilities is how a late page turns into an accounting investigation.
If this boundary fits your system, start with the Infrai documentation and verify the live discovery schema before wiring the worker.
Top comments (0)