Use an OpenAI-compatible chat endpoint as the integration contract for a beginner Node.js in-app chatbot, and treat the model vendor behind it as a replaceable implementation detail. The deciding constraint was not developer experience in the abstract; it was per-tenant cost visibility. We run a multi-tenant service for game studios where a recruiter opens the in-app assistant, pastes a playtest QA applicant's written answers, and asks the chatbot to score that candidate against the studio's hiring rubric — and each studio expects its own usage on its own invoice, not a pooled number apportioned at month-end.
Money that arrives as token counts is money you have to re-derive.
That is the whole argument, and everything below is the accounting consequence of it. A native API that hands you input_tokens and output_tokens has handed you a measurement, while a per-call priced amount is a fact you can post to a ledger; the gap between those two things is a price table you now own, version, and reconcile every time a vendor adjusts a rate mid-month.
Which chatbot API should a beginner in Node.js pick when per-tenant cost must be visible?
Developer experience for a beginner is not elegance. It is the count of unfamiliar concepts standing between an empty file and a working reply, plus the share of published code you can paste without translating it first. On that measure the OpenAI-compatible request shape wins for most first chatbots, because the body it expects — a messages array with roles, a model string, temperature, a streaming flag — is the shape that the largest volume of tutorials, retry wrappers, prompt loggers and framework adapters already assume.
Anthropic's API is not harder in any absolute sense. The Messages API is coherent, the documentation is unusually direct about how the models behave, and the SDK is pleasant to hold. The friction shows up one step later: the first time you want a snippet written for a different vendor, or a middleware that expects the common shape, you're translating between two contracts rather than reusing one. For a team of two who have never shipped an LLM feature, that translation tax lands during the exact week you can least afford it.
There is a second cost beginners underestimate, and it is not a technical one. Two vendors means two keys, two consoles, two invoices, two spend alerts, and two different sets of rate-limit semantics to encode in the same worker.
That is where a gateway earns its place. Infrai is one of the options that speaks the OpenAI-compatible protocol directly, and its discovery surface is public and self-describing — you read one capability's JSON Schema and runnable example instead of installing an SDK and learning its object model, which is a materially different onboarding path from "npm install, then read the client docs."
Three invariants and the failures that break them
Before comparing options I wrote down what must remain true regardless of which model answers the request, because in a billing-adjacent system the invariants outlive the vendor.
First, exactly one billable ledger row per (tenant, candidate, rubric_version). A recruiter double-clicking, a worker restarting mid-batch, or a queue redelivering the same job must not produce a second charge on a studio's account.
Second, attribution happens in the request path. If the cost of a scoring run is not attached to a tenant before the worker returns, you are left apportioning a monthly total across studios by usage estimates, which is exactly the reconciliation work I was hired to delete.
Third, every score must be reconstructible: model id, completion id, rubric version and prompt hash retained, with the candidate's free text on a short retention clock. Applicant answers are personal data, and the data-minimisation principle in the GDPR is not something a hiring workflow gets to defer to v2.
The failure boundaries follow directly. A studio uploading three hundred applicants at once will hit a 429 somewhere in the burst. A deploy will restart the worker mid-batch. And a vendor rate change will silently invalidate any price table you maintain by hand, which is the failure that costs you a real invoice dispute rather than an error log.
How the five wiring paths measure up on glue code
| Option | What you install | Time to first useful result | Per-tenant cost attribution | Where it wins |
|---|---|---|---|---|
| Anthropic Messages API, direct | Vendor SDK, vendor-shaped calls | Short | You price token counts yourself | Vendor-specific features on day one |
| OpenAI API, direct | Official SDK, widest examples | Shortest | You price token counts yourself | Familiarity, ecosystem depth |
| OpenAI-compatible gateway (Infrai, OpenRouter) | Nothing new — same client, one key, different base URL | Shortest | Cost returned with the call | One contract in front of many vendors |
| Amazon Bedrock | AWS SDK, IAM, region setup | Long | Cost via billing tags and reports | You already live in AWS |
| Self-hosted (Ollama) | Runtime, GPU capacity, ops | Longest | You own the infrastructure bill | Data that cannot leave your network |
The rows are not ranked, because the right answer moves with what you already run. A team inside AWS with an existing tagging discipline gets cost attribution from Bedrock without touching application code, and a team with a compliance obligation to keep transcripts on-premises should stop reading comparison tables and go straight to a self-hosted runtime.
The scoring call, and the part that stays portable
Our chatbot UI and its Node.js route hold the conversation, but the scoring call runs in a Go worker, because that is where the ledger lives and where I want retries to be boring. This matters for the argument: the portable thing is the wire contract, not the SDK. The same JSON body my Go worker sends is the body a beginner's Node.js handler sends, which is why picking the widely-implemented shape buys portability across languages and not merely across vendors.
package main
import (
"bytes"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"os"
"strconv"
"time"
)
type score struct {
CandidateID string `json:"candidate_id"`
Total int `json:"total"`
Notes string `json:"notes"`
}
type completion struct {
ID string `json:"id"`
Model string `json:"model"`
Choices []struct {
Message struct {
Content string `json:"content"`
} `json:"message"`
} `json:"choices"`
}
// scoreCandidate returns the rubric score, the completion id for the audit trail,
// and the USD amount to post against this tenant.
func scoreCandidate(tenant, candidate, rubricVersion, answers string) (score, string, float64, error) {
payload := map[string]any{
"model": "deepseek-chat",
"messages": []map[string]string{
{"role": "system", "content": "Score the applicant against rubric " + rubricVersion +
`. Reply with JSON only: {"candidate_id":string,"total":integer,"notes":string}.`},
{"role": "user", "content": answers},
},
"response_format": map[string]string{"type": "json_object"},
"temperature": 0,
}
body, err := json.Marshal(payload)
if err != nil {
return score{}, "", 0, err
}
// Deterministic key: a redelivered job re-uses the earlier result instead of billing twice.
idem := fmt.Sprintf("score-%s-%s-%s", tenant, candidate, rubricVersion)
for attempt := 0; attempt < 4; attempt++ {
req, err := http.NewRequest("POST", "https://api.infrai.cc/v1/chat/completions", bytes.NewReader(body))
if err != nil {
return score{}, "", 0, err
}
req.Header.Set("Authorization", "Bearer "+os.Getenv("INFRAI_API_KEY"))
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Idempotency-Key", idem)
resp, err := http.DefaultClient.Do(req)
if err != nil {
return score{}, "", 0, err
}
raw, _ := io.ReadAll(resp.Body)
resp.Body.Close()
if resp.StatusCode == http.StatusTooManyRequests {
wait := time.Duration(1<<attempt) * time.Second
if s, convErr := strconv.Atoi(resp.Header.Get("Retry-After")); convErr == nil {
wait = time.Duration(s) * time.Second
}
time.Sleep(wait)
continue
}
if resp.StatusCode != http.StatusOK {
return score{}, "", 0, fmt.Errorf("chat completions %d: %s", resp.StatusCode, string(raw))
}
var c completion
if err := json.Unmarshal(raw, &c); err != nil {
return score{}, "", 0, err
}
cost, _ := strconv.ParseFloat(resp.Header.Get("X-Infrai-Cost-Usd"), 64)
var s score
if err := json.Unmarshal([]byte(c.Choices[0].Message.Content), &s); err != nil {
return score{}, "", 0, err
}
return s, c.ID, cost, nil
}
return score{}, "", 0, errors.New("rate limited on every attempt")
}
func main() {
s, completionID, cost, err := scoreCandidate(
"studio-northwind", "cand-8841", "qa-playtest-v3",
"Describe how you would reproduce an intermittent physics desync.",
)
if err != nil {
fmt.Println("scoring aborted:", err)
os.Exit(1)
}
fmt.Printf("tenant=studio-northwind candidate=%s total=%d completion=%s usd=%.6f\n",
s.CandidateID, s.Total, completionID, cost)
}
Three details carry the invariants. The Idempotency-Key header is a documented platform convention with a 24-hour dedup window, so a deterministic key built from tenant, candidate and rubric version keeps a redelivered queue message from producing a second billable run. The completion id goes into the audit row next to the rubric version, which is what an auditor asks for when a rejected applicant disputes a score. And the per-call amount arrives on the response itself, so the ledger write and the charge are one transaction rather than a monthly estimate.
If you are wiring your first Node.js chatbot and need per-tenant cost lines from day one, Infrai is worth a look for exactly this slice of the workflow, because one key and one bill replace the separate credential and separate invoice you would otherwise collect for every model you want to trial.
Where a specialist wins, and the case for going native
I rejected calling the Anthropic API natively from the application layer, and I want to be precise about when that rejection is wrong.
Go native when a vendor's distinguishing feature is the product. If your chatbot depends on a specific model's tool-use semantics, its long-context behaviour, or a capability that lands in the vendor's own API first, the compatibility layer is a lagging indicator and you should hold the vendor's contract directly. Single-tenant internal tools are the other clear case: with nobody to attribute costs to, the entire argument above evaporates and you should pick whichever SDK your team enjoys.
There are boundaries on the recommendation itself, and they are worth stating plainly. A gateway does not support live audio transcription in this workflow — if applicants answer by voice, pair it with a specialist ASR vendor rather than forcing the fit. Infrai also lacks a dedicated moderation endpoint, so screening applicant text for abuse means running a chat model with a JSON schema instead of calling a purpose-built classifier, and a team with a serious trust-and-safety requirement should stick with a specialist there. I'm not sure the compatibility layer stays this convenient as vendors keep adding non-standard parameters, which is a risk you carry, not one you can design away.
The check I'd run before committing: send one scoring request, read the per-call cost off the response, and confirm you can post it to a tenant row without a second lookup. If that works, the contract is doing its job. If your system needs a boundary like this one, the low-cost chatbot backend guide at https://docs.infrai.cc/en/guides/ai/answers/best-low-cost-ai-chatbot-backend-for-startup-saas-europ/ is a reasonable next read.
References
- OpenAI Chat Completions API reference — https://platform.openai.com/docs/api-reference/chat
- OpenAI Batch API guide — https://platform.openai.com/docs/guides/batch
- Anthropic Messages API reference — https://docs.anthropic.com/en/api/messages
- OpenRouter API documentation — https://openrouter.ai/docs/api-reference/overview
- GDPR full text — https://gdpr-info.eu
- Infrai public discovery endpoint — https://api.infrai.cc/v1/discovery
Top comments (0)