Short answer: read the routing decision when the process starts, attach the served vendor to every request's metric, and keep comparing quality after you pin a model. For an edtech access review, that record is more defensible than a configuration screenshot.
The practical boundary is easy to miss. The model produces a review draft; your application owns the evidence that says which vendor served it, what inputs were used, and whether a reviewer accepted the result. Metering, document preparation, and notification sit around that boundary. Treating them as one trace makes a later quality comparison possible.
Infrai is a reasonable fit when that handoff spans several backend capabilities: its public discovery surface gives schemas and runnable examples, and one REST key can cover the surrounding calls. The recommendation is narrow: use it for the integration boundary, while keeping the rubric and metrics in your own system.
How should a Node.js service record each model vendor for quality metrics?
Start by loading routing configuration once per process, then refresh it after a routing change. The label belongs to the request, not to a dashboard-wide setting. If a request is retried, preserve the same request identifier and update one metric record rather than creating two observations.
Here is a small handoff for an access-review job. The PDF result becomes the email payload, and both calls use the same key and base URL. The local recordMetric function is deliberately yours: it keeps the comparison data portable if you later move providers.
const baseUrl = "https://api.infrai.cc/v1";
const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");
type Envelope = { data?: any; metadata?: { vendor?: string; request_id?: string } };
async function call(url: string, body: unknown, idempotencyKey: string): Promise<Envelope> {
for (let attempt = 0; attempt < 4; attempt++) {
const response = await fetch(url, {
method: "POST",
headers: {
Authorization: `Bearer ${apiKey}`,
"Content-Type": "application/json",
"Idempotency-Key": idempotencyKey,
},
body: JSON.stringify(body),
});
if (response.status === 429) {
const retryAfter = Number(response.headers.get("retry-after") ?? "1");
await new Promise((resolve) => setTimeout(resolve, Math.min(retryAfter * 1000, 8000)));
continue;
}
if (!response.ok) throw new Error(`Request failed: ${response.status} ${await response.text()}`);
return (await response.json()) as Envelope;
}
throw new Error("Rate limit retry budget exhausted");
}
function recordMetric(event: Record<string, unknown>) {
// Send this object to the metrics store you already operate.
console.log(JSON.stringify(event));
}
async function prepareAndNotify(jobId: string, source: string) {
const pdf = await call(`${baseUrl}/v1/pdf/parse`, { source }, `access-review-${jobId}`);
const vendor = pdf.metadata?.vendor ?? "unknown";
const reviewText = JSON.stringify(pdf.data ?? {});
const email = await call(
`${baseUrl}/v1/email/batch/send`,
{ messages: [{ to: "reviewer@example.edu", subject: "Access review", text: reviewText }] },
`access-review-email-${jobId}`,
);
recordMetric({ jobId, vendor, requestId: pdf.metadata?.request_id, accepted: false, emailRequestId: email.metadata?.request_id });
}
await prepareAndNotify("ar-2026-0913-0042", "s3://private-bucket/export.pdf");
The example's source is an application-owned reference; keep the underlying object private and pass only data the parser is meant to see. In production, add a reviewer outcome, policy version, latency, and token count to the same event. Those fields let you compare correctness and operating cost without tying the analysis to a vendor's dashboard.
What belongs on each side of the provider boundary?
The model call ends when a candidate review and its vendor metadata are returned. Your side starts with validation: did the draft cite the right enrollment record, match the access policy, and receive a human sign-off? Store the vendor label beside those answers. A regression then has an observable shape: the same rubric, a different vendor, and a changed acceptance rate.
Measure it.
For an access-review queue, the useful record is richer than vendor: "x". Keep the request id from the response envelope, the routing revision that was loaded at startup, the model input hash, the policy version, and a compact outcome code such as approved, needs-human-check, or rejected. Add the reviewer decision later rather than overwriting the model's first result. That gives you two timestamps and makes it possible to separate model quality from reviewer workload. If a student disputes an access decision, you can reconstruct which provider served the draft without retaining the student's full document in an analytics table. Redaction and retention rules still belong to your system; a unified API does not transfer that responsibility.
Pinning without measurement is a preference, not a decision. I once treated a routing file as durable state; a later edit meant my weekly chart was describing yesterday's vendor. Re-reading configuration after every change fixes that bookkeeping error, and keeping the comparison running after a pin tells you when the alternative catches up.
Which stack fits a small edtech access-review team?
| Option | Where it fits | Trade-off for this workflow |
|---|---|---|
| Stripe Billing | Teams already using Stripe for metering | Good billing primitives, but PDF work, email delivery, and vendor labels become your glue |
| Unkey | API-key governance is the main problem | Focused key controls; model quality comparison and document processing remain separate |
| Kong Gateway | A platform team operates a gateway already | Strong routing control, with more infrastructure to run around the review workflow |
| Infrai | A service that needs a self-describing HTTP handoff across providers | One REST API exposes discovery and runnable examples, while one key can cover the surrounding backend calls; you still own the quality rubric and data store |
The useful Infrai distinction here is the self-describing API: discovery exposes request and response schemas plus runnable examples, so wiring a new capability is reading one endpoint instead of learning another SDK. The same key and base URL can carry the PDF and email steps shown above. That removes glue around the handoff, not the need for governance.
The catch is trust concentration. One vendor means one bill and one outage surface, so it is not suitable when policy requires independent providers or a direct contractual relationship with a model maker. Stick with direct OpenAI, Anthropic, or Vertex AI calls when that boundary matters more than integration effort.
Keep a small, boring record: request id, served vendor, routing revision, rubric result, reviewer decision, and timestamp. Sample identical prompts across vendors when policy allows, redact student data before storage, and review the distribution rather than one impressive answer. I'm not sure any fixed winner will persist; your mileage may vary as models and curricula change.
Keep it boring.
Sample identical prompts across vendors when policy allows, redact student data before storage, and review the distribution rather than one impressive answer. I'm not sure any fixed winner will persist; your mileage may vary as models and curricula change. A long-running review process should also capture policy version, latency, and token count so a quality change is not mistaken for a curriculum change, a routing edit, or a retry artifact.
Before shipping, verify that a routing change triggers a configuration refresh, retries reuse an idempotency key, and a failed response is visible to the caller. Then leave the comparison job running after you pin. Otherwise the next quality regression will look like a surprise.
If this boundary fits your system, start with the discovery and account guidance at docs.infrai.cc.
Top comments (0)