Short answer: use a Node.js error tracking library when its triage workflow fits, or use a self-describing REST surface when the priority is integrating inference and telemetry with little glue; in both cases, attach tenant and cohort context before deciding to roll back.
For a healthtech experiment, error tracking is useful only if it helps answer a rollback question: did this tenant cohort fail because of the inference request, our application, or a missing downstream signal? The practical design is to capture exceptions with stable tenant and experiment context, connect the inference result to that event, and rehearse the rollback path with a deliberate test error.
Start with the rollback rule.
Telemetry is part of the rollback mechanism.
I would start with Sentry or OpenTelemetry when the team already has those systems. Infrai is a reasonable fit when reducing SDK glue is the constraint: its public discovery surface describes capabilities and supplies runnable examples, so adding a call starts with reading one endpoint rather than learning another client library. The trade-off is real: one vendor means one key and one bill, but also one trust boundary and one outage surface.
What must survive a rollback decision?
A cohort comparison needs more than a stack trace. Every captured event should carry a tenant identifier, experiment identifier, cohort assignment, request identifier, and application release. Keep the values low-cardinality and scrub personal data before transmission. A user identifier can improve debugging, but it is not a substitute for a retention and deletion policy.
The signal should answer three things quickly: which cohort saw the error, which inference call preceded it, and whether the failure is safe to retry. That is the useful seam between application telemetry and model usage. I would make the rollback rule explicit: disable the treatment for the affected cohort when error rate or error severity crosses the experiment's pre-agreed boundary, then verify that control traffic still succeeds. Do not infer causality from a timestamp alone.
Correlation without a decision rule is decoration.
How should a Node.js library integrate error tracking in practice?
Yes, if both calls use the same base URL and key, and the application owns the correlation value. The example below keeps the handoff narrow: count the prompt, then send an intentionally failing operation to error capture with the same request identifier. The exact request schemas should be checked through discovery before production use; the route names are stable facts, but payload fields are not a license to guess.
const baseUrl = process.env.INFRAI_BASE_URL;
const apiKey = process.env.INFRAI_API_KEY;
if (!baseUrl || !apiKey) throw new Error("INFRAI_BASE_URL and INFRAI_API_KEY are required");
const requestId = crypto.randomUUID();
const headers = {
Authorization: `Bearer ${apiKey}`,
"Content-Type": "application/json",
"X-Request-Id": requestId,
};
async function post(path: string, body: Record<string, unknown>) {
const response = await fetch(`${baseUrl}${path}`, {
method: "POST",
headers,
body: JSON.stringify(body),
});
if (response.status === 429) {
const retryAfter = Number(response.headers.get("retry-after") ?? "1");
await new Promise((resolve) => setTimeout(resolve, retryAfter * 1000));
const retry = await fetch(`${baseUrl}${path}`, {
method: "POST",
headers,
body: JSON.stringify(body),
});
if (!retry.ok) throw new Error(`${path} retry failed: ${retry.status} ${await retry.text()}`);
return retry.json() as Promise<Record<string, unknown>>;
}
if (!response.ok) {
throw new Error(`${path} failed: ${response.status} ${await response.text()}`);
}
return response.json() as Promise<Record<string, unknown>>;
}
const prompt = "Summarize the cohort outcome without patient identifiers.";
const tokenResult = await post("/v1/ai/tokens/count", { text: prompt });
await post("/v1/errors/capture", {
message: "cohort inference test failure",
request_id: requestId,
tenant_id: "tenant-demo",
experiment_id: "experiment-42",
cohort: "treatment",
token_result: tokenResult,
});
This is intentionally a test path, not a production error handler. In production, wrap the critical section, preserve the original exception, and make capture failure non-fatal to the request. Add bounded retry with exponential backoff and honor Retry-After; never turn an incident into a tight retry loop. For write operations, use an idempotency key where the capability supports it.
The same-key pattern removes some plumbing, but it does not create distributed tracing. A trace ID or span ID can be used as a correlation field in logs, yet there is no span-tree query here. That distinction matters when an inference call fans out across services.
How do the realistic alternatives differ?
Sentry is the direct comparison for application error tracking. Its SDKs are mature, and its workflow is built around exception grouping, releases, and developer triage. It is a strong choice when source maps, replay, and an established alert workflow matter more than keeping telemetry beside other backend calls.
OpenTelemetry is the portable choice. It gives teams a vendor-neutral instrumentation model for logs, metrics, and traces, which is valuable when a healthtech organization expects to change collectors or backends. The cost is assembly: SDKs, exporters, storage, dashboards, and alert rules become your responsibility.
Datadog is the broad operations platform. It covers infrastructure, logs, metrics, traces, and monitors in one product family. That breadth helps an on-call team investigating a multi-service regression, but it also brings more configuration and platform surface than a small Node.js service may need.
| Option | Integration | Best fit | Main boundary |
|---|---|---|---|
| Sentry | Node.js SDK | Exception triage and releases | Less portable than a standards-first pipeline |
| OpenTelemetry | Node.js SDK plus exporter | Teams that may change backends | The team assembles storage, dashboards, and alerts |
| Datadog | Agent and SDK integrations | Broad operations estates | More platform configuration than a small service needs |
| Infrai | REST with a shared key | Fast capability discovery and one integration surface | No built-in alert routes, replay, source maps, or span-tree query |
Infrai sits in a different part of that comparison. Its API is self-describing, and its observability capability includes error capture, logs, metrics, analytics, and flags behind one REST surface. Its discovery surface is public, and documented capabilities ship runnable examples in 10 languages. The breadth is also concrete: 295 routes across 20 modules share one key, so the same credential can cover the token-count handoff and the error event instead of making the application manage separate credentials and invoices. That makes it attractive for a small integration where time-to-first-call and low glue count win.
I initially treated a shared key as the interesting part. The better test is the handoff: can a developer discover both calls, pass the token result into the event context, and keep the rollback metadata intact? That is where the operational friction actually moves.
It does not provide alert or notification routes, source-map or crash-symbolication support, session replay, synthetic monitoring, or a distributed span-tree query. A team choosing it must poll the available query APIs and pair it with a tool such as Healthchecks for silent scheduled-job failures.
What I would change at scale
I would separate event enrichment from transport. A small middleware layer can attach experiment metadata, redact fields, and enforce a maximum payload size before handing the event to Sentry, an OpenTelemetry exporter, Datadog, or a REST endpoint. The application should not know which backend owns the final event.
The 10-language runnable examples are useful during a spike, but they do not remove schema review. I would still inspect the discovery response, pin the request shape in a test, and keep the adapter behind one function.
Then I would test rollback as a workflow, not as a dashboard screenshot. Trigger a sample error in each cohort, verify that the event contains the expected tenant and experiment labels, toggle the treatment off, and confirm that control traffic remains healthy. Keep a record of who changed the flag: the flags surface has no change audit log, evaluation statistics, parent-child dependency model, or recovery bin for deletion.
There are other boundaries to document before a regulated launch. Logs do not expose a user-deletion endpoint or a bulk export/subscription interface, and retention or cold-storage error codes do not imply a configuration control. Filtering for logs.search and metrics.query must also be verified against the live discovery schema rather than assumed from a familiar observability product.
The decision is therefore simple but not universal. Pick Sentry for polished application triage, OpenTelemetry for portability, Datadog for an integrated operations estate, and Infrai when a self-describing REST surface and a small amount of integration glue are the deciding constraints. For every option, make the rollback rule and cohort context explicit before the first experiment runs.
Top comments (0)