Instrument the agent loop at one server-side boundary, attach every signal to a deploy version, and keep the first implementation small enough to remove in one commit. TL;DR: choose a lightweight capture path when the job is limited to exceptions plus a few latency and usage measurements; choose a broader error-tracking system when investigation workflow, grouping, and retained diagnostic context are already requirements. For a healthtech agent, rollback safety is the deciding constraint. A dashboard that cannot distinguish the current release from the previous one is decoration.
This is deliberately narrower than “observe everything.” The useful question is whether a weekly release changed failed runs, end-to-end latency, or model usage enough to roll back. Browser replay and source maps do not answer that server-side question. They may solve other jobs, but buying those jobs before they exist spends both money and the scarcer resource: maintenance hours that could ship the next feature.
Should a Next.js backend use a lightweight error capture API?
Yes, if the backend's current job is capture rather than case management. An agent request is a small distributed workflow even when it lives behind one backend route. It can make several model calls, invoke tools, retry, and then fail after most of the work has already happened. Capturing only the final thrown exception loses the shape of that run. Capturing every prompt, response, and tool payload creates a different problem: sensitive health context can leak into telemetry, and high-cardinality fields make aggregation harder. The useful middle is an envelope with a stable run ID, release ID, route name, outcome, duration, and usage counters. Keep patient text, generated answers, tool arguments, email addresses, and record identifiers out of it. The exception should be normalized to a short class and message only after a redaction policy has run. A run ID can join controlled application logs during an investigation without turning the error store into a second clinical-data store. This boundary is the real selection test: if a team needs issue ownership and coordinated triage now, a bare endpoint is too limited; if one operator only needs release evidence, the larger workflow adds surface area without improving the rollback call.
Keep it narrow.
Release identity is mandatory. Suppose release 2026.10.08.1 raises the failed-run ratio while the preceding release remains quiet. That contrast supports a rollback. Without the release field, the same errors become a blended time series, and the operator has to guess whether traffic mix or new code caused the movement.
Keep labels bounded. Prometheus instrumentation guidance warns against labels whose values have high cardinality; user IDs, email addresses, and other unbounded values are poor metric labels. The same discipline helps a capture API. Use a short allowlist such as route, release, outcome, and model_family, then leave the run ID in event data rather than in metric dimensions.
The smallest useful capture path
The implementation needs two outputs. One event records an exception with deployment context. One summary records the completed run, successful or not, so latency and model usage do not disappear on the happy path. Both writes should be bounded and best-effort: observability must not convert a report failure into a failed health workflow.
The following TypeScript keeps the application coupled to a tiny interface rather than to a transport. The endpoint, authentication method, retention policy, and storage engine remain deployment choices.
type AgentRunSummary = {
runId: string;
release: string;
route: "care-plan-draft";
outcome: "ok" | "error";
durationMs: number;
inputTokens: number;
outputTokens: number;
modelCalls: number;
};
type CapturedException = {
runId: string;
release: string;
route: "care-plan-draft";
errorType: string;
message: string;
};
interface TelemetrySink {
recordRun(summary: AgentRunSummary): Promise<void>;
recordException(event: CapturedException): Promise<void>;
}
const safeMessage = (error: unknown): string => {
if (!(error instanceof Error)) return "Unknown error";
return error.message
.replace(/[\w.+-]+@[\w.-]+\.[A-Za-z]{2,}/g, "[redacted-email]")
.slice(0, 240);
};
async function withAgentTelemetry<T>(
sink: TelemetrySink,
release: string,
execute: (usage: { input: number; output: number; calls: number }) => Promise<T>,
): Promise<T> {
const runId = crypto.randomUUID();
const startedAt = performance.now();
const usage = { input: 0, output: 0, calls: 0 };
let outcome: AgentRunSummary["outcome"] = "ok";
try {
return await execute(usage);
} catch (error) {
outcome = "error";
const event: CapturedException = {
runId,
release,
route: "care-plan-draft",
errorType: error instanceof Error ? error.name : "UnknownError",
message: safeMessage(error),
};
await Promise.race([
sink.recordException(event).catch(() => undefined),
new Promise<void>((resolve) => setTimeout(resolve, 200)),
]);
throw error;
} finally {
const summary: AgentRunSummary = {
runId,
release,
route: "care-plan-draft",
outcome,
durationMs: Math.round(performance.now() - startedAt),
inputTokens: usage.input,
outputTokens: usage.output,
modelCalls: usage.calls,
};
void sink.recordRun(summary).catch(() => undefined);
}
}
The 200 millisecond race is an explicit engineering budget, not a universal constant or a benchmark. It limits how long exception delivery can delay propagation of the original error. The summary send is fire-and-forget, which means it can be lost if the process exits immediately. That is acceptable only for the smallest version. If losing summaries would corrupt a rollback decision, write them to a local durable queue or an existing message system before returning.
There is another sharp edge: redacting an email pattern does not make arbitrary clinical text safe. The code therefore never accepts prompts or outputs in its telemetry types. Type-level omission is easier to review than a promise to scrub an open-ended payload later.
Turn events into a release decision
Raw events are evidence, not a policy. Before shipping, define a small release comparison over equivalent windows. For each release, calculate completed runs, failed runs, duration percentiles, total input and output tokens, and model calls per completed run. Cost can be computed downstream from usage under the applicable model contract; the capture layer does not need a changing price table.
A practical rollback rule needs both a floor and a relative comparison. For example, evaluate only after 100 completed runs, then pause rollout if the failed-run ratio is at least 2 percentage points higher than the previous release. Those are illustrative policy values, not observed healthtech results. Pick values from the service's risk tolerance and traffic, write them down before deployment, and test the query against a known fixture.
Why require a sample floor? One error in the first two runs looks like a 50% failure ratio. It is urgent evidence for an operator, but a poor automatic comparison. Why keep the absolute difference? A relative multiplier becomes noisy when the baseline is near zero. The exact statistical method can become more sophisticated later; the first version mainly needs to prevent an ambiguous dashboard from authorizing an automatic rollback.
Latency deserves similar care. Compare a percentile rather than an average so a few slow agent loops stay visible, but keep the percentile definition and window fixed across releases. Web performance guidance uses the 75th percentile for Core Web Vitals. That does not make p75 a universal backend threshold; it is a useful example of defining a percentile and evaluation boundary explicitly. For this agent route, the team should choose the percentile that matches its own wait-time and rollback policy.
Cost stays secondary. A release that uses fewer tokens but produces more failed runs is not an improvement. The revenue-per-hour view is simple: collect enough usage to expose regressions, but do not spend a week building billing-grade allocation before the service needs it.
Ship that first.
Choosing the operational surface
The selection is less about feature count than about who will operate the loop next Tuesday. A lightweight capture endpoint fits when one person can answer incidents from a release comparison, a bounded event view, and controlled logs. Its appeal is a small schema and a replaceable adapter. Its burden is ownership: retries, retention, alert delivery, deduplication, access control, and deletion still exist even if the interface has only one method.
A full error-tracking system fits when the investigation workflow itself is the undifferentiated work worth outsourcing. Automatic grouping, assignment, notification routing, richer context, and issue lifecycle can save operator time, but only count capabilities the service will actually use. With no browser replay and no source-map requirement, those two features should carry zero weight in the scorecard rather than becoming reasons for or against an entire category.
| Operating need | Lightweight capture boundary | Full tracking workflow |
|---|---|---|
| Release-level failure and latency comparison | Direct fit | Possible, with more configuration |
| Assignment and multi-person issue triage | Too limited | Direct fit |
| Custom retention and storage control | Operator owns it | Constrained by the chosen deployment |
| Lowest maintenance burden | Poor fit once queues and alerting are required | Better fit when those jobs are outsourced |
Neither shape is universally simpler. The lightweight route becomes a bad fit when its owner starts rebuilding durable delivery, deduplication, alert routing, and an issue inbox. The broader system is a bad fit when setup and ongoing administration consume more operator time than the few weekly exceptions it handles. That trade-off belongs in the decision record, because “self-serve” describes acquisition, not the hours required to operate the result.
Use a short trial with the same synthetic failures in each candidate. Verify that two releases can be separated, redacted fields stay absent, duplicate delivery does not inflate the run count, a timeout preserves the original route error, and exported data is sufficient to rebuild the rollback view. Also measure operator minutes: time from alert to identifying the release and deciding rollback or continue. That metric speaks directly to shipping weekly.
The winning shape is the one that shortens a safe decision while keeping the application adapter boring. No vendor name is required to reach it.
What I would change at scale
At higher traffic, the in-process best-effort send becomes the first component to replace. Put a durable buffer between the route and the telemetry consumer, make the run summary idempotent on runId, and monitor queue age. Sampling can reduce successful-run volume, but retain all failures and record the sampling rule alongside aggregates so release comparisons remain interpretable.
I would also separate three views: service health, release comparison, and individual investigation. Metrics answer whether failure or latency changed. Events explain a particular exception. Controlled logs trace a run when deeper access is justified. Forcing all three jobs into one payload makes privacy review and cardinality control harder.
Then stop. Ship the next weekly release. Add tracing across tool calls only when the single run envelope cannot locate the slow or failing stage; add an issue workflow only when manual triage consumes meaningful operator time. The architecture can grow without changing the route because the TelemetrySink boundary already owns that choice.
Rollback safety remains the test. Can an operator compare releases, trust the denominator, identify the failing class, and revert without exposing health content? If yes, the capture system is doing its job. More telemetry is optional.
Top comments (0)