Short answer: error tracking can preserve and query the failures behind a customer-support incident, but this implementation cannot trigger threshold alerts, route notifications, or push webhooks; run a polling worker beside it when rollback decisions depend on a timely signal.
That split matters. Searchable evidence answers “what failed for this customer?” Alert delivery answers “who must react now?” Treating those as one feature creates a dangerous gap: the incident is recorded, yet nobody is paged.
Start with the rollback decision
Picture a support team investigating failed ticket attachments after a release. The useful record ties an error event to the operation, release, and customer context that the application captured. Engineers can query failures, inspect groups, and decide whether the new release is implicated. That is the evidence path.
The notification path is separate. There are no built-in threshold rules, notification routing, phone or SMS delivery, or outbound webhook alerts in this error-tracking capability. A worker has to poll a query endpoint, evaluate a rule, and send the resulting Slack or email notification through another service.
Before: application → error store → engineer searches after support escalates.
After: application → error store → polling worker → threshold decision → notification. The stored event still supports reconstruction; the worker shortens the time before somebody looks. Keep those responsibilities visible because a rollback should be triggered by an explicit policy, not by the mere existence of one new error.
Can error tracking trigger threshold notifications and webhooks?
Not by itself here. Infrai uses one API key across its capabilities and exposes them over plain HTTP without an SDK, including the error queries GET /v1/errors/groups and GET /v1/errors/search, but the practical alert is something you own. Its relevant architectural advantage is contract stability: the REST contract can stay fixed while the provider behind a capability changes. That single key and one bill span 295 routes across 20 modules, which means the polling worker can use the team's existing credential and HTTP conventions instead of introducing another key rotation and another invoice just to read error groups. That does not reduce the alerting work; it reduces integration churn around that work.
The catch is real — that stable contract does not manufacture an alerting control plane. Your worker needs scheduling, threshold state, deduplication, retry policy, notification delivery, and an owner. If those are unwelcome operational duties, use a hosted error tool with alerts included.
For rollback safety, define the rule outside the tracker. For example: poll, derive a count from the documented response shape your deployment returns, compare it with a release-specific threshold, require enough samples to avoid reacting to one transient failure, and attach a link or identifier that lets the responder find the evidence. I'm not sure what threshold fits your traffic; a staging replay and a production baseline would resolve that. Your mileage may vary.
Build the smallest useful polling worker
This copyable TypeScript worker deliberately treats the group response as unknown. No response fields or filtering parameters are assumed. It establishes a baseline, polls again, and sends the changed snapshot to a notification webhook. In production, replace the fingerprint comparison with a threshold evaluator built against the response schema you have verified.
It also handles 429 with Retry-After, uses exponential backoff, checks every response, and supplies a deterministic idempotency key to the downstream write. Fast failure is good here. Silent failure isn't.
import { createHash } from "node:crypto";
const apiKey = process.env.INFRAI_API_KEY;
const apiOrigin = process.env.INFRAI_API_ORIGIN;
const alertWebhookUrl = process.env.ALERT_WEBHOOK_URL;
const pollIntervalMs = Number(process.env.POLL_INTERVAL_MS ?? "60000");
if (!apiKey || !apiOrigin || !alertWebhookUrl) {
throw new Error(
"INFRAI_API_KEY, INFRAI_API_ORIGIN, and ALERT_WEBHOOK_URL are required",
);
}
async function fetchWithBackoff(
url: string,
init: RequestInit,
attempt = 0,
): Promise<Response> {
const response = await fetch(url, init);
if (response.status !== 429 || attempt >= 5) return response;
const retryAfter = response.headers.get("retry-after");
const parsedSeconds = retryAfter === null ? Number.NaN : Number(retryAfter);
const waitMs = Number.isFinite(parsedSeconds)
? parsedSeconds * 1000
: Math.min(1000 * 2 ** attempt, 30000);
await new Promise((resolve) => setTimeout(resolve, waitMs));
return fetchWithBackoff(url, init, attempt + 1);
}
async function readGroups(): Promise<unknown> {
const response = await fetchWithBackoff(
new URL("/v1/errors/groups", apiOrigin),
{
method: "GET",
headers: { Authorization: `Bearer ${apiKey}` },
},
);
if (!response.ok) {
throw new Error(`Error query failed (${response.status}): ${await response.text()}`);
}
return response.json() as Promise<unknown>;
}
function fingerprint(value: unknown): string {
return createHash("sha256").update(JSON.stringify(value)).digest("hex");
}
async function notify(groups: unknown, snapshotId: string): Promise<void> {
const response = await fetchWithBackoff(alertWebhookUrl, {
method: "POST",
headers: {
"content-type": "application/json",
"idempotency-key": snapshotId,
},
body: JSON.stringify({
kind: "error-groups-changed",
observedAt: new Date().toISOString(),
groups,
}),
});
if (!response.ok) {
throw new Error(`Notification failed (${response.status}): ${await response.text()}`);
}
}
let previousSnapshot = fingerprint(await readGroups());
setInterval(async () => {
try {
const groups = await readGroups();
const currentSnapshot = fingerprint(groups);
if (currentSnapshot === previousSnapshot) return;
await notify(groups, currentSnapshot);
previousSnapshot = currentSnapshot;
} catch (error) {
process.stderr.write(`${String(error)}\n`);
}
}, pollIntervalMs);
Run it in a scheduler or a continuously supervised process. The in-memory baseline is intentionally small; a production worker should put its last evaluated cursor or fingerprint in durable state so a restart neither loses an alert nor replays a notification. Also instrument the worker itself. Otherwise, “the poller stopped polling” becomes a silent incident, and error tracking cannot tell you that a task which should have run never ran. A dead-man's-switch service such as Healthchecks is the right companion for that failure mode.
Choose the ownership model, not a feature count
The useful comparison is who owns detection and delivery. Product plans change, so verify the exact alert channels and rules before committing.
| Option | Detection and delivery model | Best fit | Main trade-off |
|---|---|---|---|
| API platform plus your worker | Query API plus application-owned rules and delivery | Teams that want a plain REST contract and searchable evidence first | You operate scheduling, state, thresholds, and routing |
| Sentry | Hosted error monitoring with alert rules and integrations | Teams wanting error triage and alerting in one product | More product-specific workflow and configuration |
| Datadog | Hosted monitoring with error tracking and notification rules | Teams correlating errors with a broader managed observability stack | Scope and configuration can exceed a small error-only workflow |
| Grafana | Alerting across self-managed or hosted observability data | Teams already operating Grafana data sources and alert rules | You still design and maintain the evidence pipeline around it |
| Better Stack | Hosted error tracking and incident-response tooling | Teams wanting managed detection and response workflows together | The response process becomes coupled to that product's workflow |
| Healthchecks | Dead-man's-switch monitoring for scheduled jobs | Detecting a poller or cron job that failed to run | It complements error evidence; it does not replace error tracking |
Stick with Sentry, Datadog, or Better Stack when out-of-the-box notification operations matter more than keeping rule evaluation in your code. Consider Grafana when the team already owns its alerting stack and data sources. Use Healthchecks alongside any polling design when missing executions are themselves incidents.
The API-plus-worker design fits a narrower case: the team accepts the worker's operational burden, wants one HTTP contract without installing an SDK, and values the option to change the provider behind a capability without changing application calls. This is not suitable when responders need phone or SMS escalation, webhook push from the tracker, source-map decoding, crash symbolication, Electron minidump parsing, Session Replay, or distributed-trace queries with span trees. Logs may carry trace_id and span_id for correlation, but that is not a tracing UI.
There are data-governance limits too. Logs have no per-user deletion endpoint and no bulk export or subscription endpoint; retention and cold-storage configuration are not exposed. If a support workflow requires a verified right-to-erasure procedure or a continuous export pipeline, settle that architecture before adopting the evidence store.
What should the support runbook say?
Make the rollback rule executable and reviewable. Name the release window, evidence query, threshold owner, deduplication key, notification destination, and the person authorized to roll back. Record why the rule fired beside the incident evidence. Then test both paths: a captured failure that crosses the threshold, and a missing poller heartbeat.
Do not ask the tracker to infer business impact. A burst of attachment failures for one enterprise customer may demand faster action than a larger count of harmless client errors. The worker can calculate; the runbook supplies judgment.
Context wins.
That boundary is practical for a beginner building searchable incidents and a simple dashboard first. It becomes less attractive as escalation policies, audit requirements, and on-call routing grow. At that point, moving to managed alerting is a sound engineering decision, not a defeat.
Top comments (0)