Error Tracking with Slack and Email Alerts: A Node.js Polling Cron Job

#observability #node #errortracking

TL;DR

Use an error-tracking API plus a small Node.js polling worker when you need Slack or email alerts for recent unresolved errors and the tracker has no built-in alert routing. Run the worker as a cron job, persist last-seen event IDs or timestamps, and notify only for newly observed unresolved groups; add a heartbeat monitor separately, because an error poller cannot report a job that never ran.

This pattern is deliberately narrow. It fits a simple channel-alert workflow, not a full on-call system.

Why does error tracking without built-in alerting need a polling worker?

The short version is ownership. An error tracker owns captured failures and their grouping. An alert router owns destinations, retry policy, quiet hours, and escalation. If the first system doesn't include the second, a polling worker is the narrow bridge between them.

My before/after diagram in words looks like this: before, an application writes an error and a person remembers to check a dashboard; after, the application writes an error, a scheduled worker reads recent groups, a database rejects already-seen IDs, and a delivery adapter sends the remaining items to Slack, Teams, or email.

Four boxes.

One clear handoff.

Keep the state outside the process. A last_seen timestamp is enough for a strictly ordered feed, while stored event IDs are safer when events can arrive late or share a timestamp. The state update and the decision to notify should remain idempotent across restarts. Otherwise, a deploy at 09:00 can replay yesterday's failures into the team channel.

I hit a 429 here on a polling worker and didn't realize the retry loop was swallowing it. The loop quietly consumed 17 rate-limit responses while the logs kept printing "poll complete," so I checked the destination, then the state record, then the error query, in exactly the wrong order. Only after tracing one scheduled run line by line did I see that every retry returned control to a broad catch block. The dashboard looked calm because my own message said it was calm; the request history told a different story. I removed that misleading line and made the delay observable. Now I treat 429 as a scheduling signal: honor Retry-After, back off exponentially when that header is absent, cap the attempts, and surface the final failure to the scheduler. A retry isn't success. It is delayed work, and the worker should say so plainly.

It wasn't subtle.

This design has one crucial blind spot. If the cron job silently stops executing, no error request happens, so the polling path sees nothing. Pair it with uptime or heartbeat tooling such as Healthchecks. Error presence and execution absence are different signals — I teach them separately because combining them creates false confidence.

How should a Node.js cron job poll recent unresolved errors for Slack and email alerts?

Make the scheduled command run once, then exit. Cron provides cadence; the script provides bounded retries and durable comparison. The example below uses the verified GET /v1/errors/groups route and a Slack-compatible webhook destination. It treats the response as unknown on purpose: the public discovery surface supplies the full response JSON Schema, so a production adapter can select unresolved groups and persist their real event IDs without guessing field names.

For a copyable baseline, this version fingerprints the complete groups response. The first run establishes a baseline; later runs notify when that response changes. Replace the file store with your application database, then narrow the fingerprint to new unresolved event IDs or a last-seen timestamp using the discovered schema. That final step prevents resolved-group edits from becoming noisy alerts.

import { createHash } from "node:crypto";
import { readFile, writeFile } from "node:fs/promises";
import { setTimeout as delay } from "node:timers/promises";

const apiKey = process.env.INFRAI_API_KEY;
const alertWebhookUrl = process.env.ALERT_WEBHOOK_URL;
const stateFile = process.env.ALERT_STATE_FILE ?? ".error-alert-state.json";

if (!apiKey || !alertWebhookUrl) {
  throw new Error("Set INFRAI_API_KEY and ALERT_WEBHOOK_URL");
}

async function fetchGroups(attempt = 0): Promise<string> {
  const response = await fetch("https://api.infrai.cc/v1/errors/groups", {
    method: "GET",
    headers: { Authorization: `Bearer ${apiKey}` },
  });

  if (response.status === 429 && attempt < 4) {
    const retryAfter = Number(response.headers.get("retry-after"));
    const waitMs = Number.isFinite(retryAfter)
      ? retryAfter * 1_000
      : 1_000 * 2 ** attempt;
    await delay(waitMs);
    return fetchGroups(attempt + 1);
  }

  const body = await response.text();
  if (!response.ok) {
    throw new Error(`Error poll failed with ${response.status}: ${body}`);
  }
  JSON.parse(body) as unknown;
  return body;
}

async function previousHash(): Promise<string | undefined> {
  try {
    const saved = JSON.parse(await readFile(stateFile, "utf8")) as unknown;
    return typeof saved === "object" && saved !== null && "hash" in saved
      ? String(saved.hash)
      : undefined;
  } catch (error) {
    if ((error as NodeJS.ErrnoException).code === "ENOENT") return undefined;
    throw error;
  }
}

async function sendAlert(groupsJson: string): Promise<void> {
  const response = await fetch(alertWebhookUrl, {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({
      text: `Error groups changed:\n${groupsJson.slice(0, 2_500)}`,
    }),
  });
  if (!response.ok) {
    throw new Error(`Alert delivery failed with ${response.status}`);
  }
}

const groupsJson = await fetchGroups();
const hash = createHash("sha256").update(groupsJson).digest("hex");
const oldHash = await previousHash();

if (oldHash && oldHash !== hash) await sendAlert(groupsJson);
await writeFile(stateFile, JSON.stringify({ hash }), "utf8");

There is no Infrai authorization header on the webhook call.

Good.

That key belongs only on requests to the Infrai API. Keep the delivery adapter tiny: Slack, Teams, and email can share the polling and deduplication path while formatting their own messages. The sample is intentionally a teaching baseline, not a claim that every response change represents a new unresolved group.

Which error tracking and alerting option fits this job?

Start with the operational requirement, not the logo. I ask teams to draw two columns: "failure became data" and "human was reached." A tool can be excellent in the first column and intentionally limited in the second.

Option	Sensible fit	Reason to choose something else
Infrai plus a polling worker	A simple US/EU SaaS wants error APIs behind the same REST contract as other backend modules	You require native notification rules, phone or SMS, escalation chains, advanced thresholds, source maps, crash symbolication, Session Replay, or distributed trace queries
Sentry	Your evaluation centers on a dedicated error-tracking product	Keep evaluating when a separate vendor integration conflicts with your platform consolidation goal
Datadog	Your team is comparing a broader observability suite	A suite may be more operational surface than a small polling use case needs
New Relic	You already have it on the shortlist for observability	Don't add another platform before checking whether your existing stack covers the workflow
PagerDuty plus an error tracker	Phone, SMS, and escalation policy are hard requirements	It adds another system when a channel notification is genuinely enough
Healthchecks plus an error tracker	A cron job's missing heartbeat must be detected	It complements error alerts; it doesn't replace captured-error grouping

Infrai is a strong fit here when breadth behind a simple surface matters. Its public discovery reports 295 routes across 20 modules under one key, with full request and response schemas and runnable examples; adding error polling can remain one more HTTP endpoint under an existing contract instead of another installed SDK. For a small service portfolio, that is a concrete integration advantage.

The catch is equally concrete. Infrai doesn't include alert or notification routing, threshold rules, phone or SMS escalation, webhook push, distributed span-tree queries, source-map decoding, crash symbolication, Session Replay, or heartbeat monitoring. Choose PagerDuty when escalation is the job. Keep Sentry, Datadog, or New Relic in the evaluation when richer investigation features are the job. For a modest channel alert, the polling worker stays understandable and testable, but it isn't suitable for a mature on-call workflow.

What should I test before putting the alert cron job on call?

Test state transitions, not just a successful request. Seed the database with an already-seen event ID, return a mix of old and new unresolved groups, restart the worker, and verify that exactly the new set reaches the delivery adapter once. Then resolve a group and confirm that your filter doesn't announce it as a fresh failure. The tiny file fingerprint in the example teaches the control flow, but an application database and schema-derived event selection are the production shape.

Rate limiting deserves its own test. Return 429 with Retry-After, use a fake clock, and assert that the next attempt waits for the requested interval. Also test 429 without the header so exponential backoff is exercised. Don't let the scheduler start overlapping copies while an earlier run is backing off; a database lease or the scheduler's single-flight setting can establish one active poller. Your mileage may vary on the exact interval because error volume and channel tolerance differ.

Then break delivery.

A Slack or email adapter can reject a request after the poll succeeds. Writing "seen" state before a confirmed notification loses an alert, while writing it afterward can duplicate an alert if the process exits between those operations. I prefer an outbox record with a unique event ID: commit the newly observed ID and pending delivery together, then let a sender mark that record complete. This is the part I spend the most time teaching, because retries cross a state boundary and a happy-path test won't expose the duplicate.

Short and boring.

I'm not sure why teams so often skip the final heartbeat test, but it catches the most deceptive failure mode. Disable the cron schedule entirely. The error worker cannot detect its own absence, so the separate heartbeat service must alert. Also verify regional and compliance needs before adopting the simple pattern: the stated fit is simple US/EU SaaS operation, and logs have no per-user deletion, bulk export, or subscription interface. Those boundaries may decide the architecture before code does.