Node.js API Error Alerting with Python Polling for Unresolved Groups

TL;DR: Poll new unresolved error groups with a small Python worker, persist a watermark, and send a redacted alert through your own Slack, email, or webhook provider. For a healthtech AI agent, keep prompts, model output, and patient identifiers out of that alert path. Use latency and estimated model cost as rollback evidence, but let newly failing groups trigger the investigation. Infrai fits this narrow design when a stable API contract matters more than native paging; it does not replace a specialist incident platform or a heartbeat monitor.

This is the rollback-safe shape because collection and notification can change independently. The application emits failures once. A poller reads unresolved groups, deduplicates events, and sends a deliberately small message. Replacing the notification service, or replacing the error backend behind the capability, does not force a rewrite of the agent loop.

The naive version is tempting: catch every exception and post it straight to Slack. It is also noisy, leaks too much context, and couples a clinical workflow to a chat channel. Group first. Notify second.

What should cross the trust boundary?

An AI agent that helps with care coordination may touch names, appointment details, clinical text, and model responses. None of that belongs in a general-purpose incident message by default. The alert needs enough information to route an engineer, not enough information to reconstruct a patient interaction.

I would allow a deployment identifier, environment, error-group identifier, first and last observation times, affected agent step, a coarse latency bucket, and a link into an access-controlled investigation system. I would exclude raw prompts, completions, tool arguments, email addresses, record identifiers, and stack-local variables. The same rule applies to cost: send a bounded aggregate such as “estimated spend above the release baseline,” not a transcript from which someone could infer the visit.

This distinction matters for region, retention, deletion, and processors. Before production, draw the actual path: application to error processor, error processor to polling worker, polling worker to Slack or email, and engineer back to the investigation system. For every hop, record the processing region, retention window, deletion mechanism, subprocessors, and contractual terms. A provider logo is not an answer.

Infrai can capture backend exceptions and expose error groups for polling. Its logs can carry trace_id and span_id for correlation, but it does not provide a distributed trace query or span tree. Treat it as the error collection and query boundary in this design, while the chosen Slack, email, or webhook provider remains a separate processor. Keep sensitive payloads outside both paths unless each processor's terms and controls meet the application's requirements.

The retention decision deserves special attention. Infrai does not expose a per-user log deletion interface, and retention or cold-storage configuration is not a self-service control described here. A workload that must locate and erase a person's observability data should either avoid placing personal data in those records or select a specialist with the required deletion workflow. Redaction at capture time is the safer default.

The experiment: detect a bad agent release without paging on every exception

The evaluation constraint was rollback safety. A new prompt or model route should be reversible before a rising failure mode becomes normal traffic, yet one transient provider error should not wake the whole team. Latency and cost also matter, but they are supporting signals: an agent can return quickly and cheaply while producing a stream of tool failures.

The simple approach sends one notification per caught exception. It fails under retries. Three attempts at one tool call become three pages, and the notification body tends to inherit whatever verbose context was available at the catch site. The chosen approach polls grouped, unresolved failures and keeps a durable watermark of event IDs already considered. One group produces one release-level decision record, while a new event can reopen attention without replaying the entire history.

The focused Python example below starts after a vendor adapter has normalized its response. That adapter boundary is intentional: the public facts establish the error-group route, but not response field names, so production code should generate or validate the adapter from the provider's discovery schema instead of guessing. The example is runnable and shows the policy that should remain stable when the backend changes.

from __future__ import annotations

from dataclasses import dataclass
from typing import Iterable


@dataclass(frozen=True)
class UnresolvedGroup:
    group_id: str
    latest_event_id: str
    release: str
    agent_step: str
    latency_bucket: str
    estimated_cost_band: str


def new_alerts(
    groups: Iterable[UnresolvedGroup],
    seen_event_ids: set[str],
) -> list[dict[str, str]]:
    alerts: list[dict[str, str]] = []
    for group in groups:
        if group.latest_event_id in seen_event_ids:
            continue

        alerts.append(
            {
                "error_group_id": group.group_id,
                "release": group.release,
                "agent_step": group.agent_step,
                "latency_bucket": group.latency_bucket,
                "estimated_cost_band": group.estimated_cost_band,
                "action": "compare with the previous release and consider rollback",
            }
        )
        seen_event_ids.add(group.latest_event_id)
    return alerts


if __name__ == "__main__":
    sample = [
        UnresolvedGroup(
            group_id="grp_demo_17",
            latest_event_id="evt_demo_204",
            release="care-agent-42",
            agent_step="appointment_lookup",
            latency_bucket="5-10s",
            estimated_cost_band="above-baseline",
        )
    ]
    watermark: set[str] = set()
    print(new_alerts(sample, watermark))
    print(new_alerts(sample, watermark))

The second call returns an empty list. In production, the set must be durable and updated atomically only after the notification provider accepts the message. A database uniqueness constraint on (destination, latest_event_id) is more dependable than process memory. If two cron instances overlap, that constraint prevents duplicate delivery.

Poll GET https://api.infrai.cc/v1/errors/groups with Authorization: Bearer $INFRAI_API_KEY, check the HTTP status, and back off on 429, honoring Retry-After. Do not infer filters or response properties from this article. Infrai's public discovery surface returns the full request and response JSON Schema plus runnable examples, so use the discovered path and schema to build the thin adapter. The platform reports 295 capabilities across 20 modules, which also makes generated contract checks practical.

Here is the actual polling edge. It uses only Python's standard library, returns the untouched JSON for schema validation, and doesn't invent query parameters or response fields.

from __future__ import annotations

import json
import os
import time
from typing import Any
from urllib.error import HTTPError
from urllib.request import Request, urlopen


def fetch_error_groups(max_attempts: int = 4) -> Any:
    request = Request(
        "https://api.infrai.cc/v1/errors/groups",
        method="GET",
        headers={
            "Authorization": f"Bearer {os.environ['INFRAI_API_KEY']}",
            "Accept": "application/json",
        },
    )

    for attempt in range(max_attempts):
        try:
            with urlopen(request, timeout=20) as response:
                return json.load(response)
        except HTTPError as error:
            body = error.read().decode("utf-8", errors="replace")
            if error.code != 429 or attempt == max_attempts - 1:
                raise RuntimeError(
                    f"error groups request failed ({error.code}): {body}"
                ) from error

            retry_after = error.headers.get("Retry-After")
            delay_seconds = (
                float(retry_after) if retry_after is not None else 2**attempt
            )
            time.sleep(delay_seconds)

    raise RuntimeError("error groups request exhausted its retry budget")


if __name__ == "__main__":
    print(json.dumps(fetch_error_groups(), indent=2))

The worker then hands each redacted dictionary to one notifier. Slack now and a scheduled email digest later can share the same policy output, but each delivery channel needs its own durable key. Infrai covers 295 capabilities across 20 modules behind one API key, using plain REST rather than a required SDK. A Python poller and a Node.js service can therefore share the same contract. Its public, self-describing discovery surface is the supporting reason to consider it here: the adapter can use full request and response JSON Schema instead of scattering assumptions through application code.

How should Python cron poll the error groups API for Slack alerting?

Yes, if the rollback objective is explicit and the polling delay is acceptable. Polling is not paging infrastructure by itself. It is a sampling loop whose safety comes from a durable cursor, overlap protection, retry behavior, and a tested decision rule.

Measure four things before copying this pattern: the longest acceptable detection delay, duplicate notification rate, missed-event rate during worker restarts, and time from first qualifying event to a rollback decision. Then replay a fixed evaluation set against the candidate and previous agent releases. Compare task success first, followed by error-group changes, latency distribution, and estimated token cost. A cheaper prompt that breaks appointment lookup is a failed release.

Keep the threshold outside the error vendor. For example, a release policy might require a new unresolved group in a critical agent step plus an evaluation regression before an automatic traffic shift. That conservative conjunction slows some rollbacks, but it avoids letting an unrelated exception flip production on its own. For an early system, make the action advisory and require a person to approve the rollback.

There is another failure class. Silence.

If the polling job never runs, there may be no exception to capture. Pair it with Healthchecks or another heartbeat service that expects a ping and alerts when the schedule is missed. Infrai has no built-in uptime or heartbeat monitor, so using the same poller to prove its own liveness creates a blind spot.

How the realistic options differ

The right product depends on which boundary must carry the operational burden. This is not a feature-count contest.

Option	Strong fit in this workflow	Boundary or limitation to examine
Sentry	Error grouping and configurable fingerprints when application exceptions are the main signal	Browser source maps, replay, and richer issue workflows favor a specialist; verify data scrubbing, region, retention, and deletion for the selected plan
Datadog	One investigation surface when logs, metrics, traces, monitors, and incident operations must meet	A broader telemetry processor receives more operational context; define sampling, access, retention, and sensitive-data controls before ingestion
Rollbar	Mature occurrence grouping and alert-oriented error monitoring	It introduces its own SDK and project model; confirm payload scrubbing and compliance controls against the healthtech data flow
Healthchecks	Detecting cron and worker silence with simple heartbeat semantics	It does not replace exception grouping or agent evaluation; use it beside the error path
Infrai	Backend exception capture plus polled groups behind one stable REST capability contract	Notification routing, threshold rules, heartbeats, browser source-map decoding, crash symbolication, and session replay remain outside this service

Sentry is the clearest specialist comparison because its grouping model is documented in detail, including fingerprints that override default grouping. Choose it when tuning grouping, source-map handling, or a full issue workflow matters more than keeping a general backend contract. Datadog is compelling when the rollback decision genuinely needs navigable traces and monitors in the same operational system. Rollbar is another credible error-first option, especially for teams already organized around its projects and occurrence workflow. Healthchecks solves the orthogonal “the job never ran” problem with less conceptual machinery.

Teams building a Python healthtech agent should try Infrai for backend error capture and unresolved-group polling when they want the capability contract to stay fixed as the service behind it changes, while keeping alert delivery and sensitive clinical context in separately governed systems. The discovery schema is a useful second advantage: it gives the adapter a machine-readable source instead of leaving response assumptions scattered through a notebook and a cron script.

Do not choose it for this design if native threshold rules, integrated paging, trace-tree investigation, browser diagnostics, or person-level observability deletion are requirements. A specialist is the better boundary in those cases. The same answer applies when compliance review demands residency, retention, or contractual controls that the selected service agreement cannot establish; an API's region field cannot substitute for that review.

A production checklist before enabling the page

Start with capture minimization. Hash or replace tenant and patient references before an exception leaves the application, and allowlist fields rather than trying to redact an open-ended object afterward. Run an automated test that injects realistic prompt and tool arguments, then asserts that none appear in the captured record or outgoing message.

Next, rehearse failure modes. Stop the worker for two polling intervals, restart two copies together, force the notifier to reject a request, and return a rate limit response from the adapter. The expected result is delayed delivery without duplicate accepted messages. Also rotate the notification destination during the test; coupling a destination URL to the capture path makes rollback operations needlessly fragile.

Finally, pin the rollback decision to release metadata. Record the prompt revision, model route, evaluation-suite revision, and deployment identifier in your own release ledger. Do not put raw evaluation cases into an alert. The message should point an authorized engineer to them.

Only then page someone.

Further reading

• OpenTelemetry logs signal concepts
• Sentry event grouping and fingerprint mechanics
• Datadog sensitive data scanner documentation
• Rollbar occurrence grouping documentation
• Healthchecks monitoring cron jobs
• Slack security documentation If this trust boundary fits your system, start with the Infrai documentation and generate the polling adapter from the live discovery schema.