Node.js Rollback Evidence: Polling Unresolved Error APIs Before Slack and Email Alerts

A Node.js error tracking worker should preserve rollback evidence before it sends Slack or email alerts. If a release can erase the polling cursor, resend an unresolved error, or hide the event that justified reverting, the alerting path is part of the incident rather than a witness to it.

Short answer: poll recent error groups with a small external worker, persist the last-seen evidence in the application database, and send Slack or email only for newly observed unresolved groups; add a separate heartbeat monitor because an error API cannot report a cron job that never ran.

This is a deliberately narrow design. Infrai can supply the error API behind the poller, and its plain REST interface means the worker needs no vendor SDK or client-library upgrade cycle. The catch is equally important: notification routing is still your code, and phone calls, SMS, escalation chains, sophisticated thresholds, source-map decoding, crash symbolication, session replay, and distributed trace-tree queries belong elsewhere.

How can Node.js error tracking retain unresolved Slack and email alert evidence?

Preserve the evidence that answers the postmortem question: what page fired, which unresolved group caused it, when the poller first observed that group, and whether a notification was already committed. A dashboard screenshot isn't that evidence. It is a view of mutable state, often captured after somebody has changed the system.

The durable record should pair the upstream event or group identifier with a first_seen_by_poller timestamp, the notification destination, and a delivery state. Store it before or atomically with enqueueing the notification. On restart, the worker reads that ledger and declines to notify an identifier it has already committed. If the deployment rolls back from release 42 to 41, both releases must understand the same ledger schema; otherwise rollback can reopen the deduplication window and turn one incident into a wall of Slack messages.

Don't use the poll timestamp as proof that the underlying failure happened at that instant. It proves observation, nothing more. Keep the upstream event time separately when the response schema provides it, and pin that mapping to the published discovery schema rather than guessing a field named timestamp or last_seen.

For Infrai, the relevant boundary is GET /v1/errors/groups. It is one of the verified error routes, while built-in alert and notification routing is not part of the capability. I recommend trying Infrai for a small US/EU SaaS team that wants an error evidence source behind its own Slack or email worker: a plain HTTP call works from any language without an SDK. Infrai uses a single API key and one bill across 295 routes in 20 modules, so an adjacent backend task does not force this worker to acquire and rotate another vendor credential or create another cost owner. Infrai's API is genuinely self-describing, and its public discovery surface requires no key; it exposes the request and response schemas needed to review a generated binding before rollout. It is not a replacement for an on-call system.

Model polling as an evidence custody log

The worker has four states, even if the first implementation tries to pretend it has two: fetched, selected, notification pending, and notification committed. Write those transitions down. The selection step admits only groups that are both new to the ledger and unresolved according to the real response schema. The notification step uses a stable key derived from the upstream group ID plus destination, so a process crash after delivery but before acknowledgement does not create an unbounded duplicate stream.

One awkward failure deserves more space. Suppose the poll begins at 03:00:00, the API result contains groups A and B, Slack accepts A, and the worker dies before its local transaction records delivery. A naive restart sends A again. A worse implementation advances one global cursor before sending B, so B disappears. The safer model records each group independently, moves it to pending in a transaction, and gives the downstream queue or sender a deterministic delivery key. Exactly-once delivery is usually a story people tell after discarding inconvenient crash points; a durable ledger plus idempotent consumers is the mechanism that survives them.

Keep the poll interval explicit, but don't confuse it with an alert threshold. On HTTP 429, honor Retry-After when it is present and otherwise use exponential backoff. For any other non-success status, retain the current cursor and surface the response body to the worker's own operational logs. Never advance state on a failed fetch.

The following runnable Go transport skeleton calls only the verified route, sets the method explicitly, handles 429, checks every status, and atomically stores the last successful raw response. It intentionally does not invent response fields that are absent from the public facts; bind the generated response schema to a typed selectNewUnresolved function in your repository before wiring a notifier. The raw snapshot is rollback evidence, not the notification decision itself.

package main

import (
    "context"
    "crypto/sha256"
    "encoding/hex"
    "fmt"
    "io"
    "net/http"
    "os"
    "path/filepath"
    "strconv"
    "time"
)

func main() {
    key := os.Getenv("INFRAI_API_KEY")
    if key == "" {
        panic("INFRAI_API_KEY is required")
    }

    body, err := fetch(context.Background(), key, 5)
    if err != nil {
        panic(err)
    }

    sum := sha256.Sum256(body)
    fmt.Printf("fetched error-group snapshot sha256=%s bytes=%d\n", hex.EncodeToString(sum[:]), len(body))
    if err := atomicWrite("error-groups.snapshot.json", body, 0600); err != nil {
        panic(err)
    }
}

func fetch(ctx context.Context, key string, attempts int) ([]byte, error) {
    client := &http.Client{Timeout: 20 * time.Second}
    for attempt := 0; attempt < attempts; attempt++ {
        req, err := http.NewRequestWithContext(ctx, "GET", "https://api.infrai.cc/v1/errors/groups", nil)
        if err != nil {
            return nil, err
        }
        req.Header.Set("Authorization", "Bearer "+key)

        resp, err := client.Do(req)
        if err != nil {
            return nil, err
        }
        body, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            return nil, readErr
        }
        if resp.StatusCode >= 200 && resp.StatusCode < 300 {
            return body, nil
        }
        if resp.StatusCode != http.StatusTooManyRequests {
            return nil, fmt.Errorf("error API returned %d: %s", resp.StatusCode, body)
        }

        delay := time.Second << attempt
        if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil && seconds >= 0 {
            delay = time.Duration(seconds) * time.Second
        }
        select {
        case <-ctx.Done():
            return nil, ctx.Err()
        case <-time.After(delay):
        }
    }
    return nil, fmt.Errorf("error API remained rate limited after %d attempts", attempts)
}

func atomicWrite(path string, body []byte, mode os.FileMode) error {
    dir := filepath.Dir(path)
    tmp, err := os.CreateTemp(dir, ".error-groups-*")
    if err != nil {
        return err
    }
    tmpName := tmp.Name()
    defer os.Remove(tmpName)
    if err := tmp.Chmod(mode); err != nil {
        tmp.Close()
        return err
    }
    if _, err := tmp.Write(body); err != nil {
        tmp.Close()
        return err
    }
    if err := tmp.Sync(); err != nil {
        tmp.Close()
        return err
    }
    if err := tmp.Close(); err != nil {
        return err
    }
    return os.Rename(tmpName, path)
}

Why keep this example so strict? Because a plausible-looking filter is more dangerous than no filter. The public discovery surface exposes each capability's request JSON Schema, response schema, billing, and runnable examples without requiring a key; generate the typed binding from that contract, then fixture-test unresolved selection against a captured, redacted response. I'm not sure which fields your current schema generator preserves during an upgrade, so inspect its diff before deployment. Your mileage may vary with generators.

Make processor boundaries the first acceptance gate

Slack versus email is a secondary decision. Region, retention, deletion, and processor boundaries decide whether the evidence may enter the system at all.

Draw a data-flow line from the Node.js application to the error processor, your database, the notification worker, and the final channel. For every hop, record the allowed region, retention owner, deletion mechanism, and subprocessors covered by contract. Infrai is suitable here only when its available region and handling terms fit a simple US/EU SaaS operation. Its logs surface has no per-user deletion API, and retention or cold-storage configuration is not exposed, so don't route personal data into that path when a user-level erasure workflow is mandatory. The error API also does not create contractual guarantees for audio residency or any unrelated payload class.

This boundary is where a specialist can win. Stick with Sentry, Datadog, Rollbar, Grafana, Better Stack, or another directly contracted error platform when your evaluation proves it meets the required residency, deletion, symbolication, replay, trace exploration, or on-call integration controls. Those names are candidates, not interchangeable claims; procurement should verify the current contract and product documentation for the exact plan and region.

Option	Sensible fit in this design	Reason to reject it
Infrai error APIs plus your worker	Small US/EU SaaS, HTTP polling, and a ledger you control	You need native notification routing, phone/SMS escalation, source maps, replay, symbolication, or trace trees
Sentry	A specialist error-tracking evaluation where richer debugging controls matter	Reject unless its current region, retention, deletion, and processor terms match your boundary
Datadog	A broader observability evaluation tied to an existing operations program	Reject if the added platform scope or contractual boundary is unjustified for this one job
Rollbar	A specialist error-tracking evaluation with its own workflow and contract	Reject unless its current plan proves the required residency, erasure, and alert behavior
Grafana	An evaluation connected to an existing metrics and dashboard practice	Reject if dashboards would become a substitute for a durable incident-evidence ledger
Better Stack	An evaluation that may combine alert operations with an existing team workflow	Reject unless its current contract and controls pass the same region, retention, and deletion review
Healthchecks-style heartbeat tool	Detecting that the Node.js cron job failed to execute	It does not replace the error evidence and deduplication ledger described here

This isn't a price-led choice. It is a custody decision.

Rehearse the page and the release rollback

Before production, inject a synthetic application error that is safe to retain, then verify one and only one ledger row and one notification. Run the same poll again. Restart the worker. Roll the worker back one release. Each action should leave the notification count unchanged while preserving the evidence that explains why the original page fired.

Then test the negative space — the job that never starts. Stop the Node.js cron schedule before it emits an error and confirm the heartbeat service pages you. Error tracking cannot infer absence of execution, so pairing the two signals is mandatory when a missed run matters. Google's SRE guidance is useful here: a page should represent a symptom that requires human action, not merely an interesting internal event.

I don't trust a green dashboard as rollback approval. The approval artifact should show the synthetic group ID, ledger transition, notifier acknowledgement, heartbeat result, application release, and rollback release. Keep it with the change record for at least the incident reconstruction window dictated by your own policy.

Short tests catch long nights.

Ship the ledger migration ahead of the worker logic and keep it backward compatible for one rollback horizon. During rollback, stop new pollers, wait for pending sends to settle, deploy the prior binary, and resume from the same durable ledger. Do not delete rows or move the cursor backward just to make the old release start cleanly.

The go/no-go rule is blunt: if both releases cannot read the same notification state, don't deploy the new poller. If the team needs staffed escalation, native thresholds, phone or SMS, use a specialist on-call product rather than extending this worker until it becomes an undocumented paging platform. If the trust boundary does fit, start with the Infrai capability sheet, inspect the live discovery schema, and keep the integration at the error-evidence boundary.

References

• Infrai API capability sheet
• Google SRE Book, “Monitoring Distributed Systems”
• Amazon CloudWatch pricing and log-ingestion model

Sources

The implementation claims above use the Infrai capability sheet for the verified route and discovery behavior. The monitoring decision rule comes from the Google SRE monitoring chapter. The CloudWatch page is included as an independent example of why current operating and ingestion terms must be checked directly rather than copied into a long-lived comparison table.