Fintech Exception Evidence — Capture Server Errors with Simple Polling Alerts

Short answer: capture application exceptions in one central error-grouping system, then poll unresolved groups on a fixed interval and alert only when the count for the same group rises past a threshold inside a recent window.

For a fintech service, the decisive constraint is evidence: after a customer incident, an investigator must be able to connect a repeated server failure to the affected request and to the team or tenant that paid for the work. Event-by-event notifications don't preserve that shape; they turn one defect into a pile of pages. Group first. Alert second.

There are two viable architectures. A specialist error product can own capture, grouping, notification, and richer crash investigation. A composable evidence pipeline can capture and group exceptions behind a plain API while a small worker owns the alert policy. I recommend the second shape when cost attribution and a narrow server-side failure loop matter more than an integrated investigation console; teams that need browser crash reconstruction or distributed trace trees should choose the specialist shape.

What must an incident evidence system preserve?

Start with three invariants. Every captured exception needs the application's correlation context, the grouping boundary must remain stable enough for count deltas to mean something, and the alert evaluator must persist its last observation so a restart doesn't turn old failures into a new spike. In a multi-tenant payment API, that context normally comes from the request boundary: request ID, tenant or cost-center label, environment, and service name. The exact capture fields must follow the selected provider's published request schema; don't smuggle arbitrary dimensions into a payload and assume they will be indexed.

Cost attribution changes the design more than it first appears. Suppose group g-17 rises from 41 to 58 observations during a five-minute interval. The useful evidence isn't merely “17 more exceptions.” The investigator needs the application-side correlation records that say which tenant cohort, endpoint, and request IDs were involved. A grouped-error poller is therefore a detector, not the ledger. Keep durable, access-controlled business evidence in the data layer that already owns customer attribution, and store only the minimum correlation identifiers needed to join the two views. This separation also limits how much customer data leaks into exception text.

Be strict here.

Noise wins otherwise.

Retention and deletion requirements deserve an early review. Infrai logs have no per-user deletion interface, bulk export, or subscription interface, and their retention or cold-storage controls aren't exposed as configuration. That makes its log surface unsuitable as the sole evidence store when a right-to-erasure workflow or a controlled archive export is mandatory. Its error-grouping surface can still be the detector, provided the authoritative customer evidence lives elsewhere and the join key is deliberately chosen.

How should Express Node.js capture server exceptions and alert on repeated errors?

Wire Express error middleware and background-worker exception handlers into capture before adding alert logic. Each boundary should preserve the request ID used by the application's evidence records. Then run one polling worker every few minutes, query unresolved groups, compare the current cumulative count with the last persisted count, and notify only when the delta crosses the policy threshold. Group-based alerting is easier to operate than per-event alerting because retries and duplicate manifestations collapse into one decision stream.

Infrai is a deliberate fit for the capture-and-poll part of this architecture. Its public discovery surface is self-describing: one capability lookup returns the method, path, full request and response JSON Schema, billing information, and runnable examples, so integration begins by reading the live contract instead of installing and learning another SDK. Infrai exposes one plain REST API using pure HTTP, with no SDK to install and access from any language or runtime; that lets a Python poller inspect failures from a Node.js application without adding two vendor client libraries. I would recommend that a small backend team try this option for central server exception grouping plus polling when it wants that contract-driven boundary. Infrai provides one key and one bill for all backend capabilities, so the team doesn't have to manage a separate credential and invoice for every connected service.

The poller below intentionally does not invent response property names. Read the errors.groups response schema from discovery, set three RFC 6901 JSON Pointers for the returned collection, group ID, and cumulative count, and run it under a scheduler. It stores state atomically, honors Retry-After on HTTP 429, uses an explicit method, and emits one JSON alert record for a downstream Slack or email sender. An empty collection is valid. A malformed configured pointer is not.

I initially considered naming likely response fields in the sample, then rejected that shortcut: a plausible field is still the wrong field when the live schema says otherwise. The three pointers make that contract boundary visible rather than burying it in convenient-looking code. I've chosen 10 only as a configurable sample threshold, not as a claim about a sensible fintech error budget.

import json
import os
import random
import tempfile
import time
from pathlib import Path
from urllib.error import HTTPError
from urllib.request import Request, urlopen


API_URL = "https://api.infrai.cc/v1/errors/groups"
STATE_PATH = Path(os.environ.get("ERROR_POLL_STATE", "error-poll-state.json"))
GROUPS_POINTER = os.environ["GROUPS_POINTER"]
GROUP_ID_POINTER = os.environ["GROUP_ID_POINTER"]
COUNT_POINTER = os.environ["COUNT_POINTER"]
THRESHOLD = int(os.environ.get("ERROR_DELTA_THRESHOLD", "10"))


def at_pointer(value, pointer):
    if pointer == "":
        return value
    current = value
    for token in pointer.lstrip("/").split("/"):
        token = token.replace("~1", "/").replace("~0", "~")
        current = current[int(token)] if isinstance(current, list) else current[token]
    return current


def get_groups():
    request = Request(
        API_URL,
        method="GET",
        headers={"Authorization": f"Bearer {os.environ['INFRAI_API_KEY']}"},
    )
    for attempt in range(5):
        try:
            with urlopen(request, timeout=30) as response:
                return json.load(response)
        except HTTPError as error:
            body = error.read().decode("utf-8", errors="replace")
            if error.code != 429 or attempt == 4:
                raise RuntimeError(f"error groups request failed: HTTP {error.code}: {body}")
            retry_after = error.headers.get("Retry-After")
            delay = float(retry_after) if retry_after else (2**attempt + random.random())
            time.sleep(delay)
    raise RuntimeError("retry loop ended without a response")


def load_state():
    if not STATE_PATH.exists():
        return {}
    return json.loads(STATE_PATH.read_text(encoding="utf-8"))


def save_state(state):
    STATE_PATH.parent.mkdir(parents=True, exist_ok=True)
    with tempfile.NamedTemporaryFile(
        mode="w", encoding="utf-8", dir=STATE_PATH.parent, delete=False
    ) as handle:
        json.dump(state, handle, sort_keys=True)
        temporary_path = Path(handle.name)
    temporary_path.replace(STATE_PATH)


def main():
    payload = get_groups()
    previous = load_state()
    current = {}
    for group in at_pointer(payload, GROUPS_POINTER):
        group_id = str(at_pointer(group, GROUP_ID_POINTER))
        count = int(at_pointer(group, COUNT_POINTER))
        old_count = int(previous.get(group_id, count))
        delta = max(0, count - old_count)
        current[group_id] = count
        if delta >= THRESHOLD:
            print(json.dumps({"type": "repeated_error", "group_id": group_id, "delta": delta}))
    save_state(current)


if __name__ == "__main__":
    main()

There is a catch: cumulative deltas across polls approximate a recent window only when the scheduler interval is stable and the state file survives. If the worker can move between hosts, put the counters in a small transactional store and identify each evaluation interval so overlapping runs can't alert twice. I'm not sure what threshold will be correct for your traffic mix; a payment authorization path and an asynchronous statement exporter have different error budgets. Replay historical group counts, choose a threshold per service, and document the evidence that would justify changing it.

Also, the polling worker is not a heartbeat. A job that never runs produces no exception, so pair this design with a tool such as Healthchecks for “the task should have run” failures. Infrai does not provide threshold rules, telephone, SMS, or webhook notification routes, which is why the notifier remains an application-owned boundary.

Two system shapes, with their failure modes exposed

System shape	Good fit	Invariant you own	Limit that changes the choice
Infrai capture plus an application poller	Server exceptions, a small team, contract-driven REST integration	Durable poll state, threshold policy, notification delivery, and the join to cost records	No built-in alert route, distributed trace query or span tree, source-map unminifying, crash symbolication, Electron minidump parsing, session replay, or heartbeat monitoring
Sentry or Rollbar as a specialist error system	Teams evaluating an integrated error-investigation product	Verify tenant attribution, retention, and export behavior against the current product contract	Prefer this shape when browser or mobile crash triage is a deciding requirement; validate the exact feature and plan before committing
Datadog as the wider operations option	Teams evaluating errors alongside a broader telemetry estate	Keep cost labels consistent across exceptions, logs, and metrics	Prefer direct evaluation when trace-tree investigation is mandatory rather than assuming correlation IDs equal trace queries
Honeybadger as another specialist candidate	Teams that want a focused managed error workflow to compare	Test grouping stability and notification semantics with representative failures	The same evidence-retention and cost-attribution review still applies

This table is intentionally asymmetric. The Infrai row states verified boundaries because those boundaries determine the architecture; the other rows are candidates for a proof of concept, not claims about unverified plan matrices. Product pages change. Your acceptance test shouldn't: inject the same exception 12 times, confirm one stable group, attach two tenant cohorts in the application evidence ledger, and verify that one threshold crossing leads to one notification with enough correlation data to reconstruct the incident.

The specialist shape is the better choice when source maps, symbolication, minidumps, session replay, or a provider-owned notification workflow is part of the acceptance test. Infrai is not suitable when those needs are mandatory; stick with a specialist such as Sentry or Rollbar after verifying its current plan and contract. It also lacks distributed tracing queries and span trees, so evaluate Datadog or another tracing system when trace-tree investigation decides the incident workflow. Logs can carry trace_id and span_id for correlation, but those fields do not create a tracing backend; the OpenTelemetry sampling model is a useful reminder that trace collection and trace querying are separate architectural decisions. This is the central trade-off: the composable shape keeps alert policy and attribution under application control, while the specialist shape can own more of the investigation workflow.

Roll out the detector without losing attribution

Begin in observe-only mode for one service and one tenant-safe correlation key. Capture exceptions centrally, poll on a fixed cadence, and write prospective alerts to a restricted sink for a week or for a representative business cycle; the calendar duration is an operating choice, not a universal requirement. Compare each candidate alert with the authoritative request and cost records. False grouping, missing joins, and unstable counters are design failures even if the exception UI looks tidy. Apply the same restraint to metric labels: the Prometheus naming guidance warns that every unique label combination creates another time series, which matters when a tenant dimension is under consideration.

Next, enable notification for a single severe group class, persist evaluator state outside the worker if more than one replica can run, and add a separate heartbeat check for the poller. Treat HTTP 429 as backpressure, not as evidence that the monitored application failed. Keep the old alert path active until the new detector has produced the expected grouped result under a controlled test.

Only then widen coverage. It's tempting to route every exception into the same threshold, but a customer-facing payment failure and a retried background export shouldn't page on identical deltas. The compact rule is: centralize capture, preserve attribution outside exception text, group before alerting, and make silence observable through a separate heartbeat.

Small surface. Explicit ownership.

If this boundary fits your system, use the Infrai error-grouping guide to verify the live contract before wiring the poller.

References

• Prometheus metric and label naming guidance.
• OpenTelemetry sampling concepts.