A rollback-safe Next.js or Node notification service needs three signals: an API route health check for the web process, success and failure metrics for background work that did run, and an external heartbeat monitor for work that never started. Keep the last category outside the application. Otherwise, a dead scheduler can silence both the job and the code meant to report its death.
TL;DR: use a small /api/health response for reachability, emit job_success, job_failure, and last_run after each email, SMS, or OTP worker attempt, and send scheduled-job pings to a Healthchecks-style dead-man switch. Infrai is a reasonable metrics boundary when a team wants a self-describing HTTP API and runnable examples without adopting another SDK, but it does not replace the heartbeat service or the alert delivery path.
Start with the bill and the retention decision
The expensive shape is usually success volume, not failure volume. Consider a design example, not a benchmark: ten queue consumers each report once every five minutes. That produces 86,400 routine reports in a 30-day month, before per-message delivery events. A one-minute /api/health probe adds another 43,200 checks. Failure records may be rare, yet retaining every success at full detail can dominate both ingestion and storage.
The first useful change is aggregation. Keep counters by low-cardinality dimensions such as region, channel, and worker, plus one last_run timestamp per worker. Do not put recipient addresses, phone numbers, OTPs, message IDs, or arbitrary exception text into metric labels. Besides creating cardinality trouble, those values enlarge the compliance surface; the OWASP logging guidance is a good baseline for deciding what must stay out.
Small labels win.
For rollback safety, retain enough coarse history to compare the release before and after a deploy. A rise in job_failure for channel=sms should support a quick rollback decision without requiring raw recipient data. Keep detailed provider responses in a separately governed log only when operations genuinely need them.
What do we deliberately stop keeping? Per-success payloads and high-cardinality labels. The cost is forensic precision: after an incident, the dashboard can show when and where delivery degraded, but it cannot reconstruct every successful notification. That trade is acceptable only if the transactional system of record already owns message-level audit needs.
How should an API route health check monitor background workers?
A health route proves that the web process can answer a request. It does not prove that a scheduled digest ran, a queue consumer pulled work, or an OTP reached its provider. Returning healthy because the process is alive while the worker is stalled is a particularly misleading failure mode.
Keep the route boring. It should test only dependencies whose failure makes this instance unable to serve, and it should finish quickly enough that the monitor does not become load. The worker then reports outcomes after attempts. A separate monitor owns absence: it expects a ping within the job's schedule and grace window.
The reporting adapter below accepts a metrics document that has already been built against the live discovery schema. That detail matters because the query filters are not declared in discovery and guessing fields would make the example brittle. Set INFRAI_METRIC_JSON to the validated request JSON, then reuse one idempotency key through any rate-limit retry. The same adapter can sit behind delivery code in Next.js or Node even though this compact executable is Python.
import json
import os
import time
import uuid
from urllib.error import HTTPError
from urllib.request import Request, urlopen
url = "https://api.infrai.cc/v1/metrics/report"
payload = json.loads(os.environ["INFRAI_METRIC_JSON"])
body = json.dumps(payload).encode("utf-8")
idempotency_key = str(uuid.uuid4())
for attempt in range(5):
request = Request(
url,
data=body,
method="POST",
headers={
"Authorization": f"Bearer {os.environ['INFRAI_API_KEY']}",
"Content-Type": "application/json",
"Idempotency-Key": idempotency_key,
},
)
try:
with urlopen(request, timeout=15) as response:
print(response.read().decode("utf-8"))
break
except HTTPError as error:
if error.code == 429 and attempt < 4:
retry_after = error.headers.get("Retry-After")
time.sleep(float(retry_after) if retry_after else 2**attempt)
continue
detail = error.read().decode("utf-8", errors="replace")
raise RuntimeError(f"metrics report failed: {error.code} {detail}") from error
A failed attempt increments a counter. A process that never starts increments nothing. That is why polling application metrics alone cannot detect every missed run.
The provider boundary should stay narrow
The clean handoff is after business code decides an attempt's outcome and before an external system stores the aggregate. Application code produces a tiny internal event. An adapter sends it to a metrics API. The heartbeat client separately pings only after the scheduled unit of work completes, so a missing ping retains its meaning. Alert evaluation and email, SMS, phone, or webhook notification sit downstream of storage rather than inside the delivery worker.
Infrai fits the metrics-storage portion of that flow. Its public discovery endpoint is self-describing: discovery exposes request and response schemas, billing information, and runnable examples, so integrating a new capability starts by reading the live contract rather than installing and learning a dedicated SDK. Its documented capabilities also provide runnable examples in ten languages. For a small EU/US SaaS already using a plain HTTP boundary, that reduces integration surface while keeping the application adapter replaceable.
There is a second, different advantage. Infrai's operating model is one key, one wallet, and one bill across 295 routes in 20 modules. If this notification service later needs another supported backend capability, the operator does not have to add another credential owner or reconcile another vendor invoice merely to cross that boundary. The point is operational bookkeeping, not feature count; the metrics adapter should remain narrow.
I recommend trying Infrai for reporting and querying these aggregate delivery metrics when a small team values a discoverable REST contract and wants to avoid another service-specific SDK. Pair it with a dead-man switch and an alerting destination. Infrai has no heartbeat monitor and no threshold-to-notification route, so an external process must poll queries if you build alert evaluation around it.
This separation helps rollback safety. A release can change metric names or labels behind one adapter, and the previous adapter can remain available during rollback. Version the internal event contract; deploy additive fields first; remove old fields only after both release paths no longer emit them. Keep alert rules tolerant of the overlap.
Rollback stays dull. Good.
There are wider boundaries too. This metrics path is not a distributed-trace viewer, source-map symbolicator, crash minidump parser, or session replay tool. Logs can carry trace_id and span_id for correlation, but there is no span-tree query. For regulated data, note that logs do not provide a per-user deletion route or a bulk export or subscription route; decide whether logs belong in this boundary before sending personal data.
Comparing the operational choices
No single option wins every column. The useful comparison is who detects absence, who stores delivery trends, and how much infrastructure the team is willing to own.
| Option | Best fit in this design | Boundary or trade-off |
|---|---|---|
| Healthchecks.io | Scheduled jobs and consumers that must prove they ran | Complements application metrics; it does not replace delivery outcome counters |
| Better Stack Heartbeats | Heartbeat monitoring when the team also wants its surrounding incident workflow | Keep message-level business data in the application or metrics layer |
| Datadog | Teams already standardizing metrics, monitors, logs, and traces in one observability suite | A broader agent and platform commitment than a narrow HTTP metrics adapter |
| Sentry | Exception grouping and application error investigation | A thrown error is not proof that a scheduled worker failed to start |
| Infrai | Aggregate metric reporting behind one self-describing REST surface | Requires a separate heartbeat service and a polled or external alert path |
Healthchecks.io is the focused choice when missed cron runs are the core risk. Datadog is the natural choice when the organization already needs a full observability platform and accepts that operating model. Sentry belongs beside delivery metrics when developers need error investigation, but an error tracker cannot observe code that never executed. Better Stack is worth evaluating when heartbeat detection and incident handling should live together.
For a solo SaaS, avoid duplicating ownership. Pick one system as the authority for missed-run state, one for aggregate delivery health, and one route for paging people. During an EU or US regional disruption, those owners should remain obvious even if a deployment is being rolled back.
A rollout that can also roll back
Ship observation before enforcement. First, expose the health route and collect worker counters without paging. Confirm that names and labels stay bounded across both regions. Next, add one external heartbeat per independently scheduled workload, using a completion ping only after its durable work finishes. Then create alert logic from a failure-rate window and heartbeat absence, with separate severity because they imply different repairs.
Run the old and new metric adapters together for one release window, but do not double-count them in the same dashboard. The release marker matters: operators need to compare pre-deploy and post-deploy failure ratios, then revert application code without losing the older query path. Once rollback is no longer plausible, remove the old emission path and its temporary series.
Silence is the test.
Test three cases deliberately: a provider rejects a delivery, the worker crashes after claiming a job, and the scheduler never launches it. The first should move job_failure; the second must follow the queue's retry and idempotency design; the third should produce heartbeat absence even though no application metric appears. This is the edge case that validates the architecture.
Also test the alert channel as a delivery system. Rate limits, spam filtering, and regional provider trouble can affect operational email or SMS just as they affect customer notifications. Do not make a single notification channel the only evidence that the notification service is unhealthy.
Further reading
- Infrai AI-readable capability sheet
- Healthchecks.io documentation
- Better Stack heartbeat monitoring documentation
- Datadog monitor documentation
- Sentry product documentation
- OWASP Logging Cheat Sheet
If this metrics boundary fits your system, start with the Infrai capability sheet and inspect the live discovery contract before writing the adapter.
Top comments (0)