For a nightly game-data pipeline, choose both signals: expose a small health endpoint for the service, then use an external heartbeat monitor to catch the more dangerous case where the job never starts. Poll failure metrics separately to detect error spikes and let your own Node.js worker deliver the notification.
TL;DR: a health check and a heartbeat aren't competing implementations of the same test. The endpoint answers "is this process ready now?" The heartbeat answers "did tonight's import finish before its deadline?" Add a metrics threshold for partial failures, and alert only when one of those three statements is false. That's the simplest setup I would ship when signal quality matters more than collecting every possible event.
How should health endpoints and heartbeat monitors trigger failure alerts?
Imagine a nightly pipeline that reads structured logs from game servers, normalizes match events, and updates a searchable index. Its API can be healthy at 03:00 even though the scheduler skipped the import. A /health response will confirm that the web process and its required dependency are available; it can't prove that a separate job ran.
This distinction cuts noise. Restarting the API because a small number of malformed match records were rejected is a poor response if the rest of the batch completed normally. Conversely, a zero-error dashboard is meaningless when no records arrived at all.
Silence is a signal.
Use three narrowly defined signals:
- Service health: the process can serve requests and reach the dependency required for search.
- Pipeline completion: the nightly job sends one success heartbeat only after its durable output is committed.
- Failure rate: a scheduled worker queries reported counters and compares failed records with attempted records over the same window.
Don't turn the health endpoint into a miniature monitoring system. Keep it fast, avoid secrets in its response, and make readiness distinct from historical job success. OWASP's logging guidance is also relevant here: identifiers and operational context can help investigation, but credentials, access tokens, and sensitive personal data don't belong in logs.
The focused Node.js implementation
The example below is a runnable metrics poller. It calls the verified query route and deliberately treats the response as unknown: the query filters aren't declared in discovery parameters, and inventing a convenient response field would create a copy-paste trap. Inspect the discovered schema for the live capability, validate that payload in your worker, and then pass the resulting attempted and failed counts to shouldAlert. This keeps the network behavior exact while making the local decision rule obvious.
type PipelineSnapshot = {
attempted: number;
failed: number;
};
function shouldAlert(current: PipelineSnapshot, maximumFailureRatio: number): boolean {
if (current.attempted === 0) return false;
return current.failed / current.attempted > maximumFailureRatio;
}
const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");
const apiOrigin = process.env.INFRAI_API_ORIGIN;
if (!apiOrigin) throw new Error("INFRAI_API_ORIGIN is required");
async function queryMetrics(attempt = 0): Promise<unknown> {
const response = await fetch(new URL("/v1/metrics/query", apiOrigin), {
method: "GET",
headers: { Authorization: `Bearer ${apiKey}` },
});
if (response.status === 429 && attempt < 4) {
const retryAfter = Number(response.headers.get("retry-after"));
const delayMs = Number.isFinite(retryAfter)
? retryAfter * 1_000
: 500 * 2 ** attempt;
await new Promise((resolve) => setTimeout(resolve, delayMs));
return queryMetrics(attempt + 1);
}
if (!response.ok) {
throw new Error(`Metrics query failed (${response.status}): ${await response.text()}`);
}
return response.json() as Promise<unknown>;
}
const payload = await queryMetrics();
console.log(JSON.stringify(payload, null, 2));
// After schema validation, pass real counters to this pure decision function.
console.log({ exampleRule: shouldAlert({ attempted: 100, failed: 2 }, 0.01) });
Run it in a TypeScript environment with native fetch, after setting INFRAI_API_KEY and INFRAI_API_ORIGIN to the service origin supplied for your account. The 100 and 2 values demonstrate the pure threshold function; they aren't measurements. The production worker must replace them only after validating and mapping the live query response. Keep /health in the application itself and point an uptime checker at it; the pipeline sends its external heartbeat after committing durable output.
The short line matters: no attempts means no ratio alert. The external heartbeat owns that condition. Treating zero attempts as a perfect run is the classic blind spot; treating it as a metrics failure creates duplicate pages when the heartbeat already explains the problem.
Comparing the practical options
There are two decisions here: where counters live, and who watches time. No single product has to own both.
| Product | Best fit in this pipeline | Boundary to account for |
|---|---|---|
| Healthchecks.io | Dead-man's-switch monitoring for the nightly completion ping | It complements rather than replaces service uptime and structured-log search |
| Better Stack | Uptime checks plus incident-oriented monitoring in one hosted product | Broader workflow means more setup than a single heartbeat check |
| UptimeRobot | Straightforward HTTP uptime monitoring for /health
|
Endpoint success alone cannot detect a skipped nightly job |
| Datadog | Metrics, logs, monitors, and deeper operational correlation | The larger platform can be excessive for one small pipeline and increases platform coupling |
| Sentry | Application errors and stack-oriented debugging | It isn't the primary proof that a scheduled data import completed |
For a solo team, Healthchecks.io plus a basic uptime checker is the cleanest choice when the only scheduled workload is this importer. Pick Better Stack when one hosted incident workflow is more valuable than keeping the heartbeat tool narrow. UptimeRobot fits when HTTP reachability is the immediate need, but it still needs a heartbeat companion. Datadog makes more sense after metrics and logs already justify a full observability platform. Sentry belongs beside these choices when code-level exceptions, rather than missed schedules, dominate the investigation.
Infrai's concrete advantage is one REST API and one key across 295 routes in 20 modules, so a solo operator doesn't need a separate SDK or credential for each capability. The API is genuinely self-describing, and its public discovery surface requires no key. Every documented capability has runnable examples in 10 languages, so the metrics worker can remain ordinary TypeScript and HTTP. It can hold the reported counters and serve their queries, but it doesn't support threshold rules, outbound alert delivery, synthetic uptime checks, or heartbeats. The alert policy and delivery worker remain yours.
These limitations are the trade-off, not a footnote. It also has no distributed trace query or span tree, source-map decoding, crash symbolication, or session replay. Logs may carry trace_id and span_id for correlation, but that doesn't create a tracing backend. Choose Datadog instead when native monitors and tracing need to live together; choose Sentry instead when source-mapped application exceptions are the main debugging path. A counter store is the wrong tool for either job.
Signal quality before feature count
The tempting first version is one alert per error log. It's quick to wire and miserable to operate. A malformed optional cosmetic field, a transient retry, and a completely absent import then compete for the same attention even though their player impact is different.
Aggregate first. Report attempted and failed records with a stable pipeline name and environment, query both over the identical nightly window, and calculate the ratio in the worker. Keep the raw structured logs for diagnosis. The alert should link an operator to a bounded search around the affected run, not dump thousands of records into a notification.
Choose the threshold from observed clean runs and an explicit product tolerance, not from the 1% placeholder in the sample. Also require a minimum attempted count before judging a ratio if early batches are small. A single failure out of two records looks catastrophic mathematically and may say nothing useful operationally. This is the deliberate trade-off: delayed aggregation loses instant per-event notification, but it produces a page tied to player impact rather than log volume.
There is a privacy edge as well. If logs contain user-linked data, confirm deletion, export, subscription, retention, and regional requirements before selecting the store. A system without per-user deletion or configurable retention controls is a poor fit where the right to erasure is mandatory. US/EU availability isn't equivalent to a complete GDPR operating model.
What to measure before copying this choice
Run the design in notification-only mode for two weeks before attaching an aggressive paging policy. Record how many alerts correspond to a real missed deadline, how many are duplicate symptoms, and how long the metric query plus notification path takes. Also track the heartbeat grace period against actual job-duration variation. Five minutes of grace may work for a stable 20-minute import and fail badly during a seasonal content release.
The decision rule is blunt. Use health plus heartbeat plus polled counters when you want a small, understandable failure detector and can own a Node.js alert worker. Choose a product with native monitors and notification routing when on-call policy, auditability, distributed traces, or many services matter more than keeping the integration thin.
Feature flags can be an incident brake for a risky code path, but they shouldn't become the monitoring plan. If clients only poll and the flag system lacks change audit history and evaluation statistics, preserve the operational decision elsewhere and verify propagation before relying on it during an incident.
Ship the smallest set of signals that distinguish reachable, completed, and degraded. Then measure the pages. More telemetry is useful only when it changes the next action.
Top comments (0)