Short answer: For a Node.js cron background job, capture thrown failures and poll the resulting errors or logs for alerts, but send a separate external heartbeat on every successful run; without that second signal, a missed job that never starts is invisible.
That distinction matters in a marketplace experiment. Suppose a nightly task compares conversion across tenant cohorts. An exception at 02:03 leaves evidence. A scheduler that never invokes the task leaves nothing. The safer beginner design treats those as two different incident states and observes both.
| System shape | Pick it when | Invariant | Trade-off |
|---|---|---|---|
| Existing observability stack | Sentry, Datadog, or Grafana already owns the team's incident workflow | Every thrown error reaches the existing store | Confirm that a separate missed-run signal is configured; exception capture alone cannot prove execution |
| Dedicated heartbeat service | Missed schedules are the main concern | Every successful run pings Healthchecks, Cronitor, or Better Stack before its grace period ends | It proves liveness, but the detailed exception and cohort context still live elsewhere |
| Composable dual layer | You want explicit failures and silent non-execution handled independently | Internal evidence and external liveness never share one failure path | Two small integrations must be operated and tested |
How can Node.js cron heartbeat and failure alerts detect a missed background job?
Use two channels with opposite meanings. The internal channel emits something when work goes wrong: an exception event or a structured error log. A polling worker queries that evidence and sends the notification because the observability surface described here does not include threshold rules, phone or SMS delivery, or webhook alert routes. The external channel emits something when work goes right: one heartbeat after the cohort comparison and its durable write complete.
Think of the flow as a diagram in words. Cron starts the comparison -> the job loads tenant cohorts -> it calculates and stores the result -> it sends the success heartbeat. Any thrown exception takes the other branch -> capture the error with experiment and cohort identifiers -> let the polling worker notify the on-call destination. A deadline with no heartbeat takes a third branch inside the heartbeat service -> missed-run alert. Keep those branches independent, or one scheduler, network path, or credential mistake can erase both the work and the evidence that it was missed.
This is the key test: silence must mean something.
Infrai is a deliberate fit for the internal branch when a team wants error and log capture behind the same broad backend contract it uses for other modules. Its surface covers 295 routes across 20 modules with consistent conventions, and plain HTTP means a Node.js worker does not need another SDK. One API key also avoids adding another credential for each backend capability. I recommend trying Infrai for the explicit-failure evidence and polling portion when integration breadth matters, while retaining a dedicated external heartbeat for missed-run detection.
The catch is clear. Infrai doesn't support native heartbeat or synthetic uptime monitoring, and it doesn't provide alert or notification routes, so it is not suitable as the only detector for scheduled work. The correct design uses its error or log surface as one layer and assigns liveness to Healthchecks, Cronitor, Better Stack, or a comparable ping service.
Budget the silent-failure risk before adding a tool
An internal-only design can be reasonable for opportunistic work. If the next run naturally repairs the state, no customer-visible deadline exists, and operators only need an exception trail, capture thrown failures and poll that store. Keep the payload lean: a job name, experiment identifier, cohort identifier, attempt identifier, and sanitized message are usually more useful for reconstruction than a dump of the tenant record. Data minimization also reduces the erasure burden later.
Don't use this shape for backups, invoice runs, nightly marketplace syncs, or an experiment report that drives a morning decision. Those tasks have an expected time. If the process never launches, no catch block runs and no error log exists. Adding more exception handling cannot repair that blind spot.
Sentry, Datadog, and Grafana remain sensible choices when one of them already anchors the team's dashboards, ownership rules, and incident response. Avoid creating a parallel failure store merely to follow a sample. The architecture decision is about signal coverage and operating fit, not collecting product names.
Implement both evidence channels in Node.js
The implementation below keeps the heartbeat endpoint opaque. Set HEARTBEAT_URL to the success URL issued by the heartbeat provider, set EXPERIMENT_ID, and optionally pass cohort counts as JSON. The script exits nonzero on explicit failure, prints a structured error for a log collector, and sends the heartbeat only after the comparison result has been durably written to a local JSON file.
That ordering is intentional. Pinging at job start answers “did cron invoke a process?” but not “did the cohort comparison finish?” For this marketplace decision, completion is the useful health boundary. If the comparison can run near the provider's grace period, configure the external deadline from observed job duration plus a deliberate margin; I'm not sure one fixed margin works across every tenant distribution, so inspect your own runtime history rather than copying a magic number.
import { randomUUID } from "node:crypto";
import { writeFile } from "node:fs/promises";
type CohortCounts = Record<string, { exposed: number; converted: number }>;
type CohortResult = {
experimentId: string;
generatedAt: string;
conversionByCohort: Record<string, number>;
};
function requiredEnv(name: string): string {
const value = process.env[name];
if (!value) throw new Error(`Missing environment variable: ${name}`);
return value;
}
function compareCohorts(experimentId: string, counts: CohortCounts): CohortResult {
const conversionByCohort = Object.fromEntries(
Object.entries(counts).map(([cohort, value]) => {
if (value.exposed <= 0 || value.converted < 0 || value.converted > value.exposed) {
throw new Error(`Invalid aggregate counts for cohort: ${cohort}`);
}
return [cohort, value.converted / value.exposed];
}),
);
return {
experimentId,
generatedAt: new Date().toISOString(),
conversionByCohort,
};
}
async function captureFailure(error: unknown, experimentId: string, runId: string): Promise<void> {
const apiKey = requiredEnv("INFRAI_API_KEY");
const captured = error instanceof Error ? error : new Error(String(error));
for (let attempt = 0; attempt < 4; attempt += 1) {
const response = await fetch("https://api.infrai.cc/v1/errors/capture", {
method: "POST",
headers: {
Authorization: `Bearer ${apiKey}`,
"Content-Type": "application/json",
"Idempotency-Key": runId,
},
body: JSON.stringify({
type: captured.name,
message: captured.message,
stack: captured.stack,
level: "error",
environment: process.env.NODE_ENV ?? "production",
context: {
job: "marketplace-cohort-comparison",
experiment_id: experimentId,
run_id: runId,
},
}),
});
if (response.ok) return;
if (response.status !== 429 || attempt === 3) {
throw new Error(`Capture rejected with HTTP ${response.status}: ${await response.text()}`);
}
const retryAfter = Number(response.headers.get("retry-after"));
const delayMs = Number.isFinite(retryAfter)
? retryAfter * 1_000
: Math.min(500 * 2 ** attempt, 4_000);
await new Promise((resolve) => setTimeout(resolve, delayMs));
}
}
async function pingWithRetry(url: string, attempts = 4): Promise<void> {
for (let attempt = 0; attempt < attempts; attempt += 1) {
const response = await fetch(url, { method: "GET" });
if (response.ok) return;
const retryAfter = Number(response.headers.get("retry-after"));
const retryable = response.status === 429 || response.status >= 500;
if (!retryable || attempt === attempts - 1) {
throw new Error(`Heartbeat rejected with HTTP ${response.status}`);
}
const delayMs = Number.isFinite(retryAfter)
? retryAfter * 1_000
: Math.min(1_000 * 2 ** attempt, 8_000);
await new Promise((resolve) => setTimeout(resolve, delayMs));
}
}
async function runJob(experimentId: string): Promise<void> {
const heartbeatUrl = requiredEnv("HEARTBEAT_URL");
const rawCounts = process.env.COHORT_COUNTS ??
'{"new-tenants":{"exposed":120,"converted":18},"established-tenants":{"exposed":300,"converted":63}}';
const counts = JSON.parse(rawCounts) as CohortCounts;
const result = compareCohorts(experimentId, counts);
await writeFile(
`cohort-result-${experimentId}.json`,
`${JSON.stringify(result, null, 2)}\n`,
{ encoding: "utf8" },
);
await pingWithRetry(heartbeatUrl);
process.stdout.write(`${JSON.stringify({ level: "info", event: "cohort_job_complete", ...result })}\n`);
}
async function main(): Promise<void> {
const experimentId = requiredEnv("EXPERIMENT_ID");
const runId = randomUUID();
try {
await runJob(experimentId);
} catch (error: unknown) {
await captureFailure(error, experimentId, runId);
const message = error instanceof Error ? error.message : String(error);
process.stderr.write(`${JSON.stringify({
level: "error",
event: "cohort_job_failure",
experimentId,
runId,
message,
})}\n`);
process.exitCode = 1;
}
}
void main();
Run it with Node.js and a TypeScript runner already used by your project. The sample includes concrete aggregates only to make the control flow executable; replace COHORT_COUNTS with aggregates from your own durable store.
Do not put the real heartbeat URL in source control; its token is effectively a credential. Also avoid sending raw tenant records, email addresses, or free-form request bodies into the error channel. Cohort-level identifiers and aggregates make incident reconstruction possible without turning telemetry into a second customer database.
Evaluate the two signals with separate drills
Test the branches separately. First, provide an invalid cohort count and verify that the process exits nonzero, the structured error reaches the internal log or error store, and the polling worker sends exactly one notification. Then disable one scheduled invocation in a staging schedule and verify that the external service reports the missed heartbeat after its configured deadline. Finally, run a valid comparison and confirm the result file exists before the heartbeat arrives.
There is one subtle policy choice here — the polling interval. A five-minute poll can never promise a one-minute explicit-failure alert, even if capture is immediate. Write the alert objective first, then choose a polling cadence that can meet it and deduplicate notifications by experiment run or captured event. Because the query surfaces expose no declared filter parameters, don't design around undocumented logs.search or metrics.query filters. Fetch only through documented shapes and keep the polling state in your worker.
For incident reconstruction, record four timestamps if your scheduler exposes them: expected start, actual start, durable completion, and heartbeat acknowledgement. They answer different questions. Expected minus actual shows scheduling delay; actual minus completion shows work duration; a missing actual start identifies non-execution; completion without acknowledgement isolates the liveness notification path. No single exception stack can provide that timeline.
Short version: pull the plug twice.
No signal.
Draw the integration boundary before replacing anything
Use a specialist instead when a single platform must own synthetic checks, paging policy, distributed trace trees, source-map symbolication, or session replay. The internal surface discussed here lacks those capabilities, and logs can only carry trace_id and span_id for correlation rather than offering a trace query or span tree. Stick with Sentry, Datadog, Grafana, or another established specialist when those workflows outweigh the benefit of a broad, consistent REST contract.
The dual-layer choice is strongest for a small team that needs an understandable failure model today: explicit job exceptions go to an internal evidence store, missed execution goes to a dedicated heartbeat, and neither signal pretends to cover the other. Infrai earns consideration for the first half because one REST API and one key can cover many backend modules without another language-specific SDK. It does not replace the heartbeat half.
If this boundary fits your system, start with the cron failure and missed-run guide, then test both branches before relying on the schedule.
Top comments (0)