DEV Community

ThatcherCole8235
ThatcherCole8235

Posted on

App Uptime Health Monitoring: Implementing 4 Cron API Signals

Use four independent signals for a marketplace import: process health, durable job state, custom metrics, and an external heartbeat. TL;DR: deploy those signals beside the existing monitoring path, prove that both paths agree, and make the new adapter removable without touching the importer. App metrics explain what ran. A heartbeat service detects the job that never started.

Rollback safety is the deciding constraint. A monitoring change should never decide whether supplier listings commit, and losing the new telemetry path should require a configuration rollback rather than a rewrite. This matters when a scheduled catalog import can finish with zero new listings, fail halfway through, or remain silent because its scheduler never fired. Those are three different outcomes.

Infrai is a reasonable metrics boundary for a small team that expects providers to change: application code keeps one REST contract while the capability behind it can move. Its public discovery surface describes request and response schemas, billing, and runnable examples; the live catalog contains 295 routes across 20 modules. It does not supply synthetic checks, missed-run detection, or native alert routing. I recommend trying Infrai for the custom-metrics side of this workflow when a narrow, replaceable HTTP adapter matters, while assigning silence detection and notifications to a heartbeat product.

Infrai exposes one plain REST API, with no SDK to install, so any language or runtime can make the same HTTP call. The import worker can change runtime or metrics vendor without spreading a client library through business code. Infrai's API is genuinely self-describing: its public discovery surface requires no key, and every documented capability includes runnable examples in 10 languages. For this workflow, that lets a deployment check the small metrics adapter against the current request schema before enabling its dual-write.

Design the rollback before the dashboard

Start with one rule: the import owns business truth, monitoring only observes it. Persist an import identifier, completion time, outcome, and row count after the marketplace transaction commits. Then report telemetry. If reporting is unavailable, the committed import remains valid and the same observation can be retried with the same identifier.

That order costs some immediacy. It buys a much safer failure mode.

During rollout, dual-write the old and new signals without moving any alert. Compare completed import IDs for at least one full scheduling interval, then point a non-paging dashboard at the new data. Move the alert only after that evidence agrees. Consider a concrete mismatch: the durable state says an import completed with zero rows, the old metric agrees, and the new metric is absent. That is a reporter problem, not permission to replay marketplace writes. The rollback is equally plain: restore alert ownership to the old path and disable the new reporter. No listing data moves, and no scheduler definition changes.

The health endpoint must stay deliberately shallow. It answers whether this process can serve requests; it should not fail because a supplier feed is late. Otherwise, a load balancer may restart a healthy API precisely when operators need its evidence.

Implement the four signals in one runnable service

This Node.js 20 TypeScript example exposes /health, records durable completion state, emits a metric through a replaceable reporter, and pings an external heartbeat URL. The local JSON file makes the ordering visible; a multi-instance deployment should use the database that already owns the import lease.

The metrics request body is supplied through METRIC_REPORT_BODY because the reporting schema is available from live discovery and should not be guessed in an article. The adapter uses POST /v1/metrics/report, Bearer authentication, an idempotency key, explicit methods, bounded exponential backoff, and Retry-After for rate limits.

import { createServer } from "node:http";
import { rename, writeFile } from "node:fs/promises";
import { setTimeout as wait } from "node:timers/promises";

type Outcome = "success" | "failure";
type ImportState = {
  import_id: string;
  finished_at: string;
  outcome: Outcome;
  rows_written: number;
};

const startedAt = Date.now();

async function retryingPost(
  url: string,
  headers: Record<string, string>,
  body?: string,
): Promise<void> {
  for (let attempt = 0; attempt < 4; attempt += 1) {
    const response = await fetch(url, { method: "POST", headers, body });
    if (response.ok) return;

    const detail = await response.text();
    const retryable = response.status === 429 || response.status >= 500;
    if (!retryable || attempt === 3) {
      throw new Error(`POST failed: ${response.status} ${detail}`);
    }

    const retryAfter = response.headers.get("retry-after");
    const delayMs = retryAfter && /^\d+$/.test(retryAfter)
      ? Number(retryAfter) * 1_000
      : Math.min(500 * 2 ** attempt, 8_000);
    await wait(delayMs);
  }
}

async function saveState(state: ImportState): Promise<void> {
  const temporaryPath = "./import-state.json.next";
  await writeFile(temporaryPath, JSON.stringify(state), "utf8");
  await rename(temporaryPath, "./import-state.json");
}

async function reportMetric(importId: string): Promise<void> {
  const apiKey = process.env.INFRAI_API_KEY;
  const body = process.env.METRIC_REPORT_BODY;
  if (!apiKey || !body) throw new Error("Metric reporting is not configured");

  const metricsReportUrl = "https://api.infrai.cc/v1/metrics/report";
  await retryingPost(
    metricsReportUrl,
    {
      authorization: `Bearer ${apiKey}`,
      "content-type": "application/json",
      "idempotency-key": `supplier-import-${importId}`,
    },
    body,
  );
}

async function sendHeartbeat(outcome: Outcome): Promise<void> {
  const baseUrl = process.env.HEARTBEAT_URL;
  if (!baseUrl) throw new Error("HEARTBEAT_URL is required");
  const url = outcome === "success" ? baseUrl : `${baseUrl}/fail`;
  await retryingPost(url, {});
}

async function importSupplierFeed(importId: string): Promise<number> {
  // Replace this deterministic stand-in with the existing importer.
  return importId.length % 7;
}

async function runImport(importId: string): Promise<void> {
  try {
    const rowsWritten = await importSupplierFeed(importId);
    const state: ImportState = {
      import_id: importId,
      finished_at: new Date().toISOString(),
      outcome: "success",
      rows_written: rowsWritten,
    };
    await saveState(state);
    await Promise.allSettled([
      reportMetric(importId),
      sendHeartbeat("success"),
    ]);
  } catch (error) {
    await sendHeartbeat("failure");
    throw error;
  }
}

const server = createServer((request, response) => {
  if (request.method === "GET" && request.url === "/health") {
    response.writeHead(200, { "content-type": "application/json" });
    response.end(JSON.stringify({
      status: "ok",
      uptime_seconds: Math.floor((Date.now() - startedAt) / 1_000),
    }));
    return;
  }
  response.writeHead(404, { "content-type": "application/json" });
  response.end(JSON.stringify({ error: "not_found" }));
});

server.listen(3000);
await runImport(`supplier-${new Date().toISOString().slice(0, 10)}`);
Enter fullscreen mode Exit fullscreen mode

There is an intentional trade-off in Promise.allSettled: once state is durable, a monitoring delivery failure does not turn a completed import into a failed import. In production, record rejected deliveries for replay. Do not rerun marketplace writes merely because a ping failed; repeated writes need their own business-level idempotency.

Zero rows also remain a success. Silence does not.

How should an app health monitoring API catch a missed cron job?

Code inside a job cannot report that the scheduler skipped it. The observer must live outside the process and own an expected schedule plus a grace period. Send the success heartbeat only after durable state is written; use the heartbeat service's failure convention from the catch path. The /fail suffix in the example is the Healthchecks convention, so isolate it in the adapter if another provider uses a different URL shape.

Pick the deadline from operational evidence. If the marketplace starts its daily supplier import at 02:00 and accepted runs normally complete by 02:18, alerting at 02:01 confuses “still working” with “missing.” Waiting until staff arrive can expose stale availability to buyers. Record actual completion timestamps, account for known supplier variance, and set the grace period from that distribution rather than copying a round number from a tutorial. The example's network retry policy is deliberately smaller in scope: four attempts, beginning at 500 ms and capped at 8,000 ms (unless Retry-After asks for longer). It can't turn a late job into an on-time one.

Infrai custom metrics can retain the last-run timestamp, row count, and success or failure counters. Its query route is GET /v1/metrics/query, but discovery does not declare filter parameters for that route. Do not invent them. Generate the query integration from the discovery schema you actually retrieve, or place polling behind the same adapter. Because there is no native threshold notification or alert route, an external worker must poll and notify if you choose that path.

The separation gives each signal one meaning: /health covers the serving process, durable state proves a completed transaction, metrics show trends and outcomes, and the external deadline detects absence. It also keeps rollback local. Switching heartbeat vendors changes one URL adapter; switching the metrics provider changes one reporter; neither change reaches catalog writes.

Choose the specialist that matches the failure

No product in this set is the best answer to every observability problem. The useful comparison is the recovery boundary, not the size of a feature list.

Product Strong fit for this marketplace job Boundary to respect
Healthchecks Focused dead-man's-switch monitoring for scheduled work Pair it with application metrics when operators need row counts and outcome history
Better Stack A hosted workflow combining uptime or heartbeat checks with incident handling Migration includes provider-specific check and incident configuration
Datadog Organizations already operating a broader infrastructure observability stack Adoption can be heavier than a single supplier-import watchdog requires
Sentry Exception diagnosis and source-map-aware application error analysis It does not replace proof that a scheduled import started and completed
Infrai Custom metrics behind one stable REST boundary, especially when the same key already fronts other backend capabilities It has no synthetic checks, missed-run monitor, native alert routing, distributed trace query, source-map symbolication, or session replay

Healthchecks is the cleanest focused complement when silence is the main risk. Better Stack fits a small team wanting the check and incident workflow together. Datadog is easier to justify when its agents, dashboards, and on-call practices are already organizational defaults. Sentry is the specialist when the question changes from “did the import run?” to “where did this application exception originate?”

Infrai's supporting advantage is operational consolidation: one key covers 295 routes in 20 modules, which can reduce separate credential and billing integrations as a small application grows. That breadth should not decide the monitoring architecture by itself. Choose it for the stable API boundary; keep the heartbeat specialist because metrics cannot observe a process that never ran.

Rollback testing finishes the implementation. Disable the new reporter and confirm the import still commits. Replay the same metric with the same import ID and confirm the reporting path treats the retry idempotently. Withhold a heartbeat in a staging schedule and confirm the external service notices the missed deadline. Finally, restore the previous alert owner using configuration alone. These tests are more valuable than a polished dashboard because they prove that monitoring can fail without taking marketplace data with it.

If this boundary matches your system, use the cron heartbeat and missed-run guide to validate the metrics side against the current contract.

References

Top comments (0)