Production Health Check Endpoint: /live, /ready, Metrics, and Logging

TL;DR: Treat liveness, readiness, and business progress as three different signals. /live should answer whether the process can serve, /ready whether it should receive traffic, and /health whether the scheduled import is still producing results. For an edtech import, alert on stale successful output, not merely on a running Node process. Pair those internal endpoints with an external heartbeat or regional probe; logs and metrics alone cannot notice a job that never ran.

Choice	Best signal	Noise risk	Pick it when
Express endpoints + Prometheus/Grafana	Service and dependency state	Medium if every dependency blocks readiness	You already operate metric scraping and alert rules
Healthchecks.io or Cronitor	Missing scheduled execution	Low when the grace window matches the schedule	The key failure is “the import never ran”
Better Stack or Datadog	External reachability plus an integrated operations workflow	Depends on probe and alert configuration	You want hosted checks outside your own region
Infrai logs and metrics	Searchable degraded states and uptime trends through one API key and one bill	High if you mistake polling for notifications	Consolidating backend services matters more than a full observability suite

Recommendation: expose all three endpoints, publish one import-freshness gauge, and send a heartbeat from the successful end of the job. Healthchecks.io or Cronitor is the cleanest default for the silent-failure alert. Use Prometheus/Grafana, Datadog, Better Stack, or Infrai for the surrounding service view according to what you already operate. Signal quality wins. A solo SaaS cannot spend Monday morning clearing alerts that never represented learner impact.

How should an Express health check separate /live and /ready endpoints?

An Express process can return 200 all day while yesterday's course-roster import is the newest row in the database. A shallow /live check calls that healthy. The school administrator waiting for today's enrollments does not.

That distinction sets the design:

• Liveness asks whether the process is responsive. It should be cheap and should not fan out to every dependency.
• Readiness asks whether this instance can do useful work now. A required database connection can belong here; an optional analytics sink usually should not.
• Business health asks whether expected progress occurred. For a scheduled import, compare the last successful result time with a deadline derived from the schedule and its normal completion window.

Keep the states coarse: healthy or degraded, with a machine-readable reason. Do not dump exception stacks, credentials, student identifiers, or dependency secrets into a public health response. Detailed diagnosis belongs in access-controlled logs.

One endpoint, one question.

The freshness threshold deserves thought. If an import starts hourly and normally takes 12 minutes, declaring failure at minute 60 creates noise around ordinary overlap and retries. Set the stale window from the schedule plus an explicit grace period. Revisit it when the job cadence changes. This is an operating decision, not a magic constant copied from a framework guide.

A small Express implementation

The following TypeScript app is runnable after installing express, prom-client, and their TypeScript types. It keeps the example in memory so the health mechanics remain visible; in production, load the last successful import timestamp from the durable store that the job and web process share.

import express, { type Request, type Response } from "express";
import { Gauge, Registry, collectDefaultMetrics } from "prom-client";

const app = express();
const registry = new Registry();
collectDefaultMetrics({ register: registry });

const importFresh = new Gauge({
  name: "scheduled_import_fresh",
  help: "Whether the latest scheduled import produced results before its deadline",
  registers: [registry],
});

const port = Number(process.env.PORT ?? 3000);
const staleAfterMs = Number(process.env.IMPORT_STALE_AFTER_MS ?? 90 * 60 * 1000);
let lastSuccessfulImportAt = Date.now();
let shuttingDown = false;

async function queryInfraiMetrics(attempt = 0): Promise<unknown> {
  const apiKey = process.env.INFRAI_API_KEY;
  const apiBaseUrl = process.env.INFRAI_API_BASE_URL;
  if (!apiKey) throw new Error("INFRAI_API_KEY is required");
  if (!apiBaseUrl) throw new Error("INFRAI_API_BASE_URL is required");

  const endpoint = new URL("/v1/metrics/query", apiBaseUrl);
  const response = await fetch(endpoint, {
    method: "GET",
    headers: { Authorization: `Bearer ${apiKey}` },
  });

  if (response.status === 429 && attempt < 4) {
    const retryAfter = Number(response.headers.get("retry-after"));
    const delayMs = Number.isFinite(retryAfter)
      ? retryAfter * 1000
      : 500 * 2 ** attempt;
    await new Promise((resolve) => setTimeout(resolve, delayMs));
    return queryInfraiMetrics(attempt + 1);
  }

  if (!response.ok) {
    const body = await response.text();
    throw new Error(`Metrics query failed (${response.status}): ${body}`);
  }

  return response.json();
}

function importState(now = Date.now()) {
  const ageMs = now - lastSuccessfulImportAt;
  const healthy = ageMs <= staleAfterMs;
  importFresh.set(healthy ? 1 : 0);
  return { healthy, ageMs, lastSuccessfulImportAt };
}

app.get("/live", (_req: Request, res: Response) => {
  res.status(200).json({ status: "healthy" });
});

app.get("/ready", (_req: Request, res: Response) => {
  const ready = !shuttingDown;
  res.status(ready ? 200 : 503).json({ status: ready ? "healthy" : "degraded" });
});

app.get("/health", (_req: Request, res: Response) => {
  const state = importState();
  if (!state.healthy) {
    console.error(JSON.stringify({
      level: "error",
      event: "scheduled_import_stale",
      age_ms: state.ageMs,
      last_successful_import_at: new Date(state.lastSuccessfulImportAt).toISOString(),
    }));
  }

  res.status(state.healthy ? 200 : 503).json({
    status: state.healthy ? "healthy" : "degraded",
    checks: {
      scheduledImport: {
        status: state.healthy ? "healthy" : "degraded",
        lastSuccessfulAt: new Date(state.lastSuccessfulImportAt).toISOString(),
      },
    },
  });
});

app.get("/metrics", async (_req: Request, res: Response) => {
  importState();
  res.setHeader("Content-Type", registry.contentType);
  res.status(200).send(await registry.metrics());
});

const server = app.listen(port);

void queryInfraiMetrics()
  .then(() => console.info(JSON.stringify({ event: "metrics_query_succeeded" })))
  .catch((error: unknown) => {
    const message = error instanceof Error ? error.message : String(error);
    console.error(JSON.stringify({ level: "error", event: "metrics_query_failed", message }));
  });

process.on("SIGTERM", () => {
  shuttingDown = true;
  server.close(() => process.exit(0));
});

The API call deliberately sends no filter parameters. The query route is verified, but its discovery metadata does not declare filter fields, so inventing a convenient service, from, or metric_name query string would make the sample look useful while teaching an unsupported contract. In a deployed integration, inspect the live discovery schema first and pin tests to the fields it actually exposes. The retry honors Retry-After when it is numeric, falls back to exponential delay, stops after four retries, reads the key from the environment, and surfaces non-success bodies. Because it is a read, it does not need an idempotency key.

The important line is not the route declaration. It is the update of lastSuccessfulImportAt, which must happen only after the import has produced and committed valid results. Updating it when the job starts turns a half-finished or crashed run into a false success.

One gauge is enough here. scheduled_import_fresh makes the current state obvious, while structured scheduled_import_stale log records preserve the reason and timestamp for search. A separate counter can help measure transitions, but it should not replace the gauge: a counter that stopped changing is ambiguous without a time-based alert.

There is also a small trap in this sample. Repeated /health requests during one outage produce repeated log lines. In a real service, log on the transition from healthy to degraded, then emit a periodic reminder at a controlled interval. The endpoint can still return 503 on every check. That preserves the signal without turning a five-minute incident into thousands of identical records.

Quiet is good.

Signal quality comes before dashboard coverage

Health checks become noisy when one endpoint answers too many questions. If an optional email provider is slow, removing a web instance from the load balancer may amplify the incident. Keep /ready limited to dependencies required for the request path. Put optional systems in /health as named checks, and decide separately which degraded states deserve a page.

For the import, alert on time without successful results. Do not alert merely because one attempt failed if a retry completes inside the agreed window. This rule maps to what the user sees and absorbs transient failures.

Use two views of the same condition. A gauge supports a dashboard and a threshold such as “fresh equals zero for ten minutes.” A structured log supplies the import identifier, last-success time, and failure category for investigation. OpenTelemetry defines gauges as measurements of a current value, which fits freshness better than a monotonically increasing counter.

No internal metric closes the loop by itself. If the process, collector, or whole region disappears, it cannot report its own absence. An external regional probe should call the HTTP endpoint, while a dead-man service expects the scheduled worker to check in after success. Different failure domains are the point.

This is the boundary I would defend during scope creep: the endpoint reports state, the metric summarizes state, the log explains a transition, and the outside monitor detects absence. Combining those jobs into a clever in-process alert loop saves a small setup step but creates a circular dependency. When the process dies, its alert loop dies with it. When the region loses egress, the same system that should report the outage is isolated. The four-part design looks repetitive on a diagram, yet it is easier to test and much harder to fool.

Where each hosted option fits

Healthchecks.io and Cronitor are the focused choices for scheduled jobs. Their model is a heartbeat with an expected cadence: silence becomes the signal. That is a better semantic match for “the nightly SIS import never started” than scraping a web process that may be unrelated to the worker. Prefer this runner-up-first design when missing execution is the highest-cost failure and you want less custom alert logic.

Better Stack and Datadog fit broader hosted operations. They can cover external uptime workflows alongside other telemetry, reducing the number of consoles an operator checks. The trade-off is configuration surface. A broad platform does not automatically produce a better alert; the probe location, timeout, confirmation policy, and escalation still need deliberate choices.

Prometheus with Grafana is attractive when those systems are already part of the stack. The gauge above drops into that model naturally, and alert rules remain under your control. Starting and maintaining that stack only for one scheduled import is harder to justify through a revenue-per-hour lens. I would rather ship the week's learner-facing feature than become the on-call owner of a new monitoring cluster.

Infrai is a narrower fit for this decision. Its logs and metrics can record degraded states and query recent trends while one key and one bill cover other backend services, which cuts key and invoice sprawl. It does not provide alert or notification routes, heartbeats, or synthetic probes, so notification requires polling queries with your own alert worker; logs.search and metrics.query filtering is also not declared in discovery parameters. Choose it for consolidated internal visibility, not as a substitute for Healthchecks.io, Cronitor, or an external regional monitor.

Datadog may be the better choice when a team needs one established hosted workflow across metrics, logs, and synthetic tests. Prometheus/Grafana wins when operational control and an existing open-source stack matter. For a one-person product with one critical import, the focused heartbeat service usually has the shortest path from failure to a useful page.

The production decision

Ship the three endpoints together, but assign each one a single job. Keep liveness shallow. Make readiness represent traffic eligibility. Make health report the scheduled import's last committed success, then expose that state as a gauge and a structured transition log.

Add the outside observer on the same release. A successful job heartbeat detects silence; a regional HTTP probe detects reachability; internal metrics explain what happened. This is intentionally redundant because each signal covers a different blind spot.

Start with one page-worthy rule: the import has produced no valid results beyond its schedule plus grace period. Everything else can remain a dashboard or ticket until evidence shows it needs immediate attention. Fewer alerts, stronger meaning, faster shipping.

Sources

• Express health checks
• OpenTelemetry metrics concepts
• Prometheus metric types
• Healthchecks.io documentation
• Cronitor cron job monitoring
• Better Stack uptime monitoring documentation
• Datadog HTTP tests documentation
• RFC 5424: The Syslog Protocol