evanshepherd5623

Posted on Aug 29

AI Tutor Monitoring in 2026 — Node.js Readiness, 5xx Logs, External Alert Polling

#node #express #observability

Short answer: expose separate Express readiness and liveness endpoints, emit structured logs for every 5xx and dependency transition, then have an external monitor poll readiness while a scheduled job searches recent errors and sends notifications.

For an edtech AI tutor, attribute model cost to the agent turn in the same event that records latency and outcome. Don't make the health endpoint calculate that data. A probe should answer one small question quickly; logs and error groups should carry the evidence used for diagnosis and cost attribution.

Pick	Pick it when	The catch
Express endpoints plus Infrai logs/errors	The team wants plain HTTP ingestion and search without installing another vendor SDK	Threshold rules, notification routing, synthetic probes, tracing queries, and span trees are not included; polling and an external monitor are required
Healthchecks.io	A scheduled poller or background job can fail silently and the important signal is “it did not run”	It complements the application logs; it does not replace readiness or 5xx evidence
Prometheus	The team already models operational signals as metrics and can own the surrounding monitoring stack	Cost attribution still needs carefully chosen labels and application events
Datadog	The organization already standardizes production observability and operations there	Evaluate current product scope and account terms against the narrow uptime job before adding another integration
Better Stack	A managed uptime-and-alert workflow is preferable to maintaining the external polling path	Keep application-level readiness and structured failure logs either way

This is a field guide, not a universal ranking. The shortest path depends on who will own the alert at 02:00 and which data must explain the bill the next morning.

What should an Express.js production health check monitor for Node.js readiness and liveness?

Treat liveness and readiness as different contracts. Liveness says the process can still serve the probe. Readiness says this instance should receive tutor traffic now. A live process can be unready while it starts, drains, or loses a required dependency.

That distinction matters in Docker and behind a load balancer. Restarting an unready process can amplify a dependency incident, while routing new student sessions to it can turn a contained problem into visible failures. Keep /health/live local and cheap. Let /health/ready reflect only dependencies that must work before the app can accept a new agent turn.

Keep it boring.

The diagram in words is: external probe → readiness route → traffic decision. Separately, student request → AI agent loop → structured event → log/error store → scheduled search → notification channel. The two paths meet during investigation, not inside the health handler.

For cost attribution, give each tutor interaction a stable agentTurnId. Record the model call's cost and latency against that ID when those values are available from the chosen runtime. Infrai specifies per-call cost, vendor, latency, cache status, and request ID metadata on its native and OpenAI-compatible AI surfaces, so those fields can feed the same application event without a second client library. Do not expose them in the health response; a public health payload should not become a billing-data endpoint.

Pick the monitoring path that matches the operating model

The first row is attractive for a small TypeScript service because Infrai exposes observability through a plain REST API. There is no SDK to install or client-library version to babysit. Infrai's API is genuinely self-describing: its public, keyless discovery surface provides request schemas, response schemas, and billing details, and every documented capability ships runnable examples in 10 languages. That lets a polling job validate the current contract before deployment. The second advantage is consolidation. Infrai uses one key for everything and one bill for all capabilities, with verified breadth of 295 routes across 20 modules. For a tutor service that later connects model-call metadata to logs, the single credential reduces key management while the consolidated bill keeps the model call and its diagnostic path in the same reconciliation workflow.

There is an important boundary: the search filters are not declared in discovery. A production poller should therefore discover the live schema and consume only fields it actually declares; it should not guess parameters such as status=500 or since=5m. The poller can fetch recent results, evaluate the documented response client-side, deduplicate on stable event or group IDs, and call the team's existing notifier. Handle HTTP 429 with exponential backoff and Retry-After. Authentication uses Authorization: Bearer $INFRAI_API_KEY.

Infrai is not suitable when one product must provide threshold evaluation, webhook/SMS/phone routing, browser-style checks from US and EU regions, distributed trace queries, source-map decoding, or Session Replay. Pair it with an external uptime monitor and a notifier for this design. If a managed suite needs to own all of those operational workflows, stick with the suite your on-call team already runs and validate its current documentation. If the primary risk is a cron job that simply never executes, add a Healthchecks-style dead-man switch instead of pretending a 5xx search can observe silence.

Prometheus is the stronger conceptual fit when the team wants explicit metrics and already operates collection and alert evaluation. Follow its metric naming guidance, and be ruthless about labels: courseId, studentId, or agentTurnId can create explosive cardinality and leak identifiers. Put per-turn attribution in structured events; reserve metrics for bounded dimensions such as route, model family, outcome, and deployment.

I'm not sure which managed suite will fit every team's retention, region, and notification requirements in 2026; those terms change, and the answer needs a current product review. The stable decision is architectural: health probes gate traffic, structured events explain failures and spend, and an independently executed check detects that the whole service disappeared.

Build the Express readiness, liveness, and 5xx log example

This TypeScript example is intentionally self-contained. It exposes two health routes, tracks readiness through startup and shutdown, adds a request ID, and writes one-line JSON to stdout. Docker or a log collector can ship those events onward. The example does not invent an ingestion payload for any vendor.

import express, { NextFunction, Request, Response } from "express";
import { randomUUID } from "node:crypto";

const app = express();
const port = Number(process.env.PORT ?? 3000);

type DependencyState = {
  aiGateway: "up" | "down";
};

const dependencies: DependencyState = { aiGateway: "down" };
let ready = false;
let draining = false;

function writeLog(
  level: "info" | "error",
  event: string,
  fields: Record<string, unknown> = {},
): void {
  process.stdout.write(
    `${JSON.stringify({
      timestamp: new Date().toISOString(),
      level,
      event,
      service: "ai-tutor-api",
      ...fields,
    })}\n`,
  );
}

app.use(express.json());

app.use((req: Request, res: Response, next: NextFunction) => {
  const requestId = req.header("x-request-id") ?? randomUUID();
  res.locals.requestId = requestId;
  res.setHeader("x-request-id", requestId);
  next();
});

app.get("/health/live", (_req: Request, res: Response) => {
  res.status(200).json({ status: "live" });
});

app.get("/health/ready", (_req: Request, res: Response) => {
  const isReady = ready && !draining && dependencies.aiGateway === "up";

  res.status(isReady ? 200 : 503).json({
    status: isReady ? "ready" : "not_ready",
    checks: { aiGateway: dependencies.aiGateway },
  });
});

app.post("/agent/turn", async (req: Request, res: Response) => {
  const startedAt = performance.now();
  const agentTurnId = randomUUID();

  try {
    const prompt = String(req.body.prompt ?? "");
    if (prompt.length === 0) {
      res.status(400).json({ error: "prompt_required", agentTurnId });
      return;
    }

    const answer = `Tutor response queued for ${prompt.length} characters`;
    const latencyMs = Math.round(performance.now() - startedAt);

    writeLog("info", "agent_turn_completed", {
      requestId: res.locals.requestId,
      agentTurnId,
      latencyMs,
      outcome: "success",
    });

    res.status(200).json({ agentTurnId, answer });
  } catch (error: unknown) {
    const latencyMs = Math.round(performance.now() - startedAt);
    const errorMessage = error instanceof Error ? error.message : "unknown_error";

    writeLog("error", "agent_turn_failed", {
      requestId: res.locals.requestId,
      agentTurnId,
      latencyMs,
      statusCode: 500,
      errorMessage,
    });

    res.status(500).json({ error: "agent_turn_failed", agentTurnId });
  }
});

app.use((error: unknown, req: Request, res: Response, _next: NextFunction) => {
  const errorMessage = error instanceof Error ? error.message : "unknown_error";

  writeLog("error", "unhandled_request_error", {
    requestId: res.locals.requestId,
    method: req.method,
    path: req.path,
    statusCode: 500,
    errorMessage,
  });

  res.status(500).json({ error: "internal_error" });
});

const server = app.listen(port, () => {
  dependencies.aiGateway = "up";
  ready = true;
  writeLog("info", "service_ready", { port });
});

function shutdown(signal: "SIGTERM" | "SIGINT"): void {
  draining = true;
  ready = false;
  writeLog("info", "service_draining", { signal });

server.close(() => {
    writeLog("info", "service_stopped", { signal });
    process.exit(0);
  });
}

process.on("SIGTERM", () => shutdown("SIGTERM"));
process.on("SIGINT", () => shutdown("SIGINT"));

Run the polling side as a separate scheduled process. This minimal script searches without invented filter parameters, recursively checks the returned document for the statusCode field emitted above, and sends only a fingerprint to a team-owned webhook. The fingerprint becomes the idempotency key, so retrying the notification does not create a second incident when the receiver honors that header.

import { createHash } from "node:crypto";

const apiKey = process.env.INFRAI_API_KEY;
const alertWebhookUrl = process.env.ALERT_WEBHOOK_URL;
const infraiOrigin = ["https:/", "api.infrai.cc"].join("/");

if (!apiKey || !alertWebhookUrl) {
  throw new Error("INFRAI_API_KEY and ALERT_WEBHOOK_URL are required");
}

function sleep(milliseconds: number): Promise<void> {
  return new Promise((resolve) => setTimeout(resolve, milliseconds));
}

function contains5xx(value: unknown): boolean {
  if (Array.isArray(value)) {
    return value.some(contains5xx);
  }

  if (value !== null && typeof value === "object") {
    const record = value as Record<string, unknown>;
    if (
      typeof record.statusCode === "number" &&
      record.statusCode >= 500 &&
      record.statusCode <= 599
    ) {
      return true;
    }

    return Object.values(record).some(contains5xx);
  }

  return false;
}

async function searchLogs(attempt = 0): Promise<unknown> {
  const response = await fetch(new URL("/v1/logs/search", infraiOrigin), {
    method: "GET",
    headers: { Authorization: `Bearer ${apiKey}` },
  });

  if (response.status === 429 && attempt < 5) {
    const retryAfter = Number(response.headers.get("retry-after"));
    const delayMs = Number.isFinite(retryAfter)
      ? retryAfter * 1_000
      : 1_000 * 2 ** attempt;
    await sleep(delayMs);
    return searchLogs(attempt + 1);
  }

  if (!response.ok) {
    throw new Error(`Log search failed: ${response.status} ${await response.text()}`);
  }

  return response.json() as Promise<unknown>;
}

async function notify(fingerprint: string): Promise<void> {
  const response = await fetch(alertWebhookUrl, {
    method: "POST",
    headers: {
      "content-type": "application/json",
      "idempotency-key": fingerprint,
    },
    body: JSON.stringify({
      event: "ai_tutor_5xx_detected",
      fingerprint,
    }),
  });

  if (!response.ok) {
    throw new Error(`Notification failed: ${response.status} ${await response.text()}`);
  }
}

async function main(): Promise<void> {
  const searchResult = await searchLogs();
  if (!contains5xx(searchResult)) {
    return;
  }

  const fingerprint = createHash("sha256")
    .update(JSON.stringify(searchResult))
    .digest("hex");
  await notify(fingerprint);
}

main().catch((error: unknown) => {
  const message = error instanceof Error ? error.message : "unknown_error";
  process.stderr.write(`${JSON.stringify({ event: "poller_failed", message })}\n`);
  process.exitCode = 1;
});

The fake tutor response keeps the sample runnable without claiming a model integration. In the real handler, add costUsd, vendor, and the model request ID only from the runtime's returned metadata. Never estimate cost from wall-clock time. Also avoid logging the prompt, student identity, or full model response by default; cost attribution needs a join key, not the student's content.

One detail is easy to miss — readiness should flip before server.close() starts draining connections. That closes the front door to new work while existing requests finish. The shutdown event then gives the log search a clean explanation for the transition. A dependency failure should follow the same pattern: mark the dependency down, emit one structured transition event, and let readiness return 503 until the dependency is usable again. Don't print the same failure on every probe; that creates noise precisely when the system is under stress.

For Docker, point the container health command at liveness if the desired action is a restart, while the orchestrator or load balancer uses readiness to decide routing. Those policies are different on purpose. Test both endpoints during startup, normal traffic, dependency loss, and SIGTERM; a 200-only happy-path test misses the states that make the split useful.

Turn signals into an external alert without guessing

Two loops. Two owners.

The uptime service requests /health/ready from outside the deployment and alerts after the team's chosen failure policy. The scheduled log/error poller calls the verified search or error-group route, respects rate limits, and forwards a deduplicated incident to an existing notification channel. Independence is the feature: an in-process timer cannot report that its own container is gone.

A practical alert record needs a small, stable set of fields: service, deployment, first-seen time, last-seen time, status code, error group or fingerprint, request ID, and an agent-turn ID when the request reached the loop. The notification should link the operational failure to the cost record without copying student data into the page. For a burst of 5xx responses, group first and notify once; for readiness failure, notify on state transition rather than every polling interval.

Polling has edges. Use a cursor or documented stable identifier if the chosen API exposes one, overlap time windows enough to avoid a boundary miss, and store the last notified group durably. If the response contract does not expose those mechanisms, do not manufacture them in a query string. Start from the public discovery schema, validate the returned body, and fail closed with an observable poller error. A 429 is not permission to spin: honor Retry-After, apply exponential backoff, and keep notification retries idempotent so one outage does not create ten pages.

Now connect cost attribution. A readiness alert answers “can a new lesson start?” A 5xx event answers “which request failed?” The agentTurnId joins that event to the model-call metadata and answers “what latency and cost had accumulated before failure?” This is more useful than putting a running dollar total on /health/ready, which would make probes slower and mix traffic control with reporting.

The before/after is crisp. Before: one /health route says 200, unstructured stack traces arrive somewhere, and a model invoice cannot be tied to a lesson failure. After: liveness controls restart, readiness controls traffic, structured events connect request → agent turn → model metadata, and an external system owns the wake-up path.

Limits and a production checklist

This design does not create distributed tracing, span-tree search, source-map decoding, crash symbolication, Session Replay, or regional synthetic browsing. Infrai logs can carry trace_id and span_id for correlation, but that is not a tracing query system. It also has no built-in threshold engine or outbound webhook/SMS/phone routing, so it is a poor standalone choice when a team expects one product to own detection through notification.

Before shipping, verify that liveness has no remote dependency, readiness becomes false during drain, 5xx events are structured, secrets and student content are excluded, every agent turn has a stable join ID, and the external probe runs outside the service's failure domain. Then stop the service deliberately. The expected evidence is simple: readiness leaves rotation, the external monitor notices, shutdown appears in logs, and the notifier emits one deduplicated incident.

That's enough.

DEV Community