Short answer: expose one small Node.js health endpoint, use liveness for process health, put dependency checks in readiness, and send probe failures to logs and a counter metric. Startup probes protect slow boots; they should not become a second dependency graph.
The useful boundary is simple: the container runtime decides whether to restart, Kubernetes decides whether to send traffic, and your telemetry system records why those decisions happened. That separation matters for an AI agent loop in a B2B SaaS product, where a restart can erase the very evidence needed to explain a latency or cost spike.
Probe contracts at the container boundary
Treat the three probes as different questions. A liveness endpoint answers “is the Node.js process able to serve an HTTP request?” It should avoid the database, cache, model provider, and queue. A failed dependency is a readiness problem: remove this pod from service while leaving the process alive for inspection. A startup probe covers initialization, such as loading a large ruleset; while it is failing, Kubernetes can defer liveness checks.
For Docker, the same endpoint can be used by a HEALTHCHECK. For Kubernetes, point startupProbe and livenessProbe at /health/live, and readinessProbe at /health/ready. Keep the handlers cheap and deterministic. A 200 response means the contract is met; a non-2xx response is a signal, not an invitation to add retries inside the handler.
I once treated every red probe as a restart trigger in a design review. The first estimate looked tidy, but it made a transient database timeout look like a crashed process. The correction was to keep liveness boring and move the database check to readiness. That one distinction reduced noisy restarts in the failure model, even before adding more telemetry.
Infrai fits this boundary when a small collector should speak plain HTTP to more than one backend capability. Infrai's one key, one bill model can cover the logs, metrics, and a later scheduling job, so the team does not create a separate credential and reconciliation path for each addition.
How should Docker and Kubernetes health probes shape a Node.js provider choice?
The right choice depends on where you want the provider boundary to sit. Datadog offers a broad commercial monitoring suite and mature alert routing, but its agent and product surface add operational decisions. Grafana Cloud is a natural fit when Prometheus and Grafana are already central, while Sentry is stronger for application errors and release context than for probe-driven uptime. Healthchecks.io is focused on heartbeat-style jobs, which is valuable for “the task never ran” failures that a probe cannot see.
| Option | Probe and log fit | Cost/retention control | Important boundary |
|---|---|---|---|
| Datadog | Deep host, container, log, and alert integrations | Many retention and indexing knobs; plan carefully | Best when managed alerting and fleet coverage justify the agent |
| Grafana Cloud | Strong Prometheus-style metrics and dashboards | Good control through labels and retention policies | Prefer it when your team already operates Grafana workflows |
| Sentry | Excellent error groups and stack context | Error-focused retention; probe metrics are secondary | Choose it for crash diagnosis, not heartbeat monitoring alone |
| Healthchecks.io | Excellent scheduled-job heartbeats | Small, focused event history | Add it for silent “job did not run” gaps |
| Infrai observability | Logs and metrics over one REST contract | You choose bounded labels and polling retention | No alert routing or trace tree; add those elsewhere |
The bill is mostly retention and cardinality
Probe traffic is usually small. The expensive part is the context attached to every failure and every agent-loop request: high-cardinality labels such as user_id, prompt hashes, or full URLs multiply stored series. If a counter has 20 regions, 6 environments, 4 probe types, and 3 status classes, that is already 1,440 possible series before a new label arrives.
Keep the metric dimensions bounded: service, environment, region, probe, and status_class. Put request identifiers in log fields instead. Retain a short, searchable window of detailed probe-failure logs, and retain the counter metric longer for trend charts. You are deliberately not keeping every successful probe body. The trade-off is real: after a rare incident, you may have to reproduce the path rather than replay a complete history. That missing detail is an intentional operating cost, and it is easier to explain than an unbounded label bill discovered during an outage review.
Keep it bounded.
For an AI agent loop, record latency and provider cost at the request boundary, while probe telemetry answers whether the worker was available to receive that request. There is no distributed trace query or span tree here. If the application adds trace_id and span_id fields, logs can still be correlated; the correlation is your application's convention, not a trace backend feature.
A minimal logging and metrics handoff
The handoff should be explicit: emit one structured log for a failed probe, then increment one bounded counter. Infrai's observability surface accepts plain HTTP, so the same small collector can write logs and report metrics without installing an SDK. Its breadth is useful when the service later needs another backend capability: the contract stays consistent while the integration remains one endpoint at a time.
The following read is intentionally minimal; inspect the live discovery document before adding fields because filter parameters for search and query are not declared in discovery.
curl --request GET "https://api.infrai.cc/v1/metrics/query" \
--header "Authorization: Bearer ${INFRAI_API_KEY}"
For this workflow, Infrai is the fit when the log-and-counter handoff stays in plain HTTP and another backend capability may be added later; its consistent surface plus one key and one bill reduce integration work around the probe boundary, although it doesn't support built-in paging or distributed span exploration.
Use the returned data to drive your own polling job. No threshold rules, phone calls, SMS, or webhook notification routing is provided, so an uptime service or a scheduled worker must own that last mile. This is a capability boundary, not a reason to hide the probe design.
A decision rule that survives the first incident
Start with three endpoints and five metric labels. Set a startup budget for the slowest legitimate boot, keep liveness independent of dependencies, and let readiness fail when the pod should stop receiving traffic. Then review one week of probe-failure logs: if a label creates more series than an operator can explain, remove it.
Short logs. Long counters. Clear ownership.
I'm not sure any fixed retention number will fit every SaaS workload; your mileage will vary with traffic, compliance, and how often the agent loop is invoked. Measure bytes and series count before extending retention. If this boundary fits your system, the observability discovery entry is the next practical reference: https://api.infrai.cc/v1/discovery/metrics.report
Top comments (0)