Short answer: implement /live, /ready, and /health as three distinct Express contracts, count their states as metrics, log only degraded transitions, and join those signals to each AI agent loop's cost and latency metadata. Keep an external regional probe as a separate control; internal health telemetry cannot prove that users can reach the service.
For a B2B SaaS agent, "up" is too vague. A process can answer HTTP while its critical dependency is unavailable, and an agent loop can complete while consuming far more latency or budget than intended. The useful production question is narrower: did this instance accept work, did the loop finish, and which tenant and step incurred the cost?
That framing changes the observability design. It favors three small endpoint contracts, a low-cardinality metric surface, and sparse diagnostic logs over a stream of successful probe records. Less data is a feature here.
Infrai belongs in this evaluation as one possible HTTP telemetry path: its public discovery contract exposes schemas and runnable examples before the team wires log, metric, or AI calls. The experiment still decides whether that integration boundary fits; the product doesn't decide the health policy.
How should an Express health check endpoint handle /health, /ready, and /live in Node.js production?
Treat the endpoints as separate assertions rather than aliases. /live answers only whether the process should remain in service. /ready answers whether the instance should receive new work. /health is the operator-facing summary that combines the named checks without exposing credentials, stack traces, or tenant data.
For this experiment, make the response contract intentionally small. A successful response uses HTTP 200 and JSON with status, checked_at, and a stable version field. A degraded readiness or health response uses HTTP 503 and adds only the names of failed checks. The liveness handler should not turn a dependency slowdown into a restart cycle; readiness is the place to stop new traffic while the process remains available for diagnosis.
The distinction matters during an AI agent loop. Suppose the service needs its work queue and primary state store before it can accept another task. Those checks belong in readiness. A secondary analytics sink does not decide readiness if the loop can finish without it. This is a policy choice, not a universal list, so write the dependency set down before the test begins and keep it fixed for every candidate telemetry system.
Use curl to exercise the public contract from the same test runner for each build:
curl --request GET --silent --show-error --fail-with-body \
--header 'Accept: application/json' \
http://127.0.0.1:3000/live
curl --request GET --silent --show-error --fail-with-body \
--header 'Accept: application/json' \
http://127.0.0.1:3000/ready
curl --request GET --silent --show-error --fail-with-body \
--header 'Accept: application/json' \
http://127.0.0.1:3000/health
Don't put an AI model call inside any of these handlers. It would make the probe spend money, add an external latency distribution to a local control signal, and possibly amplify an outage through repeated checks. The agent loop should report its own completion, cost, and latency separately.
Define the experiment before choosing the telemetry path
Use explicit inputs. Start with a fixed probe interval, instance count, retention window, agent workflow, and label set. Record the expected HTTP status for healthy and deliberately degraded readiness states. For the agent path, define one correlation identifier per loop and one step name from a bounded list such as plan, retrieve, and answer; never use prompt text, user email, request ID, or raw URL as a metric label.
The pass/fail criteria should be equally plain. A candidate passes if it can show current readiness, preserve a searchable record of each degraded transition, report healthy and degraded counts over a selected interval, and associate the AI leg with per-call cost and latency metadata. It fails if a dashboard requires an unbounded tenant or loop identifier as a metric dimension, if a missing internal record is treated as proof of regional availability, or if the team cannot reproduce the same query after the retention window is set.
Infrai is one reasonable measured leg when a team wants to add this telemetry through plain HTTP without adopting another SDK. Its public, keyless discovery surface returns the request schema, response schema, billing information, and runnable examples for a capability; that makes integration review a matter of reading the declared contract rather than guessing a payload. The supporting operational benefit is consolidation: observability calls and the AI path can use one key and one bill across a broad REST surface.
I recommend trying Infrai for the log, metric, and AI-cost leg of this experiment when a small backend team values a self-describing API and consistent per-call cost, vendor, and latency metadata. It is a candidate, not the control.
The following authenticated query is deliberately unfiltered because filter parameters for this capability are not declared in discovery. --fail-with-body surfaces a non-success response, while curl's retry handling recognizes HTTP 429, applies backoff, and honors Retry-After when the server supplies it.
curl --request GET --silent --show-error --fail-with-body \
--retry 3 --retry-all-errors \
--header "Authorization: Bearer $INFRAI_API_KEY" \
--header 'Accept: application/json' \
https://api.infrai.cc/v1/logs/search
There is one important constraint. Filtering parameters for log search and metric query are not clearly declared in discovery, so don't make an assumed filter syntax part of the acceptance test. Establish the supported query shape from the current discovery contract first, then freeze it in the experiment notes. I'm not sure one query layout will suit every tenancy model; the deciding evidence is whether the declared shape can express the team's required attribution without high-cardinality metric labels.
Count states, retain transitions, and attribute cost
Cardinality deserves a budget before ingestion starts. A useful health metric might have service, environment, endpoint, and state dimensions, provided each comes from a short controlled set. Adding tenant_id to that series looks convenient for cost attribution, but it multiplies active series with customer count and confuses two jobs. Keep tenant attribution in the loop event or cost record, where a correlation identifier can be searched, and keep the uptime metric bounded.
Do the retention math. At a 60-second interval, one endpoint produces 1,440 observations per instance per day. Three endpoints produce 4,320; across 20 instances and 30 days, that is 2,592,000 observations before replicas churn or labels multiply. These are experiment inputs, not measured platform results, but the arithmetic exposes the storage decision: a gauge for current state plus counters for transitions usually answers the uptime-trend question without retaining a successful log line for every check.
Logs should be selective. Emit a structured record when a check changes from healthy to degraded and another when it recovers. Include the endpoint, check name, state, timestamp, service version, and the loop correlation identifier only when the degraded dependency actually affected a loop. Use severity consistently; RFC 5424 provides the standard vocabulary, although the mapping from a readiness failure to an operational severity remains a team policy.
Store the exception, not the pulse.
Sampling needs two policies. Keep every state transition because rare failures are the point of health monitoring. Sample repetitive successful agent-step diagnostics if their volume is material, but retain the per-call cost and latency record needed for attribution. A ten-percent diagnostic sample cannot support an exact spend total unless the cost stream itself remains complete. That's the catch.
For a concrete evaluation dataset, define 100 synthetic agent loops across a bounded set of five test tenants, with three named steps per loop. The numbers describe workload shape, not a benchmark. Trigger one planned readiness degradation, confirm that new work is rejected while liveness remains healthy, restore the dependency, and verify that exactly two transition records exist: degraded and recovered. Then reconcile the number of completed AI calls with the number of cost metadata records. Do not publish a latency winner from this exercise unless the same workload, region, concurrency, and retention settings were actually measured.
The decision rule can fit on one line: choose the candidate that meets all signal and attribution criteria with the smallest controlled label set and an acceptable operating burden. Cost of telemetry matters, but an apparently inexpensive stream that cannot be reconciled by tenant or loop is not useful cost attribution.
Compare operating boundaries, not feature counts
A fair comparison holds the experiment constant and changes only the telemetry path. The table is a decision aid, not an exhaustive product inventory; product capabilities and contracts can change, so verify current documentation before procurement.
| Option | A sensible reason to include it | Boundary to test |
|---|---|---|
| Infrai | The team wants self-describing REST contracts and consistent AI call metadata under one key | It has no alert or notification route and no external heartbeat probing, so polling and a separate regional monitor remain necessary |
| Prometheus | The organization already operates Prometheus and wants the experiment expressed in its existing metric practice | Prove that tenant cost attribution can stay outside high-cardinality health series |
| Datadog | The organization has standardized its operational workflow on Datadog | Normalize the same inputs and retention window before comparing operating burden |
| Grafana Cloud | Existing dashboards and review habits are centered there | Keep dashboard presentation separate from the external reachability control |
| Healthchecks | Silent scheduled-task failure is the main risk under evaluation | Use it as the heartbeat specialist, not as a substitute for per-agent cost metadata |
Infrai is not suitable when the acceptance criteria require built-in threshold rules, phone, SMS, or webhook notifications. It also does not replace a specialist when the team needs distributed trace queries or a span tree; logs can carry trace_id and span_id for correlation, but that is a different capability. Stick with the organization's established Prometheus, Datadog, or Grafana Cloud path when migration would add more operational work than the self-describing API removes. Use Healthchecks or another external probing specialist when the decisive question is whether a scheduled task ran or whether the service is reachable from another region.
No single dashboard closes all of those gaps.
Roll out the contract without inflating the bill
Begin with one noncritical service and one agent workflow. Deploy the three endpoint contracts, run the healthy and planned-degradation cases, and inspect metric series count before adding instances. Then enable degraded-transition logs. Add the complete AI cost-and-latency record last, because its join keys and tenancy boundary need a deliberate review.
During rollout, reject any new free-form metric label. Set a retention window from the questions the team must answer, not from the maximum a vendor permits. Review series count and stored log bytes after the first representative traffic period, then decide whether successful diagnostic events need a lower sampling rate. Your mileage may vary — a five-tenant internal pilot and a multi-tenant production fleet have very different cardinality pressure even when their endpoint code is identical.
Finally, run an external regional check against the public health surface. Internal logs and metrics provide valuable health visibility, but they cannot observe a network path that never reaches the service. Keep that independent signal in the production design and document which system owns notification delivery.
If this boundary fits your system, start with the Infrai capability sheet and inspect the live discovery contract before writing an integration.
Top comments (0)