Short answer: classify each failed Node.js fetch check by transport phase, preserve the low-level cause, and group on stable service and failure-family fields rather than raw error text. For a B2B SaaS checkout, page on sustained customer-impact signals; keep isolated timeout, connection-refused, and DNS events searchable for diagnosis.
| Approach | Pick it when | Signal-quality catch |
|---|---|---|
| Log every failed check | Engineers need a searchable evidence trail | Raw messages and stack traces fragment groups |
| Emit bounded failure metrics | Operators need rates, ratios, and alerts | Labels cannot carry request IDs or arbitrary error text |
| Create one incident per failure | Every single failure demands action | Transient network noise creates alert churn |
| Group events, then alert on policy | Checkout impact matters more than probe perfection | The policy needs explicit windows and ownership |
The last option is the useful default here. Logs explain which phase failed. Bounded metrics show how often. The alert policy decides whether someone must act now. Those are three different jobs.
What should Node.js health endpoint checks record for timeout, DNS, and connection failures?
Record the target as a controlled service identifier, not a user-supplied URL. Add the check name, region, transport phase, normalized failure family, elapsed time, attempt number, and a correlation ID. Preserve the original Error.cause chain in the event payload because Node.js fetch may surface a top-level TypeError while the cause carries the operational distinction.
For a checkout workflow, a diagram in words looks like this: scheduler -> checkout probe -> DNS lookup -> TCP connection -> HTTP exchange -> response validation -> event normalizer -> log and metric sinks -> alert evaluator. A failure should be assigned to the farthest phase it reached. No HTTP response means there is no status code to invent. A 503 response is an HTTP availability result, while a refused connection is a transport result; grouping them together destroys the clue an operator needs.
Use a small taxonomy. timeout means the caller's deadline expired. dns means name resolution failed. connection_refused means the remote address actively rejected the connection. connection_reset is separate because a connection existed before it was reset. http_status covers a completed exchange with an unacceptable response code. invalid_body covers a response that arrived but failed the checkout probe's contract.
Keep it boring.
Do not use error messages, stack traces, full URLs, correlation IDs, tenant IDs, or timestamps as metric labels. Prometheus warns that every unique label set creates a new time series, so unbounded values turn diagnosis metadata into a cardinality problem. Put rich context in structured logs. A bounded metric can use labels such as check=checkout, region=us-east, and failure_family=dns; the event retains the exact cause and correlation ID.
Pick logs when the question is “what happened?”
Structured logs are the primary search surface for a failed probe. They should let an engineer move from a noisy checkout symptom to one event family, then inspect a representative cause without parsing prose. Search on normalized fields: check_name=checkout, outcome=failure, failure_family=timeout, and a time range. Search the correlation ID only after the broad query finds the relevant event.
Start broad.
A practical fingerprint is check_name + target_id + region + failure_family + phase. Leave message, elapsed_ms, and correlation_id out. This groups repeated symptoms while preserving enough topology to avoid combining a DNS problem in one region with response validation failures everywhere. If deployments are a useful diagnostic boundary, keep the release identifier as a filter first; adding it to the fingerprint can split one continuing incident on every deploy.
Consider a hypothetical ten-minute slice in which three checkout probes time out in one region, two probes report ENOTFOUND there, and one probe in another region receives an unacceptable HTTP status. Grouping on the full message might create six groups because elapsed values and target strings differ. Grouping only on check_name would create one group and erase the network boundary. The proposed fingerprint produces three hypotheses: a regional timeout group, a regional DNS group, and a separate HTTP-status group. An engineer can search the first group, compare its timestamps with the DNS group, and ask whether both point to the same regional dependency; the system does not claim that relationship before there is evidence. Meanwhile, the status group remains distinct because an HTTP response proves that DNS and connection setup completed for that attempt. This is the crisp before and after: six message-shaped fragments become three cause-shaped leads, without collapsing every failed checkout check into a single red counter.
I'm not sure there is one universally correct grouping window. Your mileage may vary with probe frequency and checkout traffic. Resolve that uncertainty with replay: take a week of retained events, apply candidate fingerprints and windows, then compare the resulting groups with the incidents engineers actually investigated. The useful question isn't “did grouping reduce the count?” It is “did one group still mean one operational hypothesis?”
Pick metrics when the question is “how bad is it?”
Use counters for attempts and failures, then calculate a failure ratio over an explicit window. A histogram can describe successful and failed check duration if its buckets match decisions your team makes. Metrics should aggregate; they should not impersonate an event database.
For checkout, alert policy needs a customer-impact guardrail. A single external probe timeout can remain an event. A sustained failure ratio across several checks, especially when paired with a checkout request symptom from the application, is a stronger paging signal. Exact thresholds cannot be copied safely from another system because probe interval, traffic, redundancy, and response objectives differ. Test candidate rules against historical data and scheduled maintenance.
Alert on symptoms at the service boundary, then use cause-oriented signals for routing and diagnosis. This keeps a DNS failure family useful without making every resolver wobble a separate page. Fast feedback matters, but noisy feedback trains people to ignore it.
Implement one typed normalization boundary
The probe code should create one deadline, validate the response separately, and convert known cause codes into a closed vocabulary. It should not retry inside the normalization function. Retries are a policy decision, and stacking retries across layers can multiply load; the AWS Builders' Library recommends timeouts, limited retries, backoff, and jitter while warning about retry amplification.
Here is a compact TypeScript shape. The URL stays in configuration, while emitted telemetry uses the controlled targetId.
import { randomUUID } from "node:crypto";
type FailureFamily =
| "timeout"
| "dns"
| "connection_refused"
| "connection_reset"
| "http_status"
| "invalid_body"
| "unknown_transport";
type ProbeResult =
| { outcome: "ok"; elapsedMs: number; status: number }
| {
outcome: "failure";
elapsedMs: number;
family: FailureFamily;
phase: "transport" | "http" | "validation";
status?: number;
causeCode?: string;
correlationId: string;
};
function readCauseCode(error: unknown): string | undefined {
if (!(error instanceof Error)) return undefined;
const cause = error.cause;
if (typeof cause !== "object" || cause === null || !("code" in cause)) {
return undefined;
}
return typeof cause.code === "string" ? cause.code : undefined;
}
function classify(error: unknown): { family: FailureFamily; causeCode?: string } {
if (error instanceof Error && error.name === "TimeoutError") {
return { family: "timeout" };
}
const causeCode = readCauseCode(error);
const families: Partial<Record<string, FailureFamily>> = {
ENOTFOUND: "dns",
EAI_AGAIN: "dns",
ECONNREFUSED: "connection_refused",
ECONNRESET: "connection_reset"
};
return { family: families[causeCode ?? ""] ?? "unknown_transport", causeCode };
}
export async function checkCheckout(
url: URL,
timeoutMs: number
): Promise<ProbeResult> {
const started = performance.now();
const correlationId = randomUUID();
try {
const response = await fetch(url, { signal: AbortSignal.timeout(timeoutMs) });
const elapsedMs = Math.round(performance.now() - started);
if (!response.ok) {
return {
outcome: "failure",
elapsedMs,
family: "http_status",
phase: "http",
status: response.status,
correlationId
};
}
const body: unknown = await response.json();
if (typeof body !== "object" || body === null || !("status" in body) || body.status !== "ok") {
return {
outcome: "failure",
elapsedMs,
family: "invalid_body",
phase: "validation",
status: response.status,
correlationId
};
}
return { outcome: "ok", elapsedMs, status: response.status };
} catch (error: unknown) {
return {
outcome: "failure",
elapsedMs: Math.round(performance.now() - started),
...classify(error),
phase: "transport",
correlationId
};
}
}
The emitted event can now be searched without guessing which spelling a runtime chose for a message. Keep family stable. Keep causeCode for detail. Index the controlled fields used in routine triage, and retain the full cause chain according to the organization's log policy.
Test the boundary with deterministic cases: a server that accepts and delays, an unused local port for refusal, a controlled nonexistent test hostname for DNS, an unacceptable status, and a valid status with an invalid body. Assert the normalized result, not the runtime's entire message string. Also test that the configured deadline is positive and that secrets or query strings never enter the event.
Retries belong one level above this function. Give the overall operation a budget, cap attempts, and add jitter before another attempt. If the first attempt times out after the entire user-visible budget, a retry cannot help. If every monitor retries in lockstep, the monitoring system can add load during the exact moment checkout is struggling.
Limits and operational choices
This design is not suitable when an endpoint check must prove a full purchase from browser to payment settlement. A synthetic transaction with isolated test data is the better instrument there, and it needs stronger cleanup, privacy, and idempotency controls. Stick with a shallow readiness probe when the only question is whether one process can receive traffic.
The catch is that normalization deliberately loses message-level uniqueness. Keep the original cause in logs so a new runtime code can be examined, then add a reviewed mapping rather than letting unknown strings become labels. Also avoid aggressive retries when a check has side effects. Health endpoints should be safe to call repeatedly; a checkout mutation is not a health endpoint.
The resulting operating rule is short: events preserve evidence, metrics bound dimensions, groups represent hypotheses, and pages require sustained impact. That division gives checkout failures somewhere useful to go without turning every network hiccup into an incident.
Top comments (0)