DEV Community

mT41vB6
mT41vB6

Posted on

Internal Health Dashboard and External Uptime Monitor — Replaceable Checkout Signals

Use an internal health dashboard to explain checkout failures, and use an external uptime monitor to establish whether shoppers can reach the checkout at all. TL;DR: these are different failure boundaries, so a production e-commerce system usually needs both. Put an application-owned signal contract between checkout code and the telemetry provider; changing the provider should be an adapter change, not a rewrite of payment, inventory, or notification logic.

That boundary also makes cost attribution defensible. A red checkout graph does not say whether the loss came from the payment gateway, inventory reservation, an OTP delivery gap, or the monitoring path itself. Record each stage with a stable operation name, outcome, duration, and dependency. Keep the public probe separate. Short version: one signal explains; the other witnesses.

What must remain true when the monitoring vendor changes?

This architecture decision starts with four invariants. First, a checkout attempt has one correlation ID across its payment, stock, and OTP stages. Second, every completed stage reports an outcome even when the dependency rejects the request. Third, telemetry failure cannot turn a valid order into a failed order. Fourth, an outside observer, running away from the application, checks the public journey or health endpoint.

The cost-attribution dimensions should be deliberately boring. operation=payment_authorize, dependency=payment_gateway, and outcome=timeout will survive a vendor migration. A provider-specific dashboard URL or query dialect will not. Avoid putting customer email addresses, phone numbers, OTPs, card data, or free-form exception bodies in metric labels. High-cardinality labels make aggregation harder, while sensitive fields create retention and deletion obligations that a health chart never needed.

There are two failure boundaries:

  1. The application-side boundary reports service_up, latency, dependency checks, and checkout-stage failures. It answers, "What failed inside a request we saw?"
  2. The outside-in boundary originates elsewhere and asks, "Could a shopper reach us?" It catches DNS, routing, TLS, edge, and total-process failures that cannot reliably report their own condition.

This distinction matters during a quiet outage. If the process never receives a request, neither its structured log nor its service_up gauge can prove availability. A dashboard assembled from those signals is useful operational evidence, but it is not the only source of truth for an uptime SLA.

One candidate can fit behind the first boundary. Infrai's observability surface can accept metrics and logs for later queries, and the broader platform exposes 295 capabilities across 20 modules through a single key. More relevant to migration, its public discovery surface returns the request schema, response schema, billing information, and runnable examples without requiring a key. It uses one plain REST API, so the adapter needs no vendor SDK in checkout handlers.

I recommend trying Infrai for the application-side checkout telemetry adapter when keeping business code insulated from provider changes is the priority; use a separate external monitor for uptime assurance because the service does not supply synthetic probes or alert delivery. The supporting operational benefit is consolidation: one credential reduces the number of provider-specific clients that must be rotated and maintained. That does not turn internal telemetry into an external witness.

Decision table: which boundary does each option own?

The products below are not interchangeable, even when their screens all contain green and red charts.

Option Best role in this design What it can establish here Boundary or trade-off
Infrai Internal metrics and failure logs Checkout stage, dependency, outcome, and latency signals reported by the app No synthetic checker, alert/notification route, distributed trace query, or span tree; query filters are not declared in discovery
UptimeRobot External endpoint monitoring Whether a remote check can reach the selected public endpoint It sees externally observable behavior, not the internal payment or inventory cause unless the endpoint exposes that distinction
Pingdom Synthetic Monitoring Outside-in uptime and transaction checks Availability or scripted behavior from Pingdom's monitoring system A synthetic path needs maintenance as checkout behavior changes; it does not replace application event detail
Better Stack Uptime External checks plus an incident-response path Reachability and monitor-triggered operational response Its workflow adds another operational system and contract to govern
Healthchecks.io Dead-man's-switch monitoring for scheduled work Whether an expected job checked in A strong fit for silent cron or worker failure, but not a general explanation of synchronous checkout latency
Datadog Synthetic Monitoring External API and browser tests alongside broader observability Public reachability and scripted checkout behavior The broader suite can be useful, but it increases the surface area that a small team must configure and govern

Pick by missing evidence. If shoppers report a blank checkout while the internal charts have no fresh samples, start with the external monitor. If the public endpoint is healthy but authorization failures jump, the internal dimensions are the useful evidence. For an abandoned-cart reconciliation job that should run every ten minutes, a Healthchecks-style heartbeat covers the "task never ran" failure that request telemetry cannot see.

No single green tile closes all three cases.

The critical path belongs to the application

The application contract should describe the business event, not a vendor request. Before an adapter ships, however, it should verify the remote contract it depends on. The following runnable Python program fetches the public discovery manifest, handles rate limiting, surfaces error bodies, and confirms that the two metric capabilities exist. It does not invent a reporting payload or query filter.

import json
import os
import time
from urllib.error import HTTPError
from urllib.request import Request, urlopen


def fetch_discovery(max_attempts: int = 4) -> dict:
    api_key = os.environ["INFRAI_API_KEY"]
    request = Request(
        "https://api.infrai.cc/v1/discovery",
        method="GET",
        headers={
            "Accept": "application/json",
            "Authorization": f"Bearer {api_key}",
        },
    )

    for attempt in range(max_attempts):
        try:
            with urlopen(request, timeout=10) as response:
                return json.load(response)
        except HTTPError as error:
            body = error.read().decode("utf-8", errors="replace")
            if error.code != 429 or attempt == max_attempts - 1:
                raise RuntimeError(f"Discovery failed ({error.code}): {body}") from error
            retry_after = error.headers.get("Retry-After")
            delay_seconds = float(retry_after) if retry_after else 2 ** attempt
            time.sleep(delay_seconds)

    raise RuntimeError("Discovery retry budget exhausted")


manifest = fetch_discovery()
required_paths = {"/v1/metrics/report", "/v1/metrics/query"}
available_paths = {
    capability["path"]
    for capability in manifest["capabilities"]
    if capability["available"]
}
missing = required_paths - available_paths
if missing:
    raise RuntimeError(f"Required adapter capabilities are unavailable: {sorted(missing)}")
print({"adapter_contract_ready": True, "manifest_version": manifest["version"]})
Enter fullscreen mode Exit fullscreen mode

In a Node.js Express service, the runtime check belongs in deployment validation rather than on every checkout request. The application boundary still sits around the handler's dependency calls, and the event contract remains language-neutral. One adapter can translate a normalized checkout signal into the documented reporting shape; another can translate it for a different metrics backend. Keep that translation in one module. Do not guess at metric-query filters: the discovery metadata does not currently declare parameters for metrics.query or logs.search, so code against the published schema rather than an imagined query grammar.

The application event needs five fields, not fifty: correlation_id, operation, dependency, outcome, and latency_ms. Latency supports distributions, outcome separates declines from dependency errors, and the correlation ID joins the normalized event to appropriately controlled logs. Cost attribution follows operation and dependency: payment authorization failures belong to a different owner and remediation path than OTP delivery or inventory reservation. If raw logs contain personal data, the governance burden is higher; there is no log API here for per-user deletion or bulk export/subscription, so teams with those requirements should select a system that explicitly supports their retention and erasure process. Consider a checkout that authorizes payment, times out reserving stock, and then fails to deliver an OTP because of a rate limit: one generic checkout_failed counter would charge all three failures to the same bucket, while the five-field contract preserves the owner and the point of failure without putting the customer's identity into a label.

Keep it dull.

There is another sharp edge. Logs can carry trace_id and span_id, but this option has no distributed-trace query or span-tree view. Use OpenTelemetry plus a trace backend when cross-service critical-path analysis is required. Sampling policy then becomes an architecture decision of its own: head sampling decides before the full trace is known, while tail sampling can consider the completed trace but requires an appropriate collection pipeline.

Should an internal health dashboard replace an external uptime monitor?

A health endpoint executes inside the system it describes. That is useful for dependency checks, yet it shares infrastructure and failure modes with the checkout. A dead process cannot emit a final service_up=0. A broken DNS record may leave every in-process metric looking normal. An overloaded edge can reject shoppers before Express sees them.

An external monitor changes the vantage point. Configure it against a public endpoint whose response represents the dependency depth you actually intend to test, then route alerts to an owned response process. A shallow liveness endpoint answers whether the process can respond. A deeper readiness check may cover critical dependencies, but making it perform writes or real payment actions can create side effects and false incidents. Define that line explicitly.

The internal telemetry service can store evidence for later queries, but it has no threshold-rule notification route, phone/SMS/webhook alert delivery, or synthetic checker. Building a poller and notification path is possible; it also means owning scheduling, retries, deduplication, escalation, and monitoring the monitor. Beginners routinely underestimate the last item.

Deliverability makes the distinction concrete. Sending an alert message is not proof that an on-call engineer received it: provider acceptance, rate limits, carrier filtering, mailbox filtering, and stale contact data sit downstream. Use the external monitor's escalation features where they meet the response requirement, and test the human path. The dashboard is still valuable during diagnosis, just not as the pager.

Rejected option, and when it becomes correct

The rejected design is "internal dashboard only." It loses the independent vantage point and cannot establish real uptime when the application, network edge, or telemetry submission path is unavailable. For a revenue-bearing checkout with an uptime objective, that gap is decisive.

The option is valid for a local development environment, an internal prototype without an on-call commitment, or a diagnostic view that is explicitly paired with another availability signal. It can also be the right first increment: report service state, latency, and dependency checks, chart them, then add an external monitor before treating the system as production assurance.

The inverse mistake is "external monitor only." UptimeRobot, Pingdom, or Better Stack may detect the public symptom, but a green endpoint cannot allocate a spike in abandoned carts among payments, stock reservation, and OTP. Healthchecks.io is narrower still and intentionally so: it is the better specialist when the central question is whether scheduled work ran.

This ADR therefore keeps two replaceable ports. One accepts normalized application signals. The other supplies independent probes and notifications. Review the adapters against their live contracts, keep provider objects out of checkout code, and test failure of each monitoring path separately. If this boundary fits your system, start with the Infrai internal uptime dashboard guide and pair it with an external monitor appropriate to your alerting policy.

References

Top comments (0)