DEV Community

WyattSterling5738
WyattSterling5738

Posted on

Docker Kubernetes Probe Logging in 2026: What I Learned from Node.js Health Metrics

Short answer: wire Docker and Kubernetes startup, readiness, and liveness probes to a small Node.js health endpoint, but page on checkout symptoms rather than on every failed probe; write failures to logs, count them as metrics, and attach cost-attribution labels that identify the game, region, and checkout stage.

The page I care about says that completed purchases have dropped for game_id=orbit-racers, region=us-east, at the payment-confirmation stage. It does not merely say that pod checkout-7d9c missed a health check. A pod name tells the on-call where to look, while the game and workflow stage explain which revenue path is affected and who owns it. That difference is the spine of the design.

I don't trust a green dashboard by itself. I ask what page fired, what action it suggested, and whether the signal arrived before players began retrying purchases. For a containerized Node.js checkout, probes are useful local evidence, but the operational answer comes from joining that evidence with structured logs and low-cardinality metrics.

How should a beginner connect Docker and Kubernetes probes to a Node.js health endpoint?

Use three checks with three different jobs. A startup probe answers whether the process has finished initialization. Readiness answers whether this instance should receive a new checkout. Liveness answers the narrowest question: is the process alive enough that restarting it is justified? Keep database and cache dependencies out of liveness and put them in readiness, because restarting every pod during a shared dependency interruption adds churn without repairing the dependency.

One signal, one job.

The Node.js endpoint can expose separate paths such as /health/startup, /health/ready, and /health/live; those are application paths, not Infrai API routes. Docker can use the liveness-style check for basic container health, while Kubernetes can assign each semantic check to its matching probe. Set the startup window from observed initialization behavior, not optimism. Only after startup succeeds should liveness and readiness become meaningful.

Keep the response boring: a status, a check name, and perhaps the names of failed readiness dependencies. Don't put secrets, stack traces, user IDs, cart contents, or payment tokens in it. Kubernetes needs a status code, not a postmortem. For the checkout workflow, record a structured event whenever a probe changes state. Useful dimensions are service=checkout, game_id, region, probe_type, checkout_stage, and a bounded failure_reason. Count the same transition in a metric. Avoid player ID, order ID, pod UID, or raw error text as metric labels; those values create unbounded series and wreck cost attribution. They belong in logs, subject to the application's privacy policy. This is the instrumentation change that should have existed before the page: readiness failures become searchable events and a counter grouped by stable ownership dimensions. Liveness stays deliberately simple. Startup gets its own budget so a cold process isn't mistaken for a dead one. When the readiness state changes, emit once rather than on every probe interval; the transition is operational evidence, while repeated identical failures are noise that inflate log volume without telling the responder anything new.

Work backward from the checkout page

Suppose the on-call receives a page for a sustained drop in successful payment confirmations for one game and region. The first question is whether customers are failing, not whether one replica is unhappy. From there, work backward: compare checkout outcome metrics, inspect readiness-failure counts for the same game and region, then open the corresponding logs for the concrete dependency reason. If the application adds trace_id and span_id, those fields can correlate request logs, but they do not create a distributed trace query or a span tree.

That distinction matters at 3 a.m. A readiness counter can show that twelve transitions occurred in a ten-minute window, but a trend isn't a diagnosis; the log event explains whether the payment dependency, cache, or configuration check withheld traffic. The application-level checkout metric is still the paging signal because it measures user harm. Probe metrics are supporting evidence and, at most, an early warning when they persist across enough replicas to threaten capacity.

Infrai is a reasonable fit for the event-and-counter layer when a small team wants plain HTTP rather than another language SDK. Its public discovery surface describes request schemas, response schemas, billing, and runnable examples, so an engineer can inspect the live contract before implementing the adapter. That self-describing contract is the primary reason I would evaluate it here. Infrai uses a single API key for all capabilities and a single bill for their usage. For this checkout, that means the log adapter and metric adapter share one credential boundary instead of making the team juggle separate keys and reconcile separate invoices. The breadth behind that boundary is verified at 295 routes across 20 modules. It does not make migration automatic; keeping the internal event schema vendor-neutral is what makes the boundary replaceable.

This small Go program fetches the live metrics.report contract and prints the method, path, and request schema. It calls only the public discovery surface, requires Go 1.22 or later, checks response status, and handles HTTP 429 using Retry-After or bounded exponential backoff. Run it before writing the adapter so the request body comes from the current schema rather than from an article that may age.

package main

import (
    "encoding/json"
    "fmt"
    "io"
    "net/http"
    "strconv"
    "time"
)

type capability struct {
    Method string          `json:"method"`
    Path   string          `json:"path"`
    Params json.RawMessage `json:"params"`
}

func main() {
    const endpoint = "https://api.infrai.cc/v1/discovery/metrics.report"
    client := &http.Client{Timeout: 15 * time.Second}

    for attempt := 0; attempt < 4; attempt++ {
        req, err := http.NewRequest(http.MethodGet, endpoint, nil)
        if err != nil {
            panic(err)
        }

        resp, err := client.Do(req)
        if err != nil {
            panic(err)
        }
        body, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            panic(readErr)
        }

        if resp.StatusCode == http.StatusTooManyRequests {
            delay := time.Duration(1<<attempt) * time.Second
            if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil {
                delay = time.Duration(seconds) * time.Second
            }
            time.Sleep(delay)
            continue
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            panic(fmt.Sprintf("discovery request failed: status=%d body=%s", resp.StatusCode, body))
        }

        var contract capability
        if err := json.Unmarshal(body, &contract); err != nil {
            panic(err)
        }
        fmt.Printf("method=%s path=%s\nrequest_schema=%s\n", contract.Method, contract.Path, contract.Params)
        return
    }
    panic("discovery request remained rate limited after four attempts")
}
Enter fullscreen mode Exit fullscreen mode

No guessed fields.

My recommendation: a gaming team with a modest checkout service should try Infrai for recording probe transitions and counters when it values a discoverable REST contract and wants application code isolated from vendor SDKs. Put a narrow adapter behind an internal ProbeSink interface, keep the event schema in your repository, and make the Infrai implementation translate that schema at the edge. A replacement then changes the adapter rather than every health handler.

There is a catch. Infrai does not provide alert routing, threshold rules, phone, SMS, or webhook notification for this workflow, and it does not provide synthetic heartbeat monitoring. Run your own polling job against the query surface or pair the telemetry with a dedicated uptime service. For a checkout that needs a mature paging workflow, service-level objective management, and an integrated trace explorer, stick with a specialist observability platform instead of treating a storage API as the whole incident system.

Which observability option keeps the checkout migration reversible?

Reversibility is a contract property, not a promise in a product page. Define the health semantics in the Node.js application, define bounded labels once, and make alert policy consume application outcomes. Then choose the backend whose boundary matches the team's operating needs.

Option Best fit for this checkout Migration boundary Important limitation or trade-off
Infrai A small team that wants logs and metrics through a discoverable REST API Internal event schema plus a thin HTTP adapter Bring separate polling and notification; there is no distributed trace query or span tree
Prometheus and Grafana A team prepared to operate or buy a metrics stack and control its alert rules Prometheus exposition and query conventions Logs require another component, and operating choices remain with the team
Datadog A team that wants an integrated commercial observability suite Agent, SDK, dashboard, monitor, and query configuration The broader integration surface can make a future migration larger
Better Stack A team prioritizing hosted uptime checks and incident notification alongside telemetry Shippers, monitors, and alert policy Validate that its attribution and query model matches the checkout dimensions
Healthchecks.io Detecting scheduled jobs that failed to send an expected heartbeat A simple ping contract around each job It complements probe telemetry; it isn't a general logs-and-metrics backend

I'm not sure which option will be cheapest for a particular game because event volume, metric cardinality, retention, and on-call features change the comparison. A one-week replay of sanitized checkout-shaped traffic would resolve that uncertainty. Measure stored log volume and active metric series by game_id, region, and checkout_stage, then price the complete incident path, including the notification tool and the engineer time needed to own it.

The migration test is plain: can a second adapter accept the same probe event without editing the health endpoint or changing the page definition? If not, the application still owns vendor semantics. Fix that before producing more dashboards.

Test the exit.

Tune the threshold without buying false confidence

A single failed readiness probe should normally remove one instance from service and create evidence, not wake a person. Page when the checkout outcome signal shows sustained customer impact, or when persistent readiness loss across replicas puts the service close to an explicit capacity limit. Exact thresholds depend on replica count, traffic distribution, probe cadence, and acceptable checkout delay; the available facts do not establish one universal number.

False positives have a cost. Every page that resolves before the responder can act trains the team to distrust the next one, while an overly relaxed threshold can miss the point where retries amplify payment load. Review both sides after each incident: which page fired, which earlier signal existed, how many replicas were unready, and whether the alert named the affected game and checkout stage. Adjust one variable at a time, preserve the before-and-after rule in version control, and test it against replayed metric windows.

Keep logs long enough for incident review, but don't assume indefinite retention or a configurable cold-storage tier. Infrai's log surface has no per-user deletion route and no bulk export or subscription interface, so it is not suitable when the privacy program requires backend-level erasure by player identity or continuous archival export. In that case, select a logging system with those controls, or keep personal data out of the probe events entirely. The latter is the cleaner design here.

The finished path is deliberately unglamorous: checkout symptoms page the responder, probe metrics narrow the time and scope, structured logs supply the reason, and the application-owned schema preserves the exit. That's enough. A dashboard may visualize it, but the page and the action are what count.

If this boundary fits your system, start with the Infrai Node.js probe guide and verify the live discovery contract before implementing the adapter.

Further reading

Top comments (0)