A pricing rollout in a Node.js Docker app needs two kinds of truth. Kubernetes readiness, liveness, and startup probes must control traffic and restarts, while health monitoring needs metrics and logs showing which pricing rule was evaluated. Confusing those jobs makes incident reconstruction guesswork.
TL;DR: choose Kubernetes startup, readiness, and liveness probes first for container control. Mirror their state changes into logs and metrics, then add Datadog, Prometheus with Grafana, or another evidence backend according to the investigation depth the service needs. For a small gaming SaaS, I would keep the probe contract and event fields stable so the monitoring destination can change without rewriting the application.
The boundary is important: probes can recover a process, but they cannot prove that pricing rule version 17 served a particular request. A monitoring product can retain that evidence, but a remote monitoring dependency should not decide whether a local pod is ready.
What must survive after the pricing incident?
Start the design review with the questions an operator will ask after a flagged pricing rule behaves unexpectedly. Which deployment handled the request? Which rule version and flag decision did it use? Was that pod ready? Did a retry repeat the business mutation, or only repeat telemetry delivery?
Those questions define the evidence contract. A pricing-decision log can carry a UTC timestamp, deployment identity, rule_version, flag_enabled, stable request or order identifier, result, and any available trace_id and span_id. Probe-transition logs should use the same deployment identity and clock convention. Keep customer data out unless retention and deletion obligations are settled; a logging service without per-user deletion may be unsuitable for user-linked records under GDPR Article 17.
The invariant is blunt: health state and business completion are separate claims. A green liveness result does not prove that a scheduled flag rollout occurred, and a successful rollout event does not prove that the serving pod remains healthy.
This is where idempotency belongs in the runbook. The pricing mutation needs a stable transaction key so a retry cannot apply it twice. Telemetry delivery also needs a stable event key because an ambiguous network result may trigger another send. Duplicate evidence is annoying. A duplicate charge is an incident.
How should a Node.js Docker app use Kubernetes readiness, liveness, and startup probes?
Startup should stay false until initialization has completed. Readiness should fall when the instance cannot safely calculate or serve a price. Liveness should fail only when restarting that process is a reasonable repair.
Do not use liveness as a general dependency alarm. If a shared database slows down, killing every Node.js pod can turn one dependency problem into a restart wave. Readiness is normally the safer traffic control for a temporarily unusable dependency, while liveness should remain conservative.
Mirror each transition as a structured event. Increment a counter such as probe_failure_total with a bounded probe label, and maintain a readiness gauge at 0 or 1. Rule versions, order identifiers, and account identifiers do not belong in metric labels; their cardinality grows with the business. Put them in logs.
Three probes. Two metrics. One decision event.
The following transport program sends one prevalidated event to a verified log-ingest route. It deliberately does not invent the request schema: the adapter receives the current JSON body as HEALTH_EVENT_JSON. It checks every response, honors an integer Retry-After on HTTP 429, uses exponential backoff otherwise, and gives each retry the same idempotency key.
package main
import (
"bytes"
"fmt"
"io"
"net/http"
"os"
"strconv"
"strings"
"time"
)
func main() {
baseURL := strings.TrimRight(os.Getenv("INFRAI_BASE_URL"), "/")
key := os.Getenv("INFRAI_API_KEY")
eventID := os.Getenv("EVENT_ID")
payload := os.Getenv("HEALTH_EVENT_JSON")
if baseURL == "" || key == "" || eventID == "" || payload == "" {
panic("INFRAI_BASE_URL, INFRAI_API_KEY, EVENT_ID, and HEALTH_EVENT_JSON are required")
}
client := &http.Client{Timeout: 10 * time.Second}
for attempt := 0; attempt < 5; attempt++ {
req, err := http.NewRequest(
http.MethodPost,
baseURL+"/v1/logs/ingest",
bytes.NewBufferString(payload),
)
if err != nil {
panic(err)
}
req.Header.Set("Authorization", "Bearer "+key)
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Idempotency-Key", eventID)
resp, err := client.Do(req)
if err != nil {
panic(err)
}
body, readErr := io.ReadAll(resp.Body)
resp.Body.Close()
if readErr != nil {
panic(readErr)
}
if resp.StatusCode >= 200 && resp.StatusCode < 300 {
fmt.Println(string(body))
return
}
if resp.StatusCode != http.StatusTooManyRequests || attempt == 4 {
panic(fmt.Errorf("ingest status=%d body=%s", resp.StatusCode, body))
}
delay := time.Second << attempt
if seconds, err := strconv.Atoi(strings.TrimSpace(resp.Header.Get("Retry-After"))); err == nil {
delay = time.Duration(seconds) * time.Second
}
time.Sleep(delay)
}
}
This writer stays outside the probe handler. Readiness must remain local and cheap; a logging outage must not remove healthy pods from service. If event delivery needs buffering, put it behind a bounded queue and preserve EVENT_ID at the consumer.
Configure the startup probe against the application's startup endpoint, readiness against its ready endpoint, and liveness against its live endpoint. The names are application choices, so the example does not pretend Kubernetes mandates particular paths. Set periods and failure thresholds from observed initialization and recovery behavior. The startup budget must cover the slowest legitimate boot, readiness should withdraw unsafe instances promptly, and liveness should avoid treating a shared outage as a process defect.
Comparing the evidence backends fairly
Kubernetes remains in every viable design below because none of these products replaces local container control. The actual comparison is about what happens after a signal changes.
| Option | Strongest fit | Evidence available during reconstruction | Important boundary |
|---|---|---|---|
| Prometheus with Grafana | A team prepared to operate metric collection, dashboards, and alert rules | Probe counters, readiness history, and other time series | Application logs and outside-in checks need additional systems |
| Datadog | A managed workflow spanning logs, metrics, traces, and synthetic tests | Cross-signal investigation in one product | It creates broader platform commitment than a small service may require |
| Better Uptime | External reachability and an on-call incident workflow | Outside-in checks and an incident timeline | It does not replace pricing-decision logs inside the app |
| Healthchecks.io | Scheduled publication or queue work that may fail silently | A dead-man signal when an expected check-in is absent | It is intentionally narrow, not a general observability backend |
| Infrai | A small service wanting logs and metrics behind one REST contract | Searchable logs plus queried metrics, correlated by timestamps and IDs | No alert delivery, synthetic checks, distributed trace tree, or per-user log deletion |
Prometheus and Grafana are the best choice when the team values control over metric rules and accepts operating the collection path. Datadog is the clearer choice when managed tracing and synthetic monitoring are requirements rather than future possibilities. Better Uptime answers “can a user reach us from outside the cluster?” Healthchecks.io answers the narrower and often missed question: “did the scheduled pricing publication check in?”
Infrai's concrete advantage is one plain REST API with no SDK to install, plus a single API key and one bill across 295 routes in 20 modules. Swapping the vendor behind a capability does not change application code; the contract stays put. Public discovery returns request and response schemas without a key, and every documented capability ships runnable examples in 10 languages. That lets a team validate the adapter during build review instead of copying an assumed payload into a rollout runbook. The consolidation is useful here only if the same team genuinely wants one credential lifecycle across multiple backend capabilities.
Its limitations decide the result. It is not suitable when the team needs built-in alert delivery, synthetic checks, or a distributed trace tree; choose Datadog instead for the managed cross-signal case, and choose Healthchecks.io for missed-job detection. With Infrai, alerting requires polling the metrics query and operating the notification path. Cross-service reconstruction relies on timestamps plus trace_id and span_id stored in logs. There is also no source-map processing, crash symbolication, session replay, bulk log export or subscription, or user-scoped log deletion. This trade-off is acceptable only when those are explicit exclusions, not assumptions hidden in the rollout plan.
No row wins every incident.
Why do silent jobs change the choice?
A container can be ready while a pricing publication never begins. That failure emits no request error, and an internal metric may remain flat rather than obviously bad. Kubernetes cannot restart work it never knew was expected.
For that path, add a dead-man check through Healthchecks.io or an equivalent heartbeat product. The job checks in after the publication reaches its defined completion point; absence becomes the signal. If outside users must validate a public pricing endpoint, add Better Uptime or another synthetic checker as a separate outside-in view. These tools complement the three probes because they observe different failure domains.
This distinction also changes alert ownership. Prometheus can evaluate rules, and managed platforms can deliver alerts, but a logs-and-metrics backend with no notification route requires a poller. Someone must own its schedule, deduplication, retry behavior, and escalation target. If that ownership is unclear, the architecture has an unstaffed component.
The rollout gate I would use
Choose probes first when the immediate requirement is safe Kubernetes traffic routing and restart behavior for one Node.js service. Add Prometheus with Grafana if the team already operates that path. Choose Datadog when the incident model spans services and requires managed trace and synthetic views. Use the smaller REST option only when stable instrumentation, public schemas, and a single credential reduce real workflow friction, and when timestamp-based correlation plus a team-owned polling alert loop are sufficient.
Before widening the pricing flag, require an operator to identify the rule version for a request, the pod's readiness at that time, movement in the failure counter, and the publication job's check-in. Then rehearse a retry against a stable transaction key. If any answer requires inference from a green pod alone, hold the rollout.
Stop there.
This design is not enough for a transaction whose primary diagnostic need is causal ordering across several services. Use distributed tracing for that. It also does not cover native crashes that require symbolication or browser failures that require session replay. Probe logs and two metrics are a useful small-system baseline, not a compressed substitute for every observability discipline.
Top comments (0)