Short answer: for a small containerized SaaS, make startup, readiness, and liveness separate application signals, let Kubernetes act only on those signals, and mirror every health transition into logs and metrics.
I use that as an architecture decision, not as a monitoring slogan. A probe answers a narrow question about one container; it does not explain an outage, page an operator, or prove that a scheduled task ran. The useful simple setup has three parts: Kubernetes performs local recovery, the application records the same state changes, and a small operational view shows a readiness gauge beside failure counters and searchable logs. It works for a Node.js service in Docker just as well as for another runtime, because the contract lives at the HTTP boundary.
My bias comes from building storage and data layers. I distrust a green dashboard unless I can say which invariant it represents, how stale it may be, and what happens during partial dependency failure. Keep it boring.
What should a Node.js Docker Kubernetes health monitoring example prove?
The decision record starts with three different questions. A startup probe asks whether initialization has completed. A readiness probe asks whether this instance should receive new traffic now. A liveness probe asks whether the process is stuck badly enough that restarting it is the least harmful recovery. Giving all three probes the same endpoint erases those distinctions and turns a temporary database slowdown into restart churn.
For a small SaaS, I would write down these invariants before choosing any vendor. Readiness is zero when a required dependency prevents the request path from meeting its contract. Liveness stays healthy during an ordinary downstream outage if the process can still make progress. Startup remains false while migrations, cache warming, or other bounded initialization is incomplete. Each state transition produces one structured log record, while counters accumulate probe failures and a gauge exposes current readiness. Kubernetes can then restart an unhealthy container, and the evidence survives long enough for an engineer to investigate.
The failure boundary matters more than the endpoint names. A readiness failure removes traffic; a liveness failure destroys process-local evidence and state. For object storage, for example, I won't declare a process dead merely because one remote bucket check timed out. That timeout may justify refusing uploads, but a restart cannot repair the remote service. The probe should also finish well inside its configured timeout. Otherwise the orchestrator measures the timeout rather than application health.
There is a fourth question that no probe answers: can a user complete the critical journey from outside the cluster? That is uptime monitoring, and it belongs beyond the pod. The app-level pattern here is deliberately smaller.
Decision invariants and failure boundaries
The smallest useful signal model is a readiness gauge with values zero or one, plus monotonically increasing counters split by probe type. I keep the labels bounded: service, environment, probe, and result are usually enough. Request IDs, user IDs, exception messages, and URLs with arbitrary path segments belong in logs, not metric labels, because unbounded cardinality makes a tiny monitoring system expensive and hard to reason about.
Logs need a timestamp, service identity, probe type, old state, new state, and reason. If the application already carries trace_id and span_id, include them. They are correlation handles, not a trace system: this setup has no distributed tracing query or span tree, so cross-service outage analysis depends on aligned timestamps and those IDs in searchable logs. Clock discipline therefore becomes an architectural dependency — an unglamorous one, but real.
I learned to treat throttling as its own failure mode after a collector returned 429 for 37 consecutive submissions while my retry loop quietly swallowed each response and printed a cheerful success line. The first clue was not an error; it was an implausibly flat failure counter during a deployment that had plainly caused readiness churn. I followed the local timestamps, compared them with the collector responses, and found that the wrapper treated any completed HTTP exchange as success, regardless of status. No data was corrupted, but the graph had a clean gap precisely when I needed it, and the logs documenting that gap were stuck behind the same careless sender. Now I surface the status, honor Retry-After, use exponential backoff with jitter, and cap retries rather than holding a worker forever. I also make the sender expose its own dropped-attempt counter. For writes, the client needs an idempotency strategy so retrying cannot double-apply an event. That episode is why I don't accept "we emit metrics" as evidence that metrics arrived; delivery is a separate invariant, and the application must make failure visible without claiming that telemetry storage succeeded.
This leaves explicit limits. Infrai has searchable logs and metric reporting/query capabilities, but no alert or notification routing, synthetic heartbeat monitoring, distributed trace view, source-map decoding, crash symbolication, or Session Replay. Its log service also has no per-user deletion route or bulk export/subscription interface. Those boundaries affect compliance and operations: if a user-erasure workflow must remove an individual's log records, choose storage with the necessary deletion controls rather than pretending retention alone answers GDPR Article 17.
Comparing a simple SaaS observability stack
I compare tools by ownership boundary rather than feature count. Infrai is interesting when a small team wants logs and metrics behind the same plain REST contract as many other backend capabilities: one key covers 295 routes across 20 modules, and public discovery exposes request schemas and runnable examples. The advantage is integration breadth without another SDK for every capability. It does not eliminate the need to design health semantics.
| Option | Best fit | Operational trade-off | When I would choose it |
|---|---|---|---|
| Infrai | A small service that values one consistent REST surface for backend capabilities | No built-in alert routing, heartbeat monitor, or distributed trace view | Probe-state logs and basic metrics, with a separate polling alert or heartbeat tool |
| Datadog | A team wanting an integrated commercial monitoring suite | A broader platform introduces more configuration and vendor-specific operating knowledge | I need managed alerting and deeper cross-service investigation |
| Grafana Cloud | A team already comfortable with the Grafana observability ecosystem | The team still owns signal design, labels, dashboards, and collection choices | Existing Prometheus or Grafana practice is the deciding constraint |
| Better Stack | A small team prioritizing an approachable uptime and incident workflow | Application metrics and internal probe semantics remain separate concerns | External checks and operator notification are the first requirement |
| Healthchecks | Scheduled jobs and dead-man-switch heartbeats | It is complementary, not a general logs-and-metrics backend | I must detect that a task failed to run at all |
The catch is straightforward: Infrai is not suitable as the only monitoring product when an on-call workflow requires threshold rules, phone, SMS, or webhook notification. Its query APIs can be polled to build an alert, but that means your team owns the evaluator and delivery path. Stick with Datadog or another dedicated observability platform when trace navigation and managed alerting are central. Add Healthchecks when silent cron failure is the risk, and consider Better Stack when outside-in uptime is the main question. Your mileage may vary because the costly part is often the team's unfamiliarity, not ingestion.
As far as I can tell, the decisive small-team question is whether consolidating integrations is worth keeping these specialist tools at the edges. That is a real trade, not a universal ranking.
The critical path in runnable Python
The following single-file service models the application side without hiding state in a framework. It exposes distinct startup, readiness, and liveness checks, a small Prometheus-style metrics response, and structured transition logs. Run it with Python, point the Kubernetes probes at ports and paths shown in ProbeHandler, and have the real dependency checker call HealthState.set_ready. In a Node.js application I use the same state machine around the framework's route handlers; the HTTP contract is the important part.
import json
import threading
import time
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
class HealthState:
def __init__(self):
self.lock = threading.Lock()
self.started = False
self.ready = False
self.live = True
self.failures = {"startup": 0, "readiness": 0, "liveness": 0}
def set_ready(self, value, reason):
with self.lock:
previous = self.ready
self.ready = value
if previous != value:
print(json.dumps({
"timestamp": time.time(),
"service": "checkout-api",
"probe": "readiness",
"old_state": previous,
"new_state": value,
"reason": reason,
}), flush=True)
def check(self, probe):
with self.lock:
healthy = {
"startup": self.started,
"readiness": self.ready,
"liveness": self.live,
}[probe]
if not healthy:
self.failures[probe] += 1
return healthy
STATE = HealthState()
class ProbeHandler(BaseHTTPRequestHandler):
paths = {
"/health/startup": "startup",
"/health/ready": "readiness",
"/health/live": "liveness",
}
def do_GET(self):
if self.path == "/metrics":
with STATE.lock:
lines = [
f'app_ready{{service="checkout-api"}} {int(STATE.ready)}',
*(
f'probe_failures_total{{service="checkout-api",probe="{name}"}} {count}'
for name, count in STATE.failures.items()
),
]
self._send(200, "\n".join(lines) + "\n", "text/plain")
return
probe = self.paths.get(self.path)
if probe is None:
self._send(404, '{"error":"not_found"}\n', "application/json")
return
healthy = STATE.check(probe)
status = 200 if healthy else 503
self._send(status, json.dumps({"probe": probe, "ok": healthy}) + "\n",
"application/json")
def _send(self, status, body, content_type):
encoded = body.encode("utf-8")
self.send_response(status)
self.send_header("Content-Type", content_type)
self.send_header("Content-Length", str(len(encoded)))
self.end_headers()
self.wfile.write(encoded)
def log_message(self, format_string, *args):
return
def initialize():
time.sleep(2)
with STATE.lock:
STATE.started = True
STATE.set_ready(True, "required_dependencies_available")
if __name__ == "__main__":
threading.Thread(target=initialize, daemon=True).start()
ThreadingHTTPServer(("0.0.0.0", 8080), ProbeHandler).serve_forever()
The 503 responses above are application health states, not evidence of an observability service outage. Set the startup probe's failure window longer than the application's measured initialization envelope, keep readiness frequent enough to stop new traffic promptly, and make liveness conservative. I am not sure why teams still copy identical timings for all three; those timings encode different risks.
For centralized storage, ship the JSON transition records and report the bounded counters and gauge. Infrai provides POST /v1/logs/ingest and POST /v1/metrics/report for those two writes. Their exact bodies should come from the public discovery schemas rather than hand-written guesses; authenticate with Authorization: Bearer $INFRAI_API_KEY, check every response status, and apply the retry rules described above.
Rejected design and valid exceptions
I reject the common design in which one /health handler checks every downstream dependency and drives startup, readiness, and liveness. It creates a coupled restart loop: a remote database or object store degrades, every pod declares itself dead, Kubernetes restarts all of them, and the restart surge adds load while deleting local diagnostic context. It also makes the dashboard deceptively binary. A service can be alive, temporarily unready, and still capable of serving a reduced operation safely.
The rejected design does have a valid use case. For a tiny stateless worker with no degraded mode, no expensive startup, and one required dependency, a shared internal evaluator can feed three differently configured probe responses. Even there, I keep the endpoints distinct so their policies can diverge later. Likewise, a dedicated uptime platform may replace most of this simple operational view when public reachability, escalation schedules, and multi-region synthetic checks are the actual requirements. This pattern is not a full replacement for one.
The final ADR is therefore narrow: let the orchestrator recover local process failure; preserve health transitions in logs; graph readiness and probe-failure metrics; carry trace_id and span_id for timestamp-based correlation; and assign alerts, heartbeats, tracing, crash analysis, replay, and compliance deletion to systems that explicitly support them. A small SaaS can start here without confusing "simple" with "complete."
Top comments (1)
As a frontend developer, this was such an insightful look into how backend orchestration choices ripple straight up to the UI! 🎯
When a misconfigured Liveness probe triggers a restart loop, our client-side apps usually get hit with a storm of 502/504 errors, making graceful fallback UI almost impossible. Understanding the distinct role of Readiness probes—how they temporarily hold traffic without wiping the container—is super helpful for designing better retry strategies and offline/degraded states on the client.
Really appreciated the clear breakdown, even for those of us who spend most of our time in the browser! 🚀