A readiness endpoint for a Node.js metering service should report the last verified capability state, not call the account API on every probe. Put credential validity and tier eligibility behind a short, bounded cache; return ready, degraded, or not_ready; and let a spend ceiling decide when stale evidence must stop traffic. For metered invoices, that final rule matters more than shaving a few milliseconds from the probe.
TL;DR: refresh account capabilities outside the request path, retain the last successful result with its age, and fail closed when its age crosses a limit derived from the maximum unverified usage you can accept. A temporary refresh failure may be degraded while the cached result is young. An invalid credential or an unsupported metering capability is not ready.
This is an experiment note, not a vendor recipe. The evaluation constraint is concrete: a developer-tools API must attribute usage per customer without allowing unverified consumption to run past its configured ceiling. A naive live check looked appealing because it was always current. It also coupled every orchestrator probe to a remote control plane, so remote latency became local availability. The chosen design separates freshness from serving state and makes that trade-off visible.
How should readiness health checks handle a cached API credential?
Process health and business readiness answer different questions. A Node.js worker can have a responsive event loop and still be unable to meter the next customer request safely. If its API credential is invalid, it cannot refresh account capabilities. If the cached tier does not permit the required meter, accepting more work creates usage that the invoice path may not represent correctly.
Alive is not ready.
Treat those conditions separately. Liveness answers whether the process should be restarted. Readiness answers whether this instance should receive new metered work. Capability refresh answers what the external account currently permits. Folding all three into one boolean erases the reason an instance withdrew and encourages automation to react too aggressively.
I use three states because they map to decisions rather than moods:
| State | Evidence | Routing decision |
|---|---|---|
ready |
A recent successful check confirms the credential and required capability | Accept metered traffic |
degraded |
Refresh failed, but the last success remains inside the permitted stale window | Accept traffic while emitting an age signal |
not_ready |
Credential is known invalid, capability is absent, or evidence is too old | Refuse new metered traffic |
The subtle case is a timeout. It does not prove the credential is invalid. Turning one timeout into not_ready can drain every node during a shared dependency wobble. Ignoring timeouts forever is worse: the system can accumulate usage against an account state it has not verified. The stale window is the line between those risks.
That distinction drives everything else.
Derive staleness from the spend ceiling
Do not pick a cache duration because five seconds feels safe. Start with the business limit. If a node can admit at most r billable units per second and the acceptable unverified exposure is b units, the stale allowance cannot exceed b / r. With multiple independently admitting nodes, use the aggregate maximum rate or divide the budget among them.
For example, suppose two nodes can each accept 20 units per second and the team permits no more than 200 units after the last verified capability result. The upper bound is five seconds: 200 / (2 * 20). A lower operational value may be sensible, but a higher one contradicts the ceiling. This is the key trade-off: a shorter window refuses traffic sooner; a longer window increases unverified exposure. Now follow the failure through the system. Both nodes last refresh successfully at noon. The authority then becomes unreachable, so each keeps admitting at its maximum rate against young cached evidence. Together they can add 40 units each second. At the five-second boundary they have consumed the full 200-unit uncertainty budget, and both must withdraw before admitting another request. If one node owns only a quarter of the budget, its local cutoff must reflect that allocation rather than copying the fleet-wide duration. Tiny windows also amplify refresh load and correlated failure, so add randomized scheduling outside the request handler and preserve the last success when a refresh produces an indeterminate network error. Do not overwrite known-good evidence with an ambiguous failure. Do replace it when the authority definitively reports that the credential is invalid or the capability is absent.
Five seconds. No more.
Build the sentinel beside the Node.js service
The focused implementation below is a small FastAPI sentinel. That keeps the example runnable in a notebook-to-production Python workflow while the Node.js workload consumes the result over localhost or a private service boundary. The remote adapter is deliberately generic: its job is to return a typed observation, never expose the secret in the health response.
from __future__ import annotations
import asyncio
import random
import time
from dataclasses import dataclass, replace
from enum import StrEnum
from typing import Protocol
from fastapi import FastAPI, Response
from pydantic import BaseModel
class Outcome(StrEnum):
VALID = "valid"
INVALID_CREDENTIAL = "invalid_credential"
CAPABILITY_MISSING = "capability_missing"
INDETERMINATE = "indeterminate"
class CapabilityClient(Protocol):
async def inspect(self) -> Outcome: ...
@dataclass(frozen=True)
class Snapshot:
outcome: Outcome
checked_at: float | None
last_success_at: float | None
consecutive_failures: int
class HealthBody(BaseModel):
status: str
evidence_age_seconds: float | None
reason: str
class Sentinel:
def __init__(self, client: CapabilityClient, stale_after: float) -> None:
self.client = client
self.stale_after = stale_after
self.snapshot = Snapshot(Outcome.INDETERMINATE, None, None, 0)
self._lock = asyncio.Lock()
async def refresh(self) -> None:
try:
outcome = await asyncio.wait_for(self.client.inspect(), timeout=2.0)
except (TimeoutError, OSError):
outcome = Outcome.INDETERMINATE
now = time.monotonic()
async with self._lock:
if outcome is Outcome.VALID:
self.snapshot = Snapshot(outcome, now, now, 0)
elif outcome is Outcome.INDETERMINATE:
self.snapshot = replace(
self.snapshot,
checked_at=now,
consecutive_failures=self.snapshot.consecutive_failures + 1,
)
else:
self.snapshot = Snapshot(
outcome, now, self.snapshot.last_success_at, 0
)
def evaluate(self) -> tuple[int, HealthBody]:
snap = self.snapshot
age = (
None
if snap.last_success_at is None
else max(0.0, time.monotonic() - snap.last_success_at)
)
if snap.outcome in {
Outcome.INVALID_CREDENTIAL,
Outcome.CAPABILITY_MISSING,
}:
return 503, HealthBody(
status="not_ready",
evidence_age_seconds=age,
reason=snap.outcome.value,
)
if age is None or age > self.stale_after:
return 503, HealthBody(
status="not_ready",
evidence_age_seconds=age,
reason="evidence_too_old",
)
if snap.outcome is Outcome.INDETERMINATE:
return 200, HealthBody(
status="degraded",
evidence_age_seconds=age,
reason="refresh_indeterminate",
)
return 200, HealthBody(
status="ready", evidence_age_seconds=age, reason="verified"
)
app = FastAPI()
sentinel: Sentinel
@app.get("/ready", response_model=HealthBody)
async def readiness(response: Response) -> HealthBody:
status_code, body = sentinel.evaluate()
response.status_code = status_code
return body
async def refresh_loop() -> None:
while True:
await sentinel.refresh()
await asyncio.sleep(random.uniform(1.5, 2.5))
The response contains state, evidence age, and a low-cardinality reason. It contains no token, credential fingerprint, account identifier, tier name, or remote error body. Secret values belong in a managed secret lifecycle and should be protected from logging; the OWASP guidance in Further reading covers storage, rotation, auditing, and exposure concerns.
There is one production detail to add around this core: begin the refresh task during application startup and cancel it during shutdown. Also make the Node.js service depend on this sentinel only for admission, not for every unit it records. The metering path should continue to write durable, idempotent usage records after admission so a later invoice job can retry without double counting.
Test decisions instead of response shapes
A probe that returns JSON is easy to snapshot-test and easy to misunderstand. The useful tests advance a fake monotonic clock and ask whether the routing decision matches the ceiling. No real credentials are required.
import pytest
class SequenceClient:
def __init__(self, outcomes: list[Outcome]) -> None:
self.outcomes = iter(outcomes)
async def inspect(self) -> Outcome:
return next(self.outcomes)
@pytest.mark.asyncio
async def test_timeout_uses_young_success_as_degraded(monkeypatch) -> None:
clock = iter([100.0, 102.0, 103.0])
monkeypatch.setattr(time, "monotonic", lambda: next(clock))
client = SequenceClient([Outcome.VALID, Outcome.INDETERMINATE])
check = Sentinel(client, stale_after=5.0)
await check.refresh()
await check.refresh()
code, body = check.evaluate()
assert code == 200
assert body.status == "degraded"
assert body.evidence_age_seconds == 3.0
def test_missing_capability_refuses_immediately() -> None:
check = Sentinel(SequenceClient([]), stale_after=5.0)
check.snapshot = Snapshot(
Outcome.CAPABILITY_MISSING, 110.0, 109.0, 0
)
code, body = check.evaluate()
assert code == 503
assert body.reason == "capability_missing"
That first test is intentionally about a correction to the simple mental model. A failed refresh does not automatically mean a failed credential. The second locks down the opposite edge: definitive loss of the required capability bypasses the grace period. Add boundary cases at exactly the stale limit, before any successful refresh, after credential revocation, and after process restart with an empty cache.
Keep the cache per node unless you have a clear reason to coordinate it. A shared cache can align decisions, but it also introduces another dependency into admission and makes the spend budget sensitive to its consistency model. Per-node snapshots are easier to reason about when each node owns a fixed share of the exposure ceiling.
Make ownership explicit.
Measure this before copying the design
The readiness payload is for machines and operators. Metrics are for trend detection. Record refresh latency, outcome counts, evidence age, transitions by reason, and refused requests. Bound labels: reason and state are useful; customer IDs are not. Then compare admitted units during degraded intervals with the exposure budget that produced the stale window.
Watch the distribution, not only the average. A low mean evidence age can hide a node that repeatedly approaches the cutoff. Alerting on every degraded response will be noisy during brief network uncertainty; alert on sustained age, repeated transitions, or budget consumption instead. Keep logs free of credentials and remote response bodies.
The implementation is worth copying only after four inputs are known: maximum admission rate, acceptable unverified units, node count, and the external authority's distinction between definitive denial and indeterminate failure. Without those, a five-second cache is just a guess. The spend ceiling sets the clock; the clock decides when traffic must be refused.
Top comments (0)