Your AI Agent's Health Check Lies Unless It Tests the Tool Path

A process can answer 200 OK while every useful action is failing.

That is the dangerous kind of healthy state for an AI agent. The supervisor sees a live process, the container stays running, and the dashboard is green. Meanwhile the agent cannot reach its model gateway, its tool credentials are expired, its browser profile is locked, or its queue consumer has lost its lease.

The fix is not to make one giant health endpoint that performs a real side effect. The fix is to model health as several explicit signals and test the path that matters without crossing the mutation boundary.

Separate liveness, readiness, and capability

Use at least three states:

Signal	Question	Safe check
Liveness	Is the process event loop responding?	Local response with a deadline
Readiness	Can this worker accept a new run?	Queue lease, config, and dependency checks
Capability	Can this run perform its required tool calls?	Non-mutating probes for the selected capability set

Liveness should be cheap. A liveness failure usually means the process needs a restart.

Readiness is an admission decision. A worker with a stale configuration version or no durable-state connection should stop receiving new work, even if its process is alive.

Capability is run-specific. A text-only task may need a model gateway and a database, while a browser task also needs a valid browser profile, network policy, and download storage. Do not report one global green light when only half the capabilities are usable.

Make the checks typed, not boolean

A boolean loses the information needed for recovery. Return a typed result with an owner and a next action:

{
  "component": "browser",
  "state": "DEGRADED",
  "checked_at": "2026-08-20T01:00:00Z",
  "config_version": "cfg-184",
  "credential_version": "cred-77",
  "evidence": {
    "profile_lock": "held-by-worker-12",
    "read_only_probe": "passed"
  },
  "admission": "reject_browser_runs",
  "recovery": "recycle_profile_after_lease_expiry"
}

Useful states are usually PASS, DEGRADED, FAIL, and UNKNOWN. UNKNOWN matters when a probe timed out or a dependency returned an ambiguous result. It should not silently become PASS, and it should not always trigger an aggressive restart.

A typed result also makes alerts actionable. “Agent unhealthy” is not an operator instruction. “Reject browser runs, keep text runs admitted, recycle the profile after the lease expires” is.

Probe dependencies without performing user mutations

A tool-path probe should be representative but reversible. Examples:

• Model gateway: authenticate, request the smallest allowed metadata or token-count operation, then record latency and policy version.
• Database: open a connection, verify schema version, and execute a read-only query with a deadline.
• Object storage: verify access to the expected bucket and prefix, without uploading a test object unless the system has a cleanup contract.
• Browser: launch a disposable profile, navigate only to an allowlisted diagnostic page, and verify the expected browser/CDP handshake.
• MCP: list the server’s declared tools and compare them with the policy snapshot, without invoking a mutating tool.

Never use “send an email to test email.” A health check that creates an external side effect is an incident generator. If a provider cannot offer a safe probe, classify that capability as UNVERIFIED and rely on delivery receipts or a separate canary account with an explicit cleanup path.

Tie checks to the run's capability set

Before admission, compute the required capability set from the task, not from the worker's startup profile:

def admit(run, health):
    required = set(run.required_capabilities)
    missing = []

    for capability in required:
        result = health[capability]
        if result.state not in {"PASS", "DEGRADED"}:
            missing.append((capability, result.state))

    if missing:
        return {"decision": "REJECT", "missing": missing}

    return {
        "decision": "ADMIT",
        "health_snapshot": health.snapshot_id,
    }

Store the health snapshot ID with the run admission record. Otherwise an incident review cannot answer a basic question: did the worker admit the task before or after the dependency failed?

The snapshot is not permanent authorization. Recheck policy, credential version, and lease ownership at dispatch time. Health proves a narrow observation at a point in time; it does not grant permission to perform a later action.

Test the failure domains separately

A useful test matrix injects one fault at a time:

1. Keep the process responsive but block the model gateway. Liveness should pass; model capability should fail; new model runs should be rejected.
2. Expire the browser credential while the process remains alive. Browser capability should become degraded or failed; text-only work should remain unaffected.
3. Break the queue lease renewal. Readiness should fail after the lease deadline, and another worker should be able to claim new work.
4. Return a timeout from a tool dispatch. The result should be UNKNOWN, not an automatic retry that might duplicate a mutation.
5. Roll the configuration version while a run is waiting. The worker should recheck at dispatch and either refresh safely or reject the run.
6. Fill the diagnostic storage path. The probe should identify the capacity failure without turning every unrelated capability red.

For each test, record the expected admission decision, the alert, the recovery owner, and whether already-running work is allowed to finish. That last field prevents a readiness failure from becoming an unplanned cancellation storm.

Make recovery narrower than restart

A failed capability does not always justify restarting the whole agent. Prefer the smallest recovery domain:

• Refresh a short-lived credential when only its version is invalid.
• Recycle a disposable browser profile when the profile lock is stale.
• Pause admission when the queue lease is lost.
• Route around one unavailable model provider when policy permits.
• Restart the process only when liveness or local invariants fail.

For always-on OpenClaw or browser-agent deployments, the hosting layer is part of this recovery contract. A managed runtime such as always-on OpenClaw hosting on Ampere can be a hosting option to evaluate when you need a persistent process and repeatable restart boundary, but it does not remove the need to define state persistence, credential scope, health probes, and recovery policy yourself.

What to put on the dashboard

Show separate panels for:

• process liveness and restart count;
• readiness and admission rejections;
• capability state by dependency and task type;
• probe latency and timeout rate;
• configuration and credential versions;
• unknown outcomes awaiting reconciliation;
• recovery actions and their completion status.

The goal is not a prettier green dashboard. The goal is to make “alive but unable to do useful work” impossible to confuse with “ready to accept this run.”

If you build agents, add one tool-path probe this week. Then inject one dependency failure while the process stays alive. If the system cannot reject only the affected work, preserve the evidence, and recover the smallest possible domain, the health check is measuring uptime, not health.