PROOF — Planning, Reasoning, Orchestration, Observability, Feedback — is a five-category, 25-point rubric for scoring whether an "AI agent" claim is actually backed by agentic architecture. The version below breaks each category into measurable sub-criteria instead of a single subjective 0–5 gut call, and applies it to two domains where the difference between a real agent and a well-dressed prompt isn't just an ROI question — it's a safety and liability question: healthcare and manufacturing.
The Rubric, Formalized
Each category is worth 5 points, built from concrete, checkable sub-criteria rather than a single impression score.
Planning (5 pts)
- Explicit goal representation and task decomposition artifact exists — a plan graph, task tree, or structured intermediate representation, not just a prompt (0–2)
- System can revise the plan mid-execution when new information invalidates the original approach, rather than executing a fixed script (0–2)
- Constraint handling prunes invalid or unsafe plan branches before execution, not after (0–1)
Reasoning (5 pts)
- Outputs are grounded in retrievable evidence — cited tool output, sensor data, or source documents — not just fluent text (0–2)
- A verification or self-consistency pass exists (a second model, a rule-based checker, or a cross-reference step) before the reasoning is acted on (0–2)
- The system can abstain or escalate under uncertainty instead of producing a confident guess (0–1)
Orchestration (5 pts)
- Coordination runs on an explicit state machine or durable workflow engine (LangGraph, Temporal, a formal state graph), not ad hoc prompt chaining (0–2)
- Each step has defined retry, rollback, and failure-handling behavior — not "if it errors, stop" (0–2)
- Sub-agents or stages communicate through typed, validated interfaces (schemas), not free-text handoffs (0–1)
Observability (5 pts)
- Per-step tracing — tool calls, latency, token/compute cost — is exported to a real system (OpenTelemetry, a trace store, a cost dashboard), not just visible in a terminal (0–2)
- An immutable, exportable audit log exists, sufficient for a third party to reconstruct what happened and why (0–2)
- Cost and usage are attributed per task or per case, not just aggregated monthly (0–1)
Feedback (5 pts)
- An automated evaluation suite (golden test set, regression tests) runs pre-deployment and continuously, not just once at launch (0–2)
- Irreversible or high-consequence actions are gated behind a human checkpoint, not auto-executed (0–2)
- Flagged failures feed back into the system — an updated eval, a corrected prompt, a retraining trigger — closing the loop rather than being logged and forgotten (0–1)
Healthcare: Clinical Intake and Triage Support
Two versions of the same idea — an agent that reviews patient intake data and flags cases for clinician attention — scored against the same rubric.
Version A — the thin wrapper. A single LLM call takes free-text intake notes and outputs a "risk level" and a suggested next step, displayed directly to a nurse with no supporting detail.
| Category | Score | Why |
|---|---|---|
| Planning | 1/5 | One-shot classification, no decomposition, no revision if new labs come in mid-session |
| Reasoning | 1/5 | Risk label isn't traceable to specific findings; asking "why" gets a plausible-sounding paragraph, not a cited data point |
| Orchestration | 0/5 | No pipeline — a single call in, a single output out |
| Observability | 0/5 | No trace of which fields the model weighted, no audit trail beyond the final label |
| Feedback | 1/5 | No eval suite; "verification" is whether the nurse happens to disagree and overrides it |
| Total | 3/25 | Bagent — regardless of the clinical framing, this is a single unverified LLM call with real-world consequences riding on it |
Version B — the engineered version. Structured intake fields feed a decomposition step that checks against defined clinical decision rules and known red-flag combinations; every flagged risk factor links to the specific vital sign, lab value, or reported symptom that triggered it; the pipeline runs on a durable workflow engine with a retry policy per data source; every run is traced and logged to an audit store built for compliance review; nothing is escalated to "urgent" without a rule-based checker agreeing with the model's assessment, and any case the model is uncertain about is auto-routed to a clinician rather than guessed at; a continuously-run eval suite checks the system against a held-out set of historical, clinician-labeled cases before every deployment.
| Category | Score | Why |
|---|---|---|
| Planning | 4/5 | Structured decomposition against defined criteria; revises assessment as new intake data arrives |
| Reasoning | 4/5 | Every flag traces to a specific data point; uncertain cases are surfaced, not guessed |
| Orchestration | 5/5 | Durable workflow engine, explicit retry/rollback per data source, typed interfaces between stages |
| Observability | 4/5 | Full per-step tracing and an audit-grade log; cost/latency attribution present |
| Feedback | 5/5 | Continuous eval against clinician-labeled cases, hard checkpoint before any "urgent" escalation, closed-loop correction when a miss is found |
| Total | 22/25 | Real Agent — genuinely agentic, and just as importantly, built so a human can audit exactly why it did what it did |
The gap between these two isn't model quality — both could run on the same underlying LLM. It's whether Reasoning is traceable and whether Feedback includes a hard gate before anything irreversible happens. In a clinical setting, those are exactly the two categories liability actually hinges on.
Manufacturing: Production-Line Anomaly Response
Same exercise, applied to an agent monitoring sensor data on a production line and responding to anomalies — everything from a temperature drift to a full line stoppage.
Version A — the thin wrapper. A model watches a stream of sensor readings, and when something looks off, it calls an API to adjust a setpoint or halt the line, based on a prompt describing "normal" operating ranges.
| Category | Score | Why |
|---|---|---|
| Planning | 1/5 | Reacts to the latest reading; no model of what "normal drift" vs. "genuine fault" looks like over time |
| Reasoning | 1/5 | No grounding beyond the instantaneous reading — no correlation against related sensors, maintenance history, or known fault signatures |
| Orchestration | 1/5 | Single call triggers a single action; no defined behavior if the halt command itself fails to execute |
| Observability | 1/5 | Action gets logged as an event, but no trace of what data or reasoning triggered it |
| Feedback | 0/5 | No eval against historical fault data; nothing checks whether the halt call was even the right call after the fact |
| Total | 4/25 | Bagent — a live model wired directly to a physical actuator with no verification step is a hazard, not an agent |
Version B — the engineered version. Sensor streams feed a state model that distinguishes normal drift from anomaly using multi-sensor correlation and historical baselines; any proposed corrective action is checked against interlock and safety constraints before execution; the workflow runs on an orchestration layer with explicit rollback if an actuator command fails to confirm; every decision — inputs, correlated signals, the action taken, and the interlock check result — is traced to a time-series log tied to the plant's existing OPC-UA/historian infrastructure; low-confidence anomalies alert a human operator instead of triggering an automatic response, and full line stoppages always require operator confirmation regardless of model confidence; the fault-detection model is continuously evaluated against a labeled library of past fault events and re-validated whenever a new fault type is added.
| Category | Score | Why |
|---|---|---|
| Planning | 4/5 | Multi-sensor correlation against historical baselines, not single-reading reaction |
| Reasoning | 4/5 | Every action traces to specific correlated signals; low-confidence cases don't get auto-actioned |
| Orchestration | 5/5 | Explicit interlock checks, confirmed rollback on failed actuator commands |
| Observability | 5/5 | Full decision trace tied into existing historian infrastructure — auditable by plant engineers, not just data scientists |
| Feedback | 4/5 | Continuous evaluation against a labeled fault library; hard human gate on full stoppages |
| Total | 22/25 | Real Agent — same category of task as Version A, built so a wrong call gets caught before it touches a physical process |
What the Two Domains Have in Common
Line up both Version B scorecards and the pattern holds across completely different domains: the categories that separate a real agent from a dangerous wrapper aren't Planning or Orchestration — plenty of thin wrappers can fake a plausible-looking pipeline. It's Reasoning traceability and the Feedback hard-gate. In both healthcare and manufacturing, the real agent's defining trait isn't that it's smarter — it's that every consequential action is grounded in a specific, checkable piece of evidence, and nothing irreversible happens without either a rule-based check or a human confirming it first.
That's the actual engineering takeaway for anyone scoring an agent in a regulated or safety-critical domain: don't spend your review time asking whether the plan looks sophisticated. Spend it asking to see the trace for the last high-stakes decision it made, and asking who — or what — had to sign off before it acted.
Note: the scenarios above are illustrative composites built to demonstrate the scoring method, not descriptions of any specific deployed product. Nothing here is medical, safety, or compliance advice — treat it as an engineering evaluation framework, not a certification standard.
Top comments (0)