Medical AI agents face a dual constraint problem. They must stay grounded in clinical evidence while adapting explanations to individual patient context. Standard RLHF optimizes for a single holistic reward signal, which lets the model trade factuality for fluency or vice versa. G-CARL (Grounded Checklist-Aligned Reward Learning) splits the problem into verifiable atomic claims and context-dependent coverage requirements, then enforces both through structured supervision.
The architecture matters beyond medicine. Any agent system where hallucination has real-world consequences (legal interpretation, financial advice, safety-critical automation) needs a way to enforce factual grounding without freezing the model into template responses.
The Dual Constraint Problem
Patient-oriented medical report interpretation (PMRI) requires:
- Evidence grounding: Every medical claim must trace back to the radiology report or clinical knowledge base.
- Patient appropriateness: Language, detail level, and emphasis must adapt to the patient's query and dialogue history.
These objectives conflict under naive optimization. A model rewarded for "helpfulness" might invent plausible-sounding details. A model penalized for any unsupported claim might refuse to explain anything.
G-CARL solves this by separating verification (is this claim true?) from coverage (does this response address the patient's actual question?). The reward model enforces both independently, then combines them with instance-specific weights.
Grounding Architecture
G-CARL uses multi-source retrieval to verify atomic claims:
- Claim extraction: Parse the agent's response into discrete medical statements.
-
Retrieval: Query three sources in parallel:
- The original radiology report (exact match for findings)
- A medical knowledge base (for general clinical facts)
- Prior dialogue context (for patient-specific history)
- Verification: Each claim gets a binary label (grounded / ungrounded) based on whether supporting evidence exists in any source.
This is not semantic similarity search. The system checks whether a claim can be justified by the available evidence, not whether it sounds medically plausible.
Checklist-Aligned Coverage
The second component is a dynamic checklist generated for each patient query:
- Extract key information needs from the query and dialogue history.
- Build a weighted checklist of required topics (e.g., "explain the lung nodule," "compare to prior scan," "discuss next steps").
- Score the response by how many checklist items it addresses.
Weights adapt per instance. If the patient asks specifically about a nodule, that item gets higher weight than general report structure.
Reward Model Plumbing
G-CARL combines three reward components:
| Component | Measures | Verification Method |
|---|---|---|
| Factuality | Claim-level precision | Multi-source retrieval + binary grounding labels |
| Coverage | Checklist recall | Weighted topic matching against query-derived checklist |
| Expression | Fluency and tone | Lightweight LM-based scorer for readability |
Weights are instance-tunable hyperparameters optimized on a validation set.
The final reward is a weighted sum:
def compute_reward(response, report, query, dialogue_history):
# Extract discrete medical claims from the generated response
claims = extract_claims(response)
# Verify each claim against report, knowledge base, and dialogue
grounded = [verify_claim(c, report, kb, dialogue_history)
for c in claims]
factuality_score = sum(grounded) / len(claims)
# Generate query-specific checklist of required topics
checklist = generate_checklist(query, dialogue_history)
# Score how many checklist items the response addresses
coverage_score = compute_coverage(response, checklist)
# Evaluate readability and appropriateness
expression_score = score_fluency(response)
return (
w_fact * factuality_score +
w_cov * coverage_score +
w_expr * expression_score
)
Weights w_fact, w_cov, w_expr are tuned on a validation set. The key insight is that factuality and coverage are verified independently, so the model cannot game one by sacrificing the other.
Training Pipeline
G-CARL uses proximal policy optimization (PPO) with the structured reward model:
- Supervised warm-start: Fine-tune a vision-language model on clinician-written interpretations.
- Reward model training: Train the factuality verifier and checklist generator on labeled examples.
- RL fine-tuning: Run PPO with the composite reward, using KL divergence from the supervised policy as a regularizer.
The KL penalty prevents the model from drifting into adversarial solutions (e.g., refusing to answer to avoid ungrounded claims).
Inference Flow
At inference time, the agent:
- Receives a radiology report, patient query, and dialogue history.
- Generates a candidate interpretation.
- (Optional) Runs self-verification: extract claims, check grounding, regenerate if factuality score is below threshold.
- Returns the interpretation.
Self-verification adds latency but improves precision. In production, you would batch claim verification and cache knowledge base lookups.
Evaluation Protocol
The paper introduces a three-dimensional evaluation:
- Factuality: Clinician-labeled claim precision (percentage of claims supported by evidence).
- Coverage: Checklist recall (percentage of patient information needs addressed).
- Expression: Readability and appropriateness scores.
Pairwise preference evaluation by clinicians confirms that G-CARL outputs are more accurate and better aligned with patient needs than baseline RLHF or supervised fine-tuning alone.
Failure Modes and Observability
G-CARL does not eliminate hallucination. It reduces it by making factuality verifiable and penalizing ungrounded claims. Known failure modes (inferred from the architecture, not all explicitly documented in the paper):
- Retrieval gaps: If the knowledge base lacks coverage for a rare condition, the model may refuse to explain it or fall back to generic language.
- Checklist drift: If the checklist generator misinterprets the patient's query, the model optimizes for the wrong coverage target.
- Adversarial fluency: The model might learn to hedge ("this may indicate...") to avoid binary grounding penalties.
Observability requirements:
- Log claim-level grounding decisions for each response.
- Track checklist item coverage per query.
- Monitor KL divergence from the supervised policy to detect reward hacking.
Generalization Beyond Medicine
The architecture generalizes to any domain where:
- Factual grounding is verifiable (you have a source of truth).
- Context-dependent communication is required (one-size-fits-all responses are inadequate).
- Hallucination has real consequences.
Examples:
- Legal agents: Ground claims in case law and statutes while adapting explanations to client context.
- Financial advice: Verify portfolio recommendations against regulatory constraints and market data while personalizing risk communication.
- Safety-critical automation: Ensure control decisions trace back to sensor data and safety rules while adapting to operational context.
The key is splitting the reward into verifiable and contextual components, then enforcing both independently.
Technical Verdict
Use G-CARL when:
- You need factual grounding in a domain with verifiable sources of truth.
- Context-dependent communication is a hard requirement (not a nice-to-have).
- You can afford the engineering overhead of multi-source retrieval and checklist generation.
- Hallucination has real-world consequences that justify the complexity.
Avoid G-CARL when:
- Your domain lacks structured knowledge bases or verifiable evidence sources.
- Context-independence is acceptable (e.g., FAQ bots, general summarization).
- Latency constraints make multi-source retrieval infeasible.
- You need real-time inference without batching or caching.
The architecture is not a drop-in replacement for RLHF. It is a structured reward framework for high-stakes domains where factuality and context must both be enforced without adversarial trade-offs.
Top comments (0)