Dartmouth researchers find that large language models struggle with clinical reasoning, potentially increasing physician workload rather than reducing it.
A comprehensive study of AI-assisted patient communication has raised fresh questions about whether generative language models genuinely streamline clinical workflows or simply shift the burden of work from composition to revision.
Researchers at Dartmouth College analyzed roughly 146,000 messages exchanged between primary care physicians and their patients through an online portal at Dartmouth Health in Lebanon, New Hampshire. The team evaluated responses drafted by six different AI systems, including widely adopted models like Claude, Gemini, and ChatGPT, alongside smaller commercial alternatives such as Llama, Aloe, and Qwen. According to Becker's Hospital Review, the findings, presented at the Association for Computational Linguistics annual meeting in July, painted a sobering picture of current AI capabilities in clinical settings.
The core problem: while AI-generated drafts sounded professionally appropriate, they frequently contained inaccuracies, included unnecessary tangential information, and neglected to pose clinically relevant follow-up questions. Physicians tasked with reviewing and correcting these drafts often spent more time editing than they would have spent writing responses entirely from scratch.
The Editing Burden Paradox
"We find that AI can sound like a doctor but not think like one," said Sarah Preum, PhD, an assistant professor of computer science at Dartmouth College and co-corresponding author of the research. This distinction captures a fundamental limitation: language models excel at mimicking surface-level clinical communication patterns without grasping the underlying diagnostic reasoning and clinical judgment that physicians must apply.
The research team developed an intervention called TADPOLE, a technique that personalizes AI models to individual physicians' communication styles and medical decision-making patterns. This adaptation yielded measurable improvements:
Response accuracy improved by 33 percent
Editing time decreased by 26 percent
However, these gains have practical limitations. When physicians must substantially revise AI-generated content, the economics of the tool deteriorate rapidly. "If you have to edit 75 percent of the message, you may be spending more time and energy on making changes than if you were to just write it from scratch," explained Tim Burdick, MD, an associate professor of community and family medicine at Dartmouth's Geisel School of Medicine and a practicing family physician at Dartmouth Health.
Broader Implications for Clinical AI
This research joins a widening body of academic literature questioning whether generative AI actually reduces clinician workload or merely redistributes it. Healthcare organizations have rapidly adopted AI tools for documentation, message drafting, and clinical decision support, often under the assumption these systems would free up physician time. The Dartmouth findings suggest the reality is more complicated.
The distinction between sounding clinical and reasoning clinically has profound implications for AI deployment in healthcare. Patient portal communication requires not just appropriate medical tone but also clinical accuracy, knowledge of individual patient history, and sound judgment about what information is relevant and what follow-up care might be necessary.
The research underscores a critical challenge facing AI vendors and health systems: optimization for surface-level plausibility does not automatically translate to tools that enhance clinical practice. As healthcare organizations continue evaluating and implementing generative AI solutions, these empirical findings offer a cautionary framework for assessing whether new tools genuinely improve efficiency or create additional work under the guise of automation.
This article was originally published on AI Glimpse.
Top comments (0)