"The Reasoning Trap." The line the authors won't quite say out loud is that the smarter your model gets at reasoning, the more likely it is to fabricate a tool that doesn't exist.
We've spent eighteen months telling ourselves that smarter reasoning would fix the reliability problem in agents. The paper shows the opposite. Reinforcement-learned reasoning lifts task scores and amplifies tool hallucination at the same time. They don't trade off. They move together.
I've been seeing this for months at Upswing and didn't have a name for it. We run tool-using agents across hospitality ops — pricing, IoT telemetry, guest comms. The smarter models we tested were better at staying on task. The catch was their failure mode. When they got stuck, they got more confident about it. The dumber models would say "I can't do this." The smart ones would synthesize a plausible-sounding call to a function we'd never written. They didn't fail loud. They invented their way out.
The mental model I keep landing on: reasoning RL teaches a model to find a path forward at all costs. "There must be a way" becomes the prior. And when the only honest answer is "there isn't a tool for this," the model hallucinates one rather than refuse.
The fix isn't a smarter model. It's the boring runtime work — strict tool schemas validated at the call site, hard refusal scored as a first-class outcome, evals that explicitly reward graceful declines.
The agents that win in production this year won't be the ones with the deepest reasoning chains. They'll be the ones that know how to say "I don't know" — and get rewarded for it.
Top comments (0)