Instruction fine‑tuning inflates verbalized confidence while leaving predictive accuracy unchanged, and it does not mend the underlying calibration problem. Practitioners have embraced instruction tuning for its headline gains on downstream tasks, assuming that a more “helpful” model will also become better at judging its own answers. The literature, however, has focused almost exclusively on task metrics such as exact‑match or BLEU, with little attention to how confidently the model expresses those answers.
The study by Proskurina et al. systematically compares three base models and their instruction‑tuned counterparts across several question‑answering benchmarks. Their methodology isolates confidence effects by measuring answer entropy and verbalized certainty, while holding predictive performance constant. This design lets them attribute any shift in confidence directly to the tuning process rather than to data or architecture changes.
Instruction tuning consistently raises model confidence without delivering any accuracy improvement. The authors state, “Instruction Tuning Consistently Increases Model Confidence without Corresponding Improvements in Accuracy” [1]. Across all evaluated settings they also observe lower answer entropy and higher verbalized confidence scores (Table 1), confirming that the models sound more sure even though their correctness does not budge.
At the same time, the tuning process narrows the diversity of generated rationales. The paper reports, “Across the evaluated models and benchmarks, we find that instruction tuning increases model confidence and reduces cross‑rationale diversity without a corresponding improvement in predictive accuracy” [1]. Surface‑level lexical variety swings both ways, but the systematic drop in cross‑rationale diversity indicates a homogenization of reasoning patterns that accompanies the confidence boost.
These results leave open several calibration questions. The authors note that likelihood‑based calibration metrics remain poor after instruction tuning, suggesting that the training objective does not penalize miscalibrated probability estimates. Moreover, the analysis is limited to three model families and QA tasks; it remains unclear whether similar effects appear in dialogue or summarization settings. This pattern hints that fine‑tuning alone cannot guarantee trustworthy confidence scores—additional alignment signals, such as capability‑aware reinforcement learning with explicit refusal rewards, may be required to curb futile reasoning.
If the community continues to equate instruction tuning with safer outputs, it will overlook a critical failure mode. Benchmarks should now include calibrated confidence evaluations (e.g., reliability diagrams or expected calibration error) alongside traditional accuracy scores, and pipelines ought to incorporate post‑hoc calibrators or RL‑based alignment steps rather than relying on instruction fine‑tuning as a panacea.
Top comments (0)