Bare, Sonnet and Haiku pointed at wrong totals and weekdays but never wrote the sum or the true day that proves them. A skill file made them show it.
Mostly not. Our test set two Claude models, Sonnet and Haiku, to hunt for self-contradictions in short documents. On their own, both usually pointed at the right passage, yet wherever the answer needs arithmetic or a calendar they rarely wrote down the figure or the weekday that proves the problem. A budget whose lines add up to 105,000 against a stated 110,000 came back as a finding that set the lines against the total, with no 105,000 anywhere. The reader must redo the addition, and a finding without its proof looks just like a guess.
Read the full report on AISkills402: https://aiskills402.com/blog/llm-contradiction-check-skips-the-sum?utm_source=devto&utm_medium=social
Top comments (0)