Claude Sonnet and Haiku wrote correct Excel and Google Sheets formulas for 21 tasks, run in a real formula engine. The errors we saw came from our skill.
Yes, for the everyday kind. We handed 21 spreadsheet tasks to two models, Claude Sonnet and Claude Haiku, each a short description plus a few rows of the sheet, and every answer was run in a formula engine on that sample. In the final run, with our instructions loaded, neither model got a single one of the 21 wrong. Without them Haiku also got all 21, and Sonnet got 19, losing the other two only by adding a sentence after a correct formula. The finding we did not expect was about us: on the way to that result, two of the skill's own wordings caused failures that the models did not make without it.
Read the full report on AISkills402: https://aiskills402.com/blog/llm-spreadsheet-formulas-skill-was-bug
Top comments (0)