Six texts in five languages, two Claude models. Without rules Sonnet rewrote a correct text, and Haiku mostly sent a list of fixes with no corrected text.
Not on its own. Two Claude models, Sonnet and Haiku, each got six short texts and a bare proofreading request: "Proofread this text." Five of the texts carried errors we had planted, in Bulgarian, English, German, Russian and Spanish; the sixth was a correct English text that needed nothing. Without further instructions Sonnet passed four of the six cases and Haiku one. Once our rules were loaded as a skill, Sonnet passed all six and Haiku five. The interesting failures were not missed typos. Sonnet returned the correct text with two edits after calling it clean, and Haiku mostly answered with a numbered list of fixes and no corrected text at all, which leaves a program with nothing to…
Read the full report on AISkills402: https://aiskills402.com/blog/llm-proofreading-changes-correct-text
Top comments (0)