Claude Sonnet and Haiku summarized six short texts in five languages from a plain request. Without rules they ran long, switched language and bent conditions.
Partly. The names and numbers mostly come through; the length, the language and some of the conditions do not. Each of two Claude models, Sonnet and Haiku, got six short texts, two of them English and one each in Bulgarian, German, Russian and Spanish, plus the bare request "Summarize this text." Sonnet answered in the language of each text, but its summaries ran to between 68 and 89 percent of the original, and twice it appended a remark of its own about today's date. Haiku replied in English to three of the five texts that were not English, turned a condition about selling two ferries inside out, and described a price rise of about 56 percent as a near doubling. After we added our rules…
Read the full report on AISkills402: https://aiskills402.com/blog/llm-summary-adds-and-drops
Top comments (0)