We asked Claude Sonnet and Haiku for six cold emails with no rules. Most ran far too long, and the one draft that passed invented a statistic and a discount.
Two things go wrong, and only one of them shows up in an automatic check: the email gets too long. The other one, invented facts, passed every check we wrote. We put two Claude models, Sonnet and Haiku, through six short briefs, each naming a sender, a recipient and a real reason to write, and asked for a cold email. Without any written rules, eleven of the twelve answers went past our limit of 139 words, the longest reaching 297. Sonnet filled in product details nobody gave it, and Haiku made up numbers. The single answer without rules that passed every check, Haiku's Russian email, cited an unnamed study with a 60 percent figure and offered a 20 percent discount that appear nowhere in…
Read the full report on AISkills402: https://aiskills402.com/blog/llm-cold-email-invented-facts
Top comments (0)