We gave Claude Sonnet and Haiku twelve broken product searches. Sonnet fixed all twelve unaided; Haiku fixed seven, and its misses hid real costs.
A large one can, a small one only partly. Twelve short, invented product-search problems were handed to the two models, Claude Sonnet and Haiku, and nothing else came with the task. Sonnet solved all twelve. Haiku solved seven and failed five, and two of the five were the expensive kind: a query that still reads every row on each keystroke, and an accent rule that quietly damages Bulgarian. With our SQLite Search Builder skill loaded, Haiku solved all twelve under the same checks.
Read the full report on AISkills402: https://aiskills402.com/blog/llm-search-plurals-accents
Top comments (0)