Claude Sonnet and Haiku reviewed 20 SKILL.md files we seeded with problems. Both caught hidden orders and keys; Haiku missed the rules you have to count.
Half of it, reliably. When you review a SKILL.md before installing it, there are two kinds of problem to look for: text that pushes the agent toward places it has no business going, and rules that the file breaks by being too long or badly named. Claude Sonnet and Claude Haiku each reviewed 20 small skill files we had seeded with problems. Both models spotted all the hidden orders and the pasted key on their own. The counted rules were another matter: Haiku missed six of the 20 files without help and Sonnet two, almost all of them limits you can only check by counting. With a written checklist loaded as a skill, both models got all 20.
Read the full report on AISkills402: https://aiskills402.com/blog/llm-skill-md-review-counted-limits
Top comments (0)