Plain code passed all 135 checks that Claude Sonnet passed with our rules. Then one of its rules refused an ordinary sentence with a web address in it.
Yes, for these tasks, and that is less of a win than it sounds. We wrote a program for each of six repair and conversion jobs and ran it through the 135 checks we had built for AI models. The programs passed all 135, the same as Claude Sonnet with our written rules loaded. Then a sentence that was in none of the cases showed one rule refusing perfectly good text.
Read the full report on AISkills402: https://aiskills402.com/blog/data-repair-code-vs-llm
Top comments (0)