We gave Claude Sonnet and Haiku fifteen broken JSON inputs. Without rules both kept cut-off values, rewrote numbers and turned an error page into data.
Not reliably, unless you tell it what to do at the places where the input leaves room for a guess. We gave two Claude models, Sonnet and Haiku, fifteen pieces of broken JSON with a plain request: fix this so a program can parse it, return only the JSON. Counting the content alone, with any Markdown code fence removed, Sonnet gave back exactly the original data in nine cases and Haiku in six. The misses were not syntax errors. They were answers that parse and say something the input never said: a value that was cut off kept as if it were whole, a number rewritten, and in one case a full object built out of an error message. With our written rules loaded as a skill, Sonnet got all fifteen…
Read the full report on AISkills402: https://aiskills402.com/blog/llm-json-repair-changes-data
Top comments (0)