DEV Community

Discussion on: Your AI's code is "almost right". Here are 7 ways it fools you, and the 30-second check for each

Collapse
 
member_957b667c profile image
Maxwell •

the one that bites me isn't on the list: right shape, wrong value. no exception, nothing to catch, the field is filled in, it's just not what the page said. a parser can read annual as monthly across a whole layout and never once fail.

check: 5 rows at random, diffed against the source by hand. checking only the rows that already looked odd catches nothing.

Collapse
 
wst2002rk profile image
Wageesha Tripathi •

Thank you, this one should have been on the list. It's the hardest kind to catch because every check passes. Random rows is a good habit. One more that helps with parsers: compare a total from the source page (sum of a column, row count) with the same total from the parsed data. If annual was read as monthly, the totals stop matching right away. Do you run your 5-row check on every parser change, or only for new sources?