A team got a legacy CronosPro database that every existing parser said was broken. They handed the binary to Codex to reverse-engineer, and the model did its part: figured out the KOD-obfuscated schema, fixed the parser assumptions, got values decoding.
The work that actually took the time wasn't the AI decoding bytes. It was building structural validation so the output could be trusted.
The kicker from their writeup: "A CSV containing readable values under the wrong headers would be worse than an obvious error because it could look valid while being semantically corrupted."
That's the whole eval problem in one sentence. An agent that returns a string that looks right has failed in the most dangerous way possible. Grep a few headers, glance at a table, cells show dates and names, looks fine. Ship it and your data lake now has a schema that's confident and wrong.
The pattern matches what I keep seeing in coding-agent evals too. The thing that separates a useful run from a dangerous one is rarely "did it produce plausible output." It's "does some check that isn't the model's own opinion confirm the output is right against ground truth."
For the reverse-engineering job, that meant respecting physical field positions, not "it reads like a date." For code generation it's the same: an assert against a golden result, a static check, a real test run. Anything that isn't the agent judging its own work.
The model is the cheap part now. Everyone has one. The moat is the validator you build around it. If you're evaluating an agent or a tool and it doesn't show you what structural check the output had to pass, you're not looking at a result, you're looking at a vibe.
Top comments (0)