The refactor came back green. CI passed, the diff looked tidy. A week later, production threw a 500 and the log said exactly: Internal error.
That line used to read: invoice 4412 could not be voided: ledger row locked by migration job.
Now it reads nothing. The agent generalized it away.
What actually happened
There were two catch blocks with different messages. The agent saw that as duplication and consolidated them into one logAndRethrow(err) helper. The helper takes one argument: err. The invoice id, the lock holder, the calling job — all dropped, because none of them were on the happy path.
It was correct. It was clean. It was also the single most expensive change in that whole diff, and it cost nothing until 2am.
Why agents do this
They optimize for DRY and readability. A string like invoice 4412 could not be voided looks like a hardcoded smell. The fact that it was the only thing pointing at the failure at 3am is not in the signal they're optimizing for. They have no model of your on-call rotation.
Your tests don't catch it either, because they assert on the error class, not the message. Your monitoring doesn't catch it, because it still counts one 500. The human gets a wall of identical strings and has to reproduce to find out which of ten code paths broke.
The diff rule I use now
If a refactor touches a throw or a catch and the new message has fewer interpolation variables than the old one, that's a flag. Stop and ask: what context did this drop, and where does it go now?
Treat error messages like a contract your future self reads under stress. Refactor them the way you'd refactor a public API — preserve the fields callers (and logs) match on. Moving them into an enum or a structured error object is fine; flattening them into one generic string is not.
This is sharper when the errors cross a boundary. The error shape in an API response is part of the spec — the generated client parses it, monitoring groups by it, SDK users branch on it. When the agent "simplifies" an error enum or drops a field, everything downstream quietly breaks the same way. I keep that contract locally in Powerduck and re-run it against the live endpoint after agent changes; drift in error responses shows up the same way drift in success responses does, and it's cheaper to catch in CI than in the log.
Rough rule
A green refactor that shortens your error strings deserves one more look than one that doesn't.
What's the most generic error message you've ever been handed at 2am?
Top comments (0)