Before every release, somebody generates the notes. It used to take twenty minutes of scrolling the commit log. Now a model does it in four seconds, and there are a dozen products that will do it for you.
We got dramatically faster at producing a document nobody uses to decide anything.
Two different artifacts
Release notes are a communication artifact. Past tense, written outward, for people who weren't in the room — users, support, the changelog page. "Added X. Fixed Y." Their job is to tell someone what already happened.
The deploy decision needs a risk artifact. Future tense, written inward, for the person about to press the button. Its job is to say what could go wrong and what to check first.
Different tense, different reader, different content. We keep producing the first one and then making the second decision on vibes.
Put them side by side
Here is a realistic release note:
- Added dark mode
- Fixed login redirect
- Updated dependencies
Nothing in there is false. Nothing in there is what you need either.
What's worth knowing is that the login redirect fix touched session handling — the same code path as authentication. That the dependency update was a lockfile bump underneath that same auth code. That the two arrived together, in a PR one person approved at 18:00 on a Thursday.
The release note summarises what. The decision needs where, together with what, and reviewed how carefully.
Why AI made this worse rather than better
Summarising a diff is a solved problem. It's among the easiest things a language model does: the input is structured, the output is a list, and the failure mode is boring rather than dangerous. That's exactly why the category is crowded.
Deciding whether a set of changes is safe to ship is not that. It requires knowing which paths are load-bearing, what changed together, what the review actually said, and what the CI state means. Almost none of that lives in the diff text. It lives in the pull requests, the review states, the check runs, and the file paths.
So we automated the easy half and called the job done. The document got faster. The decision didn't move at all.
The tell
Ask anyone on your team what they read before approving a deploy.
Nobody says "the release notes." They say they skim the compare view — which is another way of saying that the artifact we generate is not the artifact we use.
A document produced for every release and consulted for none of them isn't a documentation problem. It's a signal that the real work is happening somewhere with no tooling at all.
Keep writing them anyway
None of this makes release notes useless. They're good at their actual job: users need to know what changed, support needs to know what to expect, and the changelog is a real product surface.
Just stop expecting them to answer a question they were never shaped to answer. Ship the notes to your users. Build something else for the person holding the deploy button.
What do you actually read before approving a deploy? I'm collecting answers, and so far "the compare view, quickly" is winning by a lot.
Top comments (0)