I built a local-first linter for vague release notes

duanjianming1981-del — Thu, 20 Aug 2026 07:34:24 +0000

Release notes often describe a change with phrases such as “improved
performance” or “fixed several issues.” Those phrases sound finished, but they
do not tell a reader what changed, who is affected, or whether any action is
required.

I built ReleaseProof to check
that narrow problem.

It takes two pieces of text:

Change evidence, such as PR titles, issue summaries, test results, or a short diff summary.
The release-note draft intended for users.

The checker then looks for vague claims, missing audiences, missing migration
steps, a lack of concrete technical or user-facing anchors, and claims that are
not clearly supported by the supplied evidence.

For example, given evidence that a dashboard query count fell from 18 to 6, a
draft saying only “Improved dashboard performance” should be challenged. A
better note would identify the dashboard, the measurable change, and the users
who notice it.

ReleaseProof is deliberately a linter rather than a ghostwriter. It does not
generate a replacement changelog or invent details. Its suggestions are narrow
prompts about missing facts, leaving the author responsible for what is true.

Privacy and scope

The current version requires no account and never accesses a repository.
Analysis runs in the browser, so pasted text does not leave the page.

It also does not claim to verify that code is correct, secure, or ready for
production. It only checks the specificity of the writing and its alignment
with the evidence supplied by the user.

How I calibrated it

The deterministic rules currently pass 20 focused cases covering vague fixes,
performance claims, breaking changes, migrations, fields, routes, and missing
evidence.

I then added five real release-note examples from uv, Ruff, pnpm, and Bun. That
second pass exposed several problems in the initial rules: technical identifiers
were not always recognized as concrete details, developer-facing notes were
being forced to name an audience, evidence matching relied too much on literal
word overlap, and a link-only release could score too highly.

Those cases are now regression tests. Four content-bearing release notes score
as specific, while the link-only example is flagged for failing to state an
actual shipped change.

What I want to learn

The rule suite can show that the checker behaves as designed, but it cannot show
whether the tool is useful in real work. I am looking for a small number of
people who write release notes, changelogs, or user-facing PR summaries to try
one sanitized example.

If you test it, I would especially like to know:

Which suggestion was useful?
What was a false positive or missed problem?
Did you revise the draft?
Would you use the checker again?

You can try ReleaseProof without
signing up and submit structured feedback through the link beside the result.

Disclosure: This post was drafted with AI assistance and must be reviewed by the
project owner before publication.

DEV Community: duanjianming1981-del

I built a local-first linter for vague release notes

Privacy and scope

How I calibrated it

What I want to learn