Ask an agent to make a video from CHANGELOG.md and it will often treat every line as content, including the dependency bump. Nothing looks wrong until the render, where the React upgrade has six seconds of screen time.
I would rather argue about a cut list than about a finished video. So I wrote a small script that sorts a changelog into keep, cut and "a human should look" before the agent sees it. The sorting turned out to be the boring part. The rows the script refuses to decide are the interesting ones.
Why the raw changelog is a bad input
A changelog is written for the team that shipped the release. It records what happened, in the order a maintainer thinks about it, and nothing in the file says which entries a customer cares about.
That leaves an agent two ways to go wrong. The first is coverage: it can treat every line as content. The second is confidence: when a line is thin, a model tends to fill the gap with a stronger headline than the line supports. A fix for one crash turns into "crashes are gone". Both mistakes are cheap to catch in a table and expensive to catch in a render.
The script
It reads a changelog in the Keep a Changelog layout, takes the newest dated release, and prints a verdict for every entry. It uses only the standard library.
#!/usr/bin/env python3
"""Draft a keep/cut table for the newest dated release in a CHANGELOG.md."""
import argparse
import re
import sys
from pathlib import Path
RELEASE = re.compile(r"^##\s+\[?(?P<ver>[^\]\s]+)\]?\s*(?:-\s*(?P<date>\d{4}-\d{2}-\d{2}))?(?P<tail>.*)$")
INTERNAL = re.compile(r"\b(bump|upgrade[sd]?|dependenc\w+|deps?|refactor\w*|lint\w*|chore|typo|tests?|ci|rename[sd]?)\b", re.I)
JARGON = re.compile(r"\b(race condition|deadlock|mutex|queue|worker|thread|pool|cache|index|orm|migration script)\b", re.I)
COMPARATIVE = re.compile(r"\b(faster|quicker|smaller|lighter|snappier|speed(s|ed)? up|reduced?)\b", re.I)
FIGURE = re.compile(r"\d[\d.,]*\s?(%|x\b|ms\b|(seconds?|secs?|s)\b|(minutes?|mins?)\b|[KMG]B\b|times\b)|\b(half|twice|double|triple)\b", re.I)
def newest_release(lines):
"""Return (version, date, {group: [entries]}) for the first dated, non-yanked release."""
release, group, found = None, None, []
for line in lines:
m = RELEASE.match(line)
if m:
release, group = m, None
if m["date"] and "yanked" not in m["tail"].lower():
found.append((m["ver"], m["date"], {}))
continue
if line.startswith("### "):
group = line[4:].strip()
elif re.match(r"^[-*]\s+", line) and release and found and release["date"] \
and "yanked" not in release["tail"].lower() and group:
found[-1][2].setdefault(group, []).append(re.sub(r"^[-*]\s+", "", line).strip())
return found[0] if found else None
def triage(group, entry):
if group == "Security":
return "ASK", "a person decides if this goes on screen"
if INTERNAL.search(entry):
return "CUT", "reads as internal work"
if len(entry.split()) < 4:
return "REWRITE", "too vague to turn into a headline"
if JARGON.search(entry):
return "REWRITE", "names a component customers never see"
if group in ("Deprecated", "Removed"):
return "SCENE", "customers lose something, say it plainly"
if COMPARATIVE.search(entry) and not FIGURE.search(entry):
return "NEEDS NUMBER", "comparative claim with no figure in the notes"
return "SCENE", ""
def main():
ap = argparse.ArgumentParser()
ap.add_argument("path")
ap.add_argument("--title", type=int, default=3, help="title card seconds")
ap.add_argument("--per", type=int, default=6, help="seconds per scene")
ap.add_argument("--cap", type=int, default=40, help="runtime cap in seconds")
ap.add_argument("--strict", action="store_true", help="exit 1 if ASK, REWRITE or NEEDS NUMBER rows remain")
args = ap.parse_args()
rel = newest_release(Path(args.path).read_text().splitlines())
if not rel:
sys.exit("No dated, non-yanked release found.")
ver, date, groups = rel
print(f"Release {ver} ({date})\n")
print("| Group | Entry | Verdict | Why |")
print("|---|---|---|---|")
scenes, open_rows = 0, 0
for group, entries in groups.items():
for e in entries:
verdict, why = triage(group, e)
scenes += verdict == "SCENE"
open_rows += verdict in ("ASK", "REWRITE", "NEEDS NUMBER")
print(f"| {group} | {e} | {verdict} | {why} |")
runtime = args.title + scenes * args.per
flag = " <- over the cap" if runtime > args.cap else ""
print(f"\n{scenes} scenes, about {runtime}s of {args.cap}s{flag}")
if args.strict and open_rows:
sys.exit(1)
if __name__ == "__main__":
main()
The rules are deliberately dumb:
The rules are deliberately dumb:
-
Skips what shouldn't be read. It ignores
[Unreleased], because those lines haven't shipped. It also ignores any release marked[YANKED], because that release was pulled. - CUT. Words like bump, upgrade, refactor, lint and chore suggest internal work.
- REWRITE. An entry under four words is too thin for a headline. An entry that names a queue, a worker or a deadlock describes something customers never see.
- NEEDS NUMBER. Words like faster, quicker or smaller with no figure in the entry. A figure means a number next to a unit, a percent or a multiplier, or a word like "twice".
- ASK. Anything under Security goes to a person, every time.
- The runtime budget. A title card plus six seconds per scene, checked against a cap. This catches the "we have eight features and 40 seconds" problem before any scene exists.
A test run on an invented changelog
Here is the file I tested with. The app, Fieldnote, is made up so there is no real customer data in it.
## [Unreleased]
### Added
- Shared folders (behind a feature flag).
## [2.8.0] - 2026-10-14
### Added
- Offline mode for iOS.
- Tag filters in search.
### Changed
- Upgrade React to 19.1.
- Search results open faster on large libraries.
### Removed
- Dropped the legacy Markdown importer.
### Fixed
- Fix race condition in sync queue.
- Fix crash when opening notes over 10,000 characters.
- Misc fixes.
### Security
- Sanitize HTML in the note preview.
## [2.7.1] - 2026-09-30 [YANKED]
### Fixed
- Fix login redirect.
And what python changelog_triage.py CHANGELOG.md printed:
| Group | Entry | Verdict | Why |
|---|---|---|---|
| Added | Offline mode for iOS. | SCENE | |
| Added | Tag filters in search. | SCENE | |
| Changed | Upgrade React to 19.1. | CUT | reads as internal work |
| Changed | Search results open faster on large libraries. | NEEDS NUMBER | comparative claim with no figure in the notes |
| Removed | Dropped the legacy Markdown importer. | SCENE | customers lose something, say it plainly |
| Fixed | Fix race condition in sync queue. | REWRITE | names a component customers never see |
| Fixed | Fix crash when opening notes over 10,000 characters. | SCENE | |
| Fixed | Misc fixes. | REWRITE | too vague to turn into a headline |
| Security | Sanitize HTML in the note preview. | ASK | a person decides if this goes on screen |
The footer reads 4 scenes, about 27s of 40s. The shared-folders flag and the yanked 2.7.1 never appear, which is the point.
Two rows deserve a second look. "Search results open faster on large libraries" is the one a reader caught. My first version let it through as a SCENE, but a speed claim invites a number, and the note doesn't contain one. If an agent writes "twice as fast" on that scene, it made the number up. The script now flags it, and the fix on screen is a side-by-side recording of the slow path and the fast one, not a figure read aloud.
The "Misc fixes" row is the other. No script can fix it. Someone who knows what shipped has to write the real line, and that is the whole argument for doing this triage before the agent runs.
Hand the table to Claude Code
Setup is the boring part and the docs cover it better than I would. In the CoAnimator app, Settings, General, Command line and agent skill has an install button for each, and the CLI docs cover the command. Claude Code comes from Anthropic's setup guide. If you want background on what the agent can do inside a project, this post covers it.
Then I would send the dated section, the table, and a brief like this:
Here is a triage table for Fieldnote 2.8.0 and the dated changelog section it came from. Treat the table as a proposal. For each row, confirm or overrule the verdict and give one sentence of reasoning. For REWRITE rows, propose a customer-facing line and quote the source words it came from. For NEEDS NUMBER rows, do not describe the improvement in words. Ask me for a before and after recording to show it. Do not state any number, speed or percentage that is not in the notes. Build nothing until I reply.
Three instructions do most of the work. "Quote the source words" forces every headline to point at its entry. The NEEDS NUMBER rule turns a claim nobody can verify into a shot someone can record. And "build nothing until I reply" turns the first answer into something you can correct for the price of one sentence. Each SCENE row also needs a screenshot or recording to attach, or the scene will be text on a background.
What I still check by hand
Once the build is approved, the render step is a single coa render <project>. For a quick look at the title card, --start 0 --end 6 renders only the first six seconds. The script can't check the rest, so I go through these:
- Headline strength. "Large notes no longer crash the app" matches the crash entry. "Crashes are gone" doesn't.
-
Version and date against the tag. The card can say 2.8.0 while the repo's latest tag is v2.8.1 after a hotfix.
git describe --tags --abbrev=0prints the latest tag in one second. - Text outside the scenes. Closing cards and subtitles never passed through the table, and that is where stock claims like "faster than ever" show up.
Gate it in CI
--strict makes the script exit 1 while any ASK, REWRITE or NEEDS NUMBER row is unresolved. Fix a REWRITE or NEEDS NUMBER row by editing the entry in the changelog, and the next run passes. I tested the exit code locally, and I haven't wired it into a release pipeline yet. The step itself is two lines:
- name: Changelog triage
run: python changelog_triage.py CHANGELOG.md --strict
Any Security entry fails strict mode on purpose, so a person has to look at it. Once someone has, run that release without --strict.
The table prints to the job log, so whoever reviews the failed run sees exactly which rows need a person.
Where the heuristics break
It matches words, not meaning, and it will be wrong in both directions:
- "Add CI status badge to dashboard" gets CUT because of "CI", though customers see that badge.
- "Upgrade your plan from the app" gets CUT because of "upgrade".
- "Add index of recent files" gets REWRITE because of "index".
- "Add reduce motion option in settings" gets NEEDS NUMBER because of "reduce", though it names an accessibility setting, not a claim.
- It only checks that a figure exists, not that it's true. "40% faster" passes even if nobody measured 40%.
- A claim worded without a trigger word, like "no more waiting", gets through.
That is why the script proposes and the agent and I decide. It also can't tell whether a change is visible to a customer, because it only sees the words.
One privacy point. The script runs locally, but whatever you paste into an agent can leave your machine for its provider. CoAnimator's quickstart says the same about files and frames sent to an AI provider. Delete embargoed rows from the table before pasting, and crop customer data out of screenshots.
FAQ
Does it work with changelogs that aren't in Keep a Changelog format?
It needs ## [version] - date headings and ### groups. GitHub's generated release notes have neither, so the parser would need a small change.
Does the script replace reading the notes?
No. It's a first draft of the cut list so the conversation with the agent starts from something concrete.
Do I need CoAnimator for this?
No. The table works with any agent or video workflow. I use CoAnimator for the render step.
Next step: let the PR decide
Regex is a fallback. The better signal is the person who wrote the change. A customer-facing label on the pull request, or a one-line release-note field in the PR template, would let the table come from labels and leave the regex to catch whatever nobody labeled. Then the triage happens at merge time, when the author still remembers why the change matters.
A second check: compare each SCENE row with the files its change touched, and flag rows where no UI files moved. Those rows tend to turn into a title card over an unchanged screen, which is where made-up claims creep in. The right unit is the release tag, not the single PR, because a feature often lands across several PRs and only the last one touches the UI. My sample entries carry no PR numbers yet, so the sketch starts there: add them to the end of each entry, take the union of files across a row's PRs with gh pr view <n> --json files, and flag the row only if none of them touched a UI path. git diff --name-only <prev-tag>..<tag> is the cheap release-level version: if no UI file moved at all, every visual row is suspect.
Try the script on your last release and see which rows you disagree with. Those are the decisions your changelog never recorded. If you want to turn the approved rows into a video, the CoAnimator desktop app is where I'd render them or you can use your existing video workflow.
Top comments (5)
The point about the speed claim is the one I'd build into the script. Any SCENE row with words like faster, quicker or smaller could get flagged as NEEDS NUMBER unless the entry itself has one, so the brief asks the agent to show the change instead of describing it. Another check that might help: compare each SCENE row against the files changed in that release, and flag rows where no UI files moved. Those usually end up as a title card over an unchanged screen, which is exactly where made-up claims creep in.
Good catch, and I ran the first one. Any SCENE row with faster, quicker or smaller and no figure now flips to NEEDS NUMBER. My first attempt counted any digit as a figure, so "Faster sync on iOS 17" slipped through. Requiring a number next to a unit fixed it:
I agree the row shouldn't just ask for a number. It should ask for a side-by-side recording of the slow path and the fast one.
The files-changed check is the one I haven't built. Changelog lines don't carry paths, so per row it needs the PR number in the entry, then
gh pr view <n> --json fileschecked against your UI folders. A SCENE row with no UI files behind it is exactly the title-card-over-an-unchanged-screen case. Do you run that comparison per PR or per release tag?Nice, thanks for running it. Did it flag many rows? The second pass, checking every number in a scene against the release note, tends to catch more than the wording check.
To your question, per release tag makes more sense to me. Run git diff --name-only between the previous tag and the new one, then map each changelog row to the PRs merged in that range. Per PR gets noisy because one feature often lands across a few PRs and only the last one touches the UI. Are your changelog entries one per PR, or grouped by feature?
Per release tag makes more sense, you're right. Per PR would be noisy, since only the last PR of a feature usually touches the UI. My sample entries are written per change and carry no PR numbers yet, so I'd add them to each entry and check the files together. I'm adding this to the post. Thanks for the push!