How I Put My Agent in CI to Automate Release Notes

I Put goose in CI to Draft Entire’s Weekly Release Notes

When I joined Entire, I noticed my boss spending a chunk of time every week writing detailed release notes, called Dispatches at Entire. It looked like a painful process. Each Dispatch had to cover changes across several repositories, explain why those changes mattered, credit external contributors, and carefully avoid leaking anything that was not public yet.

I offered to take it over. I had solved a similar problem before, so I figured it would be an easy win.

The release notes generator I built at Block

While I was at Block, I built a release notes generator for goose. It ran in GitHub Actions after a release workflow completed, checked out the new tag, compared it against the previous one, and handed goose a recipe to inspect the commit diff.

Goose organized those commits into features, bug fixes, improvements, and documentation. Each entry got a short description and a PR link. The workflow then updated the GitHub release and posted the announcement to Discord, opening a thread if the notes exceeded the message limit.

It was clean and effective, but it solved a very clean problem: one repository, one new release tag, public commit history, and concise output. So when I looked at Entire’s Dispatches, I assumed I could reuse the same playbook. Gather changes, run goose, post the draft. That assumption did not survive contact with reality.

A Dispatch is not a list of commits

A Dispatch spans multiple projects: the Entire CLI, entire.io, EntireDB, external agent integrations, and open source libraries like go-git, go-nuts, git-sync, and ForgeMark. Every project also ships on a different cadence. Some push to main and deploy continuously. Others bundle work into scheduled releases. The CLI maintains separate stable and nightly channels, which means a feature can be available to testers without being part of the latest stable tag.

Then there are feature flags. Finding changes was not the hard part because GitHub APIs handle that easily. The hard part was figuring out which changes were safe to announce. A private repository might contain code for a feature that has already launched publicly, while a merged public PR might still be hidden behind a flag. Repository visibility tells you very little about feature visibility.

The pipeline could not just answer “What changed?” It also needed to answer:

• Has this actually shipped?
• Is it live for everyone or gated behind a flag?
• Is it stable or nightly?
• Does it matter to a reader?
• Did a community contributor help build it?
• Is there anything sensitive here that should not be shared?

A script could pull the raw data, but those decisions required more context.

The first workflow and all its bad assumptions

My first GitHub Actions workflow for Entire was called Release Notes to Slack. It mirrored my setup at Block: monitor repositories for new releases, pull the diffs, and run goose inside the runner with a recipe that inspected tags and formatted notes for Slack. It made one major assumption: a release tag was the main unit of work. That worked for tagged repositories and failed for a weekly Dispatch trying to cover projects with different deployment processes.

The commit history from the first day reads like a real-time record of my assumptions falling apart. First, I tried running the workflow across every repository and aggregating the Slack output. Then I stopped relying only on release tags and used the previous Dispatch publication date as the start of the reporting window so the same work would not appear twice. Next, I added another goose recipe to turn those raw notes into a full Dispatch draft written in Marvin’s voice.

Then came the deployment edge cases. Merged code does not always mean public code. I updated the filtering so drafts only included work that was public, while keeping nightly changes clearly labeled. For entire.io, which uses continuous delivery and feature flags, I added PostHog checks so the workflow could see whether a flag was active before treating a merged PR as public.

Contributor attribution came next, followed by several experiments with the output: Slack messages, thread replies, Markdown tables, generated PDFs, and eventually a return to plain Markdown. The history was just me learning that “writing release notes” was several smaller problems stacked together.

The workflow eventually needed different goose recipes for different types of repositories. Entire.io needed rules for continuous delivery and feature flags. Tagged projects needed rules for stable releases, nightlies, and unreleased merges. Go-git needed specific community attribution rules because it is an external open source project. It worked, but the workflow became difficult to maintain. Data collection, release checks, contributor matching, writing style, and output formatting were all tangled together.

The problem was not running goose in CI. Goose had been running there from the beginning. The problem was deciding what context goose should receive and where each rule belonged.

Building V2 around structured collection

The second iteration, Release Summary to Slack V2, separated data collection from writing. Instead of asking goose to wander through repository directories and figure out what happened, the workflow gathers the facts first.

It starts by finding the date of the last published Dispatch and using that as the beginning of the reporting window. A matrix job then runs across every repository to collect releases, merged PRs, titles, descriptions, labels, authors, links, feature flag definitions, current PostHog status, commit authors, and organization membership data.

The first version of V2 also ran an entire dispatch command for every repository. I later removed that step and kept the factual collection inside the workflow. This made the inputs easier to inspect because goose received the underlying release and PR data instead of a mixture of raw facts and an earlier generated summary.

Before goose runs, the workflow turns those inputs into two files. The first is the Source Brief, which contains the releases, PRs, links, labels, feature flags, and repository metadata goose is allowed to use as facts. The second is the Style Reference, which contains excerpts from the three most recent Dispatches as examples of tone, structure, and length.

The recipe makes the boundary explicit. Previous Dispatches are for style only, so goose cannot pull an old change into the new draft simply because it appeared in an example. The recipe also defines a JSON response schema. Goose must return a payload containing the Markdown draft, and the workflow validates it before extracting the text. If goose returns a conversational preamble, malformed JSON, or an empty response, the build fails instead of posting a broken draft to Slack.

Why the workflow uses both a recipe and a skill

The system now relies on two pieces: a goose recipe and a Dispatch skill. The recipe controls the run. It tells goose which files to read, which sources it can trust, which tools are available, and what format to return.

The skill contains the reusable editorial rules, including which projects belong in each section, how to filter private or flagged work, how to separate nightly features from stable releases, how to explain the value of a change instead of repeating a commit message, and how to capture Marvin’s tone without recycling the same introduction every week.

The way I made that skill is a little recursive. I also created Entire’s session-to-skill skill, which came from a pull request I opened in the Entire skills repository. It searches Entire-tracked sessions and checkpoints for a repeated workflow, then turns the useful parts into instructions another agent can follow. It is not meant to copy a transcript into a Markdown file. It pulls out the decisions, corrections, commands, checks, and preferences that are worth repeating.

I used session-to-skill on the sessions where I wrote and revised previous Dispatches. Those sessions captured the parts that a finished blog post does not show: asking for every bullet to explain the benefit, rejecting intros that sounded corny, correcting project coverage, checking contributor credits, and removing work that was not public. That editing history became the source material for the Dispatch skill, so the workflow now benefits from decisions I had already made while doing the work manually.

The division now looks like this:

• GitHub Actions gathers and organizes the facts.
• The goose recipe controls the drafting run.
• The Dispatch skill contains the editorial and writing rules.

Privacy still needs a person

The workflow can collect evidence, but it should not make the final privacy decision on its own. For entire.io, it checks feature flag states in PostHog. If a flag is disabled, internal-only, or in the middle of a rollout, the recipe tells goose to leave that work out. For other projects, if the workflow cannot confirm that something is public, the safest choice is to omit it and review it manually.

This is why I do not automatically publish the result. An awkward sentence is easy to fix. Accidentally announcing an unreleased feature or crediting the wrong contributor damages trust. The generated Markdown goes to Slack for review first.

The current workflow looks like this:

1. GitHub Actions finds the last published Dispatch date.
2. A matrix job collects activity, feature flags, and contributor metadata across the repositories.
3. The workflow compiles a single Source Brief.
4. It adds the latest Dispatches as Style References and loads the Dispatch skill.
5. Goose runs with the recipe and generates the Markdown draft.
6. The workflow validates the JSON response and uploads the draft to Slack.
7. I review, edit, and publish.

The workflow handles the part that used to take the longest: cross-referencing activity across projects and tracking down context. I can spend that time choosing the headline, refining the narrative, checking anything that might still be private, and cleaning up sentences that sound too much like pull request titles.

What actually got automated

When I built the goose workflow at Block, I thought release note generation was mostly a matter of diffing tags. That works when the boundaries are clear: one repository, public commits, and short output. The Entire Dispatch forced me to handle messier boundaries, including several repositories, different release schedules, feature flags, private work, and a distinct voice.

I did not automate editorial judgment. I automated the research, data collection, recurring checks, and first draft. Before this workflow, writing a Dispatch meant spending hours opening tabs, reading PRs, and reconstructing the week from scratch. Now goose brings the source material together, applies the editorial rules, and gives me a draft to polish.