The team I work on at Google manages 300+ repositories, across
7 languages, developed in the open on GitHub. We release new library versions to package managers, such as Maven, npm, and PyPi, 100s of times each month:
Releasing a library isn't simply a matter of publishing code to a package manager. Most of our libraries use SemVer for versioning. SemVer formalizes the concept that the MAJOR, MINOR, and PATCH in versions ([MAJOR].[MINOR].[PATCH]) should be used to communicate the nature changes to your user:
- Increment the
MAJORversion when you make incompatible API changes.- Increment the
MINORversion when you add functionality in a backwards compatible manner.- Increment the
PATCHversion when you make backwards compatible bug fixes.
— semver.org.
It's important that we increment the correct MAJOR, MINOR, or PATCH, when releasing a new library version. However, a version number alone doesn't provide enough information when upgrading. A changelog (a human readable record of software changes, usually stored in ./CHANGELOG.md) is important because it communicates to the user:
- What new features they can look forward to.
- What bugs we've fixed that might have been affecting them.
- What breaking changes they should be careful of when updating.
Who needs a changelog? People do. Whether consumers or developers, the end users of software are human beings who care about what's in the software. When the software changes, people want to know why and how.
Humans aren't great at this
Manually choosing a new version number when releasing is error prone. Will you know that your peer introduced a small breaking change two days prior?
Manually writing release notes is time consuming. I found myself spending 20 minutes per-release (for those doing math, this would have meant we spent 130 hours writing release notes in May). It's also error prone... did we call attention to the appropriate features, did we document all the breaking changes?
Competing goals
We found ourselves with two potentially competing goals:
- We cared deeply about following SemVer, and providing an actionable, human-readable, changelog.
- If we didn't automate our release process, we'd quickly find that it was taking up 100% of our time.
The remainder of this post introduces a convention we've adopted, and a tool we've written (and open sourced 🎉), to reconcile the goal of creating releases that are both meaningful and automated.
Adopting commit conventions
Commit messages provide important context for both other collaborators and the users of your library. I believe this, but when I look back at various projects the commits are bit of a hodgepodge:
What update was made to the changelog? why was I a terrible person? what does #148 fix?
There must be a better way!
AngularJS Git Commit Message Conventions
In 2011 Igor Minar and Vojta Jína, while working on the Angular project, had the brilliant idea of introducing a lightweight convention on top of commit messages. Their original design document describes the following goals:
- Allow generating CHANGELOG.md with a script.
- Allow ignoring unimportant commits, when using git bisect.
- Provide better information when browsing the history.
They go on to propose the format:
<type>(<scope>): <subject>
<BLANK LINE>
<body>
<BLANK LINE>
<footer>
-
type: represents the type of change that occurred, valid examples being
ci:,feat:,fix:,perf:,build:. -
scope: is optional, and represents the part of the system that changed, examples of supported scopes include
localize,http,router,forms. - subject: is a description of the change.
-
body: is a longer form description of the change. It can also include the token
BREAKING CHANGE, to indicate a breaking API change.
A commit message ends up looking something like this:
fix(http): addressed bug with http module
There was a bug with the http module, it has now been addressed.
These Angular commit guidelines inspired Conventional Commits, which is the commit convention my team adopted.
Conventional Commits
The Conventional Commits spec proposes a superset of Angular's guidelines:
- The
typecan be anything a contributor likes, butfeat:andfix:have semantic meaning:fix:patches a bug in your codebase,feat:indicates a new feature was added. - Similarly, there are no restrictions on the
scope. - Conventional Commits introduces the
!shorthand, for indicating breaking changes.
refactor(http)!: removed deprecated method start()
The goals of the Conventional Commits Specification were to:
- Emphasize that these commit conventions, pioneered by Angular, were widely applicable (they need not just be used by JavaScript folks).
- Abstract out the rules so that they're applicable to arbitrary projects (underscoring how easy they are to adopt).
- Provide a formal spec that tooling authors could build parsers for.
Conventional Commits felt like the perfect choice for my goal of getting folks across 6 language teams to adopt a convention — I appreciated the slightly trimmed down rules.
Automating the release process
Once repos began adopting Conventional Commits, we were able to start automating parts of our release process that had been manual, i.e., generating a CHANGELOG.md, choosing the next library version, publishing to a package registry.
The process was gradual. Rather than forcing teams to use commit conventions, I thought that it would be better to demonstrate their value. We started by automating our JavaScript release process. As I hoped, other teams were quick to follow as they saw the time it was saving. Now, a year later, we support JavaScript, Python, Java, Ruby, PHP, Terraform, and are beginning work on Go.
In parallel with adopting conventions, we developed a tool called release-please. We've made release-please extensible, so each language team is able to customize the their release process:
- Some teams wanted to call out different types of changes in their CHANGELOG.md.
- Some teams use mono-repos, whereas other teams have a repository per-library.
- Some teams release pre-
1.x.xversions of their libraries while in beta.
Introducing release-please
release-please is the tool that grew out of my team's automating the release process for 6 languages.
It does so by parsing your git history, looking for Conventional Commit messages, and creating release PRs.
What are release PRs? Our existing release processes didn't map well to continuously releasing changes as they land on a branch:
- There will sometimes be a set release date for a library feature, even though it's ready to go on GitHub.
- We try to bump major versions rarely, so we will sometimes wait for a few breaking changes to land, before promoting a release.
- Sometimes (rarely) there will be some amount of manual QA before promoting a library release.
I’ve seen 100% code coverage deployments fail spectacularly. I am in favor of “push the big red button” releases.
— Dave
release-please creates release PRs automatically. They represent the set of changes that would be present if you were to release the library:
If a release PR is merged, release-please will create a release of your library, matching the description in the release PR.
Conclusion
While embracing the differences in release workflows between language communities, we've had a great experience adopting consistent commit conventions and tooling across our team.
We've done so without sacrificing an actionable CHANGELOG.md, and while successfully adhering to SemVer.
Implementing this process was an awesome experience, and I hope other teams are inspired by our success.
Links
- The original Angular Commit Convention design doc: written in 2011.
- conventionalcommits.org: a commit specification, inspired by Angular's.
- release-please: the release-please tool.
- release-please-action: run release please as a GitHub Action.
- semantic-release: automate releases, based on Angular conventions, when changes are pushed to a branch.
Top comments (6)
Very interesting post! I am trying evangalize this kind of mindset of library development at Roche as well. So seeing this done similarly by Google definitely gives assurance.
May I ask, why did you create a new tool opposed to semantic-release which seems to be similar?
The main differences are:
The impression I get is that semantic-release is a great library, especially if you're releasing JavaScript, and are open to continuously releasing from a branch. It's also customizable, so I think it would have been an alternate route we could have opted for (but we'd have needed to customize it significantly).
I see.. I think semantic release has very nice concepts, but its heavy focus on NPM packages has felt like lost potential. The same concepts can very well be applied to other languages, which you have implemented with release-please quite nicely!
Thanks for sharing, looking forward to see release-please grow!
I was reading through the docs for semantic-release thoroughly, in prep for this post. I do think it's customizable enough, that you could flesh out something similar to release-please on it. It would just take folks from other language communities building it out.
I went this route partially because I knew I'd have lots of edge-case requests from my own stakeholders (I would have been a nuisance on semantic-release's issue tracker
😂).
Thank you for posting this!
At uman we're currently using the GitHub built-in (pre)release functionality to release new features on a (bi-) weekly basis. Currently this approach has been working out quite well for us, but I do wonder if we (as a product-focused engineering team) would benefit from migrating our release process to use release-please? We tend to prerelease code to an acceptance environment, is that something that is possible with release-please as well?
Great write! Thank you.
I am working on my blog and looking forward to use release-please.