Reading Time: 9 minutes
H ow many times have you struggled with collecting the changes in a release to post them as release notes or include them in the changelog? How many times did you check every pull request (merge request) to check the work added? How many times you were unable to describe the work done by other team members to include it in the release notes? If you suffered from these problems (or at least one of them). Then, this post is for you! Even if you did not face any of this and you are a lazy developer (like me) or work in a small team. Then, check this post it might make you relax after every release! 🙂
I am kind of new here, what is the changelog or release note anyway?
According to Wikipedia
A changelog is a log or record of all notable changes made to a project. The project is often a website or software project, and the changelog usually includes records of changes such as bug fixes, new features, etc. Some open-source projects include a changelog as one of the top-level files in their distribution
wikipedia
In other words, it’s an announcement to the users of your application about the changes that happened to the software after the deployment of a new version. It’s really important to let the users know the changes that happened to the software to manage their expectations and also inform them about the new features added.
The purpose of maintaining a changelog or release notes documents is not only for end users. but also, the developers themselves can use it as a reference for all work done in a certain release and can be used also to inform other teams about the technical changes that happened. So they can react to these changes (Ex: integrate with a new API, or a bug fixed that was affecting them).
Ok, enough talking about the definition and the theoretical stuff, and let’s start the action 🙂
You asked for it, here you are the entire release automation flow 🙂 Don’t panic, I will explain each step in detail and then come back to this automation flow!
The Steps needed to automate the release notes/changelog.
These are the challenges that we will face to fully automate the release notes:
- Finding a way to collect the changes added.
- Determining the changes that happened in the scope of the release.
- Parsing the changes and formatting them in a human-friendly format.
- Tagging the release with a unique name or version.
- Posting the release notes to the technical and business communication channels.
1. Finding a way to collect the changes added
This is the first challenge, how can we collect the changes that happened? The answer is simple Conventional Commits!
The guys who are maintaining Angular used an expressive and strict way to describe each commit added to the git log by following a standard format. This format is simple and machine-readable. You can easily extract the information using a simple Regex. This way inspired other people to extract this guideline into a full-blown specification to make it easier for other people to standardize the commit message.
Conventional Commits
The specification is pretty simple, you have to follow the following rule while writing the commit message
<type><type>[optional scope][optional !]<description>: <description>
[optional body]
[optional footer(s)]</description></type>
The message consists of a header, [optional] body, and [optional] footer. The header line contains all the important information as the following:
<type> is the type of the commit, it can be a feat, a bug-fix or any other type you invent. The Angular team put a list of these types. but do not let this list hinders you from adding more types if it’s needed! as this is what the conventional commits specification recommends.
- build : Changes that affect the build system or external dependencies (example scopes: gulp, broccoli, npm)
- ci : Changes to our CI configuration files and scripts (example scopes: Travis, Circle, BrowserStack, SauceLabs)
- docs : Documentation only changes
- feat : A new feature
- fix : A bug fix
- perf : A code change that improves performance
- refactor : A code change that neither fixes a bug nor adds a feature
- style : Changes that do not affect the meaning of the code (white space, formatting, missing semi-colons, etc)
- test : Adding missing tests or correcting existing tests
The most important types are feat and fix, why? you will know the answer when we talk about versioning the release! be patient !🙂
The second part in the header line is the [optional scope] and it is used to define which module/part of the application is changed. You can utilize this part if you have a big application and you want to highlight the affected parts.
The third part is [optinal !] and it is very handy shortcut! to indicate that you have introduced a breaking change with your commit. We mean by breaking change, the change which is not backward compatible and the users have to adapt to this change. This also will affect the way we version the release as we will explain later.
The fourth part (which is mandatory) is a <description> of the work done. This should be a clear and concise description.
After this, you can add more details to the body and also add a footer to highlight the breaking change. You can utilize Git trailer to interpret it if needed.
2. Determining the changes that happened in the scope of the release.
The second challenge is to determine the changes that will be included in the release notes. This can be done using Git Tags, with each commit to the main branch (master) you can associate a tag to mark this commit as a release. you can also make it a meaningful tag by assigning a version to the release. But How will this make it easy to determine the changes that happened in the scope of the release? Well, let’s explain this.
First of all, you will tag the current HEAD of your main branch with an initial tag, let’s assume it’s v1.0 for simplification. then you have a new release pushed to the main branch, so you have to add another tag, let’s say v2.0 , you can utilize Git to list the commits between these 2 tags using the following commands:
last_tag=`git describe --abbrev=0 --tags`
second_to_last_tag=`git describe --abbrev=0 --tags $last_tag^`
merge_commits=`git log "${second_to_last_tag}...${last_tag}" --merges --pretty=format:'%s'
NOTE: if you want to list all commits, remove --merges option from the last command. Here we are interested only in merge commits since they contain the formatted conventional commits message
3. Parsing the changes and formatting them in a human-friendly format.
Ok, Now we have all the commits which are formatted according to the conventional commit specification. What to do next? you can guess we need to parse them to create a formatted release notes!
Here we need to write some code, This is a simple script that receives the commits and creates a markdown version. It’s written in Perl but of course, you can use also shell script for that.
# release-pr-gen.pl
! /usr/bin/perl
use strict;
use warnings;
exit if not defined $ARGV[0];
my $minor = my $major = '';
my %release_note_sections = (
'feat' => ' ⭐ Features',
'fix' => ' 🔨 Bug Fixes',
'docs' => ' 📚 Documentation',
'perf' => ' 🚀 Performance Enhancements',
'test' => ' 🩺 Test Coverage',
'refactor' => ' ⛑ Code Refactor',
'infra' => ' 🏗 Infrastructure',
'pkg' => ' 📦 Package Updates'
);
my %release_notes;
my $release_content = '';
It should receive a string as an output from (git log 'origin/main…origin/develop' --merges --pretty=format:'%s') or similar
my $merge_commits = $ARGV[0];
my @merge_commits_arr = split "\n", $merge_commits;
for(@merge_commits_arr){
if (/^(\w_)(?:(([\w\$.-\*\_\s]_)))?(!)?: _(._)$/i) {
# $1 -> work type (feat, fix, ..etc)
# $2 -> work scope (optional) (aka module name as Company, Hubspot, ..etc)
# $3 -> Breaking Change (optional) if you append (!) to any work type
# $4 -> work desc + the pull-request#id
# Determine the release type based on SemVer spec
$major ||= "true" if defined $3;
$minor ||= "true" if ($1 eq "feat");
# prepare the release notes
$release_notes{$1} = '' if not exists $release_notes{$1};
if (defined $2){
$release_notes{$1} .= '- **'.$2.'** : '.$4;
} else {
$release_notes{$1} .= '- '.$4;
}
$release_notes{$1} .= ' ~ ⚠ _BREAKING CHANGE!_' if defined $3; $release_notes{$1} .= " \n";
}
}
#---------------- Build Release Content -----------------
foreach my $key (sort keys %release_notes) {
if (exists $release_note_sections{$key}){
$release_content .= '## '.$release_note_sections{$key}." \n".$release_notes{$key};
} else {
$release_content .= '## '.$key." \n".$release_notes{$key};
}
}
# Print the release notes to STDOUT
print $release_content;
# Export the release type if needed to a file called ".release_type"
# The script must receive "with_release_type" as a 2nd argument to do the export
if (defined $ARGV[1] && $ARGV[1] eq "with_release_type") {
open(FH, '>', '.release_type') or die $!;
if ($major) {
print FH 'major';
} elsif ($minor) {
print FH 'minor';
} else {
print FH 'patch';
}
close(FH);
}
This should generate a markdown formatted changelog like the following:
📚 Documentation
- readme : Add some description to README file #3
⭐ Features
- New file added #1
🔨 Bug Fixies
- New file fixed #2
🏗 Infrastructure
- K3s specs #7
🚀 Performance Enhancements
- Performance enhanced by 2x. This is the fastest app we ever released #6
📦 Package Updates
- Update package #5
⛑ Code Refactor
- Rename new file to test_file #4 ~ ⚠ BREAKING CHANGE!
4. Tagging the release with a unique name or version
The next challenge is how to version the release, we should have a standard mechanism for naming or versioning the release. Ok, nothing prevents you from using a date tag to indicate the date of the release something like 20220220 will perfectly work. However, there are more consistent ways to versioning a release like using a versioning standard, as CalendarVersioning or SemanticVersioning. Both are widely adopted in a lot of public and private projects.
I will go with SemVer , but we need to understand the way SemVer is working first.
Semantic Versioning
The idea behind semantic versioning is fairly simple. Each release has a type of these types (patch, minor, major) and this type is determined by the work added according to the following:
-
MAJOR: If it includes a breaking change introduced. -
MINOR: If a new feature is added but it’s backward compatible. -
PATCH: If a fix or any other type is added but it’s backward compatible.
Additional labels for pre-release and build metadata are available as extensions to the MAJOR.MINOR.PATCH format.
So, the challenge here is we have to determine the type of the release while generating the release notes! and this is simple because we can check the type of the work and based on that determine the type of the release. If you checked the above script, you will find this part is responsible for determining the release type.
# Part of release-pr-gen.pl
# Determine the release type based on SemVer spec
$major ||= "true" if defined $3;
$minor ||= "true" if ($1 eq "feat");
# The script must receive "with_release_type" as a 2nd argument to do the export
if (defined $ARGV[1] && $ARGV[1] eq "with_release_type") {
open(FH, '>', '.release_type') or die $!;
if ($major) {
print FH 'major';
} elsif ($minor) {
print FH 'minor';
} else {
print FH 'patch';
}
close(FH);
}
So, calling the above script and passing the 2nd argument will save the release type in a file to be used later.
before I forget, you call the above as the following,
perl ./release-pr-gen.pl "$merge_commits" "with_release_type" > .release_notes
Now we have the release type, what should we do after?! We need to increment the version number based on that. So, if the current version is 1.0.0 and the release type is patch then the version should be 1.0.1. However if the release type minor. It should be 1.1.0 and if it’s major then the version should be 2.0.0. This can be achieved using the following Perl script, you just need to pass the last version number and the type of the release and the script will do the math for you 🙂
# release_ver.pl
! /usr/bin/perl
use strict;
use warnings;
exit if (not defined $ARGV[0] || not defined $ARGV[1]);
exit if (not $ARGV[1] =~ /^(major|minor|patch)$/) ;
my $next_version = my $major = my $minor = my $patch;
if ($ARGV[0] =~ /^v?(?0|[1-9]\d_).(?0|[1-9]\d_).(?0|[1-9]\d*)$/) {
$major = $+{major};
$minor = $+{minor};
$patch = $+{patch};
$next_version = ++$major.".0.0" if ($ARGV[1] eq "major");
$next_version = $major.".".++$minor.".0" if ($ARGV[1] eq "minor");
$next_version = $major.".".$minor.".".++$patch if ($ARGV[1] eq "patch");
}
print "v".$next_version
Now you have the release notes in a readable format and the version number, what’s next? Obviously, we need to communicate this information to the concerned people.
5. Posting the release notes to the technical and business communication channels.
This step depends on your flow, you can add the release notes to changelog.md file, create a release on GitHub, or post it to Slack. Let’s assume you will do the last two things! then you need to add the following to your CI/CD pipeline after deploying the release:
last_tag=`git describe --abbrev=0 --tags`
second_to_last_tag=`git describe --abbrev=0 --tags $last_tag^`
merge_commits=`git log "${second_to_last_tag}...${last_tag}" --merges --pretty=format:'%s'
perl ./release-pr-gen.pl "$merge_commits" "with_release_type" > .release_notes
release_notes=`cat .release_notes`
# You need to install/configure GitHub CLI, if you are using GitHub actions, it's already installed.
# Alternatively, you can use GitHub API.
gh release create "$last_tag" --title "$last_tag" \
--target "master" --notes "$release_notes"
# Post to a Slack channel
curl -H "Authorization: Bearer ${{ secrets.SLACK_TOKEN }}" \
-F filetype="post" \
-F content="$release_notes" \
-F title="$last_tag" \
-F channels="YOUR_CHANNEL" \
-F initial_comment="New Release is pushed to production :rocket:" \
"https://slack.com/api/files.upload"
Connecting all the dots to draw the big picture!
Back to this flow:
This is our release notes pipeline, It depends on several tools and is customized to incorporate all the toolchains we use to do all the mentioned steps in this article. The steps are as the following
- The developer opens a release branch against the master.
- The release branch is approved and merged to master. this will trigger two workflows as the following
- GitHub workflow to tag the master HEAD with the new version number.
- CircleCI workflow to test, build the docker image and push it to AWS Elastic Container Registry. The image is tagged with the new SemVer number.
- ArgoCD ImageUpdater deploys the newly pushed image to Kubernetes.
- ArgoCD sends a notification once the deployment is done to GitHub actions.
- The notification triggers a GitHub workflow which:
- Creates a new release on GitHub based on the last tag and adds the release notes
- Post to our Slack changelog Channel, so people know the changes introduced in this release!
In the end, I hope the flow is clear to you now, for convenience I added the scripts mentioned in this article as GitHub gists, you can find them here.
The post Keep calm and automate Release Notes! appeared first on Islam Askar Blog.
Top comments (0)