DEV Community: Jacky Efendi

Writing Your Own Changelog Generator with Git

Jacky Efendi — Sun, 20 Oct 2019 10:09:44 +0000

Turn your commit messages into release notes for your users

Changelogs are great. They give us a record of what changes in our project were made at a particular time. If you’re writing a library, it also gives your users awareness of what changed and gives the impression that your library is well-maintained and can be relied on. Tools such as conventional-changelog allow us to generate a very neat changelog easily, provided we adhere to their conventions on how to write commit messages. One of the most popular ones is Conventional Commits.

Example of a generated changelog in Angular’s repo

Under the hood, these tools use Git to do this. In this article, we’ll go through some steps to write our own fully-functional changelog generator!

What is Git?

Git is really powerful software. It’s a tool for collaborative software development. It’s used by almost every developer in the world! Even though I also use Git all the time, I’d be the first to admit that I don’t really know a whole lot about Git. I can do the basic stuff and understand some of the concepts, enough for me to be productive. But for the more advanced things you can actually do with Git? I have a long, long way to go. Seeing tweets like this makes me feel better about myself because it means that other developers are also discovering new stuff about Git.

Jason Lengstorf but spoopier 👻

@jlengstorf

Cool thing I just learned: if you want to see only the Git commits in a certain folder of a repo, you can do that!

git log --oneline folder-name

(Or leave out `--oneline` to see the full commit log.)

17:27 PM - 15 Jul 2019

15 79

Setting Up Our Repository

Let’s create a directory, and initialize it as a Git repository.

mkdir changelog-generator
cd changelog-generator
git init

In it, we’ll create a simple package.json file that only contains the version field. Think of this as a very simple JavaScript project repository. To make it simple, let’s just set "1" as the version.

{
  "version": "1"
}

After that, let’s create a CHANGELOG.md file. Just leave it empty for now.

This should be how our project looks; we’ll commit this for now. In a conventional commit, that each commit message is required to be prefixed with one of the available prefixes like feat_,_ fix, perf, refactor, chore_,_ etc. For our repository, let’s say we have to use two prefixes, chore and feature. For our initial commit, let’s use the following command:

git add .
git commit -m "chore: Initial commit for changelog generator"

Now, just so we can have two commits in our repo, let’s create a file named index.js. Leave it empty; we’ll write code there later. Do another commit, and for this one, set feature: Added index.js script as the commit message.

Writing the Changelog Generator

Now we’re getting to the main part! If we type git log into our terminal, we’ll see git provides us with a list of the commits we have made so far in the repo.

commit 1c3ec7c03f2796790eaf7271ef47b2141b22cb63 (HEAD -> master)
Author: Jacky Efendi <[not-a-real@email.com](mailto:not-a-real@email.com)>
Date: Sun Oct 20 14:39:42 2019 +0700

feature: Added index.js script

commit 05fbe5c5eee29cc33065474800e5401370e7e929
Author: Jacky Efendi <[not-a-real@email.com](mailto:not-a-real@email.com)>
Date: Sun Oct 20 14:39:37 2019 +0700

chore: Initial commit for changelog generator

Whoa, that’s a lot of stuff! Remember, we want to create a changelog that is similar to the CHANGELOG.md in Angular’s repository. This means for every commit, we only need the commit message and the commit SHA1 hash. Fortunately, git log can be configured with different formats. If we type git log --format=%B%H, Git will give us only the raw commit body and the hash.

feature: Added index.js script
1c3ec7c03f2796790eaf7271ef47b2141b22cb63
chore: Initial commit for changelog generator
05fbe5c5eee29cc33065474800e5401370e7e929

Now, we can create a script that just runs this command, gets the string output and then transforms it into an array. Let’s write some code in index.js:

Basically, we are just running the git log while adding a —--—DELIMITER—--— string to help us split the string. We also filtered out the commit if it doesn’t have a SHA hash. If we run the script, we will see the following output.

➜ node index.js
{ commitsArray:
   [ { sha: '1c3ec7c03f2796790eaf7271ef47b2141b22cb63',
       message: 'feature: Added index.js script' },
     { sha: '05fbe5c5eee29cc33065474800e5401370e7e929',
       message: 'chore: Initial commit for changelog generator' } ] }

Nice, now we have an array of commits, which are just objects with sha and message. We can use this array to write things into our CHANGELOG.md. Let’s do that for now. A lot of the code is just reading from a file, manipulating the string, and then writing the new string into a file, so I won’t bore you with the details. Here is the new code that can generate a changelog for us.

If we run our new code, we’ll see a nice changelog:

It works! Let’s manually bump the version in our package.json file to "2" and commit everything.

git commit -m "chore: Bump to version 2"

But, we have a problem. If we run the script again, we’ll see we have repeated items in the list:

Items in version 2 list should not appear again in version 3 list…

This happens because we are only running git log, which will return all the commits. What we want is to only get the logs from a particular commit, up to the current state. We need some kind of versioning.

Versioning

The easiest way to achieve versioning using Git is by using Git tags. Basically, the purpose of Git tags is to tag a particular commit (duh). Let’s try creating one. Run git log in your terminal again:

commit 50df6552c5e709b38dfd915aad3fa8e07e2b86e1 (HEAD -> master)
Author: Jacky Efendi <[not-a-real@email.com](mailto:not-a-real@email.com)>
Date: Sun Oct 20 15:27:36 2019 +0700

chore: Bump to version 2

commit 1c3ec7c03f2796790eaf7271ef47b2141b22cb63
Author: Jacky Efendi <[not-a-real@email.com](mailto:not-a-real@email.com)>
Date: Sun Oct 20 14:39:42 2019 +0700

feature: Added index.js script

commit 05fbe5c5eee29cc33065474800e5401370e7e929
Author: Jacky Efendi <[not-a-real@email.com](mailto:not-a-real@email.com)>
Date: Sun Oct 20 14:39:37 2019 +0700

chore: Initial commit for changelog generator

We see that the latest commit in our repo is the one with the hash 50df6552c5e709b38dfd915aad3fa8e07e2b86e1. yours will be different, so check it on your machine. Let’s tag this latest commit as version2. The way to do it is with the following command:

git tag -a -m "Tag for version 2" version2

The command will create an annotated tag, with “Tag for version 2” as the annotation, and version2 as the tag name. Now, let’s try running git log again:

commit 50df6552c5e709b38dfd915aad3fa8e07e2b86e1 (HEAD -> master, tag: version2)
Author: Jacky Efendi <[not-a-real@email.com](mailto:not-a-real@email.com)>
Date: Sun Oct 20 15:27:36 2019 +0700

chore: Bump to version 2
(...)

You can see that our commit now has tag: version2 written beside it. This means that the tag version2 is now referring to this particular commit, just like our current HEAD. Now, we can run git describe --long, and Git will tell us the latest tag we have in our Git repository.

➜ git describe --long
version2-0-g50df655

The output might seem a bit cryptic but is actually very simple. The string has three parts in it, all delimited by the — character. Here is the explanation for each part:

version2: The latest tag found in this Git commit history
0: The number of refs between the latest tag and the current HEAD
g50df655: The abbreviated commit SHA of the current HEAD, prefixed with “g”.

What we care about here is the name of the latest tag. We can pass this information to git log. The git log command can accept two Git refs, and return only the logs between those two refs. For example, we can do git log version2..HEAD. Git will return nothing, because version2 and HEAD currently refer to the same Git commit. Let’s try adding a dummy.txt file and just commit it.

touch dummy.txt
git add .
git commit -m "chore: Added dummy.txt file"

Now, if we run git log version2..HEAD again, we will see only one commit.

commit b9491e3c9a22bddff528e48fd599b08c0eafcce1 (HEAD -> master)
Author: Jacky Efendi <[not-a-real@email.com](mailto:not-a-real@email.com)>
Date: Sun Oct 20 15:45:19 2019 +0700

chore: Added dummy.txt file

I hope now you get the idea of what we are going to do with these commands; let’s get back to coding!

Writing the Changelog Generator (Continued)

In the first part of our code, we now want to run git describe --long to get the latest tag, and only run git log from that tag until our current HEAD.

In the last part of our code, we want to now automatically update the version number in the package.json file, create a commit, and tag that commit as the new version:

Now, let’s commit our new changelog generator:

git commit -m "feature: Implemented versioning for the changelog generator"

And let’s run the script again to generate another changelog:

It works!

Now, we have a fully functional changelog generator with support for versioning as well.

To see if this is really working, let’s put some text into the dummy.txt file and commit it. Then, we’ll run the script again.

Yep, everything still work!

Now you can just push all the tags to your remote repository so that they appear there as well. Simply run git push --tags to do so. In GitHub web, you can see the list of tags on the tags page.

Summary

We basically only needed git log, git describe, and git tag combined with some scripting to make this work. This is actually very similar to what tools like conventional-changelog do under the hood to give you that simple generated changelog. Of course, our generator is very simple and definitely doesn’t handle as many cases as conventional-changelog does, but the core idea is similar. By enforcing a specific format of writing the commit message, we can automate changelog generation.

When handling monorepo though, things get more difficult. lerna, a tool to manage a monorepo, has its own scripts, combined with conventional-changelog to handle monorepo uses cases. If you are curious and want to learn more, try improving on this generator to handle monorepo as well. I am sure you will learn much more in the process!

Resources

WebAssembly — Is it as scary as it sounds?

Jacky Efendi — Tue, 16 Jul 2019 10:10:50 +0000

WebAssembly — Is it as scary as it sounds? 😱

WebAssembly (Wasm) has been around for more than 2 years. It is still a relatively new piece of technology. For some reason, something about it always feels scary to me. Maybe because it has “ Assembly” written on its name? Or maybe simply picturing having to code in a language that is very different compared to JavaScript already stroke fear into me? Have I grown very attached to the quirks and features of JavaScript that the thought of NOT writing code in JavaScript just seems so unappetising? Whatever it was, the curiosity in me can’t stand not knowing it. After watching the talk by Surma on this year I/O, I finally brought myself to try to learn WebAssembly.

The face JavaScript developers make when trying to learn C++. (Pictures taken from Surma’s talk)

What is WebAssembly anyway? Why does it matter?

[WebAssembly] provides a way to run code written in multiple languages on the web at near native speed, with client apps running on the web that previously couldn’t have done so. — MDN

Basically, WebAssembly allows us to compile a code written in some languages like C, Rust, etc. (a complete list of languages can be found here) to a .wasm file and then run it on the browser. It is incredible in term of opening all new kinds of possibilities for the web platform. If you have tried Unity to build games before, you might be familiar with this picture.

The Unity game: Tanks!

It is a sample 3D game made with unity, ported to the browser with the help of WebAssembly! The crazy thing is: it’s not sluggish to play! You can try it out yourself here. Another story is AutoCAD. AutoCAD was ported to the web as well. You can watch this video to know more about it. WebAssembly isn’t magic. It won’t automatically take your existing program written in C++ and then compile it and it will run in the browser. What WebAssembly allows is for us to reuse code from other languages, and run it on the browser at near native speed!

I can see its benefits, but do I need to learn it though?

If your web app does not do heavy computations, but only do light tasks such as rendering the UI, making request to APIs, etc. then you probably don’t need WebAssembly. WebAssembly allows us as web developer to achieve things we previously can’t (or at least not feasibly) do with just JavaScript. It acts as a complement to JavaScript instead of a replacement for it. With that being said though, while you might not use it, it is always beneficial to have learned it and tried it out.

Foreword

In this article, we will use wasm to decode QR code from an image input. We will be using Rust and wasm-bindgen to help us in doing this. The final result Rust Crate can be seen in the following repo.

jackyef/rqrr-wasm

Be warned though, I am by no means an expert in WebAssembly and/or Rust. My experience in software engineering mostly consists of web development and writing JavaScript. This basically means what I am doing in this article might not be a good practice, and some of it maybe not technically accurate; but it is a genuine learning journey of a guy that is new to this environment. Hopefully, you will find this useful in a way or another!

Setting up

First, we will need to install Rust. Just follow this guide to do it. Once done with that, we will add wasm as a compilation target to the Rust compiler. We do it by running the following command.

rustup target add wasm32-unknown-unknown

Now, we should install the wasm-bindgen-cli because we will be using it later. We will install it using cargo. It should already be installed along with Rust. Think of it like npm, but for Rust. Run the following command to install wasm-bindgen-cli.

cargo install wasm-bindgen-cli

That’s all we need for now! Before we start, let me give you an overview of what we are going to do.

Write a function in Rust that can decode QR code
Compile the Rust code to wasm
Try using the wasm file in a simple HTML + JS webpage 🎉

Writing code in Rust

If you are familiar with Node, usually we start a project by running npm init. In Rust, we do cargo new hello_world instead. This will create a directory named hello_world with some files pre-made for us. In Rust, this project is called a package. We can import third-party crates as well, just like in Node we can import third-party modules. Now, let’s take a look at the Cargo.toml file. Yours probably looks a bit empty right now, but it’s okay. Just modify it to follow the following snippet.

You might see that this file contains the information of this package, and its dependencies. This file is much like a package.json file, it is the manifest of the package. The important thing here is that in [lib] section, we are defining the crate-type as ["cdylib"]. This is required when we are targeting Wasm. Also, we are not going to write a QR decoder ourselves, so we will be using third-party crate. We will use rqrr as the QR decoder. To create an image that will be fed to the decoder, we will use the image crate, and we will also use wasm-bindgen to help provide “Rust-to-JavaScript” bindings for us.

Now, rename src/main.rs to src/lib.rs and write the following code in it. lib.rs is the entry point of our package when we compile it to wasm later.

I am not very familiar with Rust, but I will try my best to explain. The extern crate and use statement is used to import the crates we will use, in this case, they are wasm_bindgen, rqrr and image.

Then, we create a public function using the pub fn keyword, named decode_qr. This function accepts an array of unsigned 8-bit integer named bytes (bytes:&[u8]) representing the image data. It will decode the image and return a String. The #[wasm_bindgen] attribute tells wasm-bindgen that we want this function to be exposed to our JavaScript when we use it. This information is used by wasm-bindgen to create appropriate bindings for us.

#[wasm_bindgen]
pub fn decode_qr(bytes: &[u8]) -> String {

We then create an image from this array, using the load_from_memory method provided by image crate. Since this operation can fail, we use the match keyword, and handle cases when the method returns Ok and Err results. On error, we will just return the string “[Error] Failed when trying to load image” to JavaScript side.

let img = match image::load_from_memory(&bytes) {
 Ok(v) => v,
 Err(_e) => return format!("{}", "[Error] Failed when trying to load image"),
 };

Then, we convert this image to a grayscale image before feeding it to rqrr.

let img = img.to_luma();

Note: You might be thinking: “Where do all these methods name come from?”. All of these methods along with other information can be found in the crate documentation pages. I have linked to the documentation pages above, but I will provide it again here in case you missed it. rqrr docs; image docs; wasm_bindgen docs;

Moving on. We then prepare the image and then feed it to rqrr, along with the case handling as well. Finally, we return the String to JavaScript side.

// Prepare for detection 
let mut img = rqrr::PreparedImage::prepare(img);

// Search for grids, without decoding 
let grids = img.detect_grids(); 

if grids.len() != 1 { 
 return format!("{}", "[Error] No QR code detected in image") }

// Decode the grid 
let (_meta, content) = match grids[0].decode() {
 Ok(v) => v,
 Err(_e) => return format!("{}", "[Error] Failed decoding the image"),
};

return format!("{}", content);

Compiling to Wasm

Now that we have our Rust code, we can compile it to wasm easily using the following command.

cargo build --target wasm32-unknown-unknown --release

This will generate a wasm file in target/wasm32-unknown-unknown/release.

We generated our first .wasm file! 🎉

Note that the name of the file is qr_rust.wasm. This is because the name of my package is qr-rust, so the output file is named accordingly.

We are not done though! Since we are using wasm-bindgen, we need to run wasm-bindgen against this wasm file to generate another wasm file with a JavaScript file containing the bindings needed to help us use the wasm file easily! Run the following command to do this.

wasm-bindgen target/wasm32-unknown-unknown/release/qr_rust.wasm --out-dir ./dist --no-modules --no-typescript

You should now see 2 new files in the dist directory.

The output files after using wasm-bindgen cli to create bindings

If you try to look inside the .js file, you will see a bunch of codes that is generated by wasm-bindgen so we can easily use the wasm module. 2 notable things that it does for us is that it helps us instantiate the wasm module in the init() function:

and it provides the necessary stuffs needed to pass data between JavaScript and Wasm:

If we are not using wasm-bindgen, we would have to write these stuffs ourselves.

And now…

Let’s try using it!

First, let’s create a simple HTML file that includes the JavaScript bindings generated by wasm-bindgen. The bindings have to be executed before we can do anything with the wasm module. Let’s create this index.html file in the ./dist directory.

<html>
 <!-- the javascript bindings -->
 <script src="qr_rust.js"></script>
</html>

The bindings create a wasm_bindgen variable in the global scope. Which we can use to load our wasm module.

<html>
 <!-- the javascript bindings -->
 <script src="qr_rust.js"></script>

<script>
 wasm_bindgen("qr_rust_bg.wasm")
 .then(() => {
 console.log('it is loaded!');
 })
 .catch(console.error);
 </script>

</html>

Now, let’s try serving this HTML file locally and see what happens. The easiest way to do it would be to use http-server npm module and serve our ./dist directory.

npm install http-server -g
http-server ./dist -g

Open the URL, if everything is correct, you should see It is loaded! in the browser console.

We loaded our wasm module on the browser! 🎉

The function that we wrote in Rust, can be accessed from the global wasm_bindgen variable.

const { decode_qr } = wasm_bindgen;

At this point, we can just pass an array of unsigned 8-bit integer to the function and log the output. Here, I am passing new Uint8Array([1,2,3,4,5]); to the function.

<html>
 <!-- the javascript bindings -->
 <script src="qr_rust.js"></script>

<script>
 wasm_bindgen('qr_rust_bg.wasm')
 .then(() => {
 console.log('it is loaded!');

const { decode_qr } = wasm_bindgen;
 const output = decode_qr(new Uint8Array([1,2,3,4,5]));
 console.log("output of decode_qr:", output);
 })
 .catch(console.error);
 </script>

</html>

It failed!

Apparently, it fails trying to load the image. This is expected because we are just passing a random array that is not actually representing an image data. Let’s create a <video /> element from which we will get the image data from.

<video id="video" autoplay></video>

// some other stuffs here

<script>
// other scripts

// inside wasm_bindgen().then()
 const video = document.getElementById('video');
 navigator.mediaDevices.getUserMedia({ video: true })
 .then(stream => {
 video.srcObject = stream;
 });

</script>

If you refresh the browser, your browser should be asking permission to use the camera now. Allow it, and you should see your camera feed in the browser.

Me, holding a QR code image in front of the webcam

Next, we want to be able to get the current frame of the video, get the image data as an array of unsigned 8-bit integers, and send it to the decode_qr() function. We will do this with the help of canvas and FileReader. Here is a captureImage function that will do just that.

Now, we just need to periodically call this function. We can simply do this with setInterval. We will start off the interval after the video stream is created.

navigator.mediaDevices.getUserMedia({ video: true })
 .then(stream => {
 video.srcObject = stream;

setInterval(captureImage, 300);
});

So, our final HTML file should look like this.

Refresh your browser, time to try it out!

It works! The QR code says “http://en.m.wikipedia.org” 🎉

Sometimes, you might see an error that says “unreachable code”. I haven’t figured out why that happens yet. If you do know the cause, please let me know!

What you can try to do next

We have taken a functioning QR decoder written in Rust, compile it to WebAssembly and use it in the browser with the help of wasm-bindgen.

However, if you look at the network requests in devtools, you would see that the size of this wasm file is HUGE!

736 KB for a QR decoder!?

Note that we haven’t do any compression and optimisation. You can try looking up wasm-opt to optimise the size. This guide will help you with that. Then, you can compress it down with any compression algorithm you want, a common one is gzip. See how small you can get the wasm file size down to. I am currently at 264 KB gzipped!

Next, you can try publishing your creation as an npm module so other people can use it easily. I have done this exercise myself. If you want to take a look at my implementation, you can check out my repository below. It contains mostly the same code that I included in this writing.

jackyef/rqrr-wasm

Takeaways

So, is WebAssembly scary? For me, the answer is no. The reason why it seemed so scary at first because it was a mysterious piece of technology that I was very unfamiliar with. As a guy who mostly coded in JavaScript, it was a bit odd to code in Rust. After reading and watching stuffs about WebAssembly, I could no longer ignore it and decided I had to get over my fear of WebAssembly; and apparently, a good way to get over the fear of it is to actually dive into it.

Note that you don’t even have to create your own Wasm! There is an increasing number of Wasm module created by other people being published to npm. We can just consume these modules in our projects. The purpose of the exercise we did is to get to know WebAssembly better. In practice though, we probably might not have to do that. But, if one day you need to, at least you have an overview of how it works already.

WebAssembly opens up a whole lot of possibilities for the web platform, especially for tasks that were too heavy to do with just JavaScript. Your use cases might not need WebAssembly. For example, our QR decoder (after some optimisations) is 264 KB big. There is this QR scanner written in JavaScript that is only ~12.4 KB gzipped. Depending on your use case, you could argue that even with the performance advantage, the WebAssembly solution is overkill for this purpose and you could be right. The point is, at the end of the day, tools are just tools; and WebAssembly is a great addition to our toolbox. Whether it makes sense to use the tool will be decided by us, the builder.

You can learn more about WebAssembly by watching one of the talk at 2019’s Google I/O here. You can also play around with it in the online IDE: webassembly.studio.