Shehab Hosny

Posted on Dec 4, 2021

GitHub Actions Workflow for Ultimate Laziness

#actionshackathon21 #github #opensource

My Workflow
Submission Category
Additional Resources

My Workflow

Motivation behind the Workflow

Let's all agree that we, as Human Beings, prefer laziness most of the time. We love snoozing the alarm for an extra five minutes every morning, we prefer texting our siblings who are just sitting in the next room, sometimes we don't even run through our exam paper before submitting it just because we are too lazy to do so.

Besides laziness, we usually get bored so easily, from tedious tasks at work right through to waiting in line at the supermarket. Our brains simply don't respond well to experiences that are too predictable and repetitive. This lack of amusement is what we call boredom.

That's why we invented Machines and enhanced the overall process of automation. Simply to make everything better, from increasing productivity to building more efficient and flawless products.

Well, in the Software Engineering field, we too have some tedious tasks to keep on doing daily to maintain the output product. Here comes GitHub, or more specifically GitHub Actions, to provide us a solution to automate any number of repetitive tasks and start the process with something just as simple as a mouse click! Impressive, right? Buckle up and get ready, because your mind is about to be blown away.

How did GitHub Actions help me?

Let me tell you my story from the 3rd-person point of view perspective. So, Shehab is an Open-Source enthusiast who created a GitHub repository called DiffCpp almost a year ago. At this moment in time, DiffCpp has a lot of users, forks, and stars (well, this is not the case for now, but hopefully it might be shortly). Anyway, this GitHub repo currently requires regular maintenance and updates to be alive. Since Shehab is a slothful person who avoids doing monotonous tasks, he decided to use existing GitHub Actions to build a perfect workflow to automate any kind of work needed to ensure that the codebase will not be broken in whatever way before releasing it.

So, he decided to brainstorm and list out all of the possible sets of tasks needed to get the job done. He found out that these tasks can be categorized as follows:

Documentation
- Check the Formatting of the Codebase
  - Since this is a C++ project, Clang-Format will be used.
- Generate Documentation
  - Since this is a C++ project, Doxygen will be used.
- Generate Release Notes to notify the user with changes.
Testing
- Check for a Successful Build on top of:
  - Linux (Ubuntu)
  - Windows
- Use Google Test for:
  - Unit Testing: Test the logic of each API on its own.
  - Block Testing: Test the logic of the package as a whole.
- Check the Benchmark of the package APIs using Google Benchmark.

Well obviously, this is much to do, but cautious! SPOILER ALERT is on the way, and let me just tell you not only that GitHub allowed me to do all of the previously listed tasks with ZERO effort but it also provided me with this charming pipeline visualization at the end of the process.

How I created this workflow?

For simplicity, let’s consider the following scenario. I woke up this morning, grabbed a cup of coffee, and checked out my GitHub account just to find out that someone created an issue in the DiffCpp repo mentioning the following:

Input = 🔵
Expected Output = 🔴
Actual Output = ⚪

This needs an immediate fix. So, I decided to offer $10,000,000 (I wish I had this amount of money in real life) for the first contributor who solves the issue. Well, for the first thought, you may think that this will solve the issue but have you imagined how many pull requests would I get per minute?

Apart from the number of pull requests, I wouldn't even be able to check all of them to find out which ones get the job done, this would take me forever! Thanks to Google Test that allowed me to add a new test with something just as simple as these 2 lines of code:

Output = Input(🔵);
EXPECT_EQ(Output, 🔴);

So, here comes GitHub Actions (in collaboration with Google Test) to solve the problem. So, I decided to build a workflow to check each submitted pull request for the following:

Successful Build 🔧

YAML snippet (Click to Expand)

build-linux:
  name: Linux
  runs-on: ubuntu-latest
  steps:
  - name: Checkout Repository
    uses: actions/checkout@v2

  - name: Build
    run: cmake .

  - name: Make
    run: make

  - name: Upload Linux Artifact
    uses: actions/upload-artifact@v2
    with:
      name: DiffCpp-Linux
      path: ./DiffCpp

For windows build, the same steps would be used using windows-specific CMake commands.

Actions Used:

actions/checkout@v2 is used to checkout my GitHub repo.
Here is a request from the GitHub Team
actions/upload-artifact@v2 is used to allow the user to download the built binaries later.

Code Formatting 📝

YAML snippet (Click to Expand)

clang-format-check:
  name: Format Check
  runs-on: ubuntu-latest
  steps:
  - name: Checkout Repository
    uses: actions/checkout@v2

  - name: Check Codebase Format
    uses: RafikFarhad/clang-format-github-action@v1.0.1
    with:
      sources: "src/*.cpp,include/*.h,test/*.cpp"

Actions Used:

RafikFarhad/clang-format-github-action@v1.0.1 is used to check the format of the submitted code using clang-format.

Unit and Block Testing 📋

YAML snippet (Click to Expand)

verification:
  name: Unit and Block Testing
  needs:
    - build
  runs-on: ubuntu-latest
  steps:
  - name: Checkout Repository
    uses: actions/checkout@v2

  - name: Cache GoogleTest Package
    id: cached-gtest
    uses: actions/cache@v2
    env:
      cache-name: cache-gtest-repo
    with:
      path: googletest
      key: ${{ runner.os }}-build-${{ env.cache-name }}

  - name: Install GoogleTest
    if: steps.cached-gtest.outputs.cache-hit != 'true'
    run: mkdir googletest &&
         git clone https://github.com/google/googletest.git

  - name: Build
    run: cmake -DBUILD_TESTS=ON .

  - name: Make
    run: make

  - name: Run Unit Tests
    run: ./DiffCpp_tst

Actions Used:

actions/cache@v2 is used to cache the output of the specified step. This will allow GitHub to skip the specified action if it has been already done before.

Note: If you are using any verification method other than GoogleTest, you can simply use the uploaded artifact from the build job and pass it to this one. GitHub is great isn't it? Anyway, for C++ projects, GoogleTest is perfect for verification. You will even have the ability to check the test results from the GitHub Actions log as it is shown right here 👇

When these checks pass successfully, the following will be shown in the pull request itself to facilitate direct merge.

Moreover, you can feast your eyes on the charming visualization of this pipeline showing the complete flow of the code review right here 👇

We are not done yet! Additionally, GitHub can even notify me when someone submits a valid pull request that solves the issue. So, I will be able to successfully check over millions of pull requests effortlessly.

Well, now it is time to generate a release, but again I have neither intention nor energy to do any other work (as if I have done any). Thus, I decided to build another workflow, based on the previous one, that will be in charge of generating the release from A to Z and can be triggered with a simple mouse click.

First things first, my strategy will be based on DRY (Don't Repeat Yourself). Thus, I decided to make use of what is called Reusable Workflows. Yes! it is exactly as fascinating as you read, GitHub has just introduced a new feature of reusing workflows, many thanks to @blackgirlbytes for her awesome blog post about the feature.

GitHub Actions: You Can Build Reusable Workflows!

Rizèl Scarlett for GitHub ・ Nov 17 ・ 3 min read

#actionshackathon21 #github #opensource #devops

So, I needed to add the following tasks:

Generate Documentation for the Codebase 📜

YAML snippet (Click to Expand)

doxygen-doc:
  environment:
    name: Doxygen Documentation
    url: https://shehab7osny.github.io/DiffCpp/include/
  name: Generate Documentation
  needs:
    - codebase
  runs-on: ubuntu-latest
  steps:
  - name: Checkout Repository
    uses: actions/checkout@v2

  - name: Run Doxygen
    uses: mattnotmitt/doxygen-action@v1
    with:
        working-directory: 'include/'

  - name: Deploy to GitHub Pages
    uses: peaceiris/actions-gh-pages@v3
    with:
      github_token: ${{ secrets.GITHUB_TOKEN }}
      publish_dir: ./include/html
      destination_dir: ./include

Actions Used:

mattnotmitt/doxygen-action@v1 is used to build doxygen docs.
Currently documentation is only available for the include directory but hopefully I will expand it later.
peaceiris/actions-gh-pages@v3 is used to deploy html output of doxygen into GitHub pages.

The environment specified here has a url of Doxygen output displayed just right in the workflow visualization graph itself.

This job has a single dependency ➡️ codebase formatting

The generated URL will consist of detailed code documentation as follows:

Compute the Benchmark 📈

For those of you who don't know, Benchmarking means measuring the performance of a certain program with respect to time.

YAML snippet (Click to Expand)

benchmarking:
  environment:
    name: Benchmark Results
    url: https://shehab7osny.github.io/DiffCpp/dev/bench
  name: Benchmark (Performance)
  needs:
    - verification
  runs-on: ubuntu-latest
  steps:
  - name: Checkout Repository
    uses: actions/checkout@v2

  - name: Cache Google Benchmark Package
    id: cached-benchmark
    uses: actions/cache@v2
    env:
      cache-name: cache-gbench-repo
    with:
      path: benchmark
      key: ${{ runner.os }}-build-${{ env.cache-name }}

  - name: Install Google Benchmark
    if: steps.cached-benchmark.outputs.cache-hit != 'true'
    run: mkdir benchmark &&
        git clone https://github.com/google/benchmark.git benchmark

  - name: Build
    run: cmake -DBENCHMARK_DOWNLOAD_DEPENDENCIES=ON -DCMAKE_BUILD_TYPE=Release -DBUILD_BENCH=ON .

  - name: Make
    run: make

  - name: Run Benchmark
    run: ls && ./DiffCpp_BENCH --benchmark_format=json | tee benchmark_result.json

  - name: Store benchmark result
    uses: benchmark-action/github-action-benchmark@v1
    with:
      tool: 'googlecpp'
      output-file-path: benchmark_result.json
      github-token: ${{ secrets.GITHUB_TOKEN }}
      fail-on-alert: true
      comment-on-alert: true
      auto-push: true

Actions Used:

benchmark-action/github-action-benchmark@v1 is used to convert the JSON results from Google Benchmark into a statistical graph.
actions/cache@v2 is used to cache the output of the specified step. In this case the step of checking out Google Benchmark repo is cached.

This job has a single dependency ➡️ verification

The generated URL will consist of Benchmark Graph as follows:

Create a new Release (with Release Notes) 📭

YAML snippet (Click to Expand)

release-project:
  name: Release Project
  needs:
    - benchmarking
    - doxygen-doc
  runs-on: ubuntu-latest
  steps:
  - name: Download Linux Artifact
    uses: actions/download-artifact@v2
    with:
      name: DiffCpp-Linux

  - name: Download Windows Artifact
    uses: actions/download-artifact@v2
    with:
      name: DiffCpp-Windows

  - name: Create GitHub Release
    id: create-new-release
    uses: actions/github-script@v4.0.2
    with:
      github-token: ${{secrets.GITHUB_TOKEN}}
      script: |
        await github.request(`POST /repos/${{ github.repository }}/releases`, {
          tag_name: "${{ github.event.inputs.release_ver }}",
          generate_release_notes: true
        });

Actions Used:

actions/download-artifact@v2 is used to download the previously uploaded artifact in case it is needed to be packed in the release.
actions/github-script@v4.0.2 is used to create a new release and generate an Automated Release Notes along with it.

This job has two dependencies
➡️ benchmarking
➡️ doxygen-doc

In case you are new to Auto Generate Release Notes, don't hesitate to check out this charming video created by @mishmanners , many thanks for her. Anyway, I decided to take this feature to a new level and include it in an automated action itself. So, I'm literally automating an automated task!

Here is a snapshot of the auto-generated release notes:

Besides, the built artifacts will be ready for download (both for Linux and Windows)

Finally, after the completion of all these tasks, we can have a look on this charming visualization of the pipeline showing the complete flow of the release process right here 👇

Submission Category:

Maintainer Must Haves

In conclusion, I just want to point out that relating this workflow with laziness is for nothing but simplicity of illustration. However, in real life, developers commonly tend to avoid such monotonous tasks so that they can keep focused on the development itself rather than dealing with numerous amount of tasks for verification and release purposes. So, from my point of view, using this workflow in any project is a must, and will help developers to focus more on introducing new features to the project while making sure that the codebase is not broken.

Yaml File or Link to Code

Reusable Workflows
Main Workflows:
- code-review.yml
- release.yml

Here is the DiffCpp repo

Shehab7osny / DiffCpp

Diff tool for Windows/Linux implemented in C++

DiffCpp

This package is used to compare two files line by line and show the differences.

Build and Installation

Linux

mkdir build
cd build
cmake
make

Windows

mkdir build
cd build
cmake ..
cmake --build .
cd Debug/

Usage

Linux

./DiffCpp File_1_Path File_2_Path [/N] [/A] [/W]

Windows

DiffCpp File_1_Path File_2_Path [/N] [/A] [/W]

Main Arguments

Argument	Description
File_1_Path	This represents the old version of the file
File_2_Path	This represents the new version of the file

Optional Arguments

Argument	Description
/N	Option to add line number to the displayed output
/A	Option to display the updates only and ignore common matches
/W	Option to ignore whitespaces difference while comparing files

Sample Use Case

File 1 (recursion_Old.py)

def tri_recursion(k):
  if(k > 0):
    result = k + tri_recursion(k - 1)
  else:
    result = 1
  return result

File 2 (recursion_New.py)

def tri_recursion(

…

View on GitHub

Additional Resources

Open Source projects using (hosting) the workflow ➡️ DiffCpp
Open Source projects used in the workflow:
- Google Test
- Google Benchmark
Existing Actions that have been used in the workflow:
This project is licensed under the terms of the MIT license.
Honorable mention

Many thanks to @MylesBorins and @bdougieyo for their ultimate useful info about GitHub's new feature Automated Release Notes that has been applied in this project. You can check their video right here 👇
Some enhancements requested from the GitHub team:
- Allow some sort of caching the step of checking out the repository to avoid applying the same step several times.
- Allow reusable workflows (YAML files) to be called from a child directory under the workflows directory itself.