Github Actions & Workflows for CI/CD

It's been already three months since I have joined DataHunt, but to be honest, I haven't been able to work on automating our development workflows due to lack of time. Endless march of demos and actual product delivery in a very short time span left us with untested code and no reliable CI/CD pipeline during code push and deploy.

Therefore, I decided to spend some time this week rebuilding our test suites and structuring CI/CD so that we could deliver our software in a more stable way from now on. My goal was to integrate integrations tests to Github CI and deploy to Heroku only when the test suite succeeds.

There were several options that came to my mind:

1. Jenkins CI
2. Travis CI
3. Heroku CI
4. Github Actions

Of all these solutions, I am leaning toward Github Actions.

Jenkins & Travis CI

I had some experiences with Jenkins CI at my previous company since our team used it for most of our CI/CD flow. Jenkins is easy to install, use, and quite flexible due to its configurable nature. If your team uses a lot of tech stacks, including mobile, web, backend with various languages, then I think probably this is the best option. However, its ease of use just stretches as far as to a very basic task. If you don't mind using locally installed databases or softwares, then Jenkins will probably provide you with the fasted setup and build speed, since what you have to do is just installing softwares on a machine and running scripts (probably little bit of permission settings). But if you want to use containers for your build environment for providing a consistent and isolated build environments, then things become a little more complicated in Jenkins (at least this is true for my personal experience). You have to manually install docker plugins and set up a containerized build system, which felt like pain in the ass when I tried to set up one for Kakao Android SDK project.

On the other hand, Travis CI, Heroku CI, and Github CI all offer solutions based on containers that provide isolated build environments. It is true that build speed might slow down because containers must provision necessary services in pre-test stage, but there is a larger benefit in switching because you don't have to upgrade local databases or other environments yourself.

Travis CI

Travis CI is quite good when you host an open source project on Github, since you get the service for free. However, our project was private and the pricing seemed a little expensive at our stage. I am currently using it for kakao_flutter_sdk open source project that I am maintaining for Kakao organization.

Heroku CI

Heroku CI also seemed a good option since we were on the PaaS and clicked well with the Heroku pipeline feature (dev, staging, and production env setup, deploy, etc...) we were using.

Below screenshot shows automated tests running on pushing to specific branches.

However, there were several downsides.

1. Builds seemed slower than other alternatives.
2. Databases could be provisioned in-dyno but there is no guarantee that every add-on that we use will be available. It would be impossible to run whole integration tests if just one add-on isn't available in Heroku CI env.
3. CI/CD will be stuck with infra. If we migrate to AWS in the future, redundant work must be done to setup CI/CD again with other alternative.

Therefore, I decided to try the last option, GitHub Actions.

Github Actions & Workflows

I have tried Github Actions after just testing with Heroku CI. Just putting a yaml file that specifies your build at .github/workflows/ directory and you are all set. Below are requirements for my automated testing.

1. Should be run on every push/pr to master
2. Should connect to Postgres and redis for integration tests
3. Tests should be run after dependencies are installed

One specific setting would be like below:
.github/workflows/test.yml

name: Integration Tests CI

on:
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]

jobs:
  build:

    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres
        env:
          POSTGRES_PASSWORD: postgres
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
        ports:
          - 5432:5432
      redis:
        image: redis
        ports:
        - 6379:6379
        options: --entrypoint redis-server
    strategy:
      matrix:
        node-version: [12.x]

    steps:
    # Downloads a copy of the code in your repository before running CI tests
    - name: Check out repository code
      uses: actions/checkout@v2
    - name: Use Node.js ${{ matrix.node-version }}
      uses: actions/setup-node@v1
      with:
        node-version: ${{ matrix.node-version }}
    # Performs a clean installation of all dependencies in the `package.json` file
    # For more information, see https://docs.npmjs.com/cli/ci.html
    - name: Install dependencies 
      run: npm ci
    - run: npm test
      env:
        DATABASE_URL: postgres://postgres:postgres@localhost:5432/postgres
        REDIS_URL: redis://localhost:6379
        NODE_ENV: test

Of course, this is my first basic experience with Github Actions but I will delve more into Github based solutions because this is the only platform that I am pretty sure that will not be replaced by other platforms among stuff that our team is using (with least alternative, I suppose). Familiarizing myself with GitHub Actions could definitely benefit me in other projects!

References:

1. https://help.github.com/en/actions/configuring-and-managing-workflows/creating-postgresql-service-containers#testing-the-postgresql-service-container
2. https://devcenter.heroku.com/articles/heroku-ci