Linting only changed files with GitHub Actions

In our team at Flipp, like many companies, we have a venerable and enormous Rails monolith which is feeling its age. We've started the long process of breaking it up, but in the meantime there are hundreds of thousands of lines of code which are changed piecemeal across dozens of features.

We introduced Rubocop, the Ruby linter, a few years ago, and very slowly started updating the code as we changed it. We used a pre-commit Git hook to ensure that our developers couldn't even commit code that didn't pass the linter. We used a modified version of dirty_cop - at the time this was just a gist we found online - which only runs the linter on modified files.

As time went on, this solution started to become less and less ideal. Our codebase was so big and so old that just starting the linter could take upwards of 20 seconds. We took on new team members, not all of whom installed the pre-commit hook; other teams began submitting pull requests as well. This just wasn't a great solution to ensure we had a clean codebase with a consistent style.

The Project

Our first instinct was to move the lint check to our CI tool, CircleCI. It's really powerful, but there was one big problem - CircleCI runs on pushes, not pull requests. It would have no way of knowing what files changed in the PR.

Finally, we started investigating GitHub Actions. This seemed like it ticked all the boxes - it's a CI tool that can be set to run on pull requests, meaning it would be possible to detect which files were changed in that PR. Even better, because it runs within the GitHub ecosystem, it would be relatively simple to add comments or even commits to the PR itself fixing or commenting on the issues that were found.

Luckily enough, we found two existing actions that looked right up our alley:

• file-changes-action outputs the files changed during a PR.
• lint-action runs a linter (it supports many languages, not just Ruby) and does auto-fixing and commenting as well.

On with the show!

The Solution

name: Lint
on: [pull_request]

jobs:
  build:
    name: Lint
    runs-on: ubuntu-latest
    steps:
    - name: Checkout
      uses: actions/checkout@v2
    - name: Setup
      uses: actions/setup-ruby@v1
      with:
        ruby-version: '2.4'
    - name: Cache gems
      uses: actions/cache@v1
      with:
        path: vendor/bundle
        key: ${{ runner.os }}-gems-${{ env.cache-name }}-${{ hashFiles('**/Gemfile.lock') }}
        restore-keys: |
          ${{ runner.os }}-gems-

    - name: Bundle install
      run: |
        bundle config our_gem_repo.com our_username:${{ secrets.GEM_REPO_PASSWORD }}
        bundle config set without 'production staging'
        bundle config path vendor/bundle
        bundle install
        git checkout Gemfile.lock

    - name: Get file changes
      id: get_file_changes
      uses: dorner/file-changes-action@v1.2.0
      with:
        githubToken: ${{ secrets.GITHUB_TOKEN }}
        plaintext: true
    - name: Echo file changes
      run: |
        echo Changed files: ${{ steps.get_file_changes.outputs.files }}
    - name: Run lint
      uses: dorner/lint-action@v1.3.3
      with:
        github_token: ${{ secrets.GITHUB_TOKEN }}
        rubocop_bundler: true
        auto_fix: true
        rubocop_bundler_args: -R --fail-level C ${{ steps.get_file_changes.outputs.files}}

The Issues

When developing this, we ran across a number of issues we were able to solve:

• Our gems are hosted on a private repo. To enable access to them, we had to store a secret in the repo with our credentials.
• Our Gemfile.lock uses an old version of Bundler (1.17). Newer versions of Bundler automatically update the Gemfile.lock to indicate that it was bundled with that version - which means older Bundlers will subsequently fail. This is fine if you're not doing anything with that file, but the auto_fix option on the linter means it may commit the contents of the workspace back to the repo, along with the modified Gemfile.lock. To solve this, we had to use git checkout to keep the file pristine before continuing.
• The file-changes-action outputs the files as a JSON array of strings. This isn't so convenient to pass into a shell command, so I made a fork of it and added the plaintext option. The great thing about GitHub Actions is that literally any repo can be an action - as long as it's tagged, I didn't have to wait for a PR to be accepted. I just switched the source of the action to my fork and went on my merry way.
• The lint-action Rubocop linter had two issues that needed to be fixed:
  • We use a shared gem on our private repo to share our styles across our projects. The action doesn't work with bundle exec. Since we had to use rubocop in the context of the bundle to use our shared gem, I needed to create a brand new linter called rubocop_bundler just to add the bundle exec before the command.
  • The current Rubocop linter adds "." to the end of the command. This is not only unnecessary but forces the linter to run on all files, even if we pass a list of files to it. My fork removed the period and all works well.

The action forks fixed all the problems, so now this workflow file:

• Caches the gems, so it almost always runs in less than 1 minute.
• Records all warnings and errors as comments in the PR.
• Auto-fixes fixable warnings with an additional commit in the PR.
• Adds a Check to the Checks pane where we can inspect the offenses in a nice UI.

This was done with Ruby, but you can use the linter for any language you like!

Addendum: I've added PRs to both the abovementioned actions, and have gotten a response, so this functionality should be available in the base repos soon, although it might look a bit different. (In addition, see this issue where it mentions that some of the annotations might be clobbered in some cases - we felt this was acceptable.)

Update March 30: My PRs and issues have been released! The new way to do this (using the official action repos) is as follows:

    - name: Get file changes
      id: get_file_changes
      uses: trilom/file-changes-action@v1.2.3
      with:
        output: ' '
    - name: Echo file changes
      run: |
        echo Changed files: ${{ steps.get_file_changes.outputs.files }}

    - name: Run lint
      uses: samuelmeuli/lint-action@v1.4.0
      with:
        github_token: ${{ secrets.GITHUB_TOKEN }}
        rubocop: true
        auto_fix: true
        rubocop_args: -R --fail-level C --display-only-fail-level-offenses ${{ steps.get_file_changes.outputs.files}}
        rubocop_command_prefix: bundle exec