How to integrate Brakeman Security Scanner with GitHub Code Scanning

GitHub Code Scanning automatically scans your repository for security vulnerabilities and alerts you in Pull Requests and on a dedicated alert backlog accessible via a repository's Security tab.

Out of the box, GitHub Code Scanning uses a tool called CodeQL, but you can also display alerts produced by any tool that can output its results in the SARIF format.

The Rails security scanner Brakeman supports the SARIF format, but I couldn't find any documentation about how to join all the pieces together and see Brakeman results in GitHub Code Scanning. This blog post is my answer to the question of how you do it.

Pre-requisites

This guide assumes that you already have GitHub Code Scanning turned on for your repository. Your project will also need to have Brakeman installed. You don't need it running on your local machine - you could do the install explicitly in the Actions workflow you'll be creating, but this guide assumes you instead have something like the following in your Gemfile:

group :development, :test do
  gem 'brakeman', require: false
end

Creating a GitHub Actions workflow

To upload results to Code Scanning, you'll need to create an GitHub Actions workflow. rails new creates an existing workflow which you could modify, but for simplicity I'm showing here a fresh workflow just for Brakeman.

Here's the full workflow; I'll explain it step-by-step below:

name: "Run brakeman and upload results to Code Scanning"

on:
  push:
    branches: [main]
  pull_request:

jobs:
  brakeman:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v3

      - name: Set up Ruby
        uses: ruby/setup-ruby@v1
        with:
          ruby-version: .ruby-version
          bundler-cache: true

      - name: Run brakeman
        # Without --no-exit-on-warn, brakeman returns a non-zero error code when alerts
        # are found, failing the check. We want the check to pass as results are handled
        # via GitHub Code Scanning's UI.
        run: bundle exec brakeman --no-exit-on-warn -f sarif > brakeman-results.sarif

      - name: Upload scan results to GitHub Code Scanning
        uses: github/codeql-action/upload-sarif@v3
        with:
          sarif_file: "brakeman-results.sarif"

The workflow begins with metadata - a name, and when it should run (that empty pull_request: key isn't a typo, despite appearances). We then declare a job and state that it should run on Ubuntu. Other runner types are available, or you can host your own.

The first step in the job uses actions/checkout to clone the repo it runs in. We then run the ruby/setup-ruby action which installs Ruby and runs Bundler. If Brakeman is in your Gemfile (see above), that'll be installed at this stage.

The next step actually runs Brakeman, using the run command to run the following terminal command:

bundle exec brakeman --no-exit-on-warn -f sarif > brakeman-results.sarif

The first part of this command is fairly self-explanatory, we're running Brakeman via Bundler: bundle exec brakeman.

--no-exit-on-warn is important. Normally, if Brakeman finds any alerts, it will return a non-zero return code. On UNIX-like systems (including Linux or Mac), that indicates a failure and so the workflow will fail. If you weren't uploading results to Code Scanning, that'd be the correct behaviour -- a failing workflow would indicate problems found. However, that's not what we want here -- we'll be showing alerts through the Code Scanning UI instead.

-f sarif tells Brakeman to use the SARIF format format, which is what GitHub Code Scanning needs.

Finally, we redirect the output into a file ready for the next step to upload.

The final step in the workflow uploads the results to GitHub Code Scanning.

What you get

Once this is running, results from Brakeman will appear in the Code Scanning UI:

Newly-introduced alerts will also show up as annotations on Pull Requests.

If you create the new workflow in a Pull Request, you might also see a helpful comment automatically posted on it, like this:

Troubleshooting

Not working? Try these...

1. Take a look at the log output from the workflow; chances are, you'll see an error here that helps explain what's wrong.
2. Run the Brakeman command locally and see what happens. Take away the redirection if you want to see output, and remove -f sarif to get results in a more human-readable format. You might not actually have any alerts in your project, of course!
3. Look at the Code Scanning results page for your repository (/security/code-scanning). You should see a "tool status" banner at the top, which will tell you the health of the Code Scanning tools that your repository is using. You might see Brakeman errors or warnings here; if not then press the "Tools" button and see if Brakeman is listed at all. If it's not, then the upload or a previous step is failing.
4. You could experiment with uploading results manually using the Code Scanning API. Run the workflow Brakeman command, upload results and see what status or error you get back.

See also

• GitHub's official resource article, Extend your testing with third-party tools with GitHub code scanning. This isn't Ruby-specific, though.