Faster RuboCop runs for Rails apps

Recently, Nate Berkopec shared an interesting observation: running bundle exec whatever could take seconds to boot if the Gemfile is huge (even when the executable itself requires a handful of dependencies).

Nate Berkopec

@nateberkopec

Say I have a huge Gemfile, like 500 total dependencies in the lock. I have a group in this Gemfile with ~5 gems in it. Why is Bundler.setup(:nameofgroup) so much slower than maintaining a separate Gemfile with just those 5 gems in it?

14:48 PM - 18 Feb 2022

That could be explained by the fact that Bundler has to verify the Gemfile.lock file consistency (all the gems are installed). Thus, that's an expected behaviour (that doesn't mean we shouldn't try to improve it; see, for example, Matthew Draper's Gel).

Rails developers usually put all the deps in the Gemfile, including dev tools, such as RuboCop. RuboCop is a linter, and linters must be fast. RuboCop itself complies with this statement but running it via Bundler may not.

How can we overcome this? Using a separate Gemfile!

I've been using this technique for a long time for gems development—to speed up CI RuboCop runs (by installing only the linter dependencies). Here is my typical rubocop.gemfile:

# gemfiles/rubocop.gemfile
source "https://rubygems.org" do
  gem "rubocop-md", "~> 1.0"
  gem "rubocop-rspec"
  gem "standard", "~> 1.0"
end

To use it with Bundler, we need to specify the BUNDLER_GEMFILE env variable:

# first, install the deps
BUNDLE_GEMFILE=gemfiles/rubocop.gemfile bundle install

# then, run the executable
BUNDLE_GEMFILE=gemfiles/rubocop.gemfile bundle exec rubocop

This verbose approach works well enough for machines (CI), but not for humans: maintaining a separate lockfile and using env vars in development is far from the perfect user experience.

For Rails applications development, we came up with the following trick to run commands backed by custom gemfiles—adding a simple bin/whatever wrapper. Here is our bin/rubocop:

#!/bin/bash

cd $(dirname $0)/..

export BUNDLE_GEMFILE=./gemfiles/rubocop.gemfile
bundle check > /dev/null || bundle install

bundle exec rubocop $@

The magic $@ argument proxies everything you pass to bin/rubocop, thus, making this wrapper quack like RuboCop.

We also do bundle check || bundle install to make sure all the deps are present (so, you don't need to run bundle install yourself).

That's it.

P.S. Why not use inline gemfiles (as Xavier Noria suggested)? We could write our bin/rubocop like this:

require 'bundler/inline'

gemfile(true, quiet: true) do
  gem "rubocop-md", "~> 1.0"
  gem "rubocop-rspec"
  gem "standard", "~> 1.0"
end

require 'rubocop'
RuboCop::CLI.new.run

However, with this approach, there is no lockfile at all. We want to make sure everyone is using the same versions of dependencies (to avoid "works on my computer" situations). Of course, we can use the exact version in the gemfile do ... end block, but, IMO, managing deps with Bundler is more convenient (e.g., you can run bundle update).

P.P.S. One of the benefits of this approach is the ability to run linters (and other tools, e.g., Kuby) locally while using Docker for application development; no need to spin up containers to run RuboCop. It's especially helpful if you want to use Git hooks or editor integrations.