Did you know bundler can download and install gems in parallel?
That's right, using the
--jobs option for
bundle install means you can set the number of gems that can download and install at the same time.
As a bonus, you don't even have to remember to use the
--jobs option every time. You can set this in your global bundler config with this command:
$ bundle config --global jobs 4
I found all of this out recently, even though this has been available since bundler version 1.4.0, and I wanted to test how useful it actually was. Downloading and installing gems in parallel sounds like a delight, but I needed to test it with something to be sure.
I went looking for a large Rails project to try this out on. I happened upon the GitLab Community Edition app and cloned the project. I didn't need to actually get the project running, I was just testing the installation time of the gems.
I was looking for a large application.
rake stats told me that it contains 172 controllers, 208 models and 86,019 lines of code. In the Gemfile there are 197 gem dependencies which shakes out to 369 gems in total. I checked out Discourse and Diaspora too, but GitLab certainly had the largest number of gems, so it was perfect to test my theory.
time bundle install --path=./gems --quiet --force --jobs=n 5 times each for n equal to 1 and 4. The median results for each were:
time bundle install --path=./gems --quiet --force --jobs=1 real 4m39.836s user 1m59.692s sys 0m50.291s
time bundle install --path=./gems --quiet --force --jobs=4 real 2m55.857s user 2m0.005s sys 0m47.897s
These tests were run on a MacBook Pro with 2.5 GHz Intel Core i7 and 16GB RAM.
With these results we can see that installing the gems for GitLab in parallel using 4 workers was ~1.6 times faster.
You should run your own tests with your own setup to see if this genuinely will save you time with installing gems. There has been some research into the best number of jobs. I've certainly set the default number of jobs to 4 for the future.
You might be wondering why Bundler doesn't just turn on parallel installation by default. A quick glance at the comments in the source code gives this one away.
# the order that the resolver provides is significant, since # dependencies might affect the installation of a gem. # that said, it's a rare situation (other than rake), and parallel # installation is SO MUCH FASTER. so we let people opt in.
As a final point, I'd like to point out that reading the documentation, even for a project you may use every day, can turn up some interesting options you didn't know you could use. Bundler itself is full of other useful commands and options, check out
bundle gem and
bundle outdated for some ideas of what is available.
At least if you'd not known about the
--jobs option for
bundle install you can now go out there and install your gems faster.