Run Rails 6 System Tests in Docker Using a Host Browser

The problem: we wanted to get Rails system tests running using a browser on our development machines.

The complication: we develop our Rails project within Docker, in order to make every aspect of our development environment reproduceable.

The preference: we want to see the browser-based tests running, but we don’t want to mess around with running browser inside a container and tunneling its display to an external X-Windows server. We already have perfectly good browsers on our host machines, and we’d like to use them natively.

The outcome: we can run Rails system tests inside a Docker container. It pops up Chrome windows on the host machine, and exercises our app!

Coming up with the steps below involved a lot of trial-and-error, research, and debugging. I’m going to skip all that for this write-up, and just break down our working configuration piece by piece.

I’ve also written up an in-depth breakdown of what all these parts are, and how they work together in Rails system tests.

1. Disable the webdrivers gem

The webdrivers gem is a lovely convenience for making sure you have the right browser and WebDriver tool installed before running system tests. However, we don’t want to run the browser inside the container. We want the container to talk to a browser either on the Docker host machine, or in a dedicated selenium sidecar container.

The webdrivers gem was causing system tests to fail with “couldn’t find browser” error. So we disabled it.

# Gemfile
​
group :test do
  # ...
  # webdrivers
end

And then ran bundle install again to update Gemfile.lock.

2. Choose a stable port for the test server

By default Capybara picks a new random port to run the test web server on every time system tests are run. We want tools outside the container to be able to consistently communicate with the test server.

We arbitrarily chose port 3434, and documented that choice with an entry in a brand-new .env file in the project root.

CAPYBARA\_SERVER\_PORT=3434

3. Configure Docker-Compose for external browser testing

Here’s an elided version of our .devcontainer/docker-compose.yml, with the important elements commented.

# Specify a recent version of docker-compose to ensure we have fancy environment 
# variable expansion
version: '3.2'
services:
  app:
    build: 
      context: .
      dockerfile: Dockerfile
​
    environment:
      # Convey which port Capybara should be starting test servers on; fail loudly
      # if we've neglected to configure it.
      CAPYBARA_SERVER_PORT: "${CAPYBARA_SERVER_PORT:-3434}"
      # The test server WILL NOT BE AVAILABLE from outside the container if it binds 
      # to 127.0.0.1. It must bind to 0.0.0.0 to be exposed to the outside world.
      CAPYBARA_SERVER_HOST: "${CAPYBARA_SERVER_HOST:-0.0.0.0}"

    volumes:
      - type: bind
        source: ..
        target: /workspace
        consistency: cached
​
    ports:
      # Expose the test server port. Fail loudly if CAPYBARA_SERVER_PORT isn't set.
      - "${CAPYBARA_SERVER_PORT:-3434}:${CAPYBARA_SERVER_PORT:-3434}"
      # Also expose the usual app port
      - "3000:3000"

4. Configure Capybara to use the test server and port from environment

We added two lines to our test/test_helper.rb:

ENV["RAILS_ENV"] ||= "test"
require_relative "../config/environment"
require "rails/test_help"
​
# Use `fetch` to fail loudly if these variables aren't set. We might relax this 
# and set defaults at some point, but for the moment we want to make sure we didn't 
# miss a step.
Capybara.server_host = ENV.fetch("CAPYBARA_SERVER_HOST")
Capybara.server_port = ENV.fetch("CAPYBARA_SERVER_PORT")
​
class ActiveSupport::TestCase
  # Run tests in parallel with specified workers
  parallelize(workers: :number_of_processors)
​
  # Setup all fixtures in test/fixtures/*.yml for all tests in alphabetical order.
  fixtures :all
​
  # Add more helper methods to be used by all tests here...
end

5. Update the system test base class to drive a remote browser

The configuration so far makes sure that Capybara starts up the server-under-test in a way that external tools can find it and talk to it. But we still needed to tell Capybara to interact with use an external WebDriver, instead of trying to drive a browser inside the container.

We updated our test/application_system_test.rb:

require "test_helper"
​
class ApplicationSystemTestCase < ActionDispatch::SystemTestCase
  parallelize(workers: 1)
  driven_by :selenium, using: :chrome, screen_size: [1400, 1400],
                       options: {
                         browser: :remote,
                         url: "http://host.docker.internal:9515",
                       }
end

Notes:

• Figuring out the right options to pass into driven_by was… nontrivial.
• browser: :remote are the magic options to tell Capybara not to try to talk to a local WebDriver tool.
• host.docker.internal is a special hostname that Docker sets up to refer to the host machine.
• Port 9515 is the standard port that chromedriver listens for commands on. In a future revision I think I’d like to capture this knowledge more explicitly in another environment variable.

6. Install ChromeDriver on the development host

Our first development host happened to be a Windows machine so we used:

choco install -y chromedriver

Instructions for installing ChromeDriver on various platforms can be found at the ChromeDriver project home page.

7. Start the chromedriver

…and tell it to allow connections from anywhere. This is accomplished, unintuitively, by using the --whitelisted-ips flag without any arguments.

> chromedriver --whitelisted-ipsStarting ChromeDriver 81.0.4044.69 (6813546031a4bc83f717a2ef7cd4ac6ec1199132-refs/branch-heads/4044@{#776}) on port 9515All remote connections are allowed. Use a whitelist instead!Please protect ports used by ChromeDriver and related test frameworks to prevent access by malicious code.

Maybe later we’ll figure out how to provide a whitelist for just the dev container.

8. Run system tests inside the container

$ PARALLEL\_WORKERS=1 rails test:system

We found parallel system tests to be flakey, so we disable them with an environment variable.

9. Enjoy spooky automated browser action!