Effortless Integration Testing in Rails: Mock External APIs with RSpec

It's not uncommon for our web apps to have features that make API calls to external services. In fact, if you have an application that is heavily reliant on external services, it can make integration testing a nightmare for you and your team.

In this guide, I'll show you how we can use built in RSpec features like Instance Doubles and Shared Contexts to make integration testing a breeze in our Ruby on Rails applications.

The Scenario

Imagine we're building a project management web app that heavily relies on GitLab for file management. When a user creates a new project in our app, we have to create a project on gitlab.com using their API.

In general, we never want to call an external API during a test. Remember, our tests are there to test that in a given scenario, our code behaves correctly. It would take longer to run, risk exposing API credentials and create unwanted data on our external services if we talked to them during our tests.

Furthermore, it is safe to assume that GitLab has already thoroughly tested that their create project endpoint works. There's no need for our tests to cover that as well.

So, how do we test this kind of code repeatedly and efficiently, without creating a bunch of test data on gitlab.com?

Instance Doubles

The solution is to use Instance Doubles, which are also commonly referred to as Stubs or Mocks. They allow us to modify the behaviour of our code during a test.

For example, let's consider GitLab's POST https://gitlab.com/api/v1/projects endpoint. In our application, we would use an instance of a Gitlab client to make this request

gitlab_client = Gitlab.client(Settings.gitlab.hosted_client)
gitlab_client.create_project({...})
=> {"id" => 1, ...}

So, let's tell RSpec that every time our code calls gitlab_client.create_project({...}), simply return some predefined hash instead of a real response from gitlab.com.

To implement this, we can simply define our expected response at the top of our spec like this:

# define our hardcoded expected response from GitLab
let(:project) { {"id" => 1, ...} }

# define an instance double
let(:gitlab_client) do
  # override the return for the #project and #create_project methods on every instance of Gitlab::Client
  instance_double(Gitlab::Client, project: project, create_project: project)
end

# Gitlab::Client.new will return our instance double from above, instead of a real gitlab client
before(:each) do
  allow(Gitlab::Client).to receive(:new).and_return(gitlab_client)
end

Reusable Instance Doubles as Shared Contexts

We just saw how we can override our gitlab client during a test. But, it would be really tedious to write this out every time our code makes an API call to some service. Luckily, it's pretty easy to write this once and reuse it across tests, thanks to Shared Contexts.

Building on our previous example, let's define a shared context that will allow us to easily reuse our instance double:

# spec/support/shared_contexts/gitlab_client.rb
RSpec.shared_context "with gitlab client" do
  # define our hardcoded expected response from GitLab
  let(:project) { {"id" => 1, ...} }

  # define an instance double
  let(:gitlab_client) do
    # override the return for the project and create_project methods on every instance of Gitlab::Client
    instance_double(Gitlab::Client, project: project, create_project: project)
  end

  # Gitlab::Client.new will return our instance double from above, instead of a real gitlab client
  before(:each) do
    allow(Gitlab::Client).to receive(:new).and_return(gitlab_client)
  end
end

That's it! To use it, we just need to remember to include it in any tests that interact with the gitlab client

RSpec.describe "/projects", type: :request do
  include_context "with gitlab client" # important

  describe "POST /projects" do
    context "with valid parameters" do
      it "creates a new project" do
        # thanks to the include_context above,
        # we can test our create project endpoint
        # without worrying about GitLab
        expect do
          post projects_url, params: { project: build(:project).attributes }
        end.to change(Project, :count).by(1)
      end
    end
  end
end