The alternative to shared examples

This is second part in a post series on RSpec's shared_examples, mix-ins and testing in general (pt1 here). In this final part, I will show you how to stop using shared examples in favor of something much better.

In the first part I made these promises:

1. Show how to design and spec mix-ins.
2. Demonstrate how to bring back intention and clarity to specs.
3. Illustrate how to take a more modular approach to specs.
4. Explain how to make it statements correspond to the source being specced.
5. Highlight how isolated and contract-testing can reduce the number of required specs.

I intend to keep these promises by explaining and arguing for a specific speccing philosophy that will not only make shared examples unnecessary, but improve the overall quality of code and corresponding specs in your suite.

Isolated contract-tests

I've argued for preferring isolated tests before in my On testing article. The central argument is that we want to avoid the need to re-setup, re-run, and re-assert behaviors of code in layers lower than the code under test.

The example I use is speccing a controller action. If we don't do isolated tests, we're obliged to execute persistence logic, way outside the scope of the controller action, just because it may be called somewhere deeper in the stack. Where does the rabbit-hole end?! No. We should focus on the code written in one file and speccing it in an isolated manner in the corresponding spec file!

Doing so has a nice side-effect where any failures in specs immediately point to the one corresponding source file where the problem is. And there's also an intangible, but powerful effect on our mindset - it forces us to think about the scope and contract(s)/API of the code structures we work on.

Isolated specs for mix-ins

Let's apply isolated contract-testing philosophy to mix-ins, and let's pay extra attention to David Chelimsky's advice:

1. What logic triggers at-mix-in time
2. What logic is called in response to messages to including objects
3. How is behavior/wellformedness of classes using M in the application (K and L) asserted

The offender

Let's define a naive mix-in and a shared example (boo!) that a suboptimal suite may have, so that we have a punching bag, hehe.

module PerimeterCalculations
  def perimeter
    sides.sum
  end
end

RSpec.shared_examples("an object with a perimeter") do |object, expected_perimeter|
  describe "#perimeter" do
    it "returns the sum of the side lengths" do
      expect(object.perimeter).to eq(expected_perimeter)
    end
  end  
end

class Triangle
  include PerimeterCalculations
end

RSpec.describe Triangle do
  it_behaves_like("an object with a perimeter", Triangle.new(3, 4, 5), 12)
end

class Rectangle
  include PerimeterCalculations
end

RSpec.describe Rectangle do
  it_behaves_like("an object with a perimeter", Rectangle.new(1, 2, 1, 2), 6)
end

The pro of this approach is that we're making damn sure #perimeter works as expected in including objects.
But this comes at a high cost - we don't have a spec file for PerimeterCalculations, the specs we do have in "an object with a perimeter" example group are not clearly linked to the underlying source code in PerimeterCalculations, and are expensive in that they need to perform object setup to run the logic, and we've taken on the shared example structure complexity.

It's possible to achieve assuredness of working code with less cost.

Isolation-testing an includable module

Let's rework PerimeterCalculations spec to an isolated approach. We can't use an actual includer like Triangle, so we'll need to define a dummy_includer_class instead. Needing to do this will immediately expose all dependencies and assumptions about the intended includers for the mix-in, and give an opportunity to spec this, a major win for design and maintainability! In this particular case the otherwise hidden assumption is that the includer responds to #sides, which in turn must respond to #sum.

RSpec.describe PerimeterCalculations do 
  context "when included in a class that defines required method(s)" do
    let(:dummy_includer_class) do
      Class.new do
        include PerimeterCalculations

        def sides
          [3, 4, 5]
        end
      end
    end

    describe "#perimeter" do
      subject(:perimeter) { dummy_includer_class.new.perimeter }

      it "returns the sum of the sides" do
        is_expected.to eq(12)
      end
    end
  end

  context "when included in a class that does not define required method(s)" do
    # it crashes and burns
  end
end

But what of the shared example and the includers, Triangle and Rectangle? Well, again, let's spec source code in any file in its corresponding spec file. Oftentimes just asserting that the module is in the ancestor chain is sufficient (especially if there are no inter-dependencies). No shared example group necessary.

RSpec.describe Triangle do
  it { expect(described_class).to be < PerimeterCalculations }
end

But you may be concerned that there are inter-dependencies. This is arguably the trickiest part with Ruby, where the required inter-dependencies may be provided later, through a different module, meta-programmatically or even dynamically, per-instance. Oftentimes at-mix-in time checks will not be possible, but we should still be able to spec them in the includer. In fact, in the includer we'll be asserting nothing about the mix-in, but instead we'll assert the includer's alignment to the expected API of the module. Here we can substitute shared example group with a simple context:

RSpec.shared_context("conforming to PerimeterCalculations module requirements") do
  def expect_conformity(instance)
    expect(instance.class).to be < PerimeterCalculations

    # Non-instantiation, cheap, may not work if method_missing meta-programming involved. 
    expect(instance.class.instance_methods).to include(:sides)  

    # Instantiation, expensive.
    expect(instance).to respond_to(:sides)
    expect(instance.sides).to respond_to(:sum)
  end
end

RSpec.describe Triangle do
  include_context "conforming to PerimeterCalculations module requirements" do 
    it "conforms", :aggregate_failures do      
      expect_conformity(triangle)
    end
  end
end

expect_conformity is just an ad-hoc helper method, a poor-man's custom matcher. If you can get by with just class-level introspetive asserts, great, but even if instantiation cannot be avoided, over time it's going to be (1*n)+m, rather than n*m specs.

Isolation-testing an extending module

Modules intended for extending usually define class methods which will then metaprogrammatically modify the behavior of the calling class. These are commonly called "macros" and are popular in framework libraries like Rails. Let's define a simple macro-granting module and seek to spec it in an isolated manner.

module Instrumentation
  class UnavailableMethodError < StandardError; end

  def instrument(method_name)
    raise UnavailableMethodError unless instance_methods.include?(method_name)

    original_method = instance_method(method_name)

    define_method(method_name) do |*args, &block|
      # Perform pre-instrumentation actions here
      puts "Before ##{method_name} is called"

      # Call the original method
      result = original_method.bind_call(self, *args, &block)

      # Perform post-instrumentation actions here
      puts "After ##{method_name} is called"

      result
    end
  end
end

Extending a class with this module will grant the class an .instrument macro, which allows instrumenting already defined instance methods. Let's spec what we've written, namely that extending grants .instrument, that instrumenting some method wraps it in puts statements, and that attempting to instrument a nonexistant method raises.

RSpec.describe Instrumentation do
  let(:extended_class) do
    Class.new do
      extend Instrumentation

      def call
        1
      end
    end
  end

  # Since there are no inter-dependencies, we omit a possible `context "when extending a class that defines required method(s)""` here.

  describe ".instrument(method_name)" do
    context "when called with a defined instance method's name as argument" do
      subject(:instrumenting) do
        extended_class.class_eval do
          instrument(:call)
        end
      end

      it "modifies the instance method with instrumentation wrapper" do
        # before
        expect { extended_class.new.call }.to output("").to_stdout

        instrumenting

        # after
        expect { extended_class.new.call }.to(
          output(
            "Before #call is called\n" \
            "After #call is called\n"
          ).to_stdout
        )
      end
    end

    context "when called with an undefined instance method's name as argument" do
      subject(:instrumenting) do
        extended_class.class_eval do
          instrument(:not_defined)
        end
      end

      it "raises a descriptive error" do
        expect { instrumenting }.to raise_error(Instrumentation::UnavailableMethodError)
      end
    end
  end
end

So far so good. Now we can even instrument Triangle's #perimeter, and think about how to spec that.

class Triangle
  extend Instrumentation
  include PerimeterCalculations
  instrument(:perimeter)
end

It's a bit unfortunate that instrumentation call must come after the instance method definition. This could be avoided with prepend-ing of the instrumentation logic, but would disallow validating method presence. There are no solutions, only tradeoffs. At least we can group the asserts for extension and instrumentation together, again, with assert method(s).

RSpec.shared_context("using Instrumentation") do
  def expect_extension(klass)
    expect(klass).to be < Instrumentation
  end

  def expect_instrumentation_of(method_name)   
    # what here?
  end
end

RSpec.describe Triangle do
  include_context "using Instrumentation" do 
    it "is extended with Instrumentation and calls the macro for listed methods", :aggregate_failures do      
      expect_extension(described_class)
      expect_instrumentation_of(:perimeter)   
    end
  end
end

We're almost there. The missing piece is how we could assert that Triangle did indeed call instrument(:perimeter) without instantiating a record and running the method? A suboptimal way would be to introspect the method definition's source and try regex-matching the portions added by introspection. It would be best if calls of .instrument would leave a breadcrumb, some reliable, public indicator that it was called. This can be achieved in a number of ways. One that is discussed in Module Builder pattern literature is that modifications can be made via named modules, and then asserting their presence is a straightforward and transparent approach. But for this simple case we'll expand the public API a bit with an introspection method:

module Instrumentation
  def instrumented_methods
    @instrumented_methods ||= Set.new
  end

  def instrument(method_name)
    ...
    instrumented_methods << method_name
  end
end

Expand the isolated module spec with this detail

it "modifies the instance method with instrumentation wrapper and lists the method as instrumented" do
  # before
  expect { extended_class.new.call }.to output("").to_stdout

  expect { instrumenting }.to(
    change { extended_class.instrumented_methods }.to([:call].to_set),
  )        

  # after
  expect { extended_class.new.call }.to(
    output(
      "Before #call is called\n" \
      "After #call is called\n"
    ).to_stdout
  )
end

And, finally, use it in extended class specs

RSpec.shared_context("using Instrumentation") do
  ...

  def expect_instrumentation_of(klass, *method_names)   
    expect(klass.instrumented_methods).to eq(method_names.to_set)
  end
end

RSpec.describe Triangle do
  include_context "using Instrumentation" do 
    it "is extended with Instrumentation and calls the macro for listed methods", :aggregate_failures do      
      expect_extension(described_class)
      expect_instrumentation_of(described_class, :perimeter)   
    end
  end
end

Summary of mix-in specification

Spec mix-ins like you would spec them in a gem, in an isolated manner, without any real includers yet.
Think about the inter-dependencies of mix-ins and their includers, and how to clarify and document them. Use Module Builder pattern more.
Spec inter-dependencies of mix-ins outside the defining file sparingly, exclusively in contract-spec manner.

Modular, source-colocated, intentional specs.

As I've already demonstrated, writing isolated specs for mix-ins (all code, really) in a corresponding spec file, rather than shared examples, naturally achieves all quality attributes we were aiming for.
For the few "shared examples" we are left with, switching to context groups that define asserting helper method(s) is a way to modularize, clarify and pull the examples back into the specced class.
In fact, helper methods, rather than let and before, are a powerful tool you can use everywhere in your specs, as explained in Joe Ferris' legendary "Let's Not".

It should be said that asserting helper methods can be viewed as a scoped-down variant of custom matchers. A custom matcher will be available in all suite, whereas helper methods can be deployed tactically via explicit shared context inclusion or implicit metadata use.

Do not fret about DRY, focus on KISS.

How to refactor from shared examples to isolated tests

Simple, but not easy. Luckily, stepwise progress can be made, especially the 2nd step, just keep backfilling missing specs.

1. Put a stop to new shared example definition and use. All new mix-ins must be tested in isolation.
2. Write isolated specs for existing mix-ins (existing shared examples may serve as a template here)
3. Identify what, if any, parts of existing shared examples assert behavior of code written in the including class (builder call or macro call(s)). Remove everything else, since it's already specced as part of mix-in's isolated tests.

Conclusion

I hope I've made a convincing case for doing isolated and contract testing, and provided useful examples for approaching common testing scenarios in a way that should leave both the code and the specs in a better state than is common.
I hope you learned something. Let me know what you though of this in the comments!

Further reading

• Caleb Hearth's "Write cleaner, self-documented tests by defining methods in RSpec"
• Thoughtbot dev's "Let's Not", "The Self-Contained Test", and "The Case for WET Tests"