Stubs vs Mocks in JS

Resources:

• https://martinfowler.com/articles/mocksArentStubs.html
• https://stackoverflow.com/questions/50389981/how-to-properly-mock-es6-classes-with-sinon
• https://sinonjs.org/releases/v12.0.1/stubs/
• https://spin.atomicobject.com/2011/04/28/mocking-objects-with-sinon-js/

Knowledge pre-reqs: Basic CoffeeScript testing implementation

Intro

We see the terms 'mocks' and 'stubs' thrown around a lot in JS testing. There's a difference between them, but sometimes they're used to refer to the same processes, so it can be difficult to tell what it is. This article is my attempt to get closer to the truth.

From Martin Fowler's article on testing, we see that mocks test using behavior verification while stubs test using state verification. He uses a Java example to illustrate his point. In this article, I'm going to use a JavaScript example to show what he means.

First of all, we've got to hash out the difference between state verification and behavior verification in the abstract.

State verification:

tests the state of the object in the system we're testing after the methods that comprise the behavior are exercised.

Behavior verification:

tests the behavior of the system being tested. This requires a little more dissection of the system in that we employ a mock, tell it what to expect during setup, and have it self-verify.

Now let's see how some examples in JS illustrate these definitions.

(Note that the following examples will be a bit over-engineered since I'm testing a JS method in a simple class -- but you can see how this would apply to an API or a system with similar complexity).

State verification example

require('@fatso83/mini-mocha').install()

const sinon = require('sinon')
const { assert } = require('@sinonjs/referee')

class Greeting {
    constructor(message) {
        this.message = message
    }
    greet(message) {
        return `Hello ${message}`
    }
}
describe('stub', function () {
    it('should return a string greeting the user with chosen word', function () {
        const stub = sinon.stub()
        const classInstance = new Greeting()
        const greeting = classInstance.greet('world')
        stub.returns('Hello world')

        assert.equals(stub(), greeting)

        stub.resetBehavior()

        assert.isUndefined(stub())
    })
})

Here, my stub is created using the sinon library (I'll also use this for my mock example). You can see that I'm checking the state of the value returned from the greet method once it's done executing against the stub in this line:

assert.equals(stub(), greeting)

A Bump in the Road

Now, how would I execute this test using mocks? This example was much more difficult to execute -- among the reasons why is that many developers use the term "mock" as an umbrella for a lot of testing tactics, including "stub". So when I looked up how to do this with mocks, I ended up seeing examples on how to stub!

I ended up with an example that I couldn't get to work after several hours, so I turned to my friends at Virtual Coffee in the #help-and-pairing channel for assistance.

Nikhil Seth refactored my code to the following solution (notably adding the param '1' to my .once()).

Mock verification example from Nikhil Seth

require("@fatso83/mini-mocha").install();
const sinon = require("sinon");

class Greeting {
  constructor(message) {
    this.message = message;
  }
  greet() {
    return `Hello ${this.message}`;
  }
}

describe("Test greet using Greeting", () => {
  it("should verify Greeting.greet", () => {
    const newGreeting = new Greeting("world");
    console.log("GREETING INSTANCE", newGreeting);

    // Test expects that the `greet` method of newGreeting
    // should be called once.
    const mockedGreetMethod = sinon.mock(newGreeting).expects("greet").once(1);
    console.log(mockedGreetMethod);
    // Calling the greet method
    newGreeting.greet();
    // Checking if the greet method's expectations are met
    mockedGreetMethod.verify();
  });
});

As you can see, this example employs behavior verification, testing the behavior of the system being tested. We use a mock, tell it what to expect (that the greet method is executed once) during setup, and have it self-verify using the .verify() method.

Conclusion

I learned a lot about stubs vs mocks that I never would have, if I hadn't set out to write this blog post! I like to write to synthesize my thoughts since it makes me concretize ideas with examples. Otherwise I might just read a generalized blog post and come away with a fuzzy understanding. If you're learning a new technical concept and you work with a few examples, you'll understand the general concept much better since you've seen it instantiated in different environments.

PS-- many thanks also to Ray Deck for asking pertinent questions when I was stuck!

Comments

[peerreynders]

One thing that may be worth considering: Practices inform tools but tools rarely inform practices.

To some degree your interpretation is guided by Sinon's implementation of a test double tool which ultimately reflects the vision of the Sinon.JS developers.

Martin Fowler references Gerard Mezaros's test double categorization:

• Dummy Object
• Test Stub/Spy
• Mock Object
• Fake Object

In particular (xUnit Test Patterns, p. 133):

• A Test Stub is an object that replaces the real component on which the SUT (System Under Test) depends so that the test can control the indirect inputs of the SUT. It allows the test to force the SUT down paths that it might not otherwise exercise. A Test Spy, which is a more capable version of a Test Stub, can be used to verify the indirect outputs of the SUT by giving the test a way to inspect them after exercising the SUT
• A Mock Object is an object that replaces a real component on which the SUT depends so the that test can verify its indirect outputs.

So

• A Stub is focused entirely on the value that is returned (i.e. indirect input to the SUT).
• A Spy also captures the passed inputs like argument values with each invocation ( i.e. indirect outputs from the SUT).
• A Mock goes even further as expectations can be configured directly on the test double itself — the test can then simply execute a verify command on the Mock to ensure that all expectations have been met at that point in time.

To some degree Fowler's article juxtaposes the Classical (Detroit, Chicago) vs. Mockist (London) schools of TDD (in 2007) and alludes to the fact that mockist tests can have the tendency to couple tests to implementations rather than behaviour (contract, protocol) which can lead to fragile tests.

In 2021 Ian Cooper actually goes as far as saying that mocks should only be used to solve problems of shared fixtures and slow tests rather than isolating dependencies.

[Lucia Cerchie]

This reply is a blog post in itself! I learned so much about spies! Thank you!

[Nikkhiel Seath]

Thank you for explaining the difference in simple terms.

So, I should use:
A stub if all I care about is the final change in state/value.
and a mock if I care about both the final change in state/value along with how that change was brought.
E.g.: a function that returns a certain value and has a timeout / retry functionality until a certain condition is met and if:

• I care about just the returned value: I shall use a stub
• I care about the returned value + the no. of retries: I shall use a mock

Like, I shall use a mock if I have a function that generates unique ids and part of its process is to re-generate a new id if the currently generated id already exists.
And in this case, I shall like to test:
a) That I get a unique id
b) If my dataset already contains the generated id then, I need to confirm that the generation process is called more than once until a unqiue id is found.

[Lucia Cerchie]

Wow such a thoughtful response! Another great example of the difference.