Don’t Mock What You Don’t Own, but why not?

There's a large debate going on in our office right now about how and where you should mock external libraries. On one hand, mocking an external library lets you test the boundaries of your system without testing the external system. On the other, by mocking an externality you create an unnatural opportunity for tests to drift from implementation.

We thought it would be cool to open the discussion to the larger community.

A brief overview of what we are discussing:

1. Should you mock a strongly typed external library?
2. What about something without types?
3. What does the mock vs an integration test provide?

Comments

[Kelly Stannard]

Depends on what you want to accomplish. With test mocks you are trading off certainty for cost and cost increases exponentially as you add more and more certainty.

For example, you could write a test suite that takes 20 hours and you are 99.999999999% certain it will work. Or you could write a test suite that runs in 2 seconds but you are now only 99.9999% certain it will work.

I prefer cheap tests, so I usually follow a flow chart like this:

=> Do I mock it? <=

     Is it slow?
   true         false
 =>yes<=        Do we control when changes happen?
                  true                   false
                 =>no<=                 =>yes<=

You can also do certain things to ensure mocks aren't out of sync. In Ruby you can compare method arity prior to running the tests. In static languages you could probably have the compiler do something similar.