TLTR
Mocking is often seen as a default choice when writing tests. Yet it might introduce unnecessary complexety to your system. There are other approaches to manage dependencies in tests.
More
What is mocking?
Mocking — creating objects that simulate the behaviour of real objects.
Here is how mocking looks in C# in a nutshell (JustMock library):
// Instantiate a new mock
var mockContactRepository = Mock.Create<IContactRepository>();
// Set up the mock and it's return value
Mock.Arrange(() => mockContactRepository.GetContacts())
.Returns(new List<Contact>
{
new Contact { ContactId = 1 }, new Contact { ContactId = 2 }
});
// Pass mock as a dependency
var contactManager = new ContactManager(mockContactRepository);
Although it sounds very useful it has to be taken with a pinch of salt.
Shortcomings of mocking
1. Runtime instead of compile-time feedback on changes
If we imagine a test that has Setup() but doesn’t have a return value. When we add a return value, mock doesn’t suggest to add a return type. We only find out about it when running the test.
Here is the example of this shortcoming in C# (Moq library)
public class CarsControllerTests
{
[Fact]
public async void testCreateCar()
{
var repositoryMock = new Mock<ICarRepository>();
// No return value set
repositoryMock.Setup(r => r.Create(car));
var carsController = new CarsController(repositoryMock.Object);
var car = new Car() { Name = "BMW", Available = true };
var result = await controller.Create(car);
// Use return value on result
Assert.Equal("BMW", result.Name);
}
}
The test has built sucessfully but the test will not pass. The reason is no return value set. The controller relies on returned value from the repository.
>> dotnet test
Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
[xUnit.net 00:00:01.0309627] Cars.Tests.CarsControllerTests.testCreateCar [FAIL]
Failed Cars.Tests.CarsControllerTests.testCreateCar [94 ms]
Error Message:
System.NullReferenceException : Object reference not set to an instance of an object.
Stack Trace:
at Cars.CarsController.Create(Car car) in /Users/kondrashov/Projects/unit-testing-mocking/src/Cars/Controllers/CarController.cs:line 20
at Cars.Tests.CarsControllerTests.testCreateCar() in /Users/kondrashov/Projects/unit-testing-mocking/test/Cars.Tests/CarsControllerTests.cs:line 20
at System.Threading.Tasks.Task.<>c.<ThrowAsync>b__128_0(Object state)
Failed! - Failed: 1, Passed: 0, Skipped: 0, Total: 1, Duration: < 1 ms - Cars.Tests.dll (net7.0)
We get a null reference exception at return car.Id.ToString() To make the test happy we need to use ReturnsAsync() method on our mock:
// Set return value
repositoryMock.Setup(r => r.Create(car))
.ReturnsAsync(new Car { Id = "1", Name = "BMW", Available = true });
It seems easy and straightforward to change the mock above. But more compex methods become less trivial. The value that this test delivers becomes less with all the time that we spent for it’s maintance.
We would want to know when something broke at the compile-time, instead of runtime.
2. Tight coupling to contracts
Mocking is coupled with contracts by nature. It makes any refactoring harder, because you have to change all realted mocks.
Below I have about 6 different services in my mocking. Refactoring contracts of below services will result in breaking all these mocks. Multiply the breakages by number of test files where you set up mocks.
3. Short System Under Test
System Under Test (SUT) is shorter when we mock a dependency:
A good alternative to such test is an Integration Test with longer SUT:
4. Long-winded set up in each test
A testing framework usually allows you to group your set up. However it’s not always the case as each test often requires a dedicated set up. Below is an example of how each test requires a code to set up mocking:
[TestClass]
public class MyTestClass
{
private Mock<IMyInterface> _mock;
[TestInitialize]
public void Setup()
{
_mock = new Mock<IMyInterface>();
}
[TestMethod]
public void Test1()
{
_mock.Setup(m => m.MyMethod()).Returns("Hello");
var result = _mock.Object.MyMethod();
Assert.AreEqual("Hello", result);
}
[TestMethod]
public void Test2()
{
_mock.Setup(m => m.MyMethod()).Returns("World");
_mock.Setup(m => m.AnotherMethod()).Returns(42);
var result1 = _mock.Object.MyMethod();
var result2 = _mock.Object.AnotherMethod();
Assert.AreEqual("World", result1);
Assert.AreEqual(42, result2);
}
[TestMethod]
public void Test3()
{
_mock.Setup(m => m.MyMethod()).Returns("Goodbye");
var result = _mock.Object.MyMethod();
Assert.AreEqual("Goodbye", result);
}
}
5. You can’t mock just any method
We mock methods. And they have to be public to be eligible for mocking.
Yet certain frameworks allow to you mock private methods using reflection. This is wrong as it break the incapsulation of your design. Example below in Mockito in Java:
when(spy, method(CodeWaithPrivateMethod.class, "doTheGamble", String.class, int.class))
.withArguments(anyString(), anyInt())
.thenReturn(true);
What can I do instead of mocking?
1. Integration tests
Write an Integration Test, not a Unit Test. There is more value in writing an integration test instead of a unit test with mocks. We cover more code by writing less tests.
2. End-to-end tests
Integration test might rely on an external dependency. In this case you can’t rely on that dependency since you don’t control it. Write an end-to-end test where you would hit your system as if you were a customer of this system.
3. Stubs
A stub is a small piece of code that takes the place of another component during testing. The benefit of using a stub is that it returns consistent results, making the test easier to write.
If you can’t write an end-to-end test due to dependecies you can’t control — use Stubs. It simialar to mocking, yet it provides a type checking when you introduce changes to your system. Below is an example of a simple stub for a car repository:
public class FakeCarRepository : ICarRepository
{
public async Task<Car> Create(Car car)
{
// Any logic to accomodate for creating a car
return new Car();
}
public async Task<Car> Get(string id)
{
// Any logic to accomodate for getting a car
return new Car();
}
}
Another advantage the test becomes cleaner. All setup in extracted into a separate file:
[Fact]
public async void testCreateCar()
{
var car = new Car() { Name = "BMW", Available = true };
var controller = new CarsController(new FakeCarRepository());
var createdCar = await controller.Create(car);
Assert.NotNull(createdCar);
}
Summary
The best approach depends on the specific needs of the project. Consider all trade-offs between mocking and non-mocking. There are other alternatives to mocking out there which you might enjoy.
Top comments (5)
I think this article discusses Unit Testing vs other test types, rather than the use of Mocks specifically.
You make an excellent point that unit tests are not the best type of test in every situation. However, they are a foundation of automated testing. When describing automated testing, many often refer to the testing pyramid. Here, we have layers of test types. In decreasing quantity, we have:
The idea being that a project has a large number of unit tests (for classes that are easier to test). Next, we have integration tests where (as you describe) multiple components are tested together. Next are end-to-end tests (e.g. it's difficult to set up an API controller in a test). Finally, we admit there are some things that are very difficult to test automatically (e.g. some languages don't allow mocking of mouse input events), and have a few manual tests for that.
Great point about the test pyramid, Anthony.
We should be having more unit tests than integration tests as per the test pyramid. Yet often times I see that tests scew towards the unit testing + mocking. Hence this article is a reminder that we also can write other types of tests to better balance the pyramid.
Yes - good point about the distribution of test types.
I suspect that the prevalence TDD has something to do with that. I'm not saying TDD is either good, or bad - I'm just saying that it encourages writing unit tests and then filling in the implementation.
With so many jobs on the market that want TDD as a requirement, my guess is that many people learn to do TDD with unit tests and don't think much about the other possibilities.
I agree, TDD is having an effect on this. Although I've done TDD in the past where I wrote couple integration tests which was convinient.
It's definitely worth testing a chain of components when it's the easiest way.