This should be an easy question to answer, right? Turns out that there are several definitions of unit testing, and even though they are similar, they are also different enough that answering this question is difficult.
Let's look at some examples. Form Wikipedia:
In computer programming, unit testing is a software testing method by which individual units of source code, sets of one or more computer program modules together with associated control data, usage procedures, and operating procedures, are tested to determine whether they are fit for use. Intuitively, one can view a unit as the smallest testable part of an application. In procedural programming, a unit could be an entire module, but it is more commonly an individual function or procedure. In object-oriented programming, a unit is often an entire interface, such as a class, but could be an individual method.
So, according to Wikipedia a unit can be "an entire module", "an individual function", "an entire interface/class", but also "an individual method".
From MSDN:
The primary goal of unit testing is to take the smallest piece of testable software in the application, isolate it from the remainder of the code, and determine whether it behaves exactly as you expect...
A little less vague, a unit here is described as the "smallest piece of testable software in the application". This still leaves room for debate.
Martin Fowler acknowledges that unit testing (and consequently a unit) is not tightly defined, however finds these common elements in definitions:
...Firstly there is a notion that unit tests are low-level, focusing on a small part of the software system. Secondly unit tests are usually written these days by the programmers themselves using their regular tools - the only difference being the use of some sort of unit testing framework. Thirdly unit tests are expected to be significantly faster than other kinds of tests.
One of my favourite definitions is from Roy Osherove's The Art of Unit Testing book where Unit testing is defined as:
A unit test is an automated piece of code that invokes the unit of work being tested, and then checks some assumptions about a single end result of that unit. A unit test is almost always written using a unit testing framework. It can be written easily and runs quickly. It's trustworthy, readable, and maintainable. It's consistent in its results as long as production code hasn't changed.
There's also the definition of "unit of work":
A unit of work is the sum of actions that take place between the invocation of a public method in the system and a single noticeable end result by a test of that system.
That end result is the method returning a value, some public state of the class changing or the public method being tested invoking another method in a dependency.
I really like this definition, however, I find the choice of using the term "unit of work" unnecessary, especially because it is a loaded term. For example, when discussing Object Relations Mappers (ORMs) the term "unit of work" has a completely different (and well established) meaning that is misleading in this context.
The "unit"
Can we agree that when we are writing a unit test we call a single method on the class we are testing? If so, then why can't we consider a unit to be the logical path (path that the execution of the code takes) through that method given our initial setup of the test.
The tests need to be deterministic, so for the a given setup the logical path will always be the same.
Let me show you an example using an Account class and a Withdraw method. If there are enough funds, the amount withdrawn is taken from the account's balance, if not an InsuficientFundsException is thrown.
There are two possible logical paths through the Withdraw method, one where there are sufficient funds, and another were there aren't. If we want to be thorough, we should write a test for each.
A little taste of test first development
Let's have a look at how this definition works while we write the Deposit method for our account. And let's do it test first.
For the first test we want to make sure that the balance gets updated when we make a deposit of a positive value, here's the test (I'm using NUnit):
And here's the implementation:
If the caller specifies a negative amount we want to throw an InvalidOperationException. Here's the test for that:
And here's the new implementation of the Deposit method:
We can see that up until now we have a test per logic path. We can add an extra test to cover the case where the amount is zero. However, I'd argue that that test exercises the same logical path through the code (unit) than the positive amount test (however, no harm will come in having that extra test).
If we want the method to behave differently when the amount is zero, then we'd have to add another if statement, and that would produce another possible logical path through the method, for which we could write another unit test.
You might be thinking these examples are super simple. How would something more complex, where there are other classes/dependencies involved fair? Does this notion of a unit being a logical path through the code hold?
Imagine that our Deposit method has this requirement of alerting the fraud department if the deposit looks suspicious. Let's write our test for that, starting with a "suspicious" deposit (I'm using moq as my isolation framework):
And now the test for when the deposit isn't suspicious:
And the new implementation of the Deposit method:
You've probably noticed that I added a few dependencies (IFraudDetectionService and INotificationService). We'll get to that. But first, what happens to the tests we previously had?
We have to adjust them so that they take into account the new dependencies, but should we keep all of them? Maybe we can drop the Deposit_PostiveAmount_AdjustBalance test.
If we look at all the possible logical paths through the Deposit method we can see that the there's no way that that test fails in isolation (path 2). If the balance is not updated correctly, both Deposit_SuspiciousAmount_UpdatesBalanceAndSendsEmailToFraudDeparment and Deposit_NonSuspiciousAmount_UpdatesBalanceAndDoesNotSendsEmailToFraudDeparment fail.
On the other hand, one can make the argument that we should have all tests, one for when the amount is negative, one that only asserts that the balance is updated correctly, another that verifies that the notification service is called when the deposit is suspicious, and another verifies that the notification service is not invoked when a deposit is not suspicious.
If we want that our definition of unit takes this into account, we can say that a unit is a verifiable change that occurs in a logical path through a method. That can be a change of internal state that is externally visible (e.g. the Account's Balance), a method invocation in a dependency (e.g. verify that the notification service was called) or the return value of the method.
The advantage of doing this is that it is easier to figure out what is wrong just by looking at the name of the failing test.
One thing you might have noticed is that this idea of logical path is very similar to a measure of software metric named Cyclomatic Complexity. A method's cyclomatic complexity is "the number of linearly independent paths through the code". In fact, right now, as I read the Wikipedia entry, I've just noticed that author the Cyclomatic complexity metric (Thomas McCabe) had this exact same idea (one test per logical path) as a testing strategy in 1996 (way before unit testing was popular). It is named basis path testing.
The dependencies
This is a bit of an aside, but you should have noticed that I used interfaces so that I did not have to specify a specific criteria for when a deposit is considered suspicious, or how the "fraud department" is notified.
This not only made the code easier to test, it also has the advantage of isolating the Account class from changes in the criteria used to determine if a deposit is suspicious or how the notifications should be performed.
I brought this up just to mention that if you write the tests first you'll have these problems in mind, you'll want to write your code so that it is easy to test. You will feel compelled to write small focused methods, and be very mindful to what does and doesn't belong in a class and should be considered a dependency.
Also, you should be in full control of the code that executes through the logical path you are testing. For example, in Deposit_SuspiciousAmount_UpdatesBalanceAndSendsEmailToFraudDeparment I used moq to create a stub of the IFraudDetectionService interface that always returns true when IsDepositSuspicious is called.
Say the implementation of that dependency made a call to a database or some other service over the network. That would make that test be considered an integration test. However, I'd argue that even if no database call was made, or any service called, you should still use the isolation framework and create the stub. It removes uncertainty. When the test fails and you are in full control of what was executed, you don't have to guess what went wrong.
Top comments (5)
Your last point is interesting: people write integration tests when they encounter code that has side effects and they want to have the side effects tested. Another common strategy is to use mocks (preferably generated ones) via e.g. Mockito in Java. This is a great way to test code paths where the side effect is the most interesting thing that happens in a unit tests without making it an integration test.
Unfortunately, lots of developers don't appreciate the futility of doing code coverage tests with integration tests. The whole point of a unit test is that the units are small and there is a small and finite number of paths to test so you can achieve a high degree of test coverage with little effort. This is not true for an integration test. The more stuff you gobble together the more code paths you have and it quickly explodes beyond the threshold of having reasonable coverage.
It is for that reason that I prefer to write either an end to end, blackbox integration test that is about testing realistic user scenarios against something that closely resembles the shipped product configuration, or a pure unit test that is about covering all relevant code paths. Anything in between neither achieves the goals of coverage nor of realism and tends to be a waste of time and resources.
If you are mocking half your system and then running some in memory database, your tests are not very realistic and your code coverage is likely to be poor. So, why would you do that exactly?
My personal experience is that this kind of tests become more of a burden than benefit in a long run. I should know, I've written thousands of tests like in this post. The number one reason why I want tests is safety when refactoring the solution. However, tests with too small unit don't support this. It' rare that refactoring happens in such a limited scope and therefore tests need to be modified simultaniously with production code, which scares me.
Nowadays I write unit tests with larger scope and I recommend giving it a try!
For further reading: taimila.com/blog/unit-of-unit-test...
I think unit tests are not always appropriate. If you're trying to test code which is largely business logic I'd go with unit testing, for example. If you're trying to test code which is very network intensive I would do integration testing or something along those lines instead. You end up doing so much mocking in those situations that it becomes apparent that you're making a lot of assumptions (i.e., the benefit is negligible).
Pick the right tool for the job I suppose.
To keep it simple to explain I tend to say that unit tests should be the smallest piece of code possible (like a function) and should not do any IO. To me these are the most important aspects of unit testing.
Thanks for making it clearer.