Many developers have one of two opinions about unit tests: they believe 100% of the code has to be covered by unit tests, or they believe 100% of the code has to be covered by unit tests but they have reasons for not writing any.
It isn’t common to find people who dislike the idea of unit tests. The main cost of having unit tests is spending time to write them. The benefits are outstanding. Let’s say I spend 5 hours writing 50 unit tests for a piece of code that would have taken me 1 hour to test manually. Spending 5 hours to save 1 hour of work doesn’t seem very good.
Yet those unit tests will run in seconds. There is a somewhat heavy upfront cost to writing them, but every time a developer has to go back to that code, they save an hour. The break even on the time investment is having to go back to the code 5 times, which is rather low over a long time period.
There’s also the benefit that a new developer in that code base will either not know how to properly test it or will use up a lot of another developer’s time in a knowledge transfer. Either creates a high probability for that new developer to create new bugs while fixing old ones in that code. Unit tests prevent that from happening.
The problem is that these benefits are usually long term. It’s hard to argue that unit tests won’t save time in the long run, but many developers will opt out of writing them due to time pressure in the short term. I’ve lost count of how many times I’ve said “We’ll write unit tests after this release…” Yet after every release starts development for the next one. There is rarely downtime to go back and write unit tests.
I have found though that unit tests do have a number of short term benefits. The trick is recognizing when a task will be done faster with unit tests than without. The best examples of this are math heavy problems. Anything like a recommendation engine, a physics formula, or image processing would fit in here. There are other examples such as processing spreadsheets, sanitizing HTML content, managing complex user permissions, etc.
What all of these examples have in common is that they have many many test cases. To test any of them manually would require performing a dozen or more actions to capture every case. Some may even have more test cases than a single person could possibly hold in their mind at once. The result is that a test case is likely to be missed in that developer’s testing. Maybe that bug gets found in QA, which wastes a bit of time. Maybe that bug gets released and users find it. That’s much worse.
The developer could resolve this problem by writing every test case down. But then how much time is spent going through all of them? How many times will a developer go through all of them? We often give a lot of attention to bugs in software after development has been completed, but there are plenty of bugs that appear during development. For something complex, a developer may need to tweak their code dozens of times. Even if it takes only 5 minutes to run through every test case in their list, that adds hours to the initial development. Hours of tedious testing.
We’re not looking at saving a bunch of time over the long run now. We’re looking at saving time today.
Some developers will start writing little scripts to automate some of this testing. I’ve done this plenty of times in the past. But unit tests are just another form of these scripts. One that is much more reliable. If we’re going to spend time writing scripts we’re going to throw away, why not spend that time in making unit tests instead?
Another short term benefit of unit testing involves technical debt. Often times not having unit tests is considered technical debt. However, unit tests can be used to enable you to take on more technical debt.
To understand how, we should look at where time is spent in software development. For anything complex, very little of it actually goes into typing the code. The big chunks are:
- Figuring out how the code should work ahead of time
- Figuring out all the ways to test that code
- Fixing all the (initial development) bugs with the code
When a developer is pressed for time, they take shortcuts with Chunk 1. They hack something together and make it work. It may not be pretty or efficient, but it works. The only way to tell that it works is by not taking shortcuts with Chunks 2 and 3.
A large part of Chunks 2 and 3 can be accelerated with unit tests. If a developer takes shortcuts with the initial development of the code, unit tests can significantly reduce the time it takes to clean up those shortcuts since they do not have to repeat all the work done in Chunks 2 and 3.
This sounds like a long term benefit, but having the safety net of unit tests means that shortcuts can be taken intentionally and safely. A lot more technical debt can be taken on with the knowledge that the cost of cleaning it up will be significantly reduced thanks to unit tests. That’s a huge boon in the short term.
Top comments (8)
I agree that unit tests have a lot of benefits, but what do you think about the disadvantages?
It's true that it saves time in the long run, but what if I work on a tight schedule and have no time to write unit tests?
The article was about the short term benefits of unit tests. The idea is that in some situations, it is faster to do initial development with unit tests than without.
For example: I recently wrote an email reply parser (splitting one email into many from the chain of replies quoted in it). This has dozens of test cases. The code for it is also very complex and has some messy regular expressions. I wasn't going to send an email every time I wanted to test things. I wrote some scripts with the email text in them. At this point, I realized I could convert those scripts into unit tests in less than 5 minutes. Running my script manually and maintaining those test scripts was a lot more cumbersome than unit tests.
This wasn't the long term. This was over the course of a few days. Those unit tests saved me hours of manual testing. This was a situation where if the schedule is too tight for unit tests, then the schedule is too tight to get the thing working at all.
Does that mean unit tests save you time when building basic CRUD apps? Not really. Those unit tests really are long term projects. But unit tests are almost always worth it with complex code.
I'm on your side - I'm also a big fan of unit testing.
I write unit tests whenever I can, but sometimes there is just not enough time..
About this part: "There’s also the benefit that a new developer in that code base will either not know how to properly test it or will use up a lot of another developer’s time in a knowledge transfer."
Don't you think it's a drawback? if a new developer needs to write unit tests and he doesn't know how to properly write them, he will waist both his and other developer's time..
Ah, sorry. I wasn't clear there. My point was that if unit tests already exist, it helps a new developer a lot by serving as documentation on the test cases they need to think about.
No doubt about that!
What are the unit testing frameworks that you're familiar with?
I'm a big fan of stock testing frameworks. Go and Python have decent ones.
I've worked with others, but one thing I really appreciate is simplicity. Granted, Go's testing framework is so bare bones it doesn't come with assertion functions, but that doesn't bother me that much.
I think complexity in set up or in trying to create the first test creates a lot of resistance to newcomers from building unit tests. Go and Python get you started in minutes. Frameworks tend to take a few hours at a minimum. They also have more random "gotchas" that result in lots of time on stack overflow.
I work with Typemock Isolator for C# - it's a very simple and user friendly tool, that allows you to perform unit testing by mocking almost anything you need.
It didn't take me a lot of time at all to understated how to work with it.
Have you heard about it?
Another often overlooked benefit of unit tests---at least well-written unit tests---is it can help expose flaws in your designs and abstractions. If writing tests is a struggle and you're forced to go through vast amounts of setup or mock many different kinds of objects, or worry about the internals of the implementation while testing the public methods, that's probably a sign that the design should be rethought.