I’m currently writing a series of articles about test frameworks, and it got me thinking about the importance of software testing. Ever since I started writing automated tests almost a decade ago, I’ve found them to be one of the most useful tools for building maintainable software, but I still meet developers who don’t write tests.
Sometimes it’s because their boss (or the business team) won’t give them time, sometimes it’s because they’ve never been told about the benefits of testing, and sometimes they just don’t care.
Fortunately, if the reason you’re not writing tests is one of the first two, there’s hope. You can point to real business value generated by testing, and it doesn’t take a huge software system or a long period of time to do so.
Just remember that your time as a developer is incredibly valuable , so when you say that you can save yourself (and future software developers) time, your manager can attach real dollars to the problem. In this post, I’ll introduce you to automated software testing and give you the reasons you can take to your boss to make the case for writing tests.
What is Automated Testing?
Before I jump into the benefits, I want to make sure you understand what I mean when I say “automated tests.”
Software can be tested manually (traditionally, this means passing your work to a quality assurance team), and it can be tested automatically. In the latter case, developers write code that ensures their code is working. These automated tests are often run using a command-line interface, and almost every programming language has a testing framework available.
Unit tests are probably the most widely talked about form of automated test, but there are others. I’ll briefly cover them here, but it will require a dedicated article to do this topic justice.
- Unit tests focus on the smallest possible unit of code - typically a function or class. They are fast to run, and you will usually have a lot of them.
- Integration tests ensure that interlocking pieces of your application work together as designed. Sometimes this means testing the layers between classes, and sometimes this means testing the layer between your database and application.
- Acceptance tests are a slightly broader form of integration test. In practice, I use this term to refer to service-level tests.
- End-to-end tests ensure that the entire system works as designed. This means that the frontend and backends are working and talking to one another to accomplish a series of user stories. These are the most time-consuming tests to run, and typically you won’t write as many of them.
-
“Smoke” tests are final checks that ensure your recently deployed code is working as intended. They could be as simple as a ping to your API after a deployment to make sure it responds with a
200 OK. - Performance tests ensure your application meets its SLAs by testing how long it takes to handle requests under load. These may be run on every production deployment or - because they’re so complicated and expensive - just when major new features are released.
If you’re interested in specific examples of testing patterns, I’ve written about API testing and microservice testing in the past. Those posts should be an excellent place to turn next if you want examples you can apply to your application.
Why Software Testing is Important
The business problem with testing is that it’s not immediately apparent that you need them.
You write a piece of code, and you know it works. You just wrote it. You just ran it…Six months pass. It is time to refactor your code. You would kill for some tests. - Swizec Teller, You don’t need tests
The diagram below illustrates this point. Initially, the cost of writing no tests or manually testing your code is much lower than writing automated tests. This is because setting up an automated test suite, and getting developers familiar with a test-driven workflow will take some time.
Eventually though, maintaining code without tests will lead to costly production bugs, and a manual testing strategy scales linearly (as you add more code, testers must spend more time doing tests). So in the long-run, automated tests make maintaining your code cheaper.
But why is this the case?
The Benefits of Testing
Let’s look at testing from the standpoint of a developer. Once you spend six months on a team that’s consistently writing automated tests, you’ll see the benefits. Here are the things that stand out to me:
1. Testing Minimizes Bugs
Bugs are expensive, and production bugs are the worst offenders. According to the IBM System Sciences Institute, fixing a production bug costs 100x more than fixing one at design and over 15x more than fixing a bug at implementation. Automated tests can help developers catch edge-case bugs before they make it into production and force other developers to drop everything and fight fires.
2. Automated Tests Prevent Regressions
In addition to catching new bugs, a strong automated test suite can help prevent regression. As Eric Elliot says, “Manual QA is error-prone…It’s impossible for a developer to remember all features that need testing after making a change to refactor, add new features, or remove features.”
Testing becomes even more important when multiple developers are working on the project over many years. Newcomers can’t safely work on code that doesn’t have tests. While you can rely solely on manual tests, this cost grows linearly as the number of features increases, while automated tests can be written once and run very frequently for little to no cost.
3. Writing Testable Code Improves Overall Quality
Code quality is another long-term investment that pays off for large software systems. Unit testing can dramatically improve quality when working with developers who are still learning about encapsulation, dependency injection, and scoping, and it is even more advantageous in weakly typed languages.
When every new class must have unit tests, it forces developers to stop and think about their architectural choices.
4. Tests Enhance Documentation
Good code should be easy to read and at least partially self-documenting, but there’s almost always room for use-case based documentation. That’s where testing comes in. Good test suites give other developers (or maybe just future you) a better idea of what the code was intended to do.
5. Tests Help Guide Code Reviewers
As an important part of the development process, reviewing code can be tedious. Having tests gives the reviewer a place to look for potential errors or missed edge cases. During code reviews, I often start with the tests, ensuring that they are well-written and don’t miss any important cases before I look at the actual code.
6. Tests Make It Easier to Add New Features
Code gets naturally harder to change the more interconnected and older it is. Tests counter this tendency towards calcification by helping developers add new features more confidently. As a new developer, changing older parts of your codebase can be really scary, but with tests, at least you’ll know if you broke something important.
7. Tests Can Help You Debug Edge Cases
Finally, I use tests to help debug edge cases that show up in production. For example, if we start to see a problem in the logs that wasn’t showing up in testing, I’ll try to write a test case that causes the same error. Once reproduced in a test, I use this as a blueprint to fix the issue and ensure it doesn’t show up again.
How to Get Started Testing
If you’re brand new to testing, this all probably sounds a little intimidating. How do you set up a test suite? Should you do purely test driven development? What if your team isn’t supportive? What about adding tests to a legacy application?
These questions can be overwhelming, even for an experienced software developer.
A few years ago, I started contributing to an open-source project that had almost zero test coverage. Worse, there were a few tests that the original maintainer had written, but he hadn’t bothered to fix as the library evolved. The tests were now failing, but the application was working.
I started to consider my options:
- Should I try to put a stop to new feature development until we got the tests fixed?
- Should I stop using the library until it meets the quality standards I wanted?
- Do I try to rewrite the whole thing?
Extreme solutions like this can be tempting, but they rarely make good business sense. Implementing tests on an untested app is a process. Much like the technical maturity level of a startup, you can’t expect every application to be perfect, but you can make gradual improvements to the test coverage and testability of an application as you work on it.
Here’s how I approached the problem:
1. Start With End-to-End Tests
Unit tests usually require refactoring, so while they’re great when you’re building an application from scratch, it’s easier to start by writing higher-level acceptance or end-to-end tests for a working application. An automated test that makes sure all the critical features work the whole way through is a great place to start.
2. Test Return Types
The next thing to test is the most mission-critical methods in the most important classes. Even if the code is a spaghetti string mess, you can start by testing the return type from the method. If mocking dependencies isn’t possible, then it’s okay to write some impure tests to get started. Refactoring isn’t safe until you’ve got at least some tests in place.
3. Add Tests One Piece at a Time
Unit testing a large application takes time and patience. Install a code coverage tool, and start with one small piece at a time. If it’s a big project and it’s going to take a few weeks, so try doing a few classes per week rather than taking the whole thing on at once. This is a good thing for developers who are new to a project to do as it helps them get familiar with the application without being required to change core functionality.
4. Start Refactoring
Once you feel good about the high-level test coverage, it’s time to decide if refactoring is required or not. With automated tests in place, it’s much easier to tell if you break something as you refactor the code to support unit tests.
Further Reading
Test Driven Development: By Example - Kent Beck’s classic on test driven development is a great read, even today. While the languages, frameworks, and tooling have changed, the core principles of software testing have remained largely the same since Beck wrote the book in 2002.
Refactoring: Improving the Design of Existing Code - Martin Fowler is one of my favorite technical authors, and Refactoring does not disappoint. I read the original version and then bought the new 2018 edition when it came out because the examples were rewritten in JavaScript.
Top comments (2)
This is a really great article. It's well written, and condensed but not oversimplified. Kudos for the great work on it.
I do have a minor complaint: I think the "Cost of Testing Graph" doesn't capture the cost of maintaining automated tests. In the graph, the cost levels out, but in real life, you'll need to keep that automation up to date, along with the need to write more tests as functionality is added. Again, a minor grievance, but figured it's worth mentioned.
"Tests counter this tendency towards calcification by helping developers add new features more confidently." I really like this point and haven't considered it before. By having code that has automation, devs are more likely to take chances with working in it, and therefore it's more likely to be kept up-to-date.
"Start With End-to-End Tests"
I very much agree with this strategy for the points you've made. Trying to write unit tests for spaghetti code is not fun and makes you write really poor tests.
Again, great article and I wish it got more views. Going to share it with my front-end testing newsletter though, so hopefully you'll get some more through there :)
Thanks @klamping !
Your point about my graph is spot on. While the cost of maintaining automated tests is significantly lower than manual testing over time, it does continue to grow (at a slower rate) as your test suite grows.
I created that image years ago, so I'll have to dig deep in my archives to see if I can find the original or recreate it. Thanks for the input and the share! 😁