Testing is one of the most crucial parts of any software development project. When you create a robust, comprehensive software testing suite, you can detect the bugs, mistakes, and incorrect assumptions that you’ve made while building the code base. By using many different types of testing, from unit and integration testing to user interface and regression testing, you’ll be more likely to release a higher-quality final product.
However, the unfortunate reality is that testing is far too often an afterthought, or treated merely as a box to be checked. Development teams struggling to meet the pressures of deadlines and budget constraints don’t have the time to devote to testing that the software fully deserves.
In order to counteract this trend, software developer Kent Beck reintroduced the concept of “test-driven development.” This article will provide an in-depth discussion of test-driven development when building software applications, as well as the advantages and disadvantages of doing so.
The Definition of Test-Driven Development
As the name suggests, test-driven development (abbreviated as TDD) is a software development practice that places testing first and foremost in the development process. To understand the definition of test-driven development, we first need to define unit testing, which is an essential concept in TDD.
What is unit testing?
Unit testing is a software testing method that breaks an application down into small parts known as units. Each unit is evaluated individually and independently by a thorough series of tests that are written specifically for that unit. In order to assist with debugging, the unit tests should be as short and simple as possible, so that developers can more easily discover the error.
For example, suppose that we’re writing code for a calculator application, and we want to test the program using unit testing. We could write one unit test for each of the calculator’s functions: addition, subtraction, multiplication, division, exponentials, etc. We would also need unit tests for activities such as clearing the calculator screen and composing multiple operations.
The goal of unit testing is to find bugs and problems early on in the development life cycle. A unit testing suite for a large software application may have hundreds or thousands of unit tests, each of which needs to pass in order for development to continue.
What is test-driven development?
TDD is the practice of writing software by making unit testing the most important concern. TDD is a highly structured, highly regimented approach to software development. The principles of TDD are as follows:
- Tests are always written before the code that will make them pass. The test anticipates the correct behavior of the code.
- Development proceeds one test at a time. Once a test goes from failing to passing, the next test is written and development can continue.
- Again, tests should be as simple as possible, only long enough to break the application in its current state. After the test is written, all development must focus on making the software pass the test.
TDD proceeds in cycles which are usually called “red, green, refactor.” This cycle is usually done once for each unit test that you write. The cycle consists of three stages:
- Red: Write a unit test that fails due to missing functionality in the software (since the color red usually denotes failure).
- Green: Write code that fixes the issue by making the software pass the test (since the color green usually denotes success).
- Refactor: Clean up the code base to account for the presence of this new code.
It might be helpful to compare TDD to the process of writing a long essay. You start writing the essay by creating a detailed outline of the topics that you want to cover. Next, you write one of the sections of the essay based on the outline, and you make any necessary adjustments to the rest of the essay based on the new text that you’ve just written. By having the structure in mind ahead of time, it becomes much easier to create the final product.
How to Do Test-Driven Development
In the previous section, we discussed the main ideas of test-driven development. We’ll now provide an in-depth guide about how to implement TDD in a software development project.
1. Write a test
Naturally, the first step in TDD is to create a unit test that evaluates some part of your code base. The “unit” in unit testing is often a method, a class, or a member function of that class.
For example, suppose that we’re creating a driving simulator as part of an educational course, and we want to have a Car class to represent the car that the user is driving. This Car class will include methods such as startCar(), turnOffCar(), changeGear(), changeSpeed(), etc. It will also have variables that hold information such as the car’s current status (on or off), current gear, and current speed.
The very first test that we would write would be to create an instance of the Car class:
Car c = new Car();
Of course, this test will fail at compile time, since we haven’t written the Car class yet.
Inspiration for writing tests during TDD can come from use case diagrams and user stories.
- Use case diagrams are models for how a system should behave based on the actions that a user wants to perform.
- User stories are brief text descriptions of the software requirements written by the project’s key stakeholders. In our driving simulator example, two user stories might be “Users can practice parallel parking with the software.” and “Users can practice in a variety of weather conditions.”
2. Run the test
When we run our unit testing suite (so far, consisting of just 1 test), we will receive an error during compilation informing us that the class does not exist. This message provides a clue to the developers telling them how to resolve the issue.
In some cases, the error will appear during runtime and not compile time. You can use assert statements to verify that a given condition is true or fulfilled while the program is executing. You can also throw an exception to check for one or more error conditions.
As the TDD process continues, additional tests will be added to the end of the unit testing suite. Note that in TDD, each unit test should be an independent entity. In other words, no test should depend on the behavior or success of the tests that came before it.
3. Fix the code
With the appropriate error message in hand, developers can now move to fix the problem. At this step, you should focus less on writing the perfect solution and more on writing a solution that will satisfy the test conditions.
For instance, in our Car example, the code needed to fix the failing test would be a definition of the Car class:
class Car {
}
As we mentioned above, this code is the minimum amount of effort required to pass our first test. We haven’t yet defined any member variables or member functions that belong to this class—because we don’t need to do so in order to pass the test.
The additional unit tests that we write will look for the presence of the functions and variables that we need in the Car class (e.g. turnOn() and currentState). Once we write each test, we will be able to add the function or variable that it tests.
4. Rerun the test
Once coding is complete, rerun the testing suite to see if you can now pass the test. In our basic Car example, for instance, the application will create the object and then exit silently. If all goes well and you’re following the principles of TDD, all of your tests should now be passing.
5. Refactor the code
In this (optional) step, you’ll refactor the code that you wrote in step 3 so that it integrates with the existing code base. This may involve making the code more readable, separating it into more logical parts, and renaming or moving variables and methods.
6. Repeat
TDD should continue incrementally, gradually expanding the features and functionality of the software.
You may find during TDD that you can’t easily make a new test pass, or that you break previous tests with the new code you’ve written. In these cases, the TDD best practice is usually to revert the changes you’ve made, rather than waste time on a lengthy debugging process.
Note that if you’re using any third-party libraries or frameworks, there’s no need to test the functionality of these external resources. You should only test the code that you plan to write yourself. In addition, good libraries and frameworks should already have their own unit tests defined in their code base.
The 5 Best Tools for Test-Driven Development
As a popular practice in programming, TDD has no shortage of tools to help you during your project. Some of the best tools for test-driven development are:
Travis CI: Travis CI is a tool for testing and deploying your code using the practice of continuous integration, which aligns well with TDD. Continuous integration requires developers to integrate their code into a shared repository multiple times per day and verify it using an automated build tool. Travis CI includes integration with GitHub as well as many databases and services—and it's completely free for open source projects.
CircleCI: Another tool for continuous integration, CircleCI is a highly customizable offering that offers you complete control over the development and testing process. CircleCI includes support for job orchestration and caching, and it's compatible with Docker as well as any language, toolchain, or framework that runs on Linux or Mac.
Squash: Squash replaces the traditional setup of development, staging, and QA servers with a unique virtual machine for each branch of code in your repository. You can preview a fully working version of your application based on the changes that you've made to each branch. This gives you the power to quickly and easily experiment with new tests and features, scaling your usage up and down automatically as you need it.
Selenium : is a free and open source test automation framework intended specifically for web development. You can configure tests so that they simulate different desktop environments and web browsers, and automatically generate reports of the test results.
Cypress: Cypress is another open source front-end web development testing framework that runs on JavaScript. In particular, Cypress performs end-to-end testing, making sure that the flow of your web application is correct from start to finish. Web developers at organizations like DHL, Spotify, and NASA all use Cypress to write, run, and record tests.
3 Benefits of Test-Driven Development
1. Speed
For skilled developers and testers who can move quickly, one major benefit of TDD is speed. By adding tests that fail, and then fixing the code to make them pass, TDD encourages rapid iteration and progress.
TDD may seem like the slower option at first, but the initial effort you put in will pay off later on. Few things are more disastrous for a software development project than discovering that your application logic contains a major flaw that requires the code to be refactored or rewritten.
By taking the time to invest in the quality of the code base early on, TDD can save developers time and effort and reduce the risk of a project that’s failed or delayed.
2. Easier automation
Another related benefit in terms of speed is that TDD and unit testing make it much easier to automate your software testing suite.
Manual checks are often painstakingly slow, especially if you perform functional tests that require you to follow a series of steps. The functional testing process can take seconds or minutes to complete, and must be done every time that you make a change to the system.
Unit testing, on the other hand, is rapid and extremely easy to automate. While both manual and automated testing have a place in a mature, robust software testing program, automated tests can drastically speed up the process. This allows you to run more tests in the same amount of time, improving the quality of the code.
3. Higher-quality code
If you wait to test your code until later in the development cycle, it’s a potential recipe for disaster. Writing hundreds or thousands of lines of code without making a mistake or typo is highly improbable.
As human beings, even the best developers are prone to error every so often, and it’s only a matter of time before a slip-up occurs. Without testing the code at regular intervals, bugs and unexpected behaviors are more likely to be introduced.
TDD is, above all, an incremental approach to software development. In most cases, developers will write only a few lines of code at a time—just enough to make the current test pass. This “slow yet steady” philosophy provides reassurance (although not a guarantee) that your software does not contain errors.
One additional benefit of TDD is that the code itself can serve as documentation. For example, if you want to demonstrate how a function behaves if given exceptional input (such as an empty string or a negative number), all you have to do is write a test that uses this input.
3 Common Pitfalls of Test-Driven Development
1. Time-intensive
The good news is that TDD seems to have real benefits for its practitioners. A survey of multiple studies on the impact of TDD has found that it reduces defects by 40 to 60 percent, while increasing effort and execution time by 15 to 35 percent.
While TDD generally results in higher-quality code, however, it must also be acknowledged that the extra effort isn’t always worth it. The TDD process involves a great deal of overhead in the form of unit tests. Creating and maintaining a test suite, in addition to the software itself, is a significant investment.
As a result, software that is short and/or straightforward to write will likely take longer with TDD, even when accounting for the separate coding and testing stages in traditional development. Businesses that prefer to invest time in manual QA, or that lack the technical resources to implement unit tests, may not be the ideal candidates for TDD.
2. Less flexibility
The overhead created by TDD can often be stifling and paralyzing during the project when developers want or need to make changes. Poor choices of architecture, design, or testing strategy early on can be difficult to recover from later in the project. At this point, it could be difficult or impossible to alter the code base without making dozens or hundreds of existing tests fail.
With the prospect of having to refactor both the code and the test suite, developers are caught between a rock and a hard place. Because even simple changes can be time-consuming to make, you’ll need to decide whether it’s even worth it to continue with TDD at this stage.
For this reason, TDD is difficult to apply to legacy code bases. Having a skilled team of testers and QA staff who know how to write good software tests is essential if you plan to practice TDD.
3. Not a perfect solution
TDD isn’t a guarantee that your code base will be airtight and protected from bugs and mistakes. After all, the tests in TDD are written not by the computer, but by error-prone humans.
This means that it’s nearly as likely that mistakes will be introduced during testing as during development. For example, developers may forget to write a test that covers an important feature or functionality of the software, causing bugs to go undetected. A single incorrect keystroke or lapse in judgment can easily cause an issue during the testing process.
What’s more, TDD can’t protect you from errors in comprehension, which result when developers have a fundamental misunderstanding or incorrect assumption about the problem that they’re trying to solve.
Of course, writing another testing suite for the first testing suite is a silly and impractical idea. The best you can do is to take special care when writing your tests, and to review the testing code at regular intervals.
Test-Driven Development vs. Traditional Development
TDD might sound like an excellent idea, but it hasn’t always been common practice in software development (and isn’t always used even today).
According to the traditional software development model, projects should proceed in a series of consecutive, sequential stages: requirements gathering, analysis, design, coding, testing, and deployment. This concept is often referred to as the “waterfall” model, like a waterfall cascading over a series of rocks at different levels.
In fact, the waterfall metaphor is highly accurate. The water falling in a waterfall continually flows downwards, without the possibility to return to a higher level. Similarly, the waterfall model of software development generally discourages returning to a previous stage of development. All coding and implementation must be completed before testing can begin.
Seeing the flaws with the waterfall methodology isn’t difficult. If new requirements emerge halfway through the project, or you discover a critical flaw in your assumptions, you may have no choice but to start all over again. As a result, software developed with the waterfall model can be susceptible to delays and budget overruns.
Rather than the traditional waterfall methodology, TDD fits in well with the agile and lean methodologies that are currently in vogue with software developers. Agile prioritizes flexibility, adaptability, and customer satisfaction over strict rules and regulations. In particular, Agile uses iterative development, in which software is constantly released in its newest deployable state in order to get valuable customer feedback.
The lean methodology, meanwhile, originates in concepts taken from automobile manufacturing. Lean uses the idea of "just-in-time" (JIT) development, in which the components of a system are created or ordered just in time for them to be used in the final product. JIT reduces the need to maintain excess inventory and ensures that employees are always working to add value to the product.
It's not hard to see how TDD, agile, and lean share similar philosophies about software development. With TDD, code and tests are written incrementally, using the minimum effort required in order to get feedback and start a new iteration. In fact, the practice of TDD was originally taken from extreme programming (XP), a software development methodology that falls under the agile umbrella.
Test-Driven Development vs. Behavior-Driven Development
Unit testing is an essential part of TDD, and you’ll often see the two ideas mentioned in the same breath. In this section, we’ll discuss another software testing concept that’s highly related to TDD: behavior-driven development (also known as BDD). In fact, TDD and BDD are so similar that some developers assume that they’re two terms for the same thing.
Like TDD, BDD prioritizes testing above all else during development. However, BDD goes beyond TDD by posing the question: “Is this the correct way to be testing in the first place?”
BDD considers issues such as:
- What parts of the code should and should not be tested
- How testing should be conducted
- How to understand why a test has failed or passed.
BDD encourages the use of names for unit tests that describe the expected behavior in natural language. For example, if we were testing our calculator application using BDD, we might say that a test “should (or does, or shall) return 2 when given 1+1.” The “should/does/shall” construct is extremely common for BDD test names.
Another difference between BDD and TDD is that tests in BDD evaluate the code’s expected behavior, rather than the specifics of the implementation.
For example, our Car class might have a speed variable that is initialized to 0 when creating a Car object. We’ve also written a unit test that verifies that the speed is correctly increased when using the increaseSpeed() function.
Instead of verifying that the Car’s speed is 10 after calling increaseSpeed(10), BDD would verify that the new speed is equal to the Car’s initial speed plus 10. This helps the unit test function independently of the code that it tests. For example, we can change the Car’s starting speed to 20 when the object is initialized without breaking the test.
Final Thoughts
TDD and its cousin BDD represent a departure from traditional approaches to software testing, in which tests are run only after the programming work is complete. Instead, TDD emphasizes the value of testing by binding it closely together with development.
This novel way of thinking forces developers to understand how each part of the code base should function, and helps them catch errors before it’s too late in the development process.
Being able to describe the software’s expected output and behavior has a variety of benefits. Communication improves, errors decline, and key stakeholders can be sure that their requirements for the project are being met.
Top comments (3)
Thanks for the article Evandro.
Agreed that you write a test which fails and the goal is to write enough code to get it to pass. This leads to code which may not be great but "behaves" as expected.
The key thing I have read about TDD is it a design process. It reduces big upfront design decisions. So where you say the following, I cannot agree.
"TDD is the practice of writing software by making unit testing the most important concern."
I did proper TDD ONCE for a cube library I was writing, and it naturally teases out structure. If I found myself really struggling to tease out a design, then TDD can help break it into smaller chunks/units.
Onto the second difference of opinion - Waterfall. I suggest you read - Software Project Survival Guide by Steve McConnell. He outlines, to me, what Waterfall really is about. Keeping a project to budget, and even - canning it early on. In this book he makes the point that an upstream fix is up to 200 times less expensive than a downstream fix. He doesn't say, it can't be done. This idea that once components have been released in an agile methodology, it won't take time to replace them is a myth. McConnell mentions "Thrashing" which would be the TDD phase or prototyping phase of the project. It is a bit like, planning for a journey, and keep changing your mind, but once you are on the journey it is hard to go back. This is normal.
So, many thanks for the article, very useful for many people - but I just think it a bit bold to overly sell the Unit Testing part of TDD, and to put Agile above Waterfall.
As a product guy, I tend to agree that Waterfall is needed in the first phase of the product definition and roadmap creation aspect. But, when you are in the development phase, if requirements were not clear enough , then trying to figure out the requirements while building is not good Agile. I agree that people get confused with Waterfall and Agile. You can have both, Start with Waterfall, lock your requirements, do agile development and then test in waterfall. I have seen too many projects go haywire when you don't respect the need for clarity upfront.
Okay, I took a couple of days to think about this (kind of :).
The challenge with Agile is, it is a training tool only. Agile is great as a way to train people who can't communicate, who can't plan, and can't think on their feet - to do that. Once they have those skills, there is no longer a need to keep doing it.
Agile feels like an unnatural, pervasive intrusion into the project. The processes are laboured, draining, and artificial. Conversely, despite Waterfall having many faults - it would be natural, if something wasn't working - to call a meeting. To rename a meeting and call it a 3 Amigos is stupid. Has anybody sat in a 3 amigos, when there are four people, thinking - which one of us isn't our friend? "Burndowns" is an abstraction away from what effective people do - getting things done. Have sat in retrospectives with people literally slashing their wrists because the graph wasn't a perfect line.
Drivers don't continually say 10 to 2. They move past that phase.
A Product Manager loves Agile because they think they have a handle on what the team are doing, and they can bubble up progress - but it isn't true. The tasks lose their meaning and sense, with managers saying things like we are 70% done for this sprint and our burndown spiked at this point. The product owner doesn't have a clue about it. The Product Owner wants to get on with their normal job - not spend all their time with a development team. Every project which is Agile, results in the manager/scrum master not having a Scooby about what people are actually working on.
We don't spend our time, running 2 week sprints, but if we were getting our house renovated we would not undertake an agile methodology.
Again - Waterfall wins because it is a natural process. We discuss what we would like to do, we figure out how long we think it will take, and we start on that journey, and we review it.
Am I fan of Waterfall, not entirely - many times iterative wins, but Agile just sucks.
It is a shame, because the spirit of agile, and the manifesto really made sense. Agile has been hijacked.
Okay, happy Friday.